2025-07-24T03:40:56.7326056Z Current runner version: '2.326.0'
2025-07-24T03:40:56.7349605Z ##[group]Runner Image Provisioner
2025-07-24T03:40:56.7350425Z Hosted Compute Agent
2025-07-24T03:40:56.7350970Z Version: 20250711.363
2025-07-24T03:40:56.7351599Z Commit: 6785254374ce925a23743850c1cb91912ce5c14c
2025-07-24T03:40:56.7352262Z Build Date: 2025-07-11T20:04:25Z
2025-07-24T03:40:56.7352845Z ##[endgroup]
2025-07-24T03:40:56.7353346Z ##[group]Operating System
2025-07-24T03:40:56.7353969Z Ubuntu
2025-07-24T03:40:56.7354410Z 24.04.2
2025-07-24T03:40:56.7355091Z LTS
2025-07-24T03:40:56.7355588Z ##[endgroup]
2025-07-24T03:40:56.7356077Z ##[group]Runner Image
2025-07-24T03:40:56.7356634Z Image: ubuntu-24.04
2025-07-24T03:40:56.7357165Z Version: 20250720.1.0
2025-07-24T03:40:56.7358152Z Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20250720.1/images/ubuntu/Ubuntu2404-Readme.md
2025-07-24T03:40:56.7359672Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20250720.1
2025-07-24T03:40:56.7360606Z ##[endgroup]
2025-07-24T03:40:56.7361648Z ##[group]GITHUB_TOKEN Permissions
2025-07-24T03:40:56.7363592Z Contents: read
2025-07-24T03:40:56.7364144Z Metadata: read
2025-07-24T03:40:56.7364927Z ##[endgroup]
2025-07-24T03:40:56.7367176Z Secret source: Actions
2025-07-24T03:40:56.7368027Z Prepare workflow directory
2025-07-24T03:40:56.7872428Z Prepare all required actions
2025-07-24T03:40:56.7929188Z Uses: pytorch/pytorch/.github/workflows/_runner-determinator.yml@refs/heads/main (07df6ba7f5597488a93b3855d52d2ead55675125)
2025-07-24T03:40:56.7934021Z ##[group] Inputs
2025-07-24T03:40:56.7934828Z   check_experiments: 
2025-07-24T03:40:56.7935375Z   opt_out_experiments: lf
2025-07-24T03:40:56.7936058Z   triggering_actor: pytorch-bot[bot]
2025-07-24T03:40:56.7936635Z   issue_owner: 
2025-07-24T03:40:56.7937210Z   curr_branch: ciflow/inductor/149961
2025-07-24T03:40:56.7937844Z   curr_ref_type: tag
2025-07-24T03:40:56.7938497Z   issue_number: 5132
2025-07-24T03:40:56.7939045Z ##[endgroup]
2025-07-24T03:40:56.7939717Z Complete job name: unit-test / get-label-type / runner-determinator
2025-07-24T03:40:57.5235638Z ##[group]Run cat <<EOF > runner_determinator.py
2025-07-24T03:40:57.5238240Z cat <<EOF > runner_determinator.py
2025-07-24T03:40:57.5238893Z # flake8: noqa: G004
2025-07-24T03:40:57.5239517Z 
2025-07-24T03:40:57.5240221Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2025-07-24T03:40:57.5241253Z #       must be kept in sync. You can do it easily by running the following command:
2025-07-24T03:40:57.5242236Z #           python .github/scripts/update_runner_determinator.py
2025-07-24T03:40:57.5242944Z 
2025-07-24T03:40:57.5243353Z """
2025-07-24T03:40:57.5244094Z This runner determinator is used to determine which set of runners to run a
2025-07-24T03:40:57.5245263Z GitHub job on. It uses the first comment of a GitHub issue (by default
2025-07-24T03:40:57.5246472Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2025-07-24T03:40:57.5247392Z of which runners should be used to run which job.
2025-07-24T03:40:57.5248048Z 
2025-07-24T03:40:57.5248770Z The configuration has two parts, the settings and a list of opted-in users,
2025-07-24T03:40:57.5249884Z separated by a line containing "---".  If the line is not present, the
2025-07-24T03:40:57.5250872Z settings are considered to be empty with only the second part, the user
2025-07-24T03:40:57.5251734Z list, defined.
2025-07-24T03:40:57.5252224Z 
2025-07-24T03:40:57.5252883Z The first part is a YAML block that defines the rollout settings. This can be
2025-07-24T03:40:57.5253867Z used to define any settings that are needed to determine which runners to use.
2025-07-24T03:40:57.5255018Z It's fields are defined by the RolloutSettings class below.
2025-07-24T03:40:57.5256011Z 
2025-07-24T03:40:57.5256638Z The second part is a list of users who are explicitly opted in to the LF fleet.
2025-07-24T03:40:57.5257717Z The user list is also a comma separated list of additional features or
2025-07-24T03:40:57.5258543Z experiments which the user could be opted in to.
2025-07-24T03:40:57.5259187Z 
2025-07-24T03:40:57.5259720Z The user list has the following rules:
2025-07-24T03:40:57.5260327Z 
2025-07-24T03:40:57.5260966Z - Users are GitHub usernames, which must start with the @ prefix
2025-07-24T03:40:57.5261992Z - Each user is also a comma-separated list of features/experiments to enable
2025-07-24T03:40:57.5262874Z - A "#" prefix opts the user out of all experiments
2025-07-24T03:40:57.5263489Z 
2025-07-24T03:40:57.5264001Z Example config:
2025-07-24T03:40:57.5264970Z     # A list of experiments that can be opted into.
2025-07-24T03:40:57.5265862Z     # This defines the behavior they'll induce when opted into.
2025-07-24T03:40:57.5266671Z     # Expected syntax is:
2025-07-24T03:40:57.5267419Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2025-07-24T03:40:57.5268491Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2025-07-24T03:40:57.5269394Z 
2025-07-24T03:40:57.5269811Z     experiments:
2025-07-24T03:40:57.5270352Z       lf:
2025-07-24T03:40:57.5270840Z         rollout_percent: 25
2025-07-24T03:40:57.5271443Z         all_branches: false
2025-07-24T03:40:57.5271983Z         default: true
2025-07-24T03:40:57.5272546Z     ---
2025-07-24T03:40:57.5272987Z 
2025-07-24T03:40:57.5273471Z     # Opt-ins:
2025-07-24T03:40:57.5274234Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2025-07-24T03:40:57.5275568Z     # and specifying experiments to enable in a comma-separated list.
2025-07-24T03:40:57.5276567Z     # To always opt out of an experiment, prefix it with a "-".
2025-07-24T03:40:57.5277299Z     # Experiments should be from the above list.
2025-07-24T03:40:57.5278065Z 
2025-07-24T03:40:57.5278533Z     @User1,-lf,split_build
2025-07-24T03:40:57.5279061Z     @User2,lf
2025-07-24T03:40:57.5279708Z     @User3,split_build
2025-07-24T03:40:57.5280231Z """
2025-07-24T03:40:57.5280646Z 
2025-07-24T03:40:57.5354150Z import json
2025-07-24T03:40:57.5355138Z import logging
2025-07-24T03:40:57.5355860Z import os
2025-07-24T03:40:57.5356528Z import random
2025-07-24T03:40:57.5357258Z import re
2025-07-24T03:40:57.5357893Z import sys
2025-07-24T03:40:57.5358740Z from argparse import ArgumentParser
2025-07-24T03:40:57.5359678Z from collections.abc import Iterable
2025-07-24T03:40:57.5360253Z from functools import cache
2025-07-24T03:40:57.5360772Z from logging import LogRecord
2025-07-24T03:40:57.5361302Z from typing import Any, NamedTuple
2025-07-24T03:40:57.5361941Z from urllib.request import Request, urlopen
2025-07-24T03:40:57.5362482Z 
2025-07-24T03:40:57.5362830Z import yaml
2025-07-24T03:40:57.5363260Z from github import Auth, Github
2025-07-24T03:40:57.5363788Z from github.Issue import Issue
2025-07-24T03:40:57.5364282Z 
2025-07-24T03:40:57.5364620Z 
2025-07-24T03:40:57.5365452Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2025-07-24T03:40:57.5366209Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2025-07-24T03:40:57.5367091Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2025-07-24T03:40:57.5367795Z 
2025-07-24T03:40:57.5368486Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2025-07-24T03:40:57.5369085Z GH_OUTPUT_KEY_AMI = "runner-ami"
2025-07-24T03:40:57.5369643Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2025-07-24T03:40:57.5370229Z OPT_OUT_LABEL = "no-runner-experiments"
2025-07-24T03:40:57.5370740Z 
2025-07-24T03:40:57.5371138Z SETTING_EXPERIMENTS = "experiments"
2025-07-24T03:40:57.5371637Z 
2025-07-24T03:40:57.5372009Z LF_FLEET_EXPERIMENT = "lf"
2025-07-24T03:40:57.5372501Z CANARY_FLEET_SUFFIX = ".c"
2025-07-24T03:40:57.5372950Z 
2025-07-24T03:40:57.5373289Z 
2025-07-24T03:40:57.5373668Z class Experiment(NamedTuple):
2025-07-24T03:40:57.5374188Z     rollout_perc: float = (
2025-07-24T03:40:57.5375076Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2025-07-24T03:40:57.5375780Z     )
2025-07-24T03:40:57.5376184Z     all_branches: bool = (
2025-07-24T03:40:57.5376868Z         False  # If True, the experiment is also enabled on the exception branches
2025-07-24T03:40:57.5377541Z     )
2025-07-24T03:40:57.5377916Z     default: bool = (
2025-07-24T03:40:57.5378537Z         True  # If True, the experiment is enabled by default for all queries
2025-07-24T03:40:57.5379183Z     )
2025-07-24T03:40:57.5379542Z 
2025-07-24T03:40:57.5379916Z     # Add more fields as needed
2025-07-24T03:40:57.5380400Z 
2025-07-24T03:40:57.5380755Z 
2025-07-24T03:40:57.5381126Z class Settings(NamedTuple):
2025-07-24T03:40:57.5381598Z     """
2025-07-24T03:40:57.5382085Z     Settings for the experiments that can be opted into.
2025-07-24T03:40:57.5382662Z     """
2025-07-24T03:40:57.5383022Z 
2025-07-24T03:40:57.5383447Z     experiments: dict[str, Experiment] = {}
2025-07-24T03:40:57.5383968Z 
2025-07-24T03:40:57.5384437Z 
2025-07-24T03:40:57.5385054Z class ColorFormatter(logging.Formatter):
2025-07-24T03:40:57.5385706Z     """Color codes the log messages based on the log level"""
2025-07-24T03:40:57.5386288Z 
2025-07-24T03:40:57.5386634Z     COLORS = {
2025-07-24T03:40:57.5387080Z         "WARNING": "\033[33m",  # Yellow
2025-07-24T03:40:57.5387602Z         "ERROR": "\033[31m",  # Red
2025-07-24T03:40:57.5388116Z         "CRITICAL": "\033[31m",  # Red
2025-07-24T03:40:57.5388636Z         "INFO": "\033[0m",  # Reset
2025-07-24T03:40:57.5389150Z         "DEBUG": "\033[0m",  # Reset
2025-07-24T03:40:57.5389639Z     }
2025-07-24T03:40:57.5390001Z 
2025-07-24T03:40:57.5390425Z     def format(self, record: LogRecord) -> str:
2025-07-24T03:40:57.5391188Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2025-07-24T03:40:57.5391982Z         record.msg = f"{log_color}{record.msg}\033[0m"
2025-07-24T03:40:57.5392574Z         return super().format(record)
2025-07-24T03:40:57.5393070Z 
2025-07-24T03:40:57.5393411Z 
2025-07-24T03:40:57.5393788Z handler = logging.StreamHandler()
2025-07-24T03:40:57.5394528Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2025-07-24T03:40:57.5395482Z 
2025-07-24T03:40:57.5395959Z log = logging.getLogger(os.path.basename(__file__))
2025-07-24T03:40:57.5396563Z log.addHandler(handler)
2025-07-24T03:40:57.5397044Z log.setLevel(logging.INFO)
2025-07-24T03:40:57.5397508Z 
2025-07-24T03:40:57.5397843Z 
2025-07-24T03:40:57.5398308Z def set_github_output(key: str, value: str) -> None:
2025-07-24T03:40:57.5398889Z     """
2025-07-24T03:40:57.5399426Z     Defines outputs of the github action that invokes this script
2025-07-24T03:40:57.5400195Z     """
2025-07-24T03:40:57.5400581Z     if not GITHUB_OUTPUT:
2025-07-24T03:40:57.5401659Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2025-07-24T03:40:57.5402747Z         log.warning(
2025-07-24T03:40:57.5403612Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2025-07-24T03:40:57.5404518Z         )
2025-07-24T03:40:57.5405175Z         print(f"::set-output name={key}::{value}")
2025-07-24T03:40:57.5405737Z         return
2025-07-24T03:40:57.5406142Z 
2025-07-24T03:40:57.5406529Z     with open(GITHUB_OUTPUT, "a") as f:
2025-07-24T03:40:57.5407124Z         log.info(f"Setting output: {key}='{value}'")
2025-07-24T03:40:57.5407708Z         f.write(f"{key}={value}\n")
2025-07-24T03:40:57.5408200Z 
2025-07-24T03:40:57.5408544Z 
2025-07-24T03:40:57.5409058Z def _str_comma_separated_to_set(value: str) -> frozenset[str]:
2025-07-24T03:40:57.5409713Z     return frozenset(
2025-07-24T03:40:57.5410351Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2025-07-24T03:40:57.5411079Z     )
2025-07-24T03:40:57.5411445Z 
2025-07-24T03:40:57.5411781Z 
2025-07-24T03:40:57.5412151Z def parse_args() -> Any:
2025-07-24T03:40:57.5412753Z     parser = ArgumentParser("Get dynamic rollout settings")
2025-07-24T03:40:57.5413640Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2025-07-24T03:40:57.5414393Z     parser.add_argument(
2025-07-24T03:40:57.5415121Z         "--github-issue-repo",
2025-07-24T03:40:57.5415626Z         type=str,
2025-07-24T03:40:57.5416073Z         required=False,
2025-07-24T03:40:57.5416697Z         default="pytorch/test-infra",
2025-07-24T03:40:57.5417258Z         help="GitHub repo to get the issue",
2025-07-24T03:40:57.5417770Z     )
2025-07-24T03:40:57.5418154Z     parser.add_argument(
2025-07-24T03:40:57.5418621Z         "--github-repo",
2025-07-24T03:40:57.5419077Z         type=str,
2025-07-24T03:40:57.5419514Z         required=True,
2025-07-24T03:40:57.5420029Z         help="GitHub repo where CI is running",
2025-07-24T03:40:57.5420546Z     )
2025-07-24T03:40:57.5420931Z     parser.add_argument(
2025-07-24T03:40:57.5421575Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2025-07-24T03:40:57.5422236Z     )
2025-07-24T03:40:57.5422618Z     parser.add_argument(
2025-07-24T03:40:57.5423287Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2025-07-24T03:40:57.5424028Z     )
2025-07-24T03:40:57.5424462Z     parser.add_argument(
2025-07-24T03:40:57.5425313Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2025-07-24T03:40:57.5426000Z     )
2025-07-24T03:40:57.5426384Z     parser.add_argument(
2025-07-24T03:40:57.5427067Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2025-07-24T03:40:57.5427770Z     )
2025-07-24T03:40:57.5428149Z     parser.add_argument(
2025-07-24T03:40:57.5428631Z         "--github-ref-type",
2025-07-24T03:40:57.5429117Z         type=str,
2025-07-24T03:40:57.5429550Z         required=True,
2025-07-24T03:40:57.5430094Z         help="Current GitHub ref type, branch or tag",
2025-07-24T03:40:57.5430655Z     )
2025-07-24T03:40:57.5431091Z     parser.add_argument(
2025-07-24T03:40:57.5431725Z         "--eligible-experiments",
2025-07-24T03:40:57.5432276Z         type=_str_comma_separated_to_set,
2025-07-24T03:40:57.5432808Z         required=False,
2025-07-24T03:40:57.5433261Z         default="",
2025-07-24T03:40:57.5434133Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2025-07-24T03:40:57.5435294Z     )
2025-07-24T03:40:57.5435681Z     parser.add_argument(
2025-07-24T03:40:57.5436195Z         "--opt-out-experiments",
2025-07-24T03:40:57.5436740Z         type=_str_comma_separated_to_set,
2025-07-24T03:40:57.5437285Z         required=False,
2025-07-24T03:40:57.5437748Z         default="",
2025-07-24T03:40:57.5438170Z         help=(
2025-07-24T03:40:57.5438861Z             "comma separated list of experiments to opt-out of. If unset, no opt-outs will occur. "
2025-07-24T03:40:57.5439985Z             "If the same experiment is listed both here and in '--eligible-experiments' opt-out will take priority."
2025-07-24T03:40:57.5440799Z         ),
2025-07-24T03:40:57.5441176Z     )
2025-07-24T03:40:57.5441553Z     parser.add_argument(
2025-07-24T03:40:57.5442025Z         "--pr-number",
2025-07-24T03:40:57.5442474Z         type=str,
2025-07-24T03:40:57.5442910Z         required=False,
2025-07-24T03:40:57.5443356Z         default="",
2025-07-24T03:40:57.5443882Z         help="the optional PR number where this is run",
2025-07-24T03:40:57.5444440Z     )
2025-07-24T03:40:57.5445182Z 
2025-07-24T03:40:57.5445566Z     return parser.parse_args()
2025-07-24T03:40:57.5446056Z 
2025-07-24T03:40:57.5446397Z 
2025-07-24T03:40:57.5446995Z def get_gh_client(github_token: str) -> Github:  # type: ignore[no-any-unimported]
2025-07-24T03:40:57.5447911Z     auth = Auth.Token(github_token)
2025-07-24T03:40:57.5448435Z     return Github(auth=auth)
2025-07-24T03:40:57.5448909Z 
2025-07-24T03:40:57.5449242Z 
2025-07-24T03:40:57.5449885Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:  # type: ignore[no-any-unimported]
2025-07-24T03:40:57.5450685Z     repo = gh.get_repo(repo)
2025-07-24T03:40:57.5451217Z     return repo.get_issue(number=issue_num)
2025-07-24T03:40:57.5451734Z 
2025-07-24T03:40:57.5452064Z 
2025-07-24T03:40:57.5452433Z def get_potential_pr_author(
2025-07-24T03:40:57.5453107Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2025-07-24T03:40:57.5453785Z ) -> str:
2025-07-24T03:40:57.5454335Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2025-07-24T03:40:57.5455333Z     # Fetch the actual username from the original PR. The PR number is
2025-07-24T03:40:57.5456107Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2025-07-24T03:40:57.5456672Z 
2025-07-24T03:40:57.5457061Z     gh = get_gh_client(github_token)
2025-07-24T03:40:57.5457552Z 
2025-07-24T03:40:57.5458020Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2025-07-24T03:40:57.5458683Z         split_tag = ref_name.split("/")
2025-07-24T03:40:57.5459190Z         if (
2025-07-24T03:40:57.5459609Z             len(split_tag) == 3
2025-07-24T03:40:57.5460128Z             and split_tag[0] == "ciflow"
2025-07-24T03:40:57.5460668Z             and split_tag[2].isnumeric()
2025-07-24T03:40:57.5461167Z         ):
2025-07-24T03:40:57.5461597Z             pr_number = split_tag[2]
2025-07-24T03:40:57.5462130Z             try:
2025-07-24T03:40:57.5462604Z                 repository = gh.get_repo(repo)
2025-07-24T03:40:57.5463368Z                 pull = repository.get_pull(number=int(pr_number))
2025-07-24T03:40:57.5463980Z             except Exception as e:
2025-07-24T03:40:57.5464523Z                 raise Exception(  # noqa: TRY002
2025-07-24T03:40:57.5465374Z                     f"issue with pull request {pr_number} from repo {repository}"
2025-07-24T03:40:57.5466037Z                 ) from e
2025-07-24T03:40:57.5466620Z             return pull.user.login  # type: ignore[no-any-return]
2025-07-24T03:40:57.5467340Z     # In all other cases, return the original input username
2025-07-24T03:40:57.5467935Z     return username
2025-07-24T03:40:57.5468361Z 
2025-07-24T03:40:57.5468704Z 
2025-07-24T03:40:57.5469132Z def is_exception_branch(branch: str) -> bool:
2025-07-24T03:40:57.5469669Z     """
2025-07-24T03:40:57.5470339Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2025-07-24T03:40:57.5471114Z     """
2025-07-24T03:40:57.5471681Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2025-07-24T03:40:57.5472338Z 
2025-07-24T03:40:57.5472682Z 
2025-07-24T03:40:57.5473064Z def load_yaml(yaml_text: str) -> Any:
2025-07-24T03:40:57.5473573Z     try:
2025-07-24T03:40:57.5473976Z         data = yaml.safe_load(yaml_text)
2025-07-24T03:40:57.5474496Z         return data
2025-07-24T03:40:57.5475178Z     except yaml.YAMLError:
2025-07-24T03:40:57.5475712Z         log.exception("Error loading YAML")
2025-07-24T03:40:57.5476240Z         raise
2025-07-24T03:40:57.5476626Z 
2025-07-24T03:40:57.5476963Z 
2025-07-24T03:40:57.5477575Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> tuple[str, str]:
2025-07-24T03:40:57.5478313Z     """
2025-07-24T03:40:57.5479072Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2025-07-24T03:40:57.5479817Z 
2025-07-24T03:40:57.5480355Z     If the issue body contains "---" then the text above that is the settings
2025-07-24T03:40:57.5481119Z     and the text below is the list of opted in users.
2025-07-24T03:40:57.5481670Z 
2025-07-24T03:40:57.5482236Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2025-07-24T03:40:57.5482929Z     """
2025-07-24T03:40:57.5483398Z     rollout_state_parts = rollout_state.split("---")
2025-07-24T03:40:57.5483998Z     if len(rollout_state_parts) >= 2:
2025-07-24T03:40:57.5484626Z         return rollout_state_parts[0], rollout_state_parts[1]
2025-07-24T03:40:57.5485398Z     else:
2025-07-24T03:40:57.5485891Z         return "", rollout_state
2025-07-24T03:40:57.5486536Z 
2025-07-24T03:40:57.5486873Z 
2025-07-24T03:40:57.5487267Z class UserOptins(dict[str, list[str]]):
2025-07-24T03:40:57.5487780Z     """
2025-07-24T03:40:57.5488334Z     Dictionary of users with a list of features they have opted into
2025-07-24T03:40:57.5488970Z     """
2025-07-24T03:40:57.5489323Z 
2025-07-24T03:40:57.5489655Z 
2025-07-24T03:40:57.5490190Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2025-07-24T03:40:57.5490837Z     """
2025-07-24T03:40:57.5491562Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2025-07-24T03:40:57.5492379Z 
2025-07-24T03:40:57.5493161Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2025-07-24T03:40:57.5494146Z         - Example line: "@User1,lf,split_build"
2025-07-24T03:40:57.5495181Z         - A "#" prefix indicates the user is opted out of all experiments
2025-07-24T03:40:57.5495804Z 
2025-07-24T03:40:57.5496134Z 
2025-07-24T03:40:57.5496470Z     """
2025-07-24T03:40:57.5496849Z     optins = UserOptins()
2025-07-24T03:40:57.5497363Z     for user in user_optin_text.split("\n"):
2025-07-24T03:40:57.5497936Z         user = user.strip("\r\n\t -")
2025-07-24T03:40:57.5498503Z         if not user or not user.startswith("@"):
2025-07-24T03:40:57.5499065Z             # Not a valid user. Skip
2025-07-24T03:40:57.5499559Z             continue
2025-07-24T03:40:57.5499981Z 
2025-07-24T03:40:57.5500325Z         if user:
2025-07-24T03:40:57.5500836Z             usr_name = user.split(",")[0].strip("@")
2025-07-24T03:40:57.5501531Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2025-07-24T03:40:57.5502159Z 
2025-07-24T03:40:57.5502517Z     return optins
2025-07-24T03:40:57.5502917Z 
2025-07-24T03:40:57.5503243Z 
2025-07-24T03:40:57.5503731Z def is_valid_experiment_name(experiment_name: str) -> bool:
2025-07-24T03:40:57.5504337Z     """
2025-07-24T03:40:57.5504906Z     Check if the experiment name is valid.
2025-07-24T03:40:57.5505438Z     A valid name:
2025-07-24T03:40:57.5506106Z         - Contains only alphanumeric characters and the special characters "_" & "-"
2025-07-24T03:40:57.5507026Z         - The special characters "_" & "-" shouldn't be the first or last characters
2025-07-24T03:40:57.5507729Z         - Cannot contain spaces
2025-07-24T03:40:57.5508197Z     """
2025-07-24T03:40:57.5508553Z 
2025-07-24T03:40:57.5509002Z     valid_char_regex = r"^[a-zA-Z0-9]([\w-]*[a-zA-Z0-9])?$"
2025-07-24T03:40:57.5509721Z     valid = bool(re.match(valid_char_regex, experiment_name))
2025-07-24T03:40:57.5510443Z 
2025-07-24T03:40:57.5510791Z     if valid:
2025-07-24T03:40:57.5511195Z         return True
2025-07-24T03:40:57.5511608Z 
2025-07-24T03:40:57.5511956Z     log.error(
2025-07-24T03:40:57.5513353Z         f"Invalid experiment name: {experiment_name}. Experiment names should only contain alphanumeric characters, '_', and '-'. They cannot contain spaces, and the special characters '_' and '-' cannot be the first or last characters."
2025-07-24T03:40:57.5515078Z     )
2025-07-24T03:40:57.5515448Z     return False
2025-07-24T03:40:57.5515847Z 
2025-07-24T03:40:57.5516199Z 
2025-07-24T03:40:57.5516702Z def parse_settings_from_text(settings_text: str) -> Settings:
2025-07-24T03:40:57.5517328Z     """
2025-07-24T03:40:57.5517923Z     Parse the experiments from the issue body into a list of ExperimentSettings
2025-07-24T03:40:57.5518625Z     """
2025-07-24T03:40:57.5518988Z     try:
2025-07-24T03:40:57.5519367Z         if settings_text:
2025-07-24T03:40:57.5520112Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2025-07-24T03:40:57.5520889Z             # for easy reading
2025-07-24T03:40:57.5521731Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2025-07-24T03:40:57.5522607Z             #       the backtick character in shell commands.
2025-07-24T03:40:57.5523221Z             backtick = chr(96)  # backtick character
2025-07-24T03:40:57.5523892Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2025-07-24T03:40:57.5524554Z             settings = load_yaml(settings_text)
2025-07-24T03:40:57.5525542Z 
2025-07-24T03:40:57.5526137Z             # For now we just load experiments. We can expand this if/when we add more settings
2025-07-24T03:40:57.5527013Z             experiments = {}
2025-07-24T03:40:57.5527472Z 
2025-07-24T03:40:57.5528018Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2025-07-24T03:40:57.5528779Z                 if not is_valid_experiment_name(exp_name):
2025-07-24T03:40:57.5529835Z                     # Exclude invalid experiments from the list. We log an error, but don't raise an exception so that other experiments can still be processed.
2025-07-24T03:40:57.5530840Z                     continue
2025-07-24T03:40:57.5531295Z 
2025-07-24T03:40:57.5531654Z                 valid_settings = {}
2025-07-24T03:40:57.5532355Z                 for setting in exp_settings:
2025-07-24T03:40:57.5533045Z                     if setting not in Experiment._fields:
2025-07-24T03:40:57.5533617Z                         log.warning(
2025-07-24T03:40:57.5534335Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2025-07-24T03:40:57.5535269Z                         )
2025-07-24T03:40:57.5535723Z                     else:
2025-07-24T03:40:57.5536261Z                         valid_settings[setting] = exp_settings[setting]
2025-07-24T03:40:57.5536825Z 
2025-07-24T03:40:57.5537288Z                 experiments[exp_name] = Experiment(**valid_settings)
2025-07-24T03:40:57.5537933Z             return Settings(experiments)
2025-07-24T03:40:57.5538426Z 
2025-07-24T03:40:57.5538778Z     except Exception:
2025-07-24T03:40:57.5539291Z         log.exception("Failed to parse settings")
2025-07-24T03:40:57.5539809Z 
2025-07-24T03:40:57.5540163Z     return Settings()
2025-07-24T03:40:57.5540583Z 
2025-07-24T03:40:57.5540925Z 
2025-07-24T03:40:57.5541554Z def parse_settings(rollout_state: str) -> Settings:
2025-07-24T03:40:57.5542136Z     """
2025-07-24T03:40:57.5542592Z     Parse settings, if any, from the rollout state.
2025-07-24T03:40:57.5543134Z 
2025-07-24T03:40:57.5543668Z     If the issue body contains "---" then the text above that is the settings
2025-07-24T03:40:57.5544417Z     and the text below is the list of opted in users.
2025-07-24T03:40:57.5545189Z 
2025-07-24T03:40:57.5545785Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2025-07-24T03:40:57.5546510Z     """
2025-07-24T03:40:57.5547072Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2025-07-24T03:40:57.5547841Z     return parse_settings_from_text(settings_text)
2025-07-24T03:40:57.5548379Z 
2025-07-24T03:40:57.5548700Z 
2025-07-24T03:40:57.5549157Z def parse_users(rollout_state: str) -> UserOptins:
2025-07-24T03:40:57.5549717Z     """
2025-07-24T03:40:57.5550136Z     Parse users from the rollout state.
2025-07-24T03:40:57.5550633Z 
2025-07-24T03:40:57.5550966Z     """
2025-07-24T03:40:57.5551513Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2025-07-24T03:40:57.5552260Z     return parse_user_opt_in_from_text(users_text)
2025-07-24T03:40:57.5552806Z 
2025-07-24T03:40:57.5553126Z 
2025-07-24T03:40:57.5553735Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-07-24T03:40:57.5554462Z     """
2025-07-24T03:40:57.5555018Z     Check if a user is opted into an experiment
2025-07-24T03:40:57.5555543Z     """
2025-07-24T03:40:57.5556023Z     return experiment_name in user_optins.get(user, [])
2025-07-24T03:40:57.5556741Z 
2025-07-24T03:40:57.5557063Z 
2025-07-24T03:40:57.5557681Z def is_user_opted_out(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-07-24T03:40:57.5558409Z     """
2025-07-24T03:40:57.5558895Z     Check if a user explicitly opted out of an experiment
2025-07-24T03:40:57.5559501Z     """
2025-07-24T03:40:57.5560023Z     # if the experiment is prefixed with a "-", then it's an opt-out
2025-07-24T03:40:57.5560719Z     experiment_optout = "-" + experiment_name
2025-07-24T03:40:57.5561370Z     if experiment_optout not in user_optins.get(user, []):
2025-07-24T03:40:57.5561998Z         return False
2025-07-24T03:40:57.5562406Z 
2025-07-24T03:40:57.5562866Z     if is_user_opted_in(user, user_optins, experiment_name):
2025-07-24T03:40:57.5563459Z         log.warning(
2025-07-24T03:40:57.5564262Z             f"User {user} is opted into experiment {experiment_name}, but also opted out of it. Defaulting to opting out"
2025-07-24T03:40:57.5565205Z         )
2025-07-24T03:40:57.5565565Z 
2025-07-24T03:40:57.5565911Z     return True
2025-07-24T03:40:57.5566315Z 
2025-07-24T03:40:57.5566640Z 
2025-07-24T03:40:57.5566990Z def get_runner_prefix(
2025-07-24T03:40:57.5567449Z     rollout_state: str,
2025-07-24T03:40:57.5567933Z     workflow_requestors: Iterable[str],
2025-07-24T03:40:57.5568458Z     branch: str,
2025-07-24T03:40:57.5568988Z     eligible_experiments: frozenset[str] = frozenset(),
2025-07-24T03:40:57.5569666Z     opt_out_experiments: frozenset[str] = frozenset(),
2025-07-24T03:40:57.5570247Z     is_canary: bool = False,
2025-07-24T03:40:57.5570708Z ) -> str:
2025-07-24T03:40:57.5571148Z     settings = parse_settings(rollout_state)
2025-07-24T03:40:57.5571733Z     user_optins = parse_users(rollout_state)
2025-07-24T03:40:57.5572246Z 
2025-07-24T03:40:57.5572711Z     fleet_prefix = ""
2025-07-24T03:40:57.5573165Z     prefixes = []
2025-07-24T03:40:57.5573813Z     for experiment_name, experiment_settings in settings.experiments.items():
2025-07-24T03:40:57.5574853Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2025-07-24T03:40:57.5575567Z             log.info(
2025-07-24T03:40:57.5576268Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2025-07-24T03:40:57.5577002Z             )
2025-07-24T03:40:57.5577420Z             continue
2025-07-24T03:40:57.5577832Z 
2025-07-24T03:40:57.5578202Z         if opt_out_experiments:
2025-07-24T03:40:57.5578751Z             if experiment_name in opt_out_experiments:
2025-07-24T03:40:57.5579393Z                 opt_out_exp_list = ", ".join(opt_out_experiments)
2025-07-24T03:40:57.5579986Z                 log.info(
2025-07-24T03:40:57.5580908Z                     f"Skipping experiment '{experiment_name}', as this workflow has opted-out (opted out experiments are: {opt_out_exp_list})"
2025-07-24T03:40:57.5581855Z                 )
2025-07-24T03:40:57.5582270Z                 continue
2025-07-24T03:40:57.5582713Z 
2025-07-24T03:40:57.5583111Z         if eligible_experiments:
2025-07-24T03:40:57.5583693Z             if experiment_name not in eligible_experiments:
2025-07-24T03:40:57.5584325Z                 exp_list = ", ".join(eligible_experiments)
2025-07-24T03:40:57.5584964Z                 log.info(
2025-07-24T03:40:57.5585751Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2025-07-24T03:40:57.5586548Z                 )
2025-07-24T03:40:57.5587087Z                 continue
2025-07-24T03:40:57.5587591Z         elif not experiment_settings.default:
2025-07-24T03:40:57.5588118Z             log.info(
2025-07-24T03:40:57.5588781Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2025-07-24T03:40:57.5589490Z             )
2025-07-24T03:40:57.5589885Z             continue
2025-07-24T03:40:57.5590289Z 
2025-07-24T03:40:57.5590756Z         # Is any workflow_requestor opted out to this experiment?
2025-07-24T03:40:57.5591362Z         opted_out_users = [
2025-07-24T03:40:57.5591834Z             requestor
2025-07-24T03:40:57.5592318Z             for requestor in workflow_requestors
2025-07-24T03:40:57.5593000Z             if is_user_opted_out(requestor, user_optins, experiment_name)
2025-07-24T03:40:57.5593618Z         ]
2025-07-24T03:40:57.5593980Z 
2025-07-24T03:40:57.5594349Z         if opted_out_users:
2025-07-24T03:40:57.5594922Z             log.info(
2025-07-24T03:40:57.5595559Z                 f"{', '.join(opted_out_users)} have opted out of experiment {experiment_name}."
2025-07-24T03:40:57.5596230Z             )
2025-07-24T03:40:57.5596622Z             continue
2025-07-24T03:40:57.5597032Z 
2025-07-24T03:40:57.5597488Z         # Is any workflow_requestor opted in to this experiment?
2025-07-24T03:40:57.5598095Z         opted_in_users = [
2025-07-24T03:40:57.5598557Z             requestor
2025-07-24T03:40:57.5599047Z             for requestor in workflow_requestors
2025-07-24T03:40:57.5599707Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2025-07-24T03:40:57.5600309Z         ]
2025-07-24T03:40:57.5600670Z 
2025-07-24T03:40:57.5601024Z         enabled = False
2025-07-24T03:40:57.5601490Z         if opted_in_users:
2025-07-24T03:40:57.5602068Z             log.info(
2025-07-24T03:40:57.5602711Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2025-07-24T03:40:57.5603375Z             )
2025-07-24T03:40:57.5603779Z             enabled = True
2025-07-24T03:40:57.5604218Z 
2025-07-24T03:40:57.5604623Z         elif experiment_settings.rollout_perc:
2025-07-24T03:40:57.5605547Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2025-07-24T03:40:57.5606459Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2025-07-24T03:40:57.5607098Z                 log.info(
2025-07-24T03:40:57.5607965Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2025-07-24T03:40:57.5608863Z                 )
2025-07-24T03:40:57.5609299Z                 enabled = True
2025-07-24T03:40:57.5609754Z 
2025-07-24T03:40:57.5610101Z         if enabled:
2025-07-24T03:40:57.5610556Z             label = experiment_name
2025-07-24T03:40:57.5611142Z             if experiment_name == LF_FLEET_EXPERIMENT:
2025-07-24T03:40:57.5611957Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2025-07-24T03:40:57.5612818Z                 #  - If it's enabled, then we always list it's prefix first
2025-07-24T03:40:57.5613582Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2025-07-24T03:40:57.5614229Z                 if is_canary:
2025-07-24T03:40:57.5614841Z                     label += CANARY_FLEET_SUFFIX
2025-07-24T03:40:57.5615385Z                 fleet_prefix = label
2025-07-24T03:40:57.5616006Z             else:
2025-07-24T03:40:57.5616454Z                 prefixes.append(label)
2025-07-24T03:40:57.5616945Z 
2025-07-24T03:40:57.5617304Z     if len(prefixes) > 1:
2025-07-24T03:40:57.5617760Z         log.error(
2025-07-24T03:40:57.5618783Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2025-07-24T03:40:57.5619835Z         )
2025-07-24T03:40:57.5620239Z         prefixes = prefixes[:1]
2025-07-24T03:40:57.5620711Z 
2025-07-24T03:40:57.5621078Z     # Fleet always comes first
2025-07-24T03:40:57.5621566Z     if fleet_prefix:
2025-07-24T03:40:57.5622034Z         prefixes.insert(0, fleet_prefix)
2025-07-24T03:40:57.5622535Z 
2025-07-24T03:40:57.5623055Z     return ".".join(prefixes) + "." if prefixes else ""
2025-07-24T03:40:57.5623638Z 
2025-07-24T03:40:57.5623964Z 
2025-07-24T03:40:57.5624589Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2025-07-24T03:40:57.5625443Z     """
2025-07-24T03:40:57.5626034Z     Gets the first comment of the issue, which contains the desired rollout state.
2025-07-24T03:40:57.5626729Z 
2025-07-24T03:40:57.5627288Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2025-07-24T03:40:57.5627973Z     """
2025-07-24T03:40:57.5628371Z     gh = get_gh_client(github_token)
2025-07-24T03:40:57.5628926Z     issue = get_issue(gh, repo, issue_num)
2025-07-24T03:40:57.5629572Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2025-07-24T03:40:57.5630148Z 
2025-07-24T03:40:57.5630480Z 
2025-07-24T03:40:57.5631065Z def download_json(url: str, headers: dict[str, str], num_retries: int = 3) -> Any:
2025-07-24T03:40:57.5631937Z     for _ in range(num_retries):
2025-07-24T03:40:57.5632415Z         try:
2025-07-24T03:40:57.5632866Z             req = Request(url=url, headers=headers)
2025-07-24T03:40:57.5633520Z             content = urlopen(req, timeout=5).read().decode("utf-8")
2025-07-24T03:40:57.5634161Z             return json.loads(content)
2025-07-24T03:40:57.5634790Z         except Exception as e:
2025-07-24T03:40:57.5635355Z             log.warning(f"Could not download {url}: {e}")
2025-07-24T03:40:57.5635898Z 
2025-07-24T03:40:57.5636461Z     log.warning(f"All {num_retries} retries exhausted, downloading {url} failed")
2025-07-24T03:40:57.5637167Z     return {}
2025-07-24T03:40:57.5637555Z 
2025-07-24T03:40:57.5637886Z 
2025-07-24T03:40:57.5638221Z @cache
2025-07-24T03:40:57.5638847Z def get_pr_info(github_repo: str, github_token: str, pr_number: int) -> dict[str, Any]:
2025-07-24T03:40:57.5639587Z     """
2025-07-24T03:40:57.5639991Z     Dynamically get PR information
2025-07-24T03:40:57.5640482Z     """
2025-07-24T03:40:57.5640986Z     github_api = f"https://api.github.com/repos/{github_repo}"
2025-07-24T03:40:57.5641608Z     headers = {
2025-07-24T03:40:57.5642102Z         "Accept": "application/vnd.github.v3+json",
2025-07-24T03:40:57.5642707Z         "Authorization": f"token {github_token}",
2025-07-24T03:40:57.5643231Z     }
2025-07-24T03:40:57.5643669Z     json_response: dict[str, Any] = download_json(
2025-07-24T03:40:57.5644277Z         url=f"{github_api}/issues/{pr_number}",
2025-07-24T03:40:57.5644902Z         headers=headers,
2025-07-24T03:40:57.5645349Z     )
2025-07-24T03:40:57.5645695Z 
2025-07-24T03:40:57.5646050Z     if not json_response:
2025-07-24T03:40:57.5646653Z         log.warning(f"Failed to get the labels for #{pr_number}")
2025-07-24T03:40:57.5647395Z         return {}
2025-07-24T03:40:57.5647799Z 
2025-07-24T03:40:57.5648155Z     return json_response
2025-07-24T03:40:57.5648601Z 
2025-07-24T03:40:57.5648923Z 
2025-07-24T03:40:57.5649499Z def get_labels(github_repo: str, github_token: str, pr_number: int) -> set[str]:
2025-07-24T03:40:57.5650201Z     """
2025-07-24T03:40:57.5650732Z     Dynamically get the latest list of labels from the pull request
2025-07-24T03:40:57.5651366Z     """
2025-07-24T03:40:57.5651856Z     pr_info = get_pr_info(github_repo, github_token, pr_number)
2025-07-24T03:40:57.5652450Z     return {
2025-07-24T03:40:57.5653057Z         label.get("name") for label in pr_info.get("labels", []) if label.get("name")
2025-07-24T03:40:57.5653726Z     }
2025-07-24T03:40:57.5654073Z 
2025-07-24T03:40:57.5654410Z 
2025-07-24T03:40:57.5654873Z def main() -> None:
2025-07-24T03:40:57.5655316Z     args = parse_args()
2025-07-24T03:40:57.5655755Z 
2025-07-24T03:40:57.5656164Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2025-07-24T03:40:57.5656693Z 
2025-07-24T03:40:57.5657056Z     # Check if the PR is opt-out
2025-07-24T03:40:57.5657553Z     if args.pr_number:
2025-07-24T03:40:57.5658222Z         labels = get_labels(args.github_repo, args.github_token, int(args.pr_number))
2025-07-24T03:40:57.5658962Z         if OPT_OUT_LABEL in labels:
2025-07-24T03:40:57.5659461Z             log.info(
2025-07-24T03:40:57.5660170Z                 f"Opt-out runner determinator because #{args.pr_number} has {OPT_OUT_LABEL} label"
2025-07-24T03:40:57.5660920Z             )
2025-07-24T03:40:57.5661488Z             set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-07-24T03:40:57.5662186Z             sys.exit()
2025-07-24T03:40:57.5662734Z 
2025-07-24T03:40:57.5663075Z     try:
2025-07-24T03:40:57.5663532Z         rollout_state = get_rollout_state_from_issue(
2025-07-24T03:40:57.5664232Z             args.github_token, args.github_issue_repo, args.github_issue
2025-07-24T03:40:57.5664974Z         )
2025-07-24T03:40:57.5665340Z 
2025-07-24T03:40:57.5665736Z         username = get_potential_pr_author(
2025-07-24T03:40:57.5666265Z             args.github_token,
2025-07-24T03:40:57.5666764Z             args.github_repo,
2025-07-24T03:40:57.5667249Z             args.github_actor,
2025-07-24T03:40:57.5667750Z             args.github_ref_type,
2025-07-24T03:40:57.5668256Z             args.github_branch,
2025-07-24T03:40:57.5668724Z         )
2025-07-24T03:40:57.5669090Z 
2025-07-24T03:40:57.5669567Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2025-07-24T03:40:57.5670163Z 
2025-07-24T03:40:57.5670593Z         runner_label_prefix = get_runner_prefix(
2025-07-24T03:40:57.5671147Z             rollout_state,
2025-07-24T03:40:57.5671664Z             (args.github_issue_owner, username),
2025-07-24T03:40:57.5672200Z             args.github_branch,
2025-07-24T03:40:57.5672713Z             args.eligible_experiments,
2025-07-24T03:40:57.5673250Z             args.opt_out_experiments,
2025-07-24T03:40:57.5673755Z             is_canary,
2025-07-24T03:40:57.5674179Z         )
2025-07-24T03:40:57.5674548Z 
2025-07-24T03:40:57.5675009Z     except Exception as e:
2025-07-24T03:40:57.5675478Z         log.error(
2025-07-24T03:40:57.5676170Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2025-07-24T03:40:57.5677040Z         )
2025-07-24T03:40:57.5677414Z 
2025-07-24T03:40:57.5677930Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-07-24T03:40:57.5678575Z 
2025-07-24T03:40:57.5678901Z 
2025-07-24T03:40:57.5679251Z if __name__ == "__main__":
2025-07-24T03:40:57.5679705Z     main()
2025-07-24T03:40:57.5680074Z 
2025-07-24T03:40:57.5680415Z EOF
2025-07-24T03:40:57.5680763Z 
2025-07-24T03:40:57.5681138Z cat runner_determinator.py
2025-07-24T03:40:57.5877016Z shell: /usr/bin/bash -e {0}
2025-07-24T03:40:57.5877778Z env:
2025-07-24T03:40:57.5878423Z   GITHUB_TOKEN: ***
2025-07-24T03:40:57.5878814Z   ISSUE_NUMBER: 5132
2025-07-24T03:40:57.5879243Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2025-07-24T03:40:57.5879711Z   ISSUE_OWNER: 
2025-07-24T03:40:57.5880101Z   CHECK_EXPERIMENTS: 
2025-07-24T03:40:57.5880498Z   OPT_OUT_EXPERIMENTS: lf
2025-07-24T03:40:57.5880913Z   PR_NUMBER: 
2025-07-24T03:40:57.5881285Z ##[endgroup]
2025-07-24T03:40:57.6076879Z # flake8: noqa: G004
2025-07-24T03:40:57.6077199Z 
2025-07-24T03:40:57.6077610Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2025-07-24T03:40:57.6078497Z #       must be kept in sync. You can do it easily by running the following command:
2025-07-24T03:40:57.6079264Z #           python .github/scripts/update_runner_determinator.py
2025-07-24T03:40:57.6079680Z 
2025-07-24T03:40:57.6079844Z """
2025-07-24T03:40:57.6080393Z This runner determinator is used to determine which set of runners to run a
2025-07-24T03:40:57.6081222Z GitHub job on. It uses the first comment of a GitHub issue (by default
2025-07-24T03:40:57.6082067Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2025-07-24T03:40:57.6082834Z of which runners should be used to run which job.
2025-07-24T03:40:57.6083211Z 
2025-07-24T03:40:57.6083580Z The configuration has two parts, the settings and a list of opted-in users,
2025-07-24T03:40:57.6084600Z separated by a line containing "---".  If the line is not present, the
2025-07-24T03:40:57.6085678Z settings are considered to be empty with only the second part, the user
2025-07-24T03:40:57.6086334Z list, defined.
2025-07-24T03:40:57.6086546Z 
2025-07-24T03:40:57.6086895Z The first part is a YAML block that defines the rollout settings. This can be
2025-07-24T03:40:57.6087744Z used to define any settings that are needed to determine which runners to use.
2025-07-24T03:40:57.6088518Z It's fields are defined by the RolloutSettings class below.
2025-07-24T03:40:57.6088932Z 
2025-07-24T03:40:57.6089293Z The second part is a list of users who are explicitly opted in to the LF fleet.
2025-07-24T03:40:57.6090099Z The user list is also a comma separated list of additional features or
2025-07-24T03:40:57.6090783Z experiments which the user could be opted in to.
2025-07-24T03:40:57.6091152Z 
2025-07-24T03:40:57.6091340Z The user list has the following rules:
2025-07-24T03:40:57.6091674Z 
2025-07-24T03:40:57.6091976Z - Users are GitHub usernames, which must start with the @ prefix
2025-07-24T03:40:57.6092768Z - Each user is also a comma-separated list of features/experiments to enable
2025-07-24T03:40:57.6093479Z - A "#" prefix opts the user out of all experiments
2025-07-24T03:40:57.6093837Z 
2025-07-24T03:40:57.6094010Z Example config:
2025-07-24T03:40:57.6094420Z     # A list of experiments that can be opted into.
2025-07-24T03:40:57.6095441Z     # This defines the behavior they'll induce when opted into.
2025-07-24T03:40:57.6096052Z     # Expected syntax is:
2025-07-24T03:40:57.6096651Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2025-07-24T03:40:57.6097559Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2025-07-24T03:40:57.6098137Z 
2025-07-24T03:40:57.6098303Z     experiments:
2025-07-24T03:40:57.6098677Z       lf:
2025-07-24T03:40:57.6099037Z         rollout_percent: 25
2025-07-24T03:40:57.6099638Z         all_branches: false
2025-07-24T03:40:57.6100057Z         default: true
2025-07-24T03:40:57.6100481Z     ---
2025-07-24T03:40:57.6100668Z 
2025-07-24T03:40:57.6100827Z     # Opt-ins:
2025-07-24T03:40:57.6101371Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2025-07-24T03:40:57.6102174Z     # and specifying experiments to enable in a comma-separated list.
2025-07-24T03:40:57.6102897Z     # To always opt out of an experiment, prefix it with a "-".
2025-07-24T03:40:57.6103511Z     # Experiments should be from the above list.
2025-07-24T03:40:57.6103866Z 
2025-07-24T03:40:57.6104041Z     @User1,-lf,split_build
2025-07-24T03:40:57.6104456Z     @User2,lf
2025-07-24T03:40:57.6105047Z     @User3,split_build
2025-07-24T03:40:57.6105438Z """
2025-07-24T03:40:57.6105617Z 
2025-07-24T03:40:57.6105780Z import json
2025-07-24T03:40:57.6106127Z import logging
2025-07-24T03:40:57.6106484Z import os
2025-07-24T03:40:57.6106838Z import random
2025-07-24T03:40:57.6107200Z import re
2025-07-24T03:40:57.6107540Z import sys
2025-07-24T03:40:57.6107922Z from argparse import ArgumentParser
2025-07-24T03:40:57.6108420Z from collections.abc import Iterable
2025-07-24T03:40:57.6108917Z from functools import cache
2025-07-24T03:40:57.6109365Z from logging import LogRecord
2025-07-24T03:40:57.6109833Z from typing import Any, NamedTuple
2025-07-24T03:40:57.6110332Z from urllib.request import Request, urlopen
2025-07-24T03:40:57.6110680Z 
2025-07-24T03:40:57.6110843Z import yaml
2025-07-24T03:40:57.6111203Z from github import Auth, Github
2025-07-24T03:40:57.6111662Z from github.Issue import Issue
2025-07-24T03:40:57.6111945Z 
2025-07-24T03:40:57.6111952Z 
2025-07-24T03:40:57.6112164Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2025-07-24T03:40:57.6112801Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2025-07-24T03:40:57.6113604Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2025-07-24T03:40:57.6114118Z 
2025-07-24T03:40:57.6114341Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2025-07-24T03:40:57.6115235Z GH_OUTPUT_KEY_AMI = "runner-ami"
2025-07-24T03:40:57.6115729Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2025-07-24T03:40:57.6116264Z OPT_OUT_LABEL = "no-runner-experiments"
2025-07-24T03:40:57.6116593Z 
2025-07-24T03:40:57.6116781Z SETTING_EXPERIMENTS = "experiments"
2025-07-24T03:40:57.6117085Z 
2025-07-24T03:40:57.6117260Z LF_FLEET_EXPERIMENT = "lf"
2025-07-24T03:40:57.6117698Z CANARY_FLEET_SUFFIX = ".c"
2025-07-24T03:40:57.6117956Z 
2025-07-24T03:40:57.6117963Z 
2025-07-24T03:40:57.6118141Z class Experiment(NamedTuple):
2025-07-24T03:40:57.6118597Z     rollout_perc: float = (
2025-07-24T03:40:57.6119454Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2025-07-24T03:40:57.6120094Z     )
2025-07-24T03:40:57.6120447Z     all_branches: bool = (
2025-07-24T03:40:57.6121025Z         False  # If True, the experiment is also enabled on the exception branches
2025-07-24T03:40:57.6121658Z     )
2025-07-24T03:40:57.6121997Z     default: bool = (
2025-07-24T03:40:57.6122535Z         True  # If True, the experiment is enabled by default for all queries
2025-07-24T03:40:57.6123137Z     )
2025-07-24T03:40:57.6123319Z 
2025-07-24T03:40:57.6123496Z     # Add more fields as needed
2025-07-24T03:40:57.6123771Z 
2025-07-24T03:40:57.6123777Z 
2025-07-24T03:40:57.6123965Z class Settings(NamedTuple):
2025-07-24T03:40:57.6124377Z     """
2025-07-24T03:40:57.6124911Z     Settings for the experiments that can be opted into.
2025-07-24T03:40:57.6125447Z     """
2025-07-24T03:40:57.6125630Z 
2025-07-24T03:40:57.6125835Z     experiments: dict[str, Experiment] = {}
2025-07-24T03:40:57.6126164Z 
2025-07-24T03:40:57.6126172Z 
2025-07-24T03:40:57.6126369Z class ColorFormatter(logging.Formatter):
2025-07-24T03:40:57.6126952Z     """Color codes the log messages based on the log level"""
2025-07-24T03:40:57.6127361Z 
2025-07-24T03:40:57.6127529Z     COLORS = {
2025-07-24T03:40:57.6127900Z         "WARNING": "\033[33m",  # Yellow
2025-07-24T03:40:57.6128516Z         "ERROR": "\033[31m",  # Red
2025-07-24T03:40:57.6128977Z         "CRITICAL": "\033[31m",  # Red
2025-07-24T03:40:57.6129446Z         "INFO": "\033[0m",  # Reset
2025-07-24T03:40:57.6129890Z         "DEBUG": "\033[0m",  # Reset
2025-07-24T03:40:57.6130331Z     }
2025-07-24T03:40:57.6130513Z 
2025-07-24T03:40:57.6130717Z     def format(self, record: LogRecord) -> str:
2025-07-24T03:40:57.6131415Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2025-07-24T03:40:57.6132151Z         record.msg = f"{log_color}{record.msg}\033[0m"
2025-07-24T03:40:57.6132693Z         return super().format(record)
2025-07-24T03:40:57.6132999Z 
2025-07-24T03:40:57.6133006Z 
2025-07-24T03:40:57.6133199Z handler = logging.StreamHandler()
2025-07-24T03:40:57.6133852Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2025-07-24T03:40:57.6134375Z 
2025-07-24T03:40:57.6134611Z log = logging.getLogger(os.path.basename(__file__))
2025-07-24T03:40:57.6135301Z log.addHandler(handler)
2025-07-24T03:40:57.6135737Z log.setLevel(logging.INFO)
2025-07-24T03:40:57.6136005Z 
2025-07-24T03:40:57.6136012Z 
2025-07-24T03:40:57.6136252Z def set_github_output(key: str, value: str) -> None:
2025-07-24T03:40:57.6136784Z     """
2025-07-24T03:40:57.6137264Z     Defines outputs of the github action that invokes this script
2025-07-24T03:40:57.6137845Z     """
2025-07-24T03:40:57.6138186Z     if not GITHUB_OUTPUT:
2025-07-24T03:40:57.6139184Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2025-07-24T03:40:57.6140243Z         log.warning(
2025-07-24T03:40:57.6141039Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2025-07-24T03:40:57.6141898Z         )
2025-07-24T03:40:57.6151863Z         print(f"::set-output name={key}::{value}")
2025-07-24T03:40:57.6152423Z         return
2025-07-24T03:40:57.6152658Z 
2025-07-24T03:40:57.6153025Z     with open(GITHUB_OUTPUT, "a") as f:
2025-07-24T03:40:57.6153577Z         log.info(f"Setting output: {key}='{value}'")
2025-07-24T03:40:57.6154122Z         f.write(f"{key}={value}\n")
2025-07-24T03:40:57.6154421Z 
2025-07-24T03:40:57.6154428Z 
2025-07-24T03:40:57.6154995Z def _str_comma_separated_to_set(value: str) -> frozenset[str]:
2025-07-24T03:40:57.6155613Z     return frozenset(
2025-07-24T03:40:57.6156178Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2025-07-24T03:40:57.6156811Z     )
2025-07-24T03:40:57.6157001Z 
2025-07-24T03:40:57.6157007Z 
2025-07-24T03:40:57.6157183Z def parse_args() -> Any:
2025-07-24T03:40:57.6157688Z     parser = ArgumentParser("Get dynamic rollout settings")
2025-07-24T03:40:57.6158500Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2025-07-24T03:40:57.6159229Z     parser.add_argument(
2025-07-24T03:40:57.6159657Z         "--github-issue-repo",
2025-07-24T03:40:57.6160101Z         type=str,
2025-07-24T03:40:57.6160484Z         required=False,
2025-07-24T03:40:57.6160913Z         default="pytorch/test-infra",
2025-07-24T03:40:57.6161414Z         help="GitHub repo to get the issue",
2025-07-24T03:40:57.6161892Z     )
2025-07-24T03:40:57.6162241Z     parser.add_argument(
2025-07-24T03:40:57.6162664Z         "--github-repo",
2025-07-24T03:40:57.6163066Z         type=str,
2025-07-24T03:40:57.6163444Z         required=True,
2025-07-24T03:40:57.6163865Z         help="GitHub repo where CI is running",
2025-07-24T03:40:57.6164363Z     )
2025-07-24T03:40:57.6164838Z     parser.add_argument(
2025-07-24T03:40:57.6165409Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2025-07-24T03:40:57.6166038Z     )
2025-07-24T03:40:57.6166395Z     parser.add_argument(
2025-07-24T03:40:57.6166990Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2025-07-24T03:40:57.6167628Z     )
2025-07-24T03:40:57.6167999Z     parser.add_argument(
2025-07-24T03:40:57.6168729Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2025-07-24T03:40:57.6169377Z     )
2025-07-24T03:40:57.6169728Z     parser.add_argument(
2025-07-24T03:40:57.6170327Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2025-07-24T03:40:57.6170998Z     )
2025-07-24T03:40:57.6171334Z     parser.add_argument(
2025-07-24T03:40:57.6171756Z         "--github-ref-type",
2025-07-24T03:40:57.6172173Z         type=str,
2025-07-24T03:40:57.6172549Z         required=True,
2025-07-24T03:40:57.6173001Z         help="Current GitHub ref type, branch or tag",
2025-07-24T03:40:57.6173529Z     )
2025-07-24T03:40:57.6173872Z     parser.add_argument(
2025-07-24T03:40:57.6174313Z         "--eligible-experiments",
2025-07-24T03:40:57.6175005Z         type=_str_comma_separated_to_set,
2025-07-24T03:40:57.6175511Z         required=False,
2025-07-24T03:40:57.6175902Z         default="",
2025-07-24T03:40:57.6224468Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2025-07-24T03:40:57.6225834Z     )
2025-07-24T03:40:57.6226219Z     parser.add_argument(
2025-07-24T03:40:57.6226670Z         "--opt-out-experiments",
2025-07-24T03:40:57.6227152Z         type=_str_comma_separated_to_set,
2025-07-24T03:40:57.6227651Z         required=False,
2025-07-24T03:40:57.6228044Z         default="",
2025-07-24T03:40:57.6228422Z         help=(
2025-07-24T03:40:57.6229053Z             "comma separated list of experiments to opt-out of. If unset, no opt-outs will occur. "
2025-07-24T03:40:57.6230121Z             "If the same experiment is listed both here and in '--eligible-experiments' opt-out will take priority."
2025-07-24T03:40:57.6230905Z         ),
2025-07-24T03:40:57.6231233Z     )
2025-07-24T03:40:57.6231580Z     parser.add_argument(
2025-07-24T03:40:57.6231984Z         "--pr-number",
2025-07-24T03:40:57.6232376Z         type=str,
2025-07-24T03:40:57.6232752Z         required=False,
2025-07-24T03:40:57.6233149Z         default="",
2025-07-24T03:40:57.6233790Z         help="the optional PR number where this is run",
2025-07-24T03:40:57.6234327Z     )
2025-07-24T03:40:57.6234515Z 
2025-07-24T03:40:57.6234906Z     return parser.parse_args()
2025-07-24T03:40:57.6235224Z 
2025-07-24T03:40:57.6235231Z 
2025-07-24T03:40:57.6235616Z def get_gh_client(github_token: str) -> Github:  # type: ignore[no-any-unimported]
2025-07-24T03:40:57.6236323Z     auth = Auth.Token(github_token)
2025-07-24T03:40:57.6236796Z     return Github(auth=auth)
2025-07-24T03:40:57.6237067Z 
2025-07-24T03:40:57.6237074Z 
2025-07-24T03:40:57.6237510Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:  # type: ignore[no-any-unimported]
2025-07-24T03:40:57.6238252Z     repo = gh.get_repo(repo)
2025-07-24T03:40:57.6238725Z     return repo.get_issue(number=issue_num)
2025-07-24T03:40:57.6239057Z 
2025-07-24T03:40:57.6239063Z 
2025-07-24T03:40:57.6239252Z def get_potential_pr_author(
2025-07-24T03:40:57.6239858Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2025-07-24T03:40:57.6240484Z ) -> str:
2025-07-24T03:40:57.6240958Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2025-07-24T03:40:57.6241721Z     # Fetch the actual username from the original PR. The PR number is
2025-07-24T03:40:57.6242421Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2025-07-24T03:40:57.6242816Z 
2025-07-24T03:40:57.6242998Z     gh = get_gh_client(github_token)
2025-07-24T03:40:57.6243304Z 
2025-07-24T03:40:57.6243571Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2025-07-24T03:40:57.6244151Z         split_tag = ref_name.split("/")
2025-07-24T03:40:57.6244618Z         if (
2025-07-24T03:40:57.6245092Z             len(split_tag) == 3
2025-07-24T03:40:57.6245553Z             and split_tag[0] == "ciflow"
2025-07-24T03:40:57.6246039Z             and split_tag[2].isnumeric()
2025-07-24T03:40:57.6246503Z         ):
2025-07-24T03:40:57.6246862Z             pr_number = split_tag[2]
2025-07-24T03:40:57.6247465Z             try:
2025-07-24T03:40:57.6247864Z                 repository = gh.get_repo(repo)
2025-07-24T03:40:57.6248424Z                 pull = repository.get_pull(number=int(pr_number))
2025-07-24T03:40:57.6248996Z             except Exception as e:
2025-07-24T03:40:57.6249473Z                 raise Exception(  # noqa: TRY002
2025-07-24T03:40:57.6250100Z                     f"issue with pull request {pr_number} from repo {repository}"
2025-07-24T03:40:57.6250698Z                 ) from e
2025-07-24T03:40:57.6251210Z             return pull.user.login  # type: ignore[no-any-return]
2025-07-24T03:40:57.6251844Z     # In all other cases, return the original input username
2025-07-24T03:40:57.6252392Z     return username
2025-07-24T03:40:57.6252609Z 
2025-07-24T03:40:57.6252616Z 
2025-07-24T03:40:57.6252835Z def is_exception_branch(branch: str) -> bool:
2025-07-24T03:40:57.6253324Z     """
2025-07-24T03:40:57.6253920Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2025-07-24T03:40:57.6254821Z     """
2025-07-24T03:40:57.6255389Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2025-07-24T03:40:57.6255875Z 
2025-07-24T03:40:57.6255883Z 
2025-07-24T03:40:57.6256065Z def load_yaml(yaml_text: str) -> Any:
2025-07-24T03:40:57.6256520Z     try:
2025-07-24T03:40:57.6256883Z         data = yaml.safe_load(yaml_text)
2025-07-24T03:40:57.6257354Z         return data
2025-07-24T03:40:57.6257739Z     except yaml.YAMLError:
2025-07-24T03:40:57.6258179Z         log.exception("Error loading YAML")
2025-07-24T03:40:57.6258655Z         raise
2025-07-24T03:40:57.6258849Z 
2025-07-24T03:40:57.6258856Z 
2025-07-24T03:40:57.6259247Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> tuple[str, str]:
2025-07-24T03:40:57.6259939Z     """
2025-07-24T03:40:57.6260507Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2025-07-24T03:40:57.6261066Z 
2025-07-24T03:40:57.6261526Z     If the issue body contains "---" then the text above that is the settings
2025-07-24T03:40:57.6262241Z     and the text below is the list of opted in users.
2025-07-24T03:40:57.6262616Z 
2025-07-24T03:40:57.6262966Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2025-07-24T03:40:57.6263636Z     """
2025-07-24T03:40:57.6264048Z     rollout_state_parts = rollout_state.split("---")
2025-07-24T03:40:57.6264612Z     if len(rollout_state_parts) >= 2:
2025-07-24T03:40:57.6265409Z         return rollout_state_parts[0], rollout_state_parts[1]
2025-07-24T03:40:57.6265953Z     else:
2025-07-24T03:40:57.6266312Z         return "", rollout_state
2025-07-24T03:40:57.6266590Z 
2025-07-24T03:40:57.6266598Z 
2025-07-24T03:40:57.6266787Z class UserOptins(dict[str, list[str]]):
2025-07-24T03:40:57.6267259Z     """
2025-07-24T03:40:57.6267733Z     Dictionary of users with a list of features they have opted into
2025-07-24T03:40:57.6268368Z     """
2025-07-24T03:40:57.6268552Z 
2025-07-24T03:40:57.6268559Z 
2025-07-24T03:40:57.6268894Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2025-07-24T03:40:57.6269504Z     """
2025-07-24T03:40:57.6270166Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2025-07-24T03:40:57.6270802Z 
2025-07-24T03:40:57.6271388Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2025-07-24T03:40:57.6272320Z         - Example line: "@User1,lf,split_build"
2025-07-24T03:40:57.6272954Z         - A "#" prefix indicates the user is opted out of all experiments
2025-07-24T03:40:57.6273409Z 
2025-07-24T03:40:57.6273414Z 
2025-07-24T03:40:57.6273565Z     """
2025-07-24T03:40:57.6273919Z     optins = UserOptins()
2025-07-24T03:40:57.6274372Z     for user in user_optin_text.split("\n"):
2025-07-24T03:40:57.6275004Z         user = user.strip("\r\n\t -")
2025-07-24T03:40:57.6275502Z         if not user or not user.startswith("@"):
2025-07-24T03:40:57.6276167Z             # Not a valid user. Skip
2025-07-24T03:40:57.6276618Z             continue
2025-07-24T03:40:57.6276847Z 
2025-07-24T03:40:57.6277000Z         if user:
2025-07-24T03:40:57.6277399Z             usr_name = user.split(",")[0].strip("@")
2025-07-24T03:40:57.6278046Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2025-07-24T03:40:57.6278503Z 
2025-07-24T03:40:57.6278668Z     return optins
2025-07-24T03:40:57.6278884Z 
2025-07-24T03:40:57.6278892Z 
2025-07-24T03:40:57.6279154Z def is_valid_experiment_name(experiment_name: str) -> bool:
2025-07-24T03:40:57.6279721Z     """
2025-07-24T03:40:57.6280089Z     Check if the experiment name is valid.
2025-07-24T03:40:57.6280583Z     A valid name:
2025-07-24T03:40:57.6281169Z         - Contains only alphanumeric characters and the special characters "_" & "-"
2025-07-24T03:40:57.6282039Z         - The special characters "_" & "-" shouldn't be the first or last characters
2025-07-24T03:40:57.6282717Z         - Cannot contain spaces
2025-07-24T03:40:57.6283143Z     """
2025-07-24T03:40:57.6283327Z 
2025-07-24T03:40:57.6283574Z     valid_char_regex = r"^[a-zA-Z0-9]([\w-]*[a-zA-Z0-9])?$"
2025-07-24T03:40:57.6284214Z     valid = bool(re.match(valid_char_regex, experiment_name))
2025-07-24T03:40:57.6284726Z 
2025-07-24T03:40:57.6284889Z     if valid:
2025-07-24T03:40:57.6285232Z         return True
2025-07-24T03:40:57.6285453Z 
2025-07-24T03:40:57.6285605Z     log.error(
2025-07-24T03:40:57.6286940Z         f"Invalid experiment name: {experiment_name}. Experiment names should only contain alphanumeric characters, '_', and '-'. They cannot contain spaces, and the special characters '_' and '-' cannot be the first or last characters."
2025-07-24T03:40:57.6288375Z     )
2025-07-24T03:40:57.6288713Z     return False
2025-07-24T03:40:57.6288922Z 
2025-07-24T03:40:57.6288928Z 
2025-07-24T03:40:57.6289217Z def parse_settings_from_text(settings_text: str) -> Settings:
2025-07-24T03:40:57.6289801Z     """
2025-07-24T03:40:57.6290463Z     Parse the experiments from the issue body into a list of ExperimentSettings
2025-07-24T03:40:57.6291127Z     """
2025-07-24T03:40:57.6291458Z     try:
2025-07-24T03:40:57.6291807Z         if settings_text:
2025-07-24T03:40:57.6292472Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2025-07-24T03:40:57.6293202Z             # for easy reading
2025-07-24T03:40:57.6293933Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2025-07-24T03:40:57.6294850Z             #       the backtick character in shell commands.
2025-07-24T03:40:57.6295410Z             backtick = chr(96)  # backtick character
2025-07-24T03:40:57.6296023Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2025-07-24T03:40:57.6296630Z             settings = load_yaml(settings_text)
2025-07-24T03:40:57.6296971Z 
2025-07-24T03:40:57.6297352Z             # For now we just load experiments. We can expand this if/when we add more settings
2025-07-24T03:40:57.6298048Z             experiments = {}
2025-07-24T03:40:57.6298327Z 
2025-07-24T03:40:57.6298681Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2025-07-24T03:40:57.6299376Z                 if not is_valid_experiment_name(exp_name):
2025-07-24T03:40:57.6300403Z                     # Exclude invalid experiments from the list. We log an error, but don't raise an exception so that other experiments can still be processed.
2025-07-24T03:40:57.6301371Z                     continue
2025-07-24T03:40:57.6301631Z 
2025-07-24T03:40:57.6301806Z                 valid_settings = {}
2025-07-24T03:40:57.6302278Z                 for setting in exp_settings:
2025-07-24T03:40:57.6302799Z                     if setting not in Experiment._fields:
2025-07-24T03:40:57.6303312Z                         log.warning(
2025-07-24T03:40:57.6303966Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2025-07-24T03:40:57.6305306Z                         )
2025-07-24T03:40:57.6305740Z                     else:
2025-07-24T03:40:57.6306211Z                         valid_settings[setting] = exp_settings[setting]
2025-07-24T03:40:57.6306608Z 
2025-07-24T03:40:57.6306864Z                 experiments[exp_name] = Experiment(**valid_settings)
2025-07-24T03:40:57.6307458Z             return Settings(experiments)
2025-07-24T03:40:57.6307791Z 
2025-07-24T03:40:57.6307961Z     except Exception:
2025-07-24T03:40:57.6308406Z         log.exception("Failed to parse settings")
2025-07-24T03:40:57.6308759Z 
2025-07-24T03:40:57.6308927Z     return Settings()
2025-07-24T03:40:57.6309166Z 
2025-07-24T03:40:57.6309173Z 
2025-07-24T03:40:57.6309403Z def parse_settings(rollout_state: str) -> Settings:
2025-07-24T03:40:57.6309933Z     """
2025-07-24T03:40:57.6310339Z     Parse settings, if any, from the rollout state.
2025-07-24T03:40:57.6310703Z 
2025-07-24T03:40:57.6311034Z     If the issue body contains "---" then the text above that is the settings
2025-07-24T03:40:57.6311735Z     and the text below is the list of opted in users.
2025-07-24T03:40:57.6312109Z 
2025-07-24T03:40:57.6312494Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2025-07-24T03:40:57.6313180Z     """
2025-07-24T03:40:57.6313705Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2025-07-24T03:40:57.6314405Z     return parse_settings_from_text(settings_text)
2025-07-24T03:40:57.6314982Z 
2025-07-24T03:40:57.6314988Z 
2025-07-24T03:40:57.6315227Z def parse_users(rollout_state: str) -> UserOptins:
2025-07-24T03:40:57.6315753Z     """
2025-07-24T03:40:57.6316111Z     Parse users from the rollout state.
2025-07-24T03:40:57.6316428Z 
2025-07-24T03:40:57.6316587Z     """
2025-07-24T03:40:57.6317070Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2025-07-24T03:40:57.6317759Z     return parse_user_opt_in_from_text(users_text)
2025-07-24T03:40:57.6318122Z 
2025-07-24T03:40:57.6318128Z 
2025-07-24T03:40:57.6318647Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-07-24T03:40:57.6319346Z     """
2025-07-24T03:40:57.6319727Z     Check if a user is opted into an experiment
2025-07-24T03:40:57.6320220Z     """
2025-07-24T03:40:57.6320627Z     return experiment_name in user_optins.get(user, [])
2025-07-24T03:40:57.6321006Z 
2025-07-24T03:40:57.6321013Z 
2025-07-24T03:40:57.6321401Z def is_user_opted_out(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-07-24T03:40:57.6322090Z     """
2025-07-24T03:40:57.6322503Z     Check if a user explicitly opted out of an experiment
2025-07-24T03:40:57.6323045Z     """
2025-07-24T03:40:57.6323500Z     # if the experiment is prefixed with a "-", then it's an opt-out
2025-07-24T03:40:57.6324128Z     experiment_optout = "-" + experiment_name
2025-07-24T03:40:57.6324819Z     if experiment_optout not in user_optins.get(user, []):
2025-07-24T03:40:57.6325370Z         return False
2025-07-24T03:40:57.6325597Z 
2025-07-24T03:40:57.6325848Z     if is_user_opted_in(user, user_optins, experiment_name):
2025-07-24T03:40:57.6326397Z         log.warning(
2025-07-24T03:40:57.6327128Z             f"User {user} is opted into experiment {experiment_name}, but also opted out of it. Defaulting to opting out"
2025-07-24T03:40:57.6327951Z         )
2025-07-24T03:40:57.6328135Z 
2025-07-24T03:40:57.6328289Z     return True
2025-07-24T03:40:57.6328495Z 
2025-07-24T03:40:57.6328501Z 
2025-07-24T03:40:57.6328674Z def get_runner_prefix(
2025-07-24T03:40:57.6329077Z     rollout_state: str,
2025-07-24T03:40:57.6329505Z     workflow_requestors: Iterable[str],
2025-07-24T03:40:57.6329981Z     branch: str,
2025-07-24T03:40:57.6330423Z     eligible_experiments: frozenset[str] = frozenset(),
2025-07-24T03:40:57.6331029Z     opt_out_experiments: frozenset[str] = frozenset(),
2025-07-24T03:40:57.6331569Z     is_canary: bool = False,
2025-07-24T03:40:57.6331987Z ) -> str:
2025-07-24T03:40:57.6332486Z     settings = parse_settings(rollout_state)
2025-07-24T03:40:57.6333019Z     user_optins = parse_users(rollout_state)
2025-07-24T03:40:57.6333350Z 
2025-07-24T03:40:57.6333514Z     fleet_prefix = ""
2025-07-24T03:40:57.6333902Z     prefixes = []
2025-07-24T03:40:57.6334480Z     for experiment_name, experiment_settings in settings.experiments.items():
2025-07-24T03:40:57.6335445Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2025-07-24T03:40:57.6336101Z             log.info(
2025-07-24T03:40:57.6336724Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2025-07-24T03:40:57.6337419Z             )
2025-07-24T03:40:57.6337758Z             continue
2025-07-24T03:40:57.6337982Z 
2025-07-24T03:40:57.6338159Z         if opt_out_experiments:
2025-07-24T03:40:57.6338636Z             if experiment_name in opt_out_experiments:
2025-07-24T03:40:57.6339219Z                 opt_out_exp_list = ", ".join(opt_out_experiments)
2025-07-24T03:40:57.6339758Z                 log.info(
2025-07-24T03:40:57.6340615Z                     f"Skipping experiment '{experiment_name}', as this workflow has opted-out (opted out experiments are: {opt_out_exp_list})"
2025-07-24T03:40:57.6341520Z                 )
2025-07-24T03:40:57.6341876Z                 continue
2025-07-24T03:40:57.6342115Z 
2025-07-24T03:40:57.6342297Z         if eligible_experiments:
2025-07-24T03:40:57.6342821Z             if experiment_name not in eligible_experiments:
2025-07-24T03:40:57.6343406Z                 exp_list = ", ".join(eligible_experiments)
2025-07-24T03:40:57.6343923Z                 log.info(
2025-07-24T03:40:57.6344743Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2025-07-24T03:40:57.6345515Z                 )
2025-07-24T03:40:57.6345871Z                 continue
2025-07-24T03:40:57.6346302Z         elif not experiment_settings.default:
2025-07-24T03:40:57.6346792Z             log.info(
2025-07-24T03:40:57.6347522Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2025-07-24T03:40:57.6348205Z             )
2025-07-24T03:40:57.6348548Z             continue
2025-07-24T03:40:57.6348790Z 
2025-07-24T03:40:57.6349042Z         # Is any workflow_requestor opted out to this experiment?
2025-07-24T03:40:57.6349614Z         opted_out_users = [
2025-07-24T03:40:57.6350025Z             requestor
2025-07-24T03:40:57.6350436Z             for requestor in workflow_requestors
2025-07-24T03:40:57.6351052Z             if is_user_opted_out(requestor, user_optins, experiment_name)
2025-07-24T03:40:57.6351625Z         ]
2025-07-24T03:40:57.6351815Z 
2025-07-24T03:40:57.6351980Z         if opted_out_users:
2025-07-24T03:40:57.6352385Z             log.info(
2025-07-24T03:40:57.6352948Z                 f"{', '.join(opted_out_users)} have opted out of experiment {experiment_name}."
2025-07-24T03:40:57.6353589Z             )
2025-07-24T03:40:57.6353931Z             continue
2025-07-24T03:40:57.6354157Z 
2025-07-24T03:40:57.6354406Z         # Is any workflow_requestor opted in to this experiment?
2025-07-24T03:40:57.6355064Z         opted_in_users = [
2025-07-24T03:40:57.6355470Z             requestor
2025-07-24T03:40:57.6355879Z             for requestor in workflow_requestors
2025-07-24T03:40:57.6356486Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2025-07-24T03:40:57.6357043Z         ]
2025-07-24T03:40:57.6357222Z 
2025-07-24T03:40:57.6357379Z         enabled = False
2025-07-24T03:40:57.6357775Z         if opted_in_users:
2025-07-24T03:40:57.6358177Z             log.info(
2025-07-24T03:40:57.6358713Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2025-07-24T03:40:57.6359336Z             )
2025-07-24T03:40:57.6359691Z             enabled = True
2025-07-24T03:40:57.6359938Z 
2025-07-24T03:40:57.6360142Z         elif experiment_settings.rollout_perc:
2025-07-24T03:40:57.6360907Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2025-07-24T03:40:57.6361886Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2025-07-24T03:40:57.6362484Z                 log.info(
2025-07-24T03:40:57.6363282Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2025-07-24T03:40:57.6364120Z                 )
2025-07-24T03:40:57.6364482Z                 enabled = True
2025-07-24T03:40:57.6364850Z 
2025-07-24T03:40:57.6365001Z         if enabled:
2025-07-24T03:40:57.6365384Z             label = experiment_name
2025-07-24T03:40:57.6365883Z             if experiment_name == LF_FLEET_EXPERIMENT:
2025-07-24T03:40:57.6366642Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2025-07-24T03:40:57.6367451Z                 #  - If it's enabled, then we always list it's prefix first
2025-07-24T03:40:57.6368174Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2025-07-24T03:40:57.6368781Z                 if is_canary:
2025-07-24T03:40:57.6369232Z                     label += CANARY_FLEET_SUFFIX
2025-07-24T03:40:57.6369734Z                 fleet_prefix = label
2025-07-24T03:40:57.6370170Z             else:
2025-07-24T03:40:57.6370559Z                 prefixes.append(label)
2025-07-24T03:40:57.6370874Z 
2025-07-24T03:40:57.6371050Z     if len(prefixes) > 1:
2025-07-24T03:40:57.6371448Z         log.error(
2025-07-24T03:40:57.6372390Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2025-07-24T03:40:57.6373407Z         )
2025-07-24T03:40:57.6373762Z         prefixes = prefixes[:1]
2025-07-24T03:40:57.6374036Z 
2025-07-24T03:40:57.6374208Z     # Fleet always comes first
2025-07-24T03:40:57.6374735Z     if fleet_prefix:
2025-07-24T03:40:57.6375145Z         prefixes.insert(0, fleet_prefix)
2025-07-24T03:40:57.6375482Z 
2025-07-24T03:40:57.6375838Z     return ".".join(prefixes) + "." if prefixes else ""
2025-07-24T03:40:57.6376218Z 
2025-07-24T03:40:57.6376225Z 
2025-07-24T03:40:57.6376632Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2025-07-24T03:40:57.6377333Z     """
2025-07-24T03:40:57.6377859Z     Gets the first comment of the issue, which contains the desired rollout state.
2025-07-24T03:40:57.6378366Z 
2025-07-24T03:40:57.6378720Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2025-07-24T03:40:57.6379365Z     """
2025-07-24T03:40:57.6379715Z     gh = get_gh_client(github_token)
2025-07-24T03:40:57.6380198Z     issue = get_issue(gh, repo, issue_num)
2025-07-24T03:40:57.6380772Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2025-07-24T03:40:57.6381173Z 
2025-07-24T03:40:57.6381179Z 
2025-07-24T03:40:57.6381543Z def download_json(url: str, headers: dict[str, str], num_retries: int = 3) -> Any:
2025-07-24T03:40:57.6382244Z     for _ in range(num_retries):
2025-07-24T03:40:57.6382680Z         try:
2025-07-24T03:40:57.6383068Z             req = Request(url=url, headers=headers)
2025-07-24T03:40:57.6383669Z             content = urlopen(req, timeout=5).read().decode("utf-8")
2025-07-24T03:40:57.6384252Z             return json.loads(content)
2025-07-24T03:40:57.6384832Z         except Exception as e:
2025-07-24T03:40:57.6385316Z             log.warning(f"Could not download {url}: {e}")
2025-07-24T03:40:57.6385678Z 
2025-07-24T03:40:57.6386033Z     log.warning(f"All {num_retries} retries exhausted, downloading {url} failed")
2025-07-24T03:40:57.6386681Z     return {}
2025-07-24T03:40:57.6386887Z 
2025-07-24T03:40:57.6386893Z 
2025-07-24T03:40:57.6387037Z @cache
2025-07-24T03:40:57.6387598Z def get_pr_info(github_repo: str, github_token: str, pr_number: int) -> dict[str, Any]:
2025-07-24T03:40:57.6388280Z     """
2025-07-24T03:40:57.6388636Z     Dynamically get PR information
2025-07-24T03:40:57.6389207Z     """
2025-07-24T03:40:57.6389657Z     github_api = f"https://api.github.com/repos/{github_repo}"
2025-07-24T03:40:57.6390218Z     headers = {
2025-07-24T03:40:57.6390633Z         "Accept": "application/vnd.github.v3+json",
2025-07-24T03:40:57.6391174Z         "Authorization": f"token {github_token}",
2025-07-24T03:40:57.6391665Z     }
2025-07-24T03:40:57.6392046Z     json_response: dict[str, Any] = download_json(
2025-07-24T03:40:57.6392600Z         url=f"{github_api}/issues/{pr_number}",
2025-07-24T03:40:57.6393100Z         headers=headers,
2025-07-24T03:40:57.6393482Z     )
2025-07-24T03:40:57.6393657Z 
2025-07-24T03:40:57.6393835Z     if not json_response:
2025-07-24T03:40:57.6394341Z         log.warning(f"Failed to get the labels for #{pr_number}")
2025-07-24T03:40:57.6395017Z         return {}
2025-07-24T03:40:57.6395227Z 
2025-07-24T03:40:57.6395395Z     return json_response
2025-07-24T03:40:57.6395653Z 
2025-07-24T03:40:57.6395660Z 
2025-07-24T03:40:57.6396026Z def get_labels(github_repo: str, github_token: str, pr_number: int) -> set[str]:
2025-07-24T03:40:57.6396702Z     """
2025-07-24T03:40:57.6397175Z     Dynamically get the latest list of labels from the pull request
2025-07-24T03:40:57.6397776Z     """
2025-07-24T03:40:57.6398206Z     pr_info = get_pr_info(github_repo, github_token, pr_number)
2025-07-24T03:40:57.6398764Z     return {
2025-07-24T03:40:57.6399288Z         label.get("name") for label in pr_info.get("labels", []) if label.get("name")
2025-07-24T03:40:57.6399931Z     }
2025-07-24T03:40:57.6400108Z 
2025-07-24T03:40:57.6400114Z 
2025-07-24T03:40:57.6400274Z def main() -> None:
2025-07-24T03:40:57.6400657Z     args = parse_args()
2025-07-24T03:40:57.6400893Z 
2025-07-24T03:40:57.6401098Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2025-07-24T03:40:57.6401446Z 
2025-07-24T03:40:57.6401621Z     # Check if the PR is opt-out
2025-07-24T03:40:57.6402074Z     if args.pr_number:
2025-07-24T03:40:57.6402660Z         labels = get_labels(args.github_repo, args.github_token, int(args.pr_number))
2025-07-24T03:40:57.6403473Z         if OPT_OUT_LABEL in labels:
2025-07-24T03:40:57.6403923Z             log.info(
2025-07-24T03:40:57.6404540Z                 f"Opt-out runner determinator because #{args.pr_number} has {OPT_OUT_LABEL} label"
2025-07-24T03:40:57.6405347Z             )
2025-07-24T03:40:57.6405835Z             set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-07-24T03:40:57.6406454Z             sys.exit()
2025-07-24T03:40:57.6406684Z 
2025-07-24T03:40:57.6406839Z     try:
2025-07-24T03:40:57.6407228Z         rollout_state = get_rollout_state_from_issue(
2025-07-24T03:40:57.6407869Z             args.github_token, args.github_issue_repo, args.github_issue
2025-07-24T03:40:57.6408447Z         )
2025-07-24T03:40:57.6408627Z 
2025-07-24T03:40:57.6408815Z         username = get_potential_pr_author(
2025-07-24T03:40:57.6409304Z             args.github_token,
2025-07-24T03:40:57.6409740Z             args.github_repo,
2025-07-24T03:40:57.6410169Z             args.github_actor,
2025-07-24T03:40:57.6410612Z             args.github_ref_type,
2025-07-24T03:40:57.6411061Z             args.github_branch,
2025-07-24T03:40:57.6411473Z         )
2025-07-24T03:40:57.6411655Z 
2025-07-24T03:40:57.6411910Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2025-07-24T03:40:57.6412317Z 
2025-07-24T03:40:57.6412513Z         runner_label_prefix = get_runner_prefix(
2025-07-24T03:40:57.6413014Z             rollout_state,
2025-07-24T03:40:57.6413452Z             (args.github_issue_owner, username),
2025-07-24T03:40:57.6413947Z             args.github_branch,
2025-07-24T03:40:57.6414399Z             args.eligible_experiments,
2025-07-24T03:40:57.6414984Z             args.opt_out_experiments,
2025-07-24T03:40:57.6415439Z             is_canary,
2025-07-24T03:40:57.6415812Z         )
2025-07-24T03:40:57.6415993Z 
2025-07-24T03:40:57.6416166Z     except Exception as e:
2025-07-24T03:40:57.6416574Z         log.error(
2025-07-24T03:40:57.6417171Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2025-07-24T03:40:57.6417998Z         )
2025-07-24T03:40:57.6418182Z 
2025-07-24T03:40:57.6418484Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-07-24T03:40:57.6418930Z 
2025-07-24T03:40:57.6418937Z 
2025-07-24T03:40:57.6419100Z if __name__ == "__main__":
2025-07-24T03:40:57.6419493Z     main()
2025-07-24T03:40:57.6419675Z 
2025-07-24T03:40:57.6505333Z ##[group]Run python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2025-07-24T03:40:57.6506184Z python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2025-07-24T03:40:57.6533551Z shell: /usr/bin/bash -e {0}
2025-07-24T03:40:57.6533980Z env:
2025-07-24T03:40:57.6534554Z   GITHUB_TOKEN: ***
2025-07-24T03:40:57.6535074Z   ISSUE_NUMBER: 5132
2025-07-24T03:40:57.6535482Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2025-07-24T03:40:57.6535947Z   ISSUE_OWNER: 
2025-07-24T03:40:57.6536313Z   CHECK_EXPERIMENTS: 
2025-07-24T03:40:57.6536711Z   OPT_OUT_EXPERIMENTS: lf
2025-07-24T03:40:57.6537133Z   PR_NUMBER: 
2025-07-24T03:40:57.6537483Z ##[endgroup]
2025-07-24T03:40:57.9793183Z Defaulting to user installation because normal site-packages is not writeable
2025-07-24T03:40:58.2623764Z Collecting urllib3==1.26.18
2025-07-24T03:40:58.2965497Z   Downloading urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
2025-07-24T03:40:58.3201358Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 4.1 MB/s eta 0:00:00
2025-07-24T03:40:58.3420590Z Collecting PyGithub==2.3.0
2025-07-24T03:40:58.3463365Z   Downloading PyGithub-2.3.0-py3-none-any.whl.metadata (3.8 kB)
2025-07-24T03:40:58.3887863Z Collecting pynacl>=1.4.0 (from PyGithub==2.3.0)
2025-07-24T03:40:58.3916996Z   Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (8.6 kB)
2025-07-24T03:40:58.3959135Z Requirement already satisfied: requests>=2.14.0 in /usr/lib/python3/dist-packages (from PyGithub==2.3.0) (2.31.0)
2025-07-24T03:40:58.3975386Z Requirement already satisfied: pyjwt>=2.4.0 in /usr/lib/python3/dist-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.3.0) (2.7.0)
2025-07-24T03:40:58.3989731Z Requirement already satisfied: typing-extensions>=4.0.0 in /usr/lib/python3/dist-packages (from PyGithub==2.3.0) (4.10.0)
2025-07-24T03:40:58.4227112Z Collecting Deprecated (from PyGithub==2.3.0)
2025-07-24T03:40:58.4254175Z   Downloading Deprecated-1.2.18-py2.py3-none-any.whl.metadata (5.7 kB)
2025-07-24T03:40:58.4477526Z Requirement already satisfied: cryptography>=3.4.0 in /usr/lib/python3/dist-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.3.0) (41.0.7)
2025-07-24T03:40:58.5531003Z Collecting cffi>=1.4.1 (from pynacl>=1.4.0->PyGithub==2.3.0)
2025-07-24T03:40:58.5580221Z   Downloading cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.5 kB)
2025-07-24T03:40:58.6614984Z Collecting wrapt<2,>=1.10 (from Deprecated->PyGithub==2.3.0)
2025-07-24T03:40:58.6645565Z   Downloading wrapt-1.17.2-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.4 kB)
2025-07-24T03:40:58.6822336Z Collecting pycparser (from cffi>=1.4.1->pynacl>=1.4.0->PyGithub==2.3.0)
2025-07-24T03:40:58.6856533Z   Downloading pycparser-2.22-py3-none-any.whl.metadata (943 bytes)
2025-07-24T03:40:58.7088751Z Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
2025-07-24T03:40:58.7145237Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.8/143.8 kB 36.0 MB/s eta 0:00:00
2025-07-24T03:40:58.7227416Z Downloading PyGithub-2.3.0-py3-none-any.whl (354 kB)
2025-07-24T03:40:58.7293057Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 354.4/354.4 kB 71.5 MB/s eta 0:00:00
2025-07-24T03:40:58.7327118Z Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
2025-07-24T03:40:58.7414489Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 856.7/856.7 kB 124.6 MB/s eta 0:00:00
2025-07-24T03:40:58.7444050Z Downloading Deprecated-1.2.18-py2.py3-none-any.whl (10.0 kB)
2025-07-24T03:40:58.7498651Z Downloading cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (479 kB)
2025-07-24T03:40:58.7563206Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 479.4/479.4 kB 102.4 MB/s eta 0:00:00
2025-07-24T03:40:58.7670277Z Downloading wrapt-1.17.2-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (89 kB)
2025-07-24T03:40:58.7737449Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.2/89.2 kB 15.2 MB/s eta 0:00:00
2025-07-24T03:40:58.7861314Z Downloading pycparser-2.22-py3-none-any.whl (117 kB)
2025-07-24T03:40:58.7904527Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 kB 40.5 MB/s eta 0:00:00
2025-07-24T03:40:59.0686147Z Installing collected packages: wrapt, urllib3, pycparser, Deprecated, cffi, pynacl, PyGithub
2025-07-24T03:40:59.6000026Z Successfully installed Deprecated-1.2.18 PyGithub-2.3.0 cffi-1.17.1 pycparser-2.22 pynacl-1.5.0 urllib3-1.26.18 wrapt-1.17.2
2025-07-24T03:40:59.6876730Z ##[group]Run curr_branch="ciflow/inductor/149961"
2025-07-24T03:40:59.6877205Z curr_branch="ciflow/inductor/149961"
2025-07-24T03:40:59.6877517Z curr_ref_type="tag"
2025-07-24T03:40:59.6877810Z echo "Current branch is '$curr_branch'"
2025-07-24T03:40:59.6878119Z 
2025-07-24T03:40:59.6878365Z python3 runner_determinator.py \
2025-07-24T03:40:59.6878691Z   --github-token "$GITHUB_TOKEN" \
2025-07-24T03:40:59.6878999Z   --github-issue "$ISSUE_NUMBER" \
2025-07-24T03:40:59.6879317Z   --github-branch "$curr_branch" \
2025-07-24T03:40:59.6879643Z   --github-actor "$TRIGGERING_ACTOR" \
2025-07-24T03:40:59.6879968Z   --github-issue-owner "$ISSUE_OWNER" \
2025-07-24T03:40:59.6880288Z   --github-ref-type "$curr_ref_type" \
2025-07-24T03:40:59.6880632Z   --github-repo "$GITHUB_REPOSITORY" \
2025-07-24T03:40:59.6881043Z   --eligible-experiments "$CHECK_EXPERIMENTS" \
2025-07-24T03:40:59.6881414Z   --opt-out-experiments "$OPT_OUT_EXPERIMENTS" \
2025-07-24T03:40:59.6881758Z   --pr-number "${PR_NUMBER}"
2025-07-24T03:40:59.6910633Z shell: /usr/bin/bash -e {0}
2025-07-24T03:40:59.6910911Z env:
2025-07-24T03:40:59.6911583Z   GITHUB_TOKEN: ***
2025-07-24T03:40:59.6911810Z   ISSUE_NUMBER: 5132
2025-07-24T03:40:59.6912047Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2025-07-24T03:40:59.6912310Z   ISSUE_OWNER: 
2025-07-24T03:40:59.6912535Z   CHECK_EXPERIMENTS: 
2025-07-24T03:40:59.6912764Z   OPT_OUT_EXPERIMENTS: lf
2025-07-24T03:40:59.6912998Z   PR_NUMBER: 
2025-07-24T03:40:59.6913195Z ##[endgroup]
2025-07-24T03:40:59.6961509Z Current branch is 'ciflow/inductor/149961'
2025-07-24T03:41:02.3755571Z INFO    : Skipping experiment 'lf', as this workflow has opted-out (opted out experiments are: lf)
2025-07-24T03:41:02.3756762Z INFO    : Setting output: label-type=''
2025-07-24T03:41:02.4124805Z Evaluate and set job outputs
2025-07-24T03:41:02.4132665Z Cleaning up orphan processes