2025-07-24T03:40:56.6326383Z Current runner version: '2.326.0'
2025-07-24T03:40:56.6349491Z ##[group]Runner Image Provisioner
2025-07-24T03:40:56.6350299Z Hosted Compute Agent
2025-07-24T03:40:56.6350841Z Version: 20250711.363
2025-07-24T03:40:56.6351368Z Commit: 6785254374ce925a23743850c1cb91912ce5c14c
2025-07-24T03:40:56.6352325Z Build Date: 2025-07-11T20:04:25Z
2025-07-24T03:40:56.6352907Z ##[endgroup]
2025-07-24T03:40:56.6353405Z ##[group]Operating System
2025-07-24T03:40:56.6354101Z Ubuntu
2025-07-24T03:40:56.6354546Z 24.04.2
2025-07-24T03:40:56.6354988Z LTS
2025-07-24T03:40:56.6355468Z ##[endgroup]
2025-07-24T03:40:56.6355972Z ##[group]Runner Image
2025-07-24T03:40:56.6356498Z Image: ubuntu-24.04
2025-07-24T03:40:56.6357044Z Version: 20250720.1.0
2025-07-24T03:40:56.6358039Z Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20250720.1/images/ubuntu/Ubuntu2404-Readme.md
2025-07-24T03:40:56.6359582Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20250720.1
2025-07-24T03:40:56.6360511Z ##[endgroup]
2025-07-24T03:40:56.6361920Z ##[group]GITHUB_TOKEN Permissions
2025-07-24T03:40:56.6364010Z Contents: read
2025-07-24T03:40:56.6364548Z Metadata: read
2025-07-24T03:40:56.6365077Z Packages: read
2025-07-24T03:40:56.6365622Z ##[endgroup]
2025-07-24T03:40:56.6367536Z Secret source: Actions
2025-07-24T03:40:56.6368278Z Prepare workflow directory
2025-07-24T03:40:56.6885779Z Prepare all required actions
2025-07-24T03:40:56.6941752Z Uses: pytorch/pytorch/.github/workflows/_runner-determinator.yml@refs/heads/main (07df6ba7f5597488a93b3855d52d2ead55675125)
2025-07-24T03:40:56.6946893Z ##[group] Inputs
2025-07-24T03:40:56.6947510Z   check_experiments: 
2025-07-24T03:40:56.6948018Z   opt_out_experiments: 
2025-07-24T03:40:56.6948617Z   triggering_actor: pytorch-bot[bot]
2025-07-24T03:40:56.6949202Z   issue_owner: 
2025-07-24T03:40:56.6949715Z   curr_branch: ciflow/trunk/149961
2025-07-24T03:40:56.6950478Z   curr_ref_type: tag
2025-07-24T03:40:56.6950992Z   issue_number: 5132
2025-07-24T03:40:56.6951690Z ##[endgroup]
2025-07-24T03:40:56.6952383Z Complete job name: get-label-type / runner-determinator
2025-07-24T03:40:56.7548072Z ##[group]Run cat <<EOF > runner_determinator.py
2025-07-24T03:40:56.7550409Z cat <<EOF > runner_determinator.py
2025-07-24T03:40:56.7551056Z # flake8: noqa: G004
2025-07-24T03:40:56.7551977Z 
2025-07-24T03:40:56.7552713Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2025-07-24T03:40:56.7553759Z #       must be kept in sync. You can do it easily by running the following command:
2025-07-24T03:40:56.7554750Z #           python .github/scripts/update_runner_determinator.py
2025-07-24T03:40:56.7555462Z 
2025-07-24T03:40:56.7555885Z """
2025-07-24T03:40:56.7556615Z This runner determinator is used to determine which set of runners to run a
2025-07-24T03:40:56.7557618Z GitHub job on. It uses the first comment of a GitHub issue (by default
2025-07-24T03:40:56.7558811Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2025-07-24T03:40:56.7559725Z of which runners should be used to run which job.
2025-07-24T03:40:56.7560382Z 
2025-07-24T03:40:56.7561113Z The configuration has two parts, the settings and a list of opted-in users,
2025-07-24T03:40:56.7562425Z separated by a line containing "---".  If the line is not present, the
2025-07-24T03:40:56.7563430Z settings are considered to be empty with only the second part, the user
2025-07-24T03:40:56.7564373Z list, defined.
2025-07-24T03:40:56.7564844Z 
2025-07-24T03:40:56.7565495Z The first part is a YAML block that defines the rollout settings. This can be
2025-07-24T03:40:56.7566624Z used to define any settings that are needed to determine which runners to use.
2025-07-24T03:40:56.7567599Z It's fields are defined by the RolloutSettings class below.
2025-07-24T03:40:56.7568546Z 
2025-07-24T03:40:56.7569295Z The second part is a list of users who are explicitly opted in to the LF fleet.
2025-07-24T03:40:56.7570292Z The user list is also a comma separated list of additional features or
2025-07-24T03:40:56.7571125Z experiments which the user could be opted in to.
2025-07-24T03:40:56.7572169Z 
2025-07-24T03:40:56.7572674Z The user list has the following rules:
2025-07-24T03:40:56.7573242Z 
2025-07-24T03:40:56.7573960Z - Users are GitHub usernames, which must start with the @ prefix
2025-07-24T03:40:56.7574894Z - Each user is also a comma-separated list of features/experiments to enable
2025-07-24T03:40:56.7575769Z - A "#" prefix opts the user out of all experiments
2025-07-24T03:40:56.7576481Z 
2025-07-24T03:40:56.7576895Z Example config:
2025-07-24T03:40:56.7577471Z     # A list of experiments that can be opted into.
2025-07-24T03:40:56.7578343Z     # This defines the behavior they'll induce when opted into.
2025-07-24T03:40:56.7579074Z     # Expected syntax is:
2025-07-24T03:40:56.7579824Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2025-07-24T03:40:56.7580956Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2025-07-24T03:40:56.7581937Z 
2025-07-24T03:40:56.7582346Z     experiments:
2025-07-24T03:40:56.7583058Z       lf:
2025-07-24T03:40:56.7583525Z         rollout_percent: 25
2025-07-24T03:40:56.7584109Z         all_branches: false
2025-07-24T03:40:56.7584675Z         default: true
2025-07-24T03:40:56.7585231Z     ---
2025-07-24T03:40:56.7585677Z 
2025-07-24T03:40:56.7586175Z     # Opt-ins:
2025-07-24T03:40:56.7586890Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2025-07-24T03:40:56.7588000Z     # and specifying experiments to enable in a comma-separated list.
2025-07-24T03:40:56.7588961Z     # To always opt out of an experiment, prefix it with a "-".
2025-07-24T03:40:56.7589735Z     # Experiments should be from the above list.
2025-07-24T03:40:56.7590362Z 
2025-07-24T03:40:56.7590869Z     @User1,-lf,split_build
2025-07-24T03:40:56.7591410Z     @User2,lf
2025-07-24T03:40:56.7592193Z     @User3,split_build
2025-07-24T03:40:56.7592756Z """
2025-07-24T03:40:56.7593233Z 
2025-07-24T03:40:56.7593707Z import json
2025-07-24T03:40:56.7594271Z import logging
2025-07-24T03:40:56.7594848Z import os
2025-07-24T03:40:56.7595313Z import random
2025-07-24T03:40:56.7595870Z import re
2025-07-24T03:40:56.7596320Z import sys
2025-07-24T03:40:56.7596887Z from argparse import ArgumentParser
2025-07-24T03:40:56.7597613Z from collections.abc import Iterable
2025-07-24T03:40:56.7598223Z from functools import cache
2025-07-24T03:40:56.7598826Z from logging import LogRecord
2025-07-24T03:40:56.7599457Z from typing import Any, NamedTuple
2025-07-24T03:40:56.7600148Z from urllib.request import Request, urlopen
2025-07-24T03:40:56.7600773Z 
2025-07-24T03:40:56.7601248Z import yaml
2025-07-24T03:40:56.7602078Z from github import Auth, Github
2025-07-24T03:40:56.7602673Z from github.Issue import Issue
2025-07-24T03:40:56.7603298Z 
2025-07-24T03:40:56.7603715Z 
2025-07-24T03:40:56.7604280Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2025-07-24T03:40:56.7605047Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2025-07-24T03:40:56.7606083Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2025-07-24T03:40:56.7607147Z 
2025-07-24T03:40:56.7607693Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2025-07-24T03:40:56.7608406Z GH_OUTPUT_KEY_AMI = "runner-ami"
2025-07-24T03:40:56.7609056Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2025-07-24T03:40:56.7609788Z OPT_OUT_LABEL = "no-runner-experiments"
2025-07-24T03:40:56.7610421Z 
2025-07-24T03:40:56.7610871Z SETTING_EXPERIMENTS = "experiments"
2025-07-24T03:40:56.7611728Z 
2025-07-24T03:40:56.7612186Z LF_FLEET_EXPERIMENT = "lf"
2025-07-24T03:40:56.7612808Z CANARY_FLEET_SUFFIX = ".c"
2025-07-24T03:40:56.7613538Z 
2025-07-24T03:40:56.7614043Z 
2025-07-24T03:40:56.7614487Z class Experiment(NamedTuple):
2025-07-24T03:40:56.7615122Z     rollout_perc: float = (
2025-07-24T03:40:56.7615929Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2025-07-24T03:40:56.7616705Z     )
2025-07-24T03:40:56.7617227Z     all_branches: bool = (
2025-07-24T03:40:56.7617996Z         False  # If True, the experiment is also enabled on the exception branches
2025-07-24T03:40:56.7618779Z     )
2025-07-24T03:40:56.7619281Z     default: bool = (
2025-07-24T03:40:56.7619979Z         True  # If True, the experiment is enabled by default for all queries
2025-07-24T03:40:56.7620754Z     )
2025-07-24T03:40:56.7621153Z 
2025-07-24T03:40:56.7621800Z     # Add more fields as needed
2025-07-24T03:40:56.7622350Z 
2025-07-24T03:40:56.7622759Z 
2025-07-24T03:40:56.7623249Z class Settings(NamedTuple):
2025-07-24T03:40:56.7623846Z     """
2025-07-24T03:40:56.7624422Z     Settings for the experiments that can be opted into.
2025-07-24T03:40:56.7625138Z     """
2025-07-24T03:40:56.7625590Z 
2025-07-24T03:40:56.7626048Z     experiments: dict[str, Experiment] = {}
2025-07-24T03:40:56.7626737Z 
2025-07-24T03:40:56.7627300Z 
2025-07-24T03:40:56.7627816Z class ColorFormatter(logging.Formatter):
2025-07-24T03:40:56.7628654Z     """Color codes the log messages based on the log level"""
2025-07-24T03:40:56.7629309Z 
2025-07-24T03:40:56.7629734Z     COLORS = {
2025-07-24T03:40:56.7630311Z         "WARNING": "\033[33m",  # Yellow
2025-07-24T03:40:56.7630975Z         "ERROR": "\033[31m",  # Red
2025-07-24T03:40:56.7631788Z         "CRITICAL": "\033[31m",  # Red
2025-07-24T03:40:56.7632503Z         "INFO": "\033[0m",  # Reset
2025-07-24T03:40:56.7633116Z         "DEBUG": "\033[0m",  # Reset
2025-07-24T03:40:56.7633655Z     }
2025-07-24T03:40:56.7634162Z 
2025-07-24T03:40:56.7634679Z     def format(self, record: LogRecord) -> str:
2025-07-24T03:40:56.7635531Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2025-07-24T03:40:56.7636459Z         record.msg = f"{log_color}{record.msg}\033[0m"
2025-07-24T03:40:56.7637153Z         return super().format(record)
2025-07-24T03:40:56.7637727Z 
2025-07-24T03:40:56.7638203Z 
2025-07-24T03:40:56.7638715Z handler = logging.StreamHandler()
2025-07-24T03:40:56.7639526Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2025-07-24T03:40:56.7640383Z 
2025-07-24T03:40:56.7640907Z log = logging.getLogger(os.path.basename(__file__))
2025-07-24T03:40:56.7641862Z log.addHandler(handler)
2025-07-24T03:40:56.7642539Z log.setLevel(logging.INFO)
2025-07-24T03:40:56.7643135Z 
2025-07-24T03:40:56.7643553Z 
2025-07-24T03:40:56.7644139Z def set_github_output(key: str, value: str) -> None:
2025-07-24T03:40:56.7644817Z     """
2025-07-24T03:40:56.7645420Z     Defines outputs of the github action that invokes this script
2025-07-24T03:40:56.7646381Z     """
2025-07-24T03:40:56.7646908Z     if not GITHUB_OUTPUT:
2025-07-24T03:40:56.7648057Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2025-07-24T03:40:56.7649316Z         log.warning(
2025-07-24T03:40:56.7650260Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2025-07-24T03:40:56.7651270Z         )
2025-07-24T03:40:56.7652287Z         print(f"::set-output name={key}::{value}")
2025-07-24T03:40:56.7652919Z         return
2025-07-24T03:40:56.7653401Z 
2025-07-24T03:40:56.7653927Z     with open(GITHUB_OUTPUT, "a") as f:
2025-07-24T03:40:56.7654607Z         log.info(f"Setting output: {key}='{value}'")
2025-07-24T03:40:56.7655265Z         f.write(f"{key}={value}\n")
2025-07-24T03:40:56.7655946Z 
2025-07-24T03:40:56.7656416Z 
2025-07-24T03:40:56.7656983Z def _str_comma_separated_to_set(value: str) -> frozenset[str]:
2025-07-24T03:40:56.7657788Z     return frozenset(
2025-07-24T03:40:56.7658495Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2025-07-24T03:40:56.7659273Z     )
2025-07-24T03:40:56.7659781Z 
2025-07-24T03:40:56.7660240Z 
2025-07-24T03:40:56.7660685Z def parse_args() -> Any:
2025-07-24T03:40:56.7661416Z     parser = ArgumentParser("Get dynamic rollout settings")
2025-07-24T03:40:56.7662541Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2025-07-24T03:40:56.7663377Z     parser.add_argument(
2025-07-24T03:40:56.7733863Z         "--github-issue-repo",
2025-07-24T03:40:56.7734803Z         type=str,
2025-07-24T03:40:56.7735570Z         required=False,
2025-07-24T03:40:56.7736776Z         default="pytorch/test-infra",
2025-07-24T03:40:56.7737763Z         help="GitHub repo to get the issue",
2025-07-24T03:40:56.7738739Z     )
2025-07-24T03:40:56.7739257Z     parser.add_argument(
2025-07-24T03:40:56.7739740Z         "--github-repo",
2025-07-24T03:40:56.7740209Z         type=str,
2025-07-24T03:40:56.7740638Z         required=True,
2025-07-24T03:40:56.7741144Z         help="GitHub repo where CI is running",
2025-07-24T03:40:56.7741910Z     )
2025-07-24T03:40:56.7742318Z     parser.add_argument(
2025-07-24T03:40:56.7742974Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2025-07-24T03:40:56.7743631Z     )
2025-07-24T03:40:56.7744023Z     parser.add_argument(
2025-07-24T03:40:56.7744686Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2025-07-24T03:40:56.7745384Z     )
2025-07-24T03:40:56.7745758Z     parser.add_argument(
2025-07-24T03:40:56.7746433Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2025-07-24T03:40:56.7747119Z     )
2025-07-24T03:40:56.7747496Z     parser.add_argument(
2025-07-24T03:40:56.7748192Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2025-07-24T03:40:56.7748927Z     )
2025-07-24T03:40:56.7749300Z     parser.add_argument(
2025-07-24T03:40:56.7749783Z         "--github-ref-type",
2025-07-24T03:40:56.7750257Z         type=str,
2025-07-24T03:40:56.7750683Z         required=True,
2025-07-24T03:40:56.7751220Z         help="Current GitHub ref type, branch or tag",
2025-07-24T03:40:56.7752083Z     )
2025-07-24T03:40:56.7752481Z     parser.add_argument(
2025-07-24T03:40:56.7753178Z         "--eligible-experiments",
2025-07-24T03:40:56.7753733Z         type=_str_comma_separated_to_set,
2025-07-24T03:40:56.7754253Z         required=False,
2025-07-24T03:40:56.7754711Z         default="",
2025-07-24T03:40:56.7755599Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2025-07-24T03:40:56.7756512Z     )
2025-07-24T03:40:56.7756903Z     parser.add_argument(
2025-07-24T03:40:56.7757401Z         "--opt-out-experiments",
2025-07-24T03:40:56.7757943Z         type=_str_comma_separated_to_set,
2025-07-24T03:40:56.7758477Z         required=False,
2025-07-24T03:40:56.7758937Z         default="",
2025-07-24T03:40:56.7759363Z         help=(
2025-07-24T03:40:56.7760068Z             "comma separated list of experiments to opt-out of. If unset, no opt-outs will occur. "
2025-07-24T03:40:56.7761215Z             "If the same experiment is listed both here and in '--eligible-experiments' opt-out will take priority."
2025-07-24T03:40:56.7762386Z         ),
2025-07-24T03:40:56.7762763Z     )
2025-07-24T03:40:56.7763146Z     parser.add_argument(
2025-07-24T03:40:56.7763624Z         "--pr-number",
2025-07-24T03:40:56.7764073Z         type=str,
2025-07-24T03:40:56.7764508Z         required=False,
2025-07-24T03:40:56.7764958Z         default="",
2025-07-24T03:40:56.7765477Z         help="the optional PR number where this is run",
2025-07-24T03:40:56.7766070Z     )
2025-07-24T03:40:56.7766421Z 
2025-07-24T03:40:56.7766794Z     return parser.parse_args()
2025-07-24T03:40:56.7767265Z 
2025-07-24T03:40:56.7767606Z 
2025-07-24T03:40:56.7768210Z def get_gh_client(github_token: str) -> Github:  # type: ignore[no-any-unimported]
2025-07-24T03:40:56.7769137Z     auth = Auth.Token(github_token)
2025-07-24T03:40:56.7769677Z     return Github(auth=auth)
2025-07-24T03:40:56.7770145Z 
2025-07-24T03:40:56.7770488Z 
2025-07-24T03:40:56.7771135Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:  # type: ignore[no-any-unimported]
2025-07-24T03:40:56.7772124Z     repo = gh.get_repo(repo)
2025-07-24T03:40:56.7772662Z     return repo.get_issue(number=issue_num)
2025-07-24T03:40:56.7773182Z 
2025-07-24T03:40:56.7773522Z 
2025-07-24T03:40:56.7773881Z def get_potential_pr_author(
2025-07-24T03:40:56.7774569Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2025-07-24T03:40:56.7775236Z ) -> str:
2025-07-24T03:40:56.7775790Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2025-07-24T03:40:56.7776608Z     # Fetch the actual username from the original PR. The PR number is
2025-07-24T03:40:56.7777380Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2025-07-24T03:40:56.7777951Z 
2025-07-24T03:40:56.7778331Z     gh = get_gh_client(github_token)
2025-07-24T03:40:56.7778821Z 
2025-07-24T03:40:56.7779284Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2025-07-24T03:40:56.7779921Z         split_tag = ref_name.split("/")
2025-07-24T03:40:56.7780426Z         if (
2025-07-24T03:40:56.7780847Z             len(split_tag) == 3
2025-07-24T03:40:56.7781356Z             and split_tag[0] == "ciflow"
2025-07-24T03:40:56.7782159Z             and split_tag[2].isnumeric()
2025-07-24T03:40:56.7782665Z         ):
2025-07-24T03:40:56.7783085Z             pr_number = split_tag[2]
2025-07-24T03:40:56.7783585Z             try:
2025-07-24T03:40:56.7784049Z                 repository = gh.get_repo(repo)
2025-07-24T03:40:56.7784834Z                 pull = repository.get_pull(number=int(pr_number))
2025-07-24T03:40:56.7785446Z             except Exception as e:
2025-07-24T03:40:56.7785982Z                 raise Exception(  # noqa: TRY002
2025-07-24T03:40:56.7786671Z                     f"issue with pull request {pr_number} from repo {repository}"
2025-07-24T03:40:56.7787314Z                 ) from e
2025-07-24T03:40:56.7787894Z             return pull.user.login  # type: ignore[no-any-return]
2025-07-24T03:40:56.7788601Z     # In all other cases, return the original input username
2025-07-24T03:40:56.7789203Z     return username
2025-07-24T03:40:56.7789630Z 
2025-07-24T03:40:56.7789962Z 
2025-07-24T03:40:56.7790387Z def is_exception_branch(branch: str) -> bool:
2025-07-24T03:40:56.7790927Z     """
2025-07-24T03:40:56.7792276Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2025-07-24T03:40:56.7793093Z     """
2025-07-24T03:40:56.7793675Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2025-07-24T03:40:56.7794339Z 
2025-07-24T03:40:56.7794678Z 
2025-07-24T03:40:56.7795066Z def load_yaml(yaml_text: str) -> Any:
2025-07-24T03:40:56.7795565Z     try:
2025-07-24T03:40:56.7795972Z         data = yaml.safe_load(yaml_text)
2025-07-24T03:40:56.7796483Z         return data
2025-07-24T03:40:56.7796935Z     except yaml.YAMLError:
2025-07-24T03:40:56.7797452Z         log.exception("Error loading YAML")
2025-07-24T03:40:56.7797972Z         raise
2025-07-24T03:40:56.7798359Z 
2025-07-24T03:40:56.7798697Z 
2025-07-24T03:40:56.7799312Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> tuple[str, str]:
2025-07-24T03:40:56.7800055Z     """
2025-07-24T03:40:56.7800872Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2025-07-24T03:40:56.7801742Z 
2025-07-24T03:40:56.7802285Z     If the issue body contains "---" then the text above that is the settings
2025-07-24T03:40:56.7803046Z     and the text below is the list of opted in users.
2025-07-24T03:40:56.7803606Z 
2025-07-24T03:40:56.7804173Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2025-07-24T03:40:56.7804869Z     """
2025-07-24T03:40:56.7805335Z     rollout_state_parts = rollout_state.split("---")
2025-07-24T03:40:56.7805931Z     if len(rollout_state_parts) >= 2:
2025-07-24T03:40:56.7806559Z         return rollout_state_parts[0], rollout_state_parts[1]
2025-07-24T03:40:56.7807144Z     else:
2025-07-24T03:40:56.7807546Z         return "", rollout_state
2025-07-24T03:40:56.7808026Z 
2025-07-24T03:40:56.7808346Z 
2025-07-24T03:40:56.7808754Z class UserOptins(dict[str, list[str]]):
2025-07-24T03:40:56.7809263Z     """
2025-07-24T03:40:56.7809809Z     Dictionary of users with a list of features they have opted into
2025-07-24T03:40:56.7810447Z     """
2025-07-24T03:40:56.7810803Z 
2025-07-24T03:40:56.7811126Z 
2025-07-24T03:40:56.7811925Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2025-07-24T03:40:56.7812600Z     """
2025-07-24T03:40:56.7813317Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2025-07-24T03:40:56.7814138Z 
2025-07-24T03:40:56.7814925Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2025-07-24T03:40:56.7815924Z         - Example line: "@User1,lf,split_build"
2025-07-24T03:40:56.7816775Z         - A "#" prefix indicates the user is opted out of all experiments
2025-07-24T03:40:56.7817405Z 
2025-07-24T03:40:56.7817738Z 
2025-07-24T03:40:56.7818067Z     """
2025-07-24T03:40:56.7818446Z     optins = UserOptins()
2025-07-24T03:40:56.7818962Z     for user in user_optin_text.split("\n"):
2025-07-24T03:40:56.7819533Z         user = user.strip("\r\n\t -")
2025-07-24T03:40:56.7820087Z         if not user or not user.startswith("@"):
2025-07-24T03:40:56.7820649Z             # Not a valid user. Skip
2025-07-24T03:40:56.7821140Z             continue
2025-07-24T03:40:56.7821672Z 
2025-07-24T03:40:56.7822016Z         if user:
2025-07-24T03:40:56.7822482Z             usr_name = user.split(",")[0].strip("@")
2025-07-24T03:40:56.7823174Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2025-07-24T03:40:56.7823797Z 
2025-07-24T03:40:56.7824143Z     return optins
2025-07-24T03:40:56.7824550Z 
2025-07-24T03:40:56.7824871Z 
2025-07-24T03:40:56.7825371Z def is_valid_experiment_name(experiment_name: str) -> bool:
2025-07-24T03:40:56.7825971Z     """
2025-07-24T03:40:56.7826403Z     Check if the experiment name is valid.
2025-07-24T03:40:56.7826925Z     A valid name:
2025-07-24T03:40:56.7827604Z         - Contains only alphanumeric characters and the special characters "_" & "-"
2025-07-24T03:40:56.7828536Z         - The special characters "_" & "-" shouldn't be the first or last characters
2025-07-24T03:40:56.7829248Z         - Cannot contain spaces
2025-07-24T03:40:56.7829718Z     """
2025-07-24T03:40:56.7830072Z 
2025-07-24T03:40:56.7830524Z     valid_char_regex = r"^[a-zA-Z0-9]([\w-]*[a-zA-Z0-9])?$"
2025-07-24T03:40:56.7831242Z     valid = bool(re.match(valid_char_regex, experiment_name))
2025-07-24T03:40:56.7832184Z 
2025-07-24T03:40:56.7832528Z     if valid:
2025-07-24T03:40:56.7832933Z         return True
2025-07-24T03:40:56.7833345Z 
2025-07-24T03:40:56.7833680Z     log.error(
2025-07-24T03:40:56.7835097Z         f"Invalid experiment name: {experiment_name}. Experiment names should only contain alphanumeric characters, '_', and '-'. They cannot contain spaces, and the special characters '_' and '-' cannot be the first or last characters."
2025-07-24T03:40:56.7836601Z     )
2025-07-24T03:40:56.7836965Z     return False
2025-07-24T03:40:56.7837360Z 
2025-07-24T03:40:56.7837682Z 
2025-07-24T03:40:56.7838190Z def parse_settings_from_text(settings_text: str) -> Settings:
2025-07-24T03:40:56.7838817Z     """
2025-07-24T03:40:56.7839423Z     Parse the experiments from the issue body into a list of ExperimentSettings
2025-07-24T03:40:56.7840134Z     """
2025-07-24T03:40:56.7840501Z     try:
2025-07-24T03:40:56.7840874Z         if settings_text:
2025-07-24T03:40:56.7841732Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2025-07-24T03:40:56.7842523Z             # for easy reading
2025-07-24T03:40:56.7843335Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2025-07-24T03:40:56.7844216Z             #       the backtick character in shell commands.
2025-07-24T03:40:56.7844822Z             backtick = chr(96)  # backtick character
2025-07-24T03:40:56.7845501Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2025-07-24T03:40:56.7846176Z             settings = load_yaml(settings_text)
2025-07-24T03:40:56.7846688Z 
2025-07-24T03:40:56.7847417Z             # For now we just load experiments. We can expand this if/when we add more settings
2025-07-24T03:40:56.7848158Z             experiments = {}
2025-07-24T03:40:56.7848620Z 
2025-07-24T03:40:56.7849182Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2025-07-24T03:40:56.7849955Z                 if not is_valid_experiment_name(exp_name):
2025-07-24T03:40:56.7851053Z                     # Exclude invalid experiments from the list. We log an error, but don't raise an exception so that other experiments can still be processed.
2025-07-24T03:40:56.7852351Z                     continue
2025-07-24T03:40:56.7852820Z 
2025-07-24T03:40:56.7853185Z                 valid_settings = {}
2025-07-24T03:40:56.7853728Z                 for setting in exp_settings:
2025-07-24T03:40:56.7854315Z                     if setting not in Experiment._fields:
2025-07-24T03:40:56.7854880Z                         log.warning(
2025-07-24T03:40:56.7855605Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2025-07-24T03:40:56.7856303Z                         )
2025-07-24T03:40:56.7856751Z                     else:
2025-07-24T03:40:56.7857292Z                         valid_settings[setting] = exp_settings[setting]
2025-07-24T03:40:56.7857849Z 
2025-07-24T03:40:56.7858318Z                 experiments[exp_name] = Experiment(**valid_settings)
2025-07-24T03:40:56.7858950Z             return Settings(experiments)
2025-07-24T03:40:56.7859449Z 
2025-07-24T03:40:56.7859801Z     except Exception:
2025-07-24T03:40:56.7860316Z         log.exception("Failed to parse settings")
2025-07-24T03:40:56.7860837Z 
2025-07-24T03:40:56.7861190Z     return Settings()
2025-07-24T03:40:56.7861729Z 
2025-07-24T03:40:56.7862046Z 
2025-07-24T03:40:56.7862630Z def parse_settings(rollout_state: str) -> Settings:
2025-07-24T03:40:56.7863210Z     """
2025-07-24T03:40:56.7863666Z     Parse settings, if any, from the rollout state.
2025-07-24T03:40:56.7864210Z 
2025-07-24T03:40:56.7864745Z     If the issue body contains "---" then the text above that is the settings
2025-07-24T03:40:56.7865514Z     and the text below is the list of opted in users.
2025-07-24T03:40:56.7866088Z 
2025-07-24T03:40:56.7866691Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2025-07-24T03:40:56.7867454Z     """
2025-07-24T03:40:56.7868083Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2025-07-24T03:40:56.7868849Z     return parse_settings_from_text(settings_text)
2025-07-24T03:40:56.7869388Z 
2025-07-24T03:40:56.7869716Z 
2025-07-24T03:40:56.7870159Z def parse_users(rollout_state: str) -> UserOptins:
2025-07-24T03:40:56.7870722Z     """
2025-07-24T03:40:56.7871129Z     Parse users from the rollout state.
2025-07-24T03:40:56.7871831Z 
2025-07-24T03:40:56.7872163Z     """
2025-07-24T03:40:56.7872720Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2025-07-24T03:40:56.7873474Z     return parse_user_opt_in_from_text(users_text)
2025-07-24T03:40:56.7874012Z 
2025-07-24T03:40:56.7874343Z 
2025-07-24T03:40:56.7874951Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-07-24T03:40:56.7875695Z     """
2025-07-24T03:40:56.7876147Z     Check if a user is opted into an experiment
2025-07-24T03:40:56.7876686Z     """
2025-07-24T03:40:56.7877161Z     return experiment_name in user_optins.get(user, [])
2025-07-24T03:40:56.7877875Z 
2025-07-24T03:40:56.7878197Z 
2025-07-24T03:40:56.7878821Z def is_user_opted_out(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-07-24T03:40:56.7879570Z     """
2025-07-24T03:40:56.7880048Z     Check if a user explicitly opted out of an experiment
2025-07-24T03:40:56.7880628Z     """
2025-07-24T03:40:56.7881150Z     # if the experiment is prefixed with a "-", then it's an opt-out
2025-07-24T03:40:56.7881990Z     experiment_optout = "-" + experiment_name
2025-07-24T03:40:56.7882658Z     if experiment_optout not in user_optins.get(user, []):
2025-07-24T03:40:56.7883250Z         return False
2025-07-24T03:40:56.7883665Z 
2025-07-24T03:40:56.7884116Z     if is_user_opted_in(user, user_optins, experiment_name):
2025-07-24T03:40:56.7884718Z         log.warning(
2025-07-24T03:40:56.7885537Z             f"User {user} is opted into experiment {experiment_name}, but also opted out of it. Defaulting to opting out"
2025-07-24T03:40:56.7886399Z         )
2025-07-24T03:40:56.7886761Z 
2025-07-24T03:40:56.7887101Z     return True
2025-07-24T03:40:56.7887511Z 
2025-07-24T03:40:56.7887831Z 
2025-07-24T03:40:56.7888184Z def get_runner_prefix(
2025-07-24T03:40:56.7888636Z     rollout_state: str,
2025-07-24T03:40:56.7889133Z     workflow_requestors: Iterable[str],
2025-07-24T03:40:56.7889645Z     branch: str,
2025-07-24T03:40:56.7890179Z     eligible_experiments: frozenset[str] = frozenset(),
2025-07-24T03:40:56.7890854Z     opt_out_experiments: frozenset[str] = frozenset(),
2025-07-24T03:40:56.7891443Z     is_canary: bool = False,
2025-07-24T03:40:56.7892215Z ) -> str:
2025-07-24T03:40:56.7892665Z     settings = parse_settings(rollout_state)
2025-07-24T03:40:56.7893273Z     user_optins = parse_users(rollout_state)
2025-07-24T03:40:56.7893786Z 
2025-07-24T03:40:56.7894271Z     fleet_prefix = ""
2025-07-24T03:40:56.7894726Z     prefixes = []
2025-07-24T03:40:56.7895407Z     for experiment_name, experiment_settings in settings.experiments.items():
2025-07-24T03:40:56.7896357Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2025-07-24T03:40:56.7897061Z             log.info(
2025-07-24T03:40:56.7897765Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2025-07-24T03:40:56.7898511Z             )
2025-07-24T03:40:56.7898911Z             continue
2025-07-24T03:40:56.7899326Z 
2025-07-24T03:40:56.7899692Z         if opt_out_experiments:
2025-07-24T03:40:56.7900253Z             if experiment_name in opt_out_experiments:
2025-07-24T03:40:56.7900895Z                 opt_out_exp_list = ", ".join(opt_out_experiments)
2025-07-24T03:40:56.7901664Z                 log.info(
2025-07-24T03:40:56.7902674Z                     f"Skipping experiment '{experiment_name}', as this workflow has opted-out (opted out experiments are: {opt_out_exp_list})"
2025-07-24T03:40:56.7903643Z                 )
2025-07-24T03:40:56.7904069Z                 continue
2025-07-24T03:40:56.7904494Z 
2025-07-24T03:40:56.7904873Z         if eligible_experiments:
2025-07-24T03:40:56.7905452Z             if experiment_name not in eligible_experiments:
2025-07-24T03:40:56.7906100Z                 exp_list = ", ".join(eligible_experiments)
2025-07-24T03:40:56.7906645Z                 log.info(
2025-07-24T03:40:56.7907447Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2025-07-24T03:40:56.7908275Z                 )
2025-07-24T03:40:56.7908827Z                 continue
2025-07-24T03:40:56.7909345Z         elif not experiment_settings.default:
2025-07-24T03:40:56.7909868Z             log.info(
2025-07-24T03:40:56.7910549Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2025-07-24T03:40:56.7911270Z             )
2025-07-24T03:40:56.7911877Z             continue
2025-07-24T03:40:56.7912300Z 
2025-07-24T03:40:56.7912761Z         # Is any workflow_requestor opted out to this experiment?
2025-07-24T03:40:56.7913385Z         opted_out_users = [
2025-07-24T03:40:56.7913852Z             requestor
2025-07-24T03:40:56.7914336Z             for requestor in workflow_requestors
2025-07-24T03:40:56.7915004Z             if is_user_opted_out(requestor, user_optins, experiment_name)
2025-07-24T03:40:56.7915625Z         ]
2025-07-24T03:40:56.7915997Z 
2025-07-24T03:40:56.7916347Z         if opted_out_users:
2025-07-24T03:40:56.7916822Z             log.info(
2025-07-24T03:40:56.7917461Z                 f"{', '.join(opted_out_users)} have opted out of experiment {experiment_name}."
2025-07-24T03:40:56.7918152Z             )
2025-07-24T03:40:56.7918551Z             continue
2025-07-24T03:40:56.7918966Z 
2025-07-24T03:40:56.7919420Z         # Is any workflow_requestor opted in to this experiment?
2025-07-24T03:40:56.7920032Z         opted_in_users = [
2025-07-24T03:40:56.7920498Z             requestor
2025-07-24T03:40:56.7920979Z             for requestor in workflow_requestors
2025-07-24T03:40:56.7921766Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2025-07-24T03:40:56.7922379Z         ]
2025-07-24T03:40:56.7922745Z 
2025-07-24T03:40:56.7923085Z         enabled = False
2025-07-24T03:40:56.7923550Z         if opted_in_users:
2025-07-24T03:40:56.7924140Z             log.info(
2025-07-24T03:40:56.7924774Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2025-07-24T03:40:56.7925444Z             )
2025-07-24T03:40:56.7925846Z             enabled = True
2025-07-24T03:40:56.7926299Z 
2025-07-24T03:40:56.7926698Z         elif experiment_settings.rollout_perc:
2025-07-24T03:40:56.7927529Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2025-07-24T03:40:56.7928466Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2025-07-24T03:40:56.7929095Z                 log.info(
2025-07-24T03:40:56.7929971Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2025-07-24T03:40:56.7930867Z                 )
2025-07-24T03:40:56.7931299Z                 enabled = True
2025-07-24T03:40:56.7932023Z 
2025-07-24T03:40:56.7932375Z         if enabled:
2025-07-24T03:40:56.7932831Z             label = experiment_name
2025-07-24T03:40:56.7933397Z             if experiment_name == LF_FLEET_EXPERIMENT:
2025-07-24T03:40:56.7934229Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2025-07-24T03:40:56.7935092Z                 #  - If it's enabled, then we always list it's prefix first
2025-07-24T03:40:56.7935856Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2025-07-24T03:40:56.7936517Z                 if is_canary:
2025-07-24T03:40:56.7937026Z                     label += CANARY_FLEET_SUFFIX
2025-07-24T03:40:56.7937580Z                 fleet_prefix = label
2025-07-24T03:40:56.7938208Z             else:
2025-07-24T03:40:56.7938671Z                 prefixes.append(label)
2025-07-24T03:40:56.7939163Z 
2025-07-24T03:40:56.7939518Z     if len(prefixes) > 1:
2025-07-24T03:40:56.7939984Z         log.error(
2025-07-24T03:40:56.7941018Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2025-07-24T03:40:56.7942213Z         )
2025-07-24T03:40:56.7942605Z         prefixes = prefixes[:1]
2025-07-24T03:40:56.7943077Z 
2025-07-24T03:40:56.7943433Z     # Fleet always comes first
2025-07-24T03:40:56.7943914Z     if fleet_prefix:
2025-07-24T03:40:56.7944389Z         prefixes.insert(0, fleet_prefix)
2025-07-24T03:40:56.7944888Z 
2025-07-24T03:40:56.7945332Z     return ".".join(prefixes) + "." if prefixes else ""
2025-07-24T03:40:56.7945894Z 
2025-07-24T03:40:56.7946225Z 
2025-07-24T03:40:56.7946851Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2025-07-24T03:40:56.7947611Z     """
2025-07-24T03:40:56.7948202Z     Gets the first comment of the issue, which contains the desired rollout state.
2025-07-24T03:40:56.7948901Z 
2025-07-24T03:40:56.7949473Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2025-07-24T03:40:56.7950166Z     """
2025-07-24T03:40:56.7950570Z     gh = get_gh_client(github_token)
2025-07-24T03:40:56.7951122Z     issue = get_issue(gh, repo, issue_num)
2025-07-24T03:40:56.7952022Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2025-07-24T03:40:56.7952610Z 
2025-07-24T03:40:56.7952944Z 
2025-07-24T03:40:56.7953537Z def download_json(url: str, headers: dict[str, str], num_retries: int = 3) -> Any:
2025-07-24T03:40:56.7954412Z     for _ in range(num_retries):
2025-07-24T03:40:56.7954906Z         try:
2025-07-24T03:40:56.7955354Z             req = Request(url=url, headers=headers)
2025-07-24T03:40:56.7956021Z             content = urlopen(req, timeout=5).read().decode("utf-8")
2025-07-24T03:40:56.7956660Z             return json.loads(content)
2025-07-24T03:40:56.7957189Z         except Exception as e:
2025-07-24T03:40:56.7957762Z             log.warning(f"Could not download {url}: {e}")
2025-07-24T03:40:56.7958307Z 
2025-07-24T03:40:56.7958876Z     log.warning(f"All {num_retries} retries exhausted, downloading {url} failed")
2025-07-24T03:40:56.7959564Z     return {}
2025-07-24T03:40:56.7960049Z 
2025-07-24T03:40:56.7960376Z 
2025-07-24T03:40:56.7960711Z @cache
2025-07-24T03:40:56.7961379Z def get_pr_info(github_repo: str, github_token: str, pr_number: int) -> dict[str, Any]:
2025-07-24T03:40:56.7962295Z     """
2025-07-24T03:40:56.7962758Z     Dynamically get PR information
2025-07-24T03:40:56.7963291Z     """
2025-07-24T03:40:56.7963865Z     github_api = f"https://api.github.com/repos/{github_repo}"
2025-07-24T03:40:56.7964533Z     headers = {
2025-07-24T03:40:56.7965071Z         "Accept": "application/vnd.github.v3+json",
2025-07-24T03:40:56.7965691Z         "Authorization": f"token {github_token}",
2025-07-24T03:40:56.7966238Z     }
2025-07-24T03:40:56.7966689Z     json_response: dict[str, Any] = download_json(
2025-07-24T03:40:56.7967290Z         url=f"{github_api}/issues/{pr_number}",
2025-07-24T03:40:56.7967829Z         headers=headers,
2025-07-24T03:40:56.7968262Z     )
2025-07-24T03:40:56.7968613Z 
2025-07-24T03:40:56.7968959Z     if not json_response:
2025-07-24T03:40:56.7969698Z         log.warning(f"Failed to get the labels for #{pr_number}")
2025-07-24T03:40:56.7970322Z         return {}
2025-07-24T03:40:56.7970725Z 
2025-07-24T03:40:56.7971076Z     return json_response
2025-07-24T03:40:56.7971719Z 
2025-07-24T03:40:56.7972121Z 
2025-07-24T03:40:56.7972702Z def get_labels(github_repo: str, github_token: str, pr_number: int) -> set[str]:
2025-07-24T03:40:56.7973414Z     """
2025-07-24T03:40:56.7973952Z     Dynamically get the latest list of labels from the pull request
2025-07-24T03:40:56.7974599Z     """
2025-07-24T03:40:56.7975092Z     pr_info = get_pr_info(github_repo, github_token, pr_number)
2025-07-24T03:40:56.7975695Z     return {
2025-07-24T03:40:56.7976292Z         label.get("name") for label in pr_info.get("labels", []) if label.get("name")
2025-07-24T03:40:56.7976965Z     }
2025-07-24T03:40:56.7977327Z 
2025-07-24T03:40:56.7977650Z 
2025-07-24T03:40:56.7978001Z def main() -> None:
2025-07-24T03:40:56.7978442Z     args = parse_args()
2025-07-24T03:40:56.7978882Z 
2025-07-24T03:40:56.7979297Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2025-07-24T03:40:56.7979825Z 
2025-07-24T03:40:56.7980191Z     # Check if the PR is opt-out
2025-07-24T03:40:56.7980682Z     if args.pr_number:
2025-07-24T03:40:56.7981367Z         labels = get_labels(args.github_repo, args.github_token, int(args.pr_number))
2025-07-24T03:40:56.7982333Z         if OPT_OUT_LABEL in labels:
2025-07-24T03:40:56.7982835Z             log.info(
2025-07-24T03:40:56.7983546Z                 f"Opt-out runner determinator because #{args.pr_number} has {OPT_OUT_LABEL} label"
2025-07-24T03:40:56.7984283Z             )
2025-07-24T03:40:56.7984863Z             set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-07-24T03:40:56.7985528Z             sys.exit()
2025-07-24T03:40:56.7986092Z 
2025-07-24T03:40:56.7986429Z     try:
2025-07-24T03:40:56.7986886Z         rollout_state = get_rollout_state_from_issue(
2025-07-24T03:40:56.7987606Z             args.github_token, args.github_issue_repo, args.github_issue
2025-07-24T03:40:56.7988228Z         )
2025-07-24T03:40:56.7988595Z 
2025-07-24T03:40:56.7988989Z         username = get_potential_pr_author(
2025-07-24T03:40:56.7989530Z             args.github_token,
2025-07-24T03:40:56.7990016Z             args.github_repo,
2025-07-24T03:40:56.7990510Z             args.github_actor,
2025-07-24T03:40:56.7991006Z             args.github_ref_type,
2025-07-24T03:40:56.7991707Z             args.github_branch,
2025-07-24T03:40:56.7992194Z         )
2025-07-24T03:40:56.7992560Z 
2025-07-24T03:40:56.7993045Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2025-07-24T03:40:56.7993637Z 
2025-07-24T03:40:56.7994042Z         runner_label_prefix = get_runner_prefix(
2025-07-24T03:40:56.7994582Z             rollout_state,
2025-07-24T03:40:56.7995095Z             (args.github_issue_owner, username),
2025-07-24T03:40:56.7995634Z             args.github_branch,
2025-07-24T03:40:56.7996145Z             args.eligible_experiments,
2025-07-24T03:40:56.7996685Z             args.opt_out_experiments,
2025-07-24T03:40:56.7997179Z             is_canary,
2025-07-24T03:40:56.7997605Z         )
2025-07-24T03:40:56.7997961Z 
2025-07-24T03:40:56.7998325Z     except Exception as e:
2025-07-24T03:40:56.7998786Z         log.error(
2025-07-24T03:40:56.7999484Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2025-07-24T03:40:56.8000356Z         )
2025-07-24T03:40:56.8000727Z 
2025-07-24T03:40:56.8001245Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-07-24T03:40:56.8002135Z 
2025-07-24T03:40:56.8002468Z 
2025-07-24T03:40:56.8002814Z if __name__ == "__main__":
2025-07-24T03:40:56.8003268Z     main()
2025-07-24T03:40:56.8003638Z 
2025-07-24T03:40:56.8003973Z EOF
2025-07-24T03:40:56.8004316Z 
2025-07-24T03:40:56.8004672Z cat runner_determinator.py
2025-07-24T03:40:56.8188479Z shell: /usr/bin/bash -e {0}
2025-07-24T03:40:56.8189357Z env:
2025-07-24T03:40:56.8190028Z   GITHUB_TOKEN: ***
2025-07-24T03:40:56.8190443Z   ISSUE_NUMBER: 5132
2025-07-24T03:40:56.8190876Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2025-07-24T03:40:56.8191344Z   ISSUE_OWNER: 
2025-07-24T03:40:56.8192002Z   CHECK_EXPERIMENTS: 
2025-07-24T03:40:56.8192411Z   OPT_OUT_EXPERIMENTS: 
2025-07-24T03:40:56.8192816Z   PR_NUMBER: 
2025-07-24T03:40:56.8193166Z ##[endgroup]
2025-07-24T03:40:56.8426597Z # flake8: noqa: G004
2025-07-24T03:40:56.8426933Z 
2025-07-24T03:40:56.8427360Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2025-07-24T03:40:56.8428297Z #       must be kept in sync. You can do it easily by running the following command:
2025-07-24T03:40:56.8429102Z #           python .github/scripts/update_runner_determinator.py
2025-07-24T03:40:56.8429539Z 
2025-07-24T03:40:56.8429695Z """
2025-07-24T03:40:56.8430260Z This runner determinator is used to determine which set of runners to run a
2025-07-24T03:40:56.8431110Z GitHub job on. It uses the first comment of a GitHub issue (by default
2025-07-24T03:40:56.8432259Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2025-07-24T03:40:56.8433049Z of which runners should be used to run which job.
2025-07-24T03:40:56.8433458Z 
2025-07-24T03:40:56.8433829Z The configuration has two parts, the settings and a list of opted-in users,
2025-07-24T03:40:56.8434904Z separated by a line containing "---".  If the line is not present, the
2025-07-24T03:40:56.8435776Z settings are considered to be empty with only the second part, the user
2025-07-24T03:40:56.8436449Z list, defined.
2025-07-24T03:40:56.8436679Z 
2025-07-24T03:40:56.8437032Z The first part is a YAML block that defines the rollout settings. This can be
2025-07-24T03:40:56.8437928Z used to define any settings that are needed to determine which runners to use.
2025-07-24T03:40:56.8438732Z It's fields are defined by the RolloutSettings class below.
2025-07-24T03:40:56.8439181Z 
2025-07-24T03:40:56.8439538Z The second part is a list of users who are explicitly opted in to the LF fleet.
2025-07-24T03:40:56.8440390Z The user list is also a comma separated list of additional features or
2025-07-24T03:40:56.8441099Z experiments which the user could be opted in to.
2025-07-24T03:40:56.8441680Z 
2025-07-24T03:40:56.8441881Z The user list has the following rules:
2025-07-24T03:40:56.8442238Z 
2025-07-24T03:40:56.8442549Z - Users are GitHub usernames, which must start with the @ prefix
2025-07-24T03:40:56.8443390Z - Each user is also a comma-separated list of features/experiments to enable
2025-07-24T03:40:56.8444143Z - A "#" prefix opts the user out of all experiments
2025-07-24T03:40:56.8444544Z 
2025-07-24T03:40:56.8444704Z Example config:
2025-07-24T03:40:56.8445158Z     # A list of experiments that can be opted into.
2025-07-24T03:40:56.8445812Z     # This defines the behavior they'll induce when opted into.
2025-07-24T03:40:56.8446425Z     # Expected syntax is:
2025-07-24T03:40:56.8447037Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2025-07-24T03:40:56.8448029Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2025-07-24T03:40:56.8448628Z 
2025-07-24T03:40:56.8448797Z     experiments:
2025-07-24T03:40:56.8449183Z       lf:
2025-07-24T03:40:56.8449853Z         rollout_percent: 25
2025-07-24T03:40:56.8450465Z         all_branches: false
2025-07-24T03:40:56.8450913Z         default: true
2025-07-24T03:40:56.8451315Z     ---
2025-07-24T03:40:56.8451706Z 
2025-07-24T03:40:56.8451947Z     # Opt-ins:
2025-07-24T03:40:56.8452519Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2025-07-24T03:40:56.8453364Z     # and specifying experiments to enable in a comma-separated list.
2025-07-24T03:40:56.8454122Z     # To always opt out of an experiment, prefix it with a "-".
2025-07-24T03:40:56.8454761Z     # Experiments should be from the above list.
2025-07-24T03:40:56.8455137Z 
2025-07-24T03:40:56.8455319Z     @User1,-lf,split_build
2025-07-24T03:40:56.8455740Z     @User2,lf
2025-07-24T03:40:56.8456115Z     @User3,split_build
2025-07-24T03:40:56.8456511Z """
2025-07-24T03:40:56.8456711Z 
2025-07-24T03:40:56.8456864Z import json
2025-07-24T03:40:56.8457223Z import logging
2025-07-24T03:40:56.8457601Z import os
2025-07-24T03:40:56.8457957Z import random
2025-07-24T03:40:56.8458338Z import re
2025-07-24T03:40:56.8458715Z import sys
2025-07-24T03:40:56.8459102Z from argparse import ArgumentParser
2025-07-24T03:40:56.8459617Z from collections.abc import Iterable
2025-07-24T03:40:56.8460132Z from functools import cache
2025-07-24T03:40:56.8460596Z from logging import LogRecord
2025-07-24T03:40:56.8461088Z from typing import Any, NamedTuple
2025-07-24T03:40:56.8461982Z from urllib.request import Request, urlopen
2025-07-24T03:40:56.8462461Z 
2025-07-24T03:40:56.8462622Z import yaml
2025-07-24T03:40:56.8463009Z from github import Auth, Github
2025-07-24T03:40:56.8463499Z from github.Issue import Issue
2025-07-24T03:40:56.8463797Z 
2025-07-24T03:40:56.8463803Z 
2025-07-24T03:40:56.8464010Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2025-07-24T03:40:56.8464690Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2025-07-24T03:40:56.8465536Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2025-07-24T03:40:56.8466095Z 
2025-07-24T03:40:56.8466315Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2025-07-24T03:40:56.8467014Z GH_OUTPUT_KEY_AMI = "runner-ami"
2025-07-24T03:40:56.8467528Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2025-07-24T03:40:56.8468067Z OPT_OUT_LABEL = "no-runner-experiments"
2025-07-24T03:40:56.8468420Z 
2025-07-24T03:40:56.8468602Z SETTING_EXPERIMENTS = "experiments"
2025-07-24T03:40:56.8468930Z 
2025-07-24T03:40:56.8469110Z LF_FLEET_EXPERIMENT = "lf"
2025-07-24T03:40:56.8469579Z CANARY_FLEET_SUFFIX = ".c"
2025-07-24T03:40:56.8469865Z 
2025-07-24T03:40:56.8469871Z 
2025-07-24T03:40:56.8470050Z class Experiment(NamedTuple):
2025-07-24T03:40:56.8470516Z     rollout_perc: float = (
2025-07-24T03:40:56.8471147Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2025-07-24T03:40:56.8472005Z     )
2025-07-24T03:40:56.8472360Z     all_branches: bool = (
2025-07-24T03:40:56.8472965Z         False  # If True, the experiment is also enabled on the exception branches
2025-07-24T03:40:56.8473625Z     )
2025-07-24T03:40:56.8473975Z     default: bool = (
2025-07-24T03:40:56.8474527Z         True  # If True, the experiment is enabled by default for all queries
2025-07-24T03:40:56.8475160Z     )
2025-07-24T03:40:56.8475354Z 
2025-07-24T03:40:56.8475524Z     # Add more fields as needed
2025-07-24T03:40:56.8475831Z 
2025-07-24T03:40:56.8475838Z 
2025-07-24T03:40:56.8476017Z class Settings(NamedTuple):
2025-07-24T03:40:56.8476459Z     """
2025-07-24T03:40:56.8476897Z     Settings for the experiments that can be opted into.
2025-07-24T03:40:56.8477461Z     """
2025-07-24T03:40:56.8477653Z 
2025-07-24T03:40:56.8477850Z     experiments: dict[str, Experiment] = {}
2025-07-24T03:40:56.8478224Z 
2025-07-24T03:40:56.8478229Z 
2025-07-24T03:40:56.8478434Z class ColorFormatter(logging.Formatter):
2025-07-24T03:40:56.8479043Z     """Color codes the log messages based on the log level"""
2025-07-24T03:40:56.8479478Z 
2025-07-24T03:40:56.8479633Z     COLORS = {
2025-07-24T03:40:56.8480029Z         "WARNING": "\033[33m",  # Yellow
2025-07-24T03:40:56.8480667Z         "ERROR": "\033[31m",  # Red
2025-07-24T03:40:56.8481158Z         "CRITICAL": "\033[31m",  # Red
2025-07-24T03:40:56.8481914Z         "INFO": "\033[0m",  # Reset
2025-07-24T03:40:56.8482402Z         "DEBUG": "\033[0m",  # Reset
2025-07-24T03:40:56.8482851Z     }
2025-07-24T03:40:56.8483050Z 
2025-07-24T03:40:56.8483256Z     def format(self, record: LogRecord) -> str:
2025-07-24T03:40:56.8483985Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2025-07-24T03:40:56.8484744Z         record.msg = f"{log_color}{record.msg}\033[0m"
2025-07-24T03:40:56.8485312Z         return super().format(record)
2025-07-24T03:40:56.8485640Z 
2025-07-24T03:40:56.8485646Z 
2025-07-24T03:40:56.8485831Z handler = logging.StreamHandler()
2025-07-24T03:40:56.8486518Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2025-07-24T03:40:56.8487068Z 
2025-07-24T03:40:56.8487305Z log = logging.getLogger(os.path.basename(__file__))
2025-07-24T03:40:56.8487880Z log.addHandler(handler)
2025-07-24T03:40:56.8488325Z log.setLevel(logging.INFO)
2025-07-24T03:40:56.8488608Z 
2025-07-24T03:40:56.8488614Z 
2025-07-24T03:40:56.8488848Z def set_github_output(key: str, value: str) -> None:
2025-07-24T03:40:56.8489401Z     """
2025-07-24T03:40:56.8489888Z     Defines outputs of the github action that invokes this script
2025-07-24T03:40:56.8490498Z     """
2025-07-24T03:40:56.8490850Z     if not GITHUB_OUTPUT:
2025-07-24T03:40:56.8492134Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2025-07-24T03:40:56.8493238Z         log.warning(
2025-07-24T03:40:56.8494056Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2025-07-24T03:40:56.8494969Z         )
2025-07-24T03:40:56.8504819Z         print(f"::set-output name={key}::{value}")
2025-07-24T03:40:56.8505422Z         return
2025-07-24T03:40:56.8505665Z 
2025-07-24T03:40:56.8506036Z     with open(GITHUB_OUTPUT, "a") as f:
2025-07-24T03:40:56.8506620Z         log.info(f"Setting output: {key}='{value}'")
2025-07-24T03:40:56.8507187Z         f.write(f"{key}={value}\n")
2025-07-24T03:40:56.8507516Z 
2025-07-24T03:40:56.8507522Z 
2025-07-24T03:40:56.8507814Z def _str_comma_separated_to_set(value: str) -> frozenset[str]:
2025-07-24T03:40:56.8508428Z     return frozenset(
2025-07-24T03:40:56.8509011Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2025-07-24T03:40:56.8509676Z     )
2025-07-24T03:40:56.8509865Z 
2025-07-24T03:40:56.8509873Z 
2025-07-24T03:40:56.8510043Z def parse_args() -> Any:
2025-07-24T03:40:56.8510583Z     parser = ArgumentParser("Get dynamic rollout settings")
2025-07-24T03:40:56.8511412Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2025-07-24T03:40:56.8512492Z     parser.add_argument(
2025-07-24T03:40:56.8512948Z         "--github-issue-repo",
2025-07-24T03:40:56.8513398Z         type=str,
2025-07-24T03:40:56.8513796Z         required=False,
2025-07-24T03:40:56.8514234Z         default="pytorch/test-infra",
2025-07-24T03:40:56.8514755Z         help="GitHub repo to get the issue",
2025-07-24T03:40:56.8515251Z     )
2025-07-24T03:40:56.8515603Z     parser.add_argument(
2025-07-24T03:40:56.8516034Z         "--github-repo",
2025-07-24T03:40:56.8516449Z         type=str,
2025-07-24T03:40:56.8516838Z         required=True,
2025-07-24T03:40:56.8517277Z         help="GitHub repo where CI is running",
2025-07-24T03:40:56.8517835Z     )
2025-07-24T03:40:56.8518185Z     parser.add_argument(
2025-07-24T03:40:56.8518772Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2025-07-24T03:40:56.8519404Z     )
2025-07-24T03:40:56.8519752Z     parser.add_argument(
2025-07-24T03:40:56.8520348Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2025-07-24T03:40:56.8521009Z     )
2025-07-24T03:40:56.8521700Z     parser.add_argument(
2025-07-24T03:40:56.8522374Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2025-07-24T03:40:56.8523058Z     )
2025-07-24T03:40:56.8523405Z     parser.add_argument(
2025-07-24T03:40:56.8524046Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2025-07-24T03:40:56.8524730Z     )
2025-07-24T03:40:56.8525079Z     parser.add_argument(
2025-07-24T03:40:56.8525508Z         "--github-ref-type",
2025-07-24T03:40:56.8525959Z         type=str,
2025-07-24T03:40:56.8526330Z         required=True,
2025-07-24T03:40:56.8526799Z         help="Current GitHub ref type, branch or tag",
2025-07-24T03:40:56.8527335Z     )
2025-07-24T03:40:56.8527705Z     parser.add_argument(
2025-07-24T03:40:56.8528160Z         "--eligible-experiments",
2025-07-24T03:40:56.8528663Z         type=_str_comma_separated_to_set,
2025-07-24T03:40:56.8529170Z         required=False,
2025-07-24T03:40:56.8529566Z         default="",
2025-07-24T03:40:56.8530381Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2025-07-24T03:40:56.8531277Z     )
2025-07-24T03:40:56.8531970Z     parser.add_argument(
2025-07-24T03:40:56.8532692Z         "--opt-out-experiments",
2025-07-24T03:40:56.8533476Z         type=_str_comma_separated_to_set,
2025-07-24T03:40:56.8534394Z         required=False,
2025-07-24T03:40:56.8535123Z         default="",
2025-07-24T03:40:56.8535805Z         help=(
2025-07-24T03:40:56.8536961Z             "comma separated list of experiments to opt-out of. If unset, no opt-outs will occur. "
2025-07-24T03:40:56.8538937Z             "If the same experiment is listed both here and in '--eligible-experiments' opt-out will take priority."
2025-07-24T03:40:56.8540332Z         ),
2025-07-24T03:40:56.8540906Z     )
2025-07-24T03:40:56.8541784Z     parser.add_argument(
2025-07-24T03:40:56.8542668Z         "--pr-number",
2025-07-24T03:40:56.8543350Z         type=str,
2025-07-24T03:40:56.8544017Z         required=False,
2025-07-24T03:40:56.8544714Z         default="",
2025-07-24T03:40:56.8545816Z         help="the optional PR number where this is run",
2025-07-24T03:40:56.8546829Z     )
2025-07-24T03:40:56.8547188Z 
2025-07-24T03:40:56.8547508Z     return parser.parse_args()
2025-07-24T03:40:56.8548051Z 
2025-07-24T03:40:56.8548076Z 
2025-07-24T03:40:56.8548768Z def get_gh_client(github_token: str) -> Github:  # type: ignore[no-any-unimported]
2025-07-24T03:40:56.8550046Z     auth = Auth.Token(github_token)
2025-07-24T03:40:56.8550899Z     return Github(auth=auth)
2025-07-24T03:40:56.8551387Z 
2025-07-24T03:40:56.8551400Z 
2025-07-24T03:40:56.8552372Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:  # type: ignore[no-any-unimported]
2025-07-24T03:40:56.8553758Z     repo = gh.get_repo(repo)
2025-07-24T03:40:56.8554568Z     return repo.get_issue(number=issue_num)
2025-07-24T03:40:56.8555169Z 
2025-07-24T03:40:56.8555180Z 
2025-07-24T03:40:56.8555490Z def get_potential_pr_author(
2025-07-24T03:40:56.8556568Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2025-07-24T03:40:56.8557693Z ) -> str:
2025-07-24T03:40:56.8558523Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2025-07-24T03:40:56.8559960Z     # Fetch the actual username from the original PR. The PR number is
2025-07-24T03:40:56.8561270Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2025-07-24T03:40:56.8562222Z 
2025-07-24T03:40:56.8562549Z     gh = get_gh_client(github_token)
2025-07-24T03:40:56.8563120Z 
2025-07-24T03:40:56.8563562Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2025-07-24T03:40:56.8564648Z         split_tag = ref_name.split("/")
2025-07-24T03:40:56.8565525Z         if (
2025-07-24T03:40:56.8566179Z             len(split_tag) == 3
2025-07-24T03:40:56.8567022Z             and split_tag[0] == "ciflow"
2025-07-24T03:40:56.8567942Z             and split_tag[2].isnumeric()
2025-07-24T03:40:56.8568813Z         ):
2025-07-24T03:40:56.8569775Z             pr_number = split_tag[2]
2025-07-24T03:40:56.8570577Z             try:
2025-07-24T03:40:56.8571223Z                 repository = gh.get_repo(repo)
2025-07-24T03:40:56.8572364Z                 pull = repository.get_pull(number=int(pr_number))
2025-07-24T03:40:56.8573266Z             except Exception as e:
2025-07-24T03:40:56.8573789Z                 raise Exception(  # noqa: TRY002
2025-07-24T03:40:56.8574449Z                     f"issue with pull request {pr_number} from repo {repository}"
2025-07-24T03:40:56.8575083Z                 ) from e
2025-07-24T03:40:56.8575613Z             return pull.user.login  # type: ignore[no-any-return]
2025-07-24T03:40:56.8576287Z     # In all other cases, return the original input username
2025-07-24T03:40:56.8576869Z     return username
2025-07-24T03:40:56.8577109Z 
2025-07-24T03:40:56.8577116Z 
2025-07-24T03:40:56.8577329Z def is_exception_branch(branch: str) -> bool:
2025-07-24T03:40:56.8577858Z     """
2025-07-24T03:40:56.8578470Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2025-07-24T03:40:56.8579236Z     """
2025-07-24T03:40:56.8579763Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2025-07-24T03:40:56.8580273Z 
2025-07-24T03:40:56.8580280Z 
2025-07-24T03:40:56.8580465Z def load_yaml(yaml_text: str) -> Any:
2025-07-24T03:40:56.8580949Z     try:
2025-07-24T03:40:56.8581323Z         data = yaml.safe_load(yaml_text)
2025-07-24T03:40:56.8582187Z         return data
2025-07-24T03:40:56.8582596Z     except yaml.YAMLError:
2025-07-24T03:40:56.8583059Z         log.exception("Error loading YAML")
2025-07-24T03:40:56.8583555Z         raise
2025-07-24T03:40:56.8583774Z 
2025-07-24T03:40:56.8583780Z 
2025-07-24T03:40:56.8584172Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> tuple[str, str]:
2025-07-24T03:40:56.8584894Z     """
2025-07-24T03:40:56.8585486Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2025-07-24T03:40:56.8586082Z 
2025-07-24T03:40:56.8586574Z     If the issue body contains "---" then the text above that is the settings
2025-07-24T03:40:56.8587302Z     and the text below is the list of opted in users.
2025-07-24T03:40:56.8587706Z 
2025-07-24T03:40:56.8588062Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2025-07-24T03:40:56.8588732Z     """
2025-07-24T03:40:56.8589148Z     rollout_state_parts = rollout_state.split("---")
2025-07-24T03:40:56.8589722Z     if len(rollout_state_parts) >= 2:
2025-07-24T03:40:56.8590304Z         return rollout_state_parts[0], rollout_state_parts[1]
2025-07-24T03:40:56.8590880Z     else:
2025-07-24T03:40:56.8591238Z         return "", rollout_state
2025-07-24T03:40:56.8591732Z 
2025-07-24T03:40:56.8591740Z 
2025-07-24T03:40:56.8591936Z class UserOptins(dict[str, list[str]]):
2025-07-24T03:40:56.8592430Z     """
2025-07-24T03:40:56.8592928Z     Dictionary of users with a list of features they have opted into
2025-07-24T03:40:56.8593564Z     """
2025-07-24T03:40:56.8593759Z 
2025-07-24T03:40:56.8593766Z 
2025-07-24T03:40:56.8594093Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2025-07-24T03:40:56.8594731Z     """
2025-07-24T03:40:56.8644201Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2025-07-24T03:40:56.8645436Z 
2025-07-24T03:40:56.8646088Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2025-07-24T03:40:56.8647099Z         - Example line: "@User1,lf,split_build"
2025-07-24T03:40:56.8647776Z         - A "#" prefix indicates the user is opted out of all experiments
2025-07-24T03:40:56.8648255Z 
2025-07-24T03:40:56.8648269Z 
2025-07-24T03:40:56.8648422Z     """
2025-07-24T03:40:56.8648788Z     optins = UserOptins()
2025-07-24T03:40:56.8649272Z     for user in user_optin_text.split("\n"):
2025-07-24T03:40:56.8649812Z         user = user.strip("\r\n\t -")
2025-07-24T03:40:56.8650602Z         if not user or not user.startswith("@"):
2025-07-24T03:40:56.8651149Z             # Not a valid user. Skip
2025-07-24T03:40:56.8651826Z             continue
2025-07-24T03:40:56.8652079Z 
2025-07-24T03:40:56.8652234Z         if user:
2025-07-24T03:40:56.8652661Z             usr_name = user.split(",")[0].strip("@")
2025-07-24T03:40:56.8653351Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2025-07-24T03:40:56.8653839Z 
2025-07-24T03:40:56.8653998Z     return optins
2025-07-24T03:40:56.8654238Z 
2025-07-24T03:40:56.8654244Z 
2025-07-24T03:40:56.8654523Z def is_valid_experiment_name(experiment_name: str) -> bool:
2025-07-24T03:40:56.8655101Z     """
2025-07-24T03:40:56.8655479Z     Check if the experiment name is valid.
2025-07-24T03:40:56.8655989Z     A valid name:
2025-07-24T03:40:56.8656605Z         - Contains only alphanumeric characters and the special characters "_" & "-"
2025-07-24T03:40:56.8657513Z         - The special characters "_" & "-" shouldn't be the first or last characters
2025-07-24T03:40:56.8658214Z         - Cannot contain spaces
2025-07-24T03:40:56.8658669Z     """
2025-07-24T03:40:56.8658866Z 
2025-07-24T03:40:56.8659115Z     valid_char_regex = r"^[a-zA-Z0-9]([\w-]*[a-zA-Z0-9])?$"
2025-07-24T03:40:56.8659806Z     valid = bool(re.match(valid_char_regex, experiment_name))
2025-07-24T03:40:56.8660252Z 
2025-07-24T03:40:56.8660409Z     if valid:
2025-07-24T03:40:56.8660775Z         return True
2025-07-24T03:40:56.8661009Z 
2025-07-24T03:40:56.8661169Z     log.error(
2025-07-24T03:40:56.8662918Z         f"Invalid experiment name: {experiment_name}. Experiment names should only contain alphanumeric characters, '_', and '-'. They cannot contain spaces, and the special characters '_' and '-' cannot be the first or last characters."
2025-07-24T03:40:56.8664488Z     )
2025-07-24T03:40:56.8664843Z     return False
2025-07-24T03:40:56.8665071Z 
2025-07-24T03:40:56.8665078Z 
2025-07-24T03:40:56.8665373Z def parse_settings_from_text(settings_text: str) -> Settings:
2025-07-24T03:40:56.8665984Z     """
2025-07-24T03:40:56.8666690Z     Parse the experiments from the issue body into a list of ExperimentSettings
2025-07-24T03:40:56.8667411Z     """
2025-07-24T03:40:56.8667750Z     try:
2025-07-24T03:40:56.8668115Z         if settings_text:
2025-07-24T03:40:56.8668828Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2025-07-24T03:40:56.8669598Z             # for easy reading
2025-07-24T03:40:56.8670367Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2025-07-24T03:40:56.8671218Z             #       the backtick character in shell commands.
2025-07-24T03:40:56.8672019Z             backtick = chr(96)  # backtick character
2025-07-24T03:40:56.8672675Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2025-07-24T03:40:56.8673329Z             settings = load_yaml(settings_text)
2025-07-24T03:40:56.8673689Z 
2025-07-24T03:40:56.8674095Z             # For now we just load experiments. We can expand this if/when we add more settings
2025-07-24T03:40:56.8674830Z             experiments = {}
2025-07-24T03:40:56.8675133Z 
2025-07-24T03:40:56.8675492Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2025-07-24T03:40:56.8676216Z                 if not is_valid_experiment_name(exp_name):
2025-07-24T03:40:56.8677282Z                     # Exclude invalid experiments from the list. We log an error, but don't raise an exception so that other experiments can still be processed.
2025-07-24T03:40:56.8678292Z                     continue
2025-07-24T03:40:56.8678574Z 
2025-07-24T03:40:56.8678748Z                 valid_settings = {}
2025-07-24T03:40:56.8679250Z                 for setting in exp_settings:
2025-07-24T03:40:56.8679794Z                     if setting not in Experiment._fields:
2025-07-24T03:40:56.8680338Z                         log.warning(
2025-07-24T03:40:56.8681021Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2025-07-24T03:40:56.8682082Z                         )
2025-07-24T03:40:56.8682505Z                     else:
2025-07-24T03:40:56.8683011Z                         valid_settings[setting] = exp_settings[setting]
2025-07-24T03:40:56.8683425Z 
2025-07-24T03:40:56.8683694Z                 experiments[exp_name] = Experiment(**valid_settings)
2025-07-24T03:40:56.8684305Z             return Settings(experiments)
2025-07-24T03:40:56.8684653Z 
2025-07-24T03:40:56.8684817Z     except Exception:
2025-07-24T03:40:56.8685277Z         log.exception("Failed to parse settings")
2025-07-24T03:40:56.8685671Z 
2025-07-24T03:40:56.8685832Z     return Settings()
2025-07-24T03:40:56.8686087Z 
2025-07-24T03:40:56.8686094Z 
2025-07-24T03:40:56.8686335Z def parse_settings(rollout_state: str) -> Settings:
2025-07-24T03:40:56.8686877Z     """
2025-07-24T03:40:56.8687307Z     Parse settings, if any, from the rollout state.
2025-07-24T03:40:56.8687701Z 
2025-07-24T03:40:56.8688040Z     If the issue body contains "---" then the text above that is the settings
2025-07-24T03:40:56.8688774Z     and the text below is the list of opted in users.
2025-07-24T03:40:56.8689164Z 
2025-07-24T03:40:56.8689556Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2025-07-24T03:40:56.8690262Z     """
2025-07-24T03:40:56.8690793Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2025-07-24T03:40:56.8691689Z     return parse_settings_from_text(settings_text)
2025-07-24T03:40:56.8692188Z 
2025-07-24T03:40:56.8692194Z 
2025-07-24T03:40:56.8692425Z def parse_users(rollout_state: str) -> UserOptins:
2025-07-24T03:40:56.8692967Z     """
2025-07-24T03:40:56.8693344Z     Parse users from the rollout state.
2025-07-24T03:40:56.8693688Z 
2025-07-24T03:40:56.8693840Z     """
2025-07-24T03:40:56.8694342Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2025-07-24T03:40:56.8695055Z     return parse_user_opt_in_from_text(users_text)
2025-07-24T03:40:56.8695453Z 
2025-07-24T03:40:56.8695460Z 
2025-07-24T03:40:56.8695983Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-07-24T03:40:56.8696714Z     """
2025-07-24T03:40:56.8697109Z     Check if a user is opted into an experiment
2025-07-24T03:40:56.8697631Z     """
2025-07-24T03:40:56.8698059Z     return experiment_name in user_optins.get(user, [])
2025-07-24T03:40:56.8698478Z 
2025-07-24T03:40:56.8698484Z 
2025-07-24T03:40:56.8698880Z def is_user_opted_out(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-07-24T03:40:56.8699605Z     """
2025-07-24T03:40:56.8700068Z     Check if a user explicitly opted out of an experiment
2025-07-24T03:40:56.8700626Z     """
2025-07-24T03:40:56.8701098Z     # if the experiment is prefixed with a "-", then it's an opt-out
2025-07-24T03:40:56.8702064Z     experiment_optout = "-" + experiment_name
2025-07-24T03:40:56.8702701Z     if experiment_optout not in user_optins.get(user, []):
2025-07-24T03:40:56.8703292Z         return False
2025-07-24T03:40:56.8703543Z 
2025-07-24T03:40:56.8703810Z     if is_user_opted_in(user, user_optins, experiment_name):
2025-07-24T03:40:56.8704382Z         log.warning(
2025-07-24T03:40:56.8705163Z             f"User {user} is opted into experiment {experiment_name}, but also opted out of it. Defaulting to opting out"
2025-07-24T03:40:56.8706009Z         )
2025-07-24T03:40:56.8706220Z 
2025-07-24T03:40:56.8706372Z     return True
2025-07-24T03:40:56.8706596Z 
2025-07-24T03:40:56.8706602Z 
2025-07-24T03:40:56.8706774Z def get_runner_prefix(
2025-07-24T03:40:56.8707190Z     rollout_state: str,
2025-07-24T03:40:56.8707635Z     workflow_requestors: Iterable[str],
2025-07-24T03:40:56.8708127Z     branch: str,
2025-07-24T03:40:56.8708601Z     eligible_experiments: frozenset[str] = frozenset(),
2025-07-24T03:40:56.8709239Z     opt_out_experiments: frozenset[str] = frozenset(),
2025-07-24T03:40:56.8709810Z     is_canary: bool = False,
2025-07-24T03:40:56.8710400Z ) -> str:
2025-07-24T03:40:56.8710809Z     settings = parse_settings(rollout_state)
2025-07-24T03:40:56.8711362Z     user_optins = parse_users(rollout_state)
2025-07-24T03:40:56.8711961Z 
2025-07-24T03:40:56.8712130Z     fleet_prefix = ""
2025-07-24T03:40:56.8712542Z     prefixes = []
2025-07-24T03:40:56.8713138Z     for experiment_name, experiment_settings in settings.experiments.items():
2025-07-24T03:40:56.8714037Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2025-07-24T03:40:56.8714711Z             log.info(
2025-07-24T03:40:56.8715380Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2025-07-24T03:40:56.8716125Z             )
2025-07-24T03:40:56.8716488Z             continue
2025-07-24T03:40:56.8716729Z 
2025-07-24T03:40:56.8716912Z         if opt_out_experiments:
2025-07-24T03:40:56.8717414Z             if experiment_name in opt_out_experiments:
2025-07-24T03:40:56.8718071Z                 opt_out_exp_list = ", ".join(opt_out_experiments)
2025-07-24T03:40:56.8718630Z                 log.info(
2025-07-24T03:40:56.8719514Z                     f"Skipping experiment '{experiment_name}', as this workflow has opted-out (opted out experiments are: {opt_out_exp_list})"
2025-07-24T03:40:56.8720449Z                 )
2025-07-24T03:40:56.8720832Z                 continue
2025-07-24T03:40:56.8721093Z 
2025-07-24T03:40:56.8721275Z         if eligible_experiments:
2025-07-24T03:40:56.8722036Z             if experiment_name not in eligible_experiments:
2025-07-24T03:40:56.8722663Z                 exp_list = ", ".join(eligible_experiments)
2025-07-24T03:40:56.8723195Z                 log.info(
2025-07-24T03:40:56.8723946Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2025-07-24T03:40:56.8724743Z                 )
2025-07-24T03:40:56.8725127Z                 continue
2025-07-24T03:40:56.8725592Z         elif not experiment_settings.default:
2025-07-24T03:40:56.8726120Z             log.info(
2025-07-24T03:40:56.8726893Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2025-07-24T03:40:56.8727614Z             )
2025-07-24T03:40:56.8727981Z             continue
2025-07-24T03:40:56.8728222Z 
2025-07-24T03:40:56.8728488Z         # Is any workflow_requestor opted out to this experiment?
2025-07-24T03:40:56.8729083Z         opted_out_users = [
2025-07-24T03:40:56.8729506Z             requestor
2025-07-24T03:40:56.8729949Z             for requestor in workflow_requestors
2025-07-24T03:40:56.8730591Z             if is_user_opted_out(requestor, user_optins, experiment_name)
2025-07-24T03:40:56.8731207Z         ]
2025-07-24T03:40:56.8731410Z 
2025-07-24T03:40:56.8732007Z         if opted_out_users:
2025-07-24T03:40:56.8732462Z             log.info(
2025-07-24T03:40:56.8733055Z                 f"{', '.join(opted_out_users)} have opted out of experiment {experiment_name}."
2025-07-24T03:40:56.8733727Z             )
2025-07-24T03:40:56.8734105Z             continue
2025-07-24T03:40:56.8734345Z 
2025-07-24T03:40:56.8734604Z         # Is any workflow_requestor opted in to this experiment?
2025-07-24T03:40:56.8735194Z         opted_in_users = [
2025-07-24T03:40:56.8735616Z             requestor
2025-07-24T03:40:56.8736055Z             for requestor in workflow_requestors
2025-07-24T03:40:56.8736701Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2025-07-24T03:40:56.8737307Z         ]
2025-07-24T03:40:56.8737508Z 
2025-07-24T03:40:56.8737672Z         enabled = False
2025-07-24T03:40:56.8738087Z         if opted_in_users:
2025-07-24T03:40:56.8738510Z             log.info(
2025-07-24T03:40:56.8739081Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2025-07-24T03:40:56.8739730Z             )
2025-07-24T03:40:56.8740097Z             enabled = True
2025-07-24T03:40:56.8740369Z 
2025-07-24T03:40:56.8740574Z         elif experiment_settings.rollout_perc:
2025-07-24T03:40:56.8741718Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2025-07-24T03:40:56.8742717Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2025-07-24T03:40:56.8743347Z                 log.info(
2025-07-24T03:40:56.8744181Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2025-07-24T03:40:56.8745070Z                 )
2025-07-24T03:40:56.8745452Z                 enabled = True
2025-07-24T03:40:56.8745743Z 
2025-07-24T03:40:56.8745897Z         if enabled:
2025-07-24T03:40:56.8746303Z             label = experiment_name
2025-07-24T03:40:56.8746828Z             if experiment_name == LF_FLEET_EXPERIMENT:
2025-07-24T03:40:56.8747617Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2025-07-24T03:40:56.8748461Z                 #  - If it's enabled, then we always list it's prefix first
2025-07-24T03:40:56.8749197Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2025-07-24T03:40:56.8749834Z                 if is_canary:
2025-07-24T03:40:56.8750309Z                     label += CANARY_FLEET_SUFFIX
2025-07-24T03:40:56.8750834Z                 fleet_prefix = label
2025-07-24T03:40:56.8751297Z             else:
2025-07-24T03:40:56.8751815Z                 prefixes.append(label)
2025-07-24T03:40:56.8752154Z 
2025-07-24T03:40:56.8752326Z     if len(prefixes) > 1:
2025-07-24T03:40:56.8752751Z         log.error(
2025-07-24T03:40:56.8753735Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2025-07-24T03:40:56.8754807Z         )
2025-07-24T03:40:56.8755171Z         prefixes = prefixes[:1]
2025-07-24T03:40:56.8755476Z 
2025-07-24T03:40:56.8755649Z     # Fleet always comes first
2025-07-24T03:40:56.8756103Z     if fleet_prefix:
2025-07-24T03:40:56.8756534Z         prefixes.insert(0, fleet_prefix)
2025-07-24T03:40:56.8756877Z 
2025-07-24T03:40:56.8757241Z     return ".".join(prefixes) + "." if prefixes else ""
2025-07-24T03:40:56.8757649Z 
2025-07-24T03:40:56.8757654Z 
2025-07-24T03:40:56.8758084Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2025-07-24T03:40:56.8758824Z     """
2025-07-24T03:40:56.8759388Z     Gets the first comment of the issue, which contains the desired rollout state.
2025-07-24T03:40:56.8759926Z 
2025-07-24T03:40:56.8760292Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2025-07-24T03:40:56.8760976Z     """
2025-07-24T03:40:56.8761345Z     gh = get_gh_client(github_token)
2025-07-24T03:40:56.8762159Z     issue = get_issue(gh, repo, issue_num)
2025-07-24T03:40:56.8762770Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2025-07-24T03:40:56.8763213Z 
2025-07-24T03:40:56.8763219Z 
2025-07-24T03:40:56.8763598Z def download_json(url: str, headers: dict[str, str], num_retries: int = 3) -> Any:
2025-07-24T03:40:56.8764346Z     for _ in range(num_retries):
2025-07-24T03:40:56.8764804Z         try:
2025-07-24T03:40:56.8765221Z             req = Request(url=url, headers=headers)
2025-07-24T03:40:56.8765855Z             content = urlopen(req, timeout=5).read().decode("utf-8")
2025-07-24T03:40:56.8766480Z             return json.loads(content)
2025-07-24T03:40:56.8766985Z         except Exception as e:
2025-07-24T03:40:56.8767507Z             log.warning(f"Could not download {url}: {e}")
2025-07-24T03:40:56.8767900Z 
2025-07-24T03:40:56.8768268Z     log.warning(f"All {num_retries} retries exhausted, downloading {url} failed")
2025-07-24T03:40:56.8768947Z     return {}
2025-07-24T03:40:56.8769169Z 
2025-07-24T03:40:56.8769176Z 
2025-07-24T03:40:56.8769332Z @cache
2025-07-24T03:40:56.8769919Z def get_pr_info(github_repo: str, github_token: str, pr_number: int) -> dict[str, Any]:
2025-07-24T03:40:56.8770641Z     """
2025-07-24T03:40:56.8771003Z     Dynamically get PR information
2025-07-24T03:40:56.8771735Z     """
2025-07-24T03:40:56.8772212Z     github_api = f"https://api.github.com/repos/{github_repo}"
2025-07-24T03:40:56.8772815Z     headers = {
2025-07-24T03:40:56.8773253Z         "Accept": "application/vnd.github.v3+json",
2025-07-24T03:40:56.8773833Z         "Authorization": f"token {github_token}",
2025-07-24T03:40:56.8774351Z     }
2025-07-24T03:40:56.8774756Z     json_response: dict[str, Any] = download_json(
2025-07-24T03:40:56.8775338Z         url=f"{github_api}/issues/{pr_number}",
2025-07-24T03:40:56.8775852Z         headers=headers,
2025-07-24T03:40:56.8776261Z     )
2025-07-24T03:40:56.8776457Z 
2025-07-24T03:40:56.8776622Z     if not json_response:
2025-07-24T03:40:56.8777165Z         log.warning(f"Failed to get the labels for #{pr_number}")
2025-07-24T03:40:56.8777763Z         return {}
2025-07-24T03:40:56.8777994Z 
2025-07-24T03:40:56.8778166Z     return json_response
2025-07-24T03:40:56.8778441Z 
2025-07-24T03:40:56.8778448Z 
2025-07-24T03:40:56.8778837Z def get_labels(github_repo: str, github_token: str, pr_number: int) -> set[str]:
2025-07-24T03:40:56.8779534Z     """
2025-07-24T03:40:56.8780037Z     Dynamically get the latest list of labels from the pull request
2025-07-24T03:40:56.8780656Z     """
2025-07-24T03:40:56.8781120Z     pr_info = get_pr_info(github_repo, github_token, pr_number)
2025-07-24T03:40:56.8781953Z     return {
2025-07-24T03:40:56.8782514Z         label.get("name") for label in pr_info.get("labels", []) if label.get("name")
2025-07-24T03:40:56.8783187Z     }
2025-07-24T03:40:56.8783383Z 
2025-07-24T03:40:56.8783390Z 
2025-07-24T03:40:56.8783550Z def main() -> None:
2025-07-24T03:40:56.8783953Z     args = parse_args()
2025-07-24T03:40:56.8784216Z 
2025-07-24T03:40:56.8784419Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2025-07-24T03:40:56.8784797Z 
2025-07-24T03:40:56.8784973Z     # Check if the PR is opt-out
2025-07-24T03:40:56.8785446Z     if args.pr_number:
2025-07-24T03:40:56.8786063Z         labels = get_labels(args.github_repo, args.github_token, int(args.pr_number))
2025-07-24T03:40:56.8786916Z         if OPT_OUT_LABEL in labels:
2025-07-24T03:40:56.8787407Z             log.info(
2025-07-24T03:40:56.8788067Z                 f"Opt-out runner determinator because #{args.pr_number} has {OPT_OUT_LABEL} label"
2025-07-24T03:40:56.8788799Z             )
2025-07-24T03:40:56.8789319Z             set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-07-24T03:40:56.8789960Z             sys.exit()
2025-07-24T03:40:56.8790219Z 
2025-07-24T03:40:56.8790368Z     try:
2025-07-24T03:40:56.8790788Z         rollout_state = get_rollout_state_from_issue(
2025-07-24T03:40:56.8791456Z             args.github_token, args.github_issue_repo, args.github_issue
2025-07-24T03:40:56.8792184Z         )
2025-07-24T03:40:56.8792384Z 
2025-07-24T03:40:56.8792575Z         username = get_potential_pr_author(
2025-07-24T03:40:56.8793101Z             args.github_token,
2025-07-24T03:40:56.8793559Z             args.github_repo,
2025-07-24T03:40:56.8794019Z             args.github_actor,
2025-07-24T03:40:56.8794474Z             args.github_ref_type,
2025-07-24T03:40:56.8794949Z             args.github_branch,
2025-07-24T03:40:56.8795403Z         )
2025-07-24T03:40:56.8795603Z 
2025-07-24T03:40:56.8795874Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2025-07-24T03:40:56.8796313Z 
2025-07-24T03:40:56.8796521Z         runner_label_prefix = get_runner_prefix(
2025-07-24T03:40:56.8797052Z             rollout_state,
2025-07-24T03:40:56.8797514Z             (args.github_issue_owner, username),
2025-07-24T03:40:56.8798033Z             args.github_branch,
2025-07-24T03:40:56.8798500Z             args.eligible_experiments,
2025-07-24T03:40:56.8799012Z             args.opt_out_experiments,
2025-07-24T03:40:56.8799489Z             is_canary,
2025-07-24T03:40:56.8799878Z         )
2025-07-24T03:40:56.8800081Z 
2025-07-24T03:40:56.8800252Z     except Exception as e:
2025-07-24T03:40:56.8800687Z         log.error(
2025-07-24T03:40:56.8801315Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2025-07-24T03:40:56.8802477Z         )
2025-07-24T03:40:56.8802690Z 
2025-07-24T03:40:56.8803000Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-07-24T03:40:56.8803480Z 
2025-07-24T03:40:56.8803487Z 
2025-07-24T03:40:56.8803652Z if __name__ == "__main__":
2025-07-24T03:40:56.8804072Z     main()
2025-07-24T03:40:56.8804281Z 
2025-07-24T03:40:56.8890938Z ##[group]Run python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2025-07-24T03:40:56.8891967Z python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2025-07-24T03:40:56.8919717Z shell: /usr/bin/bash -e {0}
2025-07-24T03:40:56.8920144Z env:
2025-07-24T03:40:56.8920717Z   GITHUB_TOKEN: ***
2025-07-24T03:40:56.8921086Z   ISSUE_NUMBER: 5132
2025-07-24T03:40:56.8921721Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2025-07-24T03:40:56.8922281Z   ISSUE_OWNER: 
2025-07-24T03:40:56.8922642Z   CHECK_EXPERIMENTS: 
2025-07-24T03:40:56.8923028Z   OPT_OUT_EXPERIMENTS: 
2025-07-24T03:40:56.8923422Z   PR_NUMBER: 
2025-07-24T03:40:56.8923750Z ##[endgroup]
2025-07-24T03:40:57.9032918Z Defaulting to user installation because normal site-packages is not writeable
2025-07-24T03:40:58.5131853Z Collecting urllib3==1.26.18
2025-07-24T03:40:58.5880404Z   Downloading urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
2025-07-24T03:40:58.6137450Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 2.6 MB/s eta 0:00:00
2025-07-24T03:40:58.6503020Z Collecting PyGithub==2.3.0
2025-07-24T03:40:58.6677138Z   Downloading PyGithub-2.3.0-py3-none-any.whl.metadata (3.8 kB)
2025-07-24T03:40:58.7240059Z Collecting pynacl>=1.4.0 (from PyGithub==2.3.0)
2025-07-24T03:40:58.7400505Z   Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (8.6 kB)
2025-07-24T03:40:58.7444780Z Requirement already satisfied: requests>=2.14.0 in /usr/lib/python3/dist-packages (from PyGithub==2.3.0) (2.31.0)
2025-07-24T03:40:58.7461255Z Requirement already satisfied: pyjwt>=2.4.0 in /usr/lib/python3/dist-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.3.0) (2.7.0)
2025-07-24T03:40:58.7475883Z Requirement already satisfied: typing-extensions>=4.0.0 in /usr/lib/python3/dist-packages (from PyGithub==2.3.0) (4.10.0)
2025-07-24T03:40:58.7846548Z Collecting Deprecated (from PyGithub==2.3.0)
2025-07-24T03:40:58.8019881Z   Downloading Deprecated-1.2.18-py2.py3-none-any.whl.metadata (5.7 kB)
2025-07-24T03:40:58.8250731Z Requirement already satisfied: cryptography>=3.4.0 in /usr/lib/python3/dist-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.3.0) (41.0.7)
2025-07-24T03:40:58.9715934Z Collecting cffi>=1.4.1 (from pynacl>=1.4.0->PyGithub==2.3.0)
2025-07-24T03:40:58.9878573Z   Downloading cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.5 kB)
2025-07-24T03:40:59.1026480Z Collecting wrapt<2,>=1.10 (from Deprecated->PyGithub==2.3.0)
2025-07-24T03:40:59.1194351Z   Downloading wrapt-1.17.2-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.4 kB)
2025-07-24T03:40:59.1489035Z Collecting pycparser (from cffi>=1.4.1->pynacl>=1.4.0->PyGithub==2.3.0)
2025-07-24T03:40:59.1676035Z   Downloading pycparser-2.22-py3-none-any.whl.metadata (943 bytes)
2025-07-24T03:40:59.2038535Z Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
2025-07-24T03:40:59.2219995Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.8/143.8 kB 8.2 MB/s eta 0:00:00
2025-07-24T03:40:59.2424080Z Downloading PyGithub-2.3.0-py3-none-any.whl (354 kB)
2025-07-24T03:40:59.2624480Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 354.4/354.4 kB 19.0 MB/s eta 0:00:00
2025-07-24T03:40:59.2842499Z Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
2025-07-24T03:40:59.3072390Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 856.7/856.7 kB 39.7 MB/s eta 0:00:00
2025-07-24T03:40:59.3230815Z Downloading Deprecated-1.2.18-py2.py3-none-any.whl (10.0 kB)
2025-07-24T03:40:59.3411754Z Downloading cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (479 kB)
2025-07-24T03:40:59.3476159Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 479.4/479.4 kB 100.5 MB/s eta 0:00:00
2025-07-24T03:40:59.3649920Z Downloading wrapt-1.17.2-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (89 kB)
2025-07-24T03:40:59.3690864Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.2/89.2 kB 31.2 MB/s eta 0:00:00
2025-07-24T03:40:59.3848178Z Downloading pycparser-2.22-py3-none-any.whl (117 kB)
2025-07-24T03:40:59.3891740Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 kB 39.3 MB/s eta 0:00:00
2025-07-24T03:40:59.7106512Z Installing collected packages: wrapt, urllib3, pycparser, Deprecated, cffi, pynacl, PyGithub
2025-07-24T03:41:00.2405547Z Successfully installed Deprecated-1.2.18 PyGithub-2.3.0 cffi-1.17.1 pycparser-2.22 pynacl-1.5.0 urllib3-1.26.18 wrapt-1.17.2
2025-07-24T03:41:00.3190693Z ##[group]Run curr_branch="ciflow/trunk/149961"
2025-07-24T03:41:00.3191097Z curr_branch="ciflow/trunk/149961"
2025-07-24T03:41:00.3191357Z curr_ref_type="tag"
2025-07-24T03:41:00.3191759Z echo "Current branch is '$curr_branch'"
2025-07-24T03:41:00.3192029Z 
2025-07-24T03:41:00.3192224Z python3 runner_determinator.py \
2025-07-24T03:41:00.3192514Z   --github-token "$GITHUB_TOKEN" \
2025-07-24T03:41:00.3192791Z   --github-issue "$ISSUE_NUMBER" \
2025-07-24T03:41:00.3193041Z   --github-branch "$curr_branch" \
2025-07-24T03:41:00.3193306Z   --github-actor "$TRIGGERING_ACTOR" \
2025-07-24T03:41:00.3193584Z   --github-issue-owner "$ISSUE_OWNER" \
2025-07-24T03:41:00.3193865Z   --github-ref-type "$curr_ref_type" \
2025-07-24T03:41:00.3194140Z   --github-repo "$GITHUB_REPOSITORY" \
2025-07-24T03:41:00.3194495Z   --eligible-experiments "$CHECK_EXPERIMENTS" \
2025-07-24T03:41:00.3194818Z   --opt-out-experiments "$OPT_OUT_EXPERIMENTS" \
2025-07-24T03:41:00.3195115Z   --pr-number "${PR_NUMBER}"
2025-07-24T03:41:00.3222701Z shell: /usr/bin/bash -e {0}
2025-07-24T03:41:00.3222923Z env:
2025-07-24T03:41:00.3223389Z   GITHUB_TOKEN: ***
2025-07-24T03:41:00.3223573Z   ISSUE_NUMBER: 5132
2025-07-24T03:41:00.3223781Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2025-07-24T03:41:00.3224019Z   ISSUE_OWNER: 
2025-07-24T03:41:00.3224192Z   CHECK_EXPERIMENTS: 
2025-07-24T03:41:00.3224389Z   OPT_OUT_EXPERIMENTS: 
2025-07-24T03:41:00.3224579Z   PR_NUMBER: 
2025-07-24T03:41:00.3224756Z ##[endgroup]
2025-07-24T03:41:00.3269968Z Current branch is 'ciflow/trunk/149961'
2025-07-24T03:41:02.6019480Z INFO    : Based on rollout percentage of 35%, enabling experiment lf.
2025-07-24T03:41:02.6020499Z INFO    : Setting output: label-type='lf.'
2025-07-24T03:41:02.6339099Z Evaluate and set job outputs
2025-07-24T03:41:02.6346179Z Set output 'label-type'
2025-07-24T03:41:02.6347979Z Cleaning up orphan processes