2025-04-25T04:10:05.8556283Z Current runner version: '2.323.0'
2025-04-25T04:10:05.8590596Z ##[group]Operating System
2025-04-25T04:10:05.8591849Z Ubuntu
2025-04-25T04:10:05.8592774Z 24.04.2
2025-04-25T04:10:05.8593586Z LTS
2025-04-25T04:10:05.8594277Z ##[endgroup]
2025-04-25T04:10:05.8595213Z ##[group]Runner Image
2025-04-25T04:10:05.8596165Z Image: ubuntu-24.04
2025-04-25T04:10:05.8596934Z Version: 20250420.1.0
2025-04-25T04:10:05.8598687Z Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20250420.1/images/ubuntu/Ubuntu2404-Readme.md
2025-04-25T04:10:05.8601558Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20250420.1
2025-04-25T04:10:05.8603159Z ##[endgroup]
2025-04-25T04:10:05.8604092Z ##[group]Runner Image Provisioner
2025-04-25T04:10:05.8605102Z 2.0.422.1
2025-04-25T04:10:05.8605965Z ##[endgroup]
2025-04-25T04:10:05.8607780Z ##[group]GITHUB_TOKEN Permissions
2025-04-25T04:10:05.8610808Z Contents: read
2025-04-25T04:10:05.8611797Z Metadata: read
2025-04-25T04:10:05.8612755Z Packages: read
2025-04-25T04:10:05.8613729Z ##[endgroup]
2025-04-25T04:10:05.8616917Z Secret source: Actions
2025-04-25T04:10:05.8618100Z Prepare workflow directory
2025-04-25T04:10:05.9381901Z Prepare all required actions
2025-04-25T04:10:05.9459284Z Complete job name: get-label-type / runner-determinator
2025-04-25T04:10:06.0238933Z ##[group]Run cat <<EOF > runner_determinator.py
2025-04-25T04:10:06.0241724Z cat <<EOF > runner_determinator.py
2025-04-25T04:10:06.0242386Z # flake8: noqa: G004
2025-04-25T04:10:06.0242891Z 
2025-04-25T04:10:06.0243688Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2025-04-25T04:10:06.0244727Z #       must be kept in sync. You can do it easily by running the following command:
2025-04-25T04:10:06.0245616Z #           python .github/scripts/update_runner_determinator.py
2025-04-25T04:10:06.0246353Z 
2025-04-25T04:10:06.0246764Z """
2025-04-25T04:10:06.0247593Z This runner determinator is used to determine which set of runners to run a
2025-04-25T04:10:06.0248637Z GitHub job on. It uses the first comment of a GitHub issue (by default
2025-04-25T04:10:06.0249755Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2025-04-25T04:10:06.0250890Z of which runners should be used to run which job.
2025-04-25T04:10:06.0251523Z 
2025-04-25T04:10:06.0252196Z The configuration has two parts, the settings and a list of opted-in users,
2025-04-25T04:10:06.0253165Z separated by a line containing "---".  If the line is not present, the
2025-04-25T04:10:06.0254229Z settings are considered to be empty with only the second part, the user
2025-04-25T04:10:06.0255056Z list, defined.
2025-04-25T04:10:06.0255517Z 
2025-04-25T04:10:06.0256262Z The first part is a YAML block that defines the rollout settings. This can be
2025-04-25T04:10:06.0257287Z used to define any settings that are needed to determine which runners to use.
2025-04-25T04:10:06.0258223Z It's fields are defined by the RolloutSettings class below.
2025-04-25T04:10:06.0258983Z 
2025-04-25T04:10:06.0259647Z The second part is a list of users who are explicitly opted in to the LF fleet.
2025-04-25T04:10:06.0260915Z The user list is also a comma separated list of additional features or
2025-04-25T04:10:06.0261823Z experiments which the user could be opted in to.
2025-04-25T04:10:06.0262487Z 
2025-04-25T04:10:06.0262944Z The user list has the following rules:
2025-04-25T04:10:06.0263614Z 
2025-04-25T04:10:06.0264265Z - Users are GitHub usernames, which must start with the @ prefix
2025-04-25T04:10:06.0265181Z - Each user is also a comma-separated list of features/experiments to enable
2025-04-25T04:10:06.0266120Z - A "#" prefix opts the user out of all experiments
2025-04-25T04:10:06.0266967Z 
2025-04-25T04:10:06.0267418Z Example config:
2025-04-25T04:10:06.0268075Z     # A list of experiments that can be opted into.
2025-04-25T04:10:06.0268907Z     # This defines the behavior they'll induce when opted into.
2025-04-25T04:10:06.0269629Z     # Expected syntax is:
2025-04-25T04:10:06.0270539Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2025-04-25T04:10:06.0271620Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2025-04-25T04:10:06.0272452Z 
2025-04-25T04:10:06.0272894Z     experiments:
2025-04-25T04:10:06.0273447Z       lf:
2025-04-25T04:10:06.0273907Z         rollout_percent: 25
2025-04-25T04:10:06.0274521Z         all_branches: false
2025-04-25T04:10:06.0275066Z         default: true
2025-04-25T04:10:06.0275598Z     ---
2025-04-25T04:10:06.0276006Z 
2025-04-25T04:10:06.0276535Z     # Opt-ins:
2025-04-25T04:10:06.0277247Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2025-04-25T04:10:06.0278309Z     # and specifying experiments to enable in a comma-separated list.
2025-04-25T04:10:06.0279273Z     # To always opt out of an experiment, prefix it with a "-".
2025-04-25T04:10:06.0280013Z     # Experiments should be from the above list.
2025-04-25T04:10:06.0280747Z 
2025-04-25T04:10:06.0281309Z     @User1,-lf,split_build
2025-04-25T04:10:06.0281838Z     @User2,lf
2025-04-25T04:10:06.0282342Z     @User3,split_build
2025-04-25T04:10:06.0282900Z """
2025-04-25T04:10:06.0283334Z 
2025-04-25T04:10:06.0283733Z import json
2025-04-25T04:10:06.0284307Z import logging
2025-04-25T04:10:06.0284777Z import os
2025-04-25T04:10:06.0285266Z import random
2025-04-25T04:10:06.0285807Z import re
2025-04-25T04:10:06.0286259Z import sys
2025-04-25T04:10:06.0286818Z from argparse import ArgumentParser
2025-04-25T04:10:06.0287465Z from collections.abc import Iterable
2025-04-25T04:10:06.0288105Z from functools import cache
2025-04-25T04:10:06.0288663Z from logging import LogRecord
2025-04-25T04:10:06.0289332Z from typing import Any, NamedTuple
2025-04-25T04:10:06.0290036Z from urllib.request import Request, urlopen
2025-04-25T04:10:06.0290760Z 
2025-04-25T04:10:06.0291284Z import yaml
2025-04-25T04:10:06.0291797Z from github import Auth, Github
2025-04-25T04:10:06.0292465Z from github.Issue import Issue
2025-04-25T04:10:06.0293082Z 
2025-04-25T04:10:06.0293503Z 
2025-04-25T04:10:06.0294140Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2025-04-25T04:10:06.0294906Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2025-04-25T04:10:06.0295948Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2025-04-25T04:10:06.0296743Z 
2025-04-25T04:10:06.0297224Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2025-04-25T04:10:06.0297958Z GH_OUTPUT_KEY_AMI = "runner-ami"
2025-04-25T04:10:06.0298569Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2025-04-25T04:10:06.0299307Z OPT_OUT_LABEL = "no-runner-experiments"
2025-04-25T04:10:06.0299948Z 
2025-04-25T04:10:06.0300716Z SETTING_EXPERIMENTS = "experiments"
2025-04-25T04:10:06.0301276Z 
2025-04-25T04:10:06.0301798Z LF_FLEET_EXPERIMENT = "lf"
2025-04-25T04:10:06.0302417Z CANARY_FLEET_SUFFIX = ".c"
2025-04-25T04:10:06.0302939Z 
2025-04-25T04:10:06.0303418Z 
2025-04-25T04:10:06.0303858Z class Experiment(NamedTuple):
2025-04-25T04:10:06.0304987Z     rollout_perc: float = (
2025-04-25T04:10:06.0305797Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2025-04-25T04:10:06.0306672Z     )
2025-04-25T04:10:06.0307154Z     all_branches: bool = (
2025-04-25T04:10:06.0307951Z         False  # If True, the experiment is also enabled on the exception branches
2025-04-25T04:10:06.0308728Z     )
2025-04-25T04:10:06.0309168Z     default: bool = (
2025-04-25T04:10:06.0309969Z         True  # If True, the experiment is enabled by default for all queries
2025-04-25T04:10:06.0310969Z     )
2025-04-25T04:10:06.0311389Z 
2025-04-25T04:10:06.0311889Z     # Add more fields as needed
2025-04-25T04:10:06.0312445Z 
2025-04-25T04:10:06.0312871Z 
2025-04-25T04:10:06.0313293Z class Settings(NamedTuple):
2025-04-25T04:10:06.0313950Z     """
2025-04-25T04:10:06.0314508Z     Settings for the experiments that can be opted into.
2025-04-25T04:10:06.0315175Z     """
2025-04-25T04:10:06.0315696Z 
2025-04-25T04:10:06.0316164Z     experiments: dict[str, Experiment] = {}
2025-04-25T04:10:06.0316762Z 
2025-04-25T04:10:06.0317399Z 
2025-04-25T04:10:06.0317924Z class ColorFormatter(logging.Formatter):
2025-04-25T04:10:06.0318638Z     """Color codes the log messages based on the log level"""
2025-04-25T04:10:06.0319373Z 
2025-04-25T04:10:06.0319787Z     COLORS = {
2025-04-25T04:10:06.0320456Z         "WARNING": "\033[33m",  # Yellow
2025-04-25T04:10:06.0379311Z         "ERROR": "\033[31m",  # Red
2025-04-25T04:10:06.0380461Z         "CRITICAL": "\033[31m",  # Red
2025-04-25T04:10:06.0381405Z         "INFO": "\033[0m",  # Reset
2025-04-25T04:10:06.0382333Z         "DEBUG": "\033[0m",  # Reset
2025-04-25T04:10:06.0383244Z     }
2025-04-25T04:10:06.0383888Z 
2025-04-25T04:10:06.0384496Z     def format(self, record: LogRecord) -> str:
2025-04-25T04:10:06.0385287Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2025-04-25T04:10:06.0386067Z         record.msg = f"{log_color}{record.msg}\033[0m"
2025-04-25T04:10:06.0386660Z         return super().format(record)
2025-04-25T04:10:06.0387148Z 
2025-04-25T04:10:06.0387481Z 
2025-04-25T04:10:06.0387880Z handler = logging.StreamHandler()
2025-04-25T04:10:06.0388618Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2025-04-25T04:10:06.0389338Z 
2025-04-25T04:10:06.0389810Z log = logging.getLogger(os.path.basename(__file__))
2025-04-25T04:10:06.0390829Z log.addHandler(handler)
2025-04-25T04:10:06.0391310Z log.setLevel(logging.INFO)
2025-04-25T04:10:06.0391781Z 
2025-04-25T04:10:06.0392131Z 
2025-04-25T04:10:06.0392605Z def set_github_output(key: str, value: str) -> None:
2025-04-25T04:10:06.0393168Z     """
2025-04-25T04:10:06.0393691Z     Defines outputs of the github action that invokes this script
2025-04-25T04:10:06.0394320Z     """
2025-04-25T04:10:06.0394708Z     if not GITHUB_OUTPUT:
2025-04-25T04:10:06.0395783Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2025-04-25T04:10:06.0396860Z         log.warning(
2025-04-25T04:10:06.0397718Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2025-04-25T04:10:06.0398602Z         )
2025-04-25T04:10:06.0399057Z         print(f"::set-output name={key}::{value}")
2025-04-25T04:10:06.0399599Z         return
2025-04-25T04:10:06.0399990Z 
2025-04-25T04:10:06.0400789Z     with open(GITHUB_OUTPUT, "a") as f:
2025-04-25T04:10:06.0401407Z         log.info(f"Setting output: {key}='{value}'")
2025-04-25T04:10:06.0401981Z         f.write(f"{key}={value}\n")
2025-04-25T04:10:06.0402472Z 
2025-04-25T04:10:06.0402811Z 
2025-04-25T04:10:06.0403320Z def _str_comma_separated_to_set(value: str) -> frozenset[str]:
2025-04-25T04:10:06.0403968Z     return frozenset(
2025-04-25T04:10:06.0404619Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2025-04-25T04:10:06.0405280Z     )
2025-04-25T04:10:06.0405627Z 
2025-04-25T04:10:06.0405960Z 
2025-04-25T04:10:06.0406330Z def parse_args() -> Any:
2025-04-25T04:10:06.0406922Z     parser = ArgumentParser("Get dynamic rollout settings")
2025-04-25T04:10:06.0407780Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2025-04-25T04:10:06.0408532Z     parser.add_argument(
2025-04-25T04:10:06.0409024Z         "--github-issue-repo",
2025-04-25T04:10:06.0409506Z         type=str,
2025-04-25T04:10:06.0409958Z         required=False,
2025-04-25T04:10:06.0410850Z         default="pytorch/test-infra",
2025-04-25T04:10:06.0411421Z         help="GitHub repo to get the issue",
2025-04-25T04:10:06.0411916Z     )
2025-04-25T04:10:06.0412297Z     parser.add_argument(
2025-04-25T04:10:06.0412757Z         "--github-repo",
2025-04-25T04:10:06.0413211Z         type=str,
2025-04-25T04:10:06.0413633Z         required=True,
2025-04-25T04:10:06.0414146Z         help="GitHub repo where CI is running",
2025-04-25T04:10:06.0414666Z     )
2025-04-25T04:10:06.0415048Z     parser.add_argument(
2025-04-25T04:10:06.0415696Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2025-04-25T04:10:06.0416342Z     )
2025-04-25T04:10:06.0416715Z     parser.add_argument(
2025-04-25T04:10:06.0417374Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2025-04-25T04:10:06.0418040Z     )
2025-04-25T04:10:06.0418415Z     parser.add_argument(
2025-04-25T04:10:06.0419077Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2025-04-25T04:10:06.0419754Z     )
2025-04-25T04:10:06.0420291Z     parser.add_argument(
2025-04-25T04:10:06.0420990Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2025-04-25T04:10:06.0421732Z     )
2025-04-25T04:10:06.0422106Z     parser.add_argument(
2025-04-25T04:10:06.0422584Z         "--github-ref-type",
2025-04-25T04:10:06.0423050Z         type=str,
2025-04-25T04:10:06.0423477Z         required=True,
2025-04-25T04:10:06.0424005Z         help="Current GitHub ref type, branch or tag",
2025-04-25T04:10:06.0424554Z     )
2025-04-25T04:10:06.0424931Z     parser.add_argument(
2025-04-25T04:10:06.0425427Z         "--eligible-experiments",
2025-04-25T04:10:06.0425975Z         type=_str_comma_separated_to_set,
2025-04-25T04:10:06.0426502Z         required=False,
2025-04-25T04:10:06.0426970Z         default="",
2025-04-25T04:10:06.0427837Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2025-04-25T04:10:06.0428749Z     )
2025-04-25T04:10:06.0429160Z     parser.add_argument(
2025-04-25T04:10:06.0429643Z         "--pr-number",
2025-04-25T04:10:06.0430089Z         type=str,
2025-04-25T04:10:06.0431042Z         required=False,
2025-04-25T04:10:06.0431491Z         default="",
2025-04-25T04:10:06.0432165Z         help="the optional PR number where this is run",
2025-04-25T04:10:06.0432713Z     )
2025-04-25T04:10:06.0433085Z 
2025-04-25T04:10:06.0433447Z     return parser.parse_args()
2025-04-25T04:10:06.0433966Z 
2025-04-25T04:10:06.0434301Z 
2025-04-25T04:10:06.0434915Z def get_gh_client(github_token: str) -> Github:  # type: ignore[no-any-unimported]
2025-04-25T04:10:06.0435673Z     auth = Auth.Token(github_token)
2025-04-25T04:10:06.0436191Z     return Github(auth=auth)
2025-04-25T04:10:06.0436658Z 
2025-04-25T04:10:06.0437044Z 
2025-04-25T04:10:06.0437708Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:  # type: ignore[no-any-unimported]
2025-04-25T04:10:06.0438486Z     repo = gh.get_repo(repo)
2025-04-25T04:10:06.0439021Z     return repo.get_issue(number=issue_num)
2025-04-25T04:10:06.0439539Z 
2025-04-25T04:10:06.0439874Z 
2025-04-25T04:10:06.0440422Z def get_potential_pr_author(
2025-04-25T04:10:06.0441102Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2025-04-25T04:10:06.0441772Z ) -> str:
2025-04-25T04:10:06.0442434Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2025-04-25T04:10:06.0443244Z     # Fetch the actual username from the original PR. The PR number is
2025-04-25T04:10:06.0444006Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2025-04-25T04:10:06.0444564Z 
2025-04-25T04:10:06.0444949Z     gh = get_gh_client(github_token)
2025-04-25T04:10:06.0445432Z 
2025-04-25T04:10:06.0445890Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2025-04-25T04:10:06.0446513Z         split_tag = ref_name.split("/")
2025-04-25T04:10:06.0447025Z         if (
2025-04-25T04:10:06.0447428Z             len(split_tag) == 3
2025-04-25T04:10:06.0447935Z             and split_tag[0] == "ciflow"
2025-04-25T04:10:06.0448485Z             and split_tag[2].isnumeric()
2025-04-25T04:10:06.0448976Z         ):
2025-04-25T04:10:06.0449399Z             pr_number = split_tag[2]
2025-04-25T04:10:06.0449892Z             try:
2025-04-25T04:10:06.0450521Z                 repository = gh.get_repo(repo)
2025-04-25T04:10:06.0451140Z                 pull = repository.get_pull(number=int(pr_number))
2025-04-25T04:10:06.0451746Z             except Exception as e:
2025-04-25T04:10:06.0452285Z                 raise Exception(  # noqa: TRY002
2025-04-25T04:10:06.0452960Z                     f"issue with pull request {pr_number} from repo {repository}"
2025-04-25T04:10:06.0453617Z                 ) from e
2025-04-25T04:10:06.0454180Z             return pull.user.login  # type: ignore[no-any-return]
2025-04-25T04:10:06.0454887Z     # In all other cases, return the original input username
2025-04-25T04:10:06.0455465Z     return username
2025-04-25T04:10:06.0455872Z 
2025-04-25T04:10:06.0456203Z 
2025-04-25T04:10:06.0456621Z def is_exception_branch(branch: str) -> bool:
2025-04-25T04:10:06.0457179Z     """
2025-04-25T04:10:06.0457832Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2025-04-25T04:10:06.0458584Z     """
2025-04-25T04:10:06.0459161Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2025-04-25T04:10:06.0459804Z 
2025-04-25T04:10:06.0460316Z 
2025-04-25T04:10:06.0460704Z def load_yaml(yaml_text: str) -> Any:
2025-04-25T04:10:06.0461206Z     try:
2025-04-25T04:10:06.0461608Z         data = yaml.safe_load(yaml_text)
2025-04-25T04:10:06.0462125Z         return data
2025-04-25T04:10:06.0462701Z     except yaml.YAMLError:
2025-04-25T04:10:06.0463225Z         log.exception("Error loading YAML")
2025-04-25T04:10:06.0463740Z         raise
2025-04-25T04:10:06.0464121Z 
2025-04-25T04:10:06.0464452Z 
2025-04-25T04:10:06.0465061Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> tuple[str, str]:
2025-04-25T04:10:06.0465780Z     """
2025-04-25T04:10:06.0466411Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2025-04-25T04:10:06.0467136Z 
2025-04-25T04:10:06.0467661Z     If the issue body contains "---" then the text above that is the settings
2025-04-25T04:10:06.0468403Z     and the text below is the list of opted in users.
2025-04-25T04:10:06.0468979Z 
2025-04-25T04:10:06.0469528Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2025-04-25T04:10:06.0470385Z     """
2025-04-25T04:10:06.0470870Z     rollout_state_parts = rollout_state.split("---")
2025-04-25T04:10:06.0471469Z     if len(rollout_state_parts) >= 2:
2025-04-25T04:10:06.0472204Z         return rollout_state_parts[0], rollout_state_parts[1]
2025-04-25T04:10:06.0472793Z     else:
2025-04-25T04:10:06.0473198Z         return "", rollout_state
2025-04-25T04:10:06.0473657Z 
2025-04-25T04:10:06.0473981Z 
2025-04-25T04:10:06.0474371Z class UserOptins(dict[str, list[str]]):
2025-04-25T04:10:06.0474877Z     """
2025-04-25T04:10:06.0475414Z     Dictionary of users with a list of features they have opted into
2025-04-25T04:10:06.0476047Z     """
2025-04-25T04:10:06.0476396Z 
2025-04-25T04:10:06.0476724Z 
2025-04-25T04:10:06.0477258Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2025-04-25T04:10:06.0477902Z     """
2025-04-25T04:10:06.0478616Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2025-04-25T04:10:06.0479412Z 
2025-04-25T04:10:06.0480369Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2025-04-25T04:10:06.0481347Z         - Example line: "@User1,lf,split_build"
2025-04-25T04:10:06.0482017Z         - A "#" prefix indicates the user is opted out of all experiments
2025-04-25T04:10:06.0482628Z 
2025-04-25T04:10:06.0482954Z 
2025-04-25T04:10:06.0483293Z     """
2025-04-25T04:10:06.0483669Z     optins = UserOptins()
2025-04-25T04:10:06.0484185Z     for user in user_optin_text.split("\n"):
2025-04-25T04:10:06.0484741Z         user = user.strip("\r\n\t -")
2025-04-25T04:10:06.0485297Z         if not user or not user.startswith("@"):
2025-04-25T04:10:06.0485857Z             # Not a valid user. Skip
2025-04-25T04:10:06.0486343Z             continue
2025-04-25T04:10:06.0486758Z 
2025-04-25T04:10:06.0487097Z         if user:
2025-04-25T04:10:06.0487573Z             usr_name = user.split(",")[0].strip("@")
2025-04-25T04:10:06.0488271Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2025-04-25T04:10:06.0488906Z 
2025-04-25T04:10:06.0489257Z     return optins
2025-04-25T04:10:06.0489658Z 
2025-04-25T04:10:06.0489984Z 
2025-04-25T04:10:06.0490613Z def is_valid_experiment_name(experiment_name: str) -> bool:
2025-04-25T04:10:06.0491216Z     """
2025-04-25T04:10:06.0491638Z     Check if the experiment name is valid.
2025-04-25T04:10:06.0492151Z     A valid name:
2025-04-25T04:10:06.0492811Z         - Contains only alphanumeric characters and the special characters "_" & "-"
2025-04-25T04:10:06.0493895Z         - The special characters "_" & "-" shouldn't be the first or last characters
2025-04-25T04:10:06.0494588Z         - Cannot contain spaces
2025-04-25T04:10:06.0495048Z     """
2025-04-25T04:10:06.0495398Z 
2025-04-25T04:10:06.0495844Z     valid_char_regex = r"^[a-zA-Z0-9]([\w-]*[a-zA-Z0-9])?$"
2025-04-25T04:10:06.0496538Z     valid = bool(re.match(valid_char_regex, experiment_name))
2025-04-25T04:10:06.0497101Z 
2025-04-25T04:10:06.0497431Z     if valid:
2025-04-25T04:10:06.0497818Z         return True
2025-04-25T04:10:06.0498212Z 
2025-04-25T04:10:06.0498544Z     log.error(
2025-04-25T04:10:06.0499937Z         f"Invalid experiment name: {experiment_name}. Experiment names should only contain alphanumeric characters, '_', and '-'. They cannot contain spaces, and the special characters '_' and '-' cannot be the first or last characters."
2025-04-25T04:10:06.0501596Z     )
2025-04-25T04:10:06.0501958Z     return False
2025-04-25T04:10:06.0502347Z 
2025-04-25T04:10:06.0502661Z 
2025-04-25T04:10:06.0503291Z def parse_settings_from_text(settings_text: str) -> Settings:
2025-04-25T04:10:06.0503919Z     """
2025-04-25T04:10:06.0504511Z     Parse the experiments from the issue body into a list of ExperimentSettings
2025-04-25T04:10:06.0505198Z     """
2025-04-25T04:10:06.0505545Z     try:
2025-04-25T04:10:06.0505920Z         if settings_text:
2025-04-25T04:10:06.0506647Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2025-04-25T04:10:06.0507406Z             # for easy reading
2025-04-25T04:10:06.0508208Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2025-04-25T04:10:06.0509068Z             #       the backtick character in shell commands.
2025-04-25T04:10:06.0509666Z             backtick = chr(96)  # backtick character
2025-04-25T04:10:06.0510882Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2025-04-25T04:10:06.0511559Z             settings = load_yaml(settings_text)
2025-04-25T04:10:06.0512065Z 
2025-04-25T04:10:06.0512634Z             # For now we just load experiments. We can expand this if/when we add more settings
2025-04-25T04:10:06.0513361Z             experiments = {}
2025-04-25T04:10:06.0513804Z 
2025-04-25T04:10:06.0514392Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2025-04-25T04:10:06.0515146Z                 if not is_valid_experiment_name(exp_name):
2025-04-25T04:10:06.0516394Z                     # Exclude invalid experiments from the list. We log an error, but don't raise an exception so that other experiments can still be processed.
2025-04-25T04:10:06.0517450Z                     continue
2025-04-25T04:10:06.0517891Z 
2025-04-25T04:10:06.0518259Z                 valid_settings = {}
2025-04-25T04:10:06.0518794Z                 for setting in exp_settings:
2025-04-25T04:10:06.0519357Z                     if setting not in Experiment._fields:
2025-04-25T04:10:06.0519906Z                         log.warning(
2025-04-25T04:10:06.0520801Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2025-04-25T04:10:06.0521496Z                         )
2025-04-25T04:10:06.0521939Z                     else:
2025-04-25T04:10:06.0522471Z                         valid_settings[setting] = exp_settings[setting]
2025-04-25T04:10:06.0523040Z 
2025-04-25T04:10:06.0523499Z                 experiments[exp_name] = Experiment(**valid_settings)
2025-04-25T04:10:06.0524285Z             return Settings(experiments)
2025-04-25T04:10:06.0524765Z 
2025-04-25T04:10:06.0525118Z     except Exception:
2025-04-25T04:10:06.0525622Z         log.exception("Failed to parse settings")
2025-04-25T04:10:06.0526146Z 
2025-04-25T04:10:06.0526498Z     return Settings()
2025-04-25T04:10:06.0526906Z 
2025-04-25T04:10:06.0527230Z 
2025-04-25T04:10:06.0527675Z def parse_settings(rollout_state: str) -> Settings:
2025-04-25T04:10:06.0528247Z     """
2025-04-25T04:10:06.0528695Z     Parse settings, if any, from the rollout state.
2025-04-25T04:10:06.0529233Z 
2025-04-25T04:10:06.0529755Z     If the issue body contains "---" then the text above that is the settings
2025-04-25T04:10:06.0530619Z     and the text below is the list of opted in users.
2025-04-25T04:10:06.0531164Z 
2025-04-25T04:10:06.0531753Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2025-04-25T04:10:06.0532462Z     """
2025-04-25T04:10:06.0533023Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2025-04-25T04:10:06.0533894Z     return parse_settings_from_text(settings_text)
2025-04-25T04:10:06.0534436Z 
2025-04-25T04:10:06.0534755Z 
2025-04-25T04:10:06.0535185Z def parse_users(rollout_state: str) -> UserOptins:
2025-04-25T04:10:06.0535728Z     """
2025-04-25T04:10:06.0536130Z     Parse users from the rollout state.
2025-04-25T04:10:06.0536615Z 
2025-04-25T04:10:06.0536937Z     """
2025-04-25T04:10:06.0537476Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2025-04-25T04:10:06.0538202Z     return parse_user_opt_in_from_text(users_text)
2025-04-25T04:10:06.0538727Z 
2025-04-25T04:10:06.0539047Z 
2025-04-25T04:10:06.0539649Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-04-25T04:10:06.0540512Z     """
2025-04-25T04:10:06.0540956Z     Check if a user is opted into an experiment
2025-04-25T04:10:06.0541479Z     """
2025-04-25T04:10:06.0541969Z     return experiment_name in user_optins.get(user, [])
2025-04-25T04:10:06.0542524Z 
2025-04-25T04:10:06.0542873Z 
2025-04-25T04:10:06.0543483Z def is_user_opted_out(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-04-25T04:10:06.0544205Z     """
2025-04-25T04:10:06.0544680Z     Check if a user explicitly opted out of an experiment
2025-04-25T04:10:06.0545231Z     """
2025-04-25T04:10:06.0545751Z     # if the experiment is prefixed with a "-", then it's an opt-out
2025-04-25T04:10:06.0546431Z     experiment_optout = "-" + experiment_name
2025-04-25T04:10:06.0547077Z     if experiment_optout not in user_optins.get(user, []):
2025-04-25T04:10:06.0547661Z         return False
2025-04-25T04:10:06.0548060Z 
2025-04-25T04:10:06.0548522Z     if is_user_opted_in(user, user_optins, experiment_name):
2025-04-25T04:10:06.0549098Z         log.warning(
2025-04-25T04:10:06.0549892Z             f"User {user} is opted into experiment {experiment_name}, but also opted out of it. Defaulting to opting out"
2025-04-25T04:10:06.0550934Z         )
2025-04-25T04:10:06.0551298Z 
2025-04-25T04:10:06.0551643Z     return True
2025-04-25T04:10:06.0552034Z 
2025-04-25T04:10:06.0552364Z 
2025-04-25T04:10:06.0552714Z def get_runner_prefix(
2025-04-25T04:10:06.0553169Z     rollout_state: str,
2025-04-25T04:10:06.0553652Z     workflow_requestors: Iterable[str],
2025-04-25T04:10:06.0554206Z     branch: str,
2025-04-25T04:10:06.0554857Z     eligible_experiments: frozenset[str] = frozenset(),
2025-04-25T04:10:06.0555446Z     is_canary: bool = False,
2025-04-25T04:10:06.0555907Z ) -> str:
2025-04-25T04:10:06.0556342Z     settings = parse_settings(rollout_state)
2025-04-25T04:10:06.0556919Z     user_optins = parse_users(rollout_state)
2025-04-25T04:10:06.0557413Z 
2025-04-25T04:10:06.0557760Z     fleet_prefix = ""
2025-04-25T04:10:06.0558197Z     prefixes = []
2025-04-25T04:10:06.0558845Z     for experiment_name, experiment_settings in settings.experiments.items():
2025-04-25T04:10:06.0559758Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2025-04-25T04:10:06.0560565Z             log.info(
2025-04-25T04:10:06.0561252Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2025-04-25T04:10:06.0561972Z             )
2025-04-25T04:10:06.0562365Z             continue
2025-04-25T04:10:06.0562767Z 
2025-04-25T04:10:06.0563131Z         if eligible_experiments:
2025-04-25T04:10:06.0563826Z             if experiment_name not in eligible_experiments:
2025-04-25T04:10:06.0564452Z                 exp_list = ", ".join(eligible_experiments)
2025-04-25T04:10:06.0564992Z                 log.info(
2025-04-25T04:10:06.0565771Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2025-04-25T04:10:06.0566566Z                 )
2025-04-25T04:10:06.0566977Z                 continue
2025-04-25T04:10:06.0567470Z         elif not experiment_settings.default:
2025-04-25T04:10:06.0567987Z             log.info(
2025-04-25T04:10:06.0568682Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2025-04-25T04:10:06.0569398Z             )
2025-04-25T04:10:06.0569789Z             continue
2025-04-25T04:10:06.0570298Z 
2025-04-25T04:10:06.0570757Z         # Is any workflow_requestor opted out to this experiment?
2025-04-25T04:10:06.0571370Z         opted_out_users = [
2025-04-25T04:10:06.0571840Z             requestor
2025-04-25T04:10:06.0572323Z             for requestor in workflow_requestors
2025-04-25T04:10:06.0573008Z             if is_user_opted_out(requestor, user_optins, experiment_name)
2025-04-25T04:10:06.0573608Z         ]
2025-04-25T04:10:06.0573970Z 
2025-04-25T04:10:06.0574321Z         if opted_out_users:
2025-04-25T04:10:06.0574787Z             log.info(
2025-04-25T04:10:06.0575421Z                 f"{', '.join(opted_out_users)} have opted out of experiment {experiment_name}."
2025-04-25T04:10:06.0576090Z             )
2025-04-25T04:10:06.0576486Z             continue
2025-04-25T04:10:06.0576897Z 
2025-04-25T04:10:06.0577357Z         # Is any workflow_requestor opted in to this experiment?
2025-04-25T04:10:06.0577944Z         opted_in_users = [
2025-04-25T04:10:06.0578408Z             requestor
2025-04-25T04:10:06.0578886Z             for requestor in workflow_requestors
2025-04-25T04:10:06.0579552Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2025-04-25T04:10:06.0580368Z         ]
2025-04-25T04:10:06.0580742Z 
2025-04-25T04:10:06.0581099Z         enabled = False
2025-04-25T04:10:06.0581550Z         if opted_in_users:
2025-04-25T04:10:06.0582013Z             log.info(
2025-04-25T04:10:06.0582629Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2025-04-25T04:10:06.0583284Z             )
2025-04-25T04:10:06.0583864Z             enabled = True
2025-04-25T04:10:06.0584292Z 
2025-04-25T04:10:06.0584695Z         elif experiment_settings.rollout_perc:
2025-04-25T04:10:06.0585502Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2025-04-25T04:10:06.0586414Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2025-04-25T04:10:06.0587037Z                 log.info(
2025-04-25T04:10:06.0587909Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2025-04-25T04:10:06.0588786Z                 )
2025-04-25T04:10:06.0589212Z                 enabled = True
2025-04-25T04:10:06.0589672Z 
2025-04-25T04:10:06.0590010Z         if enabled:
2025-04-25T04:10:06.0590680Z             label = experiment_name
2025-04-25T04:10:06.0591233Z             if experiment_name == LF_FLEET_EXPERIMENT:
2025-04-25T04:10:06.0592046Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2025-04-25T04:10:06.0593016Z                 #  - If it's enabled, then we always list it's prefix first
2025-04-25T04:10:06.0593753Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2025-04-25T04:10:06.0594390Z                 if is_canary:
2025-04-25T04:10:06.0594890Z                     label += CANARY_FLEET_SUFFIX
2025-04-25T04:10:06.0595427Z                 fleet_prefix = label
2025-04-25T04:10:06.0595912Z             else:
2025-04-25T04:10:06.0596350Z                 prefixes.append(label)
2025-04-25T04:10:06.0596832Z 
2025-04-25T04:10:06.0597181Z     if len(prefixes) > 1:
2025-04-25T04:10:06.0597645Z         log.error(
2025-04-25T04:10:06.0598660Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2025-04-25T04:10:06.0599707Z         )
2025-04-25T04:10:06.0600223Z         prefixes = prefixes[:1]
2025-04-25T04:10:06.0600693Z 
2025-04-25T04:10:06.0601054Z     # Fleet always comes first
2025-04-25T04:10:06.0601532Z     if fleet_prefix:
2025-04-25T04:10:06.0602077Z         prefixes.insert(0, fleet_prefix)
2025-04-25T04:10:06.0602891Z 
2025-04-25T04:10:06.0603689Z     return ".".join(prefixes) + "." if prefixes else ""
2025-04-25T04:10:06.0604706Z 
2025-04-25T04:10:06.0605271Z 
2025-04-25T04:10:06.0606454Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2025-04-25T04:10:06.0607841Z     """
2025-04-25T04:10:06.0608930Z     Gets the first comment of the issue, which contains the desired rollout state.
2025-04-25T04:10:06.0610397Z 
2025-04-25T04:10:06.0611373Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2025-04-25T04:10:06.0612728Z     """
2025-04-25T04:10:06.0613442Z     gh = get_gh_client(github_token)
2025-04-25T04:10:06.0614364Z     issue = get_issue(gh, repo, issue_num)
2025-04-25T04:10:06.0615587Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2025-04-25T04:10:06.0616647Z 
2025-04-25T04:10:06.0617300Z 
2025-04-25T04:10:06.0618354Z def download_json(url: str, headers: dict[str, str], num_retries: int = 3) -> Any:
2025-04-25T04:10:06.0619750Z     for _ in range(num_retries):
2025-04-25T04:10:06.0620956Z         try:
2025-04-25T04:10:06.0621722Z             req = Request(url=url, headers=headers)
2025-04-25T04:10:06.0622894Z             content = urlopen(req, timeout=5).read().decode("utf-8")
2025-04-25T04:10:06.0624342Z             return json.loads(content)
2025-04-25T04:10:06.0625256Z         except Exception as e:
2025-04-25T04:10:06.0626219Z             log.warning(f"Could not download {url}: {e}")
2025-04-25T04:10:06.0627170Z 
2025-04-25T04:10:06.0628163Z     log.warning(f"All {num_retries} retries exhausted, downloading {url} failed")
2025-04-25T04:10:06.0629395Z     return {}
2025-04-25T04:10:06.0630083Z 
2025-04-25T04:10:06.0630898Z 
2025-04-25T04:10:06.0631494Z @cache
2025-04-25T04:10:06.0632560Z def get_pr_info(github_repo: str, github_token: str, pr_number: int) -> dict[str, Any]:
2025-04-25T04:10:06.0633743Z     """
2025-04-25T04:10:06.0634386Z     Dynamically get PR information
2025-04-25T04:10:06.0635197Z     """
2025-04-25T04:10:06.0636030Z     github_api = f"https://api.github.com/repos/{github_repo}"
2025-04-25T04:10:06.0637083Z     headers = {
2025-04-25T04:10:06.0637897Z         "Accept": "application/vnd.github.v3+json",
2025-04-25T04:10:06.0638814Z         "Authorization": f"token {github_token}",
2025-04-25T04:10:06.0639715Z     }
2025-04-25T04:10:06.0640995Z     json_response: dict[str, Any] = download_json(
2025-04-25T04:10:06.0642101Z         url=f"{github_api}/issues/{pr_number}",
2025-04-25T04:10:06.0643097Z         headers=headers,
2025-04-25T04:10:06.0643886Z     )
2025-04-25T04:10:06.0644494Z 
2025-04-25T04:10:06.0645108Z     if not json_response:
2025-04-25T04:10:06.0646148Z         log.warning(f"Failed to get the labels for #{pr_number}")
2025-04-25T04:10:06.0647305Z         return {}
2025-04-25T04:10:06.0648042Z 
2025-04-25T04:10:06.0648669Z     return json_response
2025-04-25T04:10:06.0649498Z 
2025-04-25T04:10:06.0650074Z 
2025-04-25T04:10:06.0651315Z def get_labels(github_repo: str, github_token: str, pr_number: int) -> set[str]:
2025-04-25T04:10:06.0652527Z     """
2025-04-25T04:10:06.0653496Z     Dynamically get the latest list of labels from the pull request
2025-04-25T04:10:06.0654747Z     """
2025-04-25T04:10:06.0655618Z     pr_info = get_pr_info(github_repo, github_token, pr_number)
2025-04-25T04:10:06.0656635Z     return {
2025-04-25T04:10:06.0657592Z         label.get("name") for label in pr_info.get("labels", []) if label.get("name")
2025-04-25T04:10:06.0658699Z     }
2025-04-25T04:10:06.0659307Z 
2025-04-25T04:10:06.0659870Z 
2025-04-25T04:10:06.0660729Z def main() -> None:
2025-04-25T04:10:06.0661511Z     args = parse_args()
2025-04-25T04:10:06.0662277Z 
2025-04-25T04:10:06.0662997Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2025-04-25T04:10:06.0663801Z 
2025-04-25T04:10:06.0664359Z     # Check if the PR is opt-out
2025-04-25T04:10:06.0665160Z     if args.pr_number:
2025-04-25T04:10:06.0666288Z         labels = get_labels(args.github_repo, args.github_token, int(args.pr_number))
2025-04-25T04:10:06.0667560Z         if OPT_OUT_LABEL in labels:
2025-04-25T04:10:06.0668454Z             log.info(
2025-04-25T04:10:06.0669781Z                 f"Opt-out runner determinator because #{args.pr_number} has {OPT_OUT_LABEL} label"
2025-04-25T04:10:06.0671252Z             )
2025-04-25T04:10:06.0672220Z             set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-04-25T04:10:06.0673400Z             sys.exit()
2025-04-25T04:10:06.0674156Z 
2025-04-25T04:10:06.0674720Z     try:
2025-04-25T04:10:06.0675457Z         rollout_state = get_rollout_state_from_issue(
2025-04-25T04:10:06.0676645Z             args.github_token, args.github_issue_repo, args.github_issue
2025-04-25T04:10:06.0678053Z         )
2025-04-25T04:10:06.0678666Z 
2025-04-25T04:10:06.0679304Z         username = get_potential_pr_author(
2025-04-25T04:10:06.0680375Z             args.github_token,
2025-04-25T04:10:06.0681256Z             args.github_repo,
2025-04-25T04:10:06.0682144Z             args.github_actor,
2025-04-25T04:10:06.0682993Z             args.github_ref_type,
2025-04-25T04:10:06.0683831Z             args.github_branch,
2025-04-25T04:10:06.0684606Z         )
2025-04-25T04:10:06.0685251Z 
2025-04-25T04:10:06.0686045Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2025-04-25T04:10:06.0687035Z 
2025-04-25T04:10:06.0687720Z         runner_label_prefix = get_runner_prefix(
2025-04-25T04:10:06.0688714Z             rollout_state,
2025-04-25T04:10:06.0689567Z             (args.github_issue_owner, username),
2025-04-25T04:10:06.0690666Z             args.github_branch,
2025-04-25T04:10:06.0691234Z             args.eligible_experiments,
2025-04-25T04:10:06.0691740Z             is_canary,
2025-04-25T04:10:06.0692326Z         )
2025-04-25T04:10:06.0692694Z 
2025-04-25T04:10:06.0693054Z     except Exception as e:
2025-04-25T04:10:06.0693513Z         log.error(
2025-04-25T04:10:06.0694201Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2025-04-25T04:10:06.0694923Z         )
2025-04-25T04:10:06.0695280Z 
2025-04-25T04:10:06.0695799Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-04-25T04:10:06.0696423Z 
2025-04-25T04:10:06.0696746Z 
2025-04-25T04:10:06.0697092Z if __name__ == "__main__":
2025-04-25T04:10:06.0697541Z     main()
2025-04-25T04:10:06.0697908Z 
2025-04-25T04:10:06.0698244Z EOF
2025-04-25T04:10:06.0698588Z 
2025-04-25T04:10:06.0698948Z cat runner_determinator.py
2025-04-25T04:10:06.0911132Z shell: /usr/bin/bash -e {0}
2025-04-25T04:10:06.0911899Z env:
2025-04-25T04:10:06.0912538Z   GITHUB_TOKEN: ***
2025-04-25T04:10:06.0912945Z   ISSUE_NUMBER: 5132
2025-04-25T04:10:06.0913365Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2025-04-25T04:10:06.0913850Z   ISSUE_OWNER: 
2025-04-25T04:10:06.0914220Z   CHECK_EXPERIMENTS: 
2025-04-25T04:10:06.0914623Z   PR_NUMBER: 
2025-04-25T04:10:06.0914979Z ##[endgroup]
2025-04-25T04:10:06.1133736Z # flake8: noqa: G004
2025-04-25T04:10:06.1134078Z 
2025-04-25T04:10:06.1134504Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2025-04-25T04:10:06.1135428Z #       must be kept in sync. You can do it easily by running the following command:
2025-04-25T04:10:06.1136211Z #           python .github/scripts/update_runner_determinator.py
2025-04-25T04:10:06.1136671Z 
2025-04-25T04:10:06.1136825Z """
2025-04-25T04:10:06.1137382Z This runner determinator is used to determine which set of runners to run a
2025-04-25T04:10:06.1138243Z GitHub job on. It uses the first comment of a GitHub issue (by default
2025-04-25T04:10:06.1139143Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2025-04-25T04:10:06.1139943Z of which runners should be used to run which job.
2025-04-25T04:10:06.1140580Z 
2025-04-25T04:10:06.1140951Z The configuration has two parts, the settings and a list of opted-in users,
2025-04-25T04:10:06.1141818Z separated by a line containing "---".  If the line is not present, the
2025-04-25T04:10:06.1142660Z settings are considered to be empty with only the second part, the user
2025-04-25T04:10:06.1143323Z list, defined.
2025-04-25T04:10:06.1143559Z 
2025-04-25T04:10:06.1143905Z The first part is a YAML block that defines the rollout settings. This can be
2025-04-25T04:10:06.1144786Z used to define any settings that are needed to determine which runners to use.
2025-04-25T04:10:06.1145794Z It's fields are defined by the RolloutSettings class below.
2025-04-25T04:10:06.1146236Z 
2025-04-25T04:10:06.1146594Z The second part is a list of users who are explicitly opted in to the LF fleet.
2025-04-25T04:10:06.1147436Z The user list is also a comma separated list of additional features or
2025-04-25T04:10:06.1148159Z experiments which the user could be opted in to.
2025-04-25T04:10:06.1148553Z 
2025-04-25T04:10:06.1148741Z The user list has the following rules:
2025-04-25T04:10:06.1149092Z 
2025-04-25T04:10:06.1149387Z - Users are GitHub usernames, which must start with the @ prefix
2025-04-25T04:10:06.1150500Z - Each user is also a comma-separated list of features/experiments to enable
2025-04-25T04:10:06.1151819Z - A "#" prefix opts the user out of all experiments
2025-04-25T04:10:06.1152672Z 
2025-04-25T04:10:06.1153259Z Example config:
2025-04-25T04:10:06.1153974Z     # A list of experiments that can be opted into.
2025-04-25T04:10:06.1154672Z     # This defines the behavior they'll induce when opted into.
2025-04-25T04:10:06.1155281Z     # Expected syntax is:
2025-04-25T04:10:06.1155900Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2025-04-25T04:10:06.1157009Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2025-04-25T04:10:06.1157628Z 
2025-04-25T04:10:06.1157783Z     experiments:
2025-04-25T04:10:06.1158168Z       lf:
2025-04-25T04:10:06.1158529Z         rollout_percent: 25
2025-04-25T04:10:06.1158973Z         all_branches: false
2025-04-25T04:10:06.1159403Z         default: true
2025-04-25T04:10:06.1159800Z     ---
2025-04-25T04:10:06.1159998Z 
2025-04-25T04:10:06.1160443Z     # Opt-ins:
2025-04-25T04:10:06.1161020Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2025-04-25T04:10:06.1161859Z     # and specifying experiments to enable in a comma-separated list.
2025-04-25T04:10:06.1162610Z     # To always opt out of an experiment, prefix it with a "-".
2025-04-25T04:10:06.1163251Z     # Experiments should be from the above list.
2025-04-25T04:10:06.1163622Z 
2025-04-25T04:10:06.1163784Z     @User1,-lf,split_build
2025-04-25T04:10:06.1164209Z     @User2,lf
2025-04-25T04:10:06.1164587Z     @User3,split_build
2025-04-25T04:10:06.1164975Z """
2025-04-25T04:10:06.1165168Z 
2025-04-25T04:10:06.1165321Z import json
2025-04-25T04:10:06.1165671Z import logging
2025-04-25T04:10:06.1166039Z import os
2025-04-25T04:10:06.1166380Z import random
2025-04-25T04:10:06.1166744Z import re
2025-04-25T04:10:06.1167093Z import sys
2025-04-25T04:10:06.1167475Z from argparse import ArgumentParser
2025-04-25T04:10:06.1167979Z from collections.abc import Iterable
2025-04-25T04:10:06.1168496Z from functools import cache
2025-04-25T04:10:06.1168945Z from logging import LogRecord
2025-04-25T04:10:06.1169421Z from typing import Any, NamedTuple
2025-04-25T04:10:06.1169937Z from urllib.request import Request, urlopen
2025-04-25T04:10:06.1170555Z 
2025-04-25T04:10:06.1170712Z import yaml
2025-04-25T04:10:06.1171103Z from github import Auth, Github
2025-04-25T04:10:06.1171570Z from github.Issue import Issue
2025-04-25T04:10:06.1171872Z 
2025-04-25T04:10:06.1171878Z 
2025-04-25T04:10:06.1172085Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2025-04-25T04:10:06.1172746Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2025-04-25T04:10:06.1173589Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2025-04-25T04:10:06.1174126Z 
2025-04-25T04:10:06.1174343Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2025-04-25T04:10:06.1174903Z GH_OUTPUT_KEY_AMI = "runner-ami"
2025-04-25T04:10:06.1175399Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2025-04-25T04:10:06.1175933Z OPT_OUT_LABEL = "no-runner-experiments"
2025-04-25T04:10:06.1176287Z 
2025-04-25T04:10:06.1176471Z SETTING_EXPERIMENTS = "experiments"
2025-04-25T04:10:06.1176795Z 
2025-04-25T04:10:06.1176963Z LF_FLEET_EXPERIMENT = "lf"
2025-04-25T04:10:06.1177579Z CANARY_FLEET_SUFFIX = ".c"
2025-04-25T04:10:06.1177855Z 
2025-04-25T04:10:06.1177861Z 
2025-04-25T04:10:06.1178043Z class Experiment(NamedTuple):
2025-04-25T04:10:06.1178506Z     rollout_perc: float = (
2025-04-25T04:10:06.1179125Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2025-04-25T04:10:06.1179770Z     )
2025-04-25T04:10:06.1180324Z     all_branches: bool = (
2025-04-25T04:10:06.1180946Z         False  # If True, the experiment is also enabled on the exception branches
2025-04-25T04:10:06.1181593Z     )
2025-04-25T04:10:06.1181941Z     default: bool = (
2025-04-25T04:10:06.1182487Z         True  # If True, the experiment is enabled by default for all queries
2025-04-25T04:10:06.1183108Z     )
2025-04-25T04:10:06.1183302Z 
2025-04-25T04:10:06.1183473Z     # Add more fields as needed
2025-04-25T04:10:06.1183766Z 
2025-04-25T04:10:06.1183780Z 
2025-04-25T04:10:06.1183949Z class Settings(NamedTuple):
2025-04-25T04:10:06.1184366Z     """
2025-04-25T04:10:06.1184807Z     Settings for the experiments that can be opted into.
2025-04-25T04:10:06.1185365Z     """
2025-04-25T04:10:06.1185571Z 
2025-04-25T04:10:06.1185767Z     experiments: dict[str, Experiment] = {}
2025-04-25T04:10:06.1186133Z 
2025-04-25T04:10:06.1186140Z 
2025-04-25T04:10:06.1186468Z class ColorFormatter(logging.Formatter):
2025-04-25T04:10:06.1187088Z     """Color codes the log messages based on the log level"""
2025-04-25T04:10:06.1187525Z 
2025-04-25T04:10:06.1187675Z     COLORS = {
2025-04-25T04:10:06.1188059Z         "WARNING": "\033[33m",  # Yellow
2025-04-25T04:10:06.1188552Z         "ERROR": "\033[31m",  # Red
2025-04-25T04:10:06.1189029Z         "CRITICAL": "\033[31m",  # Red
2025-04-25T04:10:06.1189513Z         "INFO": "\033[0m",  # Reset
2025-04-25T04:10:06.1189985Z         "DEBUG": "\033[0m",  # Reset
2025-04-25T04:10:06.1190679Z     }
2025-04-25T04:10:06.1190882Z 
2025-04-25T04:10:06.1191091Z     def format(self, record: LogRecord) -> str:
2025-04-25T04:10:06.1191816Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2025-04-25T04:10:06.1192566Z         record.msg = f"{log_color}{record.msg}\033[0m"
2025-04-25T04:10:06.1193129Z         return super().format(record)
2025-04-25T04:10:06.1193469Z 
2025-04-25T04:10:06.1193475Z 
2025-04-25T04:10:06.1193663Z handler = logging.StreamHandler()
2025-04-25T04:10:06.1194348Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2025-04-25T04:10:06.1194883Z 
2025-04-25T04:10:06.1195112Z log = logging.getLogger(os.path.basename(__file__))
2025-04-25T04:10:06.1195677Z log.addHandler(handler)
2025-04-25T04:10:06.1196101Z log.setLevel(logging.INFO)
2025-04-25T04:10:06.1196394Z 
2025-04-25T04:10:06.1196401Z 
2025-04-25T04:10:06.1196635Z def set_github_output(key: str, value: str) -> None:
2025-04-25T04:10:06.1197173Z     """
2025-04-25T04:10:06.1197665Z     Defines outputs of the github action that invokes this script
2025-04-25T04:10:06.1198287Z     """
2025-04-25T04:10:06.1198645Z     if not GITHUB_OUTPUT:
2025-04-25T04:10:06.1199666Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2025-04-25T04:10:06.1200952Z         log.warning(
2025-04-25T04:10:06.1201769Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2025-04-25T04:10:06.1202629Z         )
2025-04-25T04:10:06.1212486Z         print(f"::set-output name={key}::{value}")
2025-04-25T04:10:06.1213086Z         return
2025-04-25T04:10:06.1213325Z 
2025-04-25T04:10:06.1213518Z     with open(GITHUB_OUTPUT, "a") as f:
2025-04-25T04:10:06.1214080Z         log.info(f"Setting output: {key}='{value}'")
2025-04-25T04:10:06.1214626Z         f.write(f"{key}={value}\n")
2025-04-25T04:10:06.1214948Z 
2025-04-25T04:10:06.1214960Z 
2025-04-25T04:10:06.1215243Z def _str_comma_separated_to_set(value: str) -> frozenset[str]:
2025-04-25T04:10:06.1215854Z     return frozenset(
2025-04-25T04:10:06.1216631Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2025-04-25T04:10:06.1217285Z     )
2025-04-25T04:10:06.1217491Z 
2025-04-25T04:10:06.1217498Z 
2025-04-25T04:10:06.1217670Z def parse_args() -> Any:
2025-04-25T04:10:06.1218214Z     parser = ArgumentParser("Get dynamic rollout settings")
2025-04-25T04:10:06.1219041Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2025-04-25T04:10:06.1219780Z     parser.add_argument(
2025-04-25T04:10:06.1220494Z         "--github-issue-repo",
2025-04-25T04:10:06.1220976Z         type=str,
2025-04-25T04:10:06.1221359Z         required=False,
2025-04-25T04:10:06.1221798Z         default="pytorch/test-infra",
2025-04-25T04:10:06.1222325Z         help="GitHub repo to get the issue",
2025-04-25T04:10:06.1222813Z     )
2025-04-25T04:10:06.1223175Z     parser.add_argument(
2025-04-25T04:10:06.1223600Z         "--github-repo",
2025-04-25T04:10:06.1224014Z         type=str,
2025-04-25T04:10:06.1224398Z         required=True,
2025-04-25T04:10:06.1224842Z         help="GitHub repo where CI is running",
2025-04-25T04:10:06.1225342Z     )
2025-04-25T04:10:06.1225702Z     parser.add_argument(
2025-04-25T04:10:06.1226291Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2025-04-25T04:10:06.1227085Z     )
2025-04-25T04:10:06.1227443Z     parser.add_argument(
2025-04-25T04:10:06.1228050Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2025-04-25T04:10:06.1228712Z     )
2025-04-25T04:10:06.1229069Z     parser.add_argument(
2025-04-25T04:10:06.1229673Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2025-04-25T04:10:06.1230575Z     )
2025-04-25T04:10:06.1230936Z     parser.add_argument(
2025-04-25T04:10:06.1231556Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2025-04-25T04:10:06.1232231Z     )
2025-04-25T04:10:06.1232575Z     parser.add_argument(
2025-04-25T04:10:06.1233008Z         "--github-ref-type",
2025-04-25T04:10:06.1233446Z         type=str,
2025-04-25T04:10:06.1233816Z         required=True,
2025-04-25T04:10:06.1234277Z         help="Current GitHub ref type, branch or tag",
2025-04-25T04:10:06.1234803Z     )
2025-04-25T04:10:06.1235162Z     parser.add_argument(
2025-04-25T04:10:06.1235596Z         "--eligible-experiments",
2025-04-25T04:10:06.1236086Z         type=_str_comma_separated_to_set,
2025-04-25T04:10:06.1236584Z         required=False,
2025-04-25T04:10:06.1236979Z         default="",
2025-04-25T04:10:06.1237783Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2025-04-25T04:10:06.1238667Z     )
2025-04-25T04:10:06.1239010Z     parser.add_argument(
2025-04-25T04:10:06.1239421Z         "--pr-number",
2025-04-25T04:10:06.1239901Z         type=str,
2025-04-25T04:10:06.1240391Z         required=False,
2025-04-25T04:10:06.1240793Z         default="",
2025-04-25T04:10:06.1241233Z         help="the optional PR number where this is run",
2025-04-25T04:10:06.1241768Z     )
2025-04-25T04:10:06.1241960Z 
2025-04-25T04:10:06.1242131Z     return parser.parse_args()
2025-04-25T04:10:06.1242436Z 
2025-04-25T04:10:06.1242442Z 
2025-04-25T04:10:06.1242819Z def get_gh_client(github_token: str) -> Github:  # type: ignore[no-any-unimported]
2025-04-25T04:10:06.1243544Z     auth = Auth.Token(github_token)
2025-04-25T04:10:06.1244020Z     return Github(auth=auth)
2025-04-25T04:10:06.1244307Z 
2025-04-25T04:10:06.1244313Z 
2025-04-25T04:10:06.1244739Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:  # type: ignore[no-any-unimported]
2025-04-25T04:10:06.1245503Z     repo = gh.get_repo(repo)
2025-04-25T04:10:06.1245980Z     return repo.get_issue(number=issue_num)
2025-04-25T04:10:06.1246334Z 
2025-04-25T04:10:06.1246340Z 
2025-04-25T04:10:06.1246517Z def get_potential_pr_author(
2025-04-25T04:10:06.1247125Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2025-04-25T04:10:06.1247984Z ) -> str:
2025-04-25T04:10:06.1248474Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2025-04-25T04:10:06.1249248Z     # Fetch the actual username from the original PR. The PR number is
2025-04-25T04:10:06.1249950Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2025-04-25T04:10:06.1250594Z 
2025-04-25T04:10:06.1250773Z     gh = get_gh_client(github_token)
2025-04-25T04:10:06.1251099Z 
2025-04-25T04:10:06.1251355Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2025-04-25T04:10:06.1251947Z         split_tag = ref_name.split("/")
2025-04-25T04:10:06.1252434Z         if (
2025-04-25T04:10:06.1252794Z             len(split_tag) == 3
2025-04-25T04:10:06.1253256Z             and split_tag[0] == "ciflow"
2025-04-25T04:10:06.1253765Z             and split_tag[2].isnumeric()
2025-04-25T04:10:06.1254250Z         ):
2025-04-25T04:10:06.1254611Z             pr_number = split_tag[2]
2025-04-25T04:10:06.1255083Z             try:
2025-04-25T04:10:06.1255483Z                 repository = gh.get_repo(repo)
2025-04-25T04:10:06.1256089Z                 pull = repository.get_pull(number=int(pr_number))
2025-04-25T04:10:06.1256666Z             except Exception as e:
2025-04-25T04:10:06.1257165Z                 raise Exception(  # noqa: TRY002
2025-04-25T04:10:06.1257945Z                     f"issue with pull request {pr_number} from repo {repository}"
2025-04-25T04:10:06.1258559Z                 ) from e
2025-04-25T04:10:06.1259073Z             return pull.user.login  # type: ignore[no-any-return]
2025-04-25T04:10:06.1259743Z     # In all other cases, return the original input username
2025-04-25T04:10:06.1260538Z     return username
2025-04-25T04:10:06.1260804Z 
2025-04-25T04:10:06.1260810Z 
2025-04-25T04:10:06.1261023Z def is_exception_branch(branch: str) -> bool:
2025-04-25T04:10:06.1261537Z     """
2025-04-25T04:10:06.1262147Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2025-04-25T04:10:06.1262877Z     """
2025-04-25T04:10:06.1263405Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2025-04-25T04:10:06.1263911Z 
2025-04-25T04:10:06.1263919Z 
2025-04-25T04:10:06.1264097Z def load_yaml(yaml_text: str) -> Any:
2025-04-25T04:10:06.1264577Z     try:
2025-04-25T04:10:06.1264946Z         data = yaml.safe_load(yaml_text)
2025-04-25T04:10:06.1265442Z         return data
2025-04-25T04:10:06.1265838Z     except yaml.YAMLError:
2025-04-25T04:10:06.1266293Z         log.exception("Error loading YAML")
2025-04-25T04:10:06.1266791Z         raise
2025-04-25T04:10:06.1267008Z 
2025-04-25T04:10:06.1267015Z 
2025-04-25T04:10:06.1267399Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> tuple[str, str]:
2025-04-25T04:10:06.1268115Z     """
2025-04-25T04:10:06.1268701Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2025-04-25T04:10:06.1269287Z 
2025-04-25T04:10:06.1269607Z     If the issue body contains "---" then the text above that is the settings
2025-04-25T04:10:06.1270551Z     and the text below is the list of opted in users.
2025-04-25T04:10:06.1270959Z 
2025-04-25T04:10:06.1271313Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2025-04-25T04:10:06.1271987Z     """
2025-04-25T04:10:06.1272405Z     rollout_state_parts = rollout_state.split("---")
2025-04-25T04:10:06.1272982Z     if len(rollout_state_parts) >= 2:
2025-04-25T04:10:06.1273553Z         return rollout_state_parts[0], rollout_state_parts[1]
2025-04-25T04:10:06.1274131Z     else:
2025-04-25T04:10:06.1274500Z         return "", rollout_state
2025-04-25T04:10:06.1274806Z 
2025-04-25T04:10:06.1274814Z 
2025-04-25T04:10:06.1274997Z class UserOptins(dict[str, list[str]]):
2025-04-25T04:10:06.1275480Z     """
2025-04-25T04:10:06.1275983Z     Dictionary of users with a list of features they have opted into
2025-04-25T04:10:06.1276602Z     """
2025-04-25T04:10:06.1276795Z 
2025-04-25T04:10:06.1276801Z 
2025-04-25T04:10:06.1277110Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2025-04-25T04:10:06.1323953Z     """
2025-04-25T04:10:06.1324968Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2025-04-25T04:10:06.1325655Z 
2025-04-25T04:10:06.1326264Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2025-04-25T04:10:06.1327267Z         - Example line: "@User1,lf,split_build"
2025-04-25T04:10:06.1327934Z         - A "#" prefix indicates the user is opted out of all experiments
2025-04-25T04:10:06.1328415Z 
2025-04-25T04:10:06.1328422Z 
2025-04-25T04:10:06.1328574Z     """
2025-04-25T04:10:06.1328927Z     optins = UserOptins()
2025-04-25T04:10:06.1329397Z     for user in user_optin_text.split("\n"):
2025-04-25T04:10:06.1329944Z         user = user.strip("\r\n\t -")
2025-04-25T04:10:06.1330749Z         if not user or not user.startswith("@"):
2025-04-25T04:10:06.1331285Z             # Not a valid user. Skip
2025-04-25T04:10:06.1331759Z             continue
2025-04-25T04:10:06.1332002Z 
2025-04-25T04:10:06.1332155Z         if user:
2025-04-25T04:10:06.1332567Z             usr_name = user.split(",")[0].strip("@")
2025-04-25T04:10:06.1333415Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2025-04-25T04:10:06.1333909Z 
2025-04-25T04:10:06.1334061Z     return optins
2025-04-25T04:10:06.1334301Z 
2025-04-25T04:10:06.1334308Z 
2025-04-25T04:10:06.1334570Z def is_valid_experiment_name(experiment_name: str) -> bool:
2025-04-25T04:10:06.1335150Z     """
2025-04-25T04:10:06.1335523Z     Check if the experiment name is valid.
2025-04-25T04:10:06.1336025Z     A valid name:
2025-04-25T04:10:06.1336622Z         - Contains only alphanumeric characters and the special characters "_" & "-"
2025-04-25T04:10:06.1337519Z         - The special characters "_" & "-" shouldn't be the first or last characters
2025-04-25T04:10:06.1338210Z         - Cannot contain spaces
2025-04-25T04:10:06.1338647Z     """
2025-04-25T04:10:06.1338853Z 
2025-04-25T04:10:06.1339099Z     valid_char_regex = r"^[a-zA-Z0-9]([\w-]*[a-zA-Z0-9])?$"
2025-04-25T04:10:06.1339771Z     valid = bool(re.match(valid_char_regex, experiment_name))
2025-04-25T04:10:06.1340494Z 
2025-04-25T04:10:06.1340671Z     if valid:
2025-04-25T04:10:06.1341044Z         return True
2025-04-25T04:10:06.1341293Z 
2025-04-25T04:10:06.1341441Z     log.error(
2025-04-25T04:10:06.1342803Z         f"Invalid experiment name: {experiment_name}. Experiment names should only contain alphanumeric characters, '_', and '-'. They cannot contain spaces, and the special characters '_' and '-' cannot be the first or last characters."
2025-04-25T04:10:06.1344334Z     )
2025-04-25T04:10:06.1344674Z     return False
2025-04-25T04:10:06.1344913Z 
2025-04-25T04:10:06.1344920Z 
2025-04-25T04:10:06.1345204Z def parse_settings_from_text(settings_text: str) -> Settings:
2025-04-25T04:10:06.1345793Z     """
2025-04-25T04:10:06.1346348Z     Parse the experiments from the issue body into a list of ExperimentSettings
2025-04-25T04:10:06.1347028Z     """
2025-04-25T04:10:06.1347369Z     try:
2025-04-25T04:10:06.1347722Z         if settings_text:
2025-04-25T04:10:06.1348412Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2025-04-25T04:10:06.1349169Z             # for easy reading
2025-04-25T04:10:06.1349919Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2025-04-25T04:10:06.1350982Z             #       the backtick character in shell commands.
2025-04-25T04:10:06.1351569Z             backtick = chr(96)  # backtick character
2025-04-25T04:10:06.1352208Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2025-04-25T04:10:06.1352835Z             settings = load_yaml(settings_text)
2025-04-25T04:10:06.1353192Z 
2025-04-25T04:10:06.1353573Z             # For now we just load experiments. We can expand this if/when we add more settings
2025-04-25T04:10:06.1354279Z             experiments = {}
2025-04-25T04:10:06.1354717Z 
2025-04-25T04:10:06.1355050Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2025-04-25T04:10:06.1355762Z                 if not is_valid_experiment_name(exp_name):
2025-04-25T04:10:06.1356795Z                     # Exclude invalid experiments from the list. We log an error, but don't raise an exception so that other experiments can still be processed.
2025-04-25T04:10:06.1357786Z                     continue
2025-04-25T04:10:06.1358069Z 
2025-04-25T04:10:06.1358237Z                 valid_settings = {}
2025-04-25T04:10:06.1358739Z                 for setting in exp_settings:
2025-04-25T04:10:06.1359283Z                     if setting not in Experiment._fields:
2025-04-25T04:10:06.1359818Z                         log.warning(
2025-04-25T04:10:06.1360711Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2025-04-25T04:10:06.1361394Z                         )
2025-04-25T04:10:06.1361802Z                     else:
2025-04-25T04:10:06.1362293Z                         valid_settings[setting] = exp_settings[setting]
2025-04-25T04:10:06.1362714Z 
2025-04-25T04:10:06.1362964Z                 experiments[exp_name] = Experiment(**valid_settings)
2025-04-25T04:10:06.1363684Z             return Settings(experiments)
2025-04-25T04:10:06.1364037Z 
2025-04-25T04:10:06.1364196Z     except Exception:
2025-04-25T04:10:06.1364657Z         log.exception("Failed to parse settings")
2025-04-25T04:10:06.1365033Z 
2025-04-25T04:10:06.1365190Z     return Settings()
2025-04-25T04:10:06.1365445Z 
2025-04-25T04:10:06.1365451Z 
2025-04-25T04:10:06.1365686Z def parse_settings(rollout_state: str) -> Settings:
2025-04-25T04:10:06.1366221Z     """
2025-04-25T04:10:06.1366627Z     Parse settings, if any, from the rollout state.
2025-04-25T04:10:06.1367016Z 
2025-04-25T04:10:06.1367339Z     If the issue body contains "---" then the text above that is the settings
2025-04-25T04:10:06.1368069Z     and the text below is the list of opted in users.
2025-04-25T04:10:06.1368462Z 
2025-04-25T04:10:06.1368865Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2025-04-25T04:10:06.1369550Z     """
2025-04-25T04:10:06.1370071Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2025-04-25T04:10:06.1371035Z     return parse_settings_from_text(settings_text)
2025-04-25T04:10:06.1371437Z 
2025-04-25T04:10:06.1371443Z 
2025-04-25T04:10:06.1371668Z def parse_users(rollout_state: str) -> UserOptins:
2025-04-25T04:10:06.1372195Z     """
2025-04-25T04:10:06.1372559Z     Parse users from the rollout state.
2025-04-25T04:10:06.1372895Z 
2025-04-25T04:10:06.1373045Z     """
2025-04-25T04:10:06.1373535Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2025-04-25T04:10:06.1374233Z     return parse_user_opt_in_from_text(users_text)
2025-04-25T04:10:06.1374617Z 
2025-04-25T04:10:06.1374625Z 
2025-04-25T04:10:06.1375004Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-04-25T04:10:06.1375713Z     """
2025-04-25T04:10:06.1376095Z     Check if a user is opted into an experiment
2025-04-25T04:10:06.1376602Z     """
2025-04-25T04:10:06.1377030Z     return experiment_name in user_optins.get(user, [])
2025-04-25T04:10:06.1377442Z 
2025-04-25T04:10:06.1377450Z 
2025-04-25T04:10:06.1377834Z def is_user_opted_out(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-04-25T04:10:06.1378549Z     """
2025-04-25T04:10:06.1378973Z     Check if a user explicitly opted out of an experiment
2025-04-25T04:10:06.1379518Z     """
2025-04-25T04:10:06.1379996Z     # if the experiment is prefixed with a "-", then it's an opt-out
2025-04-25T04:10:06.1380875Z     experiment_optout = "-" + experiment_name
2025-04-25T04:10:06.1381486Z     if experiment_optout not in user_optins.get(user, []):
2025-04-25T04:10:06.1382046Z         return False
2025-04-25T04:10:06.1382295Z 
2025-04-25T04:10:06.1382545Z     if is_user_opted_in(user, user_optins, experiment_name):
2025-04-25T04:10:06.1383259Z         log.warning(
2025-04-25T04:10:06.1384015Z             f"User {user} is opted into experiment {experiment_name}, but also opted out of it. Defaulting to opting out"
2025-04-25T04:10:06.1384850Z         )
2025-04-25T04:10:06.1385054Z 
2025-04-25T04:10:06.1385203Z     return True
2025-04-25T04:10:06.1385421Z 
2025-04-25T04:10:06.1385428Z 
2025-04-25T04:10:06.1385592Z def get_runner_prefix(
2025-04-25T04:10:06.1386004Z     rollout_state: str,
2025-04-25T04:10:06.1386440Z     workflow_requestors: Iterable[str],
2025-04-25T04:10:06.1386924Z     branch: str,
2025-04-25T04:10:06.1387376Z     eligible_experiments: frozenset[str] = frozenset(),
2025-04-25T04:10:06.1387950Z     is_canary: bool = False,
2025-04-25T04:10:06.1388382Z ) -> str:
2025-04-25T04:10:06.1388760Z     settings = parse_settings(rollout_state)
2025-04-25T04:10:06.1389305Z     user_optins = parse_users(rollout_state)
2025-04-25T04:10:06.1389654Z 
2025-04-25T04:10:06.1389823Z     fleet_prefix = ""
2025-04-25T04:10:06.1390414Z     prefixes = []
2025-04-25T04:10:06.1391020Z     for experiment_name, experiment_settings in settings.experiments.items():
2025-04-25T04:10:06.1391893Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2025-04-25T04:10:06.1392691Z             log.info(
2025-04-25T04:10:06.1393339Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2025-04-25T04:10:06.1394051Z             )
2025-04-25T04:10:06.1394397Z             continue
2025-04-25T04:10:06.1394639Z 
2025-04-25T04:10:06.1394809Z         if eligible_experiments:
2025-04-25T04:10:06.1395337Z             if experiment_name not in eligible_experiments:
2025-04-25T04:10:06.1395927Z                 exp_list = ", ".join(eligible_experiments)
2025-04-25T04:10:06.1396460Z                 log.info(
2025-04-25T04:10:06.1397196Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2025-04-25T04:10:06.1397995Z                 )
2025-04-25T04:10:06.1398370Z                 continue
2025-04-25T04:10:06.1398819Z         elif not experiment_settings.default:
2025-04-25T04:10:06.1399322Z             log.info(
2025-04-25T04:10:06.1399939Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2025-04-25T04:10:06.1401185Z             )
2025-04-25T04:10:06.1401539Z             continue
2025-04-25T04:10:06.1401777Z 
2025-04-25T04:10:06.1402036Z         # Is any workflow_requestor opted out to this experiment?
2025-04-25T04:10:06.1402622Z         opted_out_users = [
2025-04-25T04:10:06.1403051Z             requestor
2025-04-25T04:10:06.1403470Z             for requestor in workflow_requestors
2025-04-25T04:10:06.1404144Z             if is_user_opted_out(requestor, user_optins, experiment_name)
2025-04-25T04:10:06.1404741Z         ]
2025-04-25T04:10:06.1404943Z 
2025-04-25T04:10:06.1405102Z         if opted_out_users:
2025-04-25T04:10:06.1405535Z             log.info(
2025-04-25T04:10:06.1406114Z                 f"{', '.join(opted_out_users)} have opted out of experiment {experiment_name}."
2025-04-25T04:10:06.1406783Z             )
2025-04-25T04:10:06.1407133Z             continue
2025-04-25T04:10:06.1407373Z 
2025-04-25T04:10:06.1407622Z         # Is any workflow_requestor opted in to this experiment?
2025-04-25T04:10:06.1408198Z         opted_in_users = [
2025-04-25T04:10:06.1408619Z             requestor
2025-04-25T04:10:06.1409047Z             for requestor in workflow_requestors
2025-04-25T04:10:06.1409678Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2025-04-25T04:10:06.1410479Z         ]
2025-04-25T04:10:06.1410676Z 
2025-04-25T04:10:06.1410830Z         enabled = False
2025-04-25T04:10:06.1411243Z         if opted_in_users:
2025-04-25T04:10:06.1411655Z             log.info(
2025-04-25T04:10:06.1412223Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2025-04-25T04:10:06.1413019Z             )
2025-04-25T04:10:06.1413390Z             enabled = True
2025-04-25T04:10:06.1413658Z 
2025-04-25T04:10:06.1413861Z         elif experiment_settings.rollout_perc:
2025-04-25T04:10:06.1414633Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2025-04-25T04:10:06.1415504Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2025-04-25T04:10:06.1416118Z                 log.info(
2025-04-25T04:10:06.1416930Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2025-04-25T04:10:06.1417799Z                 )
2025-04-25T04:10:06.1418190Z                 enabled = True
2025-04-25T04:10:06.1418480Z 
2025-04-25T04:10:06.1418635Z         if enabled:
2025-04-25T04:10:06.1419033Z             label = experiment_name
2025-04-25T04:10:06.1419546Z             if experiment_name == LF_FLEET_EXPERIMENT:
2025-04-25T04:10:06.1420523Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2025-04-25T04:10:06.1421363Z                 #  - If it's enabled, then we always list it's prefix first
2025-04-25T04:10:06.1422188Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2025-04-25T04:10:06.1422818Z                 if is_canary:
2025-04-25T04:10:06.1423290Z                     label += CANARY_FLEET_SUFFIX
2025-04-25T04:10:06.1423802Z                 fleet_prefix = label
2025-04-25T04:10:06.1424269Z             else:
2025-04-25T04:10:06.1424666Z                 prefixes.append(label)
2025-04-25T04:10:06.1425010Z 
2025-04-25T04:10:06.1425174Z     if len(prefixes) > 1:
2025-04-25T04:10:06.1425576Z         log.error(
2025-04-25T04:10:06.1426536Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2025-04-25T04:10:06.1427625Z         )
2025-04-25T04:10:06.1427984Z         prefixes = prefixes[:1]
2025-04-25T04:10:06.1428280Z 
2025-04-25T04:10:06.1428457Z     # Fleet always comes first
2025-04-25T04:10:06.1428900Z     if fleet_prefix:
2025-04-25T04:10:06.1429318Z         prefixes.insert(0, fleet_prefix)
2025-04-25T04:10:06.1429667Z 
2025-04-25T04:10:06.1429894Z     return ".".join(prefixes) + "." if prefixes else ""
2025-04-25T04:10:06.1430526Z 
2025-04-25T04:10:06.1430533Z 
2025-04-25T04:10:06.1430948Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2025-04-25T04:10:06.1431684Z     """
2025-04-25T04:10:06.1432231Z     Gets the first comment of the issue, which contains the desired rollout state.
2025-04-25T04:10:06.1432764Z 
2025-04-25T04:10:06.1433122Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2025-04-25T04:10:06.1433789Z     """
2025-04-25T04:10:06.1434157Z     gh = get_gh_client(github_token)
2025-04-25T04:10:06.1434655Z     issue = get_issue(gh, repo, issue_num)
2025-04-25T04:10:06.1435264Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2025-04-25T04:10:06.1435689Z 
2025-04-25T04:10:06.1435696Z 
2025-04-25T04:10:06.1436069Z def download_json(url: str, headers: dict[str, str], num_retries: int = 3) -> Any:
2025-04-25T04:10:06.1436772Z     for _ in range(num_retries):
2025-04-25T04:10:06.1437215Z         try:
2025-04-25T04:10:06.1437609Z             req = Request(url=url, headers=headers)
2025-04-25T04:10:06.1438241Z             content = urlopen(req, timeout=5).read().decode("utf-8")
2025-04-25T04:10:06.1438845Z             return json.loads(content)
2025-04-25T04:10:06.1439349Z         except Exception as e:
2025-04-25T04:10:06.1439848Z             log.warning(f"Could not download {url}: {e}")
2025-04-25T04:10:06.1440355Z 
2025-04-25T04:10:06.1440709Z     log.warning(f"All {num_retries} retries exhausted, downloading {url} failed")
2025-04-25T04:10:06.1441393Z     return {}
2025-04-25T04:10:06.1441612Z 
2025-04-25T04:10:06.1441619Z 
2025-04-25T04:10:06.1441764Z @cache
2025-04-25T04:10:06.1442496Z def get_pr_info(github_repo: str, github_token: str, pr_number: int) -> dict[str, Any]:
2025-04-25T04:10:06.1443198Z     """
2025-04-25T04:10:06.1443575Z     Dynamically get PR information
2025-04-25T04:10:06.1444037Z     """
2025-04-25T04:10:06.1444519Z     github_api = f"https://api.github.com/repos/{github_repo}"
2025-04-25T04:10:06.1445120Z     headers = {
2025-04-25T04:10:06.1445549Z         "Accept": "application/vnd.github.v3+json",
2025-04-25T04:10:06.1446118Z         "Authorization": f"token {github_token}",
2025-04-25T04:10:06.1446622Z     }
2025-04-25T04:10:06.1447016Z     json_response: dict[str, Any] = download_json(
2025-04-25T04:10:06.1447580Z         url=f"{github_api}/issues/{pr_number}",
2025-04-25T04:10:06.1448099Z         headers=headers,
2025-04-25T04:10:06.1448499Z     )
2025-04-25T04:10:06.1448694Z 
2025-04-25T04:10:06.1448857Z     if not json_response:
2025-04-25T04:10:06.1449393Z         log.warning(f"Failed to get the labels for #{pr_number}")
2025-04-25T04:10:06.1449975Z         return {}
2025-04-25T04:10:06.1450757Z 
2025-04-25T04:10:06.1450970Z     return json_response
2025-04-25T04:10:06.1451259Z 
2025-04-25T04:10:06.1451266Z 
2025-04-25T04:10:06.1451645Z def get_labels(github_repo: str, github_token: str, pr_number: int) -> set[str]:
2025-04-25T04:10:06.1452537Z     """
2025-04-25T04:10:06.1453041Z     Dynamically get the latest list of labels from the pull request
2025-04-25T04:10:06.1453660Z     """
2025-04-25T04:10:06.1454114Z     pr_info = get_pr_info(github_repo, github_token, pr_number)
2025-04-25T04:10:06.1454680Z     return {
2025-04-25T04:10:06.1455231Z         label.get("name") for label in pr_info.get("labels", []) if label.get("name")
2025-04-25T04:10:06.1455884Z     }
2025-04-25T04:10:06.1456071Z 
2025-04-25T04:10:06.1456083Z 
2025-04-25T04:10:06.1456240Z def main() -> None:
2025-04-25T04:10:06.1456635Z     args = parse_args()
2025-04-25T04:10:06.1456899Z 
2025-04-25T04:10:06.1457095Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2025-04-25T04:10:06.1457471Z 
2025-04-25T04:10:06.1457649Z     # Check if the PR is opt-out
2025-04-25T04:10:06.1458118Z     if args.pr_number:
2025-04-25T04:10:06.1458728Z         labels = get_labels(args.github_repo, args.github_token, int(args.pr_number))
2025-04-25T04:10:06.1459440Z         if OPT_OUT_LABEL in labels:
2025-04-25T04:10:06.1459912Z             log.info(
2025-04-25T04:10:06.1460770Z                 f"Opt-out runner determinator because #{args.pr_number} has {OPT_OUT_LABEL} label"
2025-04-25T04:10:06.1461498Z             )
2025-04-25T04:10:06.1462007Z             set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-04-25T04:10:06.1462644Z             sys.exit()
2025-04-25T04:10:06.1462898Z 
2025-04-25T04:10:06.1463050Z     try:
2025-04-25T04:10:06.1463453Z         rollout_state = get_rollout_state_from_issue(
2025-04-25T04:10:06.1464120Z             args.github_token, args.github_issue_repo, args.github_issue
2025-04-25T04:10:06.1464714Z         )
2025-04-25T04:10:06.1464913Z 
2025-04-25T04:10:06.1465104Z         username = get_potential_pr_author(
2025-04-25T04:10:06.1465614Z             args.github_token,
2025-04-25T04:10:06.1466060Z             args.github_repo,
2025-04-25T04:10:06.1466503Z             args.github_actor,
2025-04-25T04:10:06.1466953Z             args.github_ref_type,
2025-04-25T04:10:06.1467423Z             args.github_branch,
2025-04-25T04:10:06.1467851Z         )
2025-04-25T04:10:06.1468047Z 
2025-04-25T04:10:06.1468318Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2025-04-25T04:10:06.1468744Z 
2025-04-25T04:10:06.1468939Z         runner_label_prefix = get_runner_prefix(
2025-04-25T04:10:06.1469460Z             rollout_state,
2025-04-25T04:10:06.1469911Z             (args.github_issue_owner, username),
2025-04-25T04:10:06.1470667Z             args.github_branch,
2025-04-25T04:10:06.1471137Z             args.eligible_experiments,
2025-04-25T04:10:06.1471625Z             is_canary,
2025-04-25T04:10:06.1472020Z         )
2025-04-25T04:10:06.1472214Z 
2025-04-25T04:10:06.1472526Z     except Exception as e:
2025-04-25T04:10:06.1472954Z         log.error(
2025-04-25T04:10:06.1473565Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2025-04-25T04:10:06.1474277Z         )
2025-04-25T04:10:06.1474476Z 
2025-04-25T04:10:06.1474781Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-04-25T04:10:06.1475262Z 
2025-04-25T04:10:06.1475268Z 
2025-04-25T04:10:06.1475428Z if __name__ == "__main__":
2025-04-25T04:10:06.1475837Z     main()
2025-04-25T04:10:06.1476039Z 
2025-04-25T04:10:06.1564017Z ##[group]Run python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2025-04-25T04:10:06.1564822Z python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2025-04-25T04:10:06.1599798Z shell: /usr/bin/bash -e {0}
2025-04-25T04:10:06.1600691Z env:
2025-04-25T04:10:06.1601487Z   GITHUB_TOKEN: ***
2025-04-25T04:10:06.1602072Z   ISSUE_NUMBER: 5132
2025-04-25T04:10:06.1602715Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2025-04-25T04:10:06.1603464Z   ISSUE_OWNER: 
2025-04-25T04:10:06.1604030Z   CHECK_EXPERIMENTS: 
2025-04-25T04:10:06.1604656Z   PR_NUMBER: 
2025-04-25T04:10:06.1605210Z ##[endgroup]
2025-04-25T04:10:07.3626310Z Defaulting to user installation because normal site-packages is not writeable
2025-04-25T04:10:08.0038177Z Collecting urllib3==1.26.18
2025-04-25T04:10:08.0396184Z   Downloading urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
2025-04-25T04:10:08.0622778Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 4.1 MB/s eta 0:00:00
2025-04-25T04:10:08.0810760Z Collecting PyGithub==2.3.0
2025-04-25T04:10:08.0844015Z   Downloading PyGithub-2.3.0-py3-none-any.whl.metadata (3.8 kB)
2025-04-25T04:10:08.1268869Z Collecting pynacl>=1.4.0 (from PyGithub==2.3.0)
2025-04-25T04:10:08.1315366Z   Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (8.6 kB)
2025-04-25T04:10:08.1365539Z Requirement already satisfied: requests>=2.14.0 in /usr/lib/python3/dist-packages (from PyGithub==2.3.0) (2.31.0)
2025-04-25T04:10:08.1386161Z Requirement already satisfied: pyjwt>=2.4.0 in /usr/lib/python3/dist-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.3.0) (2.7.0)
2025-04-25T04:10:08.1401084Z Requirement already satisfied: typing-extensions>=4.0.0 in /usr/lib/python3/dist-packages (from PyGithub==2.3.0) (4.10.0)
2025-04-25T04:10:08.1633615Z Collecting Deprecated (from PyGithub==2.3.0)
2025-04-25T04:10:08.1671990Z   Downloading Deprecated-1.2.18-py2.py3-none-any.whl.metadata (5.7 kB)
2025-04-25T04:10:08.1904486Z Requirement already satisfied: cryptography>=3.4.0 in /usr/lib/python3/dist-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.3.0) (41.0.7)
2025-04-25T04:10:08.2970073Z Collecting cffi>=1.4.1 (from pynacl>=1.4.0->PyGithub==2.3.0)
2025-04-25T04:10:08.3003943Z   Downloading cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.5 kB)
2025-04-25T04:10:08.4026157Z Collecting wrapt<2,>=1.10 (from Deprecated->PyGithub==2.3.0)
2025-04-25T04:10:08.4064660Z   Downloading wrapt-1.17.2-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.4 kB)
2025-04-25T04:10:08.4268812Z Collecting pycparser (from cffi>=1.4.1->pynacl>=1.4.0->PyGithub==2.3.0)
2025-04-25T04:10:08.4303246Z   Downloading pycparser-2.22-py3-none-any.whl.metadata (943 bytes)
2025-04-25T04:10:08.4538984Z Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
2025-04-25T04:10:08.4645222Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.8/143.8 kB 15.4 MB/s eta 0:00:00
2025-04-25T04:10:08.4695370Z Downloading PyGithub-2.3.0-py3-none-any.whl (354 kB)
2025-04-25T04:10:08.4813707Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 354.4/354.4 kB 34.3 MB/s eta 0:00:00
2025-04-25T04:10:08.4856006Z Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
2025-04-25T04:10:08.5073400Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 856.7/856.7 kB 42.7 MB/s eta 0:00:00
2025-04-25T04:10:08.5108202Z Downloading Deprecated-1.2.18-py2.py3-none-any.whl (10.0 kB)
2025-04-25T04:10:08.5162565Z Downloading cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (479 kB)
2025-04-25T04:10:08.5269557Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 479.4/479.4 kB 52.7 MB/s eta 0:00:00
2025-04-25T04:10:08.5302781Z Downloading wrapt-1.17.2-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (89 kB)
2025-04-25T04:10:08.5352147Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.2/89.2 kB 23.4 MB/s eta 0:00:00
2025-04-25T04:10:08.5383757Z Downloading pycparser-2.22-py3-none-any.whl (117 kB)
2025-04-25T04:10:08.5441182Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 kB 25.9 MB/s eta 0:00:00
2025-04-25T04:10:08.8534877Z Installing collected packages: wrapt, urllib3, pycparser, Deprecated, cffi, pynacl, PyGithub
2025-04-25T04:10:09.3901973Z Successfully installed Deprecated-1.2.18 PyGithub-2.3.0 cffi-1.17.1 pycparser-2.22 pynacl-1.5.0 urllib3-1.26.18 wrapt-1.17.2
2025-04-25T04:10:09.4765563Z ##[group]Run curr_branch="ciflow/trunk/148893"
2025-04-25T04:10:09.4765956Z curr_branch="ciflow/trunk/148893"
2025-04-25T04:10:09.4766209Z curr_ref_type="tag"
2025-04-25T04:10:09.4766463Z echo "Current branch is '$curr_branch'"
2025-04-25T04:10:09.4766722Z 
2025-04-25T04:10:09.4766932Z python3 runner_determinator.py \
2025-04-25T04:10:09.4767217Z   --github-token "$GITHUB_TOKEN" \
2025-04-25T04:10:09.4767478Z   --github-issue "$ISSUE_NUMBER" \
2025-04-25T04:10:09.4767732Z   --github-branch "$curr_branch" \
2025-04-25T04:10:09.4767990Z   --github-actor "$TRIGGERING_ACTOR" \
2025-04-25T04:10:09.4768276Z   --github-issue-owner "$ISSUE_OWNER" \
2025-04-25T04:10:09.4768551Z   --github-ref-type "$curr_ref_type" \
2025-04-25T04:10:09.4768824Z   --github-repo "$GITHUB_REPOSITORY" \
2025-04-25T04:10:09.4769171Z   --eligible-experiments "$CHECK_EXPERIMENTS" \
2025-04-25T04:10:09.4769481Z   --pr-number "${PR_NUMBER}"
2025-04-25T04:10:09.4801063Z shell: /usr/bin/bash -e {0}
2025-04-25T04:10:09.4801300Z env:
2025-04-25T04:10:09.4801877Z   GITHUB_TOKEN: ***
2025-04-25T04:10:09.4802070Z   ISSUE_NUMBER: 5132
2025-04-25T04:10:09.4802278Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2025-04-25T04:10:09.4802511Z   ISSUE_OWNER: 
2025-04-25T04:10:09.4802721Z   CHECK_EXPERIMENTS: 
2025-04-25T04:10:09.4802909Z   PR_NUMBER: 
2025-04-25T04:10:09.4803086Z ##[endgroup]
2025-04-25T04:10:09.4852698Z Current branch is 'ciflow/trunk/148893'
2025-04-25T04:10:11.4736118Z INFO    : Based on rollout percentage of 100%, enabling experiment ephemeral.
2025-04-25T04:10:11.4737448Z INFO    : Setting output: label-type='ephemeral.'
2025-04-25T04:10:11.5052936Z Evaluate and set job outputs
2025-04-25T04:10:11.5059304Z Set output 'label-type'
2025-04-25T04:10:11.5061276Z Cleaning up orphan processes