2025-07-24T03:40:53.8917914Z Current runner version: '2.326.0'
2025-07-24T03:40:53.8943691Z ##[group]Runner Image Provisioner
2025-07-24T03:40:53.8944545Z Hosted Compute Agent
2025-07-24T03:40:53.8945052Z Version: 20250711.363
2025-07-24T03:40:53.8945764Z Commit: 6785254374ce925a23743850c1cb91912ce5c14c
2025-07-24T03:40:53.8946415Z Build Date: 2025-07-11T20:04:25Z
2025-07-24T03:40:53.8946996Z ##[endgroup]
2025-07-24T03:40:53.8947590Z ##[group]Operating System
2025-07-24T03:40:53.8948158Z Ubuntu
2025-07-24T03:40:53.8948600Z 24.04.2
2025-07-24T03:40:53.8949116Z LTS
2025-07-24T03:40:53.8949536Z ##[endgroup]
2025-07-24T03:40:53.8950019Z ##[group]Runner Image
2025-07-24T03:40:53.8950641Z Image: ubuntu-24.04
2025-07-24T03:40:53.8951367Z Version: 20250720.1.0
2025-07-24T03:40:53.8952444Z Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20250720.1/images/ubuntu/Ubuntu2404-Readme.md
2025-07-24T03:40:53.8954080Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20250720.1
2025-07-24T03:40:53.8955048Z ##[endgroup]
2025-07-24T03:40:53.8956023Z ##[group]GITHUB_TOKEN Permissions
2025-07-24T03:40:53.8957868Z Metadata: read
2025-07-24T03:40:53.8958488Z ##[endgroup]
2025-07-24T03:40:53.8960556Z Secret source: Actions
2025-07-24T03:40:53.8961597Z Prepare workflow directory
2025-07-24T03:40:53.9662095Z Prepare all required actions
2025-07-24T03:40:53.9743112Z Uses: pytorch/pytorch/.github/workflows/_runner-determinator.yml@refs/heads/main (07df6ba7f5597488a93b3855d52d2ead55675125)
2025-07-24T03:40:53.9750333Z ##[group] Inputs
2025-07-24T03:40:53.9751556Z   check_experiments: 
2025-07-24T03:40:53.9752565Z   opt_out_experiments: 
2025-07-24T03:40:53.9753386Z   triggering_actor: pytorch-bot[bot]
2025-07-24T03:40:53.9754357Z   issue_owner: 
2025-07-24T03:40:53.9755200Z   curr_branch: ciflow/trunk/149961
2025-07-24T03:40:53.9756196Z   curr_ref_type: tag
2025-07-24T03:40:53.9756929Z   issue_number: 5132
2025-07-24T03:40:53.9758028Z ##[endgroup]
2025-07-24T03:40:53.9759062Z Complete job name: get-label-type / runner-determinator
2025-07-24T03:40:54.5104162Z ##[group]Run cat <<EOF > runner_determinator.py
2025-07-24T03:40:54.5106862Z cat <<EOF > runner_determinator.py
2025-07-24T03:40:54.5107522Z # flake8: noqa: G004
2025-07-24T03:40:54.5108177Z 
2025-07-24T03:40:54.5108945Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2025-07-24T03:40:54.5110020Z #       must be kept in sync. You can do it easily by running the following command:
2025-07-24T03:40:54.5111250Z #           python .github/scripts/update_runner_determinator.py
2025-07-24T03:40:54.5111976Z 
2025-07-24T03:40:54.5112432Z """
2025-07-24T03:40:54.5113228Z This runner determinator is used to determine which set of runners to run a
2025-07-24T03:40:54.5114303Z GitHub job on. It uses the first comment of a GitHub issue (by default
2025-07-24T03:40:54.5115554Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2025-07-24T03:40:54.5116612Z of which runners should be used to run which job.
2025-07-24T03:40:54.5117321Z 
2025-07-24T03:40:54.5118063Z The configuration has two parts, the settings and a list of opted-in users,
2025-07-24T03:40:54.5119188Z separated by a line containing "---".  If the line is not present, the
2025-07-24T03:40:54.5120221Z settings are considered to be empty with only the second part, the user
2025-07-24T03:40:54.5121522Z list, defined.
2025-07-24T03:40:54.5122124Z 
2025-07-24T03:40:54.5122812Z The first part is a YAML block that defines the rollout settings. This can be
2025-07-24T03:40:54.5123945Z used to define any settings that are needed to determine which runners to use.
2025-07-24T03:40:54.5124971Z It's fields are defined by the RolloutSettings class below.
2025-07-24T03:40:54.5125731Z 
2025-07-24T03:40:54.5126521Z The second part is a list of users who are explicitly opted in to the LF fleet.
2025-07-24T03:40:54.5127836Z The user list is also a comma separated list of additional features or
2025-07-24T03:40:54.5128755Z experiments which the user could be opted in to.
2025-07-24T03:40:54.5129411Z 
2025-07-24T03:40:54.5130027Z The user list has the following rules:
2025-07-24T03:40:54.5130643Z 
2025-07-24T03:40:54.5131544Z - Users are GitHub usernames, which must start with the @ prefix
2025-07-24T03:40:54.5132689Z - Each user is also a comma-separated list of features/experiments to enable
2025-07-24T03:40:54.5133607Z - A "#" prefix opts the user out of all experiments
2025-07-24T03:40:54.5134277Z 
2025-07-24T03:40:54.5134801Z Example config:
2025-07-24T03:40:54.5135440Z     # A list of experiments that can be opted into.
2025-07-24T03:40:54.5136289Z     # This defines the behavior they'll induce when opted into.
2025-07-24T03:40:54.5137114Z     # Expected syntax is:
2025-07-24T03:40:54.5137955Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2025-07-24T03:40:54.5139079Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2025-07-24T03:40:54.5140050Z 
2025-07-24T03:40:54.5140509Z     experiments:
2025-07-24T03:40:54.5141301Z       lf:
2025-07-24T03:40:54.5141899Z         rollout_percent: 25
2025-07-24T03:40:54.5142492Z         all_branches: false
2025-07-24T03:40:54.5143104Z         default: true
2025-07-24T03:40:54.5143664Z     ---
2025-07-24T03:40:54.5144178Z 
2025-07-24T03:40:54.5144648Z     # Opt-ins:
2025-07-24T03:40:54.5145444Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2025-07-24T03:40:54.5146665Z     # and specifying experiments to enable in a comma-separated list.
2025-07-24T03:40:54.5147672Z     # To always opt out of an experiment, prefix it with a "-".
2025-07-24T03:40:54.5148543Z     # Experiments should be from the above list.
2025-07-24T03:40:54.5149243Z 
2025-07-24T03:40:54.5149749Z     @User1,-lf,split_build
2025-07-24T03:40:54.5225458Z     @User2,lf
2025-07-24T03:40:54.5226105Z     @User3,split_build
2025-07-24T03:40:54.5226639Z """
2025-07-24T03:40:54.5227028Z 
2025-07-24T03:40:54.5227414Z import json
2025-07-24T03:40:54.5227853Z import logging
2025-07-24T03:40:54.5228306Z import os
2025-07-24T03:40:54.5228758Z import random
2025-07-24T03:40:54.5229212Z import re
2025-07-24T03:40:54.5229630Z import sys
2025-07-24T03:40:54.5230102Z from argparse import ArgumentParser
2025-07-24T03:40:54.5230778Z from collections.abc import Iterable
2025-07-24T03:40:54.5231550Z from functools import cache
2025-07-24T03:40:54.5232109Z from logging import LogRecord
2025-07-24T03:40:54.5232679Z from typing import Any, NamedTuple
2025-07-24T03:40:54.5233311Z from urllib.request import Request, urlopen
2025-07-24T03:40:54.5233915Z 
2025-07-24T03:40:54.5234291Z import yaml
2025-07-24T03:40:54.5234746Z from github import Auth, Github
2025-07-24T03:40:54.5235307Z from github.Issue import Issue
2025-07-24T03:40:54.5235825Z 
2025-07-24T03:40:54.5236190Z 
2025-07-24T03:40:54.5236648Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2025-07-24T03:40:54.5237428Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2025-07-24T03:40:54.5238394Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2025-07-24T03:40:54.5239170Z 
2025-07-24T03:40:54.5239648Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2025-07-24T03:40:54.5240571Z GH_OUTPUT_KEY_AMI = "runner-ami"
2025-07-24T03:40:54.5241286Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2025-07-24T03:40:54.5241935Z OPT_OUT_LABEL = "no-runner-experiments"
2025-07-24T03:40:54.5242507Z 
2025-07-24T03:40:54.5242944Z SETTING_EXPERIMENTS = "experiments"
2025-07-24T03:40:54.5243497Z 
2025-07-24T03:40:54.5243898Z LF_FLEET_EXPERIMENT = "lf"
2025-07-24T03:40:54.5244431Z CANARY_FLEET_SUFFIX = ".c"
2025-07-24T03:40:54.5244936Z 
2025-07-24T03:40:54.5245307Z 
2025-07-24T03:40:54.5245717Z class Experiment(NamedTuple):
2025-07-24T03:40:54.5246262Z     rollout_perc: float = (
2025-07-24T03:40:54.5247008Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2025-07-24T03:40:54.5247774Z     )
2025-07-24T03:40:54.5248199Z     all_branches: bool = (
2025-07-24T03:40:54.5248929Z         False  # If True, the experiment is also enabled on the exception branches
2025-07-24T03:40:54.5249662Z     )
2025-07-24T03:40:54.5250079Z     default: bool = (
2025-07-24T03:40:54.5250742Z         True  # If True, the experiment is enabled by default for all queries
2025-07-24T03:40:54.5251561Z     )
2025-07-24T03:40:54.5251974Z 
2025-07-24T03:40:54.5252377Z     # Add more fields as needed
2025-07-24T03:40:54.5252888Z 
2025-07-24T03:40:54.5253258Z 
2025-07-24T03:40:54.5253662Z class Settings(NamedTuple):
2025-07-24T03:40:54.5254182Z     """
2025-07-24T03:40:54.5254716Z     Settings for the experiments that can be opted into.
2025-07-24T03:40:54.5255342Z     """
2025-07-24T03:40:54.5255746Z 
2025-07-24T03:40:54.5256180Z     experiments: dict[str, Experiment] = {}
2025-07-24T03:40:54.5256731Z 
2025-07-24T03:40:54.5257237Z 
2025-07-24T03:40:54.5257695Z class ColorFormatter(logging.Formatter):
2025-07-24T03:40:54.5258404Z     """Color codes the log messages based on the log level"""
2025-07-24T03:40:54.5259042Z 
2025-07-24T03:40:54.5259421Z     COLORS = {
2025-07-24T03:40:54.5259897Z         "WARNING": "\033[33m",  # Yellow
2025-07-24T03:40:54.5260458Z         "ERROR": "\033[31m",  # Red
2025-07-24T03:40:54.5261012Z         "CRITICAL": "\033[31m",  # Red
2025-07-24T03:40:54.5261928Z         "INFO": "\033[0m",  # Reset
2025-07-24T03:40:54.5262477Z         "DEBUG": "\033[0m",  # Reset
2025-07-24T03:40:54.5263002Z     }
2025-07-24T03:40:54.5263385Z 
2025-07-24T03:40:54.5263842Z     def format(self, record: LogRecord) -> str:
2025-07-24T03:40:54.5264671Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2025-07-24T03:40:54.5265537Z         record.msg = f"{log_color}{record.msg}\033[0m"
2025-07-24T03:40:54.5266221Z         return super().format(record)
2025-07-24T03:40:54.5266752Z 
2025-07-24T03:40:54.5267122Z 
2025-07-24T03:40:54.5267537Z handler = logging.StreamHandler()
2025-07-24T03:40:54.5268333Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2025-07-24T03:40:54.5269114Z 
2025-07-24T03:40:54.5269603Z log = logging.getLogger(os.path.basename(__file__))
2025-07-24T03:40:54.5270256Z log.addHandler(handler)
2025-07-24T03:40:54.5270772Z log.setLevel(logging.INFO)
2025-07-24T03:40:54.5271382Z 
2025-07-24T03:40:54.5271743Z 
2025-07-24T03:40:54.5272238Z def set_github_output(key: str, value: str) -> None:
2025-07-24T03:40:54.5272856Z     """
2025-07-24T03:40:54.5273442Z     Defines outputs of the github action that invokes this script
2025-07-24T03:40:54.5274274Z     """
2025-07-24T03:40:54.5274690Z     if not GITHUB_OUTPUT:
2025-07-24T03:40:54.5275858Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2025-07-24T03:40:54.5277077Z         log.warning(
2025-07-24T03:40:54.5278031Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2025-07-24T03:40:54.5279027Z         )
2025-07-24T03:40:54.5279515Z         print(f"::set-output name={key}::{value}")
2025-07-24T03:40:54.5280109Z         return
2025-07-24T03:40:54.5280534Z 
2025-07-24T03:40:54.5280957Z     with open(GITHUB_OUTPUT, "a") as f:
2025-07-24T03:40:54.5281711Z         log.info(f"Setting output: {key}='{value}'")
2025-07-24T03:40:54.5282347Z         f.write(f"{key}={value}\n")
2025-07-24T03:40:54.5282886Z 
2025-07-24T03:40:54.5283272Z 
2025-07-24T03:40:54.5283834Z def _str_comma_separated_to_set(value: str) -> frozenset[str]:
2025-07-24T03:40:54.5284555Z     return frozenset(
2025-07-24T03:40:54.5285264Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2025-07-24T03:40:54.5286004Z     )
2025-07-24T03:40:54.5286413Z 
2025-07-24T03:40:54.5286783Z 
2025-07-24T03:40:54.5287181Z def parse_args() -> Any:
2025-07-24T03:40:54.5287842Z     parser = ArgumentParser("Get dynamic rollout settings")
2025-07-24T03:40:54.5288781Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2025-07-24T03:40:54.5289609Z     parser.add_argument(
2025-07-24T03:40:54.5290140Z         "--github-issue-repo",
2025-07-24T03:40:54.5290672Z         type=str,
2025-07-24T03:40:54.5291237Z         required=False,
2025-07-24T03:40:54.5291901Z         default="pytorch/test-infra",
2025-07-24T03:40:54.5292515Z         help="GitHub repo to get the issue",
2025-07-24T03:40:54.5293071Z     )
2025-07-24T03:40:54.5293494Z     parser.add_argument(
2025-07-24T03:40:54.5294004Z         "--github-repo",
2025-07-24T03:40:54.5294504Z         type=str,
2025-07-24T03:40:54.5294970Z         required=True,
2025-07-24T03:40:54.5295513Z         help="GitHub repo where CI is running",
2025-07-24T03:40:54.5296081Z     )
2025-07-24T03:40:54.5296501Z     parser.add_argument(
2025-07-24T03:40:54.5297200Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2025-07-24T03:40:54.5297905Z     )
2025-07-24T03:40:54.5298323Z     parser.add_argument(
2025-07-24T03:40:54.5299030Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2025-07-24T03:40:54.5299783Z     )
2025-07-24T03:40:54.5300189Z     parser.add_argument(
2025-07-24T03:40:54.5300909Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2025-07-24T03:40:54.5302224Z     )
2025-07-24T03:40:54.5302640Z     parser.add_argument(
2025-07-24T03:40:54.5303393Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2025-07-24T03:40:54.5304160Z     )
2025-07-24T03:40:54.5304593Z     parser.add_argument(
2025-07-24T03:40:54.5305125Z         "--github-ref-type",
2025-07-24T03:40:54.5305653Z         type=str,
2025-07-24T03:40:54.5306137Z         required=True,
2025-07-24T03:40:54.5306717Z         help="Current GitHub ref type, branch or tag",
2025-07-24T03:40:54.5307315Z     )
2025-07-24T03:40:54.5307738Z     parser.add_argument(
2025-07-24T03:40:54.5308274Z         "--eligible-experiments",
2025-07-24T03:40:54.5309020Z         type=_str_comma_separated_to_set,
2025-07-24T03:40:54.5309591Z         required=False,
2025-07-24T03:40:54.5310098Z         default="",
2025-07-24T03:40:54.5311168Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2025-07-24T03:40:54.5312170Z     )
2025-07-24T03:40:54.5312590Z     parser.add_argument(
2025-07-24T03:40:54.5313119Z         "--opt-out-experiments",
2025-07-24T03:40:54.5313691Z         type=_str_comma_separated_to_set,
2025-07-24T03:40:54.5314251Z         required=False,
2025-07-24T03:40:54.5314744Z         default="",
2025-07-24T03:40:54.5315214Z         help=(
2025-07-24T03:40:54.5315970Z             "comma separated list of experiments to opt-out of. If unset, no opt-outs will occur. "
2025-07-24T03:40:54.5317183Z             "If the same experiment is listed both here and in '--eligible-experiments' opt-out will take priority."
2025-07-24T03:40:54.5318077Z         ),
2025-07-24T03:40:54.5318496Z     )
2025-07-24T03:40:54.5318909Z     parser.add_argument(
2025-07-24T03:40:54.5319420Z         "--pr-number",
2025-07-24T03:40:54.5319909Z         type=str,
2025-07-24T03:40:54.5320384Z         required=False,
2025-07-24T03:40:54.5320882Z         default="",
2025-07-24T03:40:54.5321591Z         help="the optional PR number where this is run",
2025-07-24T03:40:54.5322202Z     )
2025-07-24T03:40:54.5322595Z 
2025-07-24T03:40:54.5323000Z     return parser.parse_args()
2025-07-24T03:40:54.5323516Z 
2025-07-24T03:40:54.5323883Z 
2025-07-24T03:40:54.5324530Z def get_gh_client(github_token: str) -> Github:  # type: ignore[no-any-unimported]
2025-07-24T03:40:54.5325480Z     auth = Auth.Token(github_token)
2025-07-24T03:40:54.5326092Z     return Github(auth=auth)
2025-07-24T03:40:54.5326595Z 
2025-07-24T03:40:54.5326964Z 
2025-07-24T03:40:54.5327667Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:  # type: ignore[no-any-unimported]
2025-07-24T03:40:54.5328537Z     repo = gh.get_repo(repo)
2025-07-24T03:40:54.5329114Z     return repo.get_issue(number=issue_num)
2025-07-24T03:40:54.5329686Z 
2025-07-24T03:40:54.5330057Z 
2025-07-24T03:40:54.5330455Z def get_potential_pr_author(
2025-07-24T03:40:54.5331308Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2025-07-24T03:40:54.5332041Z ) -> str:
2025-07-24T03:40:54.5332638Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2025-07-24T03:40:54.5333521Z     # Fetch the actual username from the original PR. The PR number is
2025-07-24T03:40:54.5334359Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2025-07-24T03:40:54.5334974Z 
2025-07-24T03:40:54.5335387Z     gh = get_gh_client(github_token)
2025-07-24T03:40:54.5335930Z 
2025-07-24T03:40:54.5336440Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2025-07-24T03:40:54.5337138Z         split_tag = ref_name.split("/")
2025-07-24T03:40:54.5337690Z         if (
2025-07-24T03:40:54.5338142Z             len(split_tag) == 3
2025-07-24T03:40:54.5338705Z             and split_tag[0] == "ciflow"
2025-07-24T03:40:54.5339291Z             and split_tag[2].isnumeric()
2025-07-24T03:40:54.5339841Z         ):
2025-07-24T03:40:54.5340291Z             pr_number = split_tag[2]
2025-07-24T03:40:54.5340831Z             try:
2025-07-24T03:40:54.5341443Z                 repository = gh.get_repo(repo)
2025-07-24T03:40:54.5342127Z                 pull = repository.get_pull(number=int(pr_number))
2025-07-24T03:40:54.5342944Z             except Exception as e:
2025-07-24T03:40:54.5343532Z                 raise Exception(  # noqa: TRY002
2025-07-24T03:40:54.5344273Z                     f"issue with pull request {pr_number} from repo {repository}"
2025-07-24T03:40:54.5344968Z                 ) from e
2025-07-24T03:40:54.5345599Z             return pull.user.login  # type: ignore[no-any-return]
2025-07-24T03:40:54.5346378Z     # In all other cases, return the original input username
2025-07-24T03:40:54.5347020Z     return username
2025-07-24T03:40:54.5347522Z 
2025-07-24T03:40:54.5347890Z 
2025-07-24T03:40:54.5348349Z def is_exception_branch(branch: str) -> bool:
2025-07-24T03:40:54.5348921Z     """
2025-07-24T03:40:54.5349649Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2025-07-24T03:40:54.5350485Z     """
2025-07-24T03:40:54.5351193Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2025-07-24T03:40:54.5351927Z 
2025-07-24T03:40:54.5352294Z 
2025-07-24T03:40:54.5352724Z def load_yaml(yaml_text: str) -> Any:
2025-07-24T03:40:54.5353257Z     try:
2025-07-24T03:40:54.5353704Z         data = yaml.safe_load(yaml_text)
2025-07-24T03:40:54.5354256Z         return data
2025-07-24T03:40:54.5354747Z     except yaml.YAMLError:
2025-07-24T03:40:54.5355305Z         log.exception("Error loading YAML")
2025-07-24T03:40:54.5355871Z         raise
2025-07-24T03:40:54.5356307Z 
2025-07-24T03:40:54.5356668Z 
2025-07-24T03:40:54.5357336Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> tuple[str, str]:
2025-07-24T03:40:54.5358129Z     """
2025-07-24T03:40:54.5358960Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2025-07-24T03:40:54.5359771Z 
2025-07-24T03:40:54.5360366Z     If the issue body contains "---" then the text above that is the settings
2025-07-24T03:40:54.5361315Z     and the text below is the list of opted in users.
2025-07-24T03:40:54.5361914Z 
2025-07-24T03:40:54.5362532Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2025-07-24T03:40:54.5363281Z     """
2025-07-24T03:40:54.5363791Z     rollout_state_parts = rollout_state.split("---")
2025-07-24T03:40:54.5364437Z     if len(rollout_state_parts) >= 2:
2025-07-24T03:40:54.5365115Z         return rollout_state_parts[0], rollout_state_parts[1]
2025-07-24T03:40:54.5365932Z     else:
2025-07-24T03:40:54.5366456Z         return "", rollout_state
2025-07-24T03:40:54.5366968Z 
2025-07-24T03:40:54.5367322Z 
2025-07-24T03:40:54.5367766Z class UserOptins(dict[str, list[str]]):
2025-07-24T03:40:54.5368321Z     """
2025-07-24T03:40:54.5368908Z     Dictionary of users with a list of features they have opted into
2025-07-24T03:40:54.5369598Z     """
2025-07-24T03:40:54.5369987Z 
2025-07-24T03:40:54.5370341Z 
2025-07-24T03:40:54.5370905Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2025-07-24T03:40:54.5371715Z     """
2025-07-24T03:40:54.5372495Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2025-07-24T03:40:54.5373384Z 
2025-07-24T03:40:54.5374248Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2025-07-24T03:40:54.5375320Z         - Example line: "@User1,lf,split_build"
2025-07-24T03:40:54.5376065Z         - A "#" prefix indicates the user is opted out of all experiments
2025-07-24T03:40:54.5376880Z 
2025-07-24T03:40:54.5377239Z 
2025-07-24T03:40:54.5377590Z     """
2025-07-24T03:40:54.5378007Z     optins = UserOptins()
2025-07-24T03:40:54.5378566Z     for user in user_optin_text.split("\n"):
2025-07-24T03:40:54.5379185Z         user = user.strip("\r\n\t -")
2025-07-24T03:40:54.5379797Z         if not user or not user.startswith("@"):
2025-07-24T03:40:54.5380394Z             # Not a valid user. Skip
2025-07-24T03:40:54.5380932Z             continue
2025-07-24T03:40:54.5381496Z 
2025-07-24T03:40:54.5381907Z         if user:
2025-07-24T03:40:54.5382410Z             usr_name = user.split(",")[0].strip("@")
2025-07-24T03:40:54.5383157Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2025-07-24T03:40:54.5383835Z 
2025-07-24T03:40:54.5384218Z     return optins
2025-07-24T03:40:54.5384664Z 
2025-07-24T03:40:54.5385020Z 
2025-07-24T03:40:54.5385551Z def is_valid_experiment_name(experiment_name: str) -> bool:
2025-07-24T03:40:54.5386208Z     """
2025-07-24T03:40:54.5386664Z     Check if the experiment name is valid.
2025-07-24T03:40:54.5387228Z     A valid name:
2025-07-24T03:40:54.5387958Z         - Contains only alphanumeric characters and the special characters "_" & "-"
2025-07-24T03:40:54.5388970Z         - The special characters "_" & "-" shouldn't be the first or last characters
2025-07-24T03:40:54.5389719Z         - Cannot contain spaces
2025-07-24T03:40:54.5390234Z     """
2025-07-24T03:40:54.5390613Z 
2025-07-24T03:40:54.5391209Z     valid_char_regex = r"^[a-zA-Z0-9]([\w-]*[a-zA-Z0-9])?$"
2025-07-24T03:40:54.5391994Z     valid = bool(re.match(valid_char_regex, experiment_name))
2025-07-24T03:40:54.5392776Z 
2025-07-24T03:40:54.5393410Z     if valid:
2025-07-24T03:40:54.5393855Z         return True
2025-07-24T03:40:54.5394304Z 
2025-07-24T03:40:54.5394669Z     log.error(
2025-07-24T03:40:54.5396232Z         f"Invalid experiment name: {experiment_name}. Experiment names should only contain alphanumeric characters, '_', and '-'. They cannot contain spaces, and the special characters '_' and '-' cannot be the first or last characters."
2025-07-24T03:40:54.5397850Z     )
2025-07-24T03:40:54.5398255Z     return False
2025-07-24T03:40:54.5398702Z 
2025-07-24T03:40:54.5399056Z 
2025-07-24T03:40:54.5399609Z def parse_settings_from_text(settings_text: str) -> Settings:
2025-07-24T03:40:54.5400289Z     """
2025-07-24T03:40:54.5400939Z     Parse the experiments from the issue body into a list of ExperimentSettings
2025-07-24T03:40:54.5401802Z     """
2025-07-24T03:40:54.5402210Z     try:
2025-07-24T03:40:54.5402614Z         if settings_text:
2025-07-24T03:40:54.5403427Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2025-07-24T03:40:54.5404282Z             # for easy reading
2025-07-24T03:40:54.5405186Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2025-07-24T03:40:54.5406143Z             #       the backtick character in shell commands.
2025-07-24T03:40:54.5406803Z             backtick = chr(96)  # backtick character
2025-07-24T03:40:54.5407541Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2025-07-24T03:40:54.5408264Z             settings = load_yaml(settings_text)
2025-07-24T03:40:54.5408818Z 
2025-07-24T03:40:54.5409462Z             # For now we just load experiments. We can expand this if/when we add more settings
2025-07-24T03:40:54.5410390Z             experiments = {}
2025-07-24T03:40:54.5410888Z 
2025-07-24T03:40:54.5411598Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2025-07-24T03:40:54.5412412Z                 if not is_valid_experiment_name(exp_name):
2025-07-24T03:40:54.5413591Z                     # Exclude invalid experiments from the list. We log an error, but don't raise an exception so that other experiments can still be processed.
2025-07-24T03:40:54.5414697Z                     continue
2025-07-24T03:40:54.5415192Z 
2025-07-24T03:40:54.5415587Z                 valid_settings = {}
2025-07-24T03:40:54.5416168Z                 for setting in exp_settings:
2025-07-24T03:40:54.5416791Z                     if setting not in Experiment._fields:
2025-07-24T03:40:54.5417403Z                         log.warning(
2025-07-24T03:40:54.5418193Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2025-07-24T03:40:54.5418935Z                         )
2025-07-24T03:40:54.5419418Z                     else:
2025-07-24T03:40:54.5420005Z                         valid_settings[setting] = exp_settings[setting]
2025-07-24T03:40:54.5420608Z 
2025-07-24T03:40:54.5421207Z                 experiments[exp_name] = Experiment(**valid_settings)
2025-07-24T03:40:54.5421900Z             return Settings(experiments)
2025-07-24T03:40:54.5422434Z 
2025-07-24T03:40:54.5422811Z     except Exception:
2025-07-24T03:40:54.5423368Z         log.exception("Failed to parse settings")
2025-07-24T03:40:54.5423936Z 
2025-07-24T03:40:54.5424314Z     return Settings()
2025-07-24T03:40:54.5424778Z 
2025-07-24T03:40:54.5425131Z 
2025-07-24T03:40:54.5425754Z def parse_settings(rollout_state: str) -> Settings:
2025-07-24T03:40:54.5426384Z     """
2025-07-24T03:40:54.5426878Z     Parse settings, if any, from the rollout state.
2025-07-24T03:40:54.5427463Z 
2025-07-24T03:40:54.5428040Z     If the issue body contains "---" then the text above that is the settings
2025-07-24T03:40:54.5428858Z     and the text below is the list of opted in users.
2025-07-24T03:40:54.5429452Z 
2025-07-24T03:40:54.5430092Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2025-07-24T03:40:54.5430865Z     """
2025-07-24T03:40:54.5431584Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2025-07-24T03:40:54.5432408Z     return parse_settings_from_text(settings_text)
2025-07-24T03:40:54.5432994Z 
2025-07-24T03:40:54.5433351Z 
2025-07-24T03:40:54.5433830Z def parse_users(rollout_state: str) -> UserOptins:
2025-07-24T03:40:54.5434437Z     """
2025-07-24T03:40:54.5434877Z     Parse users from the rollout state.
2025-07-24T03:40:54.5435417Z 
2025-07-24T03:40:54.5435779Z     """
2025-07-24T03:40:54.5436378Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2025-07-24T03:40:54.5437177Z     return parse_user_opt_in_from_text(users_text)
2025-07-24T03:40:54.5437764Z 
2025-07-24T03:40:54.5438129Z 
2025-07-24T03:40:54.5438795Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-07-24T03:40:54.5439595Z     """
2025-07-24T03:40:54.5440068Z     Check if a user is opted into an experiment
2025-07-24T03:40:54.5440644Z     """
2025-07-24T03:40:54.5441251Z     return experiment_name in user_optins.get(user, [])
2025-07-24T03:40:54.5441875Z 
2025-07-24T03:40:54.5442232Z 
2025-07-24T03:40:54.5443048Z def is_user_opted_out(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-07-24T03:40:54.5443860Z     """
2025-07-24T03:40:54.5444376Z     Check if a user explicitly opted out of an experiment
2025-07-24T03:40:54.5444998Z     """
2025-07-24T03:40:54.5445558Z     # if the experiment is prefixed with a "-", then it's an opt-out
2025-07-24T03:40:54.5446312Z     experiment_optout = "-" + experiment_name
2025-07-24T03:40:54.5447017Z     if experiment_optout not in user_optins.get(user, []):
2025-07-24T03:40:54.5447699Z         return False
2025-07-24T03:40:54.5448149Z 
2025-07-24T03:40:54.5448647Z     if is_user_opted_in(user, user_optins, experiment_name):
2025-07-24T03:40:54.5449293Z         log.warning(
2025-07-24T03:40:54.5450182Z             f"User {user} is opted into experiment {experiment_name}, but also opted out of it. Defaulting to opting out"
2025-07-24T03:40:54.5451211Z         )
2025-07-24T03:40:54.5451607Z 
2025-07-24T03:40:54.5451985Z     return True
2025-07-24T03:40:54.5452418Z 
2025-07-24T03:40:54.5452775Z 
2025-07-24T03:40:54.5453163Z def get_runner_prefix(
2025-07-24T03:40:54.5453654Z     rollout_state: str,
2025-07-24T03:40:54.5454191Z     workflow_requestors: Iterable[str],
2025-07-24T03:40:54.5454745Z     branch: str,
2025-07-24T03:40:54.5455314Z     eligible_experiments: frozenset[str] = frozenset(),
2025-07-24T03:40:54.5456054Z     opt_out_experiments: frozenset[str] = frozenset(),
2025-07-24T03:40:54.5456688Z     is_canary: bool = False,
2025-07-24T03:40:54.5457191Z ) -> str:
2025-07-24T03:40:54.5457671Z     settings = parse_settings(rollout_state)
2025-07-24T03:40:54.5458311Z     user_optins = parse_users(rollout_state)
2025-07-24T03:40:54.5458871Z 
2025-07-24T03:40:54.5459378Z     fleet_prefix = ""
2025-07-24T03:40:54.5459866Z     prefixes = []
2025-07-24T03:40:54.5460584Z     for experiment_name, experiment_settings in settings.experiments.items():
2025-07-24T03:40:54.5461705Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2025-07-24T03:40:54.5462466Z             log.info(
2025-07-24T03:40:54.5463223Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2025-07-24T03:40:54.5464015Z             )
2025-07-24T03:40:54.5464465Z             continue
2025-07-24T03:40:54.5464914Z 
2025-07-24T03:40:54.5465326Z         if opt_out_experiments:
2025-07-24T03:40:54.5465935Z             if experiment_name in opt_out_experiments:
2025-07-24T03:40:54.5466652Z                 opt_out_exp_list = ", ".join(opt_out_experiments)
2025-07-24T03:40:54.5467298Z                 log.info(
2025-07-24T03:40:54.5468344Z                     f"Skipping experiment '{experiment_name}', as this workflow has opted-out (opted out experiments are: {opt_out_exp_list})"
2025-07-24T03:40:54.5469374Z                 )
2025-07-24T03:40:54.5469836Z                 continue
2025-07-24T03:40:54.5470303Z 
2025-07-24T03:40:54.5470703Z         if eligible_experiments:
2025-07-24T03:40:54.5471432Z             if experiment_name not in eligible_experiments:
2025-07-24T03:40:54.5472125Z                 exp_list = ", ".join(eligible_experiments)
2025-07-24T03:40:54.5472716Z                 log.info(
2025-07-24T03:40:54.5473574Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2025-07-24T03:40:54.5474459Z                 )
2025-07-24T03:40:54.5474907Z                 continue
2025-07-24T03:40:54.5475590Z         elif not experiment_settings.default:
2025-07-24T03:40:54.5476159Z             log.info(
2025-07-24T03:40:54.5476893Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2025-07-24T03:40:54.5477661Z             )
2025-07-24T03:40:54.5478084Z             continue
2025-07-24T03:40:54.5478538Z 
2025-07-24T03:40:54.5479039Z         # Is any workflow_requestor opted out to this experiment?
2025-07-24T03:40:54.5479703Z         opted_out_users = [
2025-07-24T03:40:54.5480213Z             requestor
2025-07-24T03:40:54.5480747Z             for requestor in workflow_requestors
2025-07-24T03:40:54.5481579Z             if is_user_opted_out(requestor, user_optins, experiment_name)
2025-07-24T03:40:54.5482250Z         ]
2025-07-24T03:40:54.5482655Z 
2025-07-24T03:40:54.5483044Z         if opted_out_users:
2025-07-24T03:40:54.5483585Z             log.info(
2025-07-24T03:40:54.5484290Z                 f"{', '.join(opted_out_users)} have opted out of experiment {experiment_name}."
2025-07-24T03:40:54.5485036Z             )
2025-07-24T03:40:54.5485471Z             continue
2025-07-24T03:40:54.5485931Z 
2025-07-24T03:40:54.5486428Z         # Is any workflow_requestor opted in to this experiment?
2025-07-24T03:40:54.5487086Z         opted_in_users = [
2025-07-24T03:40:54.5487609Z             requestor
2025-07-24T03:40:54.5488149Z             for requestor in workflow_requestors
2025-07-24T03:40:54.5488887Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2025-07-24T03:40:54.5489551Z         ]
2025-07-24T03:40:54.5489958Z 
2025-07-24T03:40:54.5490344Z         enabled = False
2025-07-24T03:40:54.5490857Z         if opted_in_users:
2025-07-24T03:40:54.5491636Z             log.info(
2025-07-24T03:40:54.5492344Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2025-07-24T03:40:54.5493072Z             )
2025-07-24T03:40:54.5493519Z             enabled = True
2025-07-24T03:40:54.5494010Z 
2025-07-24T03:40:54.5494453Z         elif experiment_settings.rollout_perc:
2025-07-24T03:40:54.5495353Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2025-07-24T03:40:54.5496368Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2025-07-24T03:40:54.5497068Z                 log.info(
2025-07-24T03:40:54.5498024Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2025-07-24T03:40:54.5498996Z                 )
2025-07-24T03:40:54.5499481Z                 enabled = True
2025-07-24T03:40:54.5499984Z 
2025-07-24T03:40:54.5500379Z         if enabled:
2025-07-24T03:40:54.5500883Z             label = experiment_name
2025-07-24T03:40:54.5501605Z             if experiment_name == LF_FLEET_EXPERIMENT:
2025-07-24T03:40:54.5502500Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2025-07-24T03:40:54.5503444Z                 #  - If it's enabled, then we always list it's prefix first
2025-07-24T03:40:54.5504273Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2025-07-24T03:40:54.5504982Z                 if is_canary:
2025-07-24T03:40:54.5505540Z                     label += CANARY_FLEET_SUFFIX
2025-07-24T03:40:54.5506142Z                 fleet_prefix = label
2025-07-24T03:40:54.5506683Z             else:
2025-07-24T03:40:54.5507320Z                 prefixes.append(label)
2025-07-24T03:40:54.5507855Z 
2025-07-24T03:40:54.5508257Z     if len(prefixes) > 1:
2025-07-24T03:40:54.5508773Z         log.error(
2025-07-24T03:40:54.5509905Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2025-07-24T03:40:54.5511183Z         )
2025-07-24T03:40:54.5511628Z         prefixes = prefixes[:1]
2025-07-24T03:40:54.5512147Z 
2025-07-24T03:40:54.5512541Z     # Fleet always comes first
2025-07-24T03:40:54.5513082Z     if fleet_prefix:
2025-07-24T03:40:54.5513604Z         prefixes.insert(0, fleet_prefix)
2025-07-24T03:40:54.5514152Z 
2025-07-24T03:40:54.5514638Z     return ".".join(prefixes) + "." if prefixes else ""
2025-07-24T03:40:54.5515242Z 
2025-07-24T03:40:54.5515602Z 
2025-07-24T03:40:54.5516296Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2025-07-24T03:40:54.5517115Z     """
2025-07-24T03:40:54.5517758Z     Gets the first comment of the issue, which contains the desired rollout state.
2025-07-24T03:40:54.5518509Z 
2025-07-24T03:40:54.5519138Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2025-07-24T03:40:54.5519881Z     """
2025-07-24T03:40:54.5520326Z     gh = get_gh_client(github_token)
2025-07-24T03:40:54.5520932Z     issue = get_issue(gh, repo, issue_num)
2025-07-24T03:40:54.5521743Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2025-07-24T03:40:54.5522372Z 
2025-07-24T03:40:54.5522740Z 
2025-07-24T03:40:54.5523383Z def download_json(url: str, headers: dict[str, str], num_retries: int = 3) -> Any:
2025-07-24T03:40:54.5524316Z     for _ in range(num_retries):
2025-07-24T03:40:54.5524868Z         try:
2025-07-24T03:40:54.5525363Z             req = Request(url=url, headers=headers)
2025-07-24T03:40:54.5526079Z             content = urlopen(req, timeout=5).read().decode("utf-8")
2025-07-24T03:40:54.5526780Z             return json.loads(content)
2025-07-24T03:40:54.5527355Z         except Exception as e:
2025-07-24T03:40:54.5527973Z             log.warning(f"Could not download {url}: {e}")
2025-07-24T03:40:54.5528562Z 
2025-07-24T03:40:54.5529261Z     log.warning(f"All {num_retries} retries exhausted, downloading {url} failed")
2025-07-24T03:40:54.5530024Z     return {}
2025-07-24T03:40:54.5530451Z 
2025-07-24T03:40:54.5530814Z 
2025-07-24T03:40:54.5531286Z @cache
2025-07-24T03:40:54.5531984Z def get_pr_info(github_repo: str, github_token: str, pr_number: int) -> dict[str, Any]:
2025-07-24T03:40:54.5532805Z     """
2025-07-24T03:40:54.5533257Z     Dynamically get PR information
2025-07-24T03:40:54.5533787Z     """
2025-07-24T03:40:54.5534352Z     github_api = f"https://api.github.com/repos/{github_repo}"
2025-07-24T03:40:54.5535022Z     headers = {
2025-07-24T03:40:54.5535553Z         "Accept": "application/vnd.github.v3+json",
2025-07-24T03:40:54.5536212Z         "Authorization": f"token {github_token}",
2025-07-24T03:40:54.5536789Z     }
2025-07-24T03:40:54.5537282Z     json_response: dict[str, Any] = download_json(
2025-07-24T03:40:54.5537941Z         url=f"{github_api}/issues/{pr_number}",
2025-07-24T03:40:54.5538523Z         headers=headers,
2025-07-24T03:40:54.5539009Z     )
2025-07-24T03:40:54.5539395Z 
2025-07-24T03:40:54.5539784Z     if not json_response:
2025-07-24T03:40:54.5540434Z         log.warning(f"Failed to get the labels for #{pr_number}")
2025-07-24T03:40:54.5541361Z         return {}
2025-07-24T03:40:54.5541813Z 
2025-07-24T03:40:54.5542202Z     return json_response
2025-07-24T03:40:54.5542687Z 
2025-07-24T03:40:54.5543048Z 
2025-07-24T03:40:54.5543677Z def get_labels(github_repo: str, github_token: str, pr_number: int) -> set[str]:
2025-07-24T03:40:54.5544455Z     """
2025-07-24T03:40:54.5545046Z     Dynamically get the latest list of labels from the pull request
2025-07-24T03:40:54.5545741Z     """
2025-07-24T03:40:54.5546290Z     pr_info = get_pr_info(github_repo, github_token, pr_number)
2025-07-24T03:40:54.5546946Z     return {
2025-07-24T03:40:54.5547633Z         label.get("name") for label in pr_info.get("labels", []) if label.get("name")
2025-07-24T03:40:54.5548357Z     }
2025-07-24T03:40:54.5548741Z 
2025-07-24T03:40:54.5549095Z 
2025-07-24T03:40:54.5549494Z def main() -> None:
2025-07-24T03:40:54.5549974Z     args = parse_args()
2025-07-24T03:40:54.5550454Z 
2025-07-24T03:40:54.5550905Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2025-07-24T03:40:54.5551579Z 
2025-07-24T03:40:54.5551986Z     # Check if the PR is opt-out
2025-07-24T03:40:54.5552526Z     if args.pr_number:
2025-07-24T03:40:54.5553267Z         labels = get_labels(args.github_repo, args.github_token, int(args.pr_number))
2025-07-24T03:40:54.5554068Z         if OPT_OUT_LABEL in labels:
2025-07-24T03:40:54.5554618Z             log.info(
2025-07-24T03:40:54.5555390Z                 f"Opt-out runner determinator because #{args.pr_number} has {OPT_OUT_LABEL} label"
2025-07-24T03:40:54.5556194Z             )
2025-07-24T03:40:54.5556822Z             set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-07-24T03:40:54.5557542Z             sys.exit()
2025-07-24T03:40:54.5558144Z 
2025-07-24T03:40:54.5558520Z     try:
2025-07-24T03:40:54.5559024Z         rollout_state = get_rollout_state_from_issue(
2025-07-24T03:40:54.5559799Z             args.github_token, args.github_issue_repo, args.github_issue
2025-07-24T03:40:54.5560480Z         )
2025-07-24T03:40:54.5560881Z 
2025-07-24T03:40:54.5561416Z         username = get_potential_pr_author(
2025-07-24T03:40:54.5562003Z             args.github_token,
2025-07-24T03:40:54.5562542Z             args.github_repo,
2025-07-24T03:40:54.5563077Z             args.github_actor,
2025-07-24T03:40:54.5563618Z             args.github_ref_type,
2025-07-24T03:40:54.5564169Z             args.github_branch,
2025-07-24T03:40:54.5564684Z         )
2025-07-24T03:40:54.5565084Z 
2025-07-24T03:40:54.5565610Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2025-07-24T03:40:54.5566258Z 
2025-07-24T03:40:54.5566712Z         runner_label_prefix = get_runner_prefix(
2025-07-24T03:40:54.5567306Z             rollout_state,
2025-07-24T03:40:54.5567867Z             (args.github_issue_owner, username),
2025-07-24T03:40:54.5568465Z             args.github_branch,
2025-07-24T03:40:54.5569023Z             args.eligible_experiments,
2025-07-24T03:40:54.5569609Z             args.opt_out_experiments,
2025-07-24T03:40:54.5570152Z             is_canary,
2025-07-24T03:40:54.5570622Z         )
2025-07-24T03:40:54.5571112Z 
2025-07-24T03:40:54.5571517Z     except Exception as e:
2025-07-24T03:40:54.5572031Z         log.error(
2025-07-24T03:40:54.5572791Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2025-07-24T03:40:54.5573591Z         )
2025-07-24T03:40:54.5574133Z 
2025-07-24T03:40:54.5574705Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-07-24T03:40:54.5575402Z 
2025-07-24T03:40:54.5575767Z 
2025-07-24T03:40:54.5576151Z if __name__ == "__main__":
2025-07-24T03:40:54.5576649Z     main()
2025-07-24T03:40:54.5577052Z 
2025-07-24T03:40:54.5577422Z EOF
2025-07-24T03:40:54.5577793Z 
2025-07-24T03:40:54.5578196Z cat runner_determinator.py
2025-07-24T03:40:54.5876067Z shell: /usr/bin/bash -e {0}
2025-07-24T03:40:54.5877063Z env:
2025-07-24T03:40:54.5877820Z   GITHUB_TOKEN: ***
2025-07-24T03:40:54.5878255Z   ISSUE_NUMBER: 5132
2025-07-24T03:40:54.5878711Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2025-07-24T03:40:54.5879230Z   ISSUE_OWNER: 
2025-07-24T03:40:54.5879633Z   CHECK_EXPERIMENTS: 
2025-07-24T03:40:54.5880090Z   OPT_OUT_EXPERIMENTS: 
2025-07-24T03:40:54.5880514Z   PR_NUMBER: 
2025-07-24T03:40:54.5880904Z ##[endgroup]
2025-07-24T03:40:54.6083881Z # flake8: noqa: G004
2025-07-24T03:40:54.6084286Z 
2025-07-24T03:40:54.6084723Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2025-07-24T03:40:54.6085706Z #       must be kept in sync. You can do it easily by running the following command:
2025-07-24T03:40:54.6086508Z #           python .github/scripts/update_runner_determinator.py
2025-07-24T03:40:54.6086951Z 
2025-07-24T03:40:54.6087113Z """
2025-07-24T03:40:54.6087684Z This runner determinator is used to determine which set of runners to run a
2025-07-24T03:40:54.6088559Z GitHub job on. It uses the first comment of a GitHub issue (by default
2025-07-24T03:40:54.6089466Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2025-07-24T03:40:54.6090289Z of which runners should be used to run which job.
2025-07-24T03:40:54.6090687Z 
2025-07-24T03:40:54.6091323Z The configuration has two parts, the settings and a list of opted-in users,
2025-07-24T03:40:54.6092437Z separated by a line containing "---".  If the line is not present, the
2025-07-24T03:40:54.6093362Z settings are considered to be empty with only the second part, the user
2025-07-24T03:40:54.6094077Z list, defined.
2025-07-24T03:40:54.6094312Z 
2025-07-24T03:40:54.6094680Z The first part is a YAML block that defines the rollout settings. This can be
2025-07-24T03:40:54.6095616Z used to define any settings that are needed to determine which runners to use.
2025-07-24T03:40:54.6096449Z It's fields are defined by the RolloutSettings class below.
2025-07-24T03:40:54.6096897Z 
2025-07-24T03:40:54.6097263Z The second part is a list of users who are explicitly opted in to the LF fleet.
2025-07-24T03:40:54.6098123Z The user list is also a comma separated list of additional features or
2025-07-24T03:40:54.6098854Z experiments which the user could be opted in to.
2025-07-24T03:40:54.6099254Z 
2025-07-24T03:40:54.6099450Z The user list has the following rules:
2025-07-24T03:40:54.6099802Z 
2025-07-24T03:40:54.6100111Z - Users are GitHub usernames, which must start with the @ prefix
2025-07-24T03:40:54.6100961Z - Each user is also a comma-separated list of features/experiments to enable
2025-07-24T03:40:54.6101942Z - A "#" prefix opts the user out of all experiments
2025-07-24T03:40:54.6102335Z 
2025-07-24T03:40:54.6102508Z Example config:
2025-07-24T03:40:54.6102950Z     # A list of experiments that can be opted into.
2025-07-24T03:40:54.6103609Z     # This defines the behavior they'll induce when opted into.
2025-07-24T03:40:54.6104237Z     # Expected syntax is:
2025-07-24T03:40:54.6104885Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2025-07-24T03:40:54.6105858Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2025-07-24T03:40:54.6106482Z 
2025-07-24T03:40:54.6106651Z     experiments:
2025-07-24T03:40:54.6107040Z       lf:
2025-07-24T03:40:54.6107416Z         rollout_percent: 25
2025-07-24T03:40:54.6107875Z         all_branches: false
2025-07-24T03:40:54.6108499Z         default: true
2025-07-24T03:40:54.6108912Z     ---
2025-07-24T03:40:54.6109116Z 
2025-07-24T03:40:54.6109278Z     # Opt-ins:
2025-07-24T03:40:54.6109862Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2025-07-24T03:40:54.6110714Z     # and specifying experiments to enable in a comma-separated list.
2025-07-24T03:40:54.6111756Z     # To always opt out of an experiment, prefix it with a "-".
2025-07-24T03:40:54.6112438Z     # Experiments should be from the above list.
2025-07-24T03:40:54.6112824Z 
2025-07-24T03:40:54.6113008Z     @User1,-lf,split_build
2025-07-24T03:40:54.6113461Z     @User2,lf
2025-07-24T03:40:54.6113846Z     @User3,split_build
2025-07-24T03:40:54.6114265Z """
2025-07-24T03:40:54.6114454Z 
2025-07-24T03:40:54.6114617Z import json
2025-07-24T03:40:54.6115003Z import logging
2025-07-24T03:40:54.6115386Z import os
2025-07-24T03:40:54.6115759Z import random
2025-07-24T03:40:54.6116143Z import re
2025-07-24T03:40:54.6116507Z import sys
2025-07-24T03:40:54.6116928Z from argparse import ArgumentParser
2025-07-24T03:40:54.6117475Z from collections.abc import Iterable
2025-07-24T03:40:54.6118008Z from functools import cache
2025-07-24T03:40:54.6118475Z from logging import LogRecord
2025-07-24T03:40:54.6118970Z from typing import Any, NamedTuple
2025-07-24T03:40:54.6119495Z from urllib.request import Request, urlopen
2025-07-24T03:40:54.6119869Z 
2025-07-24T03:40:54.6120035Z import yaml
2025-07-24T03:40:54.6120424Z from github import Auth, Github
2025-07-24T03:40:54.6121697Z from github.Issue import Issue
2025-07-24T03:40:54.6122071Z 
2025-07-24T03:40:54.6122078Z 
2025-07-24T03:40:54.6122319Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2025-07-24T03:40:54.6123010Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2025-07-24T03:40:54.6123875Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2025-07-24T03:40:54.6124456Z 
2025-07-24T03:40:54.6124685Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2025-07-24T03:40:54.6125430Z GH_OUTPUT_KEY_AMI = "runner-ami"
2025-07-24T03:40:54.6125954Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2025-07-24T03:40:54.6126510Z OPT_OUT_LABEL = "no-runner-experiments"
2025-07-24T03:40:54.6126860Z 
2025-07-24T03:40:54.6127055Z SETTING_EXPERIMENTS = "experiments"
2025-07-24T03:40:54.6127392Z 
2025-07-24T03:40:54.6127578Z LF_FLEET_EXPERIMENT = "lf"
2025-07-24T03:40:54.6128040Z CANARY_FLEET_SUFFIX = ".c"
2025-07-24T03:40:54.6128318Z 
2025-07-24T03:40:54.6128324Z 
2025-07-24T03:40:54.6128511Z class Experiment(NamedTuple):
2025-07-24T03:40:54.6128994Z     rollout_perc: float = (
2025-07-24T03:40:54.6129625Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2025-07-24T03:40:54.6130313Z     )
2025-07-24T03:40:54.6130692Z     all_branches: bool = (
2025-07-24T03:40:54.6131556Z         False  # If True, the experiment is also enabled on the exception branches
2025-07-24T03:40:54.6132263Z     )
2025-07-24T03:40:54.6132621Z     default: bool = (
2025-07-24T03:40:54.6133201Z         True  # If True, the experiment is enabled by default for all queries
2025-07-24T03:40:54.6133884Z     )
2025-07-24T03:40:54.6134091Z 
2025-07-24T03:40:54.6134271Z     # Add more fields as needed
2025-07-24T03:40:54.6134568Z 
2025-07-24T03:40:54.6134574Z 
2025-07-24T03:40:54.6134766Z class Settings(NamedTuple):
2025-07-24T03:40:54.6135206Z     """
2025-07-24T03:40:54.6135662Z     Settings for the experiments that can be opted into.
2025-07-24T03:40:54.6136231Z     """
2025-07-24T03:40:54.6136427Z 
2025-07-24T03:40:54.6136638Z     experiments: dict[str, Experiment] = {}
2025-07-24T03:40:54.6136997Z 
2025-07-24T03:40:54.6137003Z 
2025-07-24T03:40:54.6137207Z class ColorFormatter(logging.Formatter):
2025-07-24T03:40:54.6137822Z     """Color codes the log messages based on the log level"""
2025-07-24T03:40:54.6138257Z 
2025-07-24T03:40:54.6138424Z     COLORS = {
2025-07-24T03:40:54.6138818Z         "WARNING": "\033[33m",  # Yellow
2025-07-24T03:40:54.6139325Z         "ERROR": "\033[31m",  # Red
2025-07-24T03:40:54.6139971Z         "CRITICAL": "\033[31m",  # Red
2025-07-24T03:40:54.6140473Z         "INFO": "\033[0m",  # Reset
2025-07-24T03:40:54.6140955Z         "DEBUG": "\033[0m",  # Reset
2025-07-24T03:40:54.6141618Z     }
2025-07-24T03:40:54.6141821Z 
2025-07-24T03:40:54.6142049Z     def format(self, record: LogRecord) -> str:
2025-07-24T03:40:54.6142804Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2025-07-24T03:40:54.6143587Z         record.msg = f"{log_color}{record.msg}\033[0m"
2025-07-24T03:40:54.6144161Z         return super().format(record)
2025-07-24T03:40:54.6144490Z 
2025-07-24T03:40:54.6144497Z 
2025-07-24T03:40:54.6144696Z handler = logging.StreamHandler()
2025-07-24T03:40:54.6145386Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2025-07-24T03:40:54.6145953Z 
2025-07-24T03:40:54.6146193Z log = logging.getLogger(os.path.basename(__file__))
2025-07-24T03:40:54.6146769Z log.addHandler(handler)
2025-07-24T03:40:54.6147222Z log.setLevel(logging.INFO)
2025-07-24T03:40:54.6147509Z 
2025-07-24T03:40:54.6147516Z 
2025-07-24T03:40:54.6147762Z def set_github_output(key: str, value: str) -> None:
2025-07-24T03:40:54.6148313Z     """
2025-07-24T03:40:54.6148811Z     Defines outputs of the github action that invokes this script
2025-07-24T03:40:54.6149437Z     """
2025-07-24T03:40:54.6149802Z     if not GITHUB_OUTPUT:
2025-07-24T03:40:54.6150873Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2025-07-24T03:40:54.6152283Z         log.warning(
2025-07-24T03:40:54.6153139Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2025-07-24T03:40:54.6154069Z         )
2025-07-24T03:40:54.6164280Z         print(f"::set-output name={key}::{value}")
2025-07-24T03:40:54.6164892Z         return
2025-07-24T03:40:54.6165132Z 
2025-07-24T03:40:54.6165513Z     with open(GITHUB_OUTPUT, "a") as f:
2025-07-24T03:40:54.6166127Z         log.info(f"Setting output: {key}='{value}'")
2025-07-24T03:40:54.6166705Z         f.write(f"{key}={value}\n")
2025-07-24T03:40:54.6167033Z 
2025-07-24T03:40:54.6167040Z 
2025-07-24T03:40:54.6167351Z def _str_comma_separated_to_set(value: str) -> frozenset[str]:
2025-07-24T03:40:54.6167989Z     return frozenset(
2025-07-24T03:40:54.6168600Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2025-07-24T03:40:54.6169285Z     )
2025-07-24T03:40:54.6169489Z 
2025-07-24T03:40:54.6169496Z 
2025-07-24T03:40:54.6169674Z def parse_args() -> Any:
2025-07-24T03:40:54.6170216Z     parser = ArgumentParser("Get dynamic rollout settings")
2025-07-24T03:40:54.6171351Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2025-07-24T03:40:54.6172144Z     parser.add_argument(
2025-07-24T03:40:54.6172595Z         "--github-issue-repo",
2025-07-24T03:40:54.6173053Z         type=str,
2025-07-24T03:40:54.6173467Z         required=False,
2025-07-24T03:40:54.6173913Z         default="pytorch/test-infra",
2025-07-24T03:40:54.6174446Z         help="GitHub repo to get the issue",
2025-07-24T03:40:54.6174952Z     )
2025-07-24T03:40:54.6175312Z     parser.add_argument(
2025-07-24T03:40:54.6175754Z         "--github-repo",
2025-07-24T03:40:54.6176171Z         type=str,
2025-07-24T03:40:54.6176562Z         required=True,
2025-07-24T03:40:54.6177019Z         help="GitHub repo where CI is running",
2025-07-24T03:40:54.6177545Z     )
2025-07-24T03:40:54.6177909Z     parser.add_argument(
2025-07-24T03:40:54.6178507Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2025-07-24T03:40:54.6179169Z     )
2025-07-24T03:40:54.6179524Z     parser.add_argument(
2025-07-24T03:40:54.6180129Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2025-07-24T03:40:54.6180800Z     )
2025-07-24T03:40:54.6181348Z     parser.add_argument(
2025-07-24T03:40:54.6181995Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2025-07-24T03:40:54.6182860Z     )
2025-07-24T03:40:54.6183227Z     parser.add_argument(
2025-07-24T03:40:54.6183869Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2025-07-24T03:40:54.6184596Z     )
2025-07-24T03:40:54.6184963Z     parser.add_argument(
2025-07-24T03:40:54.6185415Z         "--github-ref-type",
2025-07-24T03:40:54.6185860Z         type=str,
2025-07-24T03:40:54.6186257Z         required=True,
2025-07-24T03:40:54.6186723Z         help="Current GitHub ref type, branch or tag",
2025-07-24T03:40:54.6187297Z     )
2025-07-24T03:40:54.6187659Z     parser.add_argument(
2025-07-24T03:40:54.6188119Z         "--eligible-experiments",
2025-07-24T03:40:54.6188626Z         type=_str_comma_separated_to_set,
2025-07-24T03:40:54.6189144Z         required=False,
2025-07-24T03:40:54.6189556Z         default="",
2025-07-24T03:40:54.6190404Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2025-07-24T03:40:54.6191619Z     )
2025-07-24T03:40:54.6191987Z     parser.add_argument(
2025-07-24T03:40:54.6192439Z         "--opt-out-experiments",
2025-07-24T03:40:54.6192940Z         type=_str_comma_separated_to_set,
2025-07-24T03:40:54.6193457Z         required=False,
2025-07-24T03:40:54.6193870Z         default="",
2025-07-24T03:40:54.6194256Z         help=(
2025-07-24T03:40:54.6194922Z             "comma separated list of experiments to opt-out of. If unset, no opt-outs will occur. "
2025-07-24T03:40:54.6196056Z             "If the same experiment is listed both here and in '--eligible-experiments' opt-out will take priority."
2025-07-24T03:40:54.6246547Z         ),
2025-07-24T03:40:54.6247086Z     )
2025-07-24T03:40:54.6247533Z     parser.add_argument(
2025-07-24T03:40:54.6247995Z         "--pr-number",
2025-07-24T03:40:54.6248442Z         type=str,
2025-07-24T03:40:54.6249062Z         required=False,
2025-07-24T03:40:54.6249527Z         default="",
2025-07-24T03:40:54.6250239Z         help="the optional PR number where this is run",
2025-07-24T03:40:54.6250865Z     )
2025-07-24T03:40:54.6251205Z 
2025-07-24T03:40:54.6251409Z     return parser.parse_args()
2025-07-24T03:40:54.6251727Z 
2025-07-24T03:40:54.6251733Z 
2025-07-24T03:40:54.6252141Z def get_gh_client(github_token: str) -> Github:  # type: ignore[no-any-unimported]
2025-07-24T03:40:54.6252930Z     auth = Auth.Token(github_token)
2025-07-24T03:40:54.6253432Z     return Github(auth=auth)
2025-07-24T03:40:54.6253745Z 
2025-07-24T03:40:54.6253752Z 
2025-07-24T03:40:54.6254209Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:  # type: ignore[no-any-unimported]
2025-07-24T03:40:54.6255020Z     repo = gh.get_repo(repo)
2025-07-24T03:40:54.6255519Z     return repo.get_issue(number=issue_num)
2025-07-24T03:40:54.6255878Z 
2025-07-24T03:40:54.6255885Z 
2025-07-24T03:40:54.6256082Z def get_potential_pr_author(
2025-07-24T03:40:54.6256725Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2025-07-24T03:40:54.6257410Z ) -> str:
2025-07-24T03:40:54.6258156Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2025-07-24T03:40:54.6259025Z     # Fetch the actual username from the original PR. The PR number is
2025-07-24T03:40:54.6259783Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2025-07-24T03:40:54.6260203Z 
2025-07-24T03:40:54.6260391Z     gh = get_gh_client(github_token)
2025-07-24T03:40:54.6260727Z 
2025-07-24T03:40:54.6261001Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2025-07-24T03:40:54.6261871Z         split_tag = ref_name.split("/")
2025-07-24T03:40:54.6262427Z         if (
2025-07-24T03:40:54.6262817Z             len(split_tag) == 3
2025-07-24T03:40:54.6263291Z             and split_tag[0] == "ciflow"
2025-07-24T03:40:54.6263807Z             and split_tag[2].isnumeric()
2025-07-24T03:40:54.6264307Z         ):
2025-07-24T03:40:54.6264693Z             pr_number = split_tag[2]
2025-07-24T03:40:54.6265344Z             try:
2025-07-24T03:40:54.6265787Z                 repository = gh.get_repo(repo)
2025-07-24T03:40:54.6266384Z                 pull = repository.get_pull(number=int(pr_number))
2025-07-24T03:40:54.6266991Z             except Exception as e:
2025-07-24T03:40:54.6267501Z                 raise Exception(  # noqa: TRY002
2025-07-24T03:40:54.6268167Z                     f"issue with pull request {pr_number} from repo {repository}"
2025-07-24T03:40:54.6268803Z                 ) from e
2025-07-24T03:40:54.6269337Z             return pull.user.login  # type: ignore[no-any-return]
2025-07-24T03:40:54.6270026Z     # In all other cases, return the original input username
2025-07-24T03:40:54.6270641Z     return username
2025-07-24T03:40:54.6270933Z 
2025-07-24T03:40:54.6270940Z 
2025-07-24T03:40:54.6271507Z def is_exception_branch(branch: str) -> bool:
2025-07-24T03:40:54.6272081Z     """
2025-07-24T03:40:54.6272720Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2025-07-24T03:40:54.6273511Z     """
2025-07-24T03:40:54.6274052Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2025-07-24T03:40:54.6274580Z 
2025-07-24T03:40:54.6274587Z 
2025-07-24T03:40:54.6274782Z def load_yaml(yaml_text: str) -> Any:
2025-07-24T03:40:54.6275273Z     try:
2025-07-24T03:40:54.6275658Z         data = yaml.safe_load(yaml_text)
2025-07-24T03:40:54.6276166Z         return data
2025-07-24T03:40:54.6276567Z     except yaml.YAMLError:
2025-07-24T03:40:54.6277044Z         log.exception("Error loading YAML")
2025-07-24T03:40:54.6277548Z         raise
2025-07-24T03:40:54.6277768Z 
2025-07-24T03:40:54.6277776Z 
2025-07-24T03:40:54.6278196Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> tuple[str, str]:
2025-07-24T03:40:54.6278935Z     """
2025-07-24T03:40:54.6279563Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2025-07-24T03:40:54.6280158Z 
2025-07-24T03:40:54.6280648Z     If the issue body contains "---" then the text above that is the settings
2025-07-24T03:40:54.6281575Z     and the text below is the list of opted in users.
2025-07-24T03:40:54.6281976Z 
2025-07-24T03:40:54.6282353Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2025-07-24T03:40:54.6283048Z     """
2025-07-24T03:40:54.6283488Z     rollout_state_parts = rollout_state.split("---")
2025-07-24T03:40:54.6284123Z     if len(rollout_state_parts) >= 2:
2025-07-24T03:40:54.6284719Z         return rollout_state_parts[0], rollout_state_parts[1]
2025-07-24T03:40:54.6285304Z     else:
2025-07-24T03:40:54.6285688Z         return "", rollout_state
2025-07-24T03:40:54.6285993Z 
2025-07-24T03:40:54.6286000Z 
2025-07-24T03:40:54.6286202Z class UserOptins(dict[str, list[str]]):
2025-07-24T03:40:54.6286702Z     """
2025-07-24T03:40:54.6287218Z     Dictionary of users with a list of features they have opted into
2025-07-24T03:40:54.6287860Z     """
2025-07-24T03:40:54.6288062Z 
2025-07-24T03:40:54.6288069Z 
2025-07-24T03:40:54.6288412Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2025-07-24T03:40:54.6289065Z     """
2025-07-24T03:40:54.6289757Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2025-07-24T03:40:54.6290439Z 
2025-07-24T03:40:54.6291189Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2025-07-24T03:40:54.6292233Z         - Example line: "@User1,lf,split_build"
2025-07-24T03:40:54.6292911Z         - A "#" prefix indicates the user is opted out of all experiments
2025-07-24T03:40:54.6293390Z 
2025-07-24T03:40:54.6293397Z 
2025-07-24T03:40:54.6293553Z     """
2025-07-24T03:40:54.6293913Z     optins = UserOptins()
2025-07-24T03:40:54.6294391Z     for user in user_optin_text.split("\n"):
2025-07-24T03:40:54.6294949Z         user = user.strip("\r\n\t -")
2025-07-24T03:40:54.6295489Z         if not user or not user.startswith("@"):
2025-07-24T03:40:54.6296221Z             # Not a valid user. Skip
2025-07-24T03:40:54.6296725Z             continue
2025-07-24T03:40:54.6296967Z 
2025-07-24T03:40:54.6297124Z         if user:
2025-07-24T03:40:54.6297555Z             usr_name = user.split(",")[0].strip("@")
2025-07-24T03:40:54.6298238Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2025-07-24T03:40:54.6298726Z 
2025-07-24T03:40:54.6298888Z     return optins
2025-07-24T03:40:54.6299121Z 
2025-07-24T03:40:54.6299128Z 
2025-07-24T03:40:54.6299409Z def is_valid_experiment_name(experiment_name: str) -> bool:
2025-07-24T03:40:54.6300002Z     """
2025-07-24T03:40:54.6300390Z     Check if the experiment name is valid.
2025-07-24T03:40:54.6300893Z     A valid name:
2025-07-24T03:40:54.6301636Z         - Contains only alphanumeric characters and the special characters "_" & "-"
2025-07-24T03:40:54.6302601Z         - The special characters "_" & "-" shouldn't be the first or last characters
2025-07-24T03:40:54.6303330Z         - Cannot contain spaces
2025-07-24T03:40:54.6303797Z     """
2025-07-24T03:40:54.6303995Z 
2025-07-24T03:40:54.6304251Z     valid_char_regex = r"^[a-zA-Z0-9]([\w-]*[a-zA-Z0-9])?$"
2025-07-24T03:40:54.6304950Z     valid = bool(re.match(valid_char_regex, experiment_name))
2025-07-24T03:40:54.6305391Z 
2025-07-24T03:40:54.6305552Z     if valid:
2025-07-24T03:40:54.6305930Z         return True
2025-07-24T03:40:54.6306164Z 
2025-07-24T03:40:54.6306329Z     log.error(
2025-07-24T03:40:54.6307800Z         f"Invalid experiment name: {experiment_name}. Experiment names should only contain alphanumeric characters, '_', and '-'. They cannot contain spaces, and the special characters '_' and '-' cannot be the first or last characters."
2025-07-24T03:40:54.6309383Z     )
2025-07-24T03:40:54.6309729Z     return False
2025-07-24T03:40:54.6309961Z 
2025-07-24T03:40:54.6309968Z 
2025-07-24T03:40:54.6310257Z def parse_settings_from_text(settings_text: str) -> Settings:
2025-07-24T03:40:54.6310872Z     """
2025-07-24T03:40:54.6311897Z     Parse the experiments from the issue body into a list of ExperimentSettings
2025-07-24T03:40:54.6312664Z     """
2025-07-24T03:40:54.6313015Z     try:
2025-07-24T03:40:54.6313390Z         if settings_text:
2025-07-24T03:40:54.6314104Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2025-07-24T03:40:54.6314898Z             # for easy reading
2025-07-24T03:40:54.6315668Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2025-07-24T03:40:54.6316553Z             #       the backtick character in shell commands.
2025-07-24T03:40:54.6317147Z             backtick = chr(96)  # backtick character
2025-07-24T03:40:54.6317801Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2025-07-24T03:40:54.6318464Z             settings = load_yaml(settings_text)
2025-07-24T03:40:54.6318844Z 
2025-07-24T03:40:54.6319249Z             # For now we just load experiments. We can expand this if/when we add more settings
2025-07-24T03:40:54.6320015Z             experiments = {}
2025-07-24T03:40:54.6320313Z 
2025-07-24T03:40:54.6320699Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2025-07-24T03:40:54.6321586Z                 if not is_valid_experiment_name(exp_name):
2025-07-24T03:40:54.6322699Z                     # Exclude invalid experiments from the list. We log an error, but don't raise an exception so that other experiments can still be processed.
2025-07-24T03:40:54.6323747Z                     continue
2025-07-24T03:40:54.6324035Z 
2025-07-24T03:40:54.6324217Z                 valid_settings = {}
2025-07-24T03:40:54.6324722Z                 for setting in exp_settings:
2025-07-24T03:40:54.6325280Z                     if setting not in Experiment._fields:
2025-07-24T03:40:54.6325835Z                         log.warning(
2025-07-24T03:40:54.6326521Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2025-07-24T03:40:54.6327373Z                         )
2025-07-24T03:40:54.6327785Z                     else:
2025-07-24T03:40:54.6328286Z                         valid_settings[setting] = exp_settings[setting]
2025-07-24T03:40:54.6328709Z 
2025-07-24T03:40:54.6328979Z                 experiments[exp_name] = Experiment(**valid_settings)
2025-07-24T03:40:54.6329603Z             return Settings(experiments)
2025-07-24T03:40:54.6329946Z 
2025-07-24T03:40:54.6330123Z     except Exception:
2025-07-24T03:40:54.6330584Z         log.exception("Failed to parse settings")
2025-07-24T03:40:54.6330970Z 
2025-07-24T03:40:54.6331303Z     return Settings()
2025-07-24T03:40:54.6331562Z 
2025-07-24T03:40:54.6331569Z 
2025-07-24T03:40:54.6331814Z def parse_settings(rollout_state: str) -> Settings:
2025-07-24T03:40:54.6332377Z     """
2025-07-24T03:40:54.6332804Z     Parse settings, if any, from the rollout state.
2025-07-24T03:40:54.6333209Z 
2025-07-24T03:40:54.6333576Z     If the issue body contains "---" then the text above that is the settings
2025-07-24T03:40:54.6334337Z     and the text below is the list of opted in users.
2025-07-24T03:40:54.6334737Z 
2025-07-24T03:40:54.6335139Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2025-07-24T03:40:54.6335865Z     """
2025-07-24T03:40:54.6336406Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2025-07-24T03:40:54.6337154Z     return parse_settings_from_text(settings_text)
2025-07-24T03:40:54.6337548Z 
2025-07-24T03:40:54.6337555Z 
2025-07-24T03:40:54.6337801Z def parse_users(rollout_state: str) -> UserOptins:
2025-07-24T03:40:54.6338351Z     """
2025-07-24T03:40:54.6338739Z     Parse users from the rollout state.
2025-07-24T03:40:54.6339084Z 
2025-07-24T03:40:54.6339238Z     """
2025-07-24T03:40:54.6339752Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2025-07-24T03:40:54.6340499Z     return parse_user_opt_in_from_text(users_text)
2025-07-24T03:40:54.6340904Z 
2025-07-24T03:40:54.6340911Z 
2025-07-24T03:40:54.6341592Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-07-24T03:40:54.6342357Z     """
2025-07-24T03:40:54.6342764Z     Check if a user is opted into an experiment
2025-07-24T03:40:54.6343299Z     """
2025-07-24T03:40:54.6343739Z     return experiment_name in user_optins.get(user, [])
2025-07-24T03:40:54.6344157Z 
2025-07-24T03:40:54.6344171Z 
2025-07-24T03:40:54.6344581Z def is_user_opted_out(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-07-24T03:40:54.6345323Z     """
2025-07-24T03:40:54.6345775Z     Check if a user explicitly opted out of an experiment
2025-07-24T03:40:54.6346335Z     """
2025-07-24T03:40:54.6346834Z     # if the experiment is prefixed with a "-", then it's an opt-out
2025-07-24T03:40:54.6347516Z     experiment_optout = "-" + experiment_name
2025-07-24T03:40:54.6348159Z     if experiment_optout not in user_optins.get(user, []):
2025-07-24T03:40:54.6348760Z         return False
2025-07-24T03:40:54.6349020Z 
2025-07-24T03:40:54.6349286Z     if is_user_opted_in(user, user_optins, experiment_name):
2025-07-24T03:40:54.6349889Z         log.warning(
2025-07-24T03:40:54.6350695Z             f"User {user} is opted into experiment {experiment_name}, but also opted out of it. Defaulting to opting out"
2025-07-24T03:40:54.6351898Z         )
2025-07-24T03:40:54.6352111Z 
2025-07-24T03:40:54.6352283Z     return True
2025-07-24T03:40:54.6352512Z 
2025-07-24T03:40:54.6352520Z 
2025-07-24T03:40:54.6352697Z def get_runner_prefix(
2025-07-24T03:40:54.6353132Z     rollout_state: str,
2025-07-24T03:40:54.6353588Z     workflow_requestors: Iterable[str],
2025-07-24T03:40:54.6354112Z     branch: str,
2025-07-24T03:40:54.6354586Z     eligible_experiments: frozenset[str] = frozenset(),
2025-07-24T03:40:54.6355244Z     opt_out_experiments: frozenset[str] = frozenset(),
2025-07-24T03:40:54.6355824Z     is_canary: bool = False,
2025-07-24T03:40:54.6356272Z ) -> str:
2025-07-24T03:40:54.6356682Z     settings = parse_settings(rollout_state)
2025-07-24T03:40:54.6357417Z     user_optins = parse_users(rollout_state)
2025-07-24T03:40:54.6357786Z 
2025-07-24T03:40:54.6357962Z     fleet_prefix = ""
2025-07-24T03:40:54.6358368Z     prefixes = []
2025-07-24T03:40:54.6358981Z     for experiment_name, experiment_settings in settings.experiments.items():
2025-07-24T03:40:54.6359907Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2025-07-24T03:40:54.6360614Z             log.info(
2025-07-24T03:40:54.6361739Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2025-07-24T03:40:54.6362508Z             )
2025-07-24T03:40:54.6362870Z             continue
2025-07-24T03:40:54.6363122Z 
2025-07-24T03:40:54.6363303Z         if opt_out_experiments:
2025-07-24T03:40:54.6363826Z             if experiment_name in opt_out_experiments:
2025-07-24T03:40:54.6364465Z                 opt_out_exp_list = ", ".join(opt_out_experiments)
2025-07-24T03:40:54.6365034Z                 log.info(
2025-07-24T03:40:54.6365957Z                     f"Skipping experiment '{experiment_name}', as this workflow has opted-out (opted out experiments are: {opt_out_exp_list})"
2025-07-24T03:40:54.6366946Z                 )
2025-07-24T03:40:54.6367330Z                 continue
2025-07-24T03:40:54.6367600Z 
2025-07-24T03:40:54.6367785Z         if eligible_experiments:
2025-07-24T03:40:54.6368353Z             if experiment_name not in eligible_experiments:
2025-07-24T03:40:54.6368988Z                 exp_list = ", ".join(eligible_experiments)
2025-07-24T03:40:54.6369542Z                 log.info(
2025-07-24T03:40:54.6370302Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2025-07-24T03:40:54.6371281Z                 )
2025-07-24T03:40:54.6371674Z                 continue
2025-07-24T03:40:54.6372135Z         elif not experiment_settings.default:
2025-07-24T03:40:54.6372657Z             log.info(
2025-07-24T03:40:54.6373466Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2025-07-24T03:40:54.6374228Z             )
2025-07-24T03:40:54.6374593Z             continue
2025-07-24T03:40:54.6374834Z 
2025-07-24T03:40:54.6375101Z         # Is any workflow_requestor opted out to this experiment?
2025-07-24T03:40:54.6375709Z         opted_out_users = [
2025-07-24T03:40:54.6376142Z             requestor
2025-07-24T03:40:54.6376575Z             for requestor in workflow_requestors
2025-07-24T03:40:54.6377231Z             if is_user_opted_out(requestor, user_optins, experiment_name)
2025-07-24T03:40:54.6377844Z         ]
2025-07-24T03:40:54.6378047Z 
2025-07-24T03:40:54.6378224Z         if opted_out_users:
2025-07-24T03:40:54.6378656Z             log.info(
2025-07-24T03:40:54.6379269Z                 f"{', '.join(opted_out_users)} have opted out of experiment {experiment_name}."
2025-07-24T03:40:54.6379965Z             )
2025-07-24T03:40:54.6380326Z             continue
2025-07-24T03:40:54.6380568Z 
2025-07-24T03:40:54.6380853Z         # Is any workflow_requestor opted in to this experiment?
2025-07-24T03:40:54.6381592Z         opted_in_users = [
2025-07-24T03:40:54.6382036Z             requestor
2025-07-24T03:40:54.6382481Z             for requestor in workflow_requestors
2025-07-24T03:40:54.6383145Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2025-07-24T03:40:54.6383762Z         ]
2025-07-24T03:40:54.6383963Z 
2025-07-24T03:40:54.6384132Z         enabled = False
2025-07-24T03:40:54.6384563Z         if opted_in_users:
2025-07-24T03:40:54.6384998Z             log.info(
2025-07-24T03:40:54.6385597Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2025-07-24T03:40:54.6386274Z             )
2025-07-24T03:40:54.6386652Z             enabled = True
2025-07-24T03:40:54.6386929Z 
2025-07-24T03:40:54.6387136Z         elif experiment_settings.rollout_perc:
2025-07-24T03:40:54.6387976Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2025-07-24T03:40:54.6389050Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2025-07-24T03:40:54.6389692Z                 log.info(
2025-07-24T03:40:54.6390561Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2025-07-24T03:40:54.6391595Z                 )
2025-07-24T03:40:54.6391990Z                 enabled = True
2025-07-24T03:40:54.6392286Z 
2025-07-24T03:40:54.6392445Z         if enabled:
2025-07-24T03:40:54.6392855Z             label = experiment_name
2025-07-24T03:40:54.6393394Z             if experiment_name == LF_FLEET_EXPERIMENT:
2025-07-24T03:40:54.6394213Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2025-07-24T03:40:54.6395100Z                 #  - If it's enabled, then we always list it's prefix first
2025-07-24T03:40:54.6395852Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2025-07-24T03:40:54.6396524Z                 if is_canary:
2025-07-24T03:40:54.6397001Z                     label += CANARY_FLEET_SUFFIX
2025-07-24T03:40:54.6397545Z                 fleet_prefix = label
2025-07-24T03:40:54.6398019Z             else:
2025-07-24T03:40:54.6398433Z                 prefixes.append(label)
2025-07-24T03:40:54.6398776Z 
2025-07-24T03:40:54.6398956Z     if len(prefixes) > 1:
2025-07-24T03:40:54.6399387Z         log.error(
2025-07-24T03:40:54.6400429Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2025-07-24T03:40:54.6401661Z         )
2025-07-24T03:40:54.6402049Z         prefixes = prefixes[:1]
2025-07-24T03:40:54.6402357Z 
2025-07-24T03:40:54.6402539Z     # Fleet always comes first
2025-07-24T03:40:54.6403002Z     if fleet_prefix:
2025-07-24T03:40:54.6403452Z         prefixes.insert(0, fleet_prefix)
2025-07-24T03:40:54.6403804Z 
2025-07-24T03:40:54.6404210Z     return ".".join(prefixes) + "." if prefixes else ""
2025-07-24T03:40:54.6404635Z 
2025-07-24T03:40:54.6404653Z 
2025-07-24T03:40:54.6405093Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2025-07-24T03:40:54.6405865Z     """
2025-07-24T03:40:54.6406442Z     Gets the first comment of the issue, which contains the desired rollout state.
2025-07-24T03:40:54.6407000Z 
2025-07-24T03:40:54.6407376Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2025-07-24T03:40:54.6408073Z     """
2025-07-24T03:40:54.6408453Z     gh = get_gh_client(github_token)
2025-07-24T03:40:54.6408976Z     issue = get_issue(gh, repo, issue_num)
2025-07-24T03:40:54.6409600Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2025-07-24T03:40:54.6410045Z 
2025-07-24T03:40:54.6410052Z 
2025-07-24T03:40:54.6410443Z def download_json(url: str, headers: dict[str, str], num_retries: int = 3) -> Any:
2025-07-24T03:40:54.6411308Z     for _ in range(num_retries):
2025-07-24T03:40:54.6411789Z         try:
2025-07-24T03:40:54.6412203Z             req = Request(url=url, headers=headers)
2025-07-24T03:40:54.6412859Z             content = urlopen(req, timeout=5).read().decode("utf-8")
2025-07-24T03:40:54.6413493Z             return json.loads(content)
2025-07-24T03:40:54.6414009Z         except Exception as e:
2025-07-24T03:40:54.6414528Z             log.warning(f"Could not download {url}: {e}")
2025-07-24T03:40:54.6414934Z 
2025-07-24T03:40:54.6415306Z     log.warning(f"All {num_retries} retries exhausted, downloading {url} failed")
2025-07-24T03:40:54.6416013Z     return {}
2025-07-24T03:40:54.6416234Z 
2025-07-24T03:40:54.6416240Z 
2025-07-24T03:40:54.6416393Z @cache
2025-07-24T03:40:54.6417010Z def get_pr_info(github_repo: str, github_token: str, pr_number: int) -> dict[str, Any]:
2025-07-24T03:40:54.6417758Z     """
2025-07-24T03:40:54.6418138Z     Dynamically get PR information
2025-07-24T03:40:54.6418612Z     """
2025-07-24T03:40:54.6419292Z     github_api = f"https://api.github.com/repos/{github_repo}"
2025-07-24T03:40:54.6420227Z     headers = {
2025-07-24T03:40:54.6420680Z         "Accept": "application/vnd.github.v3+json",
2025-07-24T03:40:54.6421416Z         "Authorization": f"token {github_token}",
2025-07-24T03:40:54.6421992Z     }
2025-07-24T03:40:54.6422425Z     json_response: dict[str, Any] = download_json(
2025-07-24T03:40:54.6423022Z         url=f"{github_api}/issues/{pr_number}",
2025-07-24T03:40:54.6423564Z         headers=headers,
2025-07-24T03:40:54.6423976Z     )
2025-07-24T03:40:54.6424170Z 
2025-07-24T03:40:54.6424354Z     if not json_response:
2025-07-24T03:40:54.6424910Z         log.warning(f"Failed to get the labels for #{pr_number}")
2025-07-24T03:40:54.6425525Z         return {}
2025-07-24T03:40:54.6425755Z 
2025-07-24T03:40:54.6425930Z     return json_response
2025-07-24T03:40:54.6426207Z 
2025-07-24T03:40:54.6426214Z 
2025-07-24T03:40:54.6426604Z def get_labels(github_repo: str, github_token: str, pr_number: int) -> set[str]:
2025-07-24T03:40:54.6427352Z     """
2025-07-24T03:40:54.6427866Z     Dynamically get the latest list of labels from the pull request
2025-07-24T03:40:54.6428531Z     """
2025-07-24T03:40:54.6428998Z     pr_info = get_pr_info(github_repo, github_token, pr_number)
2025-07-24T03:40:54.6429608Z     return {
2025-07-24T03:40:54.6430184Z         label.get("name") for label in pr_info.get("labels", []) if label.get("name")
2025-07-24T03:40:54.6430891Z     }
2025-07-24T03:40:54.6431190Z 
2025-07-24T03:40:54.6431197Z 
2025-07-24T03:40:54.6431368Z def main() -> None:
2025-07-24T03:40:54.6431771Z     args = parse_args()
2025-07-24T03:40:54.6432036Z 
2025-07-24T03:40:54.6432256Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2025-07-24T03:40:54.6432644Z 
2025-07-24T03:40:54.6432827Z     # Check if the PR is opt-out
2025-07-24T03:40:54.6433311Z     if args.pr_number:
2025-07-24T03:40:54.6433995Z         labels = get_labels(args.github_repo, args.github_token, int(args.pr_number))
2025-07-24T03:40:54.6434889Z         if OPT_OUT_LABEL in labels:
2025-07-24T03:40:54.6435402Z             log.info(
2025-07-24T03:40:54.6436076Z                 f"Opt-out runner determinator because #{args.pr_number} has {OPT_OUT_LABEL} label"
2025-07-24T03:40:54.6436848Z             )
2025-07-24T03:40:54.6437376Z             set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-07-24T03:40:54.6438045Z             sys.exit()
2025-07-24T03:40:54.6438301Z 
2025-07-24T03:40:54.6438456Z     try:
2025-07-24T03:40:54.6438882Z         rollout_state = get_rollout_state_from_issue(
2025-07-24T03:40:54.6439582Z             args.github_token, args.github_issue_repo, args.github_issue
2025-07-24T03:40:54.6440223Z         )
2025-07-24T03:40:54.6440420Z 
2025-07-24T03:40:54.6440625Z         username = get_potential_pr_author(
2025-07-24T03:40:54.6441261Z             args.github_token,
2025-07-24T03:40:54.6441735Z             args.github_repo,
2025-07-24T03:40:54.6442197Z             args.github_actor,
2025-07-24T03:40:54.6442667Z             args.github_ref_type,
2025-07-24T03:40:54.6443159Z             args.github_branch,
2025-07-24T03:40:54.6443615Z         )
2025-07-24T03:40:54.6443819Z 
2025-07-24T03:40:54.6444095Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2025-07-24T03:40:54.6444548Z 
2025-07-24T03:40:54.6444761Z         runner_label_prefix = get_runner_prefix(
2025-07-24T03:40:54.6445305Z             rollout_state,
2025-07-24T03:40:54.6445774Z             (args.github_issue_owner, username),
2025-07-24T03:40:54.6446312Z             args.github_branch,
2025-07-24T03:40:54.6446795Z             args.eligible_experiments,
2025-07-24T03:40:54.6447324Z             args.opt_out_experiments,
2025-07-24T03:40:54.6447813Z             is_canary,
2025-07-24T03:40:54.6448217Z         )
2025-07-24T03:40:54.6448418Z 
2025-07-24T03:40:54.6448594Z     except Exception as e:
2025-07-24T03:40:54.6449039Z         log.error(
2025-07-24T03:40:54.6449707Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2025-07-24T03:40:54.6450605Z         )
2025-07-24T03:40:54.6450810Z 
2025-07-24T03:40:54.6451293Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-07-24T03:40:54.6451800Z 
2025-07-24T03:40:54.6451807Z 
2025-07-24T03:40:54.6451978Z if __name__ == "__main__":
2025-07-24T03:40:54.6452411Z     main()
2025-07-24T03:40:54.6452616Z 
2025-07-24T03:40:54.6539983Z ##[group]Run python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2025-07-24T03:40:54.6540853Z python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2025-07-24T03:40:54.6570400Z shell: /usr/bin/bash -e {0}
2025-07-24T03:40:54.6570887Z env:
2025-07-24T03:40:54.6571645Z   GITHUB_TOKEN: ***
2025-07-24T03:40:54.6572059Z   ISSUE_NUMBER: 5132
2025-07-24T03:40:54.6572504Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2025-07-24T03:40:54.6573010Z   ISSUE_OWNER: 
2025-07-24T03:40:54.6573407Z   CHECK_EXPERIMENTS: 
2025-07-24T03:40:54.6573835Z   OPT_OUT_EXPERIMENTS: 
2025-07-24T03:40:54.6574253Z   PR_NUMBER: 
2025-07-24T03:40:54.6574653Z ##[endgroup]
2025-07-24T03:40:55.0110171Z Defaulting to user installation because normal site-packages is not writeable
2025-07-24T03:40:55.7395600Z Collecting urllib3==1.26.18
2025-07-24T03:40:55.7754064Z   Downloading urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
2025-07-24T03:40:55.7992360Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 4.1 MB/s eta 0:00:00
2025-07-24T03:40:55.8206167Z Collecting PyGithub==2.3.0
2025-07-24T03:40:55.8261656Z   Downloading PyGithub-2.3.0-py3-none-any.whl.metadata (3.8 kB)
2025-07-24T03:40:55.8680865Z Collecting pynacl>=1.4.0 (from PyGithub==2.3.0)
2025-07-24T03:40:55.8716412Z   Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (8.6 kB)
2025-07-24T03:40:55.8760260Z Requirement already satisfied: requests>=2.14.0 in /usr/lib/python3/dist-packages (from PyGithub==2.3.0) (2.31.0)
2025-07-24T03:40:55.8778345Z Requirement already satisfied: pyjwt>=2.4.0 in /usr/lib/python3/dist-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.3.0) (2.7.0)
2025-07-24T03:40:55.8794759Z Requirement already satisfied: typing-extensions>=4.0.0 in /usr/lib/python3/dist-packages (from PyGithub==2.3.0) (4.10.0)
2025-07-24T03:40:55.9050296Z Collecting Deprecated (from PyGithub==2.3.0)
2025-07-24T03:40:55.9085596Z   Downloading Deprecated-1.2.18-py2.py3-none-any.whl.metadata (5.7 kB)
2025-07-24T03:40:55.9309636Z Requirement already satisfied: cryptography>=3.4.0 in /usr/lib/python3/dist-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.3.0) (41.0.7)
2025-07-24T03:40:56.0409054Z Collecting cffi>=1.4.1 (from pynacl>=1.4.0->PyGithub==2.3.0)
2025-07-24T03:40:56.0449927Z   Downloading cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.5 kB)
2025-07-24T03:40:56.1522636Z Collecting wrapt<2,>=1.10 (from Deprecated->PyGithub==2.3.0)
2025-07-24T03:40:56.1570548Z   Downloading wrapt-1.17.2-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.4 kB)
2025-07-24T03:40:56.1749790Z Collecting pycparser (from cffi>=1.4.1->pynacl>=1.4.0->PyGithub==2.3.0)
2025-07-24T03:40:56.1788544Z   Downloading pycparser-2.22-py3-none-any.whl.metadata (943 bytes)
2025-07-24T03:40:56.2076267Z Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
2025-07-24T03:40:56.2138315Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.8/143.8 kB 30.3 MB/s eta 0:00:00
2025-07-24T03:40:56.2175299Z Downloading PyGithub-2.3.0-py3-none-any.whl (354 kB)
2025-07-24T03:40:56.2242544Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 354.4/354.4 kB 67.9 MB/s eta 0:00:00
2025-07-24T03:40:56.2279301Z Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
2025-07-24T03:40:56.2369051Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 856.7/856.7 kB 117.1 MB/s eta 0:00:00
2025-07-24T03:40:56.2404651Z Downloading Deprecated-1.2.18-py2.py3-none-any.whl (10.0 kB)
2025-07-24T03:40:56.2462252Z Downloading cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (479 kB)
2025-07-24T03:40:56.2522224Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 479.4/479.4 kB 106.3 MB/s eta 0:00:00
2025-07-24T03:40:56.2559256Z Downloading wrapt-1.17.2-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (89 kB)
2025-07-24T03:40:56.2598156Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.2/89.2 kB 34.9 MB/s eta 0:00:00
2025-07-24T03:40:56.2632583Z Downloading pycparser-2.22-py3-none-any.whl (117 kB)
2025-07-24T03:40:56.2673837Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 kB 41.8 MB/s eta 0:00:00
2025-07-24T03:40:56.5469610Z Installing collected packages: wrapt, urllib3, pycparser, Deprecated, cffi, pynacl, PyGithub
2025-07-24T03:40:57.0769793Z Successfully installed Deprecated-1.2.18 PyGithub-2.3.0 cffi-1.17.1 pycparser-2.22 pynacl-1.5.0 urllib3-1.26.18 wrapt-1.17.2
2025-07-24T03:40:57.1605539Z ##[group]Run curr_branch="ciflow/trunk/149961"
2025-07-24T03:40:57.1605928Z curr_branch="ciflow/trunk/149961"
2025-07-24T03:40:57.1606244Z curr_ref_type="tag"
2025-07-24T03:40:57.1606510Z echo "Current branch is '$curr_branch'"
2025-07-24T03:40:57.1606778Z 
2025-07-24T03:40:57.1606991Z python3 runner_determinator.py \
2025-07-24T03:40:57.1607297Z   --github-token "$GITHUB_TOKEN" \
2025-07-24T03:40:57.1607586Z   --github-issue "$ISSUE_NUMBER" \
2025-07-24T03:40:57.1607866Z   --github-branch "$curr_branch" \
2025-07-24T03:40:57.1608145Z   --github-actor "$TRIGGERING_ACTOR" \
2025-07-24T03:40:57.1608444Z   --github-issue-owner "$ISSUE_OWNER" \
2025-07-24T03:40:57.1608748Z   --github-ref-type "$curr_ref_type" \
2025-07-24T03:40:57.1609058Z   --github-repo "$GITHUB_REPOSITORY" \
2025-07-24T03:40:57.1609410Z   --eligible-experiments "$CHECK_EXPERIMENTS" \
2025-07-24T03:40:57.1609760Z   --opt-out-experiments "$OPT_OUT_EXPERIMENTS" \
2025-07-24T03:40:57.1610074Z   --pr-number "${PR_NUMBER}"
2025-07-24T03:40:57.1638909Z shell: /usr/bin/bash -e {0}
2025-07-24T03:40:57.1639165Z env:
2025-07-24T03:40:57.1639719Z   GITHUB_TOKEN: ***
2025-07-24T03:40:57.1639941Z   ISSUE_NUMBER: 5132
2025-07-24T03:40:57.1640182Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2025-07-24T03:40:57.1640441Z   ISSUE_OWNER: 
2025-07-24T03:40:57.1640651Z   CHECK_EXPERIMENTS: 
2025-07-24T03:40:57.1640863Z   OPT_OUT_EXPERIMENTS: 
2025-07-24T03:40:57.1641214Z   PR_NUMBER: 
2025-07-24T03:40:57.1641403Z ##[endgroup]
2025-07-24T03:40:57.1688774Z Current branch is 'ciflow/trunk/149961'
2025-07-24T03:40:59.9032072Z INFO    : Setting output: label-type=''
2025-07-24T03:40:59.9343969Z Evaluate and set job outputs
2025-07-24T03:40:59.9350871Z Cleaning up orphan processes