2024-11-01T16:13:31.2057155Z Current runner version: '2.320.0'
2024-11-01T16:13:31.2081670Z ##[group]Operating System
2024-11-01T16:13:31.2082354Z Ubuntu
2024-11-01T16:13:31.2082699Z 22.04.5
2024-11-01T16:13:31.2083095Z LTS
2024-11-01T16:13:31.2083461Z ##[endgroup]
2024-11-01T16:13:31.2083843Z ##[group]Runner Image
2024-11-01T16:13:31.2084312Z Image: ubuntu-22.04
2024-11-01T16:13:31.2084738Z Version: 20241015.1.0
2024-11-01T16:13:31.2085741Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20241015.1/images/ubuntu/Ubuntu2204-Readme.md
2024-11-01T16:13:31.2087256Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20241015.1
2024-11-01T16:13:31.2088082Z ##[endgroup]
2024-11-01T16:13:31.2088480Z ##[group]Runner Image Provisioner
2024-11-01T16:13:31.2089041Z 2.0.384.1
2024-11-01T16:13:31.2089371Z ##[endgroup]
2024-11-01T16:13:31.2104413Z ##[group]GITHUB_TOKEN Permissions
2024-11-01T16:13:31.2105997Z Contents: read
2024-11-01T16:13:31.2106573Z Metadata: read
2024-11-01T16:13:31.2107124Z Packages: read
2024-11-01T16:13:31.2107689Z ##[endgroup]
2024-11-01T16:13:31.2110901Z Secret source: Actions
2024-11-01T16:13:31.2111504Z Prepare workflow directory
2024-11-01T16:13:31.3021875Z Prepare all required actions
2024-11-01T16:13:31.3188142Z Uses: pytorch/pytorch/.github/workflows/_runner-determinator.yml@refs/heads/main (33dce10ece5b38aa0ab76739b658cd980a6e3d8f)
2024-11-01T16:13:31.3193574Z ##[group] Inputs
2024-11-01T16:13:31.3194060Z   check_experiments: 
2024-11-01T16:13:31.3194671Z   triggering_actor: pytorch-bot[bot]
2024-11-01T16:13:31.3195182Z   issue_owner: 
2024-11-01T16:13:31.3195595Z   curr_branch: ciflow/trunk/138766
2024-11-01T16:13:31.3196349Z   curr_ref_type: tag
2024-11-01T16:13:31.3196854Z   issue_number: 5132
2024-11-01T16:13:31.3197363Z ##[endgroup]
2024-11-01T16:13:31.3198281Z Complete job name: get-label-type / runner-determinator
2024-11-01T16:13:31.4084190Z ##[group]Run cat <<EOF > runner_determinator.py
2024-11-01T16:13:31.4086041Z cat <<EOF > runner_determinator.py
2024-11-01T16:13:31.4086756Z # flake8: noqa: G004
2024-11-01T16:13:31.4087255Z 
2024-11-01T16:13:31.4087969Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2024-11-01T16:13:31.4089270Z #       must be kept in sync. You can do it easily by running the following command:
2024-11-01T16:13:31.4090310Z #           python .github/scripts/update_runner_determinator.py
2024-11-01T16:13:31.4090964Z 
2024-11-01T16:13:31.4091402Z """
2024-11-01T16:13:31.4092111Z This runner determinator is used to determine which set of runners to run a
2024-11-01T16:13:31.4093268Z GitHub job on. It uses the first comment of a GitHub issue (by default
2024-11-01T16:13:31.4094501Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2024-11-01T16:13:31.4095463Z of which runners should be used to run which job.
2024-11-01T16:13:31.4096146Z 
2024-11-01T16:13:31.4096850Z The configuration has two parts, the settings and a list of opted-in users,
2024-11-01T16:13:31.4098002Z separated by a line containing "---".  If the line is not present, the
2024-11-01T16:13:31.4099155Z settings are considered to be empty with only the second part, the user
2024-11-01T16:13:31.4100134Z list, defined.
2024-11-01T16:13:31.4100556Z 
2024-11-01T16:13:31.4101325Z The first part is a YAML block that defines the rollout settings. This can be
2024-11-01T16:13:31.4102431Z used to define any settings that are needed to determine which runners to use.
2024-11-01T16:13:31.4103462Z It's fields are defined by the RolloutSettings class below.
2024-11-01T16:13:31.4104266Z 
2024-11-01T16:13:31.4104911Z The second part is a list of users who are explicitly opted in to the LF fleet.
2024-11-01T16:13:31.4106000Z The user list is also a comma separated list of additional features or
2024-11-01T16:13:31.4107446Z experiments which the user could be opted in to.
2024-11-01T16:13:31.4108074Z 
2024-11-01T16:13:31.4108482Z The user list has the following rules:
2024-11-01T16:13:31.4109118Z 
2024-11-01T16:13:31.4109696Z - Users are GitHub usernames, which must start with the @ prefix
2024-11-01T16:13:31.4110717Z - Each user is also a comma-separated list of features/experiments to enable
2024-11-01T16:13:31.4111755Z - A "#" prefix opts the user out of all experiments
2024-11-01T16:13:31.4112385Z 
2024-11-01T16:13:31.4112729Z Example config:
2024-11-01T16:13:31.4113357Z     # A list of experiments that can be opted into.
2024-11-01T16:13:31.4114203Z     # This defines the behavior they'll induce when opted into.
2024-11-01T16:13:31.4114905Z     # Expected syntax is:
2024-11-01T16:13:31.4115778Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2024-11-01T16:13:31.4117016Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2024-11-01T16:13:31.4117984Z 
2024-11-01T16:13:31.4118318Z     experiments:
2024-11-01T16:13:31.4118767Z       lf:
2024-11-01T16:13:31.4119261Z         rollout_percent: 25
2024-11-01T16:13:31.4119748Z         all_branches: false
2024-11-01T16:13:31.4120275Z         default: true
2024-11-01T16:13:31.4120789Z     ---
2024-11-01T16:13:31.4121129Z 
2024-11-01T16:13:31.4121498Z     # Opt-ins:
2024-11-01T16:13:31.4122301Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2024-11-01T16:13:31.4123543Z     # and specifying experiments to enable in a comma-separated list.
2024-11-01T16:13:31.4124451Z     # Experiments should be from the above list.
2024-11-01T16:13:31.4125134Z 
2024-11-01T16:13:31.4125492Z     @User1,lf,split_build
2024-11-01T16:13:31.4125990Z     @User2,lf
2024-11-01T16:13:31.4126528Z     @User3,split_build
2024-11-01T16:13:31.4126980Z """
2024-11-01T16:13:31.4127334Z 
2024-11-01T16:13:31.4127774Z import logging
2024-11-01T16:13:31.4128167Z import os
2024-11-01T16:13:31.4128571Z import random
2024-11-01T16:13:31.4129145Z from argparse import ArgumentParser
2024-11-01T16:13:31.4129709Z from logging import LogRecord
2024-11-01T16:13:31.4130524Z from typing import Any, Dict, FrozenSet, Iterable, List, NamedTuple, Tuple
2024-11-01T16:13:31.4131399Z 
2024-11-01T16:13:31.4131724Z import yaml
2024-11-01T16:13:31.4132240Z from github import Auth, Github
2024-11-01T16:13:31.4132927Z from github.Issue import Issue
2024-11-01T16:13:31.4133453Z 
2024-11-01T16:13:31.4133759Z 
2024-11-01T16:13:31.4134324Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2024-11-01T16:13:31.4135166Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2024-11-01T16:13:31.4136228Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2024-11-01T16:13:31.4137067Z 
2024-11-01T16:13:31.4137532Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2024-11-01T16:13:31.4138201Z GH_OUTPUT_KEY_AMI = "runner-ami"
2024-11-01T16:13:31.4138848Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2024-11-01T16:13:31.4139416Z 
2024-11-01T16:13:31.4140154Z 
2024-11-01T16:13:31.4140618Z SETTING_EXPERIMENTS = "experiments"
2024-11-01T16:13:31.4141170Z 
2024-11-01T16:13:31.4141558Z LF_FLEET_EXPERIMENT = "lf"
2024-11-01T16:13:31.4142107Z CANARY_FLEET_SUFFIX = ".c"
2024-11-01T16:13:31.4142597Z 
2024-11-01T16:13:31.4142946Z 
2024-11-01T16:13:31.4143559Z class Experiment(NamedTuple):
2024-11-01T16:13:31.4144131Z     rollout_perc: float = (
2024-11-01T16:13:31.4144922Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2024-11-01T16:13:31.4145750Z     )
2024-11-01T16:13:31.4146163Z     all_branches: bool = (
2024-11-01T16:13:31.4146960Z         False  # If True, the experiment is also enabled on the exception branches
2024-11-01T16:13:31.4147777Z     )
2024-11-01T16:13:31.4148183Z     default: bool = (
2024-11-01T16:13:31.4148898Z         True  # If True, the experiment is enabled by default for all queries
2024-11-01T16:13:31.4149735Z     )
2024-11-01T16:13:31.4150063Z 
2024-11-01T16:13:31.4150449Z     # Add more fields as needed
2024-11-01T16:13:31.4151055Z 
2024-11-01T16:13:31.4151361Z 
2024-11-01T16:13:31.4151743Z class Settings(NamedTuple):
2024-11-01T16:13:31.4152295Z     """
2024-11-01T16:13:31.4152857Z     Settings for the experiments that can be opted into.
2024-11-01T16:13:31.4153525Z     """
2024-11-01T16:13:31.4153972Z 
2024-11-01T16:13:31.4154381Z     experiments: Dict[str, Experiment] = {}
2024-11-01T16:13:31.4154957Z 
2024-11-01T16:13:31.4155367Z 
2024-11-01T16:13:31.4155768Z class ColorFormatter(logging.Formatter):
2024-11-01T16:13:31.4156556Z     """Color codes the log messages based on the log level"""
2024-11-01T16:13:31.4157291Z 
2024-11-01T16:13:31.4157614Z     COLORS = {
2024-11-01T16:13:31.4158113Z         "WARNING": "\033[33m",  # Yellow
2024-11-01T16:13:31.4158771Z         "ERROR": "\033[31m",  # Red
2024-11-01T16:13:31.4159485Z         "CRITICAL": "\033[31m",  # Red
2024-11-01T16:13:31.4160092Z         "INFO": "\033[0m",  # Reset
2024-11-01T16:13:31.4160740Z         "DEBUG": "\033[0m",  # Reset
2024-11-01T16:13:31.4161245Z     }
2024-11-01T16:13:31.4161624Z 
2024-11-01T16:13:31.4162144Z     def format(self, record: LogRecord) -> str:
2024-11-01T16:13:31.4163054Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2024-11-01T16:13:31.4164003Z         record.msg = f"{log_color}{record.msg}\033[0m"
2024-11-01T16:13:31.4164753Z         return super().format(record)
2024-11-01T16:13:31.4165282Z 
2024-11-01T16:13:31.4165617Z 
2024-11-01T16:13:31.4166112Z handler = logging.StreamHandler()
2024-11-01T16:13:31.4166975Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2024-11-01T16:13:31.4167735Z 
2024-11-01T16:13:31.4168301Z log = logging.getLogger(os.path.basename(__file__))
2024-11-01T16:13:31.4169003Z log.addHandler(handler)
2024-11-01T16:13:31.4169483Z log.setLevel(logging.INFO)
2024-11-01T16:13:31.4170055Z 
2024-11-01T16:13:31.4170393Z 
2024-11-01T16:13:31.4170850Z def set_github_output(key: str, value: str) -> None:
2024-11-01T16:13:31.4171570Z     """
2024-11-01T16:13:31.4172174Z     Defines outputs of the github action that invokes this script
2024-11-01T16:13:31.4172863Z     """
2024-11-01T16:13:31.4173351Z     if not GITHUB_OUTPUT:
2024-11-01T16:13:31.4174700Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2024-11-01T16:13:31.4176026Z         log.warning(
2024-11-01T16:13:31.4177184Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2024-11-01T16:13:31.4178294Z         )
2024-11-01T16:13:31.4178813Z         print(f"::set-output name={key}::{value}")
2024-11-01T16:13:31.4179599Z         return
2024-11-01T16:13:31.4180166Z 
2024-11-01T16:13:31.4180605Z     with open(GITHUB_OUTPUT, "a") as f:
2024-11-01T16:13:31.4181328Z         log.info(f"Setting output: {key}='{value}'")
2024-11-01T16:13:31.4181997Z         f.write(f"{key}={value}\n")
2024-11-01T16:13:31.4182549Z 
2024-11-01T16:13:31.4182925Z 
2024-11-01T16:13:31.4183481Z def _str_comma_separated_to_set(value: str) -> FrozenSet[str]:
2024-11-01T16:13:31.4184237Z     return frozenset(
2024-11-01T16:13:31.4185037Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2024-11-01T16:13:31.4185850Z     )
2024-11-01T16:13:31.4186213Z 
2024-11-01T16:13:31.4186599Z 
2024-11-01T16:13:31.4186992Z def parse_args() -> Any:
2024-11-01T16:13:31.4187674Z     parser = ArgumentParser("Get dynamic rollout settings")
2024-11-01T16:13:31.4188766Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2024-11-01T16:13:31.4189674Z     parser.add_argument(
2024-11-01T16:13:31.4190215Z         "--github-issue-repo",
2024-11-01T16:13:31.4190800Z         type=str,
2024-11-01T16:13:31.4191239Z         required=False,
2024-11-01T16:13:31.4191776Z         default="pytorch/test-infra",
2024-11-01T16:13:31.4192489Z         help="GitHub repo to get the issue",
2024-11-01T16:13:31.4193036Z     )
2024-11-01T16:13:31.4193423Z     parser.add_argument(
2024-11-01T16:13:31.4194007Z         "--github-repo",
2024-11-01T16:13:31.4194462Z         type=str,
2024-11-01T16:13:31.4194911Z         required=True,
2024-11-01T16:13:31.4195721Z         help="GitHub repo where CI is running",
2024-11-01T16:13:31.4196291Z     )
2024-11-01T16:13:31.4196701Z     parser.add_argument(
2024-11-01T16:13:31.4197586Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2024-11-01T16:13:31.4198317Z     )
2024-11-01T16:13:31.4198722Z     parser.add_argument(
2024-11-01T16:13:31.4199560Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2024-11-01T16:13:31.4200307Z     )
2024-11-01T16:13:31.4200709Z     parser.add_argument(
2024-11-01T16:13:31.4201552Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2024-11-01T16:13:31.4202347Z     )
2024-11-01T16:13:31.4202719Z     parser.add_argument(
2024-11-01T16:13:31.4203587Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2024-11-01T16:13:31.4204421Z     )
2024-11-01T16:13:31.4204798Z     parser.add_argument(
2024-11-01T16:13:31.4205373Z         "--github-ref-type",
2024-11-01T16:13:31.4205911Z         type=str,
2024-11-01T16:13:31.4206334Z         required=True,
2024-11-01T16:13:31.4206995Z         help="Current GitHub ref type, branch or tag",
2024-11-01T16:13:31.4207640Z     )
2024-11-01T16:13:31.4207998Z     parser.add_argument(
2024-11-01T16:13:31.4208614Z         "--eligible-experiments",
2024-11-01T16:13:31.4209240Z         type=_str_comma_separated_to_set,
2024-11-01T16:13:31.4209789Z         required=False,
2024-11-01T16:13:31.4210339Z         default="",
2024-11-01T16:13:31.4211426Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2024-11-01T16:13:31.4212577Z     )
2024-11-01T16:13:31.4212963Z 
2024-11-01T16:13:31.4213329Z     return parser.parse_args()
2024-11-01T16:13:31.4213905Z 
2024-11-01T16:13:31.4214262Z 
2024-11-01T16:13:31.4214704Z def get_gh_client(github_token: str) -> Github:
2024-11-01T16:13:31.4215610Z     auth = Auth.Token(github_token)
2024-11-01T16:13:31.4216206Z     return Github(auth=auth)
2024-11-01T16:13:31.4216672Z 
2024-11-01T16:13:31.4217076Z 
2024-11-01T16:13:31.4217673Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:
2024-11-01T16:13:31.4218383Z     repo = gh.get_repo(repo)
2024-11-01T16:13:31.4219071Z     return repo.get_issue(number=issue_num)
2024-11-01T16:13:31.4219900Z 
2024-11-01T16:13:31.4220217Z 
2024-11-01T16:13:31.4220683Z def get_potential_pr_author(
2024-11-01T16:13:31.4221472Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2024-11-01T16:13:31.4222254Z ) -> str:
2024-11-01T16:13:31.4222946Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2024-11-01T16:13:31.4223954Z     # Fetch the actual username from the original PR. The PR number is
2024-11-01T16:13:31.4224871Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2024-11-01T16:13:31.4225567Z 
2024-11-01T16:13:31.4225986Z     gh = get_gh_client(github_token)
2024-11-01T16:13:31.4226539Z 
2024-11-01T16:13:31.4227087Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2024-11-01T16:13:31.4227836Z         split_tag = ref_name.split("/")
2024-11-01T16:13:31.4228417Z         if (
2024-11-01T16:13:31.4228879Z             len(split_tag) == 3
2024-11-01T16:13:31.4229464Z             and split_tag[0] == "ciflow"
2024-11-01T16:13:31.4230112Z             and split_tag[2].isnumeric()
2024-11-01T16:13:31.4230716Z         ):
2024-11-01T16:13:31.4231324Z             pr_number = split_tag[2]
2024-11-01T16:13:31.4231900Z             try:
2024-11-01T16:13:31.4232499Z                 repository = gh.get_repo(repo)
2024-11-01T16:13:31.4233230Z                 pull = repository.get_pull(number=int(pr_number))
2024-11-01T16:13:31.4233947Z             except Exception as e:
2024-11-01T16:13:31.4234638Z                 raise Exception(  # noqa: TRY002
2024-11-01T16:13:31.4235439Z                     f"issue with pull request {pr_number} from repo {repository}"
2024-11-01T16:13:31.4236197Z                 ) from e
2024-11-01T16:13:31.4236793Z             return pull.user.login
2024-11-01T16:13:31.4237486Z     # In all other cases, return the original input username
2024-11-01T16:13:31.4238178Z     return username
2024-11-01T16:13:31.4238690Z 
2024-11-01T16:13:31.4239011Z 
2024-11-01T16:13:31.4239476Z def is_exception_branch(branch: str) -> bool:
2024-11-01T16:13:31.4240277Z     """
2024-11-01T16:13:31.4241041Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2024-11-01T16:13:31.4241950Z     """
2024-11-01T16:13:31.4242684Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2024-11-01T16:13:31.4243451Z 
2024-11-01T16:13:31.4243756Z 
2024-11-01T16:13:31.4244255Z def load_yaml(yaml_text: str) -> Any:
2024-11-01T16:13:31.4244809Z     try:
2024-11-01T16:13:31.4245221Z         data = yaml.safe_load(yaml_text)
2024-11-01T16:13:31.4245884Z         return data
2024-11-01T16:13:31.4246396Z     except yaml.YAMLError as exc:
2024-11-01T16:13:31.4246987Z         log.exception("Error loading YAML")
2024-11-01T16:13:31.4247668Z         raise
2024-11-01T16:13:31.4248070Z 
2024-11-01T16:13:31.4248390Z 
2024-11-01T16:13:31.4249143Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> Tuple[str, str]:
2024-11-01T16:13:31.4249982Z     """
2024-11-01T16:13:31.4250883Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2024-11-01T16:13:31.4251842Z 
2024-11-01T16:13:31.4252482Z     If the issue body contains "---" then the text above that is the settings
2024-11-01T16:13:31.4253415Z     and the text below is the list of opted in users.
2024-11-01T16:13:31.4254079Z 
2024-11-01T16:13:31.4254773Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2024-11-01T16:13:31.4255605Z     """
2024-11-01T16:13:31.4256148Z     rollout_state_parts = rollout_state.split("---")
2024-11-01T16:13:31.4256857Z     if len(rollout_state_parts) >= 2:
2024-11-01T16:13:31.4257581Z         return rollout_state_parts[0], rollout_state_parts[1]
2024-11-01T16:13:31.4258277Z     else:
2024-11-01T16:13:31.4258722Z         return "", rollout_state
2024-11-01T16:13:31.4259239Z 
2024-11-01T16:13:31.4259601Z 
2024-11-01T16:13:31.4260171Z class UserOptins(Dict[str, List[str]]):
2024-11-01T16:13:31.4260739Z     """
2024-11-01T16:13:31.4261420Z     Dictionary of users with a list of features they have opted into
2024-11-01T16:13:31.4262185Z     """
2024-11-01T16:13:31.4262562Z 
2024-11-01T16:13:31.4262940Z 
2024-11-01T16:13:31.4263536Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2024-11-01T16:13:31.4264283Z     """
2024-11-01T16:13:31.4265218Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2024-11-01T16:13:31.4266244Z 
2024-11-01T16:13:31.4267384Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2024-11-01T16:13:31.4268698Z         - Example line: "@User1,lf,split_build"
2024-11-01T16:13:31.4269492Z         - A "#" prefix indicates the user is opted out of all experiments
2024-11-01T16:13:31.4270237Z 
2024-11-01T16:13:31.4270652Z 
2024-11-01T16:13:31.4270970Z     """
2024-11-01T16:13:31.4271390Z     optins = UserOptins()
2024-11-01T16:13:31.4272050Z     for user in user_optin_text.split("\n"):
2024-11-01T16:13:31.4272661Z         user = user.strip("\r\n\t -")
2024-11-01T16:13:31.4273333Z         if not user or not user.startswith("@"):
2024-11-01T16:13:31.4274046Z             # Not a valid user. Skip
2024-11-01T16:13:31.4274565Z             continue
2024-11-01T16:13:31.4275030Z 
2024-11-01T16:13:31.4275441Z         if user:
2024-11-01T16:13:31.4275945Z             usr_name = user.split(",")[0].strip("@")
2024-11-01T16:13:31.4276806Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2024-11-01T16:13:31.4277613Z 
2024-11-01T16:13:31.4277992Z     return optins
2024-11-01T16:13:31.4278392Z 
2024-11-01T16:13:31.4278791Z 
2024-11-01T16:13:31.4279368Z def parse_settings_from_text(settings_text: str) -> Settings:
2024-11-01T16:13:31.4280048Z     """
2024-11-01T16:13:31.4280803Z     Parse the experiments from the issue body into a list of ExperimentSettings
2024-11-01T16:13:31.4281639Z     """
2024-11-01T16:13:31.4281977Z     try:
2024-11-01T16:13:31.4282442Z         if settings_text:
2024-11-01T16:13:31.4283378Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2024-11-01T16:13:31.4284302Z             # for easy reading
2024-11-01T16:13:31.4285383Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2024-11-01T16:13:31.4286518Z             #       the backtick character in shell commands.
2024-11-01T16:13:31.4287479Z             backtick = chr(96)  # backtick character
2024-11-01T16:13:31.4288269Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2024-11-01T16:13:31.4289064Z             settings = load_yaml(settings_text)
2024-11-01T16:13:31.4289749Z 
2024-11-01T16:13:31.4290428Z             # For now we just load experiments. We can expand this if/when we add more settings
2024-11-01T16:13:31.4291320Z             experiments = {}
2024-11-01T16:13:31.4291920Z 
2024-11-01T16:13:31.4292552Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2024-11-01T16:13:31.4293409Z                 valid_settings = {}
2024-11-01T16:13:31.4294101Z                 for setting in exp_settings:
2024-11-01T16:13:31.4294800Z                     if setting not in Experiment._fields:
2024-11-01T16:13:31.4295432Z                         log.warning(
2024-11-01T16:13:31.4296354Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2024-11-01T16:13:31.4297219Z                         )
2024-11-01T16:13:31.4297665Z                     else:
2024-11-01T16:13:31.4298384Z                         valid_settings[setting] = exp_settings[setting]
2024-11-01T16:13:31.4299044Z 
2024-11-01T16:13:31.4299527Z                 experiments[exp_name] = Experiment(**valid_settings)
2024-11-01T16:13:31.4300878Z             return Settings(experiments)
2024-11-01T16:13:31.4301445Z 
2024-11-01T16:13:31.4301796Z     except Exception:
2024-11-01T16:13:31.4302617Z         log.exception("Failed to parse settings")
2024-11-01T16:13:31.4303248Z 
2024-11-01T16:13:31.4303598Z     return Settings()
2024-11-01T16:13:31.4304121Z 
2024-11-01T16:13:31.4304479Z 
2024-11-01T16:13:31.4304938Z def parse_settings(rollout_state: str) -> Settings:
2024-11-01T16:13:31.4305651Z     """
2024-11-01T16:13:31.4306179Z     Parse settings, if any, from the rollout state.
2024-11-01T16:13:31.4306805Z 
2024-11-01T16:13:31.4307468Z     If the issue body contains "---" then the text above that is the settings
2024-11-01T16:13:31.4308419Z     and the text below is the list of opted in users.
2024-11-01T16:13:31.4309050Z 
2024-11-01T16:13:31.4309801Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2024-11-01T16:13:31.4310689Z     """
2024-11-01T16:13:31.4311327Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2024-11-01T16:13:31.4312255Z     return parse_settings_from_text(settings_text)
2024-11-01T16:13:31.4312895Z 
2024-11-01T16:13:31.4313236Z 
2024-11-01T16:13:31.4313749Z def parse_users(rollout_state: str) -> UserOptins:
2024-11-01T16:13:31.4314397Z     """
2024-11-01T16:13:31.4314838Z     Parse users from the rollout state.
2024-11-01T16:13:31.4315447Z 
2024-11-01T16:13:31.4315790Z     """
2024-11-01T16:13:31.4316408Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2024-11-01T16:13:31.4317376Z     return parse_user_opt_in_from_text(users_text)
2024-11-01T16:13:31.4317950Z 
2024-11-01T16:13:31.4318288Z 
2024-11-01T16:13:31.4319075Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2024-11-01T16:13:31.4319912Z     """
2024-11-01T16:13:31.4320404Z     Check if a user is opted into an experiment
2024-11-01T16:13:31.4321083Z     """
2024-11-01T16:13:31.4321579Z     return experiment_name in user_optins.get(user, [])
2024-11-01T16:13:31.4322398Z 
2024-11-01T16:13:31.4322803Z 
2024-11-01T16:13:31.4323133Z def get_runner_prefix(
2024-11-01T16:13:31.4323649Z     rollout_state: str,
2024-11-01T16:13:31.4324257Z     workflow_requestors: Iterable[str],
2024-11-01T16:13:31.4324802Z     branch: str,
2024-11-01T16:13:31.4325412Z     eligible_experiments: FrozenSet[str] = frozenset(),
2024-11-01T16:13:31.4326167Z     is_canary: bool = False,
2024-11-01T16:13:31.4326640Z ) -> str:
2024-11-01T16:13:31.4327138Z     settings = parse_settings(rollout_state)
2024-11-01T16:13:31.4327877Z     user_optins = parse_users(rollout_state)
2024-11-01T16:13:31.4328431Z 
2024-11-01T16:13:31.4328796Z     fleet_prefix = ""
2024-11-01T16:13:31.4329353Z     prefixes = []
2024-11-01T16:13:31.4330094Z     for experiment_name, experiment_settings in settings.experiments.items():
2024-11-01T16:13:31.4331231Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2024-11-01T16:13:31.4332108Z             log.info(
2024-11-01T16:13:31.4332962Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2024-11-01T16:13:31.4333811Z             )
2024-11-01T16:13:31.4334291Z             continue
2024-11-01T16:13:31.4334750Z 
2024-11-01T16:13:31.4335102Z         if eligible_experiments:
2024-11-01T16:13:31.4335862Z             if experiment_name not in eligible_experiments:
2024-11-01T16:13:31.4336641Z                 exp_list = ", ".join(eligible_experiments)
2024-11-01T16:13:31.4337250Z                 log.info(
2024-11-01T16:13:31.4338396Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2024-11-01T16:13:31.4339384Z                 )
2024-11-01T16:13:31.4339997Z                 continue
2024-11-01T16:13:31.4340620Z         elif not experiment_settings.default:
2024-11-01T16:13:31.4341222Z             log.info(
2024-11-01T16:13:31.4342021Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2024-11-01T16:13:31.4342884Z             )
2024-11-01T16:13:31.4343299Z             continue
2024-11-01T16:13:31.4343760Z 
2024-11-01T16:13:31.4344324Z         # Is any workflow_requestor opted in to this experiment?
2024-11-01T16:13:31.4345044Z         opted_in_users = [
2024-11-01T16:13:31.4345571Z             requestor
2024-11-01T16:13:31.4346166Z             for requestor in workflow_requestors
2024-11-01T16:13:31.4346989Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2024-11-01T16:13:31.4347728Z         ]
2024-11-01T16:13:31.4348144Z 
2024-11-01T16:13:31.4348517Z         enabled = False
2024-11-01T16:13:31.4349031Z         if opted_in_users:
2024-11-01T16:13:31.4349567Z             log.info(
2024-11-01T16:13:31.4350339Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2024-11-01T16:13:31.4351130Z             )
2024-11-01T16:13:31.4351620Z             enabled = True
2024-11-01T16:13:31.4352082Z 
2024-11-01T16:13:31.4352527Z         elif experiment_settings.rollout_perc:
2024-11-01T16:13:31.4353633Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2024-11-01T16:13:31.4354793Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2024-11-01T16:13:31.4355551Z                 log.info(
2024-11-01T16:13:31.4356654Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2024-11-01T16:13:31.4357911Z                 )
2024-11-01T16:13:31.4358337Z                 enabled = True
2024-11-01T16:13:31.4358929Z 
2024-11-01T16:13:31.4359300Z         if enabled:
2024-11-01T16:13:31.4359768Z             label = experiment_name
2024-11-01T16:13:31.4360510Z             if experiment_name == LF_FLEET_EXPERIMENT:
2024-11-01T16:13:31.4361533Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2024-11-01T16:13:31.4362591Z                 #  - If it's enabled, then we always list it's prefix first
2024-11-01T16:13:31.4363658Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2024-11-01T16:13:31.4364465Z                 if is_canary:
2024-11-01T16:13:31.4365137Z                     label += CANARY_FLEET_SUFFIX
2024-11-01T16:13:31.4365741Z                 fleet_prefix = label
2024-11-01T16:13:31.4366282Z             else:
2024-11-01T16:13:31.4366852Z                 prefixes.append(label)
2024-11-01T16:13:31.4367367Z 
2024-11-01T16:13:31.4367736Z     if len(prefixes) > 1:
2024-11-01T16:13:31.4368318Z         log.error(
2024-11-01T16:13:31.4369600Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2024-11-01T16:13:31.4370942Z         )
2024-11-01T16:13:31.4371468Z         prefixes = prefixes[:1]
2024-11-01T16:13:31.4371955Z 
2024-11-01T16:13:31.4372341Z     # Fleet always comes first
2024-11-01T16:13:31.4372966Z     if fleet_prefix:
2024-11-01T16:13:31.4373611Z         prefixes.insert(0, fleet_prefix)
2024-11-01T16:13:31.4374201Z 
2024-11-01T16:13:31.4374766Z     return ".".join(prefixes) + "." if prefixes else ""
2024-11-01T16:13:31.4375377Z 
2024-11-01T16:13:31.4375730Z 
2024-11-01T16:13:31.4376493Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2024-11-01T16:13:31.4377374Z     """
2024-11-01T16:13:31.4378040Z     Gets the first comment of the issue, which contains the desired rollout state.
2024-11-01T16:13:31.4378938Z 
2024-11-01T16:13:31.4380096Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2024-11-01T16:13:31.4380982Z     """
2024-11-01T16:13:31.4381484Z     gh = get_gh_client(github_token)
2024-11-01T16:13:31.4382136Z     issue = get_issue(gh, repo, issue_num)
2024-11-01T16:13:31.4382866Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2024-11-01T16:13:31.4383600Z 
2024-11-01T16:13:31.4383956Z 
2024-11-01T16:13:31.4384286Z def main() -> None:
2024-11-01T16:13:31.4384841Z     args = parse_args()
2024-11-01T16:13:31.4385317Z 
2024-11-01T16:13:31.4385746Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2024-11-01T16:13:31.4386418Z 
2024-11-01T16:13:31.4386761Z     try:
2024-11-01T16:13:31.4387236Z         rollout_state = get_rollout_state_from_issue(
2024-11-01T16:13:31.4388171Z             args.github_token, args.github_issue_repo, args.github_issue
2024-11-01T16:13:31.4388896Z         )
2024-11-01T16:13:31.4389281Z 
2024-11-01T16:13:31.4389776Z         username = get_potential_pr_author(
2024-11-01T16:13:31.4390390Z             args.github_token,
2024-11-01T16:13:31.4390948Z             args.github_repo,
2024-11-01T16:13:31.4391516Z             args.github_actor,
2024-11-01T16:13:31.4392072Z             args.github_ref_type,
2024-11-01T16:13:31.4392656Z             args.github_branch,
2024-11-01T16:13:31.4393376Z         )
2024-11-01T16:13:31.4393756Z 
2024-11-01T16:13:31.4394313Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2024-11-01T16:13:31.4395043Z 
2024-11-01T16:13:31.4395492Z         runner_label_prefix = get_runner_prefix(
2024-11-01T16:13:31.4396129Z             rollout_state,
2024-11-01T16:13:31.4396747Z             (args.github_issue_owner, username),
2024-11-01T16:13:31.4397421Z             args.github_branch,
2024-11-01T16:13:31.4398007Z             args.eligible_experiments,
2024-11-01T16:13:31.4398604Z             is_canary,
2024-11-01T16:13:31.4399084Z         )
2024-11-01T16:13:31.4399466Z 
2024-11-01T16:13:31.4399877Z     except Exception as e:
2024-11-01T16:13:31.4400402Z         log.error(
2024-11-01T16:13:31.4401228Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2024-11-01T16:13:31.4402135Z         )
2024-11-01T16:13:31.4402528Z 
2024-11-01T16:13:31.4403103Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2024-11-01T16:13:31.4403910Z 
2024-11-01T16:13:31.4404210Z 
2024-11-01T16:13:31.4404577Z if __name__ == "__main__":
2024-11-01T16:13:31.4405152Z     main()
2024-11-01T16:13:31.4405513Z 
2024-11-01T16:13:31.4405849Z EOF
2024-11-01T16:13:31.4406252Z 
2024-11-01T16:13:31.4406637Z cat runner_determinator.py
2024-11-01T16:13:31.4793410Z shell: /usr/bin/bash -e {0}
2024-11-01T16:13:31.4793969Z env:
2024-11-01T16:13:31.4794896Z   GITHUB_TOKEN: ***
2024-11-01T16:13:31.4795363Z   ISSUE_NUMBER: 5132
2024-11-01T16:13:31.4796031Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2024-11-01T16:13:31.4796671Z   ISSUE_OWNER: 
2024-11-01T16:13:31.4797095Z   CHECK_EXPERIMENTS: 
2024-11-01T16:13:31.4797494Z ##[endgroup]
2024-11-01T16:13:31.5228927Z # flake8: noqa: G004
2024-11-01T16:13:31.5229352Z 
2024-11-01T16:13:31.5231161Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2024-11-01T16:13:31.5232471Z #       must be kept in sync. You can do it easily by running the following command:
2024-11-01T16:13:31.5233356Z #           python .github/scripts/update_runner_determinator.py
2024-11-01T16:13:31.5233858Z 
2024-11-01T16:13:31.5233982Z """
2024-11-01T16:13:31.5234669Z This runner determinator is used to determine which set of runners to run a
2024-11-01T16:13:31.5235611Z GitHub job on. It uses the first comment of a GitHub issue (by default
2024-11-01T16:13:31.5236715Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2024-11-01T16:13:31.5237711Z of which runners should be used to run which job.
2024-11-01T16:13:31.5238133Z 
2024-11-01T16:13:31.5238600Z The configuration has two parts, the settings and a list of opted-in users,
2024-11-01T16:13:31.5239667Z separated by a line containing "---".  If the line is not present, the
2024-11-01T16:13:31.5240692Z settings are considered to be empty with only the second part, the user
2024-11-01T16:13:31.5241428Z list, defined.
2024-11-01T16:13:31.5241634Z 
2024-11-01T16:13:31.5242044Z The first part is a YAML block that defines the rollout settings. This can be
2024-11-01T16:13:31.5243141Z used to define any settings that are needed to determine which runners to use.
2024-11-01T16:13:31.5244113Z It's fields are defined by the RolloutSettings class below.
2024-11-01T16:13:31.5244609Z 
2024-11-01T16:13:31.5245022Z The second part is a list of users who are explicitly opted in to the LF fleet.
2024-11-01T16:13:31.5246076Z The user list is also a comma separated list of additional features or
2024-11-01T16:13:31.5246883Z experiments which the user could be opted in to.
2024-11-01T16:13:31.5247324Z 
2024-11-01T16:13:31.5247496Z The user list has the following rules:
2024-11-01T16:13:31.5247937Z 
2024-11-01T16:13:31.5248629Z - Users are GitHub usernames, which must start with the @ prefix
2024-11-01T16:13:31.5249639Z - Each user is also a comma-separated list of features/experiments to enable
2024-11-01T16:13:31.5250537Z - A "#" prefix opts the user out of all experiments
2024-11-01T16:13:31.5251050Z 
2024-11-01T16:13:31.5251191Z Example config:
2024-11-01T16:13:31.5251681Z     # A list of experiments that can be opted into.
2024-11-01T16:13:31.5252475Z     # This defines the behavior they'll induce when opted into.
2024-11-01T16:13:31.5253174Z     # Expected syntax is:
2024-11-01T16:13:31.5253877Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2024-11-01T16:13:31.5254981Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2024-11-01T16:13:31.5255730Z 
2024-11-01T16:13:31.5255879Z     experiments:
2024-11-01T16:13:31.5256274Z       lf:
2024-11-01T16:13:31.5256647Z         rollout_percent: 25
2024-11-01T16:13:31.5257155Z         all_branches: false
2024-11-01T16:13:31.5257601Z         default: true
2024-11-01T16:13:31.5258037Z     ---
2024-11-01T16:13:31.5258279Z 
2024-11-01T16:13:31.5258457Z     # Opt-ins:
2024-11-01T16:13:31.5259103Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2024-11-01T16:13:31.5260406Z     # and specifying experiments to enable in a comma-separated list.
2024-11-01T16:13:31.5261269Z     # Experiments should be from the above list.
2024-11-01T16:13:31.5261708Z 
2024-11-01T16:13:31.5261863Z     @User1,lf,split_build
2024-11-01T16:13:31.5262285Z     @User2,lf
2024-11-01T16:13:31.5262737Z     @User3,split_build
2024-11-01T16:13:31.5263100Z """
2024-11-01T16:13:31.5263302Z 
2024-11-01T16:13:31.5263452Z import logging
2024-11-01T16:13:31.5263895Z import os
2024-11-01T16:13:31.5264216Z import random
2024-11-01T16:13:31.5264833Z from argparse import ArgumentParser
2024-11-01T16:13:31.5265423Z from logging import LogRecord
2024-11-01T16:13:31.5266135Z from typing import Any, Dict, FrozenSet, Iterable, List, NamedTuple, Tuple
2024-11-01T16:13:31.5266781Z 
2024-11-01T16:13:31.5266912Z import yaml
2024-11-01T16:13:31.5267364Z from github import Auth, Github
2024-11-01T16:13:31.5267829Z from github.Issue import Issue
2024-11-01T16:13:31.5268177Z 
2024-11-01T16:13:31.5268183Z 
2024-11-01T16:13:31.5268384Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2024-11-01T16:13:31.5269175Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2024-11-01T16:13:31.5270109Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2024-11-01T16:13:31.5270758Z 
2024-11-01T16:13:31.5270970Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2024-11-01T16:13:31.5271666Z GH_OUTPUT_KEY_AMI = "runner-ami"
2024-11-01T16:13:31.5272208Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2024-11-01T16:13:31.5272595Z 
2024-11-01T16:13:31.5272606Z 
2024-11-01T16:13:31.5272782Z SETTING_EXPERIMENTS = "experiments"
2024-11-01T16:13:31.5273234Z 
2024-11-01T16:13:31.5273394Z LF_FLEET_EXPERIMENT = "lf"
2024-11-01T16:13:31.5273872Z CANARY_FLEET_SUFFIX = ".c"
2024-11-01T16:13:31.5274144Z 
2024-11-01T16:13:31.5274149Z 
2024-11-01T16:13:31.5274312Z class Experiment(NamedTuple):
2024-11-01T16:13:31.5274846Z     rollout_perc: float = (
2024-11-01T16:13:31.5275611Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2024-11-01T16:13:31.5276309Z     )
2024-11-01T16:13:31.5276738Z     all_branches: bool = (
2024-11-01T16:13:31.5277408Z         False  # If True, the experiment is also enabled on the exception branches
2024-11-01T16:13:31.5278106Z     )
2024-11-01T16:13:31.5278527Z     default: bool = (
2024-11-01T16:13:31.5279145Z         True  # If True, the experiment is enabled by default for all queries
2024-11-01T16:13:31.5279803Z     )
2024-11-01T16:13:31.5280088Z 
2024-11-01T16:13:31.5280250Z     # Add more fields as needed
2024-11-01T16:13:31.5280598Z 
2024-11-01T16:13:31.5280603Z 
2024-11-01T16:13:31.5280797Z class Settings(NamedTuple):
2024-11-01T16:13:31.5281199Z     """
2024-11-01T16:13:31.5281925Z     Settings for the experiments that can be opted into.
2024-11-01T16:13:31.5282537Z     """
2024-11-01T16:13:31.5282712Z 
2024-11-01T16:13:31.5282916Z     experiments: Dict[str, Experiment] = {}
2024-11-01T16:13:31.5283403Z 
2024-11-01T16:13:31.5283407Z 
2024-11-01T16:13:31.5283599Z class ColorFormatter(logging.Formatter):
2024-11-01T16:13:31.5284260Z     """Color codes the log messages based on the log level"""
2024-11-01T16:13:31.5284719Z 
2024-11-01T16:13:31.5284967Z     COLORS = {
2024-11-01T16:13:31.5285339Z         "WARNING": "\033[33m",  # Yellow
2024-11-01T16:13:31.5285860Z         "ERROR": "\033[31m",  # Red
2024-11-01T16:13:31.5286446Z         "CRITICAL": "\033[31m",  # Red
2024-11-01T16:13:31.5286926Z         "INFO": "\033[0m",  # Reset
2024-11-01T16:13:31.5287419Z         "DEBUG": "\033[0m",  # Reset
2024-11-01T16:13:31.5287981Z     }
2024-11-01T16:13:31.5288156Z 
2024-11-01T16:13:31.5288416Z     def format(self, record: LogRecord) -> str:
2024-11-01T16:13:31.5289258Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2024-11-01T16:13:31.5290305Z         record.msg = f"{log_color}{record.msg}\033[0m"
2024-11-01T16:13:31.5290893Z         return super().format(record)
2024-11-01T16:13:31.5291277Z 
2024-11-01T16:13:31.5291282Z 
2024-11-01T16:13:31.5291455Z handler = logging.StreamHandler()
2024-11-01T16:13:31.5292320Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2024-11-01T16:13:31.5292918Z 
2024-11-01T16:13:31.5293185Z log = logging.getLogger(os.path.basename(__file__))
2024-11-01T16:13:31.5293779Z log.addHandler(handler)
2024-11-01T16:13:31.5294308Z log.setLevel(logging.INFO)
2024-11-01T16:13:31.5294589Z 
2024-11-01T16:13:31.5294593Z 
2024-11-01T16:13:31.5294931Z def set_github_output(key: str, value: str) -> None:
2024-11-01T16:13:31.5295477Z     """
2024-11-01T16:13:31.5296183Z     Defines outputs of the github action that invokes this script
2024-11-01T16:13:31.5296875Z     """
2024-11-01T16:13:31.5297197Z     if not GITHUB_OUTPUT:
2024-11-01T16:13:31.5298606Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2024-11-01T16:13:31.5300239Z         log.warning(
2024-11-01T16:13:31.5301198Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2024-11-01T16:13:31.5302351Z         )
2024-11-01T16:13:31.5325546Z         print(f"::set-output name={key}::{value}")
2024-11-01T16:13:31.5326169Z         return
2024-11-01T16:13:31.5326383Z 
2024-11-01T16:13:31.5326703Z     with open(GITHUB_OUTPUT, "a") as f:
2024-11-01T16:13:31.5327393Z         log.info(f"Setting output: {key}='{value}'")
2024-11-01T16:13:31.5328006Z         f.write(f"{key}={value}\n")
2024-11-01T16:13:31.5328351Z 
2024-11-01T16:13:31.5328367Z 
2024-11-01T16:13:31.5328828Z def _str_comma_separated_to_set(value: str) -> FrozenSet[str]:
2024-11-01T16:13:31.5329469Z     return frozenset(
2024-11-01T16:13:31.5330133Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2024-11-01T16:13:31.5330978Z     )
2024-11-01T16:13:31.5331157Z 
2024-11-01T16:13:31.5331162Z 
2024-11-01T16:13:31.5331349Z def parse_args() -> Any:
2024-11-01T16:13:31.5331930Z     parser = ArgumentParser("Get dynamic rollout settings")
2024-11-01T16:13:31.5333005Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2024-11-01T16:13:31.5333798Z     parser.add_argument(
2024-11-01T16:13:31.5334295Z         "--github-issue-repo",
2024-11-01T16:13:31.5334823Z         type=str,
2024-11-01T16:13:31.5335177Z         required=False,
2024-11-01T16:13:31.5335677Z         default="pytorch/test-infra",
2024-11-01T16:13:31.5336323Z         help="GitHub repo to get the issue",
2024-11-01T16:13:31.5336826Z     )
2024-11-01T16:13:31.5337200Z     parser.add_argument(
2024-11-01T16:13:31.5337736Z         "--github-repo",
2024-11-01T16:13:31.5338121Z         type=str,
2024-11-01T16:13:31.5338751Z         required=True,
2024-11-01T16:13:31.5339292Z         help="GitHub repo where CI is running",
2024-11-01T16:13:31.5340050Z     )
2024-11-01T16:13:31.5340414Z     parser.add_argument(
2024-11-01T16:13:31.5341223Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2024-11-01T16:13:31.5341930Z     )
2024-11-01T16:13:31.5342280Z     parser.add_argument(
2024-11-01T16:13:31.5343065Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2024-11-01T16:13:31.5343779Z     )
2024-11-01T16:13:31.5344127Z     parser.add_argument(
2024-11-01T16:13:31.5344942Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2024-11-01T16:13:31.5345652Z     )
2024-11-01T16:13:31.5346015Z     parser.add_argument(
2024-11-01T16:13:31.5346873Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2024-11-01T16:13:31.5347614Z     )
2024-11-01T16:13:31.5347969Z     parser.add_argument(
2024-11-01T16:13:31.5348535Z         "--github-ref-type",
2024-11-01T16:13:31.5348954Z         type=str,
2024-11-01T16:13:31.5349335Z         required=True,
2024-11-01T16:13:31.5349915Z         help="Current GitHub ref type, branch or tag",
2024-11-01T16:13:31.5350461Z     )
2024-11-01T16:13:31.5350812Z     parser.add_argument(
2024-11-01T16:13:31.5351391Z         "--eligible-experiments",
2024-11-01T16:13:31.5351881Z         type=_str_comma_separated_to_set,
2024-11-01T16:13:31.5352414Z         required=False,
2024-11-01T16:13:31.5352908Z         default="",
2024-11-01T16:13:31.5353806Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2024-11-01T16:13:31.5354849Z     )
2024-11-01T16:13:31.5355024Z 
2024-11-01T16:13:31.5355287Z     return parser.parse_args()
2024-11-01T16:13:31.5355593Z 
2024-11-01T16:13:31.5355758Z 
2024-11-01T16:13:31.5356054Z def get_gh_client(github_token: str) -> Github:
2024-11-01T16:13:31.5356678Z     auth = Auth.Token(github_token)
2024-11-01T16:13:31.5357280Z     return Github(auth=auth)
2024-11-01T16:13:31.5357567Z 
2024-11-01T16:13:31.5357572Z 
2024-11-01T16:13:31.5357979Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:
2024-11-01T16:13:31.5358627Z     repo = gh.get_repo(repo)
2024-11-01T16:13:31.5359215Z     return repo.get_issue(number=issue_num)
2024-11-01T16:13:31.5359589Z 
2024-11-01T16:13:31.5359594Z 
2024-11-01T16:13:31.5359816Z def get_potential_pr_author(
2024-11-01T16:13:31.5360454Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2024-11-01T16:13:31.5361272Z ) -> str:
2024-11-01T16:13:31.5361841Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2024-11-01T16:13:31.5362725Z     # Fetch the actual username from the original PR. The PR number is
2024-11-01T16:13:31.5363668Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2024-11-01T16:13:31.5364116Z 
2024-11-01T16:13:31.5364333Z     gh = get_gh_client(github_token)
2024-11-01T16:13:31.5364675Z 
2024-11-01T16:13:31.5364991Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2024-11-01T16:13:31.5365733Z         split_tag = ref_name.split("/")
2024-11-01T16:13:31.5366260Z         if (
2024-11-01T16:13:31.5366599Z             len(split_tag) == 3
2024-11-01T16:13:31.5367160Z             and split_tag[0] == "ciflow"
2024-11-01T16:13:31.5367723Z             and split_tag[2].isnumeric()
2024-11-01T16:13:31.5368180Z         ):
2024-11-01T16:13:31.5368634Z             pr_number = split_tag[2]
2024-11-01T16:13:31.5369122Z             try:
2024-11-01T16:13:31.5369520Z                 repository = gh.get_repo(repo)
2024-11-01T16:13:31.5370259Z                 pull = repository.get_pull(number=int(pr_number))
2024-11-01T16:13:31.5370900Z             except Exception as e:
2024-11-01T16:13:31.5371442Z                 raise Exception(  # noqa: TRY002
2024-11-01T16:13:31.5372219Z                     f"issue with pull request {pr_number} from repo {repository}"
2024-11-01T16:13:31.5372909Z                 ) from e
2024-11-01T16:13:31.5373522Z             return pull.user.login
2024-11-01T16:13:31.5374184Z     # In all other cases, return the original input username
2024-11-01T16:13:31.5374810Z     return username
2024-11-01T16:13:31.5375037Z 
2024-11-01T16:13:31.5375042Z 
2024-11-01T16:13:31.5375343Z def is_exception_branch(branch: str) -> bool:
2024-11-01T16:13:31.5375923Z     """
2024-11-01T16:13:31.5376672Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2024-11-01T16:13:31.5377531Z     """
2024-11-01T16:13:31.5378159Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2024-11-01T16:13:31.5378769Z 
2024-11-01T16:13:31.5378774Z 
2024-11-01T16:13:31.5378986Z def load_yaml(yaml_text: str) -> Any:
2024-11-01T16:13:31.5379502Z     try:
2024-11-01T16:13:31.5380162Z         data = yaml.safe_load(yaml_text)
2024-11-01T16:13:31.5380721Z         return data
2024-11-01T16:13:31.5381163Z     except yaml.YAMLError as exc:
2024-11-01T16:13:31.5381747Z         log.exception("Error loading YAML")
2024-11-01T16:13:31.5382299Z         raise
2024-11-01T16:13:31.5382496Z 
2024-11-01T16:13:31.5382501Z 
2024-11-01T16:13:31.5383034Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> Tuple[str, str]:
2024-11-01T16:13:31.5383893Z     """
2024-11-01T16:13:31.5384552Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2024-11-01T16:13:31.5385259Z 
2024-11-01T16:13:31.5385685Z     If the issue body contains "---" then the text above that is the settings
2024-11-01T16:13:31.5386591Z     and the text below is the list of opted in users.
2024-11-01T16:13:31.5387039Z 
2024-11-01T16:13:31.5387511Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2024-11-01T16:13:31.5388286Z     """
2024-11-01T16:13:31.5389004Z     rollout_state_parts = rollout_state.split("---")
2024-11-01T16:13:31.5389610Z     if len(rollout_state_parts) >= 2:
2024-11-01T16:13:31.5390236Z         return rollout_state_parts[0], rollout_state_parts[1]
2024-11-01T16:13:31.5390946Z     else:
2024-11-01T16:13:31.5391282Z         return "", rollout_state
2024-11-01T16:13:31.5391621Z 
2024-11-01T16:13:31.5391626Z 
2024-11-01T16:13:31.5391803Z class UserOptins(Dict[str, List[str]]):
2024-11-01T16:13:31.5392392Z     """
2024-11-01T16:13:31.5392889Z     Dictionary of users with a list of features they have opted into
2024-11-01T16:13:31.5393572Z     """
2024-11-01T16:13:31.5393824Z 
2024-11-01T16:13:31.5393865Z 
2024-11-01T16:13:31.5394272Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2024-11-01T16:13:31.5394963Z     """
2024-11-01T16:13:31.5395806Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2024-11-01T16:13:31.5396720Z 
2024-11-01T16:13:31.5397516Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2024-11-01T16:13:31.5398692Z         - Example line: "@User1,lf,split_build"
2024-11-01T16:13:31.5399471Z         - A "#" prefix indicates the user is opted out of all experiments
2024-11-01T16:13:31.5400084Z 
2024-11-01T16:13:31.5400088Z 
2024-11-01T16:13:31.5400218Z     """
2024-11-01T16:13:31.5400578Z     optins = UserOptins()
2024-11-01T16:13:31.5401056Z     for user in user_optin_text.split("\n"):
2024-11-01T16:13:31.5401731Z         user = user.strip("\r\n\t -")
2024-11-01T16:13:31.5402294Z         if not user or not user.startswith("@"):
2024-11-01T16:13:31.5402951Z             # Not a valid user. Skip
2024-11-01T16:13:31.5403410Z             continue
2024-11-01T16:13:31.5403670Z 
2024-11-01T16:13:31.5403800Z         if user:
2024-11-01T16:13:31.5404315Z             usr_name = user.split(",")[0].strip("@")
2024-11-01T16:13:31.5405043Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2024-11-01T16:13:31.5405609Z 
2024-11-01T16:13:31.5405767Z     return optins
2024-11-01T16:13:31.5406000Z 
2024-11-01T16:13:31.5406004Z 
2024-11-01T16:13:31.5406626Z def parse_settings_from_text(settings_text: str) -> Settings:
2024-11-01T16:13:31.5407262Z     """
2024-11-01T16:13:31.5407893Z     Parse the experiments from the issue body into a list of ExperimentSettings
2024-11-01T16:13:31.5408727Z     """
2024-11-01T16:13:31.5409029Z     try:
2024-11-01T16:13:31.5409404Z         if settings_text:
2024-11-01T16:13:31.5410265Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2024-11-01T16:13:31.5411121Z             # for easy reading
2024-11-01T16:13:31.5412105Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2024-11-01T16:13:31.5413197Z             #       the backtick character in shell commands.
2024-11-01T16:13:31.5413837Z             backtick = chr(96)  # backtick character
2024-11-01T16:13:31.5414565Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2024-11-01T16:13:31.5415337Z             settings = load_yaml(settings_text)
2024-11-01T16:13:31.5415727Z 
2024-11-01T16:13:31.5416222Z             # For now we just load experiments. We can expand this if/when we add more settings
2024-11-01T16:13:31.5417012Z             experiments = {}
2024-11-01T16:13:31.5417326Z 
2024-11-01T16:13:31.5417773Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2024-11-01T16:13:31.5418562Z                 valid_settings = {}
2024-11-01T16:13:31.5419058Z                 for setting in exp_settings:
2024-11-01T16:13:31.5419966Z                     if setting not in Experiment._fields:
2024-11-01T16:13:31.5420625Z                         log.warning(
2024-11-01T16:13:31.5421351Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2024-11-01T16:13:31.5422206Z                         )
2024-11-01T16:13:31.5422810Z                     else:
2024-11-01T16:13:31.5423321Z                         valid_settings[setting] = exp_settings[setting]
2024-11-01T16:13:31.5423837Z 
2024-11-01T16:13:31.5424172Z                 experiments[exp_name] = Experiment(**valid_settings)
2024-11-01T16:13:31.5424839Z             return Settings(experiments)
2024-11-01T16:13:31.5425188Z 
2024-11-01T16:13:31.5425390Z     except Exception:
2024-11-01T16:13:31.5425903Z         log.exception("Failed to parse settings")
2024-11-01T16:13:31.5426346Z 
2024-11-01T16:13:31.5426492Z     return Settings()
2024-11-01T16:13:31.5426736Z 
2024-11-01T16:13:31.5426741Z 
2024-11-01T16:13:31.5427095Z def parse_settings(rollout_state: str) -> Settings:
2024-11-01T16:13:31.5427707Z     """
2024-11-01T16:13:31.5428134Z     Parse settings, if any, from the rollout state.
2024-11-01T16:13:31.5428551Z 
2024-11-01T16:13:31.5429030Z     If the issue body contains "---" then the text above that is the settings
2024-11-01T16:13:31.5429905Z     and the text below is the list of opted in users.
2024-11-01T16:13:31.5430375Z 
2024-11-01T16:13:31.5430886Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2024-11-01T16:13:31.5431713Z     """
2024-11-01T16:13:31.5432346Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2024-11-01T16:13:31.5433113Z     return parse_settings_from_text(settings_text)
2024-11-01T16:13:31.5433571Z 
2024-11-01T16:13:31.5433575Z 
2024-11-01T16:13:31.5433847Z def parse_users(rollout_state: str) -> UserOptins:
2024-11-01T16:13:31.5434497Z     """
2024-11-01T16:13:31.5434835Z     Parse users from the rollout state.
2024-11-01T16:13:31.5435240Z 
2024-11-01T16:13:31.5435365Z     """
2024-11-01T16:13:31.5435959Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2024-11-01T16:13:31.5436715Z     return parse_user_opt_in_from_text(users_text)
2024-11-01T16:13:31.5437155Z 
2024-11-01T16:13:31.5437160Z 
2024-11-01T16:13:31.5437684Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2024-11-01T16:13:31.5438562Z     """
2024-11-01T16:13:31.5438964Z     Check if a user is opted into an experiment
2024-11-01T16:13:31.5439676Z     """
2024-11-01T16:13:31.5440219Z     return experiment_name in user_optins.get(user, [])
2024-11-01T16:13:31.5440658Z 
2024-11-01T16:13:31.5440663Z 
2024-11-01T16:13:31.5440844Z def get_runner_prefix(
2024-11-01T16:13:31.5441231Z     rollout_state: str,
2024-11-01T16:13:31.5441797Z     workflow_requestors: Iterable[str],
2024-11-01T16:13:31.5442317Z     branch: str,
2024-11-01T16:13:31.5442768Z     eligible_experiments: FrozenSet[str] = frozenset(),
2024-11-01T16:13:31.5443479Z     is_canary: bool = False,
2024-11-01T16:13:31.5443954Z ) -> str:
2024-11-01T16:13:31.5444329Z     settings = parse_settings(rollout_state)
2024-11-01T16:13:31.5445021Z     user_optins = parse_users(rollout_state)
2024-11-01T16:13:31.5445396Z 
2024-11-01T16:13:31.5445574Z     fleet_prefix = ""
2024-11-01T16:13:31.5445957Z     prefixes = []
2024-11-01T16:13:31.5446685Z     for experiment_name, experiment_settings in settings.experiments.items():
2024-11-01T16:13:31.5447698Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2024-11-01T16:13:31.5448423Z             log.info(
2024-11-01T16:13:31.5449246Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2024-11-01T16:13:31.5450074Z             )
2024-11-01T16:13:31.5450411Z             continue
2024-11-01T16:13:31.5450739Z 
2024-11-01T16:13:31.5450902Z         if eligible_experiments:
2024-11-01T16:13:31.5451480Z             if experiment_name not in eligible_experiments:
2024-11-01T16:13:31.5452142Z                 exp_list = ", ".join(eligible_experiments)
2024-11-01T16:13:31.5452791Z                 log.info(
2024-11-01T16:13:31.5453718Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2024-11-01T16:13:31.5454613Z                 )
2024-11-01T16:13:31.5455179Z                 continue
2024-11-01T16:13:31.5455687Z         elif not experiment_settings.default:
2024-11-01T16:13:31.5456210Z             log.info(
2024-11-01T16:13:31.5457029Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2024-11-01T16:13:31.5458064Z             )
2024-11-01T16:13:31.5458507Z             continue
2024-11-01T16:13:31.5458897Z 
2024-11-01T16:13:31.5459173Z         # Is any workflow_requestor opted in to this experiment?
2024-11-01T16:13:31.5460281Z         opted_in_users = [
2024-11-01T16:13:31.5460892Z             requestor
2024-11-01T16:13:31.5461426Z             for requestor in workflow_requestors
2024-11-01T16:13:31.5462162Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2024-11-01T16:13:31.5462824Z         ]
2024-11-01T16:13:31.5463070Z 
2024-11-01T16:13:31.5463215Z         enabled = False
2024-11-01T16:13:31.5463657Z         if opted_in_users:
2024-11-01T16:13:31.5464100Z             log.info(
2024-11-01T16:13:31.5464875Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2024-11-01T16:13:31.5465621Z             )
2024-11-01T16:13:31.5465994Z             enabled = True
2024-11-01T16:13:31.5466328Z 
2024-11-01T16:13:31.5466550Z         elif experiment_settings.rollout_perc:
2024-11-01T16:13:31.5467494Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2024-11-01T16:13:31.5468573Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2024-11-01T16:13:31.5469312Z                 log.info(
2024-11-01T16:13:31.5470235Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2024-11-01T16:13:31.5471232Z                 )
2024-11-01T16:13:31.5471664Z                 enabled = True
2024-11-01T16:13:31.5471986Z 
2024-11-01T16:13:31.5472122Z         if enabled:
2024-11-01T16:13:31.5472556Z             label = experiment_name
2024-11-01T16:13:31.5473179Z             if experiment_name == LF_FLEET_EXPERIMENT:
2024-11-01T16:13:31.5474102Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2024-11-01T16:13:31.5475376Z                 #  - If it's enabled, then we always list it's prefix first
2024-11-01T16:13:31.5476396Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2024-11-01T16:13:31.5477110Z                 if is_canary:
2024-11-01T16:13:31.5477623Z                     label += CANARY_FLEET_SUFFIX
2024-11-01T16:13:31.5478258Z                 fleet_prefix = label
2024-11-01T16:13:31.5478711Z             else:
2024-11-01T16:13:31.5479135Z                 prefixes.append(label)
2024-11-01T16:13:31.5479548Z 
2024-11-01T16:13:31.5479734Z     if len(prefixes) > 1:
2024-11-01T16:13:31.5480147Z         log.error(
2024-11-01T16:13:31.5481518Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2024-11-01T16:13:31.5482870Z         )
2024-11-01T16:13:31.5483226Z         prefixes = prefixes[:1]
2024-11-01T16:13:31.5483567Z 
2024-11-01T16:13:31.5483722Z     # Fleet always comes first
2024-11-01T16:13:31.5484279Z     if fleet_prefix:
2024-11-01T16:13:31.5484712Z         prefixes.insert(0, fleet_prefix)
2024-11-01T16:13:31.5485102Z 
2024-11-01T16:13:31.5485341Z     return ".".join(prefixes) + "." if prefixes else ""
2024-11-01T16:13:31.5485875Z 
2024-11-01T16:13:31.5485880Z 
2024-11-01T16:13:31.5486388Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2024-11-01T16:13:31.5487213Z     """
2024-11-01T16:13:31.5487787Z     Gets the first comment of the issue, which contains the desired rollout state.
2024-11-01T16:13:31.5488500Z 
2024-11-01T16:13:31.5488991Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2024-11-01T16:13:31.5489755Z     """
2024-11-01T16:13:31.5490405Z     gh = get_gh_client(github_token)
2024-11-01T16:13:31.5491068Z     issue = get_issue(gh, repo, issue_num)
2024-11-01T16:13:31.5491733Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2024-11-01T16:13:31.5492189Z 
2024-11-01T16:13:31.5492193Z 
2024-11-01T16:13:31.5492370Z def main() -> None:
2024-11-01T16:13:31.5492870Z     args = parse_args()
2024-11-01T16:13:31.5493121Z 
2024-11-01T16:13:31.5493357Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2024-11-01T16:13:31.5493746Z 
2024-11-01T16:13:31.5494003Z     try:
2024-11-01T16:13:31.5494397Z         rollout_state = get_rollout_state_from_issue(
2024-11-01T16:13:31.5495153Z             args.github_token, args.github_issue_repo, args.github_issue
2024-11-01T16:13:31.5495905Z         )
2024-11-01T16:13:31.5496094Z 
2024-11-01T16:13:31.5496276Z         username = get_potential_pr_author(
2024-11-01T16:13:31.5496830Z             args.github_token,
2024-11-01T16:13:31.5497374Z             args.github_repo,
2024-11-01T16:13:31.5497810Z             args.github_actor,
2024-11-01T16:13:31.5498272Z             args.github_ref_type,
2024-11-01T16:13:31.5498834Z             args.github_branch,
2024-11-01T16:13:31.5499265Z         )
2024-11-01T16:13:31.5499483Z 
2024-11-01T16:13:31.5500941Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2024-11-01T16:13:31.5501475Z 
2024-11-01T16:13:31.5501806Z         runner_label_prefix = get_runner_prefix(
2024-11-01T16:13:31.5502347Z             rollout_state,
2024-11-01T16:13:31.5502849Z             (args.github_issue_owner, username),
2024-11-01T16:13:31.5503472Z             args.github_branch,
2024-11-01T16:13:31.5503934Z             args.eligible_experiments,
2024-11-01T16:13:31.5504457Z             is_canary,
2024-11-01T16:13:31.5504916Z         )
2024-11-01T16:13:31.5505101Z 
2024-11-01T16:13:31.5505258Z     except Exception as e:
2024-11-01T16:13:31.5505710Z         log.error(
2024-11-01T16:13:31.5506509Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2024-11-01T16:13:31.5507315Z         )
2024-11-01T16:13:31.5507548Z 
2024-11-01T16:13:31.5507874Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2024-11-01T16:13:31.5508591Z 
2024-11-01T16:13:31.5508632Z 
2024-11-01T16:13:31.5508852Z if __name__ == "__main__":
2024-11-01T16:13:31.5509301Z     main()
2024-11-01T16:13:31.5509487Z 
2024-11-01T16:13:31.5693197Z ##[group]Run python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2024-11-01T16:13:31.5694191Z python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2024-11-01T16:13:31.5752077Z shell: /usr/bin/bash -e {0}
2024-11-01T16:13:31.5752585Z env:
2024-11-01T16:13:31.5753183Z   GITHUB_TOKEN: ***
2024-11-01T16:13:31.5753707Z   ISSUE_NUMBER: 5132
2024-11-01T16:13:31.5754179Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2024-11-01T16:13:31.5754664Z   ISSUE_OWNER: 
2024-11-01T16:13:31.5755135Z   CHECK_EXPERIMENTS: 
2024-11-01T16:13:31.5755553Z ##[endgroup]
2024-11-01T16:13:31.9929762Z Defaulting to user installation because normal site-packages is not writeable
2024-11-01T16:13:32.3788673Z Collecting urllib3==1.26.18
2024-11-01T16:13:32.4370224Z   Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
2024-11-01T16:13:32.4619069Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.8/143.8 KB 6.1 MB/s eta 0:00:00
2024-11-01T16:13:32.4998054Z Collecting PyGithub==2.3.0
2024-11-01T16:13:32.5073705Z   Downloading PyGithub-2.3.0-py3-none-any.whl (354 kB)
2024-11-01T16:13:32.5356553Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 354.4/354.4 KB 13.3 MB/s eta 0:00:00
2024-11-01T16:13:32.5805156Z Collecting Deprecated
2024-11-01T16:13:32.5877361Z   Downloading Deprecated-1.2.14-py2.py3-none-any.whl (9.6 kB)
2024-11-01T16:13:32.6279588Z Collecting pynacl>=1.4.0
2024-11-01T16:13:32.6359177Z   Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
2024-11-01T16:13:32.6880679Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 856.7/856.7 KB 17.0 MB/s eta 0:00:00
2024-11-01T16:13:32.7231894Z Collecting pyjwt[crypto]>=2.4.0
2024-11-01T16:13:32.7336084Z   Downloading PyJWT-2.9.0-py3-none-any.whl (22 kB)
2024-11-01T16:13:32.7656772Z Collecting typing-extensions>=4.0.0
2024-11-01T16:13:32.7730076Z   Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB)
2024-11-01T16:13:32.7764164Z Requirement already satisfied: requests>=2.14.0 in /usr/lib/python3/dist-packages (from PyGithub==2.3.0) (2.25.1)
2024-11-01T16:13:32.7875049Z Requirement already satisfied: cryptography>=3.4.0 in /usr/lib/python3/dist-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.3.0) (3.4.8)
2024-11-01T16:13:33.0120861Z Collecting cffi>=1.4.1
2024-11-01T16:13:33.0203057Z   Downloading cffi-1.17.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (446 kB)
2024-11-01T16:13:33.0421031Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 446.2/446.2 KB 22.3 MB/s eta 0:00:00
2024-11-01T16:13:33.2287128Z Collecting wrapt<2,>=1.10
2024-11-01T16:13:33.2375986Z   Downloading wrapt-1.16.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (80 kB)
2024-11-01T16:13:33.2416419Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 80.3/80.3 KB 33.7 MB/s eta 0:00:00
2024-11-01T16:13:33.2618057Z Collecting pycparser
2024-11-01T16:13:33.2696586Z   Downloading pycparser-2.22-py3-none-any.whl (117 kB)
2024-11-01T16:13:33.2746434Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 KB 38.5 MB/s eta 0:00:00
2024-11-01T16:13:33.4509064Z Installing collected packages: wrapt, urllib3, typing-extensions, pyjwt, pycparser, Deprecated, cffi, pynacl, PyGithub
2024-11-01T16:13:33.9186091Z Successfully installed Deprecated-1.2.14 PyGithub-2.3.0 cffi-1.17.1 pycparser-2.22 pyjwt-2.9.0 pynacl-1.5.0 typing-extensions-4.12.2 urllib3-1.26.18 wrapt-1.16.0
2024-11-01T16:13:34.0026092Z ##[group]Run curr_branch="ciflow/trunk/138766"
2024-11-01T16:13:34.0026687Z curr_branch="ciflow/trunk/138766"
2024-11-01T16:13:34.0027158Z curr_ref_type="tag"
2024-11-01T16:13:34.0027675Z echo "Current branch is '$curr_branch'"
2024-11-01T16:13:34.0028126Z 
2024-11-01T16:13:34.0028454Z python3 runner_determinator.py \
2024-11-01T16:13:34.0029237Z   --github-token "$GITHUB_TOKEN" \
2024-11-01T16:13:34.0029729Z   --github-issue "$ISSUE_NUMBER" \
2024-11-01T16:13:34.0030155Z   --github-branch "$curr_branch" \
2024-11-01T16:13:34.0030683Z   --github-actor "$TRIGGERING_ACTOR" \
2024-11-01T16:13:34.0031191Z   --github-issue-owner "$ISSUE_OWNER" \
2024-11-01T16:13:34.0031683Z   --github-ref-type "$curr_ref_type" \
2024-11-01T16:13:34.0032189Z   --github-repo "$GITHUB_REPOSITORY" \
2024-11-01T16:13:34.0032825Z   --eligible-experiments "$CHECK_EXPERIMENTS" \
2024-11-01T16:13:34.0091288Z shell: /usr/bin/bash -e {0}
2024-11-01T16:13:34.0091653Z env:
2024-11-01T16:13:34.0092512Z   GITHUB_TOKEN: ***
2024-11-01T16:13:34.0092889Z   ISSUE_NUMBER: 5132
2024-11-01T16:13:34.0093262Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2024-11-01T16:13:34.0093721Z   ISSUE_OWNER: 
2024-11-01T16:13:34.0094069Z   CHECK_EXPERIMENTS: 
2024-11-01T16:13:34.0094409Z ##[endgroup]
2024-11-01T16:13:34.0171862Z Current branch is 'ciflow/trunk/138766'
2024-11-01T16:13:35.8342175Z INFO    : Skipping experiment 'awsa100', as it is not a default experiment
2024-11-01T16:13:35.8343750Z INFO    : Setting output: label-type=''
2024-11-01T16:13:35.8766312Z Evaluate and set job outputs
2024-11-01T16:13:35.8777892Z Cleaning up orphan processes