2024-11-01T16:13:31.1774934Z Current runner version: '2.320.0'
2024-11-01T16:13:31.1798772Z ##[group]Operating System
2024-11-01T16:13:31.1799563Z Ubuntu
2024-11-01T16:13:31.1799902Z 22.04.5
2024-11-01T16:13:31.1800253Z LTS
2024-11-01T16:13:31.1800557Z ##[endgroup]
2024-11-01T16:13:31.1801014Z ##[group]Runner Image
2024-11-01T16:13:31.1801479Z Image: ubuntu-22.04
2024-11-01T16:13:31.1801943Z Version: 20241015.1.0
2024-11-01T16:13:31.1802917Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20241015.1/images/ubuntu/Ubuntu2204-Readme.md
2024-11-01T16:13:31.1804369Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20241015.1
2024-11-01T16:13:31.1805284Z ##[endgroup]
2024-11-01T16:13:31.1805861Z ##[group]Runner Image Provisioner
2024-11-01T16:13:31.1806389Z 2.0.384.1
2024-11-01T16:13:31.1806810Z ##[endgroup]
2024-11-01T16:13:31.1820744Z ##[group]GITHUB_TOKEN Permissions
2024-11-01T16:13:31.1822468Z Contents: read
2024-11-01T16:13:31.1822877Z Metadata: read
2024-11-01T16:13:31.1823575Z Packages: read
2024-11-01T16:13:31.1824047Z ##[endgroup]
2024-11-01T16:13:31.1827294Z Secret source: Actions
2024-11-01T16:13:31.1827876Z Prepare workflow directory
2024-11-01T16:13:31.2792400Z Prepare all required actions
2024-11-01T16:13:31.2973376Z Uses: pytorch/pytorch/.github/workflows/_runner-determinator.yml@refs/heads/main (33dce10ece5b38aa0ab76739b658cd980a6e3d8f)
2024-11-01T16:13:31.2978659Z ##[group] Inputs
2024-11-01T16:13:31.2979165Z   check_experiments: 
2024-11-01T16:13:31.2979779Z   triggering_actor: pytorch-bot[bot]
2024-11-01T16:13:31.2980292Z   issue_owner: 
2024-11-01T16:13:31.2980723Z   curr_branch: ciflow/trunk/138766
2024-11-01T16:13:31.2981360Z   curr_ref_type: tag
2024-11-01T16:13:31.2981766Z   issue_number: 5132
2024-11-01T16:13:31.2982173Z ##[endgroup]
2024-11-01T16:13:31.2983100Z Complete job name: get-label-type / runner-determinator
2024-11-01T16:13:31.3864282Z ##[group]Run cat <<EOF > runner_determinator.py
2024-11-01T16:13:31.3866120Z cat <<EOF > runner_determinator.py
2024-11-01T16:13:31.3866767Z # flake8: noqa: G004
2024-11-01T16:13:31.3867326Z 
2024-11-01T16:13:31.3868043Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2024-11-01T16:13:31.3869284Z #       must be kept in sync. You can do it easily by running the following command:
2024-11-01T16:13:31.3870387Z #           python .github/scripts/update_runner_determinator.py
2024-11-01T16:13:31.3871102Z 
2024-11-01T16:13:31.3871441Z """
2024-11-01T16:13:31.3872193Z This runner determinator is used to determine which set of runners to run a
2024-11-01T16:13:31.3873279Z GitHub job on. It uses the first comment of a GitHub issue (by default
2024-11-01T16:13:31.3874573Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2024-11-01T16:13:31.3875597Z of which runners should be used to run which job.
2024-11-01T16:13:31.3876219Z 
2024-11-01T16:13:31.3876948Z The configuration has two parts, the settings and a list of opted-in users,
2024-11-01T16:13:31.3878052Z separated by a line containing "---".  If the line is not present, the
2024-11-01T16:13:31.3879130Z settings are considered to be empty with only the second part, the user
2024-11-01T16:13:31.3879968Z list, defined.
2024-11-01T16:13:31.3880407Z 
2024-11-01T16:13:31.3881095Z The first part is a YAML block that defines the rollout settings. This can be
2024-11-01T16:13:31.3882271Z used to define any settings that are needed to determine which runners to use.
2024-11-01T16:13:31.3883329Z It's fields are defined by the RolloutSettings class below.
2024-11-01T16:13:31.3884029Z 
2024-11-01T16:13:31.3884775Z The second part is a list of users who are explicitly opted in to the LF fleet.
2024-11-01T16:13:31.3886198Z The user list is also a comma separated list of additional features or
2024-11-01T16:13:31.3887730Z experiments which the user could be opted in to.
2024-11-01T16:13:31.3888499Z 
2024-11-01T16:13:31.3888903Z The user list has the following rules:
2024-11-01T16:13:31.3889467Z 
2024-11-01T16:13:31.3890154Z - Users are GitHub usernames, which must start with the @ prefix
2024-11-01T16:13:31.3891164Z - Each user is also a comma-separated list of features/experiments to enable
2024-11-01T16:13:31.3892128Z - A "#" prefix opts the user out of all experiments
2024-11-01T16:13:31.3892849Z 
2024-11-01T16:13:31.3893186Z Example config:
2024-11-01T16:13:31.3893766Z     # A list of experiments that can be opted into.
2024-11-01T16:13:31.3894684Z     # This defines the behavior they'll induce when opted into.
2024-11-01T16:13:31.3895426Z     # Expected syntax is:
2024-11-01T16:13:31.3896199Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2024-11-01T16:13:31.3897513Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2024-11-01T16:13:31.3898430Z 
2024-11-01T16:13:31.3898759Z     experiments:
2024-11-01T16:13:31.3899294Z       lf:
2024-11-01T16:13:31.3899716Z         rollout_percent: 25
2024-11-01T16:13:31.3900219Z         all_branches: false
2024-11-01T16:13:31.3900799Z         default: true
2024-11-01T16:13:31.3901260Z     ---
2024-11-01T16:13:31.3901613Z 
2024-11-01T16:13:31.3902034Z     # Opt-ins:
2024-11-01T16:13:31.3902783Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2024-11-01T16:13:31.3904058Z     # and specifying experiments to enable in a comma-separated list.
2024-11-01T16:13:31.3905009Z     # Experiments should be from the above list.
2024-11-01T16:13:31.3905651Z 
2024-11-01T16:13:31.3906044Z     @User1,lf,split_build
2024-11-01T16:13:31.3906595Z     @User2,lf
2024-11-01T16:13:31.3907044Z     @User3,split_build
2024-11-01T16:13:31.3907516Z """
2024-11-01T16:13:31.3907914Z 
2024-11-01T16:13:31.3908295Z import logging
2024-11-01T16:13:31.3908725Z import os
2024-11-01T16:13:31.3909167Z import random
2024-11-01T16:13:31.3909673Z from argparse import ArgumentParser
2024-11-01T16:13:31.3910275Z from logging import LogRecord
2024-11-01T16:13:31.3911121Z from typing import Any, Dict, FrozenSet, Iterable, List, NamedTuple, Tuple
2024-11-01T16:13:31.3911939Z 
2024-11-01T16:13:31.3912304Z import yaml
2024-11-01T16:13:31.3912893Z from github import Auth, Github
2024-11-01T16:13:31.3913450Z from github.Issue import Issue
2024-11-01T16:13:31.3913972Z 
2024-11-01T16:13:31.3914400Z 
2024-11-01T16:13:31.3914833Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2024-11-01T16:13:31.3915669Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2024-11-01T16:13:31.3916803Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2024-11-01T16:13:31.3917588Z 
2024-11-01T16:13:31.3918058Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2024-11-01T16:13:31.3918798Z GH_OUTPUT_KEY_AMI = "runner-ami"
2024-11-01T16:13:31.3919376Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2024-11-01T16:13:31.3919972Z 
2024-11-01T16:13:31.3920381Z 
2024-11-01T16:13:31.3920759Z SETTING_EXPERIMENTS = "experiments"
2024-11-01T16:13:31.3921321Z 
2024-11-01T16:13:31.3921768Z LF_FLEET_EXPERIMENT = "lf"
2024-11-01T16:13:31.3922271Z CANARY_FLEET_SUFFIX = ".c"
2024-11-01T16:13:31.3922768Z 
2024-11-01T16:13:31.3923170Z 
2024-11-01T16:13:31.3923692Z class Experiment(NamedTuple):
2024-11-01T16:13:31.3924267Z     rollout_perc: float = (
2024-11-01T16:13:31.3925130Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2024-11-01T16:13:31.3926298Z     )
2024-11-01T16:13:31.3926678Z     all_branches: bool = (
2024-11-01T16:13:31.3927567Z         False  # If True, the experiment is also enabled on the exception branches
2024-11-01T16:13:31.3928383Z     )
2024-11-01T16:13:31.3928740Z     default: bool = (
2024-11-01T16:13:31.3929540Z         True  # If True, the experiment is enabled by default for all queries
2024-11-01T16:13:31.3930317Z     )
2024-11-01T16:13:31.3930646Z 
2024-11-01T16:13:31.3931101Z     # Add more fields as needed
2024-11-01T16:13:31.3931650Z 
2024-11-01T16:13:31.3931957Z 
2024-11-01T16:13:31.3932432Z class Settings(NamedTuple):
2024-11-01T16:13:31.3932974Z     """
2024-11-01T16:13:31.3933489Z     Settings for the experiments that can be opted into.
2024-11-01T16:13:31.3934240Z     """
2024-11-01T16:13:31.3934613Z 
2024-11-01T16:13:31.3935018Z     experiments: Dict[str, Experiment] = {}
2024-11-01T16:13:31.3935684Z 
2024-11-01T16:13:31.3936020Z 
2024-11-01T16:13:31.3936422Z class ColorFormatter(logging.Formatter):
2024-11-01T16:13:31.3937276Z     """Color codes the log messages based on the log level"""
2024-11-01T16:13:31.3937948Z 
2024-11-01T16:13:31.3938271Z     COLORS = {
2024-11-01T16:13:31.3938847Z         "WARNING": "\033[33m",  # Yellow
2024-11-01T16:13:31.3939462Z         "ERROR": "\033[31m",  # Red
2024-11-01T16:13:31.3940203Z         "CRITICAL": "\033[31m",  # Red
2024-11-01T16:13:31.3940906Z         "INFO": "\033[0m",  # Reset
2024-11-01T16:13:31.3941490Z         "DEBUG": "\033[0m",  # Reset
2024-11-01T16:13:31.3942012Z     }
2024-11-01T16:13:31.3942444Z 
2024-11-01T16:13:31.3942905Z     def format(self, record: LogRecord) -> str:
2024-11-01T16:13:31.3943911Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2024-11-01T16:13:31.3944831Z         record.msg = f"{log_color}{record.msg}\033[0m"
2024-11-01T16:13:31.3945535Z         return super().format(record)
2024-11-01T16:13:31.3946155Z 
2024-11-01T16:13:31.3946475Z 
2024-11-01T16:13:31.3946901Z handler = logging.StreamHandler()
2024-11-01T16:13:31.3947858Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2024-11-01T16:13:31.3948624Z 
2024-11-01T16:13:31.3949150Z log = logging.getLogger(os.path.basename(__file__))
2024-11-01T16:13:31.3949899Z log.addHandler(handler)
2024-11-01T16:13:31.3950389Z log.setLevel(logging.INFO)
2024-11-01T16:13:31.3950910Z 
2024-11-01T16:13:31.3951309Z 
2024-11-01T16:13:31.3951777Z def set_github_output(key: str, value: str) -> None:
2024-11-01T16:13:31.3952446Z     """
2024-11-01T16:13:31.3953114Z     Defines outputs of the github action that invokes this script
2024-11-01T16:13:31.3953819Z     """
2024-11-01T16:13:31.3954240Z     if not GITHUB_OUTPUT:
2024-11-01T16:13:31.3955654Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2024-11-01T16:13:31.3957038Z         log.warning(
2024-11-01T16:13:31.3958088Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2024-11-01T16:13:31.3959263Z         )
2024-11-01T16:13:31.3959789Z         print(f"::set-output name={key}::{value}")
2024-11-01T16:13:31.3960537Z         return
2024-11-01T16:13:31.3961046Z 
2024-11-01T16:13:31.3961473Z     with open(GITHUB_OUTPUT, "a") as f:
2024-11-01T16:13:31.3962127Z         log.info(f"Setting output: {key}='{value}'")
2024-11-01T16:13:31.3962887Z         f.write(f"{key}={value}\n")
2024-11-01T16:13:31.3963416Z 
2024-11-01T16:13:31.3963721Z 
2024-11-01T16:13:31.3964379Z def _str_comma_separated_to_set(value: str) -> FrozenSet[str]:
2024-11-01T16:13:31.3965122Z     return frozenset(
2024-11-01T16:13:31.3966406Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2024-11-01T16:13:31.3967293Z     )
2024-11-01T16:13:31.3967667Z 
2024-11-01T16:13:31.3968026Z 
2024-11-01T16:13:31.3968441Z def parse_args() -> Any:
2024-11-01T16:13:31.3969140Z     parser = ArgumentParser("Get dynamic rollout settings")
2024-11-01T16:13:31.3970218Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2024-11-01T16:13:31.3971147Z     parser.add_argument(
2024-11-01T16:13:31.3971686Z         "--github-issue-repo",
2024-11-01T16:13:31.3972234Z         type=str,
2024-11-01T16:13:31.3972714Z         required=False,
2024-11-01T16:13:31.3973258Z         default="pytorch/test-infra",
2024-11-01T16:13:31.3973914Z         help="GitHub repo to get the issue",
2024-11-01T16:13:31.3974519Z     )
2024-11-01T16:13:31.3974917Z     parser.add_argument(
2024-11-01T16:13:31.3975435Z         "--github-repo",
2024-11-01T16:13:31.3975959Z         type=str,
2024-11-01T16:13:31.3976417Z         required=True,
2024-11-01T16:13:31.3977179Z         help="GitHub repo where CI is running",
2024-11-01T16:13:31.3977819Z     )
2024-11-01T16:13:31.3978235Z     parser.add_argument(
2024-11-01T16:13:31.3978988Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2024-11-01T16:13:31.3979810Z     )
2024-11-01T16:13:31.3980187Z     parser.add_argument(
2024-11-01T16:13:31.3980958Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2024-11-01T16:13:31.3981830Z     )
2024-11-01T16:13:31.3982192Z     parser.add_argument(
2024-11-01T16:13:31.3982968Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2024-11-01T16:13:31.3983892Z     )
2024-11-01T16:13:31.3984256Z     parser.add_argument(
2024-11-01T16:13:31.3985073Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2024-11-01T16:13:31.3985979Z     )
2024-11-01T16:13:31.3986339Z     parser.add_argument(
2024-11-01T16:13:31.3986847Z         "--github-ref-type",
2024-11-01T16:13:31.3987457Z         type=str,
2024-11-01T16:13:31.3987882Z         required=True,
2024-11-01T16:13:31.3988482Z         help="Current GitHub ref type, branch or tag",
2024-11-01T16:13:31.3989196Z     )
2024-11-01T16:13:31.3989551Z     parser.add_argument(
2024-11-01T16:13:31.3990091Z         "--eligible-experiments",
2024-11-01T16:13:31.3990789Z         type=_str_comma_separated_to_set,
2024-11-01T16:13:31.3991370Z         required=False,
2024-11-01T16:13:31.3991846Z         default="",
2024-11-01T16:13:31.3992974Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2024-11-01T16:13:31.3994073Z     )
2024-11-01T16:13:31.3994454Z 
2024-11-01T16:13:31.3994877Z     return parser.parse_args()
2024-11-01T16:13:31.3995401Z 
2024-11-01T16:13:31.3995759Z 
2024-11-01T16:13:31.3996304Z def get_gh_client(github_token: str) -> Github:
2024-11-01T16:13:31.3997128Z     auth = Auth.Token(github_token)
2024-11-01T16:13:31.3997701Z     return Github(auth=auth)
2024-11-01T16:13:31.3998279Z 
2024-11-01T16:13:31.3998609Z 
2024-11-01T16:13:31.3999167Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:
2024-11-01T16:13:31.3999978Z     repo = gh.get_repo(repo)
2024-11-01T16:13:31.4000557Z     return repo.get_issue(number=issue_num)
2024-11-01T16:13:31.4001133Z 
2024-11-01T16:13:31.4001562Z 
2024-11-01T16:13:31.4001909Z def get_potential_pr_author(
2024-11-01T16:13:31.4002694Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2024-11-01T16:13:31.4003551Z ) -> str:
2024-11-01T16:13:31.4004167Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2024-11-01T16:13:31.4005176Z     # Fetch the actual username from the original PR. The PR number is
2024-11-01T16:13:31.4006556Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2024-11-01T16:13:31.4007203Z 
2024-11-01T16:13:31.4007613Z     gh = get_gh_client(github_token)
2024-11-01T16:13:31.4008236Z 
2024-11-01T16:13:31.4008767Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2024-11-01T16:13:31.4009481Z         split_tag = ref_name.split("/")
2024-11-01T16:13:31.4010124Z         if (
2024-11-01T16:13:31.4010566Z             len(split_tag) == 3
2024-11-01T16:13:31.4011113Z             and split_tag[0] == "ciflow"
2024-11-01T16:13:31.4011813Z             and split_tag[2].isnumeric()
2024-11-01T16:13:31.4012376Z         ):
2024-11-01T16:13:31.4012973Z             pr_number = split_tag[2]
2024-11-01T16:13:31.4013624Z             try:
2024-11-01T16:13:31.4014145Z                 repository = gh.get_repo(repo)
2024-11-01T16:13:31.4014879Z                 pull = repository.get_pull(number=int(pr_number))
2024-11-01T16:13:31.4015663Z             except Exception as e:
2024-11-01T16:13:31.4016315Z                 raise Exception(  # noqa: TRY002
2024-11-01T16:13:31.4017102Z                     f"issue with pull request {pr_number} from repo {repository}"
2024-11-01T16:13:31.4017922Z                 ) from e
2024-11-01T16:13:31.4018471Z             return pull.user.login
2024-11-01T16:13:31.4019256Z     # In all other cases, return the original input username
2024-11-01T16:13:31.4019920Z     return username
2024-11-01T16:13:31.4020386Z 
2024-11-01T16:13:31.4020894Z 
2024-11-01T16:13:31.4021350Z def is_exception_branch(branch: str) -> bool:
2024-11-01T16:13:31.4021968Z     """
2024-11-01T16:13:31.4022824Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2024-11-01T16:13:31.4023699Z     """
2024-11-01T16:13:31.4024369Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2024-11-01T16:13:31.4025231Z 
2024-11-01T16:13:31.4025561Z 
2024-11-01T16:13:31.4025982Z def load_yaml(yaml_text: str) -> Any:
2024-11-01T16:13:31.4026606Z     try:
2024-11-01T16:13:31.4027044Z         data = yaml.safe_load(yaml_text)
2024-11-01T16:13:31.4027620Z         return data
2024-11-01T16:13:31.4028215Z     except yaml.YAMLError as exc:
2024-11-01T16:13:31.4028830Z         log.exception("Error loading YAML")
2024-11-01T16:13:31.4029413Z         raise
2024-11-01T16:13:31.4029886Z 
2024-11-01T16:13:31.4030210Z 
2024-11-01T16:13:31.4030905Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> Tuple[str, str]:
2024-11-01T16:13:31.4031824Z     """
2024-11-01T16:13:31.4032760Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2024-11-01T16:13:31.4033609Z 
2024-11-01T16:13:31.4034329Z     If the issue body contains "---" then the text above that is the settings
2024-11-01T16:13:31.4035266Z     and the text below is the list of opted in users.
2024-11-01T16:13:31.4035866Z 
2024-11-01T16:13:31.4036629Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2024-11-01T16:13:31.4037465Z     """
2024-11-01T16:13:31.4037948Z     rollout_state_parts = rollout_state.split("---")
2024-11-01T16:13:31.4038721Z     if len(rollout_state_parts) >= 2:
2024-11-01T16:13:31.4039452Z         return rollout_state_parts[0], rollout_state_parts[1]
2024-11-01T16:13:31.4040112Z     else:
2024-11-01T16:13:31.4040618Z         return "", rollout_state
2024-11-01T16:13:31.4041141Z 
2024-11-01T16:13:31.4041462Z 
2024-11-01T16:13:31.4041945Z class UserOptins(Dict[str, List[str]]):
2024-11-01T16:13:31.4042515Z     """
2024-11-01T16:13:31.4043172Z     Dictionary of users with a list of features they have opted into
2024-11-01T16:13:31.4043947Z     """
2024-11-01T16:13:31.4044315Z 
2024-11-01T16:13:31.4044670Z 
2024-11-01T16:13:31.4045291Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2024-11-01T16:13:31.4046398Z     """
2024-11-01T16:13:31.4047299Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2024-11-01T16:13:31.4048389Z 
2024-11-01T16:13:31.4049532Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2024-11-01T16:13:31.4050772Z         - Example line: "@User1,lf,split_build"
2024-11-01T16:13:31.4051638Z         - A "#" prefix indicates the user is opted out of all experiments
2024-11-01T16:13:31.4052386Z 
2024-11-01T16:13:31.4052734Z 
2024-11-01T16:13:31.4053107Z     """
2024-11-01T16:13:31.4053532Z     optins = UserOptins()
2024-11-01T16:13:31.4054122Z     for user in user_optin_text.split("\n"):
2024-11-01T16:13:31.4054842Z         user = user.strip("\r\n\t -")
2024-11-01T16:13:31.4055469Z         if not user or not user.startswith("@"):
2024-11-01T16:13:31.4056125Z             # Not a valid user. Skip
2024-11-01T16:13:31.4056754Z             continue
2024-11-01T16:13:31.4057167Z 
2024-11-01T16:13:31.4057520Z         if user:
2024-11-01T16:13:31.4058134Z             usr_name = user.split(",")[0].strip("@")
2024-11-01T16:13:31.4058945Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2024-11-01T16:13:31.4059703Z 
2024-11-01T16:13:31.4060160Z     return optins
2024-11-01T16:13:31.4060572Z 
2024-11-01T16:13:31.4060929Z 
2024-11-01T16:13:31.4061548Z def parse_settings_from_text(settings_text: str) -> Settings:
2024-11-01T16:13:31.4062234Z     """
2024-11-01T16:13:31.4062960Z     Parse the experiments from the issue body into a list of ExperimentSettings
2024-11-01T16:13:31.4063863Z     """
2024-11-01T16:13:31.4064199Z     try:
2024-11-01T16:13:31.4064617Z         if settings_text:
2024-11-01T16:13:31.4065614Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2024-11-01T16:13:31.4066593Z             # for easy reading
2024-11-01T16:13:31.4067591Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2024-11-01T16:13:31.4068779Z             #       the backtick character in shell commands.
2024-11-01T16:13:31.4069701Z             backtick = chr(96)  # backtick character
2024-11-01T16:13:31.4070483Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2024-11-01T16:13:31.4071357Z             settings = load_yaml(settings_text)
2024-11-01T16:13:31.4071955Z 
2024-11-01T16:13:31.4072630Z             # For now we just load experiments. We can expand this if/when we add more settings
2024-11-01T16:13:31.4073598Z             experiments = {}
2024-11-01T16:13:31.4074112Z 
2024-11-01T16:13:31.4074755Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2024-11-01T16:13:31.4075642Z                 valid_settings = {}
2024-11-01T16:13:31.4076289Z                 for setting in exp_settings:
2024-11-01T16:13:31.4076975Z                     if setting not in Experiment._fields:
2024-11-01T16:13:31.4077688Z                         log.warning(
2024-11-01T16:13:31.4078543Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2024-11-01T16:13:31.4079367Z                         )
2024-11-01T16:13:31.4079898Z                     else:
2024-11-01T16:13:31.4080535Z                         valid_settings[setting] = exp_settings[setting]
2024-11-01T16:13:31.4081190Z 
2024-11-01T16:13:31.4081791Z                 experiments[exp_name] = Experiment(**valid_settings)
2024-11-01T16:13:31.4082513Z             return Settings(experiments)
2024-11-01T16:13:31.4083076Z 
2024-11-01T16:13:31.4083530Z     except Exception:
2024-11-01T16:13:31.4084211Z         log.exception("Failed to parse settings")
2024-11-01T16:13:31.4084847Z 
2024-11-01T16:13:31.4085294Z     return Settings()
2024-11-01T16:13:31.4086011Z 
2024-11-01T16:13:31.4086394Z 
2024-11-01T16:13:31.4086974Z def parse_settings(rollout_state: str) -> Settings:
2024-11-01T16:13:31.4087592Z     """
2024-11-01T16:13:31.4088134Z     Parse settings, if any, from the rollout state.
2024-11-01T16:13:31.4088851Z 
2024-11-01T16:13:31.4089447Z     If the issue body contains "---" then the text above that is the settings
2024-11-01T16:13:31.4090398Z     and the text below is the list of opted in users.
2024-11-01T16:13:31.4091122Z 
2024-11-01T16:13:31.4091821Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2024-11-01T16:13:31.4092704Z     """
2024-11-01T16:13:31.4093419Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2024-11-01T16:13:31.4094344Z     return parse_settings_from_text(settings_text)
2024-11-01T16:13:31.4094930Z 
2024-11-01T16:13:31.4095336Z 
2024-11-01T16:13:31.4095841Z def parse_users(rollout_state: str) -> UserOptins:
2024-11-01T16:13:31.4096444Z     """
2024-11-01T16:13:31.4096957Z     Parse users from the rollout state.
2024-11-01T16:13:31.4097539Z 
2024-11-01T16:13:31.4097851Z     """
2024-11-01T16:13:31.4098563Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2024-11-01T16:13:31.4099442Z     return parse_user_opt_in_from_text(users_text)
2024-11-01T16:13:31.4100017Z 
2024-11-01T16:13:31.4100431Z 
2024-11-01T16:13:31.4101148Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2024-11-01T16:13:31.4101984Z     """
2024-11-01T16:13:31.4102558Z     Check if a user is opted into an experiment
2024-11-01T16:13:31.4103169Z     """
2024-11-01T16:13:31.4103664Z     return experiment_name in user_optins.get(user, [])
2024-11-01T16:13:31.4104590Z 
2024-11-01T16:13:31.4104937Z 
2024-11-01T16:13:31.4105299Z def get_runner_prefix(
2024-11-01T16:13:31.4105875Z     rollout_state: str,
2024-11-01T16:13:31.4106424Z     workflow_requestors: Iterable[str],
2024-11-01T16:13:31.4106991Z     branch: str,
2024-11-01T16:13:31.4107658Z     eligible_experiments: FrozenSet[str] = frozenset(),
2024-11-01T16:13:31.4108346Z     is_canary: bool = False,
2024-11-01T16:13:31.4108934Z ) -> str:
2024-11-01T16:13:31.4109391Z     settings = parse_settings(rollout_state)
2024-11-01T16:13:31.4110068Z     user_optins = parse_users(rollout_state)
2024-11-01T16:13:31.4110728Z 
2024-11-01T16:13:31.4111066Z     fleet_prefix = ""
2024-11-01T16:13:31.4111570Z     prefixes = []
2024-11-01T16:13:31.4112401Z     for experiment_name, experiment_settings in settings.experiments.items():
2024-11-01T16:13:31.4113496Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2024-11-01T16:13:31.4114326Z             log.info(
2024-11-01T16:13:31.4115236Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2024-11-01T16:13:31.4116087Z             )
2024-11-01T16:13:31.4116523Z             continue
2024-11-01T16:13:31.4117043Z 
2024-11-01T16:13:31.4117401Z         if eligible_experiments:
2024-11-01T16:13:31.4118098Z             if experiment_name not in eligible_experiments:
2024-11-01T16:13:31.4118931Z                 exp_list = ", ".join(eligible_experiments)
2024-11-01T16:13:31.4119587Z                 log.info(
2024-11-01T16:13:31.4120677Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2024-11-01T16:13:31.4121730Z                 )
2024-11-01T16:13:31.4122210Z                 continue
2024-11-01T16:13:31.4122748Z         elif not experiment_settings.default:
2024-11-01T16:13:31.4123415Z             log.info(
2024-11-01T16:13:31.4124220Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2024-11-01T16:13:31.4125020Z             )
2024-11-01T16:13:31.4125504Z             continue
2024-11-01T16:13:31.4126170Z 
2024-11-01T16:13:31.4126674Z         # Is any workflow_requestor opted in to this experiment?
2024-11-01T16:13:31.4127486Z         opted_in_users = [
2024-11-01T16:13:31.4127989Z             requestor
2024-11-01T16:13:31.4128554Z             for requestor in workflow_requestors
2024-11-01T16:13:31.4129414Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2024-11-01T16:13:31.4130140Z         ]
2024-11-01T16:13:31.4130531Z 
2024-11-01T16:13:31.4130949Z         enabled = False
2024-11-01T16:13:31.4131448Z         if opted_in_users:
2024-11-01T16:13:31.4131947Z             log.info(
2024-11-01T16:13:31.4132753Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2024-11-01T16:13:31.4133549Z             )
2024-11-01T16:13:31.4133993Z             enabled = True
2024-11-01T16:13:31.4134509Z 
2024-11-01T16:13:31.4134975Z         elif experiment_settings.rollout_perc:
2024-11-01T16:13:31.4136031Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2024-11-01T16:13:31.4137266Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2024-11-01T16:13:31.4137996Z                 log.info(
2024-11-01T16:13:31.4139048Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2024-11-01T16:13:31.4140347Z                 )
2024-11-01T16:13:31.4140777Z                 enabled = True
2024-11-01T16:13:31.4141301Z 
2024-11-01T16:13:31.4141732Z         if enabled:
2024-11-01T16:13:31.4142259Z             label = experiment_name
2024-11-01T16:13:31.4142925Z             if experiment_name == LF_FLEET_EXPERIMENT:
2024-11-01T16:13:31.4144016Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2024-11-01T16:13:31.4145159Z                 #  - If it's enabled, then we always list it's prefix first
2024-11-01T16:13:31.4146099Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2024-11-01T16:13:31.4146971Z                 if is_canary:
2024-11-01T16:13:31.4147616Z                     label += CANARY_FLEET_SUFFIX
2024-11-01T16:13:31.4148222Z                 fleet_prefix = label
2024-11-01T16:13:31.4148835Z             else:
2024-11-01T16:13:31.4149350Z                 prefixes.append(label)
2024-11-01T16:13:31.4149866Z 
2024-11-01T16:13:31.4150321Z     if len(prefixes) > 1:
2024-11-01T16:13:31.4150831Z         log.error(
2024-11-01T16:13:31.4152088Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2024-11-01T16:13:31.4153529Z         )
2024-11-01T16:13:31.4153965Z         prefixes = prefixes[:1]
2024-11-01T16:13:31.4154483Z 
2024-11-01T16:13:31.4154915Z     # Fleet always comes first
2024-11-01T16:13:31.4155450Z     if fleet_prefix:
2024-11-01T16:13:31.4156157Z         prefixes.insert(0, fleet_prefix)
2024-11-01T16:13:31.4156783Z 
2024-11-01T16:13:31.4157282Z     return ".".join(prefixes) + "." if prefixes else ""
2024-11-01T16:13:31.4157941Z 
2024-11-01T16:13:31.4158307Z 
2024-11-01T16:13:31.4159017Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2024-11-01T16:13:31.4159898Z     """
2024-11-01T16:13:31.4160641Z     Gets the first comment of the issue, which contains the desired rollout state.
2024-11-01T16:13:31.4161460Z 
2024-11-01T16:13:31.4162135Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2024-11-01T16:13:31.4163046Z     """
2024-11-01T16:13:31.4163501Z     gh = get_gh_client(github_token)
2024-11-01T16:13:31.4164186Z     issue = get_issue(gh, repo, issue_num)
2024-11-01T16:13:31.4164950Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2024-11-01T16:13:31.4165892Z 
2024-11-01T16:13:31.4166376Z 
2024-11-01T16:13:31.4166758Z def main() -> None:
2024-11-01T16:13:31.4167222Z     args = parse_args()
2024-11-01T16:13:31.4167772Z 
2024-11-01T16:13:31.4168250Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2024-11-01T16:13:31.4168818Z 
2024-11-01T16:13:31.4169229Z     try:
2024-11-01T16:13:31.4169766Z         rollout_state = get_rollout_state_from_issue(
2024-11-01T16:13:31.4170585Z             args.github_token, args.github_issue_repo, args.github_issue
2024-11-01T16:13:31.4171386Z         )
2024-11-01T16:13:31.4171785Z 
2024-11-01T16:13:31.4172187Z         username = get_potential_pr_author(
2024-11-01T16:13:31.4172865Z             args.github_token,
2024-11-01T16:13:31.4173430Z             args.github_repo,
2024-11-01T16:13:31.4173938Z             args.github_actor,
2024-11-01T16:13:31.4174582Z             args.github_ref_type,
2024-11-01T16:13:31.4175169Z             args.github_branch,
2024-11-01T16:13:31.4175825Z         )
2024-11-01T16:13:31.4176298Z 
2024-11-01T16:13:31.4176849Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2024-11-01T16:13:31.4177502Z 
2024-11-01T16:13:31.4178034Z         runner_label_prefix = get_runner_prefix(
2024-11-01T16:13:31.4178662Z             rollout_state,
2024-11-01T16:13:31.4179210Z             (args.github_issue_owner, username),
2024-11-01T16:13:31.4179925Z             args.github_branch,
2024-11-01T16:13:31.4180514Z             args.eligible_experiments,
2024-11-01T16:13:31.4181104Z             is_canary,
2024-11-01T16:13:31.4181600Z         )
2024-11-01T16:13:31.4181980Z 
2024-11-01T16:13:31.4182376Z     except Exception as e:
2024-11-01T16:13:31.4182918Z         log.error(
2024-11-01T16:13:31.4183748Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2024-11-01T16:13:31.4184646Z         )
2024-11-01T16:13:31.4185051Z 
2024-11-01T16:13:31.4185638Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2024-11-01T16:13:31.4186386Z 
2024-11-01T16:13:31.4186757Z 
2024-11-01T16:13:31.4187126Z if __name__ == "__main__":
2024-11-01T16:13:31.4187637Z     main()
2024-11-01T16:13:31.4188059Z 
2024-11-01T16:13:31.4188397Z EOF
2024-11-01T16:13:31.4188771Z 
2024-11-01T16:13:31.4189172Z cat runner_determinator.py
2024-11-01T16:13:31.4668034Z shell: /usr/bin/bash -e {0}
2024-11-01T16:13:31.4668684Z env:
2024-11-01T16:13:31.4669410Z   GITHUB_TOKEN: ***
2024-11-01T16:13:31.4669893Z   ISSUE_NUMBER: 5132
2024-11-01T16:13:31.4670623Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2024-11-01T16:13:31.4671189Z   ISSUE_OWNER: 
2024-11-01T16:13:31.4671618Z   CHECK_EXPERIMENTS: 
2024-11-01T16:13:31.4672111Z ##[endgroup]
2024-11-01T16:13:31.5097717Z # flake8: noqa: G004
2024-11-01T16:13:31.5098148Z 
2024-11-01T16:13:31.5099792Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2024-11-01T16:13:31.5100960Z #       must be kept in sync. You can do it easily by running the following command:
2024-11-01T16:13:31.5101847Z #           python .github/scripts/update_runner_determinator.py
2024-11-01T16:13:31.5102487Z 
2024-11-01T16:13:31.5102616Z """
2024-11-01T16:13:31.5103228Z This runner determinator is used to determine which set of runners to run a
2024-11-01T16:13:31.5104179Z GitHub job on. It uses the first comment of a GitHub issue (by default
2024-11-01T16:13:31.5105371Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2024-11-01T16:13:31.5106317Z of which runners should be used to run which job.
2024-11-01T16:13:31.5106742Z 
2024-11-01T16:13:31.5107200Z The configuration has two parts, the settings and a list of opted-in users,
2024-11-01T16:13:31.5108362Z separated by a line containing "---".  If the line is not present, the
2024-11-01T16:13:31.5109311Z settings are considered to be empty with only the second part, the user
2024-11-01T16:13:31.5110137Z list, defined.
2024-11-01T16:13:31.5110352Z 
2024-11-01T16:13:31.5110755Z The first part is a YAML block that defines the rollout settings. This can be
2024-11-01T16:13:31.5111787Z used to define any settings that are needed to determine which runners to use.
2024-11-01T16:13:31.5112837Z It's fields are defined by the RolloutSettings class below.
2024-11-01T16:13:31.5113323Z 
2024-11-01T16:13:31.5113733Z The second part is a list of users who are explicitly opted in to the LF fleet.
2024-11-01T16:13:31.5114721Z The user list is also a comma separated list of additional features or
2024-11-01T16:13:31.5115639Z experiments which the user could be opted in to.
2024-11-01T16:13:31.5116063Z 
2024-11-01T16:13:31.5116283Z The user list has the following rules:
2024-11-01T16:13:31.5116636Z 
2024-11-01T16:13:31.5117258Z - Users are GitHub usernames, which must start with the @ prefix
2024-11-01T16:13:31.5118338Z - Each user is also a comma-separated list of features/experiments to enable
2024-11-01T16:13:31.5119245Z - A "#" prefix opts the user out of all experiments
2024-11-01T16:13:31.5119686Z 
2024-11-01T16:13:31.5119828Z Example config:
2024-11-01T16:13:31.5120363Z     # A list of experiments that can be opted into.
2024-11-01T16:13:31.5121145Z     # This defines the behavior they'll induce when opted into.
2024-11-01T16:13:31.5121788Z     # Expected syntax is:
2024-11-01T16:13:31.5122546Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2024-11-01T16:13:31.5123674Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2024-11-01T16:13:31.5124362Z 
2024-11-01T16:13:31.5124511Z     experiments:
2024-11-01T16:13:31.5124954Z       lf:
2024-11-01T16:13:31.5125359Z         rollout_percent: 25
2024-11-01T16:13:31.5126158Z         all_branches: false
2024-11-01T16:13:31.5126695Z         default: true
2024-11-01T16:13:31.5127158Z     ---
2024-11-01T16:13:31.5127341Z 
2024-11-01T16:13:31.5127500Z     # Opt-ins:
2024-11-01T16:13:31.5128206Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2024-11-01T16:13:31.5129253Z     # and specifying experiments to enable in a comma-separated list.
2024-11-01T16:13:31.5129988Z     # Experiments should be from the above list.
2024-11-01T16:13:31.5130424Z 
2024-11-01T16:13:31.5130642Z     @User1,lf,split_build
2024-11-01T16:13:31.5131087Z     @User2,lf
2024-11-01T16:13:31.5131454Z     @User3,split_build
2024-11-01T16:13:31.5131885Z """
2024-11-01T16:13:31.5132118Z 
2024-11-01T16:13:31.5132257Z import logging
2024-11-01T16:13:31.5132626Z import os
2024-11-01T16:13:31.5133021Z import random
2024-11-01T16:13:31.5133606Z from argparse import ArgumentParser
2024-11-01T16:13:31.5134122Z from logging import LogRecord
2024-11-01T16:13:31.5134933Z from typing import Any, Dict, FrozenSet, Iterable, List, NamedTuple, Tuple
2024-11-01T16:13:31.5135562Z 
2024-11-01T16:13:31.5135696Z import yaml
2024-11-01T16:13:31.5136098Z from github import Auth, Github
2024-11-01T16:13:31.5136639Z from github.Issue import Issue
2024-11-01T16:13:31.5136978Z 
2024-11-01T16:13:31.5136982Z 
2024-11-01T16:13:31.5137187Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2024-11-01T16:13:31.5137934Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2024-11-01T16:13:31.5138931Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2024-11-01T16:13:31.5139583Z 
2024-11-01T16:13:31.5139792Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2024-11-01T16:13:31.5140440Z GH_OUTPUT_KEY_AMI = "runner-ami"
2024-11-01T16:13:31.5141019Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2024-11-01T16:13:31.5141417Z 
2024-11-01T16:13:31.5141427Z 
2024-11-01T16:13:31.5141602Z SETTING_EXPERIMENTS = "experiments"
2024-11-01T16:13:31.5141989Z 
2024-11-01T16:13:31.5142144Z LF_FLEET_EXPERIMENT = "lf"
2024-11-01T16:13:31.5142669Z CANARY_FLEET_SUFFIX = ".c"
2024-11-01T16:13:31.5142942Z 
2024-11-01T16:13:31.5142946Z 
2024-11-01T16:13:31.5143109Z class Experiment(NamedTuple):
2024-11-01T16:13:31.5143598Z     rollout_perc: float = (
2024-11-01T16:13:31.5144419Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2024-11-01T16:13:31.5145121Z     )
2024-11-01T16:13:31.5145490Z     all_branches: bool = (
2024-11-01T16:13:31.5146235Z         False  # If True, the experiment is also enabled on the exception branches
2024-11-01T16:13:31.5146948Z     )
2024-11-01T16:13:31.5147293Z     default: bool = (
2024-11-01T16:13:31.5147956Z         True  # If True, the experiment is enabled by default for all queries
2024-11-01T16:13:31.5148634Z     )
2024-11-01T16:13:31.5148835Z 
2024-11-01T16:13:31.5148998Z     # Add more fields as needed
2024-11-01T16:13:31.5149301Z 
2024-11-01T16:13:31.5149306Z 
2024-11-01T16:13:31.5149564Z class Settings(NamedTuple):
2024-11-01T16:13:31.5149985Z     """
2024-11-01T16:13:31.5150601Z     Settings for the experiments that can be opted into.
2024-11-01T16:13:31.5151294Z     """
2024-11-01T16:13:31.5151470Z 
2024-11-01T16:13:31.5151662Z     experiments: Dict[str, Experiment] = {}
2024-11-01T16:13:31.5152084Z 
2024-11-01T16:13:31.5152089Z 
2024-11-01T16:13:31.5152282Z class ColorFormatter(logging.Formatter):
2024-11-01T16:13:31.5153029Z     """Color codes the log messages based on the log level"""
2024-11-01T16:13:31.5153488Z 
2024-11-01T16:13:31.5153661Z     COLORS = {
2024-11-01T16:13:31.5154026Z         "WARNING": "\033[33m",  # Yellow
2024-11-01T16:13:31.5154632Z         "ERROR": "\033[31m",  # Red
2024-11-01T16:13:31.5155133Z         "CRITICAL": "\033[31m",  # Red
2024-11-01T16:13:31.5155607Z         "INFO": "\033[0m",  # Reset
2024-11-01T16:13:31.5156172Z         "DEBUG": "\033[0m",  # Reset
2024-11-01T16:13:31.5156653Z     }
2024-11-01T16:13:31.5156823Z 
2024-11-01T16:13:31.5157081Z     def format(self, record: LogRecord) -> str:
2024-11-01T16:13:31.5158108Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2024-11-01T16:13:31.5158972Z         record.msg = f"{log_color}{record.msg}\033[0m"
2024-11-01T16:13:31.5159562Z         return super().format(record)
2024-11-01T16:13:31.5159994Z 
2024-11-01T16:13:31.5160000Z 
2024-11-01T16:13:31.5160174Z handler = logging.StreamHandler()
2024-11-01T16:13:31.5160973Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2024-11-01T16:13:31.5161576Z 
2024-11-01T16:13:31.5161843Z log = logging.getLogger(os.path.basename(__file__))
2024-11-01T16:13:31.5162503Z log.addHandler(handler)
2024-11-01T16:13:31.5162946Z log.setLevel(logging.INFO)
2024-11-01T16:13:31.5163239Z 
2024-11-01T16:13:31.5163243Z 
2024-11-01T16:13:31.5163559Z def set_github_output(key: str, value: str) -> None:
2024-11-01T16:13:31.5164171Z     """
2024-11-01T16:13:31.5164829Z     Defines outputs of the github action that invokes this script
2024-11-01T16:13:31.5165516Z     """
2024-11-01T16:13:31.5166279Z     if not GITHUB_OUTPUT:
2024-11-01T16:13:31.5167673Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2024-11-01T16:13:31.5168965Z         log.warning(
2024-11-01T16:13:31.5169967Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2024-11-01T16:13:31.5171029Z         )
2024-11-01T16:13:31.5193715Z         print(f"::set-output name={key}::{value}")
2024-11-01T16:13:31.5194515Z         return
2024-11-01T16:13:31.5194745Z 
2024-11-01T16:13:31.5194928Z     with open(GITHUB_OUTPUT, "a") as f:
2024-11-01T16:13:31.5195596Z         log.info(f"Setting output: {key}='{value}'")
2024-11-01T16:13:31.5196304Z         f.write(f"{key}={value}\n")
2024-11-01T16:13:31.5196632Z 
2024-11-01T16:13:31.5196648Z 
2024-11-01T16:13:31.5197035Z def _str_comma_separated_to_set(value: str) -> FrozenSet[str]:
2024-11-01T16:13:31.5197680Z     return frozenset(
2024-11-01T16:13:31.5198460Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2024-11-01T16:13:31.5199211Z     )
2024-11-01T16:13:31.5199389Z 
2024-11-01T16:13:31.5199394Z 
2024-11-01T16:13:31.5199582Z def parse_args() -> Any:
2024-11-01T16:13:31.5200239Z     parser = ArgumentParser("Get dynamic rollout settings")
2024-11-01T16:13:31.5201255Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2024-11-01T16:13:31.5202066Z     parser.add_argument(
2024-11-01T16:13:31.5202634Z         "--github-issue-repo",
2024-11-01T16:13:31.5203111Z         type=str,
2024-11-01T16:13:31.5203488Z         required=False,
2024-11-01T16:13:31.5204047Z         default="pytorch/test-infra",
2024-11-01T16:13:31.5204609Z         help="GitHub repo to get the issue",
2024-11-01T16:13:31.5205126Z     )
2024-11-01T16:13:31.5205911Z     parser.add_argument(
2024-11-01T16:13:31.5206454Z         "--github-repo",
2024-11-01T16:13:31.5206864Z         type=str,
2024-11-01T16:13:31.5207578Z         required=True,
2024-11-01T16:13:31.5208080Z         help="GitHub repo where CI is running",
2024-11-01T16:13:31.5208584Z     )
2024-11-01T16:13:31.5209003Z     parser.add_argument(
2024-11-01T16:13:31.5209741Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2024-11-01T16:13:31.5210421Z     )
2024-11-01T16:13:31.5210849Z     parser.add_argument(
2024-11-01T16:13:31.5211595Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2024-11-01T16:13:31.5212290Z     )
2024-11-01T16:13:31.5212716Z     parser.add_argument(
2024-11-01T16:13:31.5213461Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2024-11-01T16:13:31.5214167Z     )
2024-11-01T16:13:31.5214591Z     parser.add_argument(
2024-11-01T16:13:31.5215348Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2024-11-01T16:13:31.5216083Z     )
2024-11-01T16:13:31.5216514Z     parser.add_argument(
2024-11-01T16:13:31.5216988Z         "--github-ref-type",
2024-11-01T16:13:31.5217400Z         type=str,
2024-11-01T16:13:31.5217866Z         required=True,
2024-11-01T16:13:31.5218366Z         help="Current GitHub ref type, branch or tag",
2024-11-01T16:13:31.5218917Z     )
2024-11-01T16:13:31.5219341Z     parser.add_argument(
2024-11-01T16:13:31.5219829Z         "--eligible-experiments",
2024-11-01T16:13:31.5220322Z         type=_str_comma_separated_to_set,
2024-11-01T16:13:31.5220930Z         required=False,
2024-11-01T16:13:31.5221340Z         default="",
2024-11-01T16:13:31.5222237Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2024-11-01T16:13:31.5223341Z     )
2024-11-01T16:13:31.5223516Z 
2024-11-01T16:13:31.5223715Z     return parser.parse_args()
2024-11-01T16:13:31.5224017Z 
2024-11-01T16:13:31.5224193Z 
2024-11-01T16:13:31.5224477Z def get_gh_client(github_token: str) -> Github:
2024-11-01T16:13:31.5225154Z     auth = Auth.Token(github_token)
2024-11-01T16:13:31.5225677Z     return Github(auth=auth)
2024-11-01T16:13:31.5225988Z 
2024-11-01T16:13:31.5225993Z 
2024-11-01T16:13:31.5226449Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:
2024-11-01T16:13:31.5227095Z     repo = gh.get_repo(repo)
2024-11-01T16:13:31.5227609Z     return repo.get_issue(number=issue_num)
2024-11-01T16:13:31.5227983Z 
2024-11-01T16:13:31.5227988Z 
2024-11-01T16:13:31.5228250Z def get_potential_pr_author(
2024-11-01T16:13:31.5228896Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2024-11-01T16:13:31.5229663Z ) -> str:
2024-11-01T16:13:31.5230287Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2024-11-01T16:13:31.5231151Z     # Fetch the actual username from the original PR. The PR number is
2024-11-01T16:13:31.5232026Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2024-11-01T16:13:31.5232467Z 
2024-11-01T16:13:31.5232737Z     gh = get_gh_client(github_token)
2024-11-01T16:13:31.5233081Z 
2024-11-01T16:13:31.5233397Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2024-11-01T16:13:31.5234124Z         split_tag = ref_name.split("/")
2024-11-01T16:13:31.5234705Z         if (
2024-11-01T16:13:31.5235051Z             len(split_tag) == 3
2024-11-01T16:13:31.5235561Z             and split_tag[0] == "ciflow"
2024-11-01T16:13:31.5236164Z             and split_tag[2].isnumeric()
2024-11-01T16:13:31.5236627Z         ):
2024-11-01T16:13:31.5237020Z             pr_number = split_tag[2]
2024-11-01T16:13:31.5237590Z             try:
2024-11-01T16:13:31.5238010Z                 repository = gh.get_repo(repo)
2024-11-01T16:13:31.5238673Z                 pull = repository.get_pull(number=int(pr_number))
2024-11-01T16:13:31.5239367Z             except Exception as e:
2024-11-01T16:13:31.5239930Z                 raise Exception(  # noqa: TRY002
2024-11-01T16:13:31.5240634Z                     f"issue with pull request {pr_number} from repo {repository}"
2024-11-01T16:13:31.5241392Z                 ) from e
2024-11-01T16:13:31.5241992Z             return pull.user.login
2024-11-01T16:13:31.5242568Z     # In all other cases, return the original input username
2024-11-01T16:13:31.5243282Z     return username
2024-11-01T16:13:31.5243513Z 
2024-11-01T16:13:31.5243518Z 
2024-11-01T16:13:31.5243805Z def is_exception_branch(branch: str) -> bool:
2024-11-01T16:13:31.5244321Z     """
2024-11-01T16:13:31.5245169Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2024-11-01T16:13:31.5246358Z     """
2024-11-01T16:13:31.5246918Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2024-11-01T16:13:31.5247545Z 
2024-11-01T16:13:31.5247550Z 
2024-11-01T16:13:31.5247873Z def load_yaml(yaml_text: str) -> Any:
2024-11-01T16:13:31.5248390Z     try:
2024-11-01T16:13:31.5248746Z         data = yaml.safe_load(yaml_text)
2024-11-01T16:13:31.5249352Z         return data
2024-11-01T16:13:31.5249774Z     except yaml.YAMLError as exc:
2024-11-01T16:13:31.5250279Z         log.exception("Error loading YAML")
2024-11-01T16:13:31.5250893Z         raise
2024-11-01T16:13:31.5251095Z 
2024-11-01T16:13:31.5251100Z 
2024-11-01T16:13:31.5251622Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> Tuple[str, str]:
2024-11-01T16:13:31.5252421Z     """
2024-11-01T16:13:31.5253125Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2024-11-01T16:13:31.5253829Z 
2024-11-01T16:13:31.5254258Z     If the issue body contains "---" then the text above that is the settings
2024-11-01T16:13:31.5255109Z     and the text below is the list of opted in users.
2024-11-01T16:13:31.5255533Z 
2024-11-01T16:13:31.5256069Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2024-11-01T16:13:31.5256853Z     """
2024-11-01T16:13:31.5257545Z     rollout_state_parts = rollout_state.split("---")
2024-11-01T16:13:31.5258227Z     if len(rollout_state_parts) >= 2:
2024-11-01T16:13:31.5258871Z         return rollout_state_parts[0], rollout_state_parts[1]
2024-11-01T16:13:31.5259508Z     else:
2024-11-01T16:13:31.5259900Z         return "", rollout_state
2024-11-01T16:13:31.5260258Z 
2024-11-01T16:13:31.5260263Z 
2024-11-01T16:13:31.5260440Z class UserOptins(Dict[str, List[str]]):
2024-11-01T16:13:31.5260948Z     """
2024-11-01T16:13:31.5261506Z     Dictionary of users with a list of features they have opted into
2024-11-01T16:13:31.5262205Z     """
2024-11-01T16:13:31.5262412Z 
2024-11-01T16:13:31.5262418Z 
2024-11-01T16:13:31.5262814Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2024-11-01T16:13:31.5263586Z     """
2024-11-01T16:13:31.5264431Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2024-11-01T16:13:31.5265260Z 
2024-11-01T16:13:31.5266049Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2024-11-01T16:13:31.5267293Z         - Example line: "@User1,lf,split_build"
2024-11-01T16:13:31.5268055Z         - A "#" prefix indicates the user is opted out of all experiments
2024-11-01T16:13:31.5268603Z 
2024-11-01T16:13:31.5268608Z 
2024-11-01T16:13:31.5268736Z     """
2024-11-01T16:13:31.5269185Z     optins = UserOptins()
2024-11-01T16:13:31.5269650Z     for user in user_optin_text.split("\n"):
2024-11-01T16:13:31.5270264Z         user = user.strip("\r\n\t -")
2024-11-01T16:13:31.5270910Z         if not user or not user.startswith("@"):
2024-11-01T16:13:31.5271489Z             # Not a valid user. Skip
2024-11-01T16:13:31.5271946Z             continue
2024-11-01T16:13:31.5272293Z 
2024-11-01T16:13:31.5272434Z         if user:
2024-11-01T16:13:31.5272874Z             usr_name = user.split(",")[0].strip("@")
2024-11-01T16:13:31.5273595Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2024-11-01T16:13:31.5274240Z 
2024-11-01T16:13:31.5274382Z     return optins
2024-11-01T16:13:31.5274603Z 
2024-11-01T16:13:31.5274607Z 
2024-11-01T16:13:31.5275151Z def parse_settings_from_text(settings_text: str) -> Settings:
2024-11-01T16:13:31.5275783Z     """
2024-11-01T16:13:31.5276469Z     Parse the experiments from the issue body into a list of ExperimentSettings
2024-11-01T16:13:31.5277245Z     """
2024-11-01T16:13:31.5277555Z     try:
2024-11-01T16:13:31.5277975Z         if settings_text:
2024-11-01T16:13:31.5278790Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2024-11-01T16:13:31.5279637Z             # for easy reading
2024-11-01T16:13:31.5280734Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2024-11-01T16:13:31.5281774Z             #       the backtick character in shell commands.
2024-11-01T16:13:31.5282398Z             backtick = chr(96)  # backtick character
2024-11-01T16:13:31.5283179Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2024-11-01T16:13:31.5283896Z             settings = load_yaml(settings_text)
2024-11-01T16:13:31.5284280Z 
2024-11-01T16:13:31.5284753Z             # For now we just load experiments. We can expand this if/when we add more settings
2024-11-01T16:13:31.5286160Z             experiments = {}
2024-11-01T16:13:31.5286541Z 
2024-11-01T16:13:31.5286916Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2024-11-01T16:13:31.5287683Z                 valid_settings = {}
2024-11-01T16:13:31.5288269Z                 for setting in exp_settings:
2024-11-01T16:13:31.5288895Z                     if setting not in Experiment._fields:
2024-11-01T16:13:31.5289494Z                         log.warning(
2024-11-01T16:13:31.5290280Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2024-11-01T16:13:31.5291062Z                         )
2024-11-01T16:13:31.5291644Z                     else:
2024-11-01T16:13:31.5292244Z                         valid_settings[setting] = exp_settings[setting]
2024-11-01T16:13:31.5292734Z 
2024-11-01T16:13:31.5292995Z                 experiments[exp_name] = Experiment(**valid_settings)
2024-11-01T16:13:31.5293670Z             return Settings(experiments)
2024-11-01T16:13:31.5294117Z 
2024-11-01T16:13:31.5294303Z     except Exception:
2024-11-01T16:13:31.5294756Z         log.exception("Failed to parse settings")
2024-11-01T16:13:31.5295197Z 
2024-11-01T16:13:31.5295356Z     return Settings()
2024-11-01T16:13:31.5295664Z 
2024-11-01T16:13:31.5295668Z 
2024-11-01T16:13:31.5296011Z def parse_settings(rollout_state: str) -> Settings:
2024-11-01T16:13:31.5296560Z     """
2024-11-01T16:13:31.5297015Z     Parse settings, if any, from the rollout state.
2024-11-01T16:13:31.5297504Z 
2024-11-01T16:13:31.5297978Z     If the issue body contains "---" then the text above that is the settings
2024-11-01T16:13:31.5298783Z     and the text below is the list of opted in users.
2024-11-01T16:13:31.5299264Z 
2024-11-01T16:13:31.5299783Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2024-11-01T16:13:31.5300664Z     """
2024-11-01T16:13:31.5301227Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2024-11-01T16:13:31.5302010Z     return parse_settings_from_text(settings_text)
2024-11-01T16:13:31.5302518Z 
2024-11-01T16:13:31.5302523Z 
2024-11-01T16:13:31.5302797Z def parse_users(rollout_state: str) -> UserOptins:
2024-11-01T16:13:31.5303391Z     """
2024-11-01T16:13:31.5303735Z     Parse users from the rollout state.
2024-11-01T16:13:31.5304187Z 
2024-11-01T16:13:31.5304315Z     """
2024-11-01T16:13:31.5304877Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2024-11-01T16:13:31.5305617Z     return parse_user_opt_in_from_text(users_text)
2024-11-01T16:13:31.5306120Z 
2024-11-01T16:13:31.5306125Z 
2024-11-01T16:13:31.5306646Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2024-11-01T16:13:31.5307476Z     """
2024-11-01T16:13:31.5307847Z     Check if a user is opted into an experiment
2024-11-01T16:13:31.5308636Z     """
2024-11-01T16:13:31.5309106Z     return experiment_name in user_optins.get(user, [])
2024-11-01T16:13:31.5309543Z 
2024-11-01T16:13:31.5309548Z 
2024-11-01T16:13:31.5309800Z def get_runner_prefix(
2024-11-01T16:13:31.5310212Z     rollout_state: str,
2024-11-01T16:13:31.5310662Z     workflow_requestors: Iterable[str],
2024-11-01T16:13:31.5311250Z     branch: str,
2024-11-01T16:13:31.5311721Z     eligible_experiments: FrozenSet[str] = frozenset(),
2024-11-01T16:13:31.5312348Z     is_canary: bool = False,
2024-11-01T16:13:31.5312882Z ) -> str:
2024-11-01T16:13:31.5313280Z     settings = parse_settings(rollout_state)
2024-11-01T16:13:31.5313877Z     user_optins = parse_users(rollout_state)
2024-11-01T16:13:31.5314256Z 
2024-11-01T16:13:31.5314505Z     fleet_prefix = ""
2024-11-01T16:13:31.5314903Z     prefixes = []
2024-11-01T16:13:31.5315547Z     for experiment_name, experiment_settings in settings.experiments.items():
2024-11-01T16:13:31.5316640Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2024-11-01T16:13:31.5317367Z             log.info(
2024-11-01T16:13:31.5318091Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2024-11-01T16:13:31.5319000Z             )
2024-11-01T16:13:31.5319330Z             continue
2024-11-01T16:13:31.5319597Z 
2024-11-01T16:13:31.5319762Z         if eligible_experiments:
2024-11-01T16:13:31.5320413Z             if experiment_name not in eligible_experiments:
2024-11-01T16:13:31.5321058Z                 exp_list = ", ".join(eligible_experiments)
2024-11-01T16:13:31.5321635Z                 log.info(
2024-11-01T16:13:31.5322643Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2024-11-01T16:13:31.5323528Z                 )
2024-11-01T16:13:31.5324048Z                 continue
2024-11-01T16:13:31.5324602Z         elif not experiment_settings.default:
2024-11-01T16:13:31.5325126Z             log.info(
2024-11-01T16:13:31.5326241Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2024-11-01T16:13:31.5327153Z             )
2024-11-01T16:13:31.5327496Z             continue
2024-11-01T16:13:31.5327782Z 
2024-11-01T16:13:31.5328052Z         # Is any workflow_requestor opted in to this experiment?
2024-11-01T16:13:31.5328766Z         opted_in_users = [
2024-11-01T16:13:31.5329204Z             requestor
2024-11-01T16:13:31.5329641Z             for requestor in workflow_requestors
2024-11-01T16:13:31.5330454Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2024-11-01T16:13:31.5331140Z         ]
2024-11-01T16:13:31.5331328Z 
2024-11-01T16:13:31.5331474Z         enabled = False
2024-11-01T16:13:31.5331958Z         if opted_in_users:
2024-11-01T16:13:31.5332417Z             log.info(
2024-11-01T16:13:31.5333109Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2024-11-01T16:13:31.5333922Z             )
2024-11-01T16:13:31.5334317Z             enabled = True
2024-11-01T16:13:31.5334584Z 
2024-11-01T16:13:31.5334788Z         elif experiment_settings.rollout_perc:
2024-11-01T16:13:31.5335790Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2024-11-01T16:13:31.5336884Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2024-11-01T16:13:31.5337544Z                 log.info(
2024-11-01T16:13:31.5338531Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2024-11-01T16:13:31.5339547Z                 )
2024-11-01T16:13:31.5339899Z                 enabled = True
2024-11-01T16:13:31.5340219Z 
2024-11-01T16:13:31.5340423Z         if enabled:
2024-11-01T16:13:31.5340863Z             label = experiment_name
2024-11-01T16:13:31.5341400Z             if experiment_name == LF_FLEET_EXPERIMENT:
2024-11-01T16:13:31.5342419Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2024-11-01T16:13:31.5343661Z                 #  - If it's enabled, then we always list it's prefix first
2024-11-01T16:13:31.5344587Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2024-11-01T16:13:31.5345386Z                 if is_canary:
2024-11-01T16:13:31.5345881Z                     label += CANARY_FLEET_SUFFIX
2024-11-01T16:13:31.5346457Z                 fleet_prefix = label
2024-11-01T16:13:31.5346971Z             else:
2024-11-01T16:13:31.5347384Z                 prefixes.append(label)
2024-11-01T16:13:31.5347728Z 
2024-11-01T16:13:31.5347931Z     if len(prefixes) > 1:
2024-11-01T16:13:31.5348387Z         log.error(
2024-11-01T16:13:31.5349699Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2024-11-01T16:13:31.5350997Z         )
2024-11-01T16:13:31.5351415Z         prefixes = prefixes[:1]
2024-11-01T16:13:31.5351758Z 
2024-11-01T16:13:31.5351918Z     # Fleet always comes first
2024-11-01T16:13:31.5352403Z     if fleet_prefix:
2024-11-01T16:13:31.5352880Z         prefixes.insert(0, fleet_prefix)
2024-11-01T16:13:31.5353272Z 
2024-11-01T16:13:31.5353510Z     return ".".join(prefixes) + "." if prefixes else ""
2024-11-01T16:13:31.5353996Z 
2024-11-01T16:13:31.5354002Z 
2024-11-01T16:13:31.5354512Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2024-11-01T16:13:31.5355387Z     """
2024-11-01T16:13:31.5355966Z     Gets the first comment of the issue, which contains the desired rollout state.
2024-11-01T16:13:31.5356629Z 
2024-11-01T16:13:31.5357103Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2024-11-01T16:13:31.5357934Z     """
2024-11-01T16:13:31.5358430Z     gh = get_gh_client(github_token)
2024-11-01T16:13:31.5358997Z     issue = get_issue(gh, repo, issue_num)
2024-11-01T16:13:31.5359732Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2024-11-01T16:13:31.5360211Z 
2024-11-01T16:13:31.5360216Z 
2024-11-01T16:13:31.5360393Z def main() -> None:
2024-11-01T16:13:31.5360803Z     args = parse_args()
2024-11-01T16:13:31.5361055Z 
2024-11-01T16:13:31.5361370Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2024-11-01T16:13:31.5361759Z 
2024-11-01T16:13:31.5361920Z     try:
2024-11-01T16:13:31.5362312Z         rollout_state = get_rollout_state_from_issue(
2024-11-01T16:13:31.5363150Z             args.github_token, args.github_issue_repo, args.github_issue
2024-11-01T16:13:31.5363821Z         )
2024-11-01T16:13:31.5364003Z 
2024-11-01T16:13:31.5364184Z         username = get_potential_pr_author(
2024-11-01T16:13:31.5364821Z             args.github_token,
2024-11-01T16:13:31.5365288Z             args.github_repo,
2024-11-01T16:13:31.5366150Z             args.github_actor,
2024-11-01T16:13:31.5366732Z             args.github_ref_type,
2024-11-01T16:13:31.5367222Z             args.github_branch,
2024-11-01T16:13:31.5367646Z         )
2024-11-01T16:13:31.5367942Z 
2024-11-01T16:13:31.5368295Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2024-11-01T16:13:31.5368780Z 
2024-11-01T16:13:31.5369015Z         runner_label_prefix = get_runner_prefix(
2024-11-01T16:13:31.5369541Z             rollout_state,
2024-11-01T16:13:31.5370110Z             (args.github_issue_owner, username),
2024-11-01T16:13:31.5370674Z             args.github_branch,
2024-11-01T16:13:31.5371153Z             args.eligible_experiments,
2024-11-01T16:13:31.5371720Z             is_canary,
2024-11-01T16:13:31.5372120Z         )
2024-11-01T16:13:31.5372300Z 
2024-11-01T16:13:31.5372474Z     except Exception as e:
2024-11-01T16:13:31.5372986Z         log.error(
2024-11-01T16:13:31.5373714Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2024-11-01T16:13:31.5374518Z         )
2024-11-01T16:13:31.5374800Z 
2024-11-01T16:13:31.5375122Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2024-11-01T16:13:31.5375817Z 
2024-11-01T16:13:31.5375860Z 
2024-11-01T16:13:31.5376025Z if __name__ == "__main__":
2024-11-01T16:13:31.5376448Z     main()
2024-11-01T16:13:31.5376708Z 
2024-11-01T16:13:31.5550513Z ##[group]Run python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2024-11-01T16:13:31.5551418Z python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2024-11-01T16:13:31.5610167Z shell: /usr/bin/bash -e {0}
2024-11-01T16:13:31.5610706Z env:
2024-11-01T16:13:31.5611363Z   GITHUB_TOKEN: ***
2024-11-01T16:13:31.5611791Z   ISSUE_NUMBER: 5132
2024-11-01T16:13:31.5612283Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2024-11-01T16:13:31.5612831Z   ISSUE_OWNER: 
2024-11-01T16:13:31.5613230Z   CHECK_EXPERIMENTS: 
2024-11-01T16:13:31.5613677Z ##[endgroup]
2024-11-01T16:13:33.9906113Z Defaulting to user installation because normal site-packages is not writeable
2024-11-01T16:13:35.5332338Z Collecting urllib3==1.26.18
2024-11-01T16:13:35.5851473Z   Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
2024-11-01T16:13:35.6094900Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.8/143.8 KB 6.1 MB/s eta 0:00:00
2024-11-01T16:13:35.6445231Z Collecting PyGithub==2.3.0
2024-11-01T16:13:35.6526680Z   Downloading PyGithub-2.3.0-py3-none-any.whl (354 kB)
2024-11-01T16:13:35.6823131Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 354.4/354.4 KB 12.4 MB/s eta 0:00:00
2024-11-01T16:13:35.7257012Z Collecting Deprecated
2024-11-01T16:13:35.7329797Z   Downloading Deprecated-1.2.14-py2.py3-none-any.whl (9.6 kB)
2024-11-01T16:13:35.7716152Z Collecting pynacl>=1.4.0
2024-11-01T16:13:35.7789996Z   Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
2024-11-01T16:13:35.8358316Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 856.7/856.7 KB 15.4 MB/s eta 0:00:00
2024-11-01T16:13:35.8667875Z Collecting typing-extensions>=4.0.0
2024-11-01T16:13:35.8740206Z   Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB)
2024-11-01T16:13:35.9039072Z Collecting pyjwt[crypto]>=2.4.0
2024-11-01T16:13:35.9109968Z   Downloading PyJWT-2.9.0-py3-none-any.whl (22 kB)
2024-11-01T16:13:35.9182886Z Requirement already satisfied: requests>=2.14.0 in /usr/lib/python3/dist-packages (from PyGithub==2.3.0) (2.25.1)
2024-11-01T16:13:35.9289109Z Requirement already satisfied: cryptography>=3.4.0 in /usr/lib/python3/dist-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.3.0) (3.4.8)
2024-11-01T16:13:36.1533921Z Collecting cffi>=1.4.1
2024-11-01T16:13:36.1636093Z   Downloading cffi-1.17.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (446 kB)
2024-11-01T16:13:36.1901695Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 446.2/446.2 KB 17.9 MB/s eta 0:00:00
2024-11-01T16:13:36.3957622Z Collecting wrapt<2,>=1.10
2024-11-01T16:13:36.4035719Z   Downloading wrapt-1.16.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (80 kB)
2024-11-01T16:13:36.4078207Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 80.3/80.3 KB 29.9 MB/s eta 0:00:00
2024-11-01T16:13:36.4275670Z Collecting pycparser
2024-11-01T16:13:36.4408840Z   Downloading pycparser-2.22-py3-none-any.whl (117 kB)
2024-11-01T16:13:36.4459697Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 KB 35.2 MB/s eta 0:00:00
2024-11-01T16:13:36.8805442Z Installing collected packages: wrapt, urllib3, typing-extensions, pyjwt, pycparser, Deprecated, cffi, pynacl, PyGithub
2024-11-01T16:13:37.3387976Z Successfully installed Deprecated-1.2.14 PyGithub-2.3.0 cffi-1.17.1 pycparser-2.22 pyjwt-2.9.0 pynacl-1.5.0 typing-extensions-4.12.2 urllib3-1.26.18 wrapt-1.16.0
2024-11-01T16:13:37.4158366Z ##[group]Run curr_branch="ciflow/trunk/138766"
2024-11-01T16:13:37.4159069Z curr_branch="ciflow/trunk/138766"
2024-11-01T16:13:37.4159483Z curr_ref_type="tag"
2024-11-01T16:13:37.4159930Z echo "Current branch is '$curr_branch'"
2024-11-01T16:13:37.4160460Z 
2024-11-01T16:13:37.4160769Z python3 runner_determinator.py \
2024-11-01T16:13:37.4161491Z   --github-token "$GITHUB_TOKEN" \
2024-11-01T16:13:37.4162043Z   --github-issue "$ISSUE_NUMBER" \
2024-11-01T16:13:37.4162487Z   --github-branch "$curr_branch" \
2024-11-01T16:13:37.4162956Z   --github-actor "$TRIGGERING_ACTOR" \
2024-11-01T16:13:37.4163513Z   --github-issue-owner "$ISSUE_OWNER" \
2024-11-01T16:13:37.4164018Z   --github-ref-type "$curr_ref_type" \
2024-11-01T16:13:37.4164463Z   --github-repo "$GITHUB_REPOSITORY" \
2024-11-01T16:13:37.4165160Z   --eligible-experiments "$CHECK_EXPERIMENTS" \
2024-11-01T16:13:37.4223762Z shell: /usr/bin/bash -e {0}
2024-11-01T16:13:37.4224237Z env:
2024-11-01T16:13:37.4224998Z   GITHUB_TOKEN: ***
2024-11-01T16:13:37.4225380Z   ISSUE_NUMBER: 5132
2024-11-01T16:13:37.4225838Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2024-11-01T16:13:37.4226213Z   ISSUE_OWNER: 
2024-11-01T16:13:37.4226556Z   CHECK_EXPERIMENTS: 
2024-11-01T16:13:37.4226962Z ##[endgroup]
2024-11-01T16:13:37.4305312Z Current branch is 'ciflow/trunk/138766'
2024-11-01T16:13:39.3574163Z INFO    : Skipping experiment 'awsa100', as it is not a default experiment
2024-11-01T16:13:39.3575794Z INFO    : Setting output: label-type=''
2024-11-01T16:13:39.3946972Z Evaluate and set job outputs
2024-11-01T16:13:39.3957701Z Cleaning up orphan processes