2024-11-01T16:13:31.0700403Z Current runner version: '2.320.0'
2024-11-01T16:13:31.0723838Z ##[group]Operating System
2024-11-01T16:13:31.0724831Z Ubuntu
2024-11-01T16:13:31.0725192Z 22.04.5
2024-11-01T16:13:31.0725630Z LTS
2024-11-01T16:13:31.0726005Z ##[endgroup]
2024-11-01T16:13:31.0726385Z ##[group]Runner Image
2024-11-01T16:13:31.0726878Z Image: ubuntu-22.04
2024-11-01T16:13:31.0727293Z Version: 20241015.1.0
2024-11-01T16:13:31.0728291Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20241015.1/images/ubuntu/Ubuntu2204-Readme.md
2024-11-01T16:13:31.0729782Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20241015.1
2024-11-01T16:13:31.0730638Z ##[endgroup]
2024-11-01T16:13:31.0731038Z ##[group]Runner Image Provisioner
2024-11-01T16:13:31.0731568Z 2.0.384.1
2024-11-01T16:13:31.0731913Z ##[endgroup]
2024-11-01T16:13:31.0746215Z ##[group]GITHUB_TOKEN Permissions
2024-11-01T16:13:31.0747814Z Contents: read
2024-11-01T16:13:31.0748347Z Metadata: read
2024-11-01T16:13:31.0748895Z Packages: read
2024-11-01T16:13:31.0749431Z ##[endgroup]
2024-11-01T16:13:31.0752343Z Secret source: Actions
2024-11-01T16:13:31.0752945Z Prepare workflow directory
2024-11-01T16:13:31.1651789Z Prepare all required actions
2024-11-01T16:13:31.1815053Z Uses: pytorch/pytorch/.github/workflows/_runner-determinator.yml@refs/heads/main (33dce10ece5b38aa0ab76739b658cd980a6e3d8f)
2024-11-01T16:13:31.1820119Z ##[group] Inputs
2024-11-01T16:13:31.1820630Z   check_experiments: 
2024-11-01T16:13:31.1821228Z   triggering_actor: pytorch-bot[bot]
2024-11-01T16:13:31.1821734Z   issue_owner: 
2024-11-01T16:13:31.1822169Z   curr_branch: ciflow/trunk/138766
2024-11-01T16:13:31.1822802Z   curr_ref_type: tag
2024-11-01T16:13:31.1823208Z   issue_number: 5132
2024-11-01T16:13:31.1823611Z ##[endgroup]
2024-11-01T16:13:31.1824514Z Complete job name: get-label-type / runner-determinator
2024-11-01T16:13:31.2698637Z ##[group]Run cat <<EOF > runner_determinator.py
2024-11-01T16:13:31.2700378Z cat <<EOF > runner_determinator.py
2024-11-01T16:13:31.2701098Z # flake8: noqa: G004
2024-11-01T16:13:31.2701567Z 
2024-11-01T16:13:31.2702308Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2024-11-01T16:13:31.2703625Z #       must be kept in sync. You can do it easily by running the following command:
2024-11-01T16:13:31.2704646Z #           python .github/scripts/update_runner_determinator.py
2024-11-01T16:13:31.2705334Z 
2024-11-01T16:13:31.2705747Z """
2024-11-01T16:13:31.2706434Z This runner determinator is used to determine which set of runners to run a
2024-11-01T16:13:31.2707516Z GitHub job on. It uses the first comment of a GitHub issue (by default
2024-11-01T16:13:31.2708791Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2024-11-01T16:13:31.2709767Z of which runners should be used to run which job.
2024-11-01T16:13:31.2710499Z 
2024-11-01T16:13:31.2711144Z The configuration has two parts, the settings and a list of opted-in users,
2024-11-01T16:13:31.2712248Z separated by a line containing "---".  If the line is not present, the
2024-11-01T16:13:31.2713393Z settings are considered to be empty with only the second part, the user
2024-11-01T16:13:31.2714161Z list, defined.
2024-11-01T16:13:31.2714602Z 
2024-11-01T16:13:31.2715330Z The first part is a YAML block that defines the rollout settings. This can be
2024-11-01T16:13:31.2716498Z used to define any settings that are needed to determine which runners to use.
2024-11-01T16:13:31.2717518Z It's fields are defined by the RolloutSettings class below.
2024-11-01T16:13:31.2718277Z 
2024-11-01T16:13:31.2718966Z The second part is a list of users who are explicitly opted in to the LF fleet.
2024-11-01T16:13:31.2720039Z The user list is also a comma separated list of additional features or
2024-11-01T16:13:31.2721695Z experiments which the user could be opted in to.
2024-11-01T16:13:31.2722347Z 
2024-11-01T16:13:31.2722740Z The user list has the following rules:
2024-11-01T16:13:31.2723367Z 
2024-11-01T16:13:31.2723963Z - Users are GitHub usernames, which must start with the @ prefix
2024-11-01T16:13:31.2725443Z - Each user is also a comma-separated list of features/experiments to enable
2024-11-01T16:13:31.2726479Z - A "#" prefix opts the user out of all experiments
2024-11-01T16:13:31.2727139Z 
2024-11-01T16:13:31.2727502Z Example config:
2024-11-01T16:13:31.2728091Z     # A list of experiments that can be opted into.
2024-11-01T16:13:31.2728939Z     # This defines the behavior they'll induce when opted into.
2024-11-01T16:13:31.2729669Z     # Expected syntax is:
2024-11-01T16:13:31.2730513Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2024-11-01T16:13:31.2731757Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2024-11-01T16:13:31.2732660Z 
2024-11-01T16:13:31.2733085Z     experiments:
2024-11-01T16:13:31.2733528Z       lf:
2024-11-01T16:13:31.2733955Z         rollout_percent: 25
2024-11-01T16:13:31.2734543Z         all_branches: false
2024-11-01T16:13:31.2735052Z         default: true
2024-11-01T16:13:31.2735507Z     ---
2024-11-01T16:13:31.2735920Z 
2024-11-01T16:13:31.2736280Z     # Opt-ins:
2024-11-01T16:13:31.2737032Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2024-11-01T16:13:31.2738396Z     # and specifying experiments to enable in a comma-separated list.
2024-11-01T16:13:31.2739268Z     # Experiments should be from the above list.
2024-11-01T16:13:31.2739905Z 
2024-11-01T16:13:31.2740347Z     @User1,lf,split_build
2024-11-01T16:13:31.2740812Z     @User2,lf
2024-11-01T16:13:31.2741254Z     @User3,split_build
2024-11-01T16:13:31.2741817Z """
2024-11-01T16:13:31.2742139Z 
2024-11-01T16:13:31.2742514Z import logging
2024-11-01T16:13:31.2742999Z import os
2024-11-01T16:13:31.2743369Z import random
2024-11-01T16:13:31.2743870Z from argparse import ArgumentParser
2024-11-01T16:13:31.2744538Z from logging import LogRecord
2024-11-01T16:13:31.2745329Z from typing import Any, Dict, FrozenSet, Iterable, List, NamedTuple, Tuple
2024-11-01T16:13:31.2746130Z 
2024-11-01T16:13:31.2746549Z import yaml
2024-11-01T16:13:31.2747089Z from github import Auth, Github
2024-11-01T16:13:31.2747661Z from github.Issue import Issue
2024-11-01T16:13:31.2748259Z 
2024-11-01T16:13:31.2748621Z 
2024-11-01T16:13:31.2749048Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2024-11-01T16:13:31.2749951Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2024-11-01T16:13:31.2751028Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2024-11-01T16:13:31.2751813Z 
2024-11-01T16:13:31.2752358Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2024-11-01T16:13:31.2753010Z GH_OUTPUT_KEY_AMI = "runner-ami"
2024-11-01T16:13:31.2753684Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2024-11-01T16:13:31.2754233Z 
2024-11-01T16:13:31.2754578Z 
2024-11-01T16:13:31.2755071Z SETTING_EXPERIMENTS = "experiments"
2024-11-01T16:13:31.2755608Z 
2024-11-01T16:13:31.2755981Z LF_FLEET_EXPERIMENT = "lf"
2024-11-01T16:13:31.2756578Z CANARY_FLEET_SUFFIX = ".c"
2024-11-01T16:13:31.2757047Z 
2024-11-01T16:13:31.2757375Z 
2024-11-01T16:13:31.2758036Z class Experiment(NamedTuple):
2024-11-01T16:13:31.2758561Z     rollout_perc: float = (
2024-11-01T16:13:31.2759367Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2024-11-01T16:13:31.2760258Z     )
2024-11-01T16:13:31.2760629Z     all_branches: bool = (
2024-11-01T16:13:31.2761442Z         False  # If True, the experiment is also enabled on the exception branches
2024-11-01T16:13:31.2762345Z     )
2024-11-01T16:13:31.2762703Z     default: bool = (
2024-11-01T16:13:31.2763428Z         True  # If True, the experiment is enabled by default for all queries
2024-11-01T16:13:31.2764606Z     )
2024-11-01T16:13:31.2764968Z 
2024-11-01T16:13:31.2765376Z     # Add more fields as needed
2024-11-01T16:13:31.2765976Z 
2024-11-01T16:13:31.2766280Z 
2024-11-01T16:13:31.2766672Z class Settings(NamedTuple):
2024-11-01T16:13:31.2767249Z     """
2024-11-01T16:13:31.2767772Z     Settings for the experiments that can be opted into.
2024-11-01T16:13:31.2768469Z     """
2024-11-01T16:13:31.2768905Z 
2024-11-01T16:13:31.2769313Z     experiments: Dict[str, Experiment] = {}
2024-11-01T16:13:31.2769916Z 
2024-11-01T16:13:31.2770314Z 
2024-11-01T16:13:31.2770718Z class ColorFormatter(logging.Formatter):
2024-11-01T16:13:31.2771505Z     """Color codes the log messages based on the log level"""
2024-11-01T16:13:31.2772247Z 
2024-11-01T16:13:31.2772565Z     COLORS = {
2024-11-01T16:13:31.2773065Z         "WARNING": "\033[33m",  # Yellow
2024-11-01T16:13:31.2773727Z         "ERROR": "\033[31m",  # Red
2024-11-01T16:13:31.2774568Z         "CRITICAL": "\033[31m",  # Red
2024-11-01T16:13:31.2775136Z         "INFO": "\033[0m",  # Reset
2024-11-01T16:13:31.2775783Z         "DEBUG": "\033[0m",  # Reset
2024-11-01T16:13:31.2776345Z     }
2024-11-01T16:13:31.2776670Z 
2024-11-01T16:13:31.2777209Z     def format(self, record: LogRecord) -> str:
2024-11-01T16:13:31.2778144Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2024-11-01T16:13:31.2779073Z         record.msg = f"{log_color}{record.msg}\033[0m"
2024-11-01T16:13:31.2779838Z         return super().format(record)
2024-11-01T16:13:31.2780387Z 
2024-11-01T16:13:31.2780690Z 
2024-11-01T16:13:31.2781176Z handler = logging.StreamHandler()
2024-11-01T16:13:31.2782042Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2024-11-01T16:13:31.2782796Z 
2024-11-01T16:13:31.2783395Z log = logging.getLogger(os.path.basename(__file__))
2024-11-01T16:13:31.2784085Z log.addHandler(handler)
2024-11-01T16:13:31.2784568Z log.setLevel(logging.INFO)
2024-11-01T16:13:31.2785139Z 
2024-11-01T16:13:31.2785483Z 
2024-11-01T16:13:31.2785947Z def set_github_output(key: str, value: str) -> None:
2024-11-01T16:13:31.2786666Z     """
2024-11-01T16:13:31.2787271Z     Defines outputs of the github action that invokes this script
2024-11-01T16:13:31.2788013Z     """
2024-11-01T16:13:31.2788440Z     if not GITHUB_OUTPUT:
2024-11-01T16:13:31.2789803Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2024-11-01T16:13:31.2791183Z         log.warning(
2024-11-01T16:13:31.2792302Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2024-11-01T16:13:31.2793415Z         )
2024-11-01T16:13:31.2793939Z         print(f"::set-output name={key}::{value}")
2024-11-01T16:13:31.2794748Z         return
2024-11-01T16:13:31.2795177Z 
2024-11-01T16:13:31.2795599Z     with open(GITHUB_OUTPUT, "a") as f:
2024-11-01T16:13:31.2796321Z         log.info(f"Setting output: {key}='{value}'")
2024-11-01T16:13:31.2797002Z         f.write(f"{key}={value}\n")
2024-11-01T16:13:31.2797524Z 
2024-11-01T16:13:31.2797937Z 
2024-11-01T16:13:31.2798493Z def _str_comma_separated_to_set(value: str) -> FrozenSet[str]:
2024-11-01T16:13:31.2799230Z     return frozenset(
2024-11-01T16:13:31.2800091Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2024-11-01T16:13:31.2800859Z     )
2024-11-01T16:13:31.2801223Z 
2024-11-01T16:13:31.2801646Z 
2024-11-01T16:13:31.2801989Z def parse_args() -> Any:
2024-11-01T16:13:31.2802671Z     parser = ArgumentParser("Get dynamic rollout settings")
2024-11-01T16:13:31.2803838Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2024-11-01T16:13:31.2805073Z     parser.add_argument(
2024-11-01T16:13:31.2805607Z         "--github-issue-repo",
2024-11-01T16:13:31.2806246Z         type=str,
2024-11-01T16:13:31.2806666Z         required=False,
2024-11-01T16:13:31.2807209Z         default="pytorch/test-infra",
2024-11-01T16:13:31.2807933Z         help="GitHub repo to get the issue",
2024-11-01T16:13:31.2808478Z     )
2024-11-01T16:13:31.2808879Z     parser.add_argument(
2024-11-01T16:13:31.2809467Z         "--github-repo",
2024-11-01T16:13:31.2809919Z         type=str,
2024-11-01T16:13:31.2810384Z         required=True,
2024-11-01T16:13:31.2811186Z         help="GitHub repo where CI is running",
2024-11-01T16:13:31.2811797Z     )
2024-11-01T16:13:31.2812173Z     parser.add_argument(
2024-11-01T16:13:31.2813010Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2024-11-01T16:13:31.2813781Z     )
2024-11-01T16:13:31.2814134Z     parser.add_argument(
2024-11-01T16:13:31.2814969Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2024-11-01T16:13:31.2815770Z     )
2024-11-01T16:13:31.2816131Z     parser.add_argument(
2024-11-01T16:13:31.2816988Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2024-11-01T16:13:31.2817806Z     )
2024-11-01T16:13:31.2818163Z     parser.add_argument(
2024-11-01T16:13:31.2819045Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2024-11-01T16:13:31.2819892Z     )
2024-11-01T16:13:31.2820265Z     parser.add_argument(
2024-11-01T16:13:31.2820838Z         "--github-ref-type",
2024-11-01T16:13:31.2821371Z         type=str,
2024-11-01T16:13:31.2821794Z         required=True,
2024-11-01T16:13:31.2822464Z         help="Current GitHub ref type, branch or tag",
2024-11-01T16:13:31.2823109Z     )
2024-11-01T16:13:31.2823495Z     parser.add_argument(
2024-11-01T16:13:31.2824074Z         "--eligible-experiments",
2024-11-01T16:13:31.2824700Z         type=_str_comma_separated_to_set,
2024-11-01T16:13:31.2825280Z         required=False,
2024-11-01T16:13:31.2825819Z         default="",
2024-11-01T16:13:31.2826909Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2024-11-01T16:13:31.2827987Z     )
2024-11-01T16:13:31.2828443Z 
2024-11-01T16:13:31.2828850Z     return parser.parse_args()
2024-11-01T16:13:31.2829337Z 
2024-11-01T16:13:31.2829756Z 
2024-11-01T16:13:31.2830226Z def get_gh_client(github_token: str) -> Github:
2024-11-01T16:13:31.2831036Z     auth = Auth.Token(github_token)
2024-11-01T16:13:31.2831676Z     return Github(auth=auth)
2024-11-01T16:13:31.2832171Z 
2024-11-01T16:13:31.2832493Z 
2024-11-01T16:13:31.2833109Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:
2024-11-01T16:13:31.2833891Z     repo = gh.get_repo(repo)
2024-11-01T16:13:31.2834454Z     return repo.get_issue(number=issue_num)
2024-11-01T16:13:31.2835090Z 
2024-11-01T16:13:31.2835442Z 
2024-11-01T16:13:31.2835782Z def get_potential_pr_author(
2024-11-01T16:13:31.2836632Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2024-11-01T16:13:31.2837433Z ) -> str:
2024-11-01T16:13:31.2838052Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2024-11-01T16:13:31.2839120Z     # Fetch the actual username from the original PR. The PR number is
2024-11-01T16:13:31.2840059Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2024-11-01T16:13:31.2840783Z 
2024-11-01T16:13:31.2841156Z     gh = get_gh_client(github_token)
2024-11-01T16:13:31.2841726Z 
2024-11-01T16:13:31.2842321Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2024-11-01T16:13:31.2843035Z         split_tag = ref_name.split("/")
2024-11-01T16:13:31.2843621Z         if (
2024-11-01T16:13:31.2844437Z             len(split_tag) == 3
2024-11-01T16:13:31.2845071Z             and split_tag[0] == "ciflow"
2024-11-01T16:13:31.2845720Z             and split_tag[2].isnumeric()
2024-11-01T16:13:31.2846360Z         ):
2024-11-01T16:13:31.2846968Z             pr_number = split_tag[2]
2024-11-01T16:13:31.2847531Z             try:
2024-11-01T16:13:31.2848139Z                 repository = gh.get_repo(repo)
2024-11-01T16:13:31.2848866Z                 pull = repository.get_pull(number=int(pr_number))
2024-11-01T16:13:31.2849589Z             except Exception as e:
2024-11-01T16:13:31.2850316Z                 raise Exception(  # noqa: TRY002
2024-11-01T16:13:31.2851147Z                     f"issue with pull request {pr_number} from repo {repository}"
2024-11-01T16:13:31.2851872Z                 ) from e
2024-11-01T16:13:31.2852475Z             return pull.user.login
2024-11-01T16:13:31.2853201Z     # In all other cases, return the original input username
2024-11-01T16:13:31.2853861Z     return username
2024-11-01T16:13:31.2854381Z 
2024-11-01T16:13:31.2854726Z 
2024-11-01T16:13:31.2855168Z def is_exception_branch(branch: str) -> bool:
2024-11-01T16:13:31.2855941Z     """
2024-11-01T16:13:31.2856740Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2024-11-01T16:13:31.2857633Z     """
2024-11-01T16:13:31.2858361Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2024-11-01T16:13:31.2859134Z 
2024-11-01T16:13:31.2859459Z 
2024-11-01T16:13:31.2859943Z def load_yaml(yaml_text: str) -> Any:
2024-11-01T16:13:31.2860500Z     try:
2024-11-01T16:13:31.2860942Z         data = yaml.safe_load(yaml_text)
2024-11-01T16:13:31.2861578Z         return data
2024-11-01T16:13:31.2862086Z     except yaml.YAMLError as exc:
2024-11-01T16:13:31.2862734Z         log.exception("Error loading YAML")
2024-11-01T16:13:31.2863358Z         raise
2024-11-01T16:13:31.2863807Z 
2024-11-01T16:13:31.2864146Z 
2024-11-01T16:13:31.2864873Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> Tuple[str, str]:
2024-11-01T16:13:31.2865736Z     """
2024-11-01T16:13:31.2866660Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2024-11-01T16:13:31.2867583Z 
2024-11-01T16:13:31.2868230Z     If the issue body contains "---" then the text above that is the settings
2024-11-01T16:13:31.2869168Z     and the text below is the list of opted in users.
2024-11-01T16:13:31.2869834Z 
2024-11-01T16:13:31.2870529Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2024-11-01T16:13:31.2871352Z     """
2024-11-01T16:13:31.2871936Z     rollout_state_parts = rollout_state.split("---")
2024-11-01T16:13:31.2872638Z     if len(rollout_state_parts) >= 2:
2024-11-01T16:13:31.2873358Z         return rollout_state_parts[0], rollout_state_parts[1]
2024-11-01T16:13:31.2874112Z     else:
2024-11-01T16:13:31.2874505Z         return "", rollout_state
2024-11-01T16:13:31.2875024Z 
2024-11-01T16:13:31.2875438Z 
2024-11-01T16:13:31.2875829Z class UserOptins(Dict[str, List[str]]):
2024-11-01T16:13:31.2876398Z     """
2024-11-01T16:13:31.2877091Z     Dictionary of users with a list of features they have opted into
2024-11-01T16:13:31.2877813Z     """
2024-11-01T16:13:31.2878194Z 
2024-11-01T16:13:31.2878598Z 
2024-11-01T16:13:31.2879155Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2024-11-01T16:13:31.2879922Z     """
2024-11-01T16:13:31.2880878Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2024-11-01T16:13:31.2881853Z 
2024-11-01T16:13:31.2882965Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2024-11-01T16:13:31.2884615Z         - Example line: "@User1,lf,split_build"
2024-11-01T16:13:31.2885495Z         - A "#" prefix indicates the user is opted out of all experiments
2024-11-01T16:13:31.2886196Z 
2024-11-01T16:13:31.2886616Z 
2024-11-01T16:13:31.2886976Z     """
2024-11-01T16:13:31.2887365Z     optins = UserOptins()
2024-11-01T16:13:31.2888036Z     for user in user_optin_text.split("\n"):
2024-11-01T16:13:31.2888701Z         user = user.strip("\r\n\t -")
2024-11-01T16:13:31.2889330Z         if not user or not user.startswith("@"):
2024-11-01T16:13:31.2890052Z             # Not a valid user. Skip
2024-11-01T16:13:31.2890618Z             continue
2024-11-01T16:13:31.2891031Z 
2024-11-01T16:13:31.2891450Z         if user:
2024-11-01T16:13:31.2892005Z             usr_name = user.split(",")[0].strip("@")
2024-11-01T16:13:31.2892826Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2024-11-01T16:13:31.2893652Z 
2024-11-01T16:13:31.2894017Z     return optins
2024-11-01T16:13:31.2894421Z 
2024-11-01T16:13:31.2894845Z 
2024-11-01T16:13:31.2895402Z def parse_settings_from_text(settings_text: str) -> Settings:
2024-11-01T16:13:31.2896079Z     """
2024-11-01T16:13:31.2896862Z     Parse the experiments from the issue body into a list of ExperimentSettings
2024-11-01T16:13:31.2897701Z     """
2024-11-01T16:13:31.2898040Z     try:
2024-11-01T16:13:31.2898517Z         if settings_text:
2024-11-01T16:13:31.2899442Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2024-11-01T16:13:31.2900427Z             # for easy reading
2024-11-01T16:13:31.2901458Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2024-11-01T16:13:31.2902587Z             #       the backtick character in shell commands.
2024-11-01T16:13:31.2903565Z             backtick = chr(96)  # backtick character
2024-11-01T16:13:31.2904419Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2024-11-01T16:13:31.2905207Z             settings = load_yaml(settings_text)
2024-11-01T16:13:31.2905804Z 
2024-11-01T16:13:31.2906595Z             # For now we just load experiments. We can expand this if/when we add more settings
2024-11-01T16:13:31.2907453Z             experiments = {}
2024-11-01T16:13:31.2907960Z 
2024-11-01T16:13:31.2908668Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2024-11-01T16:13:31.2909505Z                 valid_settings = {}
2024-11-01T16:13:31.2910119Z                 for setting in exp_settings:
2024-11-01T16:13:31.2910867Z                     if setting not in Experiment._fields:
2024-11-01T16:13:31.2911525Z                         log.warning(
2024-11-01T16:13:31.2912373Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2024-11-01T16:13:31.2913256Z                         )
2024-11-01T16:13:31.2913757Z                     else:
2024-11-01T16:13:31.2914353Z                         valid_settings[setting] = exp_settings[setting]
2024-11-01T16:13:31.2915062Z 
2024-11-01T16:13:31.2915598Z                 experiments[exp_name] = Experiment(**valid_settings)
2024-11-01T16:13:31.2916318Z             return Settings(experiments)
2024-11-01T16:13:31.2916962Z 
2024-11-01T16:13:31.2917336Z     except Exception:
2024-11-01T16:13:31.2918033Z         log.exception("Failed to parse settings")
2024-11-01T16:13:31.2918743Z 
2024-11-01T16:13:31.2919116Z     return Settings()
2024-11-01T16:13:31.2919535Z 
2024-11-01T16:13:31.2919958Z 
2024-11-01T16:13:31.2920451Z def parse_settings(rollout_state: str) -> Settings:
2024-11-01T16:13:31.2921056Z     """
2024-11-01T16:13:31.2921657Z     Parse settings, if any, from the rollout state.
2024-11-01T16:13:31.2922281Z 
2024-11-01T16:13:31.2922885Z     If the issue body contains "---" then the text above that is the settings
2024-11-01T16:13:31.2923884Z     and the text below is the list of opted in users.
2024-11-01T16:13:31.2924795Z 
2024-11-01T16:13:31.2925626Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2024-11-01T16:13:31.2926497Z     """
2024-11-01T16:13:31.2927140Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2024-11-01T16:13:31.2928139Z     return parse_settings_from_text(settings_text)
2024-11-01T16:13:31.2928725Z 
2024-11-01T16:13:31.2929083Z 
2024-11-01T16:13:31.2929652Z def parse_users(rollout_state: str) -> UserOptins:
2024-11-01T16:13:31.2930248Z     """
2024-11-01T16:13:31.2930713Z     Parse users from the rollout state.
2024-11-01T16:13:31.2931331Z 
2024-11-01T16:13:31.2931638Z     """
2024-11-01T16:13:31.2932274Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2024-11-01T16:13:31.2933232Z     return parse_user_opt_in_from_text(users_text)
2024-11-01T16:13:31.2933808Z 
2024-11-01T16:13:31.2934161Z 
2024-11-01T16:13:31.2934940Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2024-11-01T16:13:31.2935774Z     """
2024-11-01T16:13:31.2936285Z     Check if a user is opted into an experiment
2024-11-01T16:13:31.2936963Z     """
2024-11-01T16:13:31.2937511Z     return experiment_name in user_optins.get(user, [])
2024-11-01T16:13:31.2938289Z 
2024-11-01T16:13:31.2938699Z 
2024-11-01T16:13:31.2939056Z def get_runner_prefix(
2024-11-01T16:13:31.2939548Z     rollout_state: str,
2024-11-01T16:13:31.2940177Z     workflow_requestors: Iterable[str],
2024-11-01T16:13:31.2940779Z     branch: str,
2024-11-01T16:13:31.2941337Z     eligible_experiments: FrozenSet[str] = frozenset(),
2024-11-01T16:13:31.2942079Z     is_canary: bool = False,
2024-11-01T16:13:31.2942601Z ) -> str:
2024-11-01T16:13:31.2943052Z     settings = parse_settings(rollout_state)
2024-11-01T16:13:31.2943811Z     user_optins = parse_users(rollout_state)
2024-11-01T16:13:31.2944391Z 
2024-11-01T16:13:31.2944718Z     fleet_prefix = ""
2024-11-01T16:13:31.2945283Z     prefixes = []
2024-11-01T16:13:31.2946039Z     for experiment_name, experiment_settings in settings.experiments.items():
2024-11-01T16:13:31.2947126Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2024-11-01T16:13:31.2948043Z             log.info(
2024-11-01T16:13:31.2948898Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2024-11-01T16:13:31.2949801Z             )
2024-11-01T16:13:31.2950269Z             continue
2024-11-01T16:13:31.2950717Z 
2024-11-01T16:13:31.2951119Z         if eligible_experiments:
2024-11-01T16:13:31.2951835Z             if experiment_name not in eligible_experiments:
2024-11-01T16:13:31.2952615Z                 exp_list = ", ".join(eligible_experiments)
2024-11-01T16:13:31.2953270Z                 log.info(
2024-11-01T16:13:31.2954383Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2024-11-01T16:13:31.2955363Z                 )
2024-11-01T16:13:31.2955838Z                 continue
2024-11-01T16:13:31.2956460Z         elif not experiment_settings.default:
2024-11-01T16:13:31.2957067Z             log.info(
2024-11-01T16:13:31.2957870Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2024-11-01T16:13:31.2958769Z             )
2024-11-01T16:13:31.2959178Z             continue
2024-11-01T16:13:31.2959623Z 
2024-11-01T16:13:31.2960217Z         # Is any workflow_requestor opted in to this experiment?
2024-11-01T16:13:31.2960918Z         opted_in_users = [
2024-11-01T16:13:31.2961417Z             requestor
2024-11-01T16:13:31.2962036Z             for requestor in workflow_requestors
2024-11-01T16:13:31.2962835Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2024-11-01T16:13:31.2963562Z         ]
2024-11-01T16:13:31.2964656Z 
2024-11-01T16:13:31.2965153Z         enabled = False
2024-11-01T16:13:31.2965674Z         if opted_in_users:
2024-11-01T16:13:31.2966292Z             log.info(
2024-11-01T16:13:31.2967025Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2024-11-01T16:13:31.2967830Z             )
2024-11-01T16:13:31.2968349Z             enabled = True
2024-11-01T16:13:31.2968799Z 
2024-11-01T16:13:31.2969268Z         elif experiment_settings.rollout_perc:
2024-11-01T16:13:31.2970395Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2024-11-01T16:13:31.2971589Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2024-11-01T16:13:31.2972312Z                 log.info(
2024-11-01T16:13:31.2973432Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2024-11-01T16:13:31.2974681Z                 )
2024-11-01T16:13:31.2975121Z                 enabled = True
2024-11-01T16:13:31.2975697Z 
2024-11-01T16:13:31.2976058Z         if enabled:
2024-11-01T16:13:31.2976547Z             label = experiment_name
2024-11-01T16:13:31.2977276Z             if experiment_name == LF_FLEET_EXPERIMENT:
2024-11-01T16:13:31.2978287Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2024-11-01T16:13:31.2979392Z                 #  - If it's enabled, then we always list it's prefix first
2024-11-01T16:13:31.2980398Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2024-11-01T16:13:31.2981223Z                 if is_canary:
2024-11-01T16:13:31.2981837Z                     label += CANARY_FLEET_SUFFIX
2024-11-01T16:13:31.2982509Z                 fleet_prefix = label
2024-11-01T16:13:31.2983071Z             else:
2024-11-01T16:13:31.2983575Z                 prefixes.append(label)
2024-11-01T16:13:31.2984149Z 
2024-11-01T16:13:31.2984539Z     if len(prefixes) > 1:
2024-11-01T16:13:31.2985038Z         log.error(
2024-11-01T16:13:31.2986396Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2024-11-01T16:13:31.2987729Z         )
2024-11-01T16:13:31.2988163Z         prefixes = prefixes[:1]
2024-11-01T16:13:31.2988770Z 
2024-11-01T16:13:31.2989141Z     # Fleet always comes first
2024-11-01T16:13:31.2989677Z     if fleet_prefix:
2024-11-01T16:13:31.2990445Z         prefixes.insert(0, fleet_prefix)
2024-11-01T16:13:31.2990989Z 
2024-11-01T16:13:31.2991479Z     return ".".join(prefixes) + "." if prefixes else ""
2024-11-01T16:13:31.2992217Z 
2024-11-01T16:13:31.2992529Z 
2024-11-01T16:13:31.2993231Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2024-11-01T16:13:31.2994188Z     """
2024-11-01T16:13:31.2994852Z     Gets the first comment of the issue, which contains the desired rollout state.
2024-11-01T16:13:31.2995669Z 
2024-11-01T16:13:31.2996418Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2024-11-01T16:13:31.2997291Z     """
2024-11-01T16:13:31.2997755Z     gh = get_gh_client(github_token)
2024-11-01T16:13:31.2998399Z     issue = get_issue(gh, repo, issue_num)
2024-11-01T16:13:31.2999172Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2024-11-01T16:13:31.2999890Z 
2024-11-01T16:13:31.3000226Z 
2024-11-01T16:13:31.3000612Z def main() -> None:
2024-11-01T16:13:31.3001110Z     args = parse_args()
2024-11-01T16:13:31.3001586Z 
2024-11-01T16:13:31.3002072Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2024-11-01T16:13:31.3002706Z 
2024-11-01T16:13:31.3003054Z     try:
2024-11-01T16:13:31.3003622Z         rollout_state = get_rollout_state_from_issue(
2024-11-01T16:13:31.3005082Z             args.github_token, args.github_issue_repo, args.github_issue
2024-11-01T16:13:31.3005838Z         )
2024-11-01T16:13:31.3006242Z 
2024-11-01T16:13:31.3006705Z         username = get_potential_pr_author(
2024-11-01T16:13:31.3007321Z             args.github_token,
2024-11-01T16:13:31.3007885Z             args.github_repo,
2024-11-01T16:13:31.3008461Z             args.github_actor,
2024-11-01T16:13:31.3009025Z             args.github_ref_type,
2024-11-01T16:13:31.3009610Z             args.github_branch,
2024-11-01T16:13:31.3010340Z         )
2024-11-01T16:13:31.3010747Z 
2024-11-01T16:13:31.3011297Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2024-11-01T16:13:31.3012062Z 
2024-11-01T16:13:31.3012505Z         runner_label_prefix = get_runner_prefix(
2024-11-01T16:13:31.3013130Z             rollout_state,
2024-11-01T16:13:31.3013795Z             (args.github_issue_owner, username),
2024-11-01T16:13:31.3014402Z             args.github_branch,
2024-11-01T16:13:31.3014990Z             args.eligible_experiments,
2024-11-01T16:13:31.3015634Z             is_canary,
2024-11-01T16:13:31.3016063Z         )
2024-11-01T16:13:31.3016454Z 
2024-11-01T16:13:31.3016906Z     except Exception as e:
2024-11-01T16:13:31.3017387Z         log.error(
2024-11-01T16:13:31.3018214Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2024-11-01T16:13:31.3019178Z         )
2024-11-01T16:13:31.3019523Z 
2024-11-01T16:13:31.3020120Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2024-11-01T16:13:31.3020925Z 
2024-11-01T16:13:31.3021233Z 
2024-11-01T16:13:31.3021600Z if __name__ == "__main__":
2024-11-01T16:13:31.3022165Z     main()
2024-11-01T16:13:31.3022517Z 
2024-11-01T16:13:31.3022877Z EOF
2024-11-01T16:13:31.3023297Z 
2024-11-01T16:13:31.3023648Z cat runner_determinator.py
2024-11-01T16:13:31.3571312Z shell: /usr/bin/bash -e {0}
2024-11-01T16:13:31.3571891Z env:
2024-11-01T16:13:31.3572598Z   GITHUB_TOKEN: ***
2024-11-01T16:13:31.3573195Z   ISSUE_NUMBER: 5132
2024-11-01T16:13:31.3573905Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2024-11-01T16:13:31.3574435Z   ISSUE_OWNER: 
2024-11-01T16:13:31.3574949Z   CHECK_EXPERIMENTS: 
2024-11-01T16:13:31.3575396Z ##[endgroup]
2024-11-01T16:13:31.4000176Z # flake8: noqa: G004
2024-11-01T16:13:31.4000530Z 
2024-11-01T16:13:31.4002184Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2024-11-01T16:13:31.4003476Z #       must be kept in sync. You can do it easily by running the following command:
2024-11-01T16:13:31.4004702Z #           python .github/scripts/update_runner_determinator.py
2024-11-01T16:13:31.4005238Z 
2024-11-01T16:13:31.4005368Z """
2024-11-01T16:13:31.4006089Z This runner determinator is used to determine which set of runners to run a
2024-11-01T16:13:31.4007053Z GitHub job on. It uses the first comment of a GitHub issue (by default
2024-11-01T16:13:31.4008169Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2024-11-01T16:13:31.4009186Z of which runners should be used to run which job.
2024-11-01T16:13:31.4009617Z 
2024-11-01T16:13:31.4010121Z The configuration has two parts, the settings and a list of opted-in users,
2024-11-01T16:13:31.4011141Z separated by a line containing "---".  If the line is not present, the
2024-11-01T16:13:31.4012196Z settings are considered to be empty with only the second part, the user
2024-11-01T16:13:31.4012938Z list, defined.
2024-11-01T16:13:31.4013144Z 
2024-11-01T16:13:31.4013553Z The first part is a YAML block that defines the rollout settings. This can be
2024-11-01T16:13:31.4014673Z used to define any settings that are needed to determine which runners to use.
2024-11-01T16:13:31.4015649Z It's fields are defined by the RolloutSettings class below.
2024-11-01T16:13:31.4016131Z 
2024-11-01T16:13:31.4016598Z The second part is a list of users who are explicitly opted in to the LF fleet.
2024-11-01T16:13:31.4017624Z The user list is also a comma separated list of additional features or
2024-11-01T16:13:31.4018445Z experiments which the user could be opted in to.
2024-11-01T16:13:31.4018880Z 
2024-11-01T16:13:31.4019090Z The user list has the following rules:
2024-11-01T16:13:31.4019448Z 
2024-11-01T16:13:31.4020203Z - Users are GitHub usernames, which must start with the @ prefix
2024-11-01T16:13:31.4021246Z - Each user is also a comma-separated list of features/experiments to enable
2024-11-01T16:13:31.4022142Z - A "#" prefix opts the user out of all experiments
2024-11-01T16:13:31.4022566Z 
2024-11-01T16:13:31.4022794Z Example config:
2024-11-01T16:13:31.4023281Z     # A list of experiments that can be opted into.
2024-11-01T16:13:31.4024060Z     # This defines the behavior they'll induce when opted into.
2024-11-01T16:13:31.4024761Z     # Expected syntax is:
2024-11-01T16:13:31.4025469Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2024-11-01T16:13:31.4026583Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2024-11-01T16:13:31.4027271Z 
2024-11-01T16:13:31.4027539Z     experiments:
2024-11-01T16:13:31.4027886Z       lf:
2024-11-01T16:13:31.4028245Z         rollout_percent: 25
2024-11-01T16:13:31.4028804Z         all_branches: false
2024-11-01T16:13:31.4029219Z         default: true
2024-11-01T16:13:31.4029634Z     ---
2024-11-01T16:13:31.4029817Z 
2024-11-01T16:13:31.4030090Z     # Opt-ins:
2024-11-01T16:13:31.4030698Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2024-11-01T16:13:31.4031716Z     # and specifying experiments to enable in a comma-separated list.
2024-11-01T16:13:31.4032567Z     # Experiments should be from the above list.
2024-11-01T16:13:31.4032972Z 
2024-11-01T16:13:31.4033121Z     @User1,lf,split_build
2024-11-01T16:13:31.4033561Z     @User2,lf
2024-11-01T16:13:31.4034038Z     @User3,split_build
2024-11-01T16:13:31.4034407Z """
2024-11-01T16:13:31.4034612Z 
2024-11-01T16:13:31.4034748Z import logging
2024-11-01T16:13:31.4035201Z import os
2024-11-01T16:13:31.4035520Z import random
2024-11-01T16:13:31.4036117Z from argparse import ArgumentParser
2024-11-01T16:13:31.4036751Z from logging import LogRecord
2024-11-01T16:13:31.4037430Z from typing import Any, Dict, FrozenSet, Iterable, List, NamedTuple, Tuple
2024-11-01T16:13:31.4038080Z 
2024-11-01T16:13:31.4038212Z import yaml
2024-11-01T16:13:31.4038673Z from github import Auth, Github
2024-11-01T16:13:31.4039145Z from github.Issue import Issue
2024-11-01T16:13:31.4039497Z 
2024-11-01T16:13:31.4039502Z 
2024-11-01T16:13:31.4039705Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2024-11-01T16:13:31.4040506Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2024-11-01T16:13:31.4041423Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2024-11-01T16:13:31.4042090Z 
2024-11-01T16:13:31.4042311Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2024-11-01T16:13:31.4043009Z GH_OUTPUT_KEY_AMI = "runner-ami"
2024-11-01T16:13:31.4043584Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2024-11-01T16:13:31.4043967Z 
2024-11-01T16:13:31.4043978Z 
2024-11-01T16:13:31.4044405Z SETTING_EXPERIMENTS = "experiments"
2024-11-01T16:13:31.4044880Z 
2024-11-01T16:13:31.4045038Z LF_FLEET_EXPERIMENT = "lf"
2024-11-01T16:13:31.4045521Z CANARY_FLEET_SUFFIX = ".c"
2024-11-01T16:13:31.4045789Z 
2024-11-01T16:13:31.4045794Z 
2024-11-01T16:13:31.4045960Z class Experiment(NamedTuple):
2024-11-01T16:13:31.4046509Z     rollout_perc: float = (
2024-11-01T16:13:31.4047300Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2024-11-01T16:13:31.4047999Z     )
2024-11-01T16:13:31.4048421Z     all_branches: bool = (
2024-11-01T16:13:31.4049109Z         False  # If True, the experiment is also enabled on the exception branches
2024-11-01T16:13:31.4049807Z     )
2024-11-01T16:13:31.4050230Z     default: bool = (
2024-11-01T16:13:31.4050842Z         True  # If True, the experiment is enabled by default for all queries
2024-11-01T16:13:31.4051525Z     )
2024-11-01T16:13:31.4051791Z 
2024-11-01T16:13:31.4051972Z     # Add more fields as needed
2024-11-01T16:13:31.4052316Z 
2024-11-01T16:13:31.4052322Z 
2024-11-01T16:13:31.4052477Z class Settings(NamedTuple):
2024-11-01T16:13:31.4052907Z     """
2024-11-01T16:13:31.4053605Z     Settings for the experiments that can be opted into.
2024-11-01T16:13:31.4054212Z     """
2024-11-01T16:13:31.4054384Z 
2024-11-01T16:13:31.4054623Z     experiments: Dict[str, Experiment] = {}
2024-11-01T16:13:31.4055069Z 
2024-11-01T16:13:31.4055074Z 
2024-11-01T16:13:31.4055267Z class ColorFormatter(logging.Formatter):
2024-11-01T16:13:31.4055937Z     """Color codes the log messages based on the log level"""
2024-11-01T16:13:31.4056393Z 
2024-11-01T16:13:31.4056577Z     COLORS = {
2024-11-01T16:13:31.4057012Z         "WARNING": "\033[33m",  # Yellow
2024-11-01T16:13:31.4057555Z         "ERROR": "\033[31m",  # Red
2024-11-01T16:13:31.4058073Z         "CRITICAL": "\033[31m",  # Red
2024-11-01T16:13:31.4058608Z         "INFO": "\033[0m",  # Reset
2024-11-01T16:13:31.4059101Z         "DEBUG": "\033[0m",  # Reset
2024-11-01T16:13:31.4059593Z     }
2024-11-01T16:13:31.4059825Z 
2024-11-01T16:13:31.4060086Z     def format(self, record: LogRecord) -> str:
2024-11-01T16:13:31.4060924Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2024-11-01T16:13:31.4061912Z         record.msg = f"{log_color}{record.msg}\033[0m"
2024-11-01T16:13:31.4062577Z         return super().format(record)
2024-11-01T16:13:31.4062947Z 
2024-11-01T16:13:31.4062952Z 
2024-11-01T16:13:31.4063141Z handler = logging.StreamHandler()
2024-11-01T16:13:31.4063941Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2024-11-01T16:13:31.4064609Z 
2024-11-01T16:13:31.4064894Z log = logging.getLogger(os.path.basename(__file__))
2024-11-01T16:13:31.4065488Z log.addHandler(handler)
2024-11-01T16:13:31.4065930Z log.setLevel(logging.INFO)
2024-11-01T16:13:31.4066294Z 
2024-11-01T16:13:31.4066299Z 
2024-11-01T16:13:31.4066623Z def set_github_output(key: str, value: str) -> None:
2024-11-01T16:13:31.4067189Z     """
2024-11-01T16:13:31.4067859Z     Defines outputs of the github action that invokes this script
2024-11-01T16:13:31.4068625Z     """
2024-11-01T16:13:31.4068997Z     if not GITHUB_OUTPUT:
2024-11-01T16:13:31.4070327Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2024-11-01T16:13:31.4071707Z         log.warning(
2024-11-01T16:13:31.4072630Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2024-11-01T16:13:31.4073671Z         )
2024-11-01T16:13:31.4097562Z         print(f"::set-output name={key}::{value}")
2024-11-01T16:13:31.4098198Z         return
2024-11-01T16:13:31.4098424Z 
2024-11-01T16:13:31.4098788Z     with open(GITHUB_OUTPUT, "a") as f:
2024-11-01T16:13:31.4099448Z         log.info(f"Setting output: {key}='{value}'")
2024-11-01T16:13:31.4100052Z         f.write(f"{key}={value}\n")
2024-11-01T16:13:31.4100379Z 
2024-11-01T16:13:31.4100403Z 
2024-11-01T16:13:31.4100890Z def _str_comma_separated_to_set(value: str) -> FrozenSet[str]:
2024-11-01T16:13:31.4101535Z     return frozenset(
2024-11-01T16:13:31.4102209Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2024-11-01T16:13:31.4103051Z     )
2024-11-01T16:13:31.4103228Z 
2024-11-01T16:13:31.4103233Z 
2024-11-01T16:13:31.4103422Z def parse_args() -> Any:
2024-11-01T16:13:31.4103996Z     parser = ArgumentParser("Get dynamic rollout settings")
2024-11-01T16:13:31.4105101Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2024-11-01T16:13:31.4105901Z     parser.add_argument(
2024-11-01T16:13:31.4106402Z         "--github-issue-repo",
2024-11-01T16:13:31.4106937Z         type=str,
2024-11-01T16:13:31.4107298Z         required=False,
2024-11-01T16:13:31.4107806Z         default="pytorch/test-infra",
2024-11-01T16:13:31.4108446Z         help="GitHub repo to get the issue",
2024-11-01T16:13:31.4108944Z     )
2024-11-01T16:13:31.4109312Z     parser.add_argument(
2024-11-01T16:13:31.4109850Z         "--github-repo",
2024-11-01T16:13:31.4110233Z         type=str,
2024-11-01T16:13:31.4110857Z         required=True,
2024-11-01T16:13:31.4111414Z         help="GitHub repo where CI is running",
2024-11-01T16:13:31.4111949Z     )
2024-11-01T16:13:31.4112302Z     parser.add_argument(
2024-11-01T16:13:31.4113094Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2024-11-01T16:13:31.4113794Z     )
2024-11-01T16:13:31.4114140Z     parser.add_argument(
2024-11-01T16:13:31.4114942Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2024-11-01T16:13:31.4115665Z     )
2024-11-01T16:13:31.4116010Z     parser.add_argument(
2024-11-01T16:13:31.4116803Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2024-11-01T16:13:31.4117532Z     )
2024-11-01T16:13:31.4117875Z     parser.add_argument(
2024-11-01T16:13:31.4118724Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2024-11-01T16:13:31.4119465Z     )
2024-11-01T16:13:31.4119810Z     parser.add_argument(
2024-11-01T16:13:31.4120374Z         "--github-ref-type",
2024-11-01T16:13:31.4120784Z         type=str,
2024-11-01T16:13:31.4121162Z         required=True,
2024-11-01T16:13:31.4121796Z         help="Current GitHub ref type, branch or tag",
2024-11-01T16:13:31.4122364Z     )
2024-11-01T16:13:31.4122713Z     parser.add_argument(
2024-11-01T16:13:31.4123291Z         "--eligible-experiments",
2024-11-01T16:13:31.4123782Z         type=_str_comma_separated_to_set,
2024-11-01T16:13:31.4125414Z         required=False,
2024-11-01T16:13:31.4125951Z         default="",
2024-11-01T16:13:31.4126864Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2024-11-01T16:13:31.4127891Z     )
2024-11-01T16:13:31.4128081Z 
2024-11-01T16:13:31.4128344Z     return parser.parse_args()
2024-11-01T16:13:31.4128654Z 
2024-11-01T16:13:31.4128839Z 
2024-11-01T16:13:31.4129186Z def get_gh_client(github_token: str) -> Github:
2024-11-01T16:13:31.4129768Z     auth = Auth.Token(github_token)
2024-11-01T16:13:31.4130503Z     return Github(auth=auth)
2024-11-01T16:13:31.4130967Z 
2024-11-01T16:13:31.4130974Z 
2024-11-01T16:13:31.4131469Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:
2024-11-01T16:13:31.4132154Z     repo = gh.get_repo(repo)
2024-11-01T16:13:31.4132745Z     return repo.get_issue(number=issue_num)
2024-11-01T16:13:31.4133127Z 
2024-11-01T16:13:31.4133131Z 
2024-11-01T16:13:31.4133341Z def get_potential_pr_author(
2024-11-01T16:13:31.4134011Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2024-11-01T16:13:31.4134837Z ) -> str:
2024-11-01T16:13:31.4135411Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2024-11-01T16:13:31.4136287Z     # Fetch the actual username from the original PR. The PR number is
2024-11-01T16:13:31.4137222Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2024-11-01T16:13:31.4137720Z 
2024-11-01T16:13:31.4137883Z     gh = get_gh_client(github_token)
2024-11-01T16:13:31.4138226Z 
2024-11-01T16:13:31.4138571Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2024-11-01T16:13:31.4139263Z         split_tag = ref_name.split("/")
2024-11-01T16:13:31.4139792Z         if (
2024-11-01T16:13:31.4140174Z             len(split_tag) == 3
2024-11-01T16:13:31.4140688Z             and split_tag[0] == "ciflow"
2024-11-01T16:13:31.4141258Z             and split_tag[2].isnumeric()
2024-11-01T16:13:31.4141752Z         ):
2024-11-01T16:13:31.4142157Z             pr_number = split_tag[2]
2024-11-01T16:13:31.4142661Z             try:
2024-11-01T16:13:31.4143085Z                 repository = gh.get_repo(repo)
2024-11-01T16:13:31.4143787Z                 pull = repository.get_pull(number=int(pr_number))
2024-11-01T16:13:31.4144426Z             except Exception as e:
2024-11-01T16:13:31.4144969Z                 raise Exception(  # noqa: TRY002
2024-11-01T16:13:31.4145759Z                     f"issue with pull request {pr_number} from repo {repository}"
2024-11-01T16:13:31.4146457Z                 ) from e
2024-11-01T16:13:31.4147099Z             return pull.user.login
2024-11-01T16:13:31.4147750Z     # In all other cases, return the original input username
2024-11-01T16:13:31.4148384Z     return username
2024-11-01T16:13:31.4148611Z 
2024-11-01T16:13:31.4148615Z 
2024-11-01T16:13:31.4148920Z def is_exception_branch(branch: str) -> bool:
2024-11-01T16:13:31.4149505Z     """
2024-11-01T16:13:31.4150264Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2024-11-01T16:13:31.4151117Z     """
2024-11-01T16:13:31.4151731Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2024-11-01T16:13:31.4152338Z 
2024-11-01T16:13:31.4152343Z 
2024-11-01T16:13:31.4152552Z def load_yaml(yaml_text: str) -> Any:
2024-11-01T16:13:31.4153059Z     try:
2024-11-01T16:13:31.4153509Z         data = yaml.safe_load(yaml_text)
2024-11-01T16:13:31.4153993Z         return data
2024-11-01T16:13:31.4154425Z     except yaml.YAMLError as exc:
2024-11-01T16:13:31.4155036Z         log.exception("Error loading YAML")
2024-11-01T16:13:31.4155531Z         raise
2024-11-01T16:13:31.4155773Z 
2024-11-01T16:13:31.4155778Z 
2024-11-01T16:13:31.4156262Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> Tuple[str, str]:
2024-11-01T16:13:31.4157112Z     """
2024-11-01T16:13:31.4157761Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2024-11-01T16:13:31.4158468Z 
2024-11-01T16:13:31.4158896Z     If the issue body contains "---" then the text above that is the settings
2024-11-01T16:13:31.4159803Z     and the text below is the list of opted in users.
2024-11-01T16:13:31.4160234Z 
2024-11-01T16:13:31.4160724Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2024-11-01T16:13:31.4161494Z     """
2024-11-01T16:13:31.4162186Z     rollout_state_parts = rollout_state.split("---")
2024-11-01T16:13:31.4162820Z     if len(rollout_state_parts) >= 2:
2024-11-01T16:13:31.4163403Z         return rollout_state_parts[0], rollout_state_parts[1]
2024-11-01T16:13:31.4164337Z     else:
2024-11-01T16:13:31.4164736Z         return "", rollout_state
2024-11-01T16:13:31.4165044Z 
2024-11-01T16:13:31.4165049Z 
2024-11-01T16:13:31.4165223Z class UserOptins(Dict[str, List[str]]):
2024-11-01T16:13:31.4165843Z     """
2024-11-01T16:13:31.4166373Z     Dictionary of users with a list of features they have opted into
2024-11-01T16:13:31.4167023Z     """
2024-11-01T16:13:31.4167305Z 
2024-11-01T16:13:31.4167309Z 
2024-11-01T16:13:31.4167730Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2024-11-01T16:13:31.4168424Z     """
2024-11-01T16:13:31.4169306Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2024-11-01T16:13:31.4170233Z 
2024-11-01T16:13:31.4171038Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2024-11-01T16:13:31.4172221Z         - Example line: "@User1,lf,split_build"
2024-11-01T16:13:31.4173037Z         - A "#" prefix indicates the user is opted out of all experiments
2024-11-01T16:13:31.4173619Z 
2024-11-01T16:13:31.4173623Z 
2024-11-01T16:13:31.4173755Z     """
2024-11-01T16:13:31.4174111Z     optins = UserOptins()
2024-11-01T16:13:31.4174622Z     for user in user_optin_text.split("\n"):
2024-11-01T16:13:31.4175269Z         user = user.strip("\r\n\t -")
2024-11-01T16:13:31.4175839Z         if not user or not user.startswith("@"):
2024-11-01T16:13:31.4176438Z             # Not a valid user. Skip
2024-11-01T16:13:31.4176961Z             continue
2024-11-01T16:13:31.4177225Z 
2024-11-01T16:13:31.4177354Z         if user:
2024-11-01T16:13:31.4177807Z             usr_name = user.split(",")[0].strip("@")
2024-11-01T16:13:31.4178595Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2024-11-01T16:13:31.4179169Z 
2024-11-01T16:13:31.4179310Z     return optins
2024-11-01T16:13:31.4179546Z 
2024-11-01T16:13:31.4179551Z 
2024-11-01T16:13:31.4180120Z def parse_settings_from_text(settings_text: str) -> Settings:
2024-11-01T16:13:31.4180831Z     """
2024-11-01T16:13:31.4181468Z     Parse the experiments from the issue body into a list of ExperimentSettings
2024-11-01T16:13:31.4182234Z     """
2024-11-01T16:13:31.4182606Z     try:
2024-11-01T16:13:31.4182978Z         if settings_text:
2024-11-01T16:13:31.4183783Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2024-11-01T16:13:31.4184759Z             # for easy reading
2024-11-01T16:13:31.4185691Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2024-11-01T16:13:31.4186731Z             #       the backtick character in shell commands.
2024-11-01T16:13:31.4187465Z             backtick = chr(96)  # backtick character
2024-11-01T16:13:31.4188146Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2024-11-01T16:13:31.4188869Z             settings = load_yaml(settings_text)
2024-11-01T16:13:31.4189319Z 
2024-11-01T16:13:31.4189812Z             # For now we just load experiments. We can expand this if/when we add more settings
2024-11-01T16:13:31.4190605Z             experiments = {}
2024-11-01T16:13:31.4190919Z 
2024-11-01T16:13:31.4191295Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2024-11-01T16:13:31.4192147Z                 valid_settings = {}
2024-11-01T16:13:31.4192642Z                 for setting in exp_settings:
2024-11-01T16:13:31.4193250Z                     if setting not in Experiment._fields:
2024-11-01T16:13:31.4193910Z                         log.warning(
2024-11-01T16:13:31.4194658Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2024-11-01T16:13:31.4195387Z                         )
2024-11-01T16:13:31.4196038Z                     else:
2024-11-01T16:13:31.4196585Z                         valid_settings[setting] = exp_settings[setting]
2024-11-01T16:13:31.4197053Z 
2024-11-01T16:13:31.4197316Z                 experiments[exp_name] = Experiment(**valid_settings)
2024-11-01T16:13:31.4198057Z             return Settings(experiments)
2024-11-01T16:13:31.4198409Z 
2024-11-01T16:13:31.4198586Z     except Exception:
2024-11-01T16:13:31.4199044Z         log.exception("Failed to parse settings")
2024-11-01T16:13:31.4199547Z 
2024-11-01T16:13:31.4199687Z     return Settings()
2024-11-01T16:13:31.4199922Z 
2024-11-01T16:13:31.4199927Z 
2024-11-01T16:13:31.4200264Z def parse_settings(rollout_state: str) -> Settings:
2024-11-01T16:13:31.4200816Z     """
2024-11-01T16:13:31.4201304Z     Parse settings, if any, from the rollout state.
2024-11-01T16:13:31.4201764Z 
2024-11-01T16:13:31.4202238Z     If the issue body contains "---" then the text above that is the settings
2024-11-01T16:13:31.4203155Z     and the text below is the list of opted in users.
2024-11-01T16:13:31.4203591Z 
2024-11-01T16:13:31.4204705Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2024-11-01T16:13:31.4205605Z     """
2024-11-01T16:13:31.4206288Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2024-11-01T16:13:31.4207072Z     return parse_settings_from_text(settings_text)
2024-11-01T16:13:31.4207538Z 
2024-11-01T16:13:31.4207542Z 
2024-11-01T16:13:31.4207835Z def parse_users(rollout_state: str) -> UserOptins:
2024-11-01T16:13:31.4208481Z     """
2024-11-01T16:13:31.4208835Z     Parse users from the rollout state.
2024-11-01T16:13:31.4209238Z 
2024-11-01T16:13:31.4209364Z     """
2024-11-01T16:13:31.4209958Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2024-11-01T16:13:31.4210703Z     return parse_user_opt_in_from_text(users_text)
2024-11-01T16:13:31.4211164Z 
2024-11-01T16:13:31.4211169Z 
2024-11-01T16:13:31.4211692Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2024-11-01T16:13:31.4212574Z     """
2024-11-01T16:13:31.4213009Z     Check if a user is opted into an experiment
2024-11-01T16:13:31.4213698Z     """
2024-11-01T16:13:31.4214219Z     return experiment_name in user_optins.get(user, [])
2024-11-01T16:13:31.4214683Z 
2024-11-01T16:13:31.4214687Z 
2024-11-01T16:13:31.4214867Z def get_runner_prefix(
2024-11-01T16:13:31.4215256Z     rollout_state: str,
2024-11-01T16:13:31.4215781Z     workflow_requestors: Iterable[str],
2024-11-01T16:13:31.4216306Z     branch: str,
2024-11-01T16:13:31.4216752Z     eligible_experiments: FrozenSet[str] = frozenset(),
2024-11-01T16:13:31.4217479Z     is_canary: bool = False,
2024-11-01T16:13:31.4217968Z ) -> str:
2024-11-01T16:13:31.4218348Z     settings = parse_settings(rollout_state)
2024-11-01T16:13:31.4219045Z     user_optins = parse_users(rollout_state)
2024-11-01T16:13:31.4219438Z 
2024-11-01T16:13:31.4219616Z     fleet_prefix = ""
2024-11-01T16:13:31.4219997Z     prefixes = []
2024-11-01T16:13:31.4220727Z     for experiment_name, experiment_settings in settings.experiments.items():
2024-11-01T16:13:31.4221741Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2024-11-01T16:13:31.4222472Z             log.info(
2024-11-01T16:13:31.4223276Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2024-11-01T16:13:31.4224105Z             )
2024-11-01T16:13:31.4224430Z             continue
2024-11-01T16:13:31.4224708Z 
2024-11-01T16:13:31.4224931Z         if eligible_experiments:
2024-11-01T16:13:31.4225507Z             if experiment_name not in eligible_experiments:
2024-11-01T16:13:31.4226201Z                 exp_list = ", ".join(eligible_experiments)
2024-11-01T16:13:31.4226812Z                 log.info(
2024-11-01T16:13:31.4227756Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2024-11-01T16:13:31.4228682Z                 )
2024-11-01T16:13:31.4229243Z                 continue
2024-11-01T16:13:31.4229761Z         elif not experiment_settings.default:
2024-11-01T16:13:31.4230311Z             log.info(
2024-11-01T16:13:31.4231100Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2024-11-01T16:13:31.4231902Z             )
2024-11-01T16:13:31.4232257Z             continue
2024-11-01T16:13:31.4232484Z 
2024-11-01T16:13:31.4232827Z         # Is any workflow_requestor opted in to this experiment?
2024-11-01T16:13:31.4233491Z         opted_in_users = [
2024-11-01T16:13:31.4233913Z             requestor
2024-11-01T16:13:31.4234412Z             for requestor in workflow_requestors
2024-11-01T16:13:31.4235147Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2024-11-01T16:13:31.4235806Z         ]
2024-11-01T16:13:31.4235993Z 
2024-11-01T16:13:31.4236198Z         enabled = False
2024-11-01T16:13:31.4236640Z         if opted_in_users:
2024-11-01T16:13:31.4237080Z             log.info(
2024-11-01T16:13:31.4237818Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2024-11-01T16:13:31.4238581Z             )
2024-11-01T16:13:31.4238947Z             enabled = True
2024-11-01T16:13:31.4239208Z 
2024-11-01T16:13:31.4239498Z         elif experiment_settings.rollout_perc:
2024-11-01T16:13:31.4240450Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2024-11-01T16:13:31.4241525Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2024-11-01T16:13:31.4242316Z                 log.info(
2024-11-01T16:13:31.4243210Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2024-11-01T16:13:31.4244455Z                 )
2024-11-01T16:13:31.4244964Z                 enabled = True
2024-11-01T16:13:31.4245258Z 
2024-11-01T16:13:31.4245394Z         if enabled:
2024-11-01T16:13:31.4245820Z             label = experiment_name
2024-11-01T16:13:31.4246474Z             if experiment_name == LF_FLEET_EXPERIMENT:
2024-11-01T16:13:31.4247374Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2024-11-01T16:13:31.4248635Z                 #  - If it's enabled, then we always list it's prefix first
2024-11-01T16:13:31.4249644Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2024-11-01T16:13:31.4250353Z                 if is_canary:
2024-11-01T16:13:31.4250871Z                     label += CANARY_FLEET_SUFFIX
2024-11-01T16:13:31.4251504Z                 fleet_prefix = label
2024-11-01T16:13:31.4251962Z             else:
2024-11-01T16:13:31.4252395Z                 prefixes.append(label)
2024-11-01T16:13:31.4252738Z 
2024-11-01T16:13:31.4252987Z     if len(prefixes) > 1:
2024-11-01T16:13:31.4253383Z         log.error(
2024-11-01T16:13:31.4254712Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2024-11-01T16:13:31.4256069Z         )
2024-11-01T16:13:31.4256474Z         prefixes = prefixes[:1]
2024-11-01T16:13:31.4256778Z 
2024-11-01T16:13:31.4256939Z     # Fleet always comes first
2024-11-01T16:13:31.4257468Z     if fleet_prefix:
2024-11-01T16:13:31.4257927Z         prefixes.insert(0, fleet_prefix)
2024-11-01T16:13:31.4258293Z 
2024-11-01T16:13:31.4258530Z     return ".".join(prefixes) + "." if prefixes else ""
2024-11-01T16:13:31.4259062Z 
2024-11-01T16:13:31.4259066Z 
2024-11-01T16:13:31.4259578Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2024-11-01T16:13:31.4260398Z     """
2024-11-01T16:13:31.4260972Z     Gets the first comment of the issue, which contains the desired rollout state.
2024-11-01T16:13:31.4261694Z 
2024-11-01T16:13:31.4262164Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2024-11-01T16:13:31.4262944Z     """
2024-11-01T16:13:31.4263603Z     gh = get_gh_client(github_token)
2024-11-01T16:13:31.4264269Z     issue = get_issue(gh, repo, issue_num)
2024-11-01T16:13:31.4264929Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2024-11-01T16:13:31.4265384Z 
2024-11-01T16:13:31.4265389Z 
2024-11-01T16:13:31.4265599Z def main() -> None:
2024-11-01T16:13:31.4266053Z     args = parse_args()
2024-11-01T16:13:31.4266347Z 
2024-11-01T16:13:31.4266548Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2024-11-01T16:13:31.4266936Z 
2024-11-01T16:13:31.4267110Z     try:
2024-11-01T16:13:31.4267574Z         rollout_state = get_rollout_state_from_issue(
2024-11-01T16:13:31.4268327Z             args.github_token, args.github_issue_repo, args.github_issue
2024-11-01T16:13:31.4269054Z         )
2024-11-01T16:13:31.4269305Z 
2024-11-01T16:13:31.4269492Z         username = get_potential_pr_author(
2024-11-01T16:13:31.4270041Z             args.github_token,
2024-11-01T16:13:31.4270527Z             args.github_repo,
2024-11-01T16:13:31.4271033Z             args.github_actor,
2024-11-01T16:13:31.4271503Z             args.github_ref_type,
2024-11-01T16:13:31.4272010Z             args.github_branch,
2024-11-01T16:13:31.4272498Z         )
2024-11-01T16:13:31.4272723Z 
2024-11-01T16:13:31.4273064Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2024-11-01T16:13:31.4273555Z 
2024-11-01T16:13:31.4273785Z         runner_label_prefix = get_runner_prefix(
2024-11-01T16:13:31.4274384Z             rollout_state,
2024-11-01T16:13:31.4274874Z             (args.github_issue_owner, username),
2024-11-01T16:13:31.4275453Z             args.github_branch,
2024-11-01T16:13:31.4275982Z             args.eligible_experiments,
2024-11-01T16:13:31.4276504Z             is_canary,
2024-11-01T16:13:31.4276893Z         )
2024-11-01T16:13:31.4277137Z 
2024-11-01T16:13:31.4277295Z     except Exception as e:
2024-11-01T16:13:31.4277751Z         log.error(
2024-11-01T16:13:31.4278476Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2024-11-01T16:13:31.4279391Z         )
2024-11-01T16:13:31.4279590Z 
2024-11-01T16:13:31.4279914Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2024-11-01T16:13:31.4280607Z 
2024-11-01T16:13:31.4280612Z 
2024-11-01T16:13:31.4280755Z if __name__ == "__main__":
2024-11-01T16:13:31.4281271Z     main()
2024-11-01T16:13:31.4281464Z 
2024-11-01T16:13:31.4459617Z ##[group]Run python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2024-11-01T16:13:31.4460589Z python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2024-11-01T16:13:31.4518061Z shell: /usr/bin/bash -e {0}
2024-11-01T16:13:31.4518568Z env:
2024-11-01T16:13:31.4519166Z   GITHUB_TOKEN: ***
2024-11-01T16:13:31.4519702Z   ISSUE_NUMBER: 5132
2024-11-01T16:13:31.4520167Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2024-11-01T16:13:31.4520643Z   ISSUE_OWNER: 
2024-11-01T16:13:31.4521114Z   CHECK_EXPERIMENTS: 
2024-11-01T16:13:31.4521523Z ##[endgroup]
2024-11-01T16:13:31.8907094Z Defaulting to user installation because normal site-packages is not writeable
2024-11-01T16:13:32.2815047Z Collecting urllib3==1.26.18
2024-11-01T16:13:32.3208095Z   Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
2024-11-01T16:13:32.3407131Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.8/143.8 KB 7.8 MB/s eta 0:00:00
2024-11-01T16:13:32.3737242Z Collecting PyGithub==2.3.0
2024-11-01T16:13:32.3776190Z   Downloading PyGithub-2.3.0-py3-none-any.whl (354 kB)
2024-11-01T16:13:32.3866383Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 354.4/354.4 KB 50.4 MB/s eta 0:00:00
2024-11-01T16:13:32.4399795Z Collecting pynacl>=1.4.0
2024-11-01T16:13:32.4441551Z   Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
2024-11-01T16:13:32.4565688Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 856.7/856.7 KB 82.6 MB/s eta 0:00:00
2024-11-01T16:13:32.4854304Z Collecting Deprecated
2024-11-01T16:13:32.4899679Z   Downloading Deprecated-1.2.14-py2.py3-none-any.whl (9.6 kB)
2024-11-01T16:13:32.4931514Z Requirement already satisfied: requests>=2.14.0 in /usr/lib/python3/dist-packages (from PyGithub==2.3.0) (2.25.1)
2024-11-01T16:13:32.5211788Z Collecting pyjwt[crypto]>=2.4.0
2024-11-01T16:13:32.5213403Z   Downloading PyJWT-2.9.0-py3-none-any.whl (22 kB)
2024-11-01T16:13:32.5507074Z Collecting typing-extensions>=4.0.0
2024-11-01T16:13:32.5551466Z   Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB)
2024-11-01T16:13:32.5697005Z Requirement already satisfied: cryptography>=3.4.0 in /usr/lib/python3/dist-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.3.0) (3.4.8)
2024-11-01T16:13:32.7872624Z Collecting cffi>=1.4.1
2024-11-01T16:13:32.7925264Z   Downloading cffi-1.17.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (446 kB)
2024-11-01T16:13:32.8017377Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 446.2/446.2 KB 70.3 MB/s eta 0:00:00
2024-11-01T16:13:32.9782697Z Collecting wrapt<2,>=1.10
2024-11-01T16:13:32.9847065Z   Downloading wrapt-1.16.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (80 kB)
2024-11-01T16:13:32.9889662Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 80.3/80.3 KB 32.3 MB/s eta 0:00:00
2024-11-01T16:13:33.0084748Z Collecting pycparser
2024-11-01T16:13:33.0138203Z   Downloading pycparser-2.22-py3-none-any.whl (117 kB)
2024-11-01T16:13:33.0191111Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 KB 34.9 MB/s eta 0:00:00
2024-11-01T16:13:33.2117953Z Installing collected packages: wrapt, urllib3, typing-extensions, pyjwt, pycparser, Deprecated, cffi, pynacl, PyGithub
2024-11-01T16:13:33.6821562Z Successfully installed Deprecated-1.2.14 PyGithub-2.3.0 cffi-1.17.1 pycparser-2.22 pyjwt-2.9.0 pynacl-1.5.0 typing-extensions-4.12.2 urllib3-1.26.18 wrapt-1.16.0
2024-11-01T16:13:33.7550474Z ##[group]Run curr_branch="ciflow/trunk/138766"
2024-11-01T16:13:33.7551090Z curr_branch="ciflow/trunk/138766"
2024-11-01T16:13:33.7551548Z curr_ref_type="tag"
2024-11-01T16:13:33.7552108Z echo "Current branch is '$curr_branch'"
2024-11-01T16:13:33.7552528Z 
2024-11-01T16:13:33.7552878Z python3 runner_determinator.py \
2024-11-01T16:13:33.7553698Z   --github-token "$GITHUB_TOKEN" \
2024-11-01T16:13:33.7554134Z   --github-issue "$ISSUE_NUMBER" \
2024-11-01T16:13:33.7554609Z   --github-branch "$curr_branch" \
2024-11-01T16:13:33.7555144Z   --github-actor "$TRIGGERING_ACTOR" \
2024-11-01T16:13:33.7555627Z   --github-issue-owner "$ISSUE_OWNER" \
2024-11-01T16:13:33.7556144Z   --github-ref-type "$curr_ref_type" \
2024-11-01T16:13:33.7556698Z   --github-repo "$GITHUB_REPOSITORY" \
2024-11-01T16:13:33.7557332Z   --eligible-experiments "$CHECK_EXPERIMENTS" \
2024-11-01T16:13:33.7615903Z shell: /usr/bin/bash -e {0}
2024-11-01T16:13:33.7616373Z env:
2024-11-01T16:13:33.7617134Z   GITHUB_TOKEN: ***
2024-11-01T16:13:33.7617513Z   ISSUE_NUMBER: 5132
2024-11-01T16:13:33.7617935Z   TRIGGERING_ACTOR: pytorch-bot[bot]
2024-11-01T16:13:33.7618367Z   ISSUE_OWNER: 
2024-11-01T16:13:33.7618696Z   CHECK_EXPERIMENTS: 
2024-11-01T16:13:33.7619079Z ##[endgroup]
2024-11-01T16:13:33.7697363Z Current branch is 'ciflow/trunk/138766'
2024-11-01T16:13:35.5628556Z INFO    : Skipping experiment 'awsa100', as it is not a default experiment
2024-11-01T16:13:35.5630085Z INFO    : Setting output: label-type=''
2024-11-01T16:13:35.6011682Z Evaluate and set job outputs
2024-11-01T16:13:35.6023233Z Cleaning up orphan processes