2025-06-01T21:03:58.6812514Z Current runner version: '2.324.0'
2025-06-01T21:03:58.6848477Z ##[group]Operating System
2025-06-01T21:03:58.6849705Z Ubuntu
2025-06-01T21:03:58.6850468Z 24.04.2
2025-06-01T21:03:58.6851191Z LTS
2025-06-01T21:03:58.6851964Z ##[endgroup]
2025-06-01T21:03:58.6852758Z ##[group]Runner Image
2025-06-01T21:03:58.6853985Z Image: ubuntu-24.04
2025-06-01T21:03:58.6854830Z Version: 20250527.1.0
2025-06-01T21:03:58.6856555Z Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20250527.1/images/ubuntu/Ubuntu2404-Readme.md
2025-06-01T21:03:58.6858869Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20250527.1
2025-06-01T21:03:58.6860495Z ##[endgroup]
2025-06-01T21:03:58.6861304Z ##[group]Runner Image Provisioner
2025-06-01T21:03:58.6862234Z 2.0.437.1
2025-06-01T21:03:58.6863087Z ##[endgroup]
2025-06-01T21:03:58.6864810Z ##[group]GITHUB_TOKEN Permissions
2025-06-01T21:03:58.6867157Z Contents: read
2025-06-01T21:03:58.6868096Z Metadata: read
2025-06-01T21:03:58.6869067Z ##[endgroup]
2025-06-01T21:03:58.6872007Z Secret source: Actions
2025-06-01T21:03:58.6873045Z Prepare workflow directory
2025-06-01T21:03:58.7601283Z Prepare all required actions
2025-06-01T21:03:58.7679248Z Complete job name: before-test / get-label-type / runner-determinator
2025-06-01T21:03:59.2642827Z ##[group]Run cat <<EOF > runner_determinator.py
2025-06-01T21:03:59.2645463Z cat <<EOF > runner_determinator.py
2025-06-01T21:03:59.2646210Z # flake8: noqa: G004
2025-06-01T21:03:59.2646784Z 
2025-06-01T21:03:59.2647543Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2025-06-01T21:03:59.2648726Z #       must be kept in sync. You can do it easily by running the following command:
2025-06-01T21:03:59.2649745Z #           python .github/scripts/update_runner_determinator.py
2025-06-01T21:03:59.2650465Z 
2025-06-01T21:03:59.2650993Z """
2025-06-01T21:03:59.2651725Z This runner determinator is used to determine which set of runners to run a
2025-06-01T21:03:59.2652815Z GitHub job on. It uses the first comment of a GitHub issue (by default
2025-06-01T21:03:59.2654315Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2025-06-01T21:03:59.2655325Z of which runners should be used to run which job.
2025-06-01T21:03:59.2656112Z 
2025-06-01T21:03:59.2656845Z The configuration has two parts, the settings and a list of opted-in users,
2025-06-01T21:03:59.2657960Z separated by a line containing "---".  If the line is not present, the
2025-06-01T21:03:59.2658974Z settings are considered to be empty with only the second part, the user
2025-06-01T21:03:59.2659933Z list, defined.
2025-06-01T21:03:59.2660439Z 
2025-06-01T21:03:59.2661146Z The first part is a YAML block that defines the rollout settings. This can be
2025-06-01T21:03:59.2662325Z used to define any settings that are needed to determine which runners to use.
2025-06-01T21:03:59.2663570Z It's fields are defined by the RolloutSettings class below.
2025-06-01T21:03:59.2664335Z 
2025-06-01T21:03:59.2665141Z The second part is a list of users who are explicitly opted in to the LF fleet.
2025-06-01T21:03:59.2666209Z The user list is also a comma separated list of additional features or
2025-06-01T21:03:59.2667148Z experiments which the user could be opted in to.
2025-06-01T21:03:59.2667894Z 
2025-06-01T21:03:59.2668424Z The user list has the following rules:
2025-06-01T21:03:59.2669037Z 
2025-06-01T21:03:59.2669807Z - Users are GitHub usernames, which must start with the @ prefix
2025-06-01T21:03:59.2670846Z - Each user is also a comma-separated list of features/experiments to enable
2025-06-01T21:03:59.2671757Z - A "#" prefix opts the user out of all experiments
2025-06-01T21:03:59.2672775Z 
2025-06-01T21:03:59.2673497Z Example config:
2025-06-01T21:03:59.2674171Z     # A list of experiments that can be opted into.
2025-06-01T21:03:59.2675117Z     # This defines the behavior they'll induce when opted into.
2025-06-01T21:03:59.2675881Z     # Expected syntax is:
2025-06-01T21:03:59.2676732Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2025-06-01T21:03:59.2677884Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2025-06-01T21:03:59.2678806Z 
2025-06-01T21:03:59.2679293Z     experiments:
2025-06-01T21:03:59.2679888Z       lf:
2025-06-01T21:03:59.2680442Z         rollout_percent: 25
2025-06-01T21:03:59.2681025Z         all_branches: false
2025-06-01T21:03:59.2681673Z         default: true
2025-06-01T21:03:59.2682231Z     ---
2025-06-01T21:03:59.2682734Z 
2025-06-01T21:03:59.2683301Z     # Opt-ins:
2025-06-01T21:03:59.2684168Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2025-06-01T21:03:59.2685392Z     # and specifying experiments to enable in a comma-separated list.
2025-06-01T21:03:59.2686318Z     # To always opt out of an experiment, prefix it with a "-".
2025-06-01T21:03:59.2687601Z     # Experiments should be from the above list.
2025-06-01T21:03:59.2688243Z 
2025-06-01T21:03:59.2688760Z     @User1,-lf,split_build
2025-06-01T21:03:59.2689436Z     @User2,lf
2025-06-01T21:03:59.2689956Z     @User3,split_build
2025-06-01T21:03:59.2690507Z """
2025-06-01T21:03:59.2691013Z 
2025-06-01T21:03:59.2691516Z import json
2025-06-01T21:03:59.2692014Z import logging
2025-06-01T21:03:59.2766931Z import os
2025-06-01T21:03:59.2767446Z import random
2025-06-01T21:03:59.2767960Z import re
2025-06-01T21:03:59.2768381Z import sys
2025-06-01T21:03:59.2768866Z from argparse import ArgumentParser
2025-06-01T21:03:59.2769479Z from collections.abc import Iterable
2025-06-01T21:03:59.2770067Z from functools import cache
2025-06-01T21:03:59.2770662Z from logging import LogRecord
2025-06-01T21:03:59.2771246Z from typing import Any, NamedTuple
2025-06-01T21:03:59.2771871Z from urllib.request import Request, urlopen
2025-06-01T21:03:59.2772473Z 
2025-06-01T21:03:59.2772852Z import yaml
2025-06-01T21:03:59.2773453Z from github import Auth, Github
2025-06-01T21:03:59.2774015Z from github.Issue import Issue
2025-06-01T21:03:59.2774538Z 
2025-06-01T21:03:59.2774900Z 
2025-06-01T21:03:59.2775370Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2025-06-01T21:03:59.2776145Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2025-06-01T21:03:59.2777125Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2025-06-01T21:03:59.2777884Z 
2025-06-01T21:03:59.2778357Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2025-06-01T21:03:59.2779008Z GH_OUTPUT_KEY_AMI = "runner-ami"
2025-06-01T21:03:59.2779600Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2025-06-01T21:03:59.2780249Z OPT_OUT_LABEL = "no-runner-experiments"
2025-06-01T21:03:59.2780820Z 
2025-06-01T21:03:59.2781237Z SETTING_EXPERIMENTS = "experiments"
2025-06-01T21:03:59.2781783Z 
2025-06-01T21:03:59.2782176Z LF_FLEET_EXPERIMENT = "lf"
2025-06-01T21:03:59.2782711Z CANARY_FLEET_SUFFIX = ".c"
2025-06-01T21:03:59.2783490Z 
2025-06-01T21:03:59.2783874Z 
2025-06-01T21:03:59.2784285Z class Experiment(NamedTuple):
2025-06-01T21:03:59.2784861Z     rollout_perc: float = (
2025-06-01T21:03:59.2785888Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2025-06-01T21:03:59.2786625Z     )
2025-06-01T21:03:59.2787056Z     all_branches: bool = (
2025-06-01T21:03:59.2787793Z         False  # If True, the experiment is also enabled on the exception branches
2025-06-01T21:03:59.2788580Z     )
2025-06-01T21:03:59.2788992Z     default: bool = (
2025-06-01T21:03:59.2789647Z         True  # If True, the experiment is enabled by default for all queries
2025-06-01T21:03:59.2790336Z     )
2025-06-01T21:03:59.2790726Z 
2025-06-01T21:03:59.2791135Z     # Add more fields as needed
2025-06-01T21:03:59.2791650Z 
2025-06-01T21:03:59.2792016Z 
2025-06-01T21:03:59.2792421Z class Settings(NamedTuple):
2025-06-01T21:03:59.2792935Z     """
2025-06-01T21:03:59.2793614Z     Settings for the experiments that can be opted into.
2025-06-01T21:03:59.2794287Z     """
2025-06-01T21:03:59.2794683Z 
2025-06-01T21:03:59.2795126Z     experiments: dict[str, Experiment] = {}
2025-06-01T21:03:59.2795700Z 
2025-06-01T21:03:59.2796200Z 
2025-06-01T21:03:59.2796673Z class ColorFormatter(logging.Formatter):
2025-06-01T21:03:59.2797387Z     """Color codes the log messages based on the log level"""
2025-06-01T21:03:59.2798019Z 
2025-06-01T21:03:59.2798400Z     COLORS = {
2025-06-01T21:03:59.2798869Z         "WARNING": "\033[33m",  # Yellow
2025-06-01T21:03:59.2799435Z         "ERROR": "\033[31m",  # Red
2025-06-01T21:03:59.2799992Z         "CRITICAL": "\033[31m",  # Red
2025-06-01T21:03:59.2800558Z         "INFO": "\033[0m",  # Reset
2025-06-01T21:03:59.2801113Z         "DEBUG": "\033[0m",  # Reset
2025-06-01T21:03:59.2801639Z     }
2025-06-01T21:03:59.2802030Z 
2025-06-01T21:03:59.2802494Z     def format(self, record: LogRecord) -> str:
2025-06-01T21:03:59.2803441Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2025-06-01T21:03:59.2804296Z         record.msg = f"{log_color}{record.msg}\033[0m"
2025-06-01T21:03:59.2804938Z         return super().format(record)
2025-06-01T21:03:59.2805487Z 
2025-06-01T21:03:59.2805852Z 
2025-06-01T21:03:59.2806275Z handler = logging.StreamHandler()
2025-06-01T21:03:59.2807074Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2025-06-01T21:03:59.2807846Z 
2025-06-01T21:03:59.2808340Z log = logging.getLogger(os.path.basename(__file__))
2025-06-01T21:03:59.2808990Z log.addHandler(handler)
2025-06-01T21:03:59.2809506Z log.setLevel(logging.INFO)
2025-06-01T21:03:59.2810001Z 
2025-06-01T21:03:59.2810362Z 
2025-06-01T21:03:59.2810868Z def set_github_output(key: str, value: str) -> None:
2025-06-01T21:03:59.2811495Z     """
2025-06-01T21:03:59.2812076Z     Defines outputs of the github action that invokes this script
2025-06-01T21:03:59.2812766Z     """
2025-06-01T21:03:59.2813279Z     if not GITHUB_OUTPUT:
2025-06-01T21:03:59.2814462Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2025-06-01T21:03:59.2815668Z         log.warning(
2025-06-01T21:03:59.2816629Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2025-06-01T21:03:59.2817624Z         )
2025-06-01T21:03:59.2818121Z         print(f"::set-output name={key}::{value}")
2025-06-01T21:03:59.2818727Z         return
2025-06-01T21:03:59.2819159Z 
2025-06-01T21:03:59.2819721Z     with open(GITHUB_OUTPUT, "a") as f:
2025-06-01T21:03:59.2820362Z         log.info(f"Setting output: {key}='{value}'")
2025-06-01T21:03:59.2820992Z         f.write(f"{key}={value}\n")
2025-06-01T21:03:59.2821528Z 
2025-06-01T21:03:59.2821899Z 
2025-06-01T21:03:59.2822460Z def _str_comma_separated_to_set(value: str) -> frozenset[str]:
2025-06-01T21:03:59.2823372Z     return frozenset(
2025-06-01T21:03:59.2824105Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2025-06-01T21:03:59.2824841Z     )
2025-06-01T21:03:59.2825255Z 
2025-06-01T21:03:59.2825626Z 
2025-06-01T21:03:59.2826022Z def parse_args() -> Any:
2025-06-01T21:03:59.2826699Z     parser = ArgumentParser("Get dynamic rollout settings")
2025-06-01T21:03:59.2827659Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2025-06-01T21:03:59.2828498Z     parser.add_argument(
2025-06-01T21:03:59.2829026Z         "--github-issue-repo",
2025-06-01T21:03:59.2829566Z         type=str,
2025-06-01T21:03:59.2830048Z         required=False,
2025-06-01T21:03:59.2830711Z         default="pytorch/test-infra",
2025-06-01T21:03:59.2831346Z         help="GitHub repo to get the issue",
2025-06-01T21:03:59.2831907Z     )
2025-06-01T21:03:59.2832326Z     parser.add_argument(
2025-06-01T21:03:59.2832836Z         "--github-repo",
2025-06-01T21:03:59.2833444Z         type=str,
2025-06-01T21:03:59.2833921Z         required=True,
2025-06-01T21:03:59.2834471Z         help="GitHub repo where CI is running",
2025-06-01T21:03:59.2835042Z     )
2025-06-01T21:03:59.2835458Z     parser.add_argument(
2025-06-01T21:03:59.2836163Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2025-06-01T21:03:59.2836879Z     )
2025-06-01T21:03:59.2837293Z     parser.add_argument(
2025-06-01T21:03:59.2838012Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2025-06-01T21:03:59.2838761Z     )
2025-06-01T21:03:59.2839180Z     parser.add_argument(
2025-06-01T21:03:59.2839912Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2025-06-01T21:03:59.2840657Z     )
2025-06-01T21:03:59.2841073Z     parser.add_argument(
2025-06-01T21:03:59.2841837Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2025-06-01T21:03:59.2842594Z     )
2025-06-01T21:03:59.2843042Z     parser.add_argument(
2025-06-01T21:03:59.2843670Z         "--github-ref-type",
2025-06-01T21:03:59.2844194Z         type=str,
2025-06-01T21:03:59.2844672Z         required=True,
2025-06-01T21:03:59.2845261Z         help="Current GitHub ref type, branch or tag",
2025-06-01T21:03:59.2845860Z     )
2025-06-01T21:03:59.2846276Z     parser.add_argument(
2025-06-01T21:03:59.2846818Z         "--eligible-experiments",
2025-06-01T21:03:59.2847413Z         type=_str_comma_separated_to_set,
2025-06-01T21:03:59.2847994Z         required=False,
2025-06-01T21:03:59.2848490Z         default="",
2025-06-01T21:03:59.2849442Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2025-06-01T21:03:59.2850436Z     )
2025-06-01T21:03:59.2850860Z     parser.add_argument(
2025-06-01T21:03:59.2851391Z         "--opt-out-experiments",
2025-06-01T21:03:59.2851971Z         type=_str_comma_separated_to_set,
2025-06-01T21:03:59.2852550Z         required=False,
2025-06-01T21:03:59.2853049Z         default="",
2025-06-01T21:03:59.2853842Z         help=(
2025-06-01T21:03:59.2854615Z             "comma separated list of experiments to opt-out of. If unset, no opt-outs will occur. "
2025-06-01T21:03:59.2855842Z             "If the same experiment is listed both here and in '--eligible-experiments' opt-out will take priority."
2025-06-01T21:03:59.2856744Z         ),
2025-06-01T21:03:59.2857152Z     )
2025-06-01T21:03:59.2857572Z     parser.add_argument(
2025-06-01T21:03:59.2858084Z         "--pr-number",
2025-06-01T21:03:59.2858576Z         type=str,
2025-06-01T21:03:59.2859054Z         required=False,
2025-06-01T21:03:59.2859544Z         default="",
2025-06-01T21:03:59.2860109Z         help="the optional PR number where this is run",
2025-06-01T21:03:59.2860715Z     )
2025-06-01T21:03:59.2861104Z 
2025-06-01T21:03:59.2861516Z     return parser.parse_args()
2025-06-01T21:03:59.2862043Z 
2025-06-01T21:03:59.2862407Z 
2025-06-01T21:03:59.2863062Z def get_gh_client(github_token: str) -> Github:  # type: ignore[no-any-unimported]
2025-06-01T21:03:59.2864543Z     auth = Auth.Token(github_token)
2025-06-01T21:03:59.2865170Z     return Github(auth=auth)
2025-06-01T21:03:59.2865675Z 
2025-06-01T21:03:59.2866048Z 
2025-06-01T21:03:59.2866764Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:  # type: ignore[no-any-unimported]
2025-06-01T21:03:59.2867643Z     repo = gh.get_repo(repo)
2025-06-01T21:03:59.2868229Z     return repo.get_issue(number=issue_num)
2025-06-01T21:03:59.2868804Z 
2025-06-01T21:03:59.2869178Z 
2025-06-01T21:03:59.2869577Z def get_potential_pr_author(
2025-06-01T21:03:59.2870326Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2025-06-01T21:03:59.2871057Z ) -> str:
2025-06-01T21:03:59.2871658Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2025-06-01T21:03:59.2872546Z     # Fetch the actual username from the original PR. The PR number is
2025-06-01T21:03:59.2873522Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2025-06-01T21:03:59.2874158Z 
2025-06-01T21:03:59.2874581Z     gh = get_gh_client(github_token)
2025-06-01T21:03:59.2875117Z 
2025-06-01T21:03:59.2875631Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2025-06-01T21:03:59.2876338Z         split_tag = ref_name.split("/")
2025-06-01T21:03:59.2876887Z         if (
2025-06-01T21:03:59.2877330Z             len(split_tag) == 3
2025-06-01T21:03:59.2877897Z             and split_tag[0] == "ciflow"
2025-06-01T21:03:59.2878489Z             and split_tag[2].isnumeric()
2025-06-01T21:03:59.2879037Z         ):
2025-06-01T21:03:59.2879506Z             pr_number = split_tag[2]
2025-06-01T21:03:59.2880057Z             try:
2025-06-01T21:03:59.2880566Z                 repository = gh.get_repo(repo)
2025-06-01T21:03:59.2881266Z                 pull = repository.get_pull(number=int(pr_number))
2025-06-01T21:03:59.2881943Z             except Exception as e:
2025-06-01T21:03:59.2882541Z                 raise Exception(  # noqa: TRY002
2025-06-01T21:03:59.2883394Z                     f"issue with pull request {pr_number} from repo {repository}"
2025-06-01T21:03:59.2884120Z                 ) from e
2025-06-01T21:03:59.2884760Z             return pull.user.login  # type: ignore[no-any-return]
2025-06-01T21:03:59.2885572Z     # In all other cases, return the original input username
2025-06-01T21:03:59.2886222Z     return username
2025-06-01T21:03:59.2886687Z 
2025-06-01T21:03:59.2887054Z 
2025-06-01T21:03:59.2887696Z def is_exception_branch(branch: str) -> bool:
2025-06-01T21:03:59.2888286Z     """
2025-06-01T21:03:59.2889038Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2025-06-01T21:03:59.2889883Z     """
2025-06-01T21:03:59.2890505Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2025-06-01T21:03:59.2891224Z 
2025-06-01T21:03:59.2891591Z 
2025-06-01T21:03:59.2892020Z def load_yaml(yaml_text: str) -> Any:
2025-06-01T21:03:59.2892567Z     try:
2025-06-01T21:03:59.2893025Z         data = yaml.safe_load(yaml_text)
2025-06-01T21:03:59.2893695Z         return data
2025-06-01T21:03:59.2894197Z     except yaml.YAMLError:
2025-06-01T21:03:59.2894774Z         log.exception("Error loading YAML")
2025-06-01T21:03:59.2895338Z         raise
2025-06-01T21:03:59.2895779Z 
2025-06-01T21:03:59.2896150Z 
2025-06-01T21:03:59.2896820Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> tuple[str, str]:
2025-06-01T21:03:59.2897625Z     """
2025-06-01T21:03:59.2898476Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2025-06-01T21:03:59.2899299Z 
2025-06-01T21:03:59.2899891Z     If the issue body contains "---" then the text above that is the settings
2025-06-01T21:03:59.2900747Z     and the text below is the list of opted in users.
2025-06-01T21:03:59.2901355Z 
2025-06-01T21:03:59.2901984Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2025-06-01T21:03:59.2902740Z     """
2025-06-01T21:03:59.2903359Z     rollout_state_parts = rollout_state.split("---")
2025-06-01T21:03:59.2904028Z     if len(rollout_state_parts) >= 2:
2025-06-01T21:03:59.2904793Z         return rollout_state_parts[0], rollout_state_parts[1]
2025-06-01T21:03:59.2905610Z     else:
2025-06-01T21:03:59.2906051Z         return "", rollout_state
2025-06-01T21:03:59.2906564Z 
2025-06-01T21:03:59.2906929Z 
2025-06-01T21:03:59.2907369Z class UserOptins(dict[str, list[str]]):
2025-06-01T21:03:59.2907932Z     """
2025-06-01T21:03:59.2908532Z     Dictionary of users with a list of features they have opted into
2025-06-01T21:03:59.2909239Z     """
2025-06-01T21:03:59.2909629Z 
2025-06-01T21:03:59.2909993Z 
2025-06-01T21:03:59.2910572Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2025-06-01T21:03:59.2911290Z     """
2025-06-01T21:03:59.2912088Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2025-06-01T21:03:59.2912980Z 
2025-06-01T21:03:59.2913963Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2025-06-01T21:03:59.2915059Z         - Example line: "@User1,lf,split_build"
2025-06-01T21:03:59.2915820Z         - A "#" prefix indicates the user is opted out of all experiments
2025-06-01T21:03:59.2916508Z 
2025-06-01T21:03:59.2916876Z 
2025-06-01T21:03:59.2917241Z     """
2025-06-01T21:03:59.2917665Z     optins = UserOptins()
2025-06-01T21:03:59.2918241Z     for user in user_optin_text.split("\n"):
2025-06-01T21:03:59.2918859Z         user = user.strip("\r\n\t -")
2025-06-01T21:03:59.2919478Z         if not user or not user.startswith("@"):
2025-06-01T21:03:59.2920095Z             # Not a valid user. Skip
2025-06-01T21:03:59.2920648Z             continue
2025-06-01T21:03:59.2921316Z 
2025-06-01T21:03:59.2921887Z         if user:
2025-06-01T21:03:59.2922577Z             usr_name = user.split(",")[0].strip("@")
2025-06-01T21:03:59.2923471Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2025-06-01T21:03:59.2924163Z 
2025-06-01T21:03:59.2924550Z     return optins
2025-06-01T21:03:59.2924995Z 
2025-06-01T21:03:59.2925347Z 
2025-06-01T21:03:59.2925885Z def is_valid_experiment_name(experiment_name: str) -> bool:
2025-06-01T21:03:59.2926641Z     """
2025-06-01T21:03:59.2927106Z     Check if the experiment name is valid.
2025-06-01T21:03:59.2927682Z     A valid name:
2025-06-01T21:03:59.2928411Z         - Contains only alphanumeric characters and the special characters "_" & "-"
2025-06-01T21:03:59.2929432Z         - The special characters "_" & "-" shouldn't be the first or last characters
2025-06-01T21:03:59.2930621Z         - Cannot contain spaces
2025-06-01T21:03:59.2931159Z     """
2025-06-01T21:03:59.2931554Z 
2025-06-01T21:03:59.2932133Z     valid_char_regex = r"^[a-zA-Z0-9]([\w-]*[a-zA-Z0-9])?$"
2025-06-01T21:03:59.2933390Z     valid = bool(re.match(valid_char_regex, experiment_name))
2025-06-01T21:03:59.2934284Z 
2025-06-01T21:03:59.2934784Z     if valid:
2025-06-01T21:03:59.2935227Z         return True
2025-06-01T21:03:59.2935677Z 
2025-06-01T21:03:59.2936050Z     log.error(
2025-06-01T21:03:59.2937638Z         f"Invalid experiment name: {experiment_name}. Experiment names should only contain alphanumeric characters, '_', and '-'. They cannot contain spaces, and the special characters '_' and '-' cannot be the first or last characters."
2025-06-01T21:03:59.2939302Z     )
2025-06-01T21:03:59.2939697Z     return False
2025-06-01T21:03:59.2940143Z 
2025-06-01T21:03:59.2940500Z 
2025-06-01T21:03:59.2941059Z def parse_settings_from_text(settings_text: str) -> Settings:
2025-06-01T21:03:59.2941752Z     """
2025-06-01T21:03:59.2942416Z     Parse the experiments from the issue body into a list of ExperimentSettings
2025-06-01T21:03:59.2943351Z     """
2025-06-01T21:03:59.2944036Z     try:
2025-06-01T21:03:59.2944767Z         if settings_text:
2025-06-01T21:03:59.2946322Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2025-06-01T21:03:59.2947942Z             # for easy reading
2025-06-01T21:03:59.2949365Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2025-06-01T21:03:59.2950903Z             #       the backtick character in shell commands.
2025-06-01T21:03:59.2951951Z             backtick = chr(96)  # backtick character
2025-06-01T21:03:59.2953160Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2025-06-01T21:03:59.2954711Z             settings = load_yaml(settings_text)
2025-06-01T21:03:59.2955305Z 
2025-06-01T21:03:59.2955975Z             # For now we just load experiments. We can expand this if/when we add more settings
2025-06-01T21:03:59.2956785Z             experiments = {}
2025-06-01T21:03:59.2957276Z 
2025-06-01T21:03:59.2957886Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2025-06-01T21:03:59.2958704Z                 if not is_valid_experiment_name(exp_name):
2025-06-01T21:03:59.2959906Z                     # Exclude invalid experiments from the list. We log an error, but don't raise an exception so that other experiments can still be processed.
2025-06-01T21:03:59.2961016Z                     continue
2025-06-01T21:03:59.2961514Z 
2025-06-01T21:03:59.2961919Z                 valid_settings = {}
2025-06-01T21:03:59.2962682Z                 for setting in exp_settings:
2025-06-01T21:03:59.2963496Z                     if setting not in Experiment._fields:
2025-06-01T21:03:59.2964149Z                         log.warning(
2025-06-01T21:03:59.2964943Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2025-06-01T21:03:59.2965711Z                         )
2025-06-01T21:03:59.2966198Z                     else:
2025-06-01T21:03:59.2966798Z                         valid_settings[setting] = exp_settings[setting]
2025-06-01T21:03:59.2967410Z 
2025-06-01T21:03:59.2967927Z                 experiments[exp_name] = Experiment(**valid_settings)
2025-06-01T21:03:59.2968622Z             return Settings(experiments)
2025-06-01T21:03:59.2969161Z 
2025-06-01T21:03:59.2969546Z     except Exception:
2025-06-01T21:03:59.2970103Z         log.exception("Failed to parse settings")
2025-06-01T21:03:59.2970685Z 
2025-06-01T21:03:59.2971062Z     return Settings()
2025-06-01T21:03:59.2971521Z 
2025-06-01T21:03:59.2971868Z 
2025-06-01T21:03:59.2972492Z def parse_settings(rollout_state: str) -> Settings:
2025-06-01T21:03:59.2973118Z     """
2025-06-01T21:03:59.2974106Z     Parse settings, if any, from the rollout state.
2025-06-01T21:03:59.2974800Z 
2025-06-01T21:03:59.2975399Z     If the issue body contains "---" then the text above that is the settings
2025-06-01T21:03:59.2976228Z     and the text below is the list of opted in users.
2025-06-01T21:03:59.2976822Z 
2025-06-01T21:03:59.2977477Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2025-06-01T21:03:59.2978251Z     """
2025-06-01T21:03:59.2978884Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2025-06-01T21:03:59.2979727Z     return parse_settings_from_text(settings_text)
2025-06-01T21:03:59.2980315Z 
2025-06-01T21:03:59.2980682Z 
2025-06-01T21:03:59.2981175Z def parse_users(rollout_state: str) -> UserOptins:
2025-06-01T21:03:59.2981795Z     """
2025-06-01T21:03:59.2982238Z     Parse users from the rollout state.
2025-06-01T21:03:59.2982788Z 
2025-06-01T21:03:59.2983145Z     """
2025-06-01T21:03:59.2983876Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2025-06-01T21:03:59.2984699Z     return parse_user_opt_in_from_text(users_text)
2025-06-01T21:03:59.2985283Z 
2025-06-01T21:03:59.2985653Z 
2025-06-01T21:03:59.2986325Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-06-01T21:03:59.2987126Z     """
2025-06-01T21:03:59.2987611Z     Check if a user is opted into an experiment
2025-06-01T21:03:59.2988198Z     """
2025-06-01T21:03:59.2988715Z     return experiment_name in user_optins.get(user, [])
2025-06-01T21:03:59.2989338Z 
2025-06-01T21:03:59.2989705Z 
2025-06-01T21:03:59.2990380Z def is_user_opted_out(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-06-01T21:03:59.2991188Z     """
2025-06-01T21:03:59.2991710Z     Check if a user explicitly opted out of an experiment
2025-06-01T21:03:59.2992336Z     """
2025-06-01T21:03:59.2992903Z     # if the experiment is prefixed with a "-", then it's an opt-out
2025-06-01T21:03:59.2993768Z     experiment_optout = "-" + experiment_name
2025-06-01T21:03:59.2994470Z     if experiment_optout not in user_optins.get(user, []):
2025-06-01T21:03:59.2995119Z         return False
2025-06-01T21:03:59.2995585Z 
2025-06-01T21:03:59.2996089Z     if is_user_opted_in(user, user_optins, experiment_name):
2025-06-01T21:03:59.2996910Z         log.warning(
2025-06-01T21:03:59.2997821Z             f"User {user} is opted into experiment {experiment_name}, but also opted out of it. Defaulting to opting out"
2025-06-01T21:03:59.2998752Z         )
2025-06-01T21:03:59.2999157Z 
2025-06-01T21:03:59.2999531Z     return True
2025-06-01T21:03:59.2999967Z 
2025-06-01T21:03:59.3000318Z 
2025-06-01T21:03:59.3000705Z def get_runner_prefix(
2025-06-01T21:03:59.3001207Z     rollout_state: str,
2025-06-01T21:03:59.3001745Z     workflow_requestors: Iterable[str],
2025-06-01T21:03:59.3002305Z     branch: str,
2025-06-01T21:03:59.3002883Z     eligible_experiments: frozenset[str] = frozenset(),
2025-06-01T21:03:59.3003737Z     opt_out_experiments: frozenset[str] = frozenset(),
2025-06-01T21:03:59.3004375Z     is_canary: bool = False,
2025-06-01T21:03:59.3004893Z ) -> str:
2025-06-01T21:03:59.3005367Z     settings = parse_settings(rollout_state)
2025-06-01T21:03:59.3006019Z     user_optins = parse_users(rollout_state)
2025-06-01T21:03:59.3006585Z 
2025-06-01T21:03:59.3007097Z     fleet_prefix = ""
2025-06-01T21:03:59.3007583Z     prefixes = []
2025-06-01T21:03:59.3008301Z     for experiment_name, experiment_settings in settings.experiments.items():
2025-06-01T21:03:59.3009321Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2025-06-01T21:03:59.3010078Z             log.info(
2025-06-01T21:03:59.3010839Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2025-06-01T21:03:59.3011631Z             )
2025-06-01T21:03:59.3012067Z             continue
2025-06-01T21:03:59.3012523Z 
2025-06-01T21:03:59.3012921Z         if opt_out_experiments:
2025-06-01T21:03:59.3013637Z             if experiment_name in opt_out_experiments:
2025-06-01T21:03:59.3014354Z                 opt_out_exp_list = ", ".join(opt_out_experiments)
2025-06-01T21:03:59.3015000Z                 log.info(
2025-06-01T21:03:59.3016009Z                     f"Skipping experiment '{experiment_name}', as this workflow has opted-out (opted out experiments are: {opt_out_exp_list})"
2025-06-01T21:03:59.3017035Z                 )
2025-06-01T21:03:59.3017506Z                 continue
2025-06-01T21:03:59.3017973Z 
2025-06-01T21:03:59.3018379Z         if eligible_experiments:
2025-06-01T21:03:59.3019011Z             if experiment_name not in eligible_experiments:
2025-06-01T21:03:59.3019709Z                 exp_list = ", ".join(eligible_experiments)
2025-06-01T21:03:59.3020301Z                 log.info(
2025-06-01T21:03:59.3021162Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2025-06-01T21:03:59.3022059Z                 )
2025-06-01T21:03:59.3022513Z                 continue
2025-06-01T21:03:59.3023076Z         elif not experiment_settings.default:
2025-06-01T21:03:59.3023861Z             log.info(
2025-06-01T21:03:59.3024616Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2025-06-01T21:03:59.3025401Z             )
2025-06-01T21:03:59.3025838Z             continue
2025-06-01T21:03:59.3026293Z 
2025-06-01T21:03:59.3026811Z         # Is any workflow_requestor opted out to this experiment?
2025-06-01T21:03:59.3027478Z         opted_out_users = [
2025-06-01T21:03:59.3027991Z             requestor
2025-06-01T21:03:59.3028565Z             for requestor in workflow_requestors
2025-06-01T21:03:59.3029422Z             if is_user_opted_out(requestor, user_optins, experiment_name)
2025-06-01T21:03:59.3030097Z         ]
2025-06-01T21:03:59.3030495Z 
2025-06-01T21:03:59.3030877Z         if opted_out_users:
2025-06-01T21:03:59.3031417Z             log.info(
2025-06-01T21:03:59.3032136Z                 f"{', '.join(opted_out_users)} have opted out of experiment {experiment_name}."
2025-06-01T21:03:59.3032886Z             )
2025-06-01T21:03:59.3033419Z             continue
2025-06-01T21:03:59.3033877Z 
2025-06-01T21:03:59.3034383Z         # Is any workflow_requestor opted in to this experiment?
2025-06-01T21:03:59.3035036Z         opted_in_users = [
2025-06-01T21:03:59.3035562Z             requestor
2025-06-01T21:03:59.3036116Z             for requestor in workflow_requestors
2025-06-01T21:03:59.3036857Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2025-06-01T21:03:59.3037533Z         ]
2025-06-01T21:03:59.3037935Z 
2025-06-01T21:03:59.3038328Z         enabled = False
2025-06-01T21:03:59.3038842Z         if opted_in_users:
2025-06-01T21:03:59.3039475Z             log.info(
2025-06-01T21:03:59.3040179Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2025-06-01T21:03:59.3040911Z             )
2025-06-01T21:03:59.3041359Z             enabled = True
2025-06-01T21:03:59.3041851Z 
2025-06-01T21:03:59.3042294Z         elif experiment_settings.rollout_perc:
2025-06-01T21:03:59.3043316Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2025-06-01T21:03:59.3044365Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2025-06-01T21:03:59.3045086Z                 log.info(
2025-06-01T21:03:59.3046071Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2025-06-01T21:03:59.3047051Z                 )
2025-06-01T21:03:59.3047532Z                 enabled = True
2025-06-01T21:03:59.3048046Z 
2025-06-01T21:03:59.3048428Z         if enabled:
2025-06-01T21:03:59.3048927Z             label = experiment_name
2025-06-01T21:03:59.3049536Z             if experiment_name == LF_FLEET_EXPERIMENT:
2025-06-01T21:03:59.3050432Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2025-06-01T21:03:59.3051371Z                 #  - If it's enabled, then we always list it's prefix first
2025-06-01T21:03:59.3052199Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2025-06-01T21:03:59.3052915Z                 if is_canary:
2025-06-01T21:03:59.3053578Z                     label += CANARY_FLEET_SUFFIX
2025-06-01T21:03:59.3054183Z                 fleet_prefix = label
2025-06-01T21:03:59.3054718Z             else:
2025-06-01T21:03:59.3055229Z                 prefixes.append(label)
2025-06-01T21:03:59.3055776Z 
2025-06-01T21:03:59.3056165Z     if len(prefixes) > 1:
2025-06-01T21:03:59.3056671Z         log.error(
2025-06-01T21:03:59.3057804Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2025-06-01T21:03:59.3058988Z         )
2025-06-01T21:03:59.3059428Z         prefixes = prefixes[:1]
2025-06-01T21:03:59.3059955Z 
2025-06-01T21:03:59.3060361Z     # Fleet always comes first
2025-06-01T21:03:59.3060896Z     if fleet_prefix:
2025-06-01T21:03:59.3061424Z         prefixes.insert(0, fleet_prefix)
2025-06-01T21:03:59.3062092Z 
2025-06-01T21:03:59.3062583Z     return ".".join(prefixes) + "." if prefixes else ""
2025-06-01T21:03:59.3063292Z 
2025-06-01T21:03:59.3063666Z 
2025-06-01T21:03:59.3064384Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2025-06-01T21:03:59.3065206Z     """
2025-06-01T21:03:59.3065868Z     Gets the first comment of the issue, which contains the desired rollout state.
2025-06-01T21:03:59.3066622Z 
2025-06-01T21:03:59.3067243Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2025-06-01T21:03:59.3067990Z     """
2025-06-01T21:03:59.3068435Z     gh = get_gh_client(github_token)
2025-06-01T21:03:59.3069038Z     issue = get_issue(gh, repo, issue_num)
2025-06-01T21:03:59.3069750Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2025-06-01T21:03:59.3070397Z 
2025-06-01T21:03:59.3070760Z 
2025-06-01T21:03:59.3071405Z def download_json(url: str, headers: dict[str, str], num_retries: int = 3) -> Any:
2025-06-01T21:03:59.3072558Z     for _ in range(num_retries):
2025-06-01T21:03:59.3073106Z         try:
2025-06-01T21:03:59.3073721Z             req = Request(url=url, headers=headers)
2025-06-01T21:03:59.3074447Z             content = urlopen(req, timeout=5).read().decode("utf-8")
2025-06-01T21:03:59.3075159Z             return json.loads(content)
2025-06-01T21:03:59.3075747Z         except Exception as e:
2025-06-01T21:03:59.3076368Z             log.warning(f"Could not download {url}: {e}")
2025-06-01T21:03:59.3076959Z 
2025-06-01T21:03:59.3077581Z     log.warning(f"All {num_retries} retries exhausted, downloading {url} failed")
2025-06-01T21:03:59.3078343Z     return {}
2025-06-01T21:03:59.3078779Z 
2025-06-01T21:03:59.3079135Z 
2025-06-01T21:03:59.3079501Z @cache
2025-06-01T21:03:59.3080201Z def get_pr_info(github_repo: str, github_token: str, pr_number: int) -> dict[str, Any]:
2025-06-01T21:03:59.3081007Z     """
2025-06-01T21:03:59.3081455Z     Dynamically get PR information
2025-06-01T21:03:59.3081990Z     """
2025-06-01T21:03:59.3082549Z     github_api = f"https://api.github.com/repos/{github_repo}"
2025-06-01T21:03:59.3083320Z     headers = {
2025-06-01T21:03:59.3083862Z         "Accept": "application/vnd.github.v3+json",
2025-06-01T21:03:59.3084531Z         "Authorization": f"token {github_token}",
2025-06-01T21:03:59.3085101Z     }
2025-06-01T21:03:59.3085633Z     json_response: dict[str, Any] = download_json(
2025-06-01T21:03:59.3086306Z         url=f"{github_api}/issues/{pr_number}",
2025-06-01T21:03:59.3086900Z         headers=headers,
2025-06-01T21:03:59.3087391Z     )
2025-06-01T21:03:59.3087779Z 
2025-06-01T21:03:59.3088170Z     if not json_response:
2025-06-01T21:03:59.3088831Z         log.warning(f"Failed to get the labels for #{pr_number}")
2025-06-01T21:03:59.3089517Z         return {}
2025-06-01T21:03:59.3089961Z 
2025-06-01T21:03:59.3090358Z     return json_response
2025-06-01T21:03:59.3090833Z 
2025-06-01T21:03:59.3091194Z 
2025-06-01T21:03:59.3091829Z def get_labels(github_repo: str, github_token: str, pr_number: int) -> set[str]:
2025-06-01T21:03:59.3092609Z     """
2025-06-01T21:03:59.3093306Z     Dynamically get the latest list of labels from the pull request
2025-06-01T21:03:59.3094012Z     """
2025-06-01T21:03:59.3094556Z     pr_info = get_pr_info(github_repo, github_token, pr_number)
2025-06-01T21:03:59.3095208Z     return {
2025-06-01T21:03:59.3095994Z         label.get("name") for label in pr_info.get("labels", []) if label.get("name")
2025-06-01T21:03:59.3096724Z     }
2025-06-01T21:03:59.3097115Z 
2025-06-01T21:03:59.3097475Z 
2025-06-01T21:03:59.3097861Z def main() -> None:
2025-06-01T21:03:59.3098347Z     args = parse_args()
2025-06-01T21:03:59.3098824Z 
2025-06-01T21:03:59.3099281Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2025-06-01T21:03:59.3099859Z 
2025-06-01T21:03:59.3100267Z     # Check if the PR is opt-out
2025-06-01T21:03:59.3100802Z     if args.pr_number:
2025-06-01T21:03:59.3101547Z         labels = get_labels(args.github_repo, args.github_token, int(args.pr_number))
2025-06-01T21:03:59.3102357Z         if OPT_OUT_LABEL in labels:
2025-06-01T21:03:59.3102912Z             log.info(
2025-06-01T21:03:59.3103799Z                 f"Opt-out runner determinator because #{args.pr_number} has {OPT_OUT_LABEL} label"
2025-06-01T21:03:59.3104615Z             )
2025-06-01T21:03:59.3105245Z             set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-06-01T21:03:59.3105980Z             sys.exit()
2025-06-01T21:03:59.3106575Z 
2025-06-01T21:03:59.3106948Z     try:
2025-06-01T21:03:59.3107448Z         rollout_state = get_rollout_state_from_issue(
2025-06-01T21:03:59.3108237Z             args.github_token, args.github_issue_repo, args.github_issue
2025-06-01T21:03:59.3108919Z         )
2025-06-01T21:03:59.3109326Z 
2025-06-01T21:03:59.3109753Z         username = get_potential_pr_author(
2025-06-01T21:03:59.3110344Z             args.github_token,
2025-06-01T21:03:59.3110882Z             args.github_repo,
2025-06-01T21:03:59.3111422Z             args.github_actor,
2025-06-01T21:03:59.3111975Z             args.github_ref_type,
2025-06-01T21:03:59.3112535Z             args.github_branch,
2025-06-01T21:03:59.3113057Z         )
2025-06-01T21:03:59.3113559Z 
2025-06-01T21:03:59.3114089Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2025-06-01T21:03:59.3114738Z 
2025-06-01T21:03:59.3115189Z         runner_label_prefix = get_runner_prefix(
2025-06-01T21:03:59.3115790Z             rollout_state,
2025-06-01T21:03:59.3116358Z             (args.github_issue_owner, username),
2025-06-01T21:03:59.3116954Z             args.github_branch,
2025-06-01T21:03:59.3117515Z             args.eligible_experiments,
2025-06-01T21:03:59.3118111Z             args.opt_out_experiments,
2025-06-01T21:03:59.3118661Z             is_canary,
2025-06-01T21:03:59.3119129Z         )
2025-06-01T21:03:59.3119524Z 
2025-06-01T21:03:59.3119928Z     except Exception as e:
2025-06-01T21:03:59.3120448Z         log.error(
2025-06-01T21:03:59.3121207Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2025-06-01T21:03:59.3122027Z         )
2025-06-01T21:03:59.3122431Z 
2025-06-01T21:03:59.3123003Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-06-01T21:03:59.3123804Z 
2025-06-01T21:03:59.3124171Z 
2025-06-01T21:03:59.3124554Z if __name__ == "__main__":
2025-06-01T21:03:59.3125050Z     main()
2025-06-01T21:03:59.3125458Z 
2025-06-01T21:03:59.3125823Z EOF
2025-06-01T21:03:59.3126201Z 
2025-06-01T21:03:59.3126601Z cat runner_determinator.py
2025-06-01T21:03:59.3539791Z shell: /usr/bin/bash -e {0}
2025-06-01T21:03:59.3540621Z env:
2025-06-01T21:03:59.3541390Z   GITHUB_TOKEN: ***
2025-06-01T21:03:59.3541823Z   ISSUE_NUMBER: 5132
2025-06-01T21:03:59.3542272Z   TRIGGERING_ACTOR: pytorchmergebot
2025-06-01T21:03:59.3543010Z   ISSUE_OWNER: 
2025-06-01T21:03:59.3543585Z   CHECK_EXPERIMENTS: 
2025-06-01T21:03:59.3544029Z   OPT_OUT_EXPERIMENTS: 
2025-06-01T21:03:59.3544460Z   PR_NUMBER: 
2025-06-01T21:03:59.3544846Z ##[endgroup]
2025-06-01T21:03:59.3766997Z # flake8: noqa: G004
2025-06-01T21:03:59.3767360Z 
2025-06-01T21:03:59.3767800Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2025-06-01T21:03:59.3768777Z #       must be kept in sync. You can do it easily by running the following command:
2025-06-01T21:03:59.3769602Z #           python .github/scripts/update_runner_determinator.py
2025-06-01T21:03:59.3770049Z 
2025-06-01T21:03:59.3770216Z """
2025-06-01T21:03:59.3770803Z This runner determinator is used to determine which set of runners to run a
2025-06-01T21:03:59.3771701Z GitHub job on. It uses the first comment of a GitHub issue (by default
2025-06-01T21:03:59.3772608Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2025-06-01T21:03:59.3773666Z of which runners should be used to run which job.
2025-06-01T21:03:59.3774079Z 
2025-06-01T21:03:59.3774472Z The configuration has two parts, the settings and a list of opted-in users,
2025-06-01T21:03:59.3775582Z separated by a line containing "---".  If the line is not present, the
2025-06-01T21:03:59.3776511Z settings are considered to be empty with only the second part, the user
2025-06-01T21:03:59.3777218Z list, defined.
2025-06-01T21:03:59.3777453Z 
2025-06-01T21:03:59.3777820Z The first part is a YAML block that defines the rollout settings. This can be
2025-06-01T21:03:59.3778752Z used to define any settings that are needed to determine which runners to use.
2025-06-01T21:03:59.3779603Z It's fields are defined by the RolloutSettings class below.
2025-06-01T21:03:59.3780050Z 
2025-06-01T21:03:59.3780430Z The second part is a list of users who are explicitly opted in to the LF fleet.
2025-06-01T21:03:59.3781303Z The user list is also a comma separated list of additional features or
2025-06-01T21:03:59.3782058Z experiments which the user could be opted in to.
2025-06-01T21:03:59.3782465Z 
2025-06-01T21:03:59.3782665Z The user list has the following rules:
2025-06-01T21:03:59.3783022Z 
2025-06-01T21:03:59.3783514Z - Users are GitHub usernames, which must start with the @ prefix
2025-06-01T21:03:59.3784395Z - Each user is also a comma-separated list of features/experiments to enable
2025-06-01T21:03:59.3785185Z - A "#" prefix opts the user out of all experiments
2025-06-01T21:03:59.3785582Z 
2025-06-01T21:03:59.3785758Z Example config:
2025-06-01T21:03:59.3786213Z     # A list of experiments that can be opted into.
2025-06-01T21:03:59.3786901Z     # This defines the behavior they'll induce when opted into.
2025-06-01T21:03:59.3787523Z     # Expected syntax is:
2025-06-01T21:03:59.3788190Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2025-06-01T21:03:59.3789189Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2025-06-01T21:03:59.3789826Z 
2025-06-01T21:03:59.3790001Z     experiments:
2025-06-01T21:03:59.3790404Z       lf:
2025-06-01T21:03:59.3790789Z         rollout_percent: 25
2025-06-01T21:03:59.3791262Z         all_branches: false
2025-06-01T21:03:59.3791719Z         default: true
2025-06-01T21:03:59.3792143Z     ---
2025-06-01T21:03:59.3792349Z 
2025-06-01T21:03:59.3792516Z     # Opt-ins:
2025-06-01T21:03:59.3793108Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2025-06-01T21:03:59.3794202Z     # and specifying experiments to enable in a comma-separated list.
2025-06-01T21:03:59.3794995Z     # To always opt out of an experiment, prefix it with a "-".
2025-06-01T21:03:59.3795663Z     # Experiments should be from the above list.
2025-06-01T21:03:59.3796051Z 
2025-06-01T21:03:59.3796235Z     @User1,-lf,split_build
2025-06-01T21:03:59.3796673Z     @User2,lf
2025-06-01T21:03:59.3797067Z     @User3,split_build
2025-06-01T21:03:59.3797483Z """
2025-06-01T21:03:59.3797869Z 
2025-06-01T21:03:59.3798040Z import json
2025-06-01T21:03:59.3798426Z import logging
2025-06-01T21:03:59.3798807Z import os
2025-06-01T21:03:59.3799186Z import random
2025-06-01T21:03:59.3799571Z import re
2025-06-01T21:03:59.3799941Z import sys
2025-06-01T21:03:59.3800359Z from argparse import ArgumentParser
2025-06-01T21:03:59.3800895Z from collections.abc import Iterable
2025-06-01T21:03:59.3801429Z from functools import cache
2025-06-01T21:03:59.3801906Z from logging import LogRecord
2025-06-01T21:03:59.3802408Z from typing import Any, NamedTuple
2025-06-01T21:03:59.3802993Z from urllib.request import Request, urlopen
2025-06-01T21:03:59.3803573Z 
2025-06-01T21:03:59.3803750Z import yaml
2025-06-01T21:03:59.3804146Z from github import Auth, Github
2025-06-01T21:03:59.3804653Z from github.Issue import Issue
2025-06-01T21:03:59.3804961Z 
2025-06-01T21:03:59.3804968Z 
2025-06-01T21:03:59.3805191Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2025-06-01T21:03:59.3805889Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2025-06-01T21:03:59.3806781Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2025-06-01T21:03:59.3807363Z 
2025-06-01T21:03:59.3807595Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2025-06-01T21:03:59.3808315Z GH_OUTPUT_KEY_AMI = "runner-ami"
2025-06-01T21:03:59.3808856Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2025-06-01T21:03:59.3809420Z OPT_OUT_LABEL = "no-runner-experiments"
2025-06-01T21:03:59.3809778Z 
2025-06-01T21:03:59.3809984Z SETTING_EXPERIMENTS = "experiments"
2025-06-01T21:03:59.3810324Z 
2025-06-01T21:03:59.3810516Z LF_FLEET_EXPERIMENT = "lf"
2025-06-01T21:03:59.3810993Z CANARY_FLEET_SUFFIX = ".c"
2025-06-01T21:03:59.3811278Z 
2025-06-01T21:03:59.3811284Z 
2025-06-01T21:03:59.3811473Z class Experiment(NamedTuple):
2025-06-01T21:03:59.3811966Z     rollout_perc: float = (
2025-06-01T21:03:59.3812601Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2025-06-01T21:03:59.3813570Z     )
2025-06-01T21:03:59.3813983Z     all_branches: bool = (
2025-06-01T21:03:59.3814638Z         False  # If True, the experiment is also enabled on the exception branches
2025-06-01T21:03:59.3815333Z     )
2025-06-01T21:03:59.3815701Z     default: bool = (
2025-06-01T21:03:59.3816293Z         True  # If True, the experiment is enabled by default for all queries
2025-06-01T21:03:59.3816949Z     )
2025-06-01T21:03:59.3817160Z 
2025-06-01T21:03:59.3817344Z     # Add more fields as needed
2025-06-01T21:03:59.3817649Z 
2025-06-01T21:03:59.3817655Z 
2025-06-01T21:03:59.3817853Z class Settings(NamedTuple):
2025-06-01T21:03:59.3818300Z     """
2025-06-01T21:03:59.3818767Z     Settings for the experiments that can be opted into.
2025-06-01T21:03:59.3819349Z     """
2025-06-01T21:03:59.3819552Z 
2025-06-01T21:03:59.3819771Z     experiments: dict[str, Experiment] = {}
2025-06-01T21:03:59.3820139Z 
2025-06-01T21:03:59.3820147Z 
2025-06-01T21:03:59.3820361Z class ColorFormatter(logging.Formatter):
2025-06-01T21:03:59.3820995Z     """Color codes the log messages based on the log level"""
2025-06-01T21:03:59.3821454Z 
2025-06-01T21:03:59.3821628Z     COLORS = {
2025-06-01T21:03:59.3822032Z         "WARNING": "\033[33m",  # Yellow
2025-06-01T21:03:59.3822545Z         "ERROR": "\033[31m",  # Red
2025-06-01T21:03:59.3823044Z         "CRITICAL": "\033[31m",  # Red
2025-06-01T21:03:59.3823759Z         "INFO": "\033[0m",  # Reset
2025-06-01T21:03:59.3824255Z         "DEBUG": "\033[0m",  # Reset
2025-06-01T21:03:59.3824737Z     }
2025-06-01T21:03:59.3824934Z 
2025-06-01T21:03:59.3825157Z     def format(self, record: LogRecord) -> str:
2025-06-01T21:03:59.3825945Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2025-06-01T21:03:59.3826753Z         record.msg = f"{log_color}{record.msg}\033[0m"
2025-06-01T21:03:59.3827346Z         return super().format(record)
2025-06-01T21:03:59.3827684Z 
2025-06-01T21:03:59.3827691Z 
2025-06-01T21:03:59.3827892Z handler = logging.StreamHandler()
2025-06-01T21:03:59.3828767Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2025-06-01T21:03:59.3829345Z 
2025-06-01T21:03:59.3829593Z log = logging.getLogger(os.path.basename(__file__))
2025-06-01T21:03:59.3830175Z log.addHandler(handler)
2025-06-01T21:03:59.3830640Z log.setLevel(logging.INFO)
2025-06-01T21:03:59.3830928Z 
2025-06-01T21:03:59.3830935Z 
2025-06-01T21:03:59.3831186Z def set_github_output(key: str, value: str) -> None:
2025-06-01T21:03:59.3831754Z     """
2025-06-01T21:03:59.3832265Z     Defines outputs of the github action that invokes this script
2025-06-01T21:03:59.3832903Z     """
2025-06-01T21:03:59.3833452Z     if not GITHUB_OUTPUT:
2025-06-01T21:03:59.3834553Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2025-06-01T21:03:59.3835700Z         log.warning(
2025-06-01T21:03:59.3836560Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2025-06-01T21:03:59.3837504Z         )
2025-06-01T21:03:59.3847570Z         print(f"::set-output name={key}::{value}")
2025-06-01T21:03:59.3848190Z         return
2025-06-01T21:03:59.3848438Z 
2025-06-01T21:03:59.3848824Z     with open(GITHUB_OUTPUT, "a") as f:
2025-06-01T21:03:59.3849457Z         log.info(f"Setting output: {key}='{value}'")
2025-06-01T21:03:59.3850059Z         f.write(f"{key}={value}\n")
2025-06-01T21:03:59.3850394Z 
2025-06-01T21:03:59.3850401Z 
2025-06-01T21:03:59.3850714Z def _str_comma_separated_to_set(value: str) -> frozenset[str]:
2025-06-01T21:03:59.3851366Z     return frozenset(
2025-06-01T21:03:59.3851988Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2025-06-01T21:03:59.3852682Z     )
2025-06-01T21:03:59.3852888Z 
2025-06-01T21:03:59.3852895Z 
2025-06-01T21:03:59.3853080Z def parse_args() -> Any:
2025-06-01T21:03:59.3853867Z     parser = ArgumentParser("Get dynamic rollout settings")
2025-06-01T21:03:59.3854771Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2025-06-01T21:03:59.3855603Z     parser.add_argument(
2025-06-01T21:03:59.3856071Z         "--github-issue-repo",
2025-06-01T21:03:59.3856553Z         type=str,
2025-06-01T21:03:59.3856973Z         required=False,
2025-06-01T21:03:59.3857449Z         default="pytorch/test-infra",
2025-06-01T21:03:59.3857997Z         help="GitHub repo to get the issue",
2025-06-01T21:03:59.3858519Z     )
2025-06-01T21:03:59.3858892Z     parser.add_argument(
2025-06-01T21:03:59.3859361Z         "--github-repo",
2025-06-01T21:03:59.3859793Z         type=str,
2025-06-01T21:03:59.3860199Z         required=True,
2025-06-01T21:03:59.3860661Z         help="GitHub repo where CI is running",
2025-06-01T21:03:59.3861209Z     )
2025-06-01T21:03:59.3861586Z     parser.add_argument(
2025-06-01T21:03:59.3862207Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2025-06-01T21:03:59.3862886Z     )
2025-06-01T21:03:59.3863424Z     parser.add_argument(
2025-06-01T21:03:59.3864088Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2025-06-01T21:03:59.3864772Z     )
2025-06-01T21:03:59.3865144Z     parser.add_argument(
2025-06-01T21:03:59.3865794Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2025-06-01T21:03:59.3866497Z     )
2025-06-01T21:03:59.3866878Z     parser.add_argument(
2025-06-01T21:03:59.3867538Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2025-06-01T21:03:59.3868262Z     )
2025-06-01T21:03:59.3868634Z     parser.add_argument(
2025-06-01T21:03:59.3869090Z         "--github-ref-type",
2025-06-01T21:03:59.3869547Z         type=str,
2025-06-01T21:03:59.3869951Z         required=True,
2025-06-01T21:03:59.3870450Z         help="Current GitHub ref type, branch or tag",
2025-06-01T21:03:59.3871020Z     )
2025-06-01T21:03:59.3871397Z     parser.add_argument(
2025-06-01T21:03:59.3871866Z         "--eligible-experiments",
2025-06-01T21:03:59.3872553Z         type=_str_comma_separated_to_set,
2025-06-01T21:03:59.3873085Z         required=False,
2025-06-01T21:03:59.3873740Z         default="",
2025-06-01T21:03:59.3874622Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2025-06-01T21:03:59.3875586Z     )
2025-06-01T21:03:59.3875962Z     parser.add_argument(
2025-06-01T21:03:59.3876439Z         "--opt-out-experiments",
2025-06-01T21:03:59.3876956Z         type=_str_comma_separated_to_set,
2025-06-01T21:03:59.3877498Z         required=False,
2025-06-01T21:03:59.3877923Z         default="",
2025-06-01T21:03:59.3878328Z         help=(
2025-06-01T21:03:59.3879016Z             "comma separated list of experiments to opt-out of. If unset, no opt-outs will occur. "
2025-06-01T21:03:59.3880179Z             "If the same experiment is listed both here and in '--eligible-experiments' opt-out will take priority."
2025-06-01T21:03:59.3881035Z         ),
2025-06-01T21:03:59.3881404Z     )
2025-06-01T21:03:59.3881788Z     parser.add_argument(
2025-06-01T21:03:59.3882233Z         "--pr-number",
2025-06-01T21:03:59.3931758Z         type=str,
2025-06-01T21:03:59.3932354Z         required=False,
2025-06-01T21:03:59.3932846Z         default="",
2025-06-01T21:03:59.3933698Z         help="the optional PR number where this is run",
2025-06-01T21:03:59.3934312Z     )
2025-06-01T21:03:59.3934519Z 
2025-06-01T21:03:59.3934722Z     return parser.parse_args()
2025-06-01T21:03:59.3935039Z 
2025-06-01T21:03:59.3935046Z 
2025-06-01T21:03:59.3935467Z def get_gh_client(github_token: str) -> Github:  # type: ignore[no-any-unimported]
2025-06-01T21:03:59.3936263Z     auth = Auth.Token(github_token)
2025-06-01T21:03:59.3936792Z     return Github(auth=auth)
2025-06-01T21:03:59.3937104Z 
2025-06-01T21:03:59.3937111Z 
2025-06-01T21:03:59.3937587Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:  # type: ignore[no-any-unimported]
2025-06-01T21:03:59.3938414Z     repo = gh.get_repo(repo)
2025-06-01T21:03:59.3938945Z     return repo.get_issue(number=issue_num)
2025-06-01T21:03:59.3939321Z 
2025-06-01T21:03:59.3939327Z 
2025-06-01T21:03:59.3939529Z def get_potential_pr_author(
2025-06-01T21:03:59.3940194Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2025-06-01T21:03:59.3940893Z ) -> str:
2025-06-01T21:03:59.3941417Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2025-06-01T21:03:59.3942234Z     # Fetch the actual username from the original PR. The PR number is
2025-06-01T21:03:59.3942978Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2025-06-01T21:03:59.3943547Z 
2025-06-01T21:03:59.3943745Z     gh = get_gh_client(github_token)
2025-06-01T21:03:59.3944082Z 
2025-06-01T21:03:59.3944362Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2025-06-01T21:03:59.3944994Z         split_tag = ref_name.split("/")
2025-06-01T21:03:59.3945507Z         if (
2025-06-01T21:03:59.3945894Z             len(split_tag) == 3
2025-06-01T21:03:59.3946392Z             and split_tag[0] == "ciflow"
2025-06-01T21:03:59.3946927Z             and split_tag[2].isnumeric()
2025-06-01T21:03:59.3947439Z         ):
2025-06-01T21:03:59.3947840Z             pr_number = split_tag[2]
2025-06-01T21:03:59.3948324Z             try:
2025-06-01T21:03:59.3948766Z                 repository = gh.get_repo(repo)
2025-06-01T21:03:59.3949390Z                 pull = repository.get_pull(number=int(pr_number))
2025-06-01T21:03:59.3950006Z             except Exception as e:
2025-06-01T21:03:59.3950530Z                 raise Exception(  # noqa: TRY002
2025-06-01T21:03:59.3951208Z                     f"issue with pull request {pr_number} from repo {repository}"
2025-06-01T21:03:59.3951854Z                 ) from e
2025-06-01T21:03:59.3952406Z             return pull.user.login  # type: ignore[no-any-return]
2025-06-01T21:03:59.3953108Z     # In all other cases, return the original input username
2025-06-01T21:03:59.3953831Z     return username
2025-06-01T21:03:59.3954079Z 
2025-06-01T21:03:59.3954220Z 
2025-06-01T21:03:59.3954460Z def is_exception_branch(branch: str) -> bool:
2025-06-01T21:03:59.3954999Z     """
2025-06-01T21:03:59.3955784Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2025-06-01T21:03:59.3956569Z     """
2025-06-01T21:03:59.3957124Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2025-06-01T21:03:59.3957660Z 
2025-06-01T21:03:59.3957667Z 
2025-06-01T21:03:59.3957873Z def load_yaml(yaml_text: str) -> Any:
2025-06-01T21:03:59.3958377Z     try:
2025-06-01T21:03:59.3958780Z         data = yaml.safe_load(yaml_text)
2025-06-01T21:03:59.3959294Z         return data
2025-06-01T21:03:59.3959732Z     except yaml.YAMLError:
2025-06-01T21:03:59.3960213Z         log.exception("Error loading YAML")
2025-06-01T21:03:59.3960737Z         raise
2025-06-01T21:03:59.3960958Z 
2025-06-01T21:03:59.3960966Z 
2025-06-01T21:03:59.3961380Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> tuple[str, str]:
2025-06-01T21:03:59.3962149Z     """
2025-06-01T21:03:59.3962789Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2025-06-01T21:03:59.3963526Z 
2025-06-01T21:03:59.3964005Z     If the issue body contains "---" then the text above that is the settings
2025-06-01T21:03:59.3964888Z     and the text below is the list of opted in users.
2025-06-01T21:03:59.3965301Z 
2025-06-01T21:03:59.3965679Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2025-06-01T21:03:59.3966401Z     """
2025-06-01T21:03:59.3966849Z     rollout_state_parts = rollout_state.split("---")
2025-06-01T21:03:59.3967479Z     if len(rollout_state_parts) >= 2:
2025-06-01T21:03:59.3968091Z         return rollout_state_parts[0], rollout_state_parts[1]
2025-06-01T21:03:59.3968685Z     else:
2025-06-01T21:03:59.3969079Z         return "", rollout_state
2025-06-01T21:03:59.3969390Z 
2025-06-01T21:03:59.3969397Z 
2025-06-01T21:03:59.3969601Z class UserOptins(dict[str, list[str]]):
2025-06-01T21:03:59.3970120Z     """
2025-06-01T21:03:59.3970641Z     Dictionary of users with a list of features they have opted into
2025-06-01T21:03:59.3971304Z     """
2025-06-01T21:03:59.3971511Z 
2025-06-01T21:03:59.3971519Z 
2025-06-01T21:03:59.3971876Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2025-06-01T21:03:59.3972546Z     """
2025-06-01T21:03:59.3973432Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2025-06-01T21:03:59.3974157Z 
2025-06-01T21:03:59.3974787Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2025-06-01T21:03:59.3976131Z         - Example line: "@User1,lf,split_build"
2025-06-01T21:03:59.3977123Z         - A "#" prefix indicates the user is opted out of all experiments
2025-06-01T21:03:59.3977617Z 
2025-06-01T21:03:59.3977624Z 
2025-06-01T21:03:59.3977789Z     """
2025-06-01T21:03:59.3978173Z     optins = UserOptins()
2025-06-01T21:03:59.3978700Z     for user in user_optin_text.split("\n"):
2025-06-01T21:03:59.3979266Z         user = user.strip("\r\n\t -")
2025-06-01T21:03:59.3979815Z         if not user or not user.startswith("@"):
2025-06-01T21:03:59.3980401Z             # Not a valid user. Skip
2025-06-01T21:03:59.3980909Z             continue
2025-06-01T21:03:59.3981164Z 
2025-06-01T21:03:59.3981341Z         if user:
2025-06-01T21:03:59.3981790Z             usr_name = user.split(",")[0].strip("@")
2025-06-01T21:03:59.3982490Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2025-06-01T21:03:59.3982991Z 
2025-06-01T21:03:59.3983365Z     return optins
2025-06-01T21:03:59.3983663Z 
2025-06-01T21:03:59.3983670Z 
2025-06-01T21:03:59.3983965Z def is_valid_experiment_name(experiment_name: str) -> bool:
2025-06-01T21:03:59.3984796Z     """
2025-06-01T21:03:59.3985222Z     Check if the experiment name is valid.
2025-06-01T21:03:59.3985763Z     A valid name:
2025-06-01T21:03:59.3986787Z         - Contains only alphanumeric characters and the special characters "_" & "-"
2025-06-01T21:03:59.3987957Z         - The special characters "_" & "-" shouldn't be the first or last characters
2025-06-01T21:03:59.3988720Z         - Cannot contain spaces
2025-06-01T21:03:59.3989200Z     """
2025-06-01T21:03:59.3989414Z 
2025-06-01T21:03:59.3989681Z     valid_char_regex = r"^[a-zA-Z0-9]([\w-]*[a-zA-Z0-9])?$"
2025-06-01T21:03:59.3990393Z     valid = bool(re.match(valid_char_regex, experiment_name))
2025-06-01T21:03:59.3990849Z 
2025-06-01T21:03:59.3991013Z     if valid:
2025-06-01T21:03:59.3991412Z         return True
2025-06-01T21:03:59.3991660Z 
2025-06-01T21:03:59.3991824Z     log.error(
2025-06-01T21:03:59.3993434Z         f"Invalid experiment name: {experiment_name}. Experiment names should only contain alphanumeric characters, '_', and '-'. They cannot contain spaces, and the special characters '_' and '-' cannot be the first or last characters."
2025-06-01T21:03:59.3995069Z     )
2025-06-01T21:03:59.3995436Z     return False
2025-06-01T21:03:59.3995673Z 
2025-06-01T21:03:59.3995679Z 
2025-06-01T21:03:59.3995999Z def parse_settings_from_text(settings_text: str) -> Settings:
2025-06-01T21:03:59.3996632Z     """
2025-06-01T21:03:59.3997537Z     Parse the experiments from the issue body into a list of ExperimentSettings
2025-06-01T21:03:59.3998463Z     """
2025-06-01T21:03:59.3998836Z     try:
2025-06-01T21:03:59.3999233Z         if settings_text:
2025-06-01T21:03:59.3999965Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2025-06-01T21:03:59.4000770Z             # for easy reading
2025-06-01T21:03:59.4001561Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2025-06-01T21:03:59.4002468Z             #       the backtick character in shell commands.
2025-06-01T21:03:59.4003077Z             backtick = chr(96)  # backtick character
2025-06-01T21:03:59.4003893Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2025-06-01T21:03:59.4004575Z             settings = load_yaml(settings_text)
2025-06-01T21:03:59.4004960Z 
2025-06-01T21:03:59.4005385Z             # For now we just load experiments. We can expand this if/when we add more settings
2025-06-01T21:03:59.4006177Z             experiments = {}
2025-06-01T21:03:59.4006482Z 
2025-06-01T21:03:59.4006861Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2025-06-01T21:03:59.4007654Z                 if not is_valid_experiment_name(exp_name):
2025-06-01T21:03:59.4008779Z                     # Exclude invalid experiments from the list. We log an error, but don't raise an exception so that other experiments can still be processed.
2025-06-01T21:03:59.4009838Z                     continue
2025-06-01T21:03:59.4010126Z 
2025-06-01T21:03:59.4010319Z                 valid_settings = {}
2025-06-01T21:03:59.4010837Z                 for setting in exp_settings:
2025-06-01T21:03:59.4011422Z                     if setting not in Experiment._fields:
2025-06-01T21:03:59.4011986Z                         log.warning(
2025-06-01T21:03:59.4012705Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2025-06-01T21:03:59.4013572Z                         )
2025-06-01T21:03:59.4014009Z                     else:
2025-06-01T21:03:59.4014526Z                         valid_settings[setting] = exp_settings[setting]
2025-06-01T21:03:59.4014956Z 
2025-06-01T21:03:59.4015230Z                 experiments[exp_name] = Experiment(**valid_settings)
2025-06-01T21:03:59.4015870Z             return Settings(experiments)
2025-06-01T21:03:59.4016225Z 
2025-06-01T21:03:59.4016403Z     except Exception:
2025-06-01T21:03:59.4016883Z         log.exception("Failed to parse settings")
2025-06-01T21:03:59.4017278Z 
2025-06-01T21:03:59.4017456Z     return Settings()
2025-06-01T21:03:59.4017715Z 
2025-06-01T21:03:59.4017722Z 
2025-06-01T21:03:59.4017968Z def parse_settings(rollout_state: str) -> Settings:
2025-06-01T21:03:59.4018695Z     """
2025-06-01T21:03:59.4019137Z     Parse settings, if any, from the rollout state.
2025-06-01T21:03:59.4019550Z 
2025-06-01T21:03:59.4019902Z     If the issue body contains "---" then the text above that is the settings
2025-06-01T21:03:59.4020672Z     and the text below is the list of opted in users.
2025-06-01T21:03:59.4021089Z 
2025-06-01T21:03:59.4021497Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2025-06-01T21:03:59.4022244Z     """
2025-06-01T21:03:59.4022801Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2025-06-01T21:03:59.4024144Z     return parse_settings_from_text(settings_text)
2025-06-01T21:03:59.4024564Z 
2025-06-01T21:03:59.4024571Z 
2025-06-01T21:03:59.4024816Z def parse_users(rollout_state: str) -> UserOptins:
2025-06-01T21:03:59.4025391Z     """
2025-06-01T21:03:59.4025784Z     Parse users from the rollout state.
2025-06-01T21:03:59.4026149Z 
2025-06-01T21:03:59.4026317Z     """
2025-06-01T21:03:59.4026854Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2025-06-01T21:03:59.4027613Z     return parse_user_opt_in_from_text(users_text)
2025-06-01T21:03:59.4028024Z 
2025-06-01T21:03:59.4028031Z 
2025-06-01T21:03:59.4028595Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-06-01T21:03:59.4029366Z     """
2025-06-01T21:03:59.4029787Z     Check if a user is opted into an experiment
2025-06-01T21:03:59.4030337Z     """
2025-06-01T21:03:59.4030793Z     return experiment_name in user_optins.get(user, [])
2025-06-01T21:03:59.4031214Z 
2025-06-01T21:03:59.4031220Z 
2025-06-01T21:03:59.4031647Z def is_user_opted_out(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-06-01T21:03:59.4032403Z     """
2025-06-01T21:03:59.4032863Z     Check if a user explicitly opted out of an experiment
2025-06-01T21:03:59.4033667Z     """
2025-06-01T21:03:59.4034191Z     # if the experiment is prefixed with a "-", then it's an opt-out
2025-06-01T21:03:59.4034894Z     experiment_optout = "-" + experiment_name
2025-06-01T21:03:59.4035542Z     if experiment_optout not in user_optins.get(user, []):
2025-06-01T21:03:59.4036161Z         return False
2025-06-01T21:03:59.4036414Z 
2025-06-01T21:03:59.4036695Z     if is_user_opted_in(user, user_optins, experiment_name):
2025-06-01T21:03:59.4037309Z         log.warning(
2025-06-01T21:03:59.4038125Z             f"User {user} is opted into experiment {experiment_name}, but also opted out of it. Defaulting to opting out"
2025-06-01T21:03:59.4039046Z         )
2025-06-01T21:03:59.4039259Z 
2025-06-01T21:03:59.4039426Z     return True
2025-06-01T21:03:59.4039685Z 
2025-06-01T21:03:59.4039692Z 
2025-06-01T21:03:59.4039873Z def get_runner_prefix(
2025-06-01T21:03:59.4040317Z     rollout_state: str,
2025-06-01T21:03:59.4040776Z     workflow_requestors: Iterable[str],
2025-06-01T21:03:59.4041303Z     branch: str,
2025-06-01T21:03:59.4041793Z     eligible_experiments: frozenset[str] = frozenset(),
2025-06-01T21:03:59.4042646Z     opt_out_experiments: frozenset[str] = frozenset(),
2025-06-01T21:03:59.4043458Z     is_canary: bool = False,
2025-06-01T21:03:59.4043935Z ) -> str:
2025-06-01T21:03:59.4044397Z     settings = parse_settings(rollout_state)
2025-06-01T21:03:59.4045004Z     user_optins = parse_users(rollout_state)
2025-06-01T21:03:59.4045377Z 
2025-06-01T21:03:59.4045557Z     fleet_prefix = ""
2025-06-01T21:03:59.4045975Z     prefixes = []
2025-06-01T21:03:59.4046607Z     for experiment_name, experiment_settings in settings.experiments.items():
2025-06-01T21:03:59.4047549Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2025-06-01T21:03:59.4048289Z             log.info(
2025-06-01T21:03:59.4048977Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2025-06-01T21:03:59.4049744Z             )
2025-06-01T21:03:59.4050124Z             continue
2025-06-01T21:03:59.4050382Z 
2025-06-01T21:03:59.4050569Z         if opt_out_experiments:
2025-06-01T21:03:59.4051274Z             if experiment_name in opt_out_experiments:
2025-06-01T21:03:59.4051915Z                 opt_out_exp_list = ", ".join(opt_out_experiments)
2025-06-01T21:03:59.4052502Z                 log.info(
2025-06-01T21:03:59.4053549Z                     f"Skipping experiment '{experiment_name}', as this workflow has opted-out (opted out experiments are: {opt_out_exp_list})"
2025-06-01T21:03:59.4054547Z                 )
2025-06-01T21:03:59.4054960Z                 continue
2025-06-01T21:03:59.4055259Z 
2025-06-01T21:03:59.4055450Z         if eligible_experiments:
2025-06-01T21:03:59.4056040Z             if experiment_name not in eligible_experiments:
2025-06-01T21:03:59.4056676Z                 exp_list = ", ".join(eligible_experiments)
2025-06-01T21:03:59.4057232Z                 log.info(
2025-06-01T21:03:59.4058015Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2025-06-01T21:03:59.4058870Z                 )
2025-06-01T21:03:59.4059262Z                 continue
2025-06-01T21:03:59.4059733Z         elif not experiment_settings.default:
2025-06-01T21:03:59.4060268Z             log.info(
2025-06-01T21:03:59.4061046Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2025-06-01T21:03:59.4061805Z             )
2025-06-01T21:03:59.4062175Z             continue
2025-06-01T21:03:59.4062429Z 
2025-06-01T21:03:59.4062699Z         # Is any workflow_requestor opted out to this experiment?
2025-06-01T21:03:59.4063520Z         opted_out_users = [
2025-06-01T21:03:59.4063977Z             requestor
2025-06-01T21:03:59.4064427Z             for requestor in workflow_requestors
2025-06-01T21:03:59.4065093Z             if is_user_opted_out(requestor, user_optins, experiment_name)
2025-06-01T21:03:59.4065723Z         ]
2025-06-01T21:03:59.4065928Z 
2025-06-01T21:03:59.4066106Z         if opted_out_users:
2025-06-01T21:03:59.4066560Z             log.info(
2025-06-01T21:03:59.4067181Z                 f"{', '.join(opted_out_users)} have opted out of experiment {experiment_name}."
2025-06-01T21:03:59.4067898Z             )
2025-06-01T21:03:59.4068287Z             continue
2025-06-01T21:03:59.4068534Z 
2025-06-01T21:03:59.4068804Z         # Is any workflow_requestor opted in to this experiment?
2025-06-01T21:03:59.4069443Z         opted_in_users = [
2025-06-01T21:03:59.4069909Z             requestor
2025-06-01T21:03:59.4070372Z             for requestor in workflow_requestors
2025-06-01T21:03:59.4071039Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2025-06-01T21:03:59.4071676Z         ]
2025-06-01T21:03:59.4071883Z 
2025-06-01T21:03:59.4072057Z         enabled = False
2025-06-01T21:03:59.4072502Z         if opted_in_users:
2025-06-01T21:03:59.4072951Z             log.info(
2025-06-01T21:03:59.4073694Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2025-06-01T21:03:59.4074393Z             )
2025-06-01T21:03:59.4074785Z             enabled = True
2025-06-01T21:03:59.4075071Z 
2025-06-01T21:03:59.4075286Z         elif experiment_settings.rollout_perc:
2025-06-01T21:03:59.4076125Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2025-06-01T21:03:59.4077081Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2025-06-01T21:03:59.4077739Z                 log.info(
2025-06-01T21:03:59.4078603Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2025-06-01T21:03:59.4079544Z                 )
2025-06-01T21:03:59.4079946Z                 enabled = True
2025-06-01T21:03:59.4080253Z 
2025-06-01T21:03:59.4080419Z         if enabled:
2025-06-01T21:03:59.4080843Z             label = experiment_name
2025-06-01T21:03:59.4081391Z             if experiment_name == LF_FLEET_EXPERIMENT:
2025-06-01T21:03:59.4082226Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2025-06-01T21:03:59.4083412Z                 #  - If it's enabled, then we always list it's prefix first
2025-06-01T21:03:59.4084205Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2025-06-01T21:03:59.4084877Z                 if is_canary:
2025-06-01T21:03:59.4085375Z                     label += CANARY_FLEET_SUFFIX
2025-06-01T21:03:59.4085930Z                 fleet_prefix = label
2025-06-01T21:03:59.4086438Z             else:
2025-06-01T21:03:59.4086863Z                 prefixes.append(label)
2025-06-01T21:03:59.4087216Z 
2025-06-01T21:03:59.4087398Z     if len(prefixes) > 1:
2025-06-01T21:03:59.4087841Z         log.error(
2025-06-01T21:03:59.4088878Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2025-06-01T21:03:59.4090030Z         )
2025-06-01T21:03:59.4090417Z         prefixes = prefixes[:1]
2025-06-01T21:03:59.4090737Z 
2025-06-01T21:03:59.4090938Z     # Fleet always comes first
2025-06-01T21:03:59.4091405Z     if fleet_prefix:
2025-06-01T21:03:59.4091855Z         prefixes.insert(0, fleet_prefix)
2025-06-01T21:03:59.4092215Z 
2025-06-01T21:03:59.4092593Z     return ".".join(prefixes) + "." if prefixes else ""
2025-06-01T21:03:59.4093019Z 
2025-06-01T21:03:59.4093026Z 
2025-06-01T21:03:59.4093574Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2025-06-01T21:03:59.4094367Z     """
2025-06-01T21:03:59.4094948Z     Gets the first comment of the issue, which contains the desired rollout state.
2025-06-01T21:03:59.4095525Z 
2025-06-01T21:03:59.4095913Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2025-06-01T21:03:59.4096631Z     """
2025-06-01T21:03:59.4097014Z     gh = get_gh_client(github_token)
2025-06-01T21:03:59.4097554Z     issue = get_issue(gh, repo, issue_num)
2025-06-01T21:03:59.4098194Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2025-06-01T21:03:59.4098651Z 
2025-06-01T21:03:59.4098663Z 
2025-06-01T21:03:59.4099065Z def download_json(url: str, headers: dict[str, str], num_retries: int = 3) -> Any:
2025-06-01T21:03:59.4099833Z     for _ in range(num_retries):
2025-06-01T21:03:59.4100321Z         try:
2025-06-01T21:03:59.4100742Z             req = Request(url=url, headers=headers)
2025-06-01T21:03:59.4101412Z             content = urlopen(req, timeout=5).read().decode("utf-8")
2025-06-01T21:03:59.4102066Z             return json.loads(content)
2025-06-01T21:03:59.4102590Z         except Exception as e:
2025-06-01T21:03:59.4103127Z             log.warning(f"Could not download {url}: {e}")
2025-06-01T21:03:59.4103632Z 
2025-06-01T21:03:59.4104015Z     log.warning(f"All {num_retries} retries exhausted, downloading {url} failed")
2025-06-01T21:03:59.4104739Z     return {}
2025-06-01T21:03:59.4104960Z 
2025-06-01T21:03:59.4104966Z 
2025-06-01T21:03:59.4105123Z @cache
2025-06-01T21:03:59.4105749Z def get_pr_info(github_repo: str, github_token: str, pr_number: int) -> dict[str, Any]:
2025-06-01T21:03:59.4106525Z     """
2025-06-01T21:03:59.4106911Z     Dynamically get PR information
2025-06-01T21:03:59.4107402Z     """
2025-06-01T21:03:59.4107906Z     github_api = f"https://api.github.com/repos/{github_repo}"
2025-06-01T21:03:59.4108539Z     headers = {
2025-06-01T21:03:59.4108990Z         "Accept": "application/vnd.github.v3+json",
2025-06-01T21:03:59.4109602Z         "Authorization": f"token {github_token}",
2025-06-01T21:03:59.4110142Z     }
2025-06-01T21:03:59.4110567Z     json_response: dict[str, Any] = download_json(
2025-06-01T21:03:59.4111178Z         url=f"{github_api}/issues/{pr_number}",
2025-06-01T21:03:59.4111724Z         headers=headers,
2025-06-01T21:03:59.4112150Z     )
2025-06-01T21:03:59.4112344Z 
2025-06-01T21:03:59.4112526Z     if not json_response:
2025-06-01T21:03:59.4113100Z         log.warning(f"Failed to get the labels for #{pr_number}")
2025-06-01T21:03:59.4113812Z         return {}
2025-06-01T21:03:59.4114224Z 
2025-06-01T21:03:59.4114401Z     return json_response
2025-06-01T21:03:59.4114678Z 
2025-06-01T21:03:59.4114685Z 
2025-06-01T21:03:59.4115086Z def get_labels(github_repo: str, github_token: str, pr_number: int) -> set[str]:
2025-06-01T21:03:59.4115825Z     """
2025-06-01T21:03:59.4116354Z     Dynamically get the latest list of labels from the pull request
2025-06-01T21:03:59.4117019Z     """
2025-06-01T21:03:59.4117506Z     pr_info = get_pr_info(github_repo, github_token, pr_number)
2025-06-01T21:03:59.4118125Z     return {
2025-06-01T21:03:59.4118723Z         label.get("name") for label in pr_info.get("labels", []) if label.get("name")
2025-06-01T21:03:59.4119448Z     }
2025-06-01T21:03:59.4119659Z 
2025-06-01T21:03:59.4119665Z 
2025-06-01T21:03:59.4119835Z def main() -> None:
2025-06-01T21:03:59.4120262Z     args = parse_args()
2025-06-01T21:03:59.4120535Z 
2025-06-01T21:03:59.4120756Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2025-06-01T21:03:59.4121148Z 
2025-06-01T21:03:59.4121353Z     # Check if the PR is opt-out
2025-06-01T21:03:59.4121840Z     if args.pr_number:
2025-06-01T21:03:59.4122492Z         labels = get_labels(args.github_repo, args.github_token, int(args.pr_number))
2025-06-01T21:03:59.4123466Z         if OPT_OUT_LABEL in labels:
2025-06-01T21:03:59.4123992Z             log.info(
2025-06-01T21:03:59.4124683Z                 f"Opt-out runner determinator because #{args.pr_number} has {OPT_OUT_LABEL} label"
2025-06-01T21:03:59.4125461Z             )
2025-06-01T21:03:59.4126013Z             set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-06-01T21:03:59.4126685Z             sys.exit()
2025-06-01T21:03:59.4126946Z 
2025-06-01T21:03:59.4127117Z     try:
2025-06-01T21:03:59.4127555Z         rollout_state = get_rollout_state_from_issue(
2025-06-01T21:03:59.4128272Z             args.github_token, args.github_issue_repo, args.github_issue
2025-06-01T21:03:59.4128911Z         )
2025-06-01T21:03:59.4129123Z 
2025-06-01T21:03:59.4129331Z         username = get_potential_pr_author(
2025-06-01T21:03:59.4129882Z             args.github_token,
2025-06-01T21:03:59.4130353Z             args.github_repo,
2025-06-01T21:03:59.4130835Z             args.github_actor,
2025-06-01T21:03:59.4131310Z             args.github_ref_type,
2025-06-01T21:03:59.4131809Z             args.github_branch,
2025-06-01T21:03:59.4132258Z         )
2025-06-01T21:03:59.4132470Z 
2025-06-01T21:03:59.4132756Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2025-06-01T21:03:59.4133323Z 
2025-06-01T21:03:59.4133544Z         runner_label_prefix = get_runner_prefix(
2025-06-01T21:03:59.4134102Z             rollout_state,
2025-06-01T21:03:59.4134582Z             (args.github_issue_owner, username),
2025-06-01T21:03:59.4135128Z             args.github_branch,
2025-06-01T21:03:59.4135625Z             args.eligible_experiments,
2025-06-01T21:03:59.4136166Z             args.opt_out_experiments,
2025-06-01T21:03:59.4136670Z             is_canary,
2025-06-01T21:03:59.4137075Z         )
2025-06-01T21:03:59.4137299Z 
2025-06-01T21:03:59.4137482Z     except Exception as e:
2025-06-01T21:03:59.4137934Z         log.error(
2025-06-01T21:03:59.4138609Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2025-06-01T21:03:59.4139387Z         )
2025-06-01T21:03:59.4139599Z 
2025-06-01T21:03:59.4139932Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-06-01T21:03:59.4140448Z 
2025-06-01T21:03:59.4140454Z 
2025-06-01T21:03:59.4140638Z if __name__ == "__main__":
2025-06-01T21:03:59.4141071Z     main()
2025-06-01T21:03:59.4141290Z 
2025-06-01T21:03:59.4229915Z ##[group]Run python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2025-06-01T21:03:59.4230822Z python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2025-06-01T21:03:59.4281941Z shell: /usr/bin/bash -e {0}
2025-06-01T21:03:59.4282428Z env:
2025-06-01T21:03:59.4283058Z   GITHUB_TOKEN: ***
2025-06-01T21:03:59.4283626Z   ISSUE_NUMBER: 5132
2025-06-01T21:03:59.4284280Z   TRIGGERING_ACTOR: pytorchmergebot
2025-06-01T21:03:59.4284798Z   ISSUE_OWNER: 
2025-06-01T21:03:59.4285207Z   CHECK_EXPERIMENTS: 
2025-06-01T21:03:59.4285637Z   OPT_OUT_EXPERIMENTS: 
2025-06-01T21:03:59.4286079Z   PR_NUMBER: 
2025-06-01T21:03:59.4286474Z ##[endgroup]
2025-06-01T21:03:59.8182218Z Defaulting to user installation because normal site-packages is not writeable
2025-06-01T21:04:00.2510289Z Collecting urllib3==1.26.18
2025-06-01T21:04:00.2991730Z   Downloading urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
2025-06-01T21:04:00.3227696Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 3.8 MB/s eta 0:00:00
2025-06-01T21:04:00.3502066Z Collecting PyGithub==2.3.0
2025-06-01T21:04:00.3586709Z   Downloading PyGithub-2.3.0-py3-none-any.whl.metadata (3.8 kB)
2025-06-01T21:04:00.4045335Z Collecting pynacl>=1.4.0 (from PyGithub==2.3.0)
2025-06-01T21:04:00.4122133Z   Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (8.6 kB)
2025-06-01T21:04:00.4166026Z Requirement already satisfied: requests>=2.14.0 in /usr/lib/python3/dist-packages (from PyGithub==2.3.0) (2.31.0)
2025-06-01T21:04:00.4182538Z Requirement already satisfied: pyjwt>=2.4.0 in /usr/lib/python3/dist-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.3.0) (2.7.0)
2025-06-01T21:04:00.4197138Z Requirement already satisfied: typing-extensions>=4.0.0 in /usr/lib/python3/dist-packages (from PyGithub==2.3.0) (4.10.0)
2025-06-01T21:04:00.4488701Z Collecting Deprecated (from PyGithub==2.3.0)
2025-06-01T21:04:00.4567985Z   Downloading Deprecated-1.2.18-py2.py3-none-any.whl.metadata (5.7 kB)
2025-06-01T21:04:00.4796695Z Requirement already satisfied: cryptography>=3.4.0 in /usr/lib/python3/dist-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.3.0) (41.0.7)
2025-06-01T21:04:00.5966096Z Collecting cffi>=1.4.1 (from pynacl>=1.4.0->PyGithub==2.3.0)
2025-06-01T21:04:00.6038141Z   Downloading cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.5 kB)
2025-06-01T21:04:00.7107104Z Collecting wrapt<2,>=1.10 (from Deprecated->PyGithub==2.3.0)
2025-06-01T21:04:00.7197360Z   Downloading wrapt-1.17.2-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.4 kB)
2025-06-01T21:04:00.7421748Z Collecting pycparser (from cffi>=1.4.1->pynacl>=1.4.0->PyGithub==2.3.0)
2025-06-01T21:04:00.7491160Z   Downloading pycparser-2.22-py3-none-any.whl.metadata (943 bytes)
2025-06-01T21:04:00.7765563Z Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
2025-06-01T21:04:00.7904304Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.8/143.8 kB 11.2 MB/s eta 0:00:00
2025-06-01T21:04:00.8011015Z Downloading PyGithub-2.3.0-py3-none-any.whl (354 kB)
2025-06-01T21:04:00.8325546Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 354.4/354.4 kB 11.6 MB/s eta 0:00:00
2025-06-01T21:04:00.8395997Z Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
2025-06-01T21:04:00.8876363Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 856.7/856.7 kB 18.3 MB/s eta 0:00:00
2025-06-01T21:04:00.8983682Z Downloading Deprecated-1.2.18-py2.py3-none-any.whl (10.0 kB)
2025-06-01T21:04:00.9092244Z Downloading cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (479 kB)
2025-06-01T21:04:00.9406191Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 479.4/479.4 kB 15.8 MB/s eta 0:00:00
2025-06-01T21:04:00.9484578Z Downloading wrapt-1.17.2-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (89 kB)
2025-06-01T21:04:00.9534101Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.2/89.2 kB 23.7 MB/s eta 0:00:00
2025-06-01T21:04:00.9602066Z Downloading pycparser-2.22-py3-none-any.whl (117 kB)
2025-06-01T21:04:00.9666682Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 kB 21.9 MB/s eta 0:00:00
2025-06-01T21:04:01.2544986Z Installing collected packages: wrapt, urllib3, pycparser, Deprecated, cffi, pynacl, PyGithub
2025-06-01T21:04:01.7929260Z Successfully installed Deprecated-1.2.18 PyGithub-2.3.0 cffi-1.17.1 pycparser-2.22 pynacl-1.5.0 urllib3-1.26.18 wrapt-1.17.2
2025-06-01T21:04:01.8702629Z ##[group]Run curr_branch="main"
2025-06-01T21:04:01.8702933Z curr_branch="main"
2025-06-01T21:04:01.8703149Z curr_ref_type="branch"
2025-06-01T21:04:01.8703681Z echo "Current branch is '$curr_branch'"
2025-06-01T21:04:01.8703944Z 
2025-06-01T21:04:01.8704119Z python3 runner_determinator.py \
2025-06-01T21:04:01.8704396Z   --github-token "$GITHUB_TOKEN" \
2025-06-01T21:04:01.8704659Z   --github-issue "$ISSUE_NUMBER" \
2025-06-01T21:04:01.8704926Z   --github-branch "$curr_branch" \
2025-06-01T21:04:01.8705189Z   --github-actor "$TRIGGERING_ACTOR" \
2025-06-01T21:04:01.8705483Z   --github-issue-owner "$ISSUE_OWNER" \
2025-06-01T21:04:01.8705760Z   --github-ref-type "$curr_ref_type" \
2025-06-01T21:04:01.8706023Z   --github-repo "$GITHUB_REPOSITORY" \
2025-06-01T21:04:01.8706345Z   --eligible-experiments "$CHECK_EXPERIMENTS" \
2025-06-01T21:04:01.8706707Z   --opt-out-experiments "$OPT_OUT_EXPERIMENTS" \
2025-06-01T21:04:01.8707000Z   --pr-number "${PR_NUMBER}"
2025-06-01T21:04:01.8759820Z shell: /usr/bin/bash -e {0}
2025-06-01T21:04:01.8760043Z env:
2025-06-01T21:04:01.8760624Z   GITHUB_TOKEN: ***
2025-06-01T21:04:01.8760811Z   ISSUE_NUMBER: 5132
2025-06-01T21:04:01.8761016Z   TRIGGERING_ACTOR: pytorchmergebot
2025-06-01T21:04:01.8761241Z   ISSUE_OWNER: 
2025-06-01T21:04:01.8761417Z   CHECK_EXPERIMENTS: 
2025-06-01T21:04:01.8761604Z   OPT_OUT_EXPERIMENTS: 
2025-06-01T21:04:01.8761790Z   PR_NUMBER: 
2025-06-01T21:04:01.8761952Z ##[endgroup]
2025-06-01T21:04:01.8830490Z Current branch is 'main'
2025-06-01T21:04:03.1061876Z INFO    : Branch main is an exception branch. Not enabling experiment ephemeral.
2025-06-01T21:04:03.1062642Z INFO    : Branch main is an exception branch. Not enabling experiment wincanary.
2025-06-01T21:04:03.1084915Z INFO    : Setting output: label-type=''
2025-06-01T21:04:03.1387314Z Evaluate and set job outputs
2025-06-01T21:04:03.1394332Z Cleaning up orphan processes