2025-09-07T06:09:28.4740396Z Current runner version: '2.328.0'
2025-09-07T06:09:28.4767739Z ##[group]Runner Image Provisioner
2025-09-07T06:09:28.4768602Z Hosted Compute Agent
2025-09-07T06:09:28.4769108Z Version: 20250829.383
2025-09-07T06:09:28.4769712Z Commit: 27cb235aab5b0e52e153a26cd86b4742e89dac5d
2025-09-07T06:09:28.4770401Z Build Date: 2025-08-29T13:48:48Z
2025-09-07T06:09:28.4771007Z ##[endgroup]
2025-09-07T06:09:28.4771536Z ##[group]Operating System
2025-09-07T06:09:28.4772109Z Ubuntu
2025-09-07T06:09:28.4772617Z 24.04.3
2025-09-07T06:09:28.4773042Z LTS
2025-09-07T06:09:28.4773540Z ##[endgroup]
2025-09-07T06:09:28.4773985Z ##[group]Runner Image
2025-09-07T06:09:28.4774554Z Image: ubuntu-24.04
2025-09-07T06:09:28.4775060Z Version: 20250831.1.0
2025-09-07T06:09:28.4776313Z Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20250831.1/images/ubuntu/Ubuntu2404-Readme.md
2025-09-07T06:09:28.4777989Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20250831.1
2025-09-07T06:09:28.4779032Z ##[endgroup]
2025-09-07T06:09:28.4780110Z ##[group]GITHUB_TOKEN Permissions
2025-09-07T06:09:28.4782426Z Contents: read
2025-09-07T06:09:28.4782967Z Metadata: read
2025-09-07T06:09:28.4783913Z ##[endgroup]
2025-09-07T06:09:28.4786612Z Secret source: Actions
2025-09-07T06:09:28.4787531Z Prepare workflow directory
2025-09-07T06:09:28.5339752Z Prepare all required actions
2025-09-07T06:09:28.5403381Z Uses: pytorch/pytorch/.github/workflows/_runner-determinator.yml@refs/heads/main (93fb23d6fae7c4e82c4239a1033e522088742634)
2025-09-07T06:09:28.5409651Z ##[group] Inputs
2025-09-07T06:09:28.5410288Z   check_experiments: 
2025-09-07T06:09:28.5410821Z   opt_out_experiments: lf
2025-09-07T06:09:28.5411469Z   triggering_actor: pytorchmergebot
2025-09-07T06:09:28.5412041Z   issue_owner: 
2025-09-07T06:09:28.5412538Z   curr_branch: main
2025-09-07T06:09:28.5413108Z   curr_ref_type: branch
2025-09-07T06:09:28.5413803Z   issue_number: 5132
2025-09-07T06:09:28.5414326Z ##[endgroup]
2025-09-07T06:09:28.5414961Z Complete job name: get-label-type / runner-determinator
2025-09-07T06:09:29.0582300Z ##[group]Run cat <<EOF > runner_determinator.py
2025-09-07T06:09:29.0584805Z cat <<EOF > runner_determinator.py
2025-09-07T06:09:29.0585577Z # flake8: noqa: G004
2025-09-07T06:09:29.0586328Z 
2025-09-07T06:09:29.0587305Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2025-09-07T06:09:29.0588459Z #       must be kept in sync. You can do it easily by running the following command:
2025-09-07T06:09:29.0589536Z #           python .github/scripts/update_runner_determinator.py
2025-09-07T06:09:29.0590387Z 
2025-09-07T06:09:29.0590908Z """
2025-09-07T06:09:29.0591710Z This runner determinator is used to determine which set of runners to run a
2025-09-07T06:09:29.0592898Z GitHub job on. It uses the first comment of a GitHub issue (by default
2025-09-07T06:09:29.0594263Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2025-09-07T06:09:29.0595377Z of which runners should be used to run which job.
2025-09-07T06:09:29.0596447Z 
2025-09-07T06:09:29.0597263Z The configuration has two parts, the settings and a list of opted-in users,
2025-09-07T06:09:29.0598491Z separated by a line containing "---".  If the line is not present, the
2025-09-07T06:09:29.0599666Z settings are considered to be empty with only the second part, the user
2025-09-07T06:09:29.0600608Z list, defined.
2025-09-07T06:09:29.0601229Z 
2025-09-07T06:09:29.0601985Z The first part is a YAML block that defines the rollout settings. This can be
2025-09-07T06:09:29.0603134Z used to define any settings that are needed to determine which runners to use.
2025-09-07T06:09:29.0604257Z It's fields are defined by the RolloutSettings class below.
2025-09-07T06:09:29.0605051Z 
2025-09-07T06:09:29.0606278Z The second part is a list of users who are explicitly opted in to the LF fleet.
2025-09-07T06:09:29.0607451Z The user list is also a comma separated list of additional features or
2025-09-07T06:09:29.0608540Z experiments which the user could be opted in to.
2025-09-07T06:09:29.0609267Z 
2025-09-07T06:09:29.0609846Z The user list has the following rules:
2025-09-07T06:09:29.0610618Z 
2025-09-07T06:09:29.0611370Z - Users are GitHub usernames, which must start with the @ prefix
2025-09-07T06:09:29.0612453Z - Each user is also a comma-separated list of features/experiments to enable
2025-09-07T06:09:29.0613481Z - A "#" prefix opts the user out of all experiments
2025-09-07T06:09:29.0614226Z 
2025-09-07T06:09:29.0614736Z Example config:
2025-09-07T06:09:29.0615508Z     # A list of experiments that can be opted into.
2025-09-07T06:09:29.0616803Z     # This defines the behavior they'll induce when opted into.
2025-09-07T06:09:29.0617639Z     # Expected syntax is:
2025-09-07T06:09:29.0705500Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2025-09-07T06:09:29.0707023Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2025-09-07T06:09:29.0707971Z 
2025-09-07T06:09:29.0708419Z     experiments:
2025-09-07T06:09:29.0708967Z       lf:
2025-09-07T06:09:29.0709472Z         rollout_percent: 25
2025-09-07T06:09:29.0710087Z         all_branches: false
2025-09-07T06:09:29.0710677Z         default: true
2025-09-07T06:09:29.0711218Z     ---
2025-09-07T06:09:29.0711684Z 
2025-09-07T06:09:29.0712116Z     # Opt-ins:
2025-09-07T06:09:29.0712846Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2025-09-07T06:09:29.0714105Z     # and specifying experiments to enable in a comma-separated list.
2025-09-07T06:09:29.0715030Z     # To always opt out of an experiment, prefix it with a "-".
2025-09-07T06:09:29.0715814Z     # Experiments should be from the above list.
2025-09-07T06:09:29.0716586Z 
2025-09-07T06:09:29.0717040Z     @User1,-lf,split_build
2025-09-07T06:09:29.0717601Z     @User2,lf
2025-09-07T06:09:29.0718105Z     @User3,split_build
2025-09-07T06:09:29.0718647Z """
2025-09-07T06:09:29.0719079Z 
2025-09-07T06:09:29.0719514Z import json
2025-09-07T06:09:29.0719993Z import logging
2025-09-07T06:09:29.0720479Z import os
2025-09-07T06:09:29.0720963Z import random
2025-09-07T06:09:29.0721451Z import re
2025-09-07T06:09:29.0721921Z import sys
2025-09-07T06:09:29.0722459Z from argparse import ArgumentParser
2025-09-07T06:09:29.0723194Z from collections.abc import Iterable
2025-09-07T06:09:29.0723845Z from functools import cache
2025-09-07T06:09:29.0724445Z from logging import LogRecord
2025-09-07T06:09:29.0725064Z from typing import Any, NamedTuple
2025-09-07T06:09:29.0725734Z from urllib.request import Request, urlopen
2025-09-07T06:09:29.0726472Z 
2025-09-07T06:09:29.0726919Z import yaml
2025-09-07T06:09:29.0727434Z from github import Auth, Github
2025-09-07T06:09:29.0728041Z from github.Issue import Issue
2025-09-07T06:09:29.0728601Z 
2025-09-07T06:09:29.0729014Z 
2025-09-07T06:09:29.0729530Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2025-09-07T06:09:29.0730364Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2025-09-07T06:09:29.0731386Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2025-09-07T06:09:29.0732203Z 
2025-09-07T06:09:29.0732914Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2025-09-07T06:09:29.0733604Z GH_OUTPUT_KEY_AMI = "runner-ami"
2025-09-07T06:09:29.0734251Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2025-09-07T06:09:29.0734947Z OPT_OUT_LABEL = "no-runner-experiments"
2025-09-07T06:09:29.0735579Z 
2025-09-07T06:09:29.0736254Z SETTING_EXPERIMENTS = "experiments"
2025-09-07T06:09:29.0736957Z 
2025-09-07T06:09:29.0737422Z LF_FLEET_EXPERIMENT = "lf"
2025-09-07T06:09:29.0737998Z CANARY_FLEET_SUFFIX = ".c"
2025-09-07T06:09:29.0738527Z 
2025-09-07T06:09:29.0738944Z 
2025-09-07T06:09:29.0739398Z class Experiment(NamedTuple):
2025-09-07T06:09:29.0739991Z     rollout_perc: float = (
2025-09-07T06:09:29.0740787Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2025-09-07T06:09:29.0741571Z     )
2025-09-07T06:09:29.0742050Z     all_branches: bool = (
2025-09-07T06:09:29.0742831Z         False  # If True, the experiment is also enabled on the exception branches
2025-09-07T06:09:29.0743592Z     )
2025-09-07T06:09:29.0744057Z     default: bool = (
2025-09-07T06:09:29.0744756Z         True  # If True, the experiment is enabled by default for all queries
2025-09-07T06:09:29.0745492Z     )
2025-09-07T06:09:29.0745928Z 
2025-09-07T06:09:29.0746526Z     # Add more fields as needed
2025-09-07T06:09:29.0747080Z 
2025-09-07T06:09:29.0747493Z 
2025-09-07T06:09:29.0747944Z class Settings(NamedTuple):
2025-09-07T06:09:29.0748504Z     """
2025-09-07T06:09:29.0749092Z     Settings for the experiments that can be opted into.
2025-09-07T06:09:29.0749764Z     """
2025-09-07T06:09:29.0750198Z 
2025-09-07T06:09:29.0750692Z     experiments: dict[str, Experiment] = {}
2025-09-07T06:09:29.0751296Z 
2025-09-07T06:09:29.0751882Z 
2025-09-07T06:09:29.0752399Z class ColorFormatter(logging.Formatter):
2025-09-07T06:09:29.0753149Z     """Color codes the log messages based on the log level"""
2025-09-07T06:09:29.0753840Z 
2025-09-07T06:09:29.0754263Z     COLORS = {
2025-09-07T06:09:29.0754789Z         "WARNING": "\033[33m",  # Yellow
2025-09-07T06:09:29.0755405Z         "ERROR": "\033[31m",  # Red
2025-09-07T06:09:29.0756011Z         "CRITICAL": "\033[31m",  # Red
2025-09-07T06:09:29.0756724Z         "INFO": "\033[0m",  # Reset
2025-09-07T06:09:29.0757328Z         "DEBUG": "\033[0m",  # Reset
2025-09-07T06:09:29.0757892Z     }
2025-09-07T06:09:29.0758333Z 
2025-09-07T06:09:29.0758850Z     def format(self, record: LogRecord) -> str:
2025-09-07T06:09:29.0759778Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2025-09-07T06:09:29.0760666Z         record.msg = f"{log_color}{record.msg}\033[0m"
2025-09-07T06:09:29.0761358Z         return super().format(record)
2025-09-07T06:09:29.0761924Z 
2025-09-07T06:09:29.0762341Z 
2025-09-07T06:09:29.0762815Z handler = logging.StreamHandler()
2025-09-07T06:09:29.0763658Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2025-09-07T06:09:29.0764467Z 
2025-09-07T06:09:29.0765014Z log = logging.getLogger(os.path.basename(__file__))
2025-09-07T06:09:29.0765712Z log.addHandler(handler)
2025-09-07T06:09:29.0766383Z log.setLevel(logging.INFO)
2025-09-07T06:09:29.0766939Z 
2025-09-07T06:09:29.0767359Z 
2025-09-07T06:09:29.0767913Z def set_github_output(key: str, value: str) -> None:
2025-09-07T06:09:29.0768567Z     """
2025-09-07T06:09:29.0769205Z     Defines outputs of the github action that invokes this script
2025-09-07T06:09:29.0770064Z     """
2025-09-07T06:09:29.0770530Z     if not GITHUB_OUTPUT:
2025-09-07T06:09:29.0771742Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2025-09-07T06:09:29.0772987Z         log.warning(
2025-09-07T06:09:29.0774001Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2025-09-07T06:09:29.0775033Z         )
2025-09-07T06:09:29.0775587Z         print(f"::set-output name={key}::{value}")
2025-09-07T06:09:29.0776509Z         return
2025-09-07T06:09:29.0777006Z 
2025-09-07T06:09:29.0777484Z     with open(GITHUB_OUTPUT, "a") as f:
2025-09-07T06:09:29.0778162Z         log.info(f"Setting output: {key}='{value}'")
2025-09-07T06:09:29.0778840Z         f.write(f"{key}={value}\n")
2025-09-07T06:09:29.0779423Z 
2025-09-07T06:09:29.0779846Z 
2025-09-07T06:09:29.0780452Z def _str_comma_separated_to_set(value: str) -> frozenset[str]:
2025-09-07T06:09:29.0781216Z     return frozenset(
2025-09-07T06:09:29.0781977Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2025-09-07T06:09:29.0782754Z     )
2025-09-07T06:09:29.0783206Z 
2025-09-07T06:09:29.0783619Z 
2025-09-07T06:09:29.0784078Z def parse_args() -> Any:
2025-09-07T06:09:29.0784797Z     parser = ArgumentParser("Get dynamic rollout settings")
2025-09-07T06:09:29.0785819Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2025-09-07T06:09:29.0786968Z     parser.add_argument(
2025-09-07T06:09:29.0787548Z         "--github-issue-repo",
2025-09-07T06:09:29.0788150Z         type=str,
2025-09-07T06:09:29.0788688Z         required=False,
2025-09-07T06:09:29.0789435Z         default="pytorch/test-infra",
2025-09-07T06:09:29.0790126Z         help="GitHub repo to get the issue",
2025-09-07T06:09:29.0790732Z     )
2025-09-07T06:09:29.0791215Z     parser.add_argument(
2025-09-07T06:09:29.0791777Z         "--github-repo",
2025-09-07T06:09:29.0792335Z         type=str,
2025-09-07T06:09:29.0792856Z         required=True,
2025-09-07T06:09:29.0793468Z         help="GitHub repo where CI is running",
2025-09-07T06:09:29.0794081Z     )
2025-09-07T06:09:29.0794605Z     parser.add_argument(
2025-09-07T06:09:29.0795356Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2025-09-07T06:09:29.0796226Z     )
2025-09-07T06:09:29.0796703Z     parser.add_argument(
2025-09-07T06:09:29.0797475Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2025-09-07T06:09:29.0798260Z     )
2025-09-07T06:09:29.0798741Z     parser.add_argument(
2025-09-07T06:09:29.0799533Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2025-09-07T06:09:29.0800331Z     )
2025-09-07T06:09:29.0800807Z     parser.add_argument(
2025-09-07T06:09:29.0801623Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2025-09-07T06:09:29.0802418Z     )
2025-09-07T06:09:29.0802907Z     parser.add_argument(
2025-09-07T06:09:29.0803472Z         "--github-ref-type",
2025-09-07T06:09:29.0804069Z         type=str,
2025-09-07T06:09:29.0804593Z         required=True,
2025-09-07T06:09:29.0805232Z         help="Current GitHub ref type, branch or tag",
2025-09-07T06:09:29.0805879Z     )
2025-09-07T06:09:29.0806498Z     parser.add_argument(
2025-09-07T06:09:29.0807087Z         "--eligible-experiments",
2025-09-07T06:09:29.0807859Z         type=_str_comma_separated_to_set,
2025-09-07T06:09:29.0808468Z         required=False,
2025-09-07T06:09:29.0808999Z         default="",
2025-09-07T06:09:29.0809988Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2025-09-07T06:09:29.0811005Z     )
2025-09-07T06:09:29.0811486Z     parser.add_argument(
2025-09-07T06:09:29.0812056Z         "--opt-out-experiments",
2025-09-07T06:09:29.0812691Z         type=_str_comma_separated_to_set,
2025-09-07T06:09:29.0813307Z         required=False,
2025-09-07T06:09:29.0813865Z         default="",
2025-09-07T06:09:29.0814385Z         help=(
2025-09-07T06:09:29.0815194Z             "comma separated list of experiments to opt-out of. If unset, no opt-outs will occur. "
2025-09-07T06:09:29.0816761Z             "If the same experiment is listed both here and in '--eligible-experiments' opt-out will take priority."
2025-09-07T06:09:29.0817721Z         ),
2025-09-07T06:09:29.0818198Z     )
2025-09-07T06:09:29.0818670Z     parser.add_argument(
2025-09-07T06:09:29.0819253Z         "--pr-number",
2025-09-07T06:09:29.0819791Z         type=str,
2025-09-07T06:09:29.0820326Z         required=False,
2025-09-07T06:09:29.0820889Z         default="",
2025-09-07T06:09:29.0821515Z         help="the optional PR number where this is run",
2025-09-07T06:09:29.0822176Z     )
2025-09-07T06:09:29.0822620Z 
2025-09-07T06:09:29.0823093Z     return parser.parse_args()
2025-09-07T06:09:29.0823665Z 
2025-09-07T06:09:29.0824095Z 
2025-09-07T06:09:29.0824807Z def get_gh_client(github_token: str) -> Github:  # type: ignore[no-any-unimported]
2025-09-07T06:09:29.0825820Z     auth = Auth.Token(github_token)
2025-09-07T06:09:29.0826632Z     return Github(auth=auth)
2025-09-07T06:09:29.0827193Z 
2025-09-07T06:09:29.0827622Z 
2025-09-07T06:09:29.0828381Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:  # type: ignore[no-any-unimported]
2025-09-07T06:09:29.0829296Z     repo = gh.get_repo(repo)
2025-09-07T06:09:29.0829922Z     return repo.get_issue(number=issue_num)
2025-09-07T06:09:29.0830532Z 
2025-09-07T06:09:29.0830965Z 
2025-09-07T06:09:29.0831427Z def get_potential_pr_author(
2025-09-07T06:09:29.0832211Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2025-09-07T06:09:29.0832996Z ) -> str:
2025-09-07T06:09:29.0833656Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2025-09-07T06:09:29.0834587Z     # Fetch the actual username from the original PR. The PR number is
2025-09-07T06:09:29.0835469Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2025-09-07T06:09:29.0836242Z 
2025-09-07T06:09:29.0836728Z     gh = get_gh_client(github_token)
2025-09-07T06:09:29.0837317Z 
2025-09-07T06:09:29.0837894Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2025-09-07T06:09:29.0838642Z         split_tag = ref_name.split("/")
2025-09-07T06:09:29.0839240Z         if (
2025-09-07T06:09:29.0839762Z             len(split_tag) == 3
2025-09-07T06:09:29.0840373Z             and split_tag[0] == "ciflow"
2025-09-07T06:09:29.0841009Z             and split_tag[2].isnumeric()
2025-09-07T06:09:29.0841596Z         ):
2025-09-07T06:09:29.0842112Z             pr_number = split_tag[2]
2025-09-07T06:09:29.0842688Z             try:
2025-09-07T06:09:29.0843248Z                 repository = gh.get_repo(repo)
2025-09-07T06:09:29.0844124Z                 pull = repository.get_pull(number=int(pr_number))
2025-09-07T06:09:29.0844859Z             except Exception as e:
2025-09-07T06:09:29.0845502Z                 raise Exception(  # noqa: TRY002
2025-09-07T06:09:29.0846391Z                     f"issue with pull request {pr_number} from repo {repository}"
2025-09-07T06:09:29.0847131Z                 ) from e
2025-09-07T06:09:29.0847810Z             return pull.user.login  # type: ignore[no-any-return]
2025-09-07T06:09:29.0848626Z     # In all other cases, return the original input username
2025-09-07T06:09:29.0849318Z     return username
2025-09-07T06:09:29.0849832Z 
2025-09-07T06:09:29.0850255Z 
2025-09-07T06:09:29.0850783Z def is_exception_branch(branch: str) -> bool:
2025-09-07T06:09:29.0851416Z     """
2025-09-07T06:09:29.0852197Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2025-09-07T06:09:29.0853070Z     """
2025-09-07T06:09:29.0853739Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2025-09-07T06:09:29.0854518Z 
2025-09-07T06:09:29.0854939Z 
2025-09-07T06:09:29.0855419Z def load_yaml(yaml_text: str) -> Any:
2025-09-07T06:09:29.0856011Z     try:
2025-09-07T06:09:29.0856782Z         data = yaml.safe_load(yaml_text)
2025-09-07T06:09:29.0857406Z         return data
2025-09-07T06:09:29.0857953Z     except yaml.YAMLError:
2025-09-07T06:09:29.0858565Z         log.exception("Error loading YAML")
2025-09-07T06:09:29.0859190Z         raise
2025-09-07T06:09:29.0859685Z 
2025-09-07T06:09:29.0860105Z 
2025-09-07T06:09:29.0860840Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> tuple[str, str]:
2025-09-07T06:09:29.0861677Z     """
2025-09-07T06:09:29.0862578Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2025-09-07T06:09:29.0863437Z 
2025-09-07T06:09:29.0864079Z     If the issue body contains "---" then the text above that is the settings
2025-09-07T06:09:29.0864970Z     and the text below is the list of opted in users.
2025-09-07T06:09:29.0865614Z 
2025-09-07T06:09:29.0866869Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2025-09-07T06:09:29.0867695Z     """
2025-09-07T06:09:29.0868267Z     rollout_state_parts = rollout_state.split("---")
2025-09-07T06:09:29.0868983Z     if len(rollout_state_parts) >= 2:
2025-09-07T06:09:29.0869915Z         return rollout_state_parts[0], rollout_state_parts[1]
2025-09-07T06:09:29.0870652Z     else:
2025-09-07T06:09:29.0871146Z         return "", rollout_state
2025-09-07T06:09:29.0871714Z 
2025-09-07T06:09:29.0872148Z 
2025-09-07T06:09:29.0872642Z class UserOptins(dict[str, list[str]]):
2025-09-07T06:09:29.0873253Z     """
2025-09-07T06:09:29.0873907Z     Dictionary of users with a list of features they have opted into
2025-09-07T06:09:29.0874646Z     """
2025-09-07T06:09:29.0875097Z 
2025-09-07T06:09:29.0875516Z 
2025-09-07T06:09:29.0876285Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2025-09-07T06:09:29.0877052Z     """
2025-09-07T06:09:29.0877894Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2025-09-07T06:09:29.0878816Z 
2025-09-07T06:09:29.0879738Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2025-09-07T06:09:29.0880865Z         - Example line: "@User1,lf,split_build"
2025-09-07T06:09:29.0881822Z         - A "#" prefix indicates the user is opted out of all experiments
2025-09-07T06:09:29.0882554Z 
2025-09-07T06:09:29.0882986Z 
2025-09-07T06:09:29.0883403Z     """
2025-09-07T06:09:29.0883889Z     optins = UserOptins()
2025-09-07T06:09:29.0884505Z     for user in user_optin_text.split("\n"):
2025-09-07T06:09:29.0885173Z         user = user.strip("\r\n\t -")
2025-09-07T06:09:29.0885834Z         if not user or not user.startswith("@"):
2025-09-07T06:09:29.0886611Z             # Not a valid user. Skip
2025-09-07T06:09:29.0887210Z             continue
2025-09-07T06:09:29.0887745Z 
2025-09-07T06:09:29.0888188Z         if user:
2025-09-07T06:09:29.0888759Z             usr_name = user.split(",")[0].strip("@")
2025-09-07T06:09:29.0889573Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2025-09-07T06:09:29.0890311Z 
2025-09-07T06:09:29.0890749Z     return optins
2025-09-07T06:09:29.0891258Z 
2025-09-07T06:09:29.0891686Z 
2025-09-07T06:09:29.0892286Z def is_valid_experiment_name(experiment_name: str) -> bool:
2025-09-07T06:09:29.0892999Z     """
2025-09-07T06:09:29.0893522Z     Check if the experiment name is valid.
2025-09-07T06:09:29.0894134Z     A valid name:
2025-09-07T06:09:29.0894950Z         - Contains only alphanumeric characters and the special characters "_" & "-"
2025-09-07T06:09:29.0896014Z         - The special characters "_" & "-" shouldn't be the first or last characters
2025-09-07T06:09:29.0896935Z         - Cannot contain spaces
2025-09-07T06:09:29.0897513Z     """
2025-09-07T06:09:29.0897950Z 
2025-09-07T06:09:29.0898512Z     valid_char_regex = r"^[a-zA-Z0-9]([\w-]*[a-zA-Z0-9])?$"
2025-09-07T06:09:29.0899338Z     valid = bool(re.match(valid_char_regex, experiment_name))
2025-09-07T06:09:29.0900363Z 
2025-09-07T06:09:29.0900881Z     if valid:
2025-09-07T06:09:29.0901381Z         return True
2025-09-07T06:09:29.0901890Z 
2025-09-07T06:09:29.0902323Z     log.error(
2025-09-07T06:09:29.0903927Z         f"Invalid experiment name: {experiment_name}. Experiment names should only contain alphanumeric characters, '_', and '-'. They cannot contain spaces, and the special characters '_' and '-' cannot be the first or last characters."
2025-09-07T06:09:29.0905587Z     )
2025-09-07T06:09:29.0906228Z     return False
2025-09-07T06:09:29.0906751Z 
2025-09-07T06:09:29.0907173Z 
2025-09-07T06:09:29.0907796Z def parse_settings_from_text(settings_text: str) -> Settings:
2025-09-07T06:09:29.0908573Z     """
2025-09-07T06:09:29.0909287Z     Parse the experiments from the issue body into a list of ExperimentSettings
2025-09-07T06:09:29.0910094Z     """
2025-09-07T06:09:29.0910563Z     try:
2025-09-07T06:09:29.0911026Z         if settings_text:
2025-09-07T06:09:29.0911878Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2025-09-07T06:09:29.0912791Z             # for easy reading
2025-09-07T06:09:29.0913709Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2025-09-07T06:09:29.0914704Z             #       the backtick character in shell commands.
2025-09-07T06:09:29.0915415Z             backtick = chr(96)  # backtick character
2025-09-07T06:09:29.0916328Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2025-09-07T06:09:29.0917103Z             settings = load_yaml(settings_text)
2025-09-07T06:09:29.0917701Z 
2025-09-07T06:09:29.0918411Z             # For now we just load experiments. We can expand this if/when we add more settings
2025-09-07T06:09:29.0919406Z             experiments = {}
2025-09-07T06:09:29.0919951Z 
2025-09-07T06:09:29.0920616Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2025-09-07T06:09:29.0921496Z                 if not is_valid_experiment_name(exp_name):
2025-09-07T06:09:29.0922736Z                     # Exclude invalid experiments from the list. We log an error, but don't raise an exception so that other experiments can still be processed.
2025-09-07T06:09:29.0923894Z                     continue
2025-09-07T06:09:29.0924442Z 
2025-09-07T06:09:29.0924902Z                 valid_settings = {}
2025-09-07T06:09:29.0925543Z                 for setting in exp_settings:
2025-09-07T06:09:29.0926317Z                     if setting not in Experiment._fields:
2025-09-07T06:09:29.0926981Z                         log.warning(
2025-09-07T06:09:29.0927824Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2025-09-07T06:09:29.0928633Z                         )
2025-09-07T06:09:29.0929180Z                     else:
2025-09-07T06:09:29.0929834Z                         valid_settings[setting] = exp_settings[setting]
2025-09-07T06:09:29.0930494Z 
2025-09-07T06:09:29.0931065Z                 experiments[exp_name] = Experiment(**valid_settings)
2025-09-07T06:09:29.0931798Z             return Settings(experiments)
2025-09-07T06:09:29.0932385Z 
2025-09-07T06:09:29.0932832Z     except Exception:
2025-09-07T06:09:29.0933437Z         log.exception("Failed to parse settings")
2025-09-07T06:09:29.0934058Z 
2025-09-07T06:09:29.0934506Z     return Settings()
2025-09-07T06:09:29.0935024Z 
2025-09-07T06:09:29.0935455Z 
2025-09-07T06:09:29.0936241Z def parse_settings(rollout_state: str) -> Settings:
2025-09-07T06:09:29.0936951Z     """
2025-09-07T06:09:29.0937530Z     Parse settings, if any, from the rollout state.
2025-09-07T06:09:29.0938162Z 
2025-09-07T06:09:29.0938809Z     If the issue body contains "---" then the text above that is the settings
2025-09-07T06:09:29.0939670Z     and the text below is the list of opted in users.
2025-09-07T06:09:29.0940314Z 
2025-09-07T06:09:29.0941010Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2025-09-07T06:09:29.0941834Z     """
2025-09-07T06:09:29.0942535Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2025-09-07T06:09:29.0943416Z     return parse_settings_from_text(settings_text)
2025-09-07T06:09:29.0944051Z 
2025-09-07T06:09:29.0944464Z 
2025-09-07T06:09:29.0945018Z def parse_users(rollout_state: str) -> UserOptins:
2025-09-07T06:09:29.0945682Z     """
2025-09-07T06:09:29.0946277Z     Parse users from the rollout state.
2025-09-07T06:09:29.0946871Z 
2025-09-07T06:09:29.0947286Z     """
2025-09-07T06:09:29.0947941Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2025-09-07T06:09:29.0948790Z     return parse_user_opt_in_from_text(users_text)
2025-09-07T06:09:29.0949434Z 
2025-09-07T06:09:29.0949861Z 
2025-09-07T06:09:29.0950600Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-09-07T06:09:29.0951439Z     """
2025-09-07T06:09:29.0951983Z     Check if a user is opted into an experiment
2025-09-07T06:09:29.0952604Z     """
2025-09-07T06:09:29.0953183Z     return experiment_name in user_optins.get(user, [])
2025-09-07T06:09:29.0953846Z 
2025-09-07T06:09:29.0954416Z 
2025-09-07T06:09:29.0955158Z def is_user_opted_out(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-09-07T06:09:29.0956017Z     """
2025-09-07T06:09:29.0956714Z     Check if a user explicitly opted out of an experiment
2025-09-07T06:09:29.0957383Z     """
2025-09-07T06:09:29.0958011Z     # if the experiment is prefixed with a "-", then it's an opt-out
2025-09-07T06:09:29.0958812Z     experiment_optout = "-" + experiment_name
2025-09-07T06:09:29.0959567Z     if experiment_optout not in user_optins.get(user, []):
2025-09-07T06:09:29.0960282Z         return False
2025-09-07T06:09:29.0960797Z 
2025-09-07T06:09:29.0961354Z     if is_user_opted_in(user, user_optins, experiment_name):
2025-09-07T06:09:29.0962050Z         log.warning(
2025-09-07T06:09:29.0963003Z             f"User {user} is opted into experiment {experiment_name}, but also opted out of it. Defaulting to opting out"
2025-09-07T06:09:29.0963975Z         )
2025-09-07T06:09:29.0964429Z 
2025-09-07T06:09:29.0964871Z     return True
2025-09-07T06:09:29.0965375Z 
2025-09-07T06:09:29.0965791Z 
2025-09-07T06:09:29.0966342Z def get_runner_prefix(
2025-09-07T06:09:29.0966897Z     rollout_state: str,
2025-09-07T06:09:29.0967489Z     workflow_requestors: Iterable[str],
2025-09-07T06:09:29.0968090Z     branch: str,
2025-09-07T06:09:29.0968730Z     eligible_experiments: frozenset[str] = frozenset(),
2025-09-07T06:09:29.0969527Z     opt_out_experiments: frozenset[str] = frozenset(),
2025-09-07T06:09:29.0970219Z     is_canary: bool = False,
2025-09-07T06:09:29.0970783Z ) -> str:
2025-09-07T06:09:29.0971330Z     settings = parse_settings(rollout_state)
2025-09-07T06:09:29.0972033Z     user_optins = parse_users(rollout_state)
2025-09-07T06:09:29.0972647Z 
2025-09-07T06:09:29.0973221Z     fleet_prefix = ""
2025-09-07T06:09:29.0973775Z     prefixes = []
2025-09-07T06:09:29.0974551Z     for experiment_name, experiment_settings in settings.experiments.items():
2025-09-07T06:09:29.0975610Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2025-09-07T06:09:29.0976516Z             log.info(
2025-09-07T06:09:29.0977336Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2025-09-07T06:09:29.0978178Z             )
2025-09-07T06:09:29.0978681Z             continue
2025-09-07T06:09:29.0979170Z 
2025-09-07T06:09:29.0979634Z         if opt_out_experiments:
2025-09-07T06:09:29.0980305Z             if experiment_name in opt_out_experiments:
2025-09-07T06:09:29.0981051Z                 opt_out_exp_list = ", ".join(opt_out_experiments)
2025-09-07T06:09:29.0981748Z                 log.info(
2025-09-07T06:09:29.0982811Z                     f"Skipping experiment '{experiment_name}', as this workflow has opted-out (opted out experiments are: {opt_out_exp_list})"
2025-09-07T06:09:29.0983884Z                 )
2025-09-07T06:09:29.0984390Z                 continue
2025-09-07T06:09:29.0984922Z 
2025-09-07T06:09:29.0985385Z         if eligible_experiments:
2025-09-07T06:09:29.0986209Z             if experiment_name not in eligible_experiments:
2025-09-07T06:09:29.0986961Z                 exp_list = ", ".join(eligible_experiments)
2025-09-07T06:09:29.0987605Z                 log.info(
2025-09-07T06:09:29.0988515Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2025-09-07T06:09:29.0989454Z                 )
2025-09-07T06:09:29.0989959Z                 continue
2025-09-07T06:09:29.0990704Z         elif not experiment_settings.default:
2025-09-07T06:09:29.0991328Z             log.info(
2025-09-07T06:09:29.0992109Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2025-09-07T06:09:29.0992923Z             )
2025-09-07T06:09:29.0993413Z             continue
2025-09-07T06:09:29.0993910Z 
2025-09-07T06:09:29.0994502Z         # Is any workflow_requestor opted out to this experiment?
2025-09-07T06:09:29.0995217Z         opted_out_users = [
2025-09-07T06:09:29.0995780Z             requestor
2025-09-07T06:09:29.0996474Z             for requestor in workflow_requestors
2025-09-07T06:09:29.0997256Z             if is_user_opted_out(requestor, user_optins, experiment_name)
2025-09-07T06:09:29.0997976Z         ]
2025-09-07T06:09:29.0998436Z 
2025-09-07T06:09:29.0998902Z         if opted_out_users:
2025-09-07T06:09:29.0999487Z             log.info(
2025-09-07T06:09:29.1000252Z                 f"{', '.join(opted_out_users)} have opted out of experiment {experiment_name}."
2025-09-07T06:09:29.1001047Z             )
2025-09-07T06:09:29.1001540Z             continue
2025-09-07T06:09:29.1002060Z 
2025-09-07T06:09:29.1002613Z         # Is any workflow_requestor opted in to this experiment?
2025-09-07T06:09:29.1003325Z         opted_in_users = [
2025-09-07T06:09:29.1003898Z             requestor
2025-09-07T06:09:29.1004480Z             for requestor in workflow_requestors
2025-09-07T06:09:29.1005253Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2025-09-07T06:09:29.1005954Z         ]
2025-09-07T06:09:29.1006511Z 
2025-09-07T06:09:29.1006955Z         enabled = False
2025-09-07T06:09:29.1007509Z         if opted_in_users:
2025-09-07T06:09:29.1008189Z             log.info(
2025-09-07T06:09:29.1008933Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2025-09-07T06:09:29.1009709Z             )
2025-09-07T06:09:29.1010210Z             enabled = True
2025-09-07T06:09:29.1010741Z 
2025-09-07T06:09:29.1011239Z         elif experiment_settings.rollout_perc:
2025-09-07T06:09:29.1012187Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2025-09-07T06:09:29.1013234Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2025-09-07T06:09:29.1013998Z                 log.info(
2025-09-07T06:09:29.1015012Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2025-09-07T06:09:29.1016026Z                 )
2025-09-07T06:09:29.1016672Z                 enabled = True
2025-09-07T06:09:29.1017222Z 
2025-09-07T06:09:29.1017672Z         if enabled:
2025-09-07T06:09:29.1018226Z             label = experiment_name
2025-09-07T06:09:29.1018891Z             if experiment_name == LF_FLEET_EXPERIMENT:
2025-09-07T06:09:29.1019826Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2025-09-07T06:09:29.1020803Z                 #  - If it's enabled, then we always list it's prefix first
2025-09-07T06:09:29.1021687Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2025-09-07T06:09:29.1022443Z                 if is_canary:
2025-09-07T06:09:29.1023047Z                     label += CANARY_FLEET_SUFFIX
2025-09-07T06:09:29.1023694Z                 fleet_prefix = label
2025-09-07T06:09:29.1024268Z             else:
2025-09-07T06:09:29.1024963Z                 prefixes.append(label)
2025-09-07T06:09:29.1025562Z 
2025-09-07T06:09:29.1026000Z     if len(prefixes) > 1:
2025-09-07T06:09:29.1026653Z         log.error(
2025-09-07T06:09:29.1027829Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2025-09-07T06:09:29.1029043Z         )
2025-09-07T06:09:29.1029530Z         prefixes = prefixes[:1]
2025-09-07T06:09:29.1030093Z 
2025-09-07T06:09:29.1030548Z     # Fleet always comes first
2025-09-07T06:09:29.1031141Z     if fleet_prefix:
2025-09-07T06:09:29.1031719Z         prefixes.insert(0, fleet_prefix)
2025-09-07T06:09:29.1032315Z 
2025-09-07T06:09:29.1032865Z     return ".".join(prefixes) + "." if prefixes else ""
2025-09-07T06:09:29.1033511Z 
2025-09-07T06:09:29.1033947Z 
2025-09-07T06:09:29.1034687Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2025-09-07T06:09:29.1035549Z     """
2025-09-07T06:09:29.1036331Z     Gets the first comment of the issue, which contains the desired rollout state.
2025-09-07T06:09:29.1037140Z 
2025-09-07T06:09:29.1037826Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2025-09-07T06:09:29.1038613Z     """
2025-09-07T06:09:29.1039120Z     gh = get_gh_client(github_token)
2025-09-07T06:09:29.1039781Z     issue = get_issue(gh, repo, issue_num)
2025-09-07T06:09:29.1040534Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2025-09-07T06:09:29.1041216Z 
2025-09-07T06:09:29.1041642Z 
2025-09-07T06:09:29.1042338Z def download_json(url: str, headers: dict[str, str], num_retries: int = 3) -> Any:
2025-09-07T06:09:29.1043324Z     for _ in range(num_retries):
2025-09-07T06:09:29.1043917Z         try:
2025-09-07T06:09:29.1044465Z             req = Request(url=url, headers=headers)
2025-09-07T06:09:29.1045232Z             content = urlopen(req, timeout=5).read().decode("utf-8")
2025-09-07T06:09:29.1045968Z             return json.loads(content)
2025-09-07T06:09:29.1046704Z         except Exception as e:
2025-09-07T06:09:29.1047375Z             log.warning(f"Could not download {url}: {e}")
2025-09-07T06:09:29.1048004Z 
2025-09-07T06:09:29.1048675Z     log.warning(f"All {num_retries} retries exhausted, downloading {url} failed")
2025-09-07T06:09:29.1049472Z     return {}
2025-09-07T06:09:29.1050033Z 
2025-09-07T06:09:29.1050458Z 
2025-09-07T06:09:29.1050877Z @cache
2025-09-07T06:09:29.1051625Z def get_pr_info(github_repo: str, github_token: str, pr_number: int) -> dict[str, Any]:
2025-09-07T06:09:29.1052468Z     """
2025-09-07T06:09:29.1052972Z     Dynamically get PR information
2025-09-07T06:09:29.1053548Z     """
2025-09-07T06:09:29.1054170Z     github_api = f"https://api.github.com/repos/{github_repo}"
2025-09-07T06:09:29.1054887Z     headers = {
2025-09-07T06:09:29.1055469Z         "Accept": "application/vnd.github.v3+json",
2025-09-07T06:09:29.1056284Z         "Authorization": f"token {github_token}",
2025-09-07T06:09:29.1056908Z     }
2025-09-07T06:09:29.1057467Z     json_response: dict[str, Any] = download_json(
2025-09-07T06:09:29.1058171Z         url=f"{github_api}/issues/{pr_number}",
2025-09-07T06:09:29.1058814Z         headers=headers,
2025-09-07T06:09:29.1059354Z     )
2025-09-07T06:09:29.1059795Z 
2025-09-07T06:09:29.1060248Z     if not json_response:
2025-09-07T06:09:29.1060957Z         log.warning(f"Failed to get the labels for #{pr_number}")
2025-09-07T06:09:29.1061806Z         return {}
2025-09-07T06:09:29.1062322Z 
2025-09-07T06:09:29.1062770Z     return json_response
2025-09-07T06:09:29.1063301Z 
2025-09-07T06:09:29.1063724Z 
2025-09-07T06:09:29.1064414Z def get_labels(github_repo: str, github_token: str, pr_number: int) -> set[str]:
2025-09-07T06:09:29.1065243Z     """
2025-09-07T06:09:29.1065887Z     Dynamically get the latest list of labels from the pull request
2025-09-07T06:09:29.1066775Z     """
2025-09-07T06:09:29.1067384Z     pr_info = get_pr_info(github_repo, github_token, pr_number)
2025-09-07T06:09:29.1068088Z     return {
2025-09-07T06:09:29.1068796Z         label.get("name") for label in pr_info.get("labels", []) if label.get("name")
2025-09-07T06:09:29.1069579Z     }
2025-09-07T06:09:29.1070031Z 
2025-09-07T06:09:29.1070459Z 
2025-09-07T06:09:29.1070918Z def main() -> None:
2025-09-07T06:09:29.1071462Z     args = parse_args()
2025-09-07T06:09:29.1071999Z 
2025-09-07T06:09:29.1072523Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2025-09-07T06:09:29.1073152Z 
2025-09-07T06:09:29.1073627Z     # Check if the PR is opt-out
2025-09-07T06:09:29.1074211Z     if args.pr_number:
2025-09-07T06:09:29.1075005Z         labels = get_labels(args.github_repo, args.github_token, int(args.pr_number))
2025-09-07T06:09:29.1075850Z         if OPT_OUT_LABEL in labels:
2025-09-07T06:09:29.1076549Z             log.info(
2025-09-07T06:09:29.1077366Z                 f"Opt-out runner determinator because #{args.pr_number} has {OPT_OUT_LABEL} label"
2025-09-07T06:09:29.1078215Z             )
2025-09-07T06:09:29.1078894Z             set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-09-07T06:09:29.1079680Z             sys.exit()
2025-09-07T06:09:29.1080328Z 
2025-09-07T06:09:29.1080767Z     try:
2025-09-07T06:09:29.1081324Z         rollout_state = get_rollout_state_from_issue(
2025-09-07T06:09:29.1082142Z             args.github_token, args.github_issue_repo, args.github_issue
2025-09-07T06:09:29.1082866Z         )
2025-09-07T06:09:29.1083323Z 
2025-09-07T06:09:29.1083809Z         username = get_potential_pr_author(
2025-09-07T06:09:29.1084448Z             args.github_token,
2025-09-07T06:09:29.1085044Z             args.github_repo,
2025-09-07T06:09:29.1085632Z             args.github_actor,
2025-09-07T06:09:29.1086330Z             args.github_ref_type,
2025-09-07T06:09:29.1086945Z             args.github_branch,
2025-09-07T06:09:29.1087511Z         )
2025-09-07T06:09:29.1087963Z 
2025-09-07T06:09:29.1088544Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2025-09-07T06:09:29.1089232Z 
2025-09-07T06:09:29.1089745Z         runner_label_prefix = get_runner_prefix(
2025-09-07T06:09:29.1090387Z             rollout_state,
2025-09-07T06:09:29.1091005Z             (args.github_issue_owner, username),
2025-09-07T06:09:29.1091642Z             args.github_branch,
2025-09-07T06:09:29.1092266Z             args.eligible_experiments,
2025-09-07T06:09:29.1092904Z             args.opt_out_experiments,
2025-09-07T06:09:29.1093499Z             is_canary,
2025-09-07T06:09:29.1094024Z         )
2025-09-07T06:09:29.1094515Z 
2025-09-07T06:09:29.1094973Z     except Exception as e:
2025-09-07T06:09:29.1095523Z         log.error(
2025-09-07T06:09:29.1096434Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2025-09-07T06:09:29.1097284Z         )
2025-09-07T06:09:29.1097865Z 
2025-09-07T06:09:29.1098509Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-09-07T06:09:29.1099250Z 
2025-09-07T06:09:29.1099676Z 
2025-09-07T06:09:29.1100122Z if __name__ == "__main__":
2025-09-07T06:09:29.1100663Z     main()
2025-09-07T06:09:29.1101117Z 
2025-09-07T06:09:29.1101553Z EOF
2025-09-07T06:09:29.1101979Z 
2025-09-07T06:09:29.1102444Z cat runner_determinator.py
2025-09-07T06:09:29.2568743Z shell: /usr/bin/bash -e {0}
2025-09-07T06:09:29.2569715Z env:
2025-09-07T06:09:29.2570549Z   GITHUB_TOKEN: ***
2025-09-07T06:09:29.2571102Z   ISSUE_NUMBER: 5132
2025-09-07T06:09:29.2571676Z   TRIGGERING_ACTOR: pytorchmergebot
2025-09-07T06:09:29.2572344Z   ISSUE_OWNER: 
2025-09-07T06:09:29.2572863Z   CHECK_EXPERIMENTS: 
2025-09-07T06:09:29.2573430Z   OPT_OUT_EXPERIMENTS: lf
2025-09-07T06:09:29.2574008Z   PR_NUMBER: 
2025-09-07T06:09:29.2574547Z ##[endgroup]
2025-09-07T06:09:29.2781048Z # flake8: noqa: G004
2025-09-07T06:09:29.2781386Z 
2025-09-07T06:09:29.2781835Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml
2025-09-07T06:09:29.2782806Z #       must be kept in sync. You can do it easily by running the following command:
2025-09-07T06:09:29.2783616Z #           python .github/scripts/update_runner_determinator.py
2025-09-07T06:09:29.2784070Z 
2025-09-07T06:09:29.2784223Z """
2025-09-07T06:09:29.2784807Z This runner determinator is used to determine which set of runners to run a
2025-09-07T06:09:29.2785684Z GitHub job on. It uses the first comment of a GitHub issue (by default
2025-09-07T06:09:29.2786894Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration
2025-09-07T06:09:29.2787711Z of which runners should be used to run which job.
2025-09-07T06:09:29.2788119Z 
2025-09-07T06:09:29.2788501Z The configuration has two parts, the settings and a list of opted-in users,
2025-09-07T06:09:29.2789615Z separated by a line containing "---".  If the line is not present, the
2025-09-07T06:09:29.2790553Z settings are considered to be empty with only the second part, the user
2025-09-07T06:09:29.2791313Z list, defined.
2025-09-07T06:09:29.2791566Z 
2025-09-07T06:09:29.2791954Z The first part is a YAML block that defines the rollout settings. This can be
2025-09-07T06:09:29.2792953Z used to define any settings that are needed to determine which runners to use.
2025-09-07T06:09:29.2793831Z It's fields are defined by the RolloutSettings class below.
2025-09-07T06:09:29.2794310Z 
2025-09-07T06:09:29.2794705Z The second part is a list of users who are explicitly opted in to the LF fleet.
2025-09-07T06:09:29.2795626Z The user list is also a comma separated list of additional features or
2025-09-07T06:09:29.2796690Z experiments which the user could be opted in to.
2025-09-07T06:09:29.2797100Z 
2025-09-07T06:09:29.2797314Z The user list has the following rules:
2025-09-07T06:09:29.2797666Z 
2025-09-07T06:09:29.2798001Z - Users are GitHub usernames, which must start with the @ prefix
2025-09-07T06:09:29.2798869Z - Each user is also a comma-separated list of features/experiments to enable
2025-09-07T06:09:29.2799636Z - A "#" prefix opts the user out of all experiments
2025-09-07T06:09:29.2800034Z 
2025-09-07T06:09:29.2800202Z Example config:
2025-09-07T06:09:29.2800659Z     # A list of experiments that can be opted into.
2025-09-07T06:09:29.2801328Z     # This defines the behavior they'll induce when opted into.
2025-09-07T06:09:29.2801953Z     # Expected syntax is:
2025-09-07T06:09:29.2802590Z     #   [experiment_name]: # Name of the experiment. Also used for the label prefix.
2025-09-07T06:09:29.2803564Z     #      rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in.
2025-09-07T06:09:29.2804182Z 
2025-09-07T06:09:29.2804357Z     experiments:
2025-09-07T06:09:29.2804751Z       lf:
2025-09-07T06:09:29.2805128Z         rollout_percent: 25
2025-09-07T06:09:29.2805586Z         all_branches: false
2025-09-07T06:09:29.2806427Z         default: true
2025-09-07T06:09:29.2806850Z     ---
2025-09-07T06:09:29.2807060Z 
2025-09-07T06:09:29.2807223Z     # Opt-ins:
2025-09-07T06:09:29.2807810Z     # Users can opt into the LF fleet by adding their GitHub username to this list
2025-09-07T06:09:29.2808735Z     # and specifying experiments to enable in a comma-separated list.
2025-09-07T06:09:29.2809504Z     # To always opt out of an experiment, prefix it with a "-".
2025-09-07T06:09:29.2810157Z     # Experiments should be from the above list.
2025-09-07T06:09:29.2810541Z 
2025-09-07T06:09:29.2810727Z     @User1,-lf,split_build
2025-09-07T06:09:29.2811160Z     @User2,lf
2025-09-07T06:09:29.2811546Z     @User3,split_build
2025-09-07T06:09:29.2811945Z """
2025-09-07T06:09:29.2812148Z 
2025-09-07T06:09:29.2812311Z import json
2025-09-07T06:09:29.2812678Z import logging
2025-09-07T06:09:29.2813065Z import os
2025-09-07T06:09:29.2813424Z import random
2025-09-07T06:09:29.2813801Z import re
2025-09-07T06:09:29.2814162Z import sys
2025-09-07T06:09:29.2814567Z from argparse import ArgumentParser
2025-09-07T06:09:29.2815098Z from collections.abc import Iterable
2025-09-07T06:09:29.2815616Z from functools import cache
2025-09-07T06:09:29.2816287Z from logging import LogRecord
2025-09-07T06:09:29.2816863Z from typing import Any, NamedTuple
2025-09-07T06:09:29.2817402Z from urllib.request import Request, urlopen
2025-09-07T06:09:29.2817775Z 
2025-09-07T06:09:29.2817938Z import yaml
2025-09-07T06:09:29.2818329Z from github import Auth, Github
2025-09-07T06:09:29.2818807Z from github.Issue import Issue
2025-09-07T06:09:29.2819122Z 
2025-09-07T06:09:29.2819129Z 
2025-09-07T06:09:29.2819350Z DEFAULT_LABEL_PREFIX = ""  # use meta runners
2025-09-07T06:09:29.2820040Z WORKFLOW_LABEL_LF = "lf."  # use runners from the linux foundation
2025-09-07T06:09:29.2820919Z WORKFLOW_LABEL_LF_CANARY = "lf.c."  # use canary runners from the linux foundation
2025-09-07T06:09:29.2821484Z 
2025-09-07T06:09:29.2821716Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "")
2025-09-07T06:09:29.2822438Z GH_OUTPUT_KEY_AMI = "runner-ami"
2025-09-07T06:09:29.2822968Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type"
2025-09-07T06:09:29.2823515Z OPT_OUT_LABEL = "no-runner-experiments"
2025-09-07T06:09:29.2823877Z 
2025-09-07T06:09:29.2824076Z SETTING_EXPERIMENTS = "experiments"
2025-09-07T06:09:29.2824411Z 
2025-09-07T06:09:29.2824599Z LF_FLEET_EXPERIMENT = "lf"
2025-09-07T06:09:29.2825056Z CANARY_FLEET_SUFFIX = ".c"
2025-09-07T06:09:29.2825340Z 
2025-09-07T06:09:29.2825353Z 
2025-09-07T06:09:29.2825543Z class Experiment(NamedTuple):
2025-09-07T06:09:29.2826023Z     rollout_perc: float = (
2025-09-07T06:09:29.2826927Z         0  # Percentage of workflows to experiment on when user is not opted-in.
2025-09-07T06:09:29.2827608Z     )
2025-09-07T06:09:29.2827980Z     all_branches: bool = (
2025-09-07T06:09:29.2828611Z         False  # If True, the experiment is also enabled on the exception branches
2025-09-07T06:09:29.2829281Z     )
2025-09-07T06:09:29.2829645Z     default: bool = (
2025-09-07T06:09:29.2830210Z         True  # If True, the experiment is enabled by default for all queries
2025-09-07T06:09:29.2830852Z     )
2025-09-07T06:09:29.2831045Z 
2025-09-07T06:09:29.2831223Z     # Add more fields as needed
2025-09-07T06:09:29.2831535Z 
2025-09-07T06:09:29.2831542Z 
2025-09-07T06:09:29.2831723Z class Settings(NamedTuple):
2025-09-07T06:09:29.2832167Z     """
2025-09-07T06:09:29.2832609Z     Settings for the experiments that can be opted into.
2025-09-07T06:09:29.2833183Z     """
2025-09-07T06:09:29.2833370Z 
2025-09-07T06:09:29.2833574Z     experiments: dict[str, Experiment] = {}
2025-09-07T06:09:29.2833940Z 
2025-09-07T06:09:29.2833953Z 
2025-09-07T06:09:29.2834159Z class ColorFormatter(logging.Formatter):
2025-09-07T06:09:29.2834774Z     """Color codes the log messages based on the log level"""
2025-09-07T06:09:29.2835213Z 
2025-09-07T06:09:29.2835370Z     COLORS = {
2025-09-07T06:09:29.2835769Z         "WARNING": "\033[33m",  # Yellow
2025-09-07T06:09:29.2836634Z         "ERROR": "\033[31m",  # Red
2025-09-07T06:09:29.2837140Z         "CRITICAL": "\033[31m",  # Red
2025-09-07T06:09:29.2837652Z         "INFO": "\033[0m",  # Reset
2025-09-07T06:09:29.2838138Z         "DEBUG": "\033[0m",  # Reset
2025-09-07T06:09:29.2838697Z     }
2025-09-07T06:09:29.2838900Z 
2025-09-07T06:09:29.2839119Z     def format(self, record: LogRecord) -> str:
2025-09-07T06:09:29.2839866Z         log_color = self.COLORS.get(record.levelname, "\033[0m")  # Default to reset
2025-09-07T06:09:29.2840645Z         record.msg = f"{log_color}{record.msg}\033[0m"
2025-09-07T06:09:29.2841228Z         return super().format(record)
2025-09-07T06:09:29.2841562Z 
2025-09-07T06:09:29.2841568Z 
2025-09-07T06:09:29.2841760Z handler = logging.StreamHandler()
2025-09-07T06:09:29.2842468Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s"))
2025-09-07T06:09:29.2843029Z 
2025-09-07T06:09:29.2843271Z log = logging.getLogger(os.path.basename(__file__))
2025-09-07T06:09:29.2843851Z log.addHandler(handler)
2025-09-07T06:09:29.2844299Z log.setLevel(logging.INFO)
2025-09-07T06:09:29.2844593Z 
2025-09-07T06:09:29.2844600Z 
2025-09-07T06:09:29.2844847Z def set_github_output(key: str, value: str) -> None:
2025-09-07T06:09:29.2845405Z     """
2025-09-07T06:09:29.2845897Z     Defines outputs of the github action that invokes this script
2025-09-07T06:09:29.2846725Z     """
2025-09-07T06:09:29.2847095Z     if not GITHUB_OUTPUT:
2025-09-07T06:09:29.2848179Z         # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice
2025-09-07T06:09:29.2849300Z         log.warning(
2025-09-07T06:09:29.2850162Z             "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method."
2025-09-07T06:09:29.2851102Z         )
2025-09-07T06:09:29.2861269Z         print(f"::set-output name={key}::{value}")
2025-09-07T06:09:29.2861899Z         return
2025-09-07T06:09:29.2862140Z 
2025-09-07T06:09:29.2862542Z     with open(GITHUB_OUTPUT, "a") as f:
2025-09-07T06:09:29.2863155Z         log.info(f"Setting output: {key}='{value}'")
2025-09-07T06:09:29.2863724Z         f.write(f"{key}={value}\n")
2025-09-07T06:09:29.2864064Z 
2025-09-07T06:09:29.2864072Z 
2025-09-07T06:09:29.2864379Z def _str_comma_separated_to_set(value: str) -> frozenset[str]:
2025-09-07T06:09:29.2865018Z     return frozenset(
2025-09-07T06:09:29.2865625Z         filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(",")))
2025-09-07T06:09:29.2866618Z     )
2025-09-07T06:09:29.2866822Z 
2025-09-07T06:09:29.2866829Z 
2025-09-07T06:09:29.2867009Z def parse_args() -> Any:
2025-09-07T06:09:29.2867572Z     parser = ArgumentParser("Get dynamic rollout settings")
2025-09-07T06:09:29.2868439Z     parser.add_argument("--github-token", type=str, required=True, help="GitHub token")
2025-09-07T06:09:29.2869226Z     parser.add_argument(
2025-09-07T06:09:29.2869689Z         "--github-issue-repo",
2025-09-07T06:09:29.2870147Z         type=str,
2025-09-07T06:09:29.2870562Z         required=False,
2025-09-07T06:09:29.2871009Z         default="pytorch/test-infra",
2025-09-07T06:09:29.2871547Z         help="GitHub repo to get the issue",
2025-09-07T06:09:29.2872053Z     )
2025-09-07T06:09:29.2872419Z     parser.add_argument(
2025-09-07T06:09:29.2872859Z         "--github-repo",
2025-09-07T06:09:29.2873284Z         type=str,
2025-09-07T06:09:29.2873676Z         required=True,
2025-09-07T06:09:29.2874139Z         help="GitHub repo where CI is running",
2025-09-07T06:09:29.2874665Z     )
2025-09-07T06:09:29.2875035Z     parser.add_argument(
2025-09-07T06:09:29.2875660Z         "--github-issue", type=int, required=True, help="GitHub issue number"
2025-09-07T06:09:29.2876562Z     )
2025-09-07T06:09:29.2876929Z     parser.add_argument(
2025-09-07T06:09:29.2930718Z         "--github-actor", type=str, required=True, help="GitHub triggering_actor"
2025-09-07T06:09:29.2931471Z     )
2025-09-07T06:09:29.2931854Z     parser.add_argument(
2025-09-07T06:09:29.2932760Z         "--github-issue-owner", type=str, required=True, help="GitHub issue owner"
2025-09-07T06:09:29.2933461Z     )
2025-09-07T06:09:29.2933835Z     parser.add_argument(
2025-09-07T06:09:29.2934483Z         "--github-branch", type=str, required=True, help="Current GitHub branch or tag"
2025-09-07T06:09:29.2935206Z     )
2025-09-07T06:09:29.2935566Z     parser.add_argument(
2025-09-07T06:09:29.2936021Z         "--github-ref-type",
2025-09-07T06:09:29.2936656Z         type=str,
2025-09-07T06:09:29.2937054Z         required=True,
2025-09-07T06:09:29.2937531Z         help="Current GitHub ref type, branch or tag",
2025-09-07T06:09:29.2938082Z     )
2025-09-07T06:09:29.2938447Z     parser.add_argument(
2025-09-07T06:09:29.2938893Z         "--eligible-experiments",
2025-09-07T06:09:29.2939411Z         type=_str_comma_separated_to_set,
2025-09-07T06:09:29.2939914Z         required=False,
2025-09-07T06:09:29.2940324Z         default="",
2025-09-07T06:09:29.2941152Z         help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked",
2025-09-07T06:09:29.2942061Z     )
2025-09-07T06:09:29.2942421Z     parser.add_argument(
2025-09-07T06:09:29.2942874Z         "--opt-out-experiments",
2025-09-07T06:09:29.2943367Z         type=_str_comma_separated_to_set,
2025-09-07T06:09:29.2943877Z         required=False,
2025-09-07T06:09:29.2944284Z         default="",
2025-09-07T06:09:29.2944662Z         help=(
2025-09-07T06:09:29.2945323Z             "comma separated list of experiments to opt-out of. If unset, no opt-outs will occur. "
2025-09-07T06:09:29.2946542Z             "If the same experiment is listed both here and in '--eligible-experiments' opt-out will take priority."
2025-09-07T06:09:29.2947364Z         ),
2025-09-07T06:09:29.2947705Z     )
2025-09-07T06:09:29.2948067Z     parser.add_argument(
2025-09-07T06:09:29.2948489Z         "--pr-number",
2025-09-07T06:09:29.2949064Z         type=str,
2025-09-07T06:09:29.2949726Z         required=False,
2025-09-07T06:09:29.2950411Z         default="",
2025-09-07T06:09:29.2951187Z         help="the optional PR number where this is run",
2025-09-07T06:09:29.2951924Z     )
2025-09-07T06:09:29.2952146Z 
2025-09-07T06:09:29.2952345Z     return parser.parse_args()
2025-09-07T06:09:29.2952772Z 
2025-09-07T06:09:29.2952779Z 
2025-09-07T06:09:29.2953190Z def get_gh_client(github_token: str) -> Github:  # type: ignore[no-any-unimported]
2025-09-07T06:09:29.2954146Z     auth = Auth.Token(github_token)
2025-09-07T06:09:29.2954801Z     return Github(auth=auth)
2025-09-07T06:09:29.2955108Z 
2025-09-07T06:09:29.2955115Z 
2025-09-07T06:09:29.2955557Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue:  # type: ignore[no-any-unimported]
2025-09-07T06:09:29.2956476Z     repo = gh.get_repo(repo)
2025-09-07T06:09:29.2956969Z     return repo.get_issue(number=issue_num)
2025-09-07T06:09:29.2957332Z 
2025-09-07T06:09:29.2957339Z 
2025-09-07T06:09:29.2957524Z def get_potential_pr_author(
2025-09-07T06:09:29.2958166Z     github_token: str, repo: str, username: str, ref_type: str, ref_name: str
2025-09-07T06:09:29.2958835Z ) -> str:
2025-09-07T06:09:29.2959346Z     # If the trigger was a new tag added by a bot, this is a ciflow case
2025-09-07T06:09:29.2960134Z     # Fetch the actual username from the original PR. The PR number is
2025-09-07T06:09:29.2960859Z     # embedded in the tag name: ciflow/<name>/<pr-number>
2025-09-07T06:09:29.2961269Z 
2025-09-07T06:09:29.2961454Z     gh = get_gh_client(github_token)
2025-09-07T06:09:29.2961787Z 
2025-09-07T06:09:29.2962050Z     if username == "pytorch-bot[bot]" and ref_type == "tag":
2025-09-07T06:09:29.2962658Z         split_tag = ref_name.split("/")
2025-09-07T06:09:29.2963153Z         if (
2025-09-07T06:09:29.2963533Z             len(split_tag) == 3
2025-09-07T06:09:29.2963997Z             and split_tag[0] == "ciflow"
2025-09-07T06:09:29.2964514Z             and split_tag[2].isnumeric()
2025-09-07T06:09:29.2964994Z         ):
2025-09-07T06:09:29.2965365Z             pr_number = split_tag[2]
2025-09-07T06:09:29.2966028Z             try:
2025-09-07T06:09:29.2966560Z                 repository = gh.get_repo(repo)
2025-09-07T06:09:29.2967156Z                 pull = repository.get_pull(number=int(pr_number))
2025-09-07T06:09:29.2967748Z             except Exception as e:
2025-09-07T06:09:29.2968254Z                 raise Exception(  # noqa: TRY002
2025-09-07T06:09:29.2968911Z                     f"issue with pull request {pr_number} from repo {repository}"
2025-09-07T06:09:29.2969539Z                 ) from e
2025-09-07T06:09:29.2970057Z             return pull.user.login  # type: ignore[no-any-return]
2025-09-07T06:09:29.2970738Z     # In all other cases, return the original input username
2025-09-07T06:09:29.2971310Z     return username
2025-09-07T06:09:29.2971557Z 
2025-09-07T06:09:29.2971564Z 
2025-09-07T06:09:29.2971783Z def is_exception_branch(branch: str) -> bool:
2025-09-07T06:09:29.2972299Z     """
2025-09-07T06:09:29.2972928Z     Branches that get opted out of experiments by default, until they're explicitly enabled.
2025-09-07T06:09:29.2973686Z     """
2025-09-07T06:09:29.2974210Z     return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"}
2025-09-07T06:09:29.2974720Z 
2025-09-07T06:09:29.2974728Z 
2025-09-07T06:09:29.2974926Z def load_yaml(yaml_text: str) -> Any:
2025-09-07T06:09:29.2975399Z     try:
2025-09-07T06:09:29.2975774Z         data = yaml.safe_load(yaml_text)
2025-09-07T06:09:29.2976425Z         return data
2025-09-07T06:09:29.2976839Z     except yaml.YAMLError:
2025-09-07T06:09:29.2977306Z         log.exception("Error loading YAML")
2025-09-07T06:09:29.2977811Z         raise
2025-09-07T06:09:29.2978024Z 
2025-09-07T06:09:29.2978031Z 
2025-09-07T06:09:29.2978454Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> tuple[str, str]:
2025-09-07T06:09:29.2979172Z     """
2025-09-07T06:09:29.2979778Z     Extracts the text with settings, if any, and the opted in users from the rollout state.
2025-09-07T06:09:29.2980367Z 
2025-09-07T06:09:29.2980837Z     If the issue body contains "---" then the text above that is the settings
2025-09-07T06:09:29.2981595Z     and the text below is the list of opted in users.
2025-09-07T06:09:29.2981994Z 
2025-09-07T06:09:29.2982367Z     If it doesn't contain "---" then the settings are empty and the rest is the users.
2025-09-07T06:09:29.2983051Z     """
2025-09-07T06:09:29.2983490Z     rollout_state_parts = rollout_state.split("---")
2025-09-07T06:09:29.2984094Z     if len(rollout_state_parts) >= 2:
2025-09-07T06:09:29.2984695Z         return rollout_state_parts[0], rollout_state_parts[1]
2025-09-07T06:09:29.2985267Z     else:
2025-09-07T06:09:29.2985640Z         return "", rollout_state
2025-09-07T06:09:29.2985945Z 
2025-09-07T06:09:29.2985952Z 
2025-09-07T06:09:29.2986296Z class UserOptins(dict[str, list[str]]):
2025-09-07T06:09:29.2986801Z     """
2025-09-07T06:09:29.2987317Z     Dictionary of users with a list of features they have opted into
2025-09-07T06:09:29.2987949Z     """
2025-09-07T06:09:29.2988140Z 
2025-09-07T06:09:29.2988154Z 
2025-09-07T06:09:29.2988489Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins:
2025-09-07T06:09:29.2989129Z     """
2025-09-07T06:09:29.2989826Z     Parse the user opt-in text into a key value pair of username and the list of features they have opted into
2025-09-07T06:09:29.2990497Z 
2025-09-07T06:09:29.2991112Z     Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable.
2025-09-07T06:09:29.2992077Z         - Example line: "@User1,lf,split_build"
2025-09-07T06:09:29.2992748Z         - A "#" prefix indicates the user is opted out of all experiments
2025-09-07T06:09:29.2993217Z 
2025-09-07T06:09:29.2993224Z 
2025-09-07T06:09:29.2993374Z     """
2025-09-07T06:09:29.2993739Z     optins = UserOptins()
2025-09-07T06:09:29.2994211Z     for user in user_optin_text.split("\n"):
2025-09-07T06:09:29.2994757Z         user = user.strip("\r\n\t -")
2025-09-07T06:09:29.2995289Z         if not user or not user.startswith("@"):
2025-09-07T06:09:29.2995976Z             # Not a valid user. Skip
2025-09-07T06:09:29.2996592Z             continue
2025-09-07T06:09:29.2996834Z 
2025-09-07T06:09:29.2996991Z         if user:
2025-09-07T06:09:29.2997420Z             usr_name = user.split(",")[0].strip("@")
2025-09-07T06:09:29.2998089Z             optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]]
2025-09-07T06:09:29.2998575Z 
2025-09-07T06:09:29.2998736Z     return optins
2025-09-07T06:09:29.2998965Z 
2025-09-07T06:09:29.2998972Z 
2025-09-07T06:09:29.2999258Z def is_valid_experiment_name(experiment_name: str) -> bool:
2025-09-07T06:09:29.2999846Z     """
2025-09-07T06:09:29.3000234Z     Check if the experiment name is valid.
2025-09-07T06:09:29.3000747Z     A valid name:
2025-09-07T06:09:29.3001370Z         - Contains only alphanumeric characters and the special characters "_" & "-"
2025-09-07T06:09:29.3002321Z         - The special characters "_" & "-" shouldn't be the first or last characters
2025-09-07T06:09:29.3003044Z         - Cannot contain spaces
2025-09-07T06:09:29.3003495Z     """
2025-09-07T06:09:29.3003698Z 
2025-09-07T06:09:29.3003955Z     valid_char_regex = r"^[a-zA-Z0-9]([\w-]*[a-zA-Z0-9])?$"
2025-09-07T06:09:29.3004638Z     valid = bool(re.match(valid_char_regex, experiment_name))
2025-09-07T06:09:29.3005070Z 
2025-09-07T06:09:29.3005225Z     if valid:
2025-09-07T06:09:29.3005592Z         return True
2025-09-07T06:09:29.3005825Z 
2025-09-07T06:09:29.3005984Z     log.error(
2025-09-07T06:09:29.3007524Z         f"Invalid experiment name: {experiment_name}. Experiment names should only contain alphanumeric characters, '_', and '-'. They cannot contain spaces, and the special characters '_' and '-' cannot be the first or last characters."
2025-09-07T06:09:29.3009083Z     )
2025-09-07T06:09:29.3009430Z     return False
2025-09-07T06:09:29.3009660Z 
2025-09-07T06:09:29.3009667Z 
2025-09-07T06:09:29.3009974Z def parse_settings_from_text(settings_text: str) -> Settings:
2025-09-07T06:09:29.3010575Z     """
2025-09-07T06:09:29.3011283Z     Parse the experiments from the issue body into a list of ExperimentSettings
2025-09-07T06:09:29.3012001Z     """
2025-09-07T06:09:29.3012345Z     try:
2025-09-07T06:09:29.3012702Z         if settings_text:
2025-09-07T06:09:29.3013416Z             # Escape the backtick as well so that we can have the settings in a code block on the GH issue
2025-09-07T06:09:29.3014200Z             # for easy reading
2025-09-07T06:09:29.3014967Z             # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on
2025-09-07T06:09:29.3015839Z             #       the backtick character in shell commands.
2025-09-07T06:09:29.3016592Z             backtick = chr(96)  # backtick character
2025-09-07T06:09:29.3017260Z             settings_text = settings_text.strip(f"\r\n\t{backtick} ")
2025-09-07T06:09:29.3017910Z             settings = load_yaml(settings_text)
2025-09-07T06:09:29.3018455Z 
2025-09-07T06:09:29.3018868Z             # For now we just load experiments. We can expand this if/when we add more settings
2025-09-07T06:09:29.3019627Z             experiments = {}
2025-09-07T06:09:29.3019924Z 
2025-09-07T06:09:29.3020300Z             for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items():
2025-09-07T06:09:29.3021104Z                 if not is_valid_experiment_name(exp_name):
2025-09-07T06:09:29.3022199Z                     # Exclude invalid experiments from the list. We log an error, but don't raise an exception so that other experiments can still be processed.
2025-09-07T06:09:29.3023234Z                     continue
2025-09-07T06:09:29.3023515Z 
2025-09-07T06:09:29.3023701Z                 valid_settings = {}
2025-09-07T06:09:29.3024207Z                 for setting in exp_settings:
2025-09-07T06:09:29.3024766Z                     if setting not in Experiment._fields:
2025-09-07T06:09:29.3025310Z                         log.warning(
2025-09-07T06:09:29.3026003Z                             f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}"
2025-09-07T06:09:29.3026995Z                         )
2025-09-07T06:09:29.3027418Z                     else:
2025-09-07T06:09:29.3027920Z                         valid_settings[setting] = exp_settings[setting]
2025-09-07T06:09:29.3028349Z 
2025-09-07T06:09:29.3028618Z                 experiments[exp_name] = Experiment(**valid_settings)
2025-09-07T06:09:29.3029239Z             return Settings(experiments)
2025-09-07T06:09:29.3029584Z 
2025-09-07T06:09:29.3029751Z     except Exception:
2025-09-07T06:09:29.3030218Z         log.exception("Failed to parse settings")
2025-09-07T06:09:29.3030599Z 
2025-09-07T06:09:29.3030766Z     return Settings()
2025-09-07T06:09:29.3031022Z 
2025-09-07T06:09:29.3031030Z 
2025-09-07T06:09:29.3031272Z def parse_settings(rollout_state: str) -> Settings:
2025-09-07T06:09:29.3031825Z     """
2025-09-07T06:09:29.3032258Z     Parse settings, if any, from the rollout state.
2025-09-07T06:09:29.3032648Z 
2025-09-07T06:09:29.3032991Z     If the issue body contains "---" then the text above that is the settings
2025-09-07T06:09:29.3033741Z     and the text below is the list of opted in users.
2025-09-07T06:09:29.3034143Z 
2025-09-07T06:09:29.3034538Z     If it doesn't contain "---" then the settings are empty and the default values are used.
2025-09-07T06:09:29.3035254Z     """
2025-09-07T06:09:29.3035788Z     settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state)
2025-09-07T06:09:29.3036657Z     return parse_settings_from_text(settings_text)
2025-09-07T06:09:29.3037050Z 
2025-09-07T06:09:29.3037057Z 
2025-09-07T06:09:29.3037293Z def parse_users(rollout_state: str) -> UserOptins:
2025-09-07T06:09:29.3037842Z     """
2025-09-07T06:09:29.3038362Z     Parse users from the rollout state.
2025-09-07T06:09:29.3038733Z 
2025-09-07T06:09:29.3038889Z     """
2025-09-07T06:09:29.3039400Z     _, users_text = extract_settings_user_opt_in_from_text(rollout_state)
2025-09-07T06:09:29.3040139Z     return parse_user_opt_in_from_text(users_text)
2025-09-07T06:09:29.3040532Z 
2025-09-07T06:09:29.3040548Z 
2025-09-07T06:09:29.3041094Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-09-07T06:09:29.3041840Z     """
2025-09-07T06:09:29.3042246Z     Check if a user is opted into an experiment
2025-09-07T06:09:29.3042764Z     """
2025-09-07T06:09:29.3043210Z     return experiment_name in user_optins.get(user, [])
2025-09-07T06:09:29.3043625Z 
2025-09-07T06:09:29.3043633Z 
2025-09-07T06:09:29.3044038Z def is_user_opted_out(user: str, user_optins: UserOptins, experiment_name: str) -> bool:
2025-09-07T06:09:29.3044769Z     """
2025-09-07T06:09:29.3045217Z     Check if a user explicitly opted out of an experiment
2025-09-07T06:09:29.3045774Z     """
2025-09-07T06:09:29.3046392Z     # if the experiment is prefixed with a "-", then it's an opt-out
2025-09-07T06:09:29.3047063Z     experiment_optout = "-" + experiment_name
2025-09-07T06:09:29.3047682Z     if experiment_optout not in user_optins.get(user, []):
2025-09-07T06:09:29.3048269Z         return False
2025-09-07T06:09:29.3048532Z 
2025-09-07T06:09:29.3048808Z     if is_user_opted_in(user, user_optins, experiment_name):
2025-09-07T06:09:29.3049401Z         log.warning(
2025-09-07T06:09:29.3050187Z             f"User {user} is opted into experiment {experiment_name}, but also opted out of it. Defaulting to opting out"
2025-09-07T06:09:29.3051056Z         )
2025-09-07T06:09:29.3051254Z 
2025-09-07T06:09:29.3051416Z     return True
2025-09-07T06:09:29.3051648Z 
2025-09-07T06:09:29.3051656Z 
2025-09-07T06:09:29.3051830Z def get_runner_prefix(
2025-09-07T06:09:29.3052256Z     rollout_state: str,
2025-09-07T06:09:29.3052718Z     workflow_requestors: Iterable[str],
2025-09-07T06:09:29.3053229Z     branch: str,
2025-09-07T06:09:29.3053706Z     eligible_experiments: frozenset[str] = frozenset(),
2025-09-07T06:09:29.3054358Z     opt_out_experiments: frozenset[str] = frozenset(),
2025-09-07T06:09:29.3054934Z     is_canary: bool = False,
2025-09-07T06:09:29.3055391Z ) -> str:
2025-09-07T06:09:29.3055795Z     settings = parse_settings(rollout_state)
2025-09-07T06:09:29.3056685Z     user_optins = parse_users(rollout_state)
2025-09-07T06:09:29.3057050Z 
2025-09-07T06:09:29.3057225Z     fleet_prefix = ""
2025-09-07T06:09:29.3057645Z     prefixes = []
2025-09-07T06:09:29.3058260Z     for experiment_name, experiment_settings in settings.experiments.items():
2025-09-07T06:09:29.3059178Z         if not experiment_settings.all_branches and is_exception_branch(branch):
2025-09-07T06:09:29.3059884Z             log.info(
2025-09-07T06:09:29.3060544Z                 f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}."
2025-09-07T06:09:29.3061282Z             )
2025-09-07T06:09:29.3061649Z             continue
2025-09-07T06:09:29.3061899Z 
2025-09-07T06:09:29.3062082Z         if opt_out_experiments:
2025-09-07T06:09:29.3062601Z             if experiment_name in opt_out_experiments:
2025-09-07T06:09:29.3063232Z                 opt_out_exp_list = ", ".join(opt_out_experiments)
2025-09-07T06:09:29.3063811Z                 log.info(
2025-09-07T06:09:29.3064718Z                     f"Skipping experiment '{experiment_name}', as this workflow has opted-out (opted out experiments are: {opt_out_exp_list})"
2025-09-07T06:09:29.3065686Z                 )
2025-09-07T06:09:29.3066539Z                 continue
2025-09-07T06:09:29.3066829Z 
2025-09-07T06:09:29.3067019Z         if eligible_experiments:
2025-09-07T06:09:29.3067595Z             if experiment_name not in eligible_experiments:
2025-09-07T06:09:29.3068220Z                 exp_list = ", ".join(eligible_experiments)
2025-09-07T06:09:29.3068770Z                 log.info(
2025-09-07T06:09:29.3069539Z                     f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}"
2025-09-07T06:09:29.3070369Z                 )
2025-09-07T06:09:29.3070758Z                 continue
2025-09-07T06:09:29.3071224Z         elif not experiment_settings.default:
2025-09-07T06:09:29.3071745Z             log.info(
2025-09-07T06:09:29.3072536Z                 f"Skipping experiment '{experiment_name}', as it is not a default experiment"
2025-09-07T06:09:29.3073266Z             )
2025-09-07T06:09:29.3073634Z             continue
2025-09-07T06:09:29.3073876Z 
2025-09-07T06:09:29.3074151Z         # Is any workflow_requestor opted out to this experiment?
2025-09-07T06:09:29.3074758Z         opted_out_users = [
2025-09-07T06:09:29.3075196Z             requestor
2025-09-07T06:09:29.3075641Z             for requestor in workflow_requestors
2025-09-07T06:09:29.3076418Z             if is_user_opted_out(requestor, user_optins, experiment_name)
2025-09-07T06:09:29.3077037Z         ]
2025-09-07T06:09:29.3077247Z 
2025-09-07T06:09:29.3077422Z         if opted_out_users:
2025-09-07T06:09:29.3077860Z             log.info(
2025-09-07T06:09:29.3078463Z                 f"{', '.join(opted_out_users)} have opted out of experiment {experiment_name}."
2025-09-07T06:09:29.3079148Z             )
2025-09-07T06:09:29.3079514Z             continue
2025-09-07T06:09:29.3079759Z 
2025-09-07T06:09:29.3080038Z         # Is any workflow_requestor opted in to this experiment?
2025-09-07T06:09:29.3080667Z         opted_in_users = [
2025-09-07T06:09:29.3081100Z             requestor
2025-09-07T06:09:29.3081541Z             for requestor in workflow_requestors
2025-09-07T06:09:29.3082196Z             if is_user_opted_in(requestor, user_optins, experiment_name)
2025-09-07T06:09:29.3082801Z         ]
2025-09-07T06:09:29.3082999Z 
2025-09-07T06:09:29.3083170Z         enabled = False
2025-09-07T06:09:29.3083594Z         if opted_in_users:
2025-09-07T06:09:29.3084023Z             log.info(
2025-09-07T06:09:29.3084610Z                 f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}."
2025-09-07T06:09:29.3085274Z             )
2025-09-07T06:09:29.3085659Z             enabled = True
2025-09-07T06:09:29.3085936Z 
2025-09-07T06:09:29.3086251Z         elif experiment_settings.rollout_perc:
2025-09-07T06:09:29.3087085Z             # If no user is opted in, then we randomly enable the experiment based on the rollout percentage
2025-09-07T06:09:29.3088146Z             if random.uniform(0, 100) <= experiment_settings.rollout_perc:
2025-09-07T06:09:29.3088786Z                 log.info(
2025-09-07T06:09:29.3089636Z                     f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}."
2025-09-07T06:09:29.3090541Z                 )
2025-09-07T06:09:29.3090943Z                 enabled = True
2025-09-07T06:09:29.3091239Z 
2025-09-07T06:09:29.3091400Z         if enabled:
2025-09-07T06:09:29.3091817Z             label = experiment_name
2025-09-07T06:09:29.3092361Z             if experiment_name == LF_FLEET_EXPERIMENT:
2025-09-07T06:09:29.3093172Z                 # We give some special treatment to the "lf" experiment since determines the fleet we use
2025-09-07T06:09:29.3094046Z                 #  - If it's enabled, then we always list it's prefix first
2025-09-07T06:09:29.3094800Z                 #  - If we're in the canary branch, then we append ".c" to the lf prefix
2025-09-07T06:09:29.3095464Z                 if is_canary:
2025-09-07T06:09:29.3095945Z                     label += CANARY_FLEET_SUFFIX
2025-09-07T06:09:29.3096599Z                 fleet_prefix = label
2025-09-07T06:09:29.3097080Z             else:
2025-09-07T06:09:29.3097492Z                 prefixes.append(label)
2025-09-07T06:09:29.3097830Z 
2025-09-07T06:09:29.3098011Z     if len(prefixes) > 1:
2025-09-07T06:09:29.3098445Z         log.error(
2025-09-07T06:09:29.3099460Z             f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}"
2025-09-07T06:09:29.3100557Z         )
2025-09-07T06:09:29.3100933Z         prefixes = prefixes[:1]
2025-09-07T06:09:29.3101237Z 
2025-09-07T06:09:29.3101420Z     # Fleet always comes first
2025-09-07T06:09:29.3101886Z     if fleet_prefix:
2025-09-07T06:09:29.3102330Z         prefixes.insert(0, fleet_prefix)
2025-09-07T06:09:29.3102688Z 
2025-09-07T06:09:29.3103057Z     return ".".join(prefixes) + "." if prefixes else ""
2025-09-07T06:09:29.3103474Z 
2025-09-07T06:09:29.3103490Z 
2025-09-07T06:09:29.3103927Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str:
2025-09-07T06:09:29.3104686Z     """
2025-09-07T06:09:29.3105266Z     Gets the first comment of the issue, which contains the desired rollout state.
2025-09-07T06:09:29.3105821Z 
2025-09-07T06:09:29.3106314Z     The default issue we use - https://github.com/pytorch/test-infra/issues/5132
2025-09-07T06:09:29.3107009Z     """
2025-09-07T06:09:29.3107388Z     gh = get_gh_client(github_token)
2025-09-07T06:09:29.3107916Z     issue = get_issue(gh, repo, issue_num)
2025-09-07T06:09:29.3108572Z     return str(issue.get_comments()[0].body.strip("\n\t "))
2025-09-07T06:09:29.3109031Z 
2025-09-07T06:09:29.3109038Z 
2025-09-07T06:09:29.3109456Z def download_json(url: str, headers: dict[str, str], num_retries: int = 3) -> Any:
2025-09-07T06:09:29.3110217Z     for _ in range(num_retries):
2025-09-07T06:09:29.3110688Z         try:
2025-09-07T06:09:29.3111099Z             req = Request(url=url, headers=headers)
2025-09-07T06:09:29.3111749Z             content = urlopen(req, timeout=5).read().decode("utf-8")
2025-09-07T06:09:29.3112373Z             return json.loads(content)
2025-09-07T06:09:29.3112895Z         except Exception as e:
2025-09-07T06:09:29.3113420Z             log.warning(f"Could not download {url}: {e}")
2025-09-07T06:09:29.3113820Z 
2025-09-07T06:09:29.3114192Z     log.warning(f"All {num_retries} retries exhausted, downloading {url} failed")
2025-09-07T06:09:29.3114889Z     return {}
2025-09-07T06:09:29.3115113Z 
2025-09-07T06:09:29.3115119Z 
2025-09-07T06:09:29.3115275Z @cache
2025-09-07T06:09:29.3115887Z def get_pr_info(github_repo: str, github_token: str, pr_number: int) -> dict[str, Any]:
2025-09-07T06:09:29.3116731Z     """
2025-09-07T06:09:29.3117118Z     Dynamically get PR information
2025-09-07T06:09:29.3117599Z     """
2025-09-07T06:09:29.3118232Z     github_api = f"https://api.github.com/repos/{github_repo}"
2025-09-07T06:09:29.3118844Z     headers = {
2025-09-07T06:09:29.3119297Z         "Accept": "application/vnd.github.v3+json",
2025-09-07T06:09:29.3119888Z         "Authorization": f"token {github_token}",
2025-09-07T06:09:29.3120417Z     }
2025-09-07T06:09:29.3120833Z     json_response: dict[str, Any] = download_json(
2025-09-07T06:09:29.3121426Z         url=f"{github_api}/issues/{pr_number}",
2025-09-07T06:09:29.3121964Z         headers=headers,
2025-09-07T06:09:29.3122378Z     )
2025-09-07T06:09:29.3122577Z 
2025-09-07T06:09:29.3122756Z     if not json_response:
2025-09-07T06:09:29.3123305Z         log.warning(f"Failed to get the labels for #{pr_number}")
2025-09-07T06:09:29.3123908Z         return {}
2025-09-07T06:09:29.3124138Z 
2025-09-07T06:09:29.3124315Z     return json_response
2025-09-07T06:09:29.3124592Z 
2025-09-07T06:09:29.3124599Z 
2025-09-07T06:09:29.3124992Z def get_labels(github_repo: str, github_token: str, pr_number: int) -> set[str]:
2025-09-07T06:09:29.3125720Z     """
2025-09-07T06:09:29.3126345Z     Dynamically get the latest list of labels from the pull request
2025-09-07T06:09:29.3127000Z     """
2025-09-07T06:09:29.3127469Z     pr_info = get_pr_info(github_repo, github_token, pr_number)
2025-09-07T06:09:29.3128067Z     return {
2025-09-07T06:09:29.3128642Z         label.get("name") for label in pr_info.get("labels", []) if label.get("name")
2025-09-07T06:09:29.3129335Z     }
2025-09-07T06:09:29.3129528Z 
2025-09-07T06:09:29.3129535Z 
2025-09-07T06:09:29.3129708Z def main() -> None:
2025-09-07T06:09:29.3130116Z     args = parse_args()
2025-09-07T06:09:29.3130380Z 
2025-09-07T06:09:29.3130602Z     runner_label_prefix = DEFAULT_LABEL_PREFIX
2025-09-07T06:09:29.3130986Z 
2025-09-07T06:09:29.3131174Z     # Check if the PR is opt-out
2025-09-07T06:09:29.3131658Z     if args.pr_number:
2025-09-07T06:09:29.3132291Z         labels = get_labels(args.github_repo, args.github_token, int(args.pr_number))
2025-09-07T06:09:29.3133163Z         if OPT_OUT_LABEL in labels:
2025-09-07T06:09:29.3133665Z             log.info(
2025-09-07T06:09:29.3134332Z                 f"Opt-out runner determinator because #{args.pr_number} has {OPT_OUT_LABEL} label"
2025-09-07T06:09:29.3135092Z             )
2025-09-07T06:09:29.3135628Z             set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-09-07T06:09:29.3136397Z             sys.exit()
2025-09-07T06:09:29.3136655Z 
2025-09-07T06:09:29.3136814Z     try:
2025-09-07T06:09:29.3137237Z         rollout_state = get_rollout_state_from_issue(
2025-09-07T06:09:29.3137929Z             args.github_token, args.github_issue_repo, args.github_issue
2025-09-07T06:09:29.3138556Z         )
2025-09-07T06:09:29.3138758Z 
2025-09-07T06:09:29.3138965Z         username = get_potential_pr_author(
2025-09-07T06:09:29.3139498Z             args.github_token,
2025-09-07T06:09:29.3139972Z             args.github_repo,
2025-09-07T06:09:29.3140435Z             args.github_actor,
2025-09-07T06:09:29.3140918Z             args.github_ref_type,
2025-09-07T06:09:29.3141405Z             args.github_branch,
2025-09-07T06:09:29.3141867Z         )
2025-09-07T06:09:29.3142069Z 
2025-09-07T06:09:29.3142348Z         is_canary = args.github_repo == "pytorch/pytorch-canary"
2025-09-07T06:09:29.3142800Z 
2025-09-07T06:09:29.3143013Z         runner_label_prefix = get_runner_prefix(
2025-09-07T06:09:29.3143562Z             rollout_state,
2025-09-07T06:09:29.3144031Z             (args.github_issue_owner, username),
2025-09-07T06:09:29.3144570Z             args.github_branch,
2025-09-07T06:09:29.3145054Z             args.eligible_experiments,
2025-09-07T06:09:29.3145586Z             args.opt_out_experiments,
2025-09-07T06:09:29.3146178Z             is_canary,
2025-09-07T06:09:29.3146592Z         )
2025-09-07T06:09:29.3146793Z 
2025-09-07T06:09:29.3146973Z     except Exception as e:
2025-09-07T06:09:29.3147415Z         log.error(
2025-09-07T06:09:29.3148076Z             f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}"
2025-09-07T06:09:29.3148974Z         )
2025-09-07T06:09:29.3149175Z 
2025-09-07T06:09:29.3149506Z     set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix)
2025-09-07T06:09:29.3149996Z 
2025-09-07T06:09:29.3150002Z 
2025-09-07T06:09:29.3150176Z if __name__ == "__main__":
2025-09-07T06:09:29.3150606Z     main()
2025-09-07T06:09:29.3150808Z 
2025-09-07T06:09:29.3244589Z ##[group]Run python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2025-09-07T06:09:29.3245482Z python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0
2025-09-07T06:09:29.3275528Z shell: /usr/bin/bash -e {0}
2025-09-07T06:09:29.3275986Z env:
2025-09-07T06:09:29.3276790Z   GITHUB_TOKEN: ***
2025-09-07T06:09:29.3277201Z   ISSUE_NUMBER: 5132
2025-09-07T06:09:29.3277647Z   TRIGGERING_ACTOR: pytorchmergebot
2025-09-07T06:09:29.3278146Z   ISSUE_OWNER: 
2025-09-07T06:09:29.3278551Z   CHECK_EXPERIMENTS: 
2025-09-07T06:09:29.3278979Z   OPT_OUT_EXPERIMENTS: lf
2025-09-07T06:09:29.3279410Z   PR_NUMBER: 
2025-09-07T06:09:29.3279793Z ##[endgroup]
2025-09-07T06:09:32.9827945Z Defaulting to user installation because normal site-packages is not writeable
2025-09-07T06:09:35.5803839Z Collecting urllib3==1.26.18
2025-09-07T06:09:35.6177047Z   Downloading urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB)
2025-09-07T06:09:35.6382449Z      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 4.1 MB/s eta 0:00:00
2025-09-07T06:09:35.6612900Z Collecting PyGithub==2.3.0
2025-09-07T06:09:35.6656353Z   Downloading PyGithub-2.3.0-py3-none-any.whl.metadata (3.8 kB)
2025-09-07T06:09:35.7075911Z Collecting pynacl>=1.4.0 (from PyGithub==2.3.0)
2025-09-07T06:09:35.7117340Z   Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (8.6 kB)
2025-09-07T06:09:35.7160252Z Requirement already satisfied: requests>=2.14.0 in /usr/lib/python3/dist-packages (from PyGithub==2.3.0) (2.31.0)
2025-09-07T06:09:35.7173067Z Requirement already satisfied: pyjwt>=2.4.0 in /usr/lib/python3/dist-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.3.0) (2.7.0)
2025-09-07T06:09:35.7192879Z Requirement already satisfied: typing-extensions>=4.0.0 in /usr/lib/python3/dist-packages (from PyGithub==2.3.0) (4.10.0)
2025-09-07T06:09:35.7439317Z Collecting Deprecated (from PyGithub==2.3.0)
2025-09-07T06:09:35.7473925Z   Downloading Deprecated-1.2.18-py2.py3-none-any.whl.metadata (5.7 kB)
2025-09-07T06:09:35.7697580Z Requirement already satisfied: cryptography>=3.4.0 in /usr/lib/python3/dist-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.3.0) (41.0.7)
2025-09-07T06:09:35.8825230Z Collecting cffi>=1.4.1 (from pynacl>=1.4.0->PyGithub==2.3.0)
2025-09-07T06:09:35.8860636Z   Downloading cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.5 kB)
2025-09-07T06:09:36.0107799Z Collecting wrapt<2,>=1.10 (from Deprecated->PyGithub==2.3.0)
2025-09-07T06:09:36.0149656Z   Downloading wrapt-1.17.3-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.metadata (6.4 kB)
2025-09-07T06:09:36.0342568Z Collecting pycparser (from cffi>=1.4.1->pynacl>=1.4.0->PyGithub==2.3.0)
2025-09-07T06:09:36.0382489Z   Downloading pycparser-2.22-py3-none-any.whl.metadata (943 bytes)
2025-09-07T06:09:36.0631089Z Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
2025-09-07T06:09:36.0701823Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.8/143.8 kB 28.4 MB/s eta 0:00:00
2025-09-07T06:09:36.0765269Z Downloading PyGithub-2.3.0-py3-none-any.whl (354 kB)
2025-09-07T06:09:36.0829922Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 354.4/354.4 kB 83.7 MB/s eta 0:00:00
2025-09-07T06:09:36.0876704Z Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
2025-09-07T06:09:36.0970752Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 856.7/856.7 kB 117.1 MB/s eta 0:00:00
2025-09-07T06:09:36.1013364Z Downloading Deprecated-1.2.18-py2.py3-none-any.whl (10.0 kB)
2025-09-07T06:09:36.1080463Z Downloading cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (479 kB)
2025-09-07T06:09:36.1148325Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 479.4/479.4 kB 94.5 MB/s eta 0:00:00
2025-09-07T06:09:36.1190214Z Downloading wrapt-1.17.3-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (88 kB)
2025-09-07T06:09:36.1235824Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 88.0/88.0 kB 30.4 MB/s eta 0:00:00
2025-09-07T06:09:36.1272617Z Downloading pycparser-2.22-py3-none-any.whl (117 kB)
2025-09-07T06:09:36.1318655Z    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 kB 40.7 MB/s eta 0:00:00
2025-09-07T06:09:36.5033938Z Installing collected packages: wrapt, urllib3, pycparser, Deprecated, cffi, pynacl, PyGithub
2025-09-07T06:09:37.0341844Z Successfully installed Deprecated-1.2.18 PyGithub-2.3.0 cffi-1.17.1 pycparser-2.22 pynacl-1.5.0 urllib3-1.26.18 wrapt-1.17.3
2025-09-07T06:09:37.1106952Z ##[group]Run curr_branch="main"
2025-09-07T06:09:37.1107280Z curr_branch="main"
2025-09-07T06:09:37.1107499Z curr_ref_type="branch"
2025-09-07T06:09:37.1107794Z echo "Current branch is '$curr_branch'"
2025-09-07T06:09:37.1108040Z 
2025-09-07T06:09:37.1108229Z python3 runner_determinator.py \
2025-09-07T06:09:37.1108508Z   --github-token "$GITHUB_TOKEN" \
2025-09-07T06:09:37.1108785Z   --github-issue "$ISSUE_NUMBER" \
2025-09-07T06:09:37.1109034Z   --github-branch "$curr_branch" \
2025-09-07T06:09:37.1109292Z   --github-actor "$TRIGGERING_ACTOR" \
2025-09-07T06:09:37.1109581Z   --github-issue-owner "$ISSUE_OWNER" \
2025-09-07T06:09:37.1109853Z   --github-ref-type "$curr_ref_type" \
2025-09-07T06:09:37.1110126Z   --github-repo "$GITHUB_REPOSITORY" \
2025-09-07T06:09:37.1110419Z   --eligible-experiments "$CHECK_EXPERIMENTS" \
2025-09-07T06:09:37.1110778Z   --opt-out-experiments "$OPT_OUT_EXPERIMENTS" \
2025-09-07T06:09:37.1111062Z   --pr-number "${PR_NUMBER}"
2025-09-07T06:09:37.1142586Z shell: /usr/bin/bash -e {0}
2025-09-07T06:09:37.1142808Z env:
2025-09-07T06:09:37.1143370Z   GITHUB_TOKEN: ***
2025-09-07T06:09:37.1143563Z   ISSUE_NUMBER: 5132
2025-09-07T06:09:37.1143761Z   TRIGGERING_ACTOR: pytorchmergebot
2025-09-07T06:09:37.1143991Z   ISSUE_OWNER: 
2025-09-07T06:09:37.1144182Z   CHECK_EXPERIMENTS: 
2025-09-07T06:09:37.1144375Z   OPT_OUT_EXPERIMENTS: lf
2025-09-07T06:09:37.1144561Z   PR_NUMBER: 
2025-09-07T06:09:37.1144732Z ##[endgroup]
2025-09-07T06:09:37.1193619Z Current branch is 'main'
2025-09-07T06:09:38.8795726Z INFO    : Skipping experiment 'lf', as this workflow has opted-out (opted out experiments are: lf)
2025-09-07T06:09:38.8796825Z INFO    : Branch main is an exception branch. Not enabling experiment ephemeral.
2025-09-07T06:09:38.8797544Z INFO    : Branch main is an exception branch. Not enabling experiment wincanary.
2025-09-07T06:09:38.8798222Z INFO    : Branch main is an exception branch. Not enabling experiment wincanarylf.
2025-09-07T06:09:38.8798975Z INFO    : Setting output: label-type=''
2025-09-07T06:09:38.9104398Z Evaluate and set job outputs
2025-09-07T06:09:38.9111095Z Cleaning up orphan processes