2024-12-17T23:28:36.1197314Z Current runner version: '2.321.0' 2024-12-17T23:28:36.1221920Z ##[group]Operating System 2024-12-17T23:28:36.1222738Z Ubuntu 2024-12-17T23:28:36.1223237Z 22.04.5 2024-12-17T23:28:36.1223729Z LTS 2024-12-17T23:28:36.1224242Z ##[endgroup] 2024-12-17T23:28:36.1224792Z ##[group]Runner Image 2024-12-17T23:28:36.1225359Z Image: ubuntu-22.04 2024-12-17T23:28:36.1225876Z Version: 20241215.1.0 2024-12-17T23:28:36.1226872Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20241215.1/images/ubuntu/Ubuntu2204-Readme.md 2024-12-17T23:28:36.1228225Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20241215.1 2024-12-17T23:28:36.1229109Z ##[endgroup] 2024-12-17T23:28:36.1229617Z ##[group]Runner Image Provisioner 2024-12-17T23:28:36.1230197Z 2.0.404.1 2024-12-17T23:28:36.1230661Z ##[endgroup] 2024-12-17T23:28:36.1231709Z ##[group]GITHUB_TOKEN Permissions 2024-12-17T23:28:36.1233665Z Contents: read 2024-12-17T23:28:36.1234331Z Metadata: read 2024-12-17T23:28:36.1235009Z Packages: read 2024-12-17T23:28:36.1235618Z ##[endgroup] 2024-12-17T23:28:36.1238801Z Secret source: Actions 2024-12-17T23:28:36.1239590Z Prepare workflow directory 2024-12-17T23:28:36.1727450Z Prepare all required actions 2024-12-17T23:28:36.1781103Z Uses: pytorch/pytorch/.github/workflows/_runner-determinator.yml@refs/heads/release/2.6 (0cdf8b1d09254cfda66191d1bd01e3041c3c76f7) 2024-12-17T23:28:36.1786470Z ##[group] Inputs 2024-12-17T23:28:36.1787088Z check_experiments: 2024-12-17T23:28:36.1787654Z triggering_actor: malfet 2024-12-17T23:28:36.1788224Z issue_owner: 2024-12-17T23:28:36.1788745Z curr_branch: release/2.6 2024-12-17T23:28:36.1789307Z curr_ref_type: branch 2024-12-17T23:28:36.1789865Z issue_number: 5132 2024-12-17T23:28:36.1790395Z ##[endgroup] 2024-12-17T23:28:36.1790993Z Complete job name: get-label-type / runner-determinator 2024-12-17T23:28:36.2426353Z ##[group]Run cat < runner_determinator.py 2024-12-17T23:28:36.2428657Z cat < runner_determinator.py 2024-12-17T23:28:36.2429371Z # flake8: noqa: G004 2024-12-17T23:28:36.2429867Z  2024-12-17T23:28:36.2430653Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml 2024-12-17T23:28:36.2431732Z # must be kept in sync. You can do it easily by running the following command: 2024-12-17T23:28:36.2432605Z # python .github/scripts/update_runner_determinator.py 2024-12-17T23:28:36.2433367Z  2024-12-17T23:28:36.2433842Z """ 2024-12-17T23:28:36.2434529Z This runner determinator is used to determine which set of runners to run a 2024-12-17T23:28:36.2435514Z GitHub job on. It uses the first comment of a GitHub issue (by default 2024-12-17T23:28:36.2436655Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration 2024-12-17T23:28:36.2437780Z of which runners should be used to run which job. 2024-12-17T23:28:36.2439051Z  2024-12-17T23:28:36.2440074Z The configuration has two parts, the settings and a list of opted-in users, 2024-12-17T23:28:36.2441163Z separated by a line containing "---". If the line is not present, the 2024-12-17T23:28:36.2442153Z settings are considered to be empty with only the second part, the user 2024-12-17T23:28:36.2442958Z list, defined. 2024-12-17T23:28:36.2443498Z  2024-12-17T23:28:36.2444208Z The first part is a YAML block that defines the rollout settings. This can be 2024-12-17T23:28:36.2445253Z used to define any settings that are needed to determine which runners to use. 2024-12-17T23:28:36.2446263Z It's fields are defined by the RolloutSettings class below. 2024-12-17T23:28:36.2446949Z  2024-12-17T23:28:36.2447608Z The second part is a list of users who are explicitly opted in to the LF fleet. 2024-12-17T23:28:36.2448679Z The user list is also a comma separated list of additional features or 2024-12-17T23:28:36.2450188Z experiments which the user could be opted in to. 2024-12-17T23:28:36.2450805Z  2024-12-17T23:28:36.2451364Z The user list has the following rules: 2024-12-17T23:28:36.2451976Z  2024-12-17T23:28:36.2452585Z - Users are GitHub usernames, which must start with the @ prefix 2024-12-17T23:28:36.2453630Z - Each user is also a comma-separated list of features/experiments to enable 2024-12-17T23:28:36.2454505Z - A "#" prefix opts the user out of all experiments 2024-12-17T23:28:36.2455116Z  2024-12-17T23:28:36.2455621Z Example config: 2024-12-17T23:28:36.2456210Z  # A list of experiments that can be opted into. 2024-12-17T23:28:36.2456964Z  # This defines the behavior they'll induce when opted into. 2024-12-17T23:28:36.2457750Z  # Expected syntax is: 2024-12-17T23:28:36.2458519Z  # [experiment_name]: # Name of the experiment. Also used for the label prefix. 2024-12-17T23:28:36.2460252Z  # rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in. 2024-12-17T23:28:36.2461191Z  2024-12-17T23:28:36.2461678Z  experiments: 2024-12-17T23:28:36.2462176Z  lf: 2024-12-17T23:28:36.2462716Z  rollout_percent: 25 2024-12-17T23:28:36.2463312Z  all_branches: false 2024-12-17T23:28:36.2463872Z  default: true 2024-12-17T23:28:36.2464460Z  --- 2024-12-17T23:28:36.2464945Z  2024-12-17T23:28:36.2465375Z  # Opt-ins: 2024-12-17T23:28:36.2466141Z  # Users can opt into the LF fleet by adding their GitHub username to this list 2024-12-17T23:28:36.2467347Z  # and specifying experiments to enable in a comma-separated list. 2024-12-17T23:28:36.2468279Z  # To always opt out of an experiment, prefix it with a "-". 2024-12-17T23:28:36.2469344Z  # Experiments should be from the above list. 2024-12-17T23:28:36.2469986Z  2024-12-17T23:28:36.2470453Z  @User1,-lf,split_build 2024-12-17T23:28:36.2471128Z  @User2,lf 2024-12-17T23:28:36.2471610Z  @User3,split_build 2024-12-17T23:28:36.2472133Z """ 2024-12-17T23:28:36.2472796Z  2024-12-17T23:28:36.2473486Z import json 2024-12-17T23:28:36.2474306Z import logging 2024-12-17T23:28:36.2475275Z import os 2024-12-17T23:28:36.2475998Z import random 2024-12-17T23:28:36.2476769Z import re 2024-12-17T23:28:36.2477704Z import sys 2024-12-17T23:28:36.2478473Z from argparse import ArgumentParser 2024-12-17T23:28:36.2479121Z from functools import lru_cache 2024-12-17T23:28:36.2479842Z from logging import LogRecord 2024-12-17T23:28:36.2480673Z from typing import Any, Dict, FrozenSet, Iterable, List, NamedTuple, Set, Tuple 2024-12-17T23:28:36.2481530Z from urllib.request import Request, urlopen 2024-12-17T23:28:36.2482240Z  2024-12-17T23:28:36.2482684Z import yaml 2024-12-17T23:28:36.2483176Z from github import Auth, Github 2024-12-17T23:28:36.2483879Z from github.Issue import Issue 2024-12-17T23:28:36.2484469Z  2024-12-17T23:28:36.2484848Z  2024-12-17T23:28:36.2485449Z DEFAULT_LABEL_PREFIX = "" # use meta runners 2024-12-17T23:28:36.2486270Z WORKFLOW_LABEL_LF = "lf." # use runners from the linux foundation 2024-12-17T23:28:36.2487182Z WORKFLOW_LABEL_LF_CANARY = "lf.c." # use canary runners from the linux foundation 2024-12-17T23:28:36.2488043Z  2024-12-17T23:28:36.2488589Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "") 2024-12-17T23:28:36.2489213Z GH_OUTPUT_KEY_AMI = "runner-ami" 2024-12-17T23:28:36.2489926Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type" 2024-12-17T23:28:36.2490886Z OPT_OUT_LABEL = "no-runner-experiments" 2024-12-17T23:28:36.2491449Z  2024-12-17T23:28:36.2492050Z SETTING_EXPERIMENTS = "experiments" 2024-12-17T23:28:36.2492643Z  2024-12-17T23:28:36.2493161Z LF_FLEET_EXPERIMENT = "lf" 2024-12-17T23:28:36.2493760Z CANARY_FLEET_SUFFIX = ".c" 2024-12-17T23:28:36.2494331Z  2024-12-17T23:28:36.2494797Z  2024-12-17T23:28:36.2495263Z class Experiment(NamedTuple): 2024-12-17T23:28:36.2495874Z  rollout_perc: float = ( 2024-12-17T23:28:36.2496695Z  0 # Percentage of workflows to experiment on when user is not opted-in. 2024-12-17T23:28:36.2497461Z  ) 2024-12-17T23:28:36.2497932Z  all_branches: bool = ( 2024-12-17T23:28:36.2499355Z  False # If True, the experiment is also enabled on the exception branches 2024-12-17T23:28:36.2500202Z  ) 2024-12-17T23:28:36.2500672Z  default: bool = ( 2024-12-17T23:28:36.2501471Z  True # If True, the experiment is enabled by default for all queries 2024-12-17T23:28:36.2502204Z  ) 2024-12-17T23:28:36.2502655Z  2024-12-17T23:28:36.2503181Z  # Add more fields as needed 2024-12-17T23:28:36.2503736Z  2024-12-17T23:28:36.2504160Z  2024-12-17T23:28:36.2504678Z class Settings(NamedTuple): 2024-12-17T23:28:36.2505225Z  """ 2024-12-17T23:28:36.2505802Z  Settings for the experiments that can be opted into. 2024-12-17T23:28:36.2506534Z  """ 2024-12-17T23:28:36.2506973Z  2024-12-17T23:28:36.2507470Z  experiments: Dict[str, Experiment] = {} 2024-12-17T23:28:36.2508154Z  2024-12-17T23:28:36.2508564Z  2024-12-17T23:28:36.2509258Z class ColorFormatter(logging.Formatter): 2024-12-17T23:28:36.2510075Z  """Color codes the log messages based on the log level""" 2024-12-17T23:28:36.2510742Z  2024-12-17T23:28:36.2511195Z  COLORS = { 2024-12-17T23:28:36.2511948Z  "WARNING": "\033[33m", # Yellow 2024-12-17T23:28:36.2512586Z  "ERROR": "\033[31m", # Red 2024-12-17T23:28:36.2513228Z  "CRITICAL": "\033[31m", # Red 2024-12-17T23:28:36.2513889Z  "INFO": "\033[0m", # Reset 2024-12-17T23:28:36.2514523Z  "DEBUG": "\033[0m", # Reset 2024-12-17T23:28:36.2515058Z  } 2024-12-17T23:28:36.2515883Z  2024-12-17T23:28:36.2516463Z  def format(self, record: LogRecord) -> str: 2024-12-17T23:28:36.2517290Z  log_color = self.COLORS.get(record.levelname, "\033[0m") # Default to reset 2024-12-17T23:28:36.2518602Z  record.msg = f"{log_color}{record.msg}\033[0m" 2024-12-17T23:28:36.2519340Z  return super().format(record) 2024-12-17T23:28:36.2519889Z  2024-12-17T23:28:36.2520390Z  2024-12-17T23:28:36.2520888Z handler = logging.StreamHandler() 2024-12-17T23:28:36.2521707Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s")) 2024-12-17T23:28:36.2522551Z  2024-12-17T23:28:36.2523118Z log = logging.getLogger(os.path.basename(__file__)) 2024-12-17T23:28:36.2523806Z log.addHandler(handler) 2024-12-17T23:28:36.2524415Z log.setLevel(logging.INFO) 2024-12-17T23:28:36.2524980Z  2024-12-17T23:28:36.2525387Z  2024-12-17T23:28:36.2525975Z def set_github_output(key: str, value: str) -> None: 2024-12-17T23:28:36.2526655Z  """ 2024-12-17T23:28:36.2527260Z  Defines outputs of the github action that invokes this script 2024-12-17T23:28:36.2528027Z  """ 2024-12-17T23:28:36.2528539Z  if not GITHUB_OUTPUT: 2024-12-17T23:28:36.2529714Z  # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice 2024-12-17T23:28:36.2531110Z  log.warning( 2024-12-17T23:28:36.2532129Z  "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method." 2024-12-17T23:28:36.2533284Z  ) 2024-12-17T23:28:36.2533876Z  print(f"::set-output name={key}::{value}") 2024-12-17T23:28:36.2534578Z  return 2024-12-17T23:28:36.2535095Z  2024-12-17T23:28:36.2535673Z  with open(GITHUB_OUTPUT, "a") as f: 2024-12-17T23:28:36.2536324Z  log.info(f"Setting output: {key}='{value}'") 2024-12-17T23:28:36.2537054Z  f.write(f"{key}={value}\n") 2024-12-17T23:28:36.2537858Z  2024-12-17T23:28:36.2538519Z  2024-12-17T23:28:36.2539813Z def _str_comma_separated_to_set(value: str) -> FrozenSet[str]: 2024-12-17T23:28:36.2541152Z  return frozenset( 2024-12-17T23:28:36.2542280Z  filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(","))) 2024-12-17T23:28:36.2543213Z  ) 2024-12-17T23:28:36.2543692Z  2024-12-17T23:28:36.2544066Z  2024-12-17T23:28:36.2544623Z def parse_args() -> Any: 2024-12-17T23:28:36.2545429Z  parser = ArgumentParser("Get dynamic rollout settings") 2024-12-17T23:28:36.2546356Z  parser.add_argument("--github-token", type=str, required=True, help="GitHub token") 2024-12-17T23:28:36.2547295Z  parser.add_argument( 2024-12-17T23:28:36.2547895Z  "--github-issue-repo", 2024-12-17T23:28:36.2548520Z  type=str, 2024-12-17T23:28:36.2549043Z  required=False, 2024-12-17T23:28:36.2549811Z  default="pytorch/test-infra", 2024-12-17T23:28:36.2550557Z  help="GitHub repo to get the issue", 2024-12-17T23:28:36.2551164Z  ) 2024-12-17T23:28:36.2551644Z  parser.add_argument( 2024-12-17T23:28:36.2552240Z  "--github-repo", 2024-12-17T23:28:36.2552791Z  type=str, 2024-12-17T23:28:36.2553317Z  required=True, 2024-12-17T23:28:36.2553969Z  help="GitHub repo where CI is running", 2024-12-17T23:28:36.2554564Z  ) 2024-12-17T23:28:36.2555038Z  parser.add_argument( 2024-12-17T23:28:36.2555838Z  "--github-issue", type=int, required=True, help="GitHub issue number" 2024-12-17T23:28:36.2556564Z  ) 2024-12-17T23:28:36.2557051Z  parser.add_argument( 2024-12-17T23:28:36.2557864Z  "--github-actor", type=str, required=True, help="GitHub triggering_actor" 2024-12-17T23:28:36.2558748Z  ) 2024-12-17T23:28:36.2559254Z  parser.add_argument( 2024-12-17T23:28:36.2560154Z  "--github-issue-owner", type=str, required=True, help="GitHub issue owner" 2024-12-17T23:28:36.2560956Z  ) 2024-12-17T23:28:36.2561389Z  parser.add_argument( 2024-12-17T23:28:36.2562235Z  "--github-branch", type=str, required=True, help="Current GitHub branch or tag" 2024-12-17T23:28:36.2563054Z  ) 2024-12-17T23:28:36.2563505Z  parser.add_argument( 2024-12-17T23:28:36.2564122Z  "--github-ref-type", 2024-12-17T23:28:36.2564702Z  type=str, 2024-12-17T23:28:36.2565213Z  required=True, 2024-12-17T23:28:36.2565938Z  help="Current GitHub ref type, branch or tag", 2024-12-17T23:28:36.2566553Z  ) 2024-12-17T23:28:36.2567042Z  parser.add_argument( 2024-12-17T23:28:36.2567690Z  "--eligible-experiments", 2024-12-17T23:28:36.2568297Z  type=_str_comma_separated_to_set, 2024-12-17T23:28:36.2568924Z  required=False, 2024-12-17T23:28:36.2569524Z  default="", 2024-12-17T23:28:36.2570686Z  help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked", 2024-12-17T23:28:36.2571683Z  ) 2024-12-17T23:28:36.2572220Z  parser.add_argument( 2024-12-17T23:28:36.2572783Z  "--pr-number", 2024-12-17T23:28:36.2573317Z  type=str, 2024-12-17T23:28:36.2573918Z  required=False, 2024-12-17T23:28:36.2574425Z  default="", 2024-12-17T23:28:36.2575198Z  help="the optional PR number where this is run", 2024-12-17T23:28:36.2576002Z  ) 2024-12-17T23:28:36.2576708Z  2024-12-17T23:28:36.2577475Z  return parser.parse_args() 2024-12-17T23:28:36.2578317Z  2024-12-17T23:28:36.2579190Z  2024-12-17T23:28:36.2579746Z def get_gh_client(github_token: str) -> Github: 2024-12-17T23:28:36.2580533Z  auth = Auth.Token(github_token) 2024-12-17T23:28:36.2581137Z  return Github(auth=auth) 2024-12-17T23:28:36.2581692Z  2024-12-17T23:28:36.2582198Z  2024-12-17T23:28:36.2582762Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue: 2024-12-17T23:28:36.2583500Z  repo = gh.get_repo(repo) 2024-12-17T23:28:36.2584218Z  return repo.get_issue(number=issue_num) 2024-12-17T23:28:36.2584826Z  2024-12-17T23:28:36.2585231Z  2024-12-17T23:28:36.2585742Z def get_potential_pr_author( 2024-12-17T23:28:36.2586503Z  github_token: str, repo: str, username: str, ref_type: str, ref_name: str 2024-12-17T23:28:36.2587238Z ) -> str: 2024-12-17T23:28:36.2587948Z  # If the trigger was a new tag added by a bot, this is a ciflow case 2024-12-17T23:28:36.2589050Z  # Fetch the actual username from the original PR. The PR number is 2024-12-17T23:28:36.2589887Z  # embedded in the tag name: ciflow// 2024-12-17T23:28:36.2590623Z  2024-12-17T23:28:36.2591115Z  gh = get_gh_client(github_token) 2024-12-17T23:28:36.2591665Z  2024-12-17T23:28:36.2592282Z  if username == "pytorch-bot[bot]" and ref_type == "tag": 2024-12-17T23:28:36.2593020Z  split_tag = ref_name.split("/") 2024-12-17T23:28:36.2593579Z  if ( 2024-12-17T23:28:36.2594140Z  len(split_tag) == 3 2024-12-17T23:28:36.2594761Z  and split_tag[0] == "ciflow" 2024-12-17T23:28:36.2595351Z  and split_tag[2].isnumeric() 2024-12-17T23:28:36.2596012Z  ): 2024-12-17T23:28:36.2596539Z  pr_number = split_tag[2] 2024-12-17T23:28:36.2597115Z  try: 2024-12-17T23:28:36.2597708Z  repository = gh.get_repo(repo) 2024-12-17T23:28:36.2598440Z  pull = repository.get_pull(number=int(pr_number)) 2024-12-17T23:28:36.2599133Z  except Exception as e: 2024-12-17T23:28:36.2599827Z  raise Exception( # noqa: TRY002 2024-12-17T23:28:36.2600599Z  f"issue with pull request {pr_number} from repo {repository}" 2024-12-17T23:28:36.2601315Z  ) from e 2024-12-17T23:28:36.2601943Z  return pull.user.login 2024-12-17T23:28:36.2602632Z  # In all other cases, return the original input username 2024-12-17T23:28:36.2603294Z  return username 2024-12-17T23:28:36.2603888Z  2024-12-17T23:28:36.2604316Z  2024-12-17T23:28:36.2604817Z def is_exception_branch(branch: str) -> bool: 2024-12-17T23:28:36.2605501Z  """ 2024-12-17T23:28:36.2606256Z  Branches that get opted out of experiments by default, until they're explicitly enabled. 2024-12-17T23:28:36.2607093Z  """ 2024-12-17T23:28:36.2607996Z  return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"} 2024-12-17T23:28:36.2608711Z  2024-12-17T23:28:36.2609139Z  2024-12-17T23:28:36.2609679Z def load_yaml(yaml_text: str) -> Any: 2024-12-17T23:28:36.2610236Z  try: 2024-12-17T23:28:36.2610747Z  data = yaml.safe_load(yaml_text) 2024-12-17T23:28:36.2611424Z  return data 2024-12-17T23:28:36.2611927Z  except yaml.YAMLError: 2024-12-17T23:28:36.2612534Z  log.exception("Error loading YAML") 2024-12-17T23:28:36.2613211Z  raise 2024-12-17T23:28:36.2613668Z  2024-12-17T23:28:36.2614082Z  2024-12-17T23:28:36.2614870Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> Tuple[str, str]: 2024-12-17T23:28:36.2615677Z  """ 2024-12-17T23:28:36.2616391Z  Extracts the text with settings, if any, and the opted in users from the rollout state. 2024-12-17T23:28:36.2617298Z  2024-12-17T23:28:36.2617901Z  If the issue body contains "---" then the text above that is the settings 2024-12-17T23:28:36.2619004Z  and the text below is the list of opted in users. 2024-12-17T23:28:36.2619785Z  2024-12-17T23:28:36.2620459Z  If it doesn't contain "---" then the settings are empty and the rest is the users. 2024-12-17T23:28:36.2621183Z  """ 2024-12-17T23:28:36.2621826Z  rollout_state_parts = rollout_state.split("---") 2024-12-17T23:28:36.2622517Z  if len(rollout_state_parts) >= 2: 2024-12-17T23:28:36.2623178Z  return rollout_state_parts[0], rollout_state_parts[1] 2024-12-17T23:28:36.2623942Z  else: 2024-12-17T23:28:36.2624602Z  return "", rollout_state 2024-12-17T23:28:36.2625136Z  2024-12-17T23:28:36.2625655Z  2024-12-17T23:28:36.2626152Z class UserOptins(Dict[str, List[str]]): 2024-12-17T23:28:36.2626707Z  """ 2024-12-17T23:28:36.2627455Z  Dictionary of users with a list of features they have opted into 2024-12-17T23:28:36.2628177Z  """ 2024-12-17T23:28:36.2628563Z  2024-12-17T23:28:36.2629073Z  2024-12-17T23:28:36.2629687Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins: 2024-12-17T23:28:36.2630383Z  """ 2024-12-17T23:28:36.2631743Z  Parse the user opt-in text into a key value pair of username and the list of features they have opted into 2024-12-17T23:28:36.2633277Z  2024-12-17T23:28:36.2634820Z  Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable. 2024-12-17T23:28:36.2636458Z  - Example line: "@User1,lf,split_build" 2024-12-17T23:28:36.2637249Z  - A "#" prefix indicates the user is opted out of all experiments 2024-12-17T23:28:36.2638100Z  2024-12-17T23:28:36.2638489Z  2024-12-17T23:28:36.2638896Z  """ 2024-12-17T23:28:36.2639463Z  optins = UserOptins() 2024-12-17T23:28:36.2640029Z  for user in user_optin_text.split("\n"): 2024-12-17T23:28:36.2640684Z  user = user.strip("\r\n\t -") 2024-12-17T23:28:36.2641415Z  if not user or not user.startswith("@"): 2024-12-17T23:28:36.2642033Z  # Not a valid user. Skip 2024-12-17T23:28:36.2642616Z  continue 2024-12-17T23:28:36.2643240Z  2024-12-17T23:28:36.2643632Z  if user: 2024-12-17T23:28:36.2644202Z  usr_name = user.split(",")[0].strip("@") 2024-12-17T23:28:36.2645073Z  optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]] 2024-12-17T23:28:36.2645810Z  2024-12-17T23:28:36.2646193Z  return optins 2024-12-17T23:28:36.2646777Z  2024-12-17T23:28:36.2647423Z  2024-12-17T23:28:36.2648090Z def is_valid_experiment_name(experiment_name: str) -> bool: 2024-12-17T23:28:36.2648933Z  """ 2024-12-17T23:28:36.2649470Z  Check if the experiment name is valid. 2024-12-17T23:28:36.2650027Z  A valid name: 2024-12-17T23:28:36.2650882Z  - Contains only alphanumeric characters and the special characters "_" & "-" 2024-12-17T23:28:36.2651914Z  - The special characters "_" & "-" shouldn't be the first or last characters 2024-12-17T23:28:36.2652654Z  - Cannot contain spaces 2024-12-17T23:28:36.2653311Z  """ 2024-12-17T23:28:36.2653754Z  2024-12-17T23:28:36.2654244Z  valid_char_regex = r"^[a-zA-Z0-9]([\w-]*[a-zA-Z0-9])?$" 2024-12-17T23:28:36.2655147Z  valid = bool(re.match(valid_char_regex, experiment_name)) 2024-12-17T23:28:36.2655820Z  2024-12-17T23:28:36.2656285Z  if valid: 2024-12-17T23:28:36.2656781Z  return True 2024-12-17T23:28:36.2657288Z  2024-12-17T23:28:36.2657755Z  log.error( 2024-12-17T23:28:36.2659859Z  f"Invalid experiment name: {experiment_name}. Experiment names should only contain alphanumeric characters, '_', and '-'. They cannot contain spaces, and the special characters '_' and '-' cannot be the first or last characters." 2024-12-17T23:28:36.2661479Z  ) 2024-12-17T23:28:36.2662020Z  return False 2024-12-17T23:28:36.2662498Z  2024-12-17T23:28:36.2662918Z  2024-12-17T23:28:36.2663584Z def parse_settings_from_text(settings_text: str) -> Settings: 2024-12-17T23:28:36.2664278Z  """ 2024-12-17T23:28:36.2665136Z  Parse the experiments from the issue body into a list of ExperimentSettings 2024-12-17T23:28:36.2666025Z  """ 2024-12-17T23:28:36.2666457Z  try: 2024-12-17T23:28:36.2666957Z  if settings_text: 2024-12-17T23:28:36.2667827Z  # Escape the backtick as well so that we can have the settings in a code block on the GH issue 2024-12-17T23:28:36.2668712Z  # for easy reading 2024-12-17T23:28:36.2669584Z  # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on 2024-12-17T23:28:36.2670589Z  # the backtick character in shell commands. 2024-12-17T23:28:36.2671301Z  backtick = chr(96) # backtick character 2024-12-17T23:28:36.2672043Z  settings_text = settings_text.strip(f"\r\n\t{backtick} ") 2024-12-17T23:28:36.2672834Z  settings = load_yaml(settings_text) 2024-12-17T23:28:36.2673440Z  2024-12-17T23:28:36.2674124Z  # For now we just load experiments. We can expand this if/when we add more settings 2024-12-17T23:28:36.2674948Z  experiments = {} 2024-12-17T23:28:36.2675525Z  2024-12-17T23:28:36.2676165Z  for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items(): 2024-12-17T23:28:36.2677008Z  if not is_valid_experiment_name(exp_name): 2024-12-17T23:28:36.2678208Z  # Exclude invalid experiments from the list. We log an error, but don't raise an exception so that other experiments can still be processed. 2024-12-17T23:28:36.2679294Z  continue 2024-12-17T23:28:36.2679893Z  2024-12-17T23:28:36.2680330Z  valid_settings = {} 2024-12-17T23:28:36.2680950Z  for setting in exp_settings: 2024-12-17T23:28:36.2681667Z  if setting not in Experiment._fields: 2024-12-17T23:28:36.2682297Z  log.warning( 2024-12-17T23:28:36.2683088Z  f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}" 2024-12-17T23:28:36.2684113Z  ) 2024-12-17T23:28:36.2684618Z  else: 2024-12-17T23:28:36.2685246Z  valid_settings[setting] = exp_settings[setting] 2024-12-17T23:28:36.2685987Z  2024-12-17T23:28:36.2686505Z  experiments[exp_name] = Experiment(**valid_settings) 2024-12-17T23:28:36.2687229Z  return Settings(experiments) 2024-12-17T23:28:36.2687905Z  2024-12-17T23:28:36.2688313Z  except Exception: 2024-12-17T23:28:36.2688960Z  log.exception("Failed to parse settings") 2024-12-17T23:28:36.2690197Z  2024-12-17T23:28:36.2690989Z  return Settings() 2024-12-17T23:28:36.2691836Z  2024-12-17T23:28:36.2692623Z  2024-12-17T23:28:36.2693553Z def parse_settings(rollout_state: str) -> Settings: 2024-12-17T23:28:36.2694673Z  """ 2024-12-17T23:28:36.2695719Z  Parse settings, if any, from the rollout state. 2024-12-17T23:28:36.2696582Z  2024-12-17T23:28:36.2697198Z  If the issue body contains "---" then the text above that is the settings 2024-12-17T23:28:36.2698140Z  and the text below is the list of opted in users. 2024-12-17T23:28:36.2699105Z  2024-12-17T23:28:36.2699835Z  If it doesn't contain "---" then the settings are empty and the default values are used. 2024-12-17T23:28:36.2700732Z  """ 2024-12-17T23:28:36.2701422Z  settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state) 2024-12-17T23:28:36.2702242Z  return parse_settings_from_text(settings_text) 2024-12-17T23:28:36.2702937Z  2024-12-17T23:28:36.2703566Z  2024-12-17T23:28:36.2704114Z def parse_users(rollout_state: str) -> UserOptins: 2024-12-17T23:28:36.2704839Z  """ 2024-12-17T23:28:36.2705371Z  Parse users from the rollout state. 2024-12-17T23:28:36.2705930Z  2024-12-17T23:28:36.2706420Z  """ 2024-12-17T23:28:36.2707070Z  _, users_text = extract_settings_user_opt_in_from_text(rollout_state) 2024-12-17T23:28:36.2707889Z  return parse_user_opt_in_from_text(users_text) 2024-12-17T23:28:36.2708570Z  2024-12-17T23:28:36.2708988Z  2024-12-17T23:28:36.2709671Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool: 2024-12-17T23:28:36.2710553Z  """ 2024-12-17T23:28:36.2711080Z  Check if a user is opted into an experiment 2024-12-17T23:28:36.2711692Z  """ 2024-12-17T23:28:36.2712324Z  return experiment_name in user_optins.get(user, []) 2024-12-17T23:28:36.2712977Z  2024-12-17T23:28:36.2713378Z  2024-12-17T23:28:36.2714134Z def is_user_opted_out(user: str, user_optins: UserOptins, experiment_name: str) -> bool: 2024-12-17T23:28:36.2714972Z  """ 2024-12-17T23:28:36.2715524Z  Check if a user explicitly opted out of an experiment 2024-12-17T23:28:36.2716241Z  """ 2024-12-17T23:28:36.2716853Z  # if the experiment is prefixed with a "-", then it's an opt-out 2024-12-17T23:28:36.2717647Z  experiment_optout = "-" + experiment_name 2024-12-17T23:28:36.2718459Z  if experiment_optout not in user_optins.get(user, []): 2024-12-17T23:28:36.2719106Z  return False 2024-12-17T23:28:36.2719623Z  2024-12-17T23:28:36.2720250Z  if is_user_opted_in(user, user_optins, experiment_name): 2024-12-17T23:28:36.2720898Z  log.warning( 2024-12-17T23:28:36.2721796Z  f"User {user} is opted into experiment {experiment_name}, but also opted out of it. Defaulting to opting out" 2024-12-17T23:28:36.2722803Z  ) 2024-12-17T23:28:36.2723396Z  2024-12-17T23:28:36.2723814Z  return True 2024-12-17T23:28:36.2724381Z  2024-12-17T23:28:36.2724780Z  2024-12-17T23:28:36.2725209Z def get_runner_prefix( 2024-12-17T23:28:36.2725836Z  rollout_state: str, 2024-12-17T23:28:36.2726436Z  workflow_requestors: Iterable[str], 2024-12-17T23:28:36.2727028Z  branch: str, 2024-12-17T23:28:36.2727718Z  eligible_experiments: FrozenSet[str] = frozenset(), 2024-12-17T23:28:36.2728386Z  is_canary: bool = False, 2024-12-17T23:28:36.2728926Z ) -> str: 2024-12-17T23:28:36.2729552Z  settings = parse_settings(rollout_state) 2024-12-17T23:28:36.2730194Z  user_optins = parse_users(rollout_state) 2024-12-17T23:28:36.2730783Z  2024-12-17T23:28:36.2731313Z  fleet_prefix = "" 2024-12-17T23:28:36.2731815Z  prefixes = [] 2024-12-17T23:28:36.2732549Z  for experiment_name, experiment_settings in settings.experiments.items(): 2024-12-17T23:28:36.2733658Z  if not experiment_settings.all_branches and is_exception_branch(branch): 2024-12-17T23:28:36.2734457Z  log.info( 2024-12-17T23:28:36.2735698Z  f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}." 2024-12-17T23:28:36.2737229Z  ) 2024-12-17T23:28:36.2738121Z  continue 2024-12-17T23:28:36.2739567Z  2024-12-17T23:28:36.2740498Z  if eligible_experiments: 2024-12-17T23:28:36.2741626Z  if experiment_name not in eligible_experiments: 2024-12-17T23:28:36.2742792Z  exp_list = ", ".join(eligible_experiments) 2024-12-17T23:28:36.2744279Z  log.info( 2024-12-17T23:28:36.2745822Z  f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}" 2024-12-17T23:28:36.2747535Z  ) 2024-12-17T23:28:36.2748371Z  continue 2024-12-17T23:28:36.2749371Z  elif not experiment_settings.default: 2024-12-17T23:28:36.2750581Z  log.info( 2024-12-17T23:28:36.2751963Z  f"Skipping experiment '{experiment_name}', as it is not a default experiment" 2024-12-17T23:28:36.2753362Z  ) 2024-12-17T23:28:36.2754367Z  continue 2024-12-17T23:28:36.2755158Z  2024-12-17T23:28:36.2756109Z  # Is any workflow_requestor opted out to this experiment? 2024-12-17T23:28:36.2757377Z  opted_out_users = [ 2024-12-17T23:28:36.2758276Z  requestor 2024-12-17T23:28:36.2759285Z  for requestor in workflow_requestors 2024-12-17T23:28:36.2760807Z  if is_user_opted_out(requestor, user_optins, experiment_name) 2024-12-17T23:28:36.2762066Z  ] 2024-12-17T23:28:36.2762774Z  2024-12-17T23:28:36.2763592Z  if opted_out_users: 2024-12-17T23:28:36.2764575Z  log.info( 2024-12-17T23:28:36.2766083Z  f"{', '.join(opted_out_users)} have opted out of experiment {experiment_name}." 2024-12-17T23:28:36.2767355Z  ) 2024-12-17T23:28:36.2768137Z  continue 2024-12-17T23:28:36.2769086Z  2024-12-17T23:28:36.2769978Z  # Is any workflow_requestor opted in to this experiment? 2024-12-17T23:28:36.2770665Z  opted_in_users = [ 2024-12-17T23:28:36.2771308Z  requestor 2024-12-17T23:28:36.2771909Z  for requestor in workflow_requestors 2024-12-17T23:28:36.2772643Z  if is_user_opted_in(requestor, user_optins, experiment_name) 2024-12-17T23:28:36.2773423Z  ] 2024-12-17T23:28:36.2774115Z  2024-12-17T23:28:36.2774499Z  enabled = False 2024-12-17T23:28:36.2775128Z  if opted_in_users: 2024-12-17T23:28:36.2775695Z  log.info( 2024-12-17T23:28:36.2776363Z  f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}." 2024-12-17T23:28:36.2777191Z  ) 2024-12-17T23:28:36.2777698Z  enabled = True 2024-12-17T23:28:36.2778176Z  2024-12-17T23:28:36.2779044Z  elif experiment_settings.rollout_perc: 2024-12-17T23:28:36.2780024Z  # If no user is opted in, then we randomly enable the experiment based on the rollout percentage 2024-12-17T23:28:36.2781095Z  if random.uniform(0, 100) <= experiment_settings.rollout_perc: 2024-12-17T23:28:36.2781815Z  log.info( 2024-12-17T23:28:36.2782764Z  f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}." 2024-12-17T23:28:36.2783785Z  ) 2024-12-17T23:28:36.2784299Z  enabled = True 2024-12-17T23:28:36.2784867Z  2024-12-17T23:28:36.2785355Z  if enabled: 2024-12-17T23:28:36.2785897Z  label = experiment_name 2024-12-17T23:28:36.2786542Z  if experiment_name == LF_FLEET_EXPERIMENT: 2024-12-17T23:28:36.2787481Z  # We give some special treatment to the "lf" experiment since determines the fleet we use 2024-12-17T23:28:36.2788475Z  # - If it's enabled, then we always list it's prefix first 2024-12-17T23:28:36.2789451Z  # - If we're in the canary branch, then we append ".c" to the lf prefix 2024-12-17T23:28:36.2790284Z  if is_canary: 2024-12-17T23:28:36.2790907Z  label += CANARY_FLEET_SUFFIX 2024-12-17T23:28:36.2791512Z  fleet_prefix = label 2024-12-17T23:28:36.2792152Z  else: 2024-12-17T23:28:36.2792716Z  prefixes.append(label) 2024-12-17T23:28:36.2793257Z  2024-12-17T23:28:36.2793756Z  if len(prefixes) > 1: 2024-12-17T23:28:36.2794317Z  log.error( 2024-12-17T23:28:36.2795390Z  f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}" 2024-12-17T23:28:36.2796698Z  ) 2024-12-17T23:28:36.2797220Z  prefixes = prefixes[:1] 2024-12-17T23:28:36.2797793Z  2024-12-17T23:28:36.2798246Z  # Fleet always comes first 2024-12-17T23:28:36.2798838Z  if fleet_prefix: 2024-12-17T23:28:36.2799409Z  prefixes.insert(0, fleet_prefix) 2024-12-17T23:28:36.2800011Z  2024-12-17T23:28:36.2800560Z  return ".".join(prefixes) + "." if prefixes else "" 2024-12-17T23:28:36.2801212Z  2024-12-17T23:28:36.2801637Z  2024-12-17T23:28:36.2802354Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str: 2024-12-17T23:28:36.2803200Z  """ 2024-12-17T23:28:36.2803882Z  Gets the first comment of the issue, which contains the desired rollout state. 2024-12-17T23:28:36.2804700Z  2024-12-17T23:28:36.2805347Z  The default issue we use - https://github.com/pytorch/test-infra/issues/5132 2024-12-17T23:28:36.2806120Z  """ 2024-12-17T23:28:36.2806719Z  gh = get_gh_client(github_token) 2024-12-17T23:28:36.2807774Z  issue = get_issue(gh, repo, issue_num) 2024-12-17T23:28:36.2808847Z  return str(issue.get_comments()[0].body.strip("\n\t ")) 2024-12-17T23:28:36.2809522Z  2024-12-17T23:28:36.2809933Z  2024-12-17T23:28:36.2810878Z def download_json(url: str, headers: Dict[str, str], num_retries: int = 3) -> Any: 2024-12-17T23:28:36.2811688Z  for _ in range(num_retries): 2024-12-17T23:28:36.2812254Z  try: 2024-12-17T23:28:36.2812871Z  req = Request(url=url, headers=headers) 2024-12-17T23:28:36.2813579Z  content = urlopen(req, timeout=5).read().decode("utf-8") 2024-12-17T23:28:36.2814300Z  return json.loads(content) 2024-12-17T23:28:36.2814994Z  except Exception as e: 2024-12-17T23:28:36.2815611Z  log.warning(f"Could not download {url}: {e}") 2024-12-17T23:28:36.2816242Z  2024-12-17T23:28:36.2816957Z  log.warning(f"All {num_retries} retries exhausted, downloading {url} failed") 2024-12-17T23:28:36.2817750Z  return {} 2024-12-17T23:28:36.2818202Z  2024-12-17T23:28:36.2818927Z  2024-12-17T23:28:36.2819407Z @lru_cache(maxsize=None) 2024-12-17T23:28:36.2820217Z def get_pr_info(github_repo: str, github_token: str, pr_number: int) -> Dict[str, Any]: 2024-12-17T23:28:36.2821114Z  """ 2024-12-17T23:28:36.2821608Z  Dynamically get PR information 2024-12-17T23:28:36.2822168Z  """ 2024-12-17T23:28:36.2822827Z  github_api = f"https://api.github.com/repos/{github_repo}" 2024-12-17T23:28:36.2823537Z  headers = { 2024-12-17T23:28:36.2824089Z  "Accept": "application/vnd.github.v3+json", 2024-12-17T23:28:36.2824849Z  "Authorization": f"token {github_token}", 2024-12-17T23:28:36.2825461Z  } 2024-12-17T23:28:36.2825970Z  json_response: Dict[str, Any] = download_json( 2024-12-17T23:28:36.2826879Z  url=f"{github_api}/issues/{pr_number}", 2024-12-17T23:28:36.2827531Z  headers=headers, 2024-12-17T23:28:36.2828039Z  ) 2024-12-17T23:28:36.2828627Z  2024-12-17T23:28:36.2829415Z  if not json_response: 2024-12-17T23:28:36.2830269Z  log.warning(f"Failed to get the labels for #{pr_number}") 2024-12-17T23:28:36.2831309Z  return {} 2024-12-17T23:28:36.2831915Z  2024-12-17T23:28:36.2832439Z  return json_response 2024-12-17T23:28:36.2833031Z  2024-12-17T23:28:36.2833475Z  2024-12-17T23:28:36.2834441Z def get_labels(github_repo: str, github_token: str, pr_number: int) -> Set[str]: 2024-12-17T23:28:36.2835641Z  """ 2024-12-17T23:28:36.2836317Z  Dynamically get the latest list of labels from the pull request 2024-12-17T23:28:36.2837061Z  """ 2024-12-17T23:28:36.2837669Z  pr_info = get_pr_info(github_repo, github_token, pr_number) 2024-12-17T23:28:36.2838493Z  return { 2024-12-17T23:28:36.2839199Z  label.get("name") for label in pr_info.get("labels", []) if label.get("name") 2024-12-17T23:28:36.2839971Z  } 2024-12-17T23:28:36.2840474Z  2024-12-17T23:28:36.2840876Z  2024-12-17T23:28:36.2841301Z def main() -> None: 2024-12-17T23:28:36.2841881Z  args = parse_args() 2024-12-17T23:28:36.2842381Z  2024-12-17T23:28:36.2842882Z  runner_label_prefix = DEFAULT_LABEL_PREFIX 2024-12-17T23:28:36.2843556Z  2024-12-17T23:28:36.2843982Z  # Check if the PR is opt-out 2024-12-17T23:28:36.2844564Z  if args.pr_number: 2024-12-17T23:28:36.2845393Z  labels = get_labels(args.github_repo, args.github_token, int(args.pr_number)) 2024-12-17T23:28:36.2846204Z  if OPT_OUT_LABEL in labels: 2024-12-17T23:28:36.2846787Z  log.info( 2024-12-17T23:28:36.2847654Z  f"Opt-out runner determinator because #{args.pr_number} has {OPT_OUT_LABEL} label" 2024-12-17T23:28:36.2848444Z  ) 2024-12-17T23:28:36.2849285Z  set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix) 2024-12-17T23:28:36.2850116Z  sys.exit() 2024-12-17T23:28:36.2850604Z  2024-12-17T23:28:36.2851028Z  try: 2024-12-17T23:28:36.2851641Z  rollout_state = get_rollout_state_from_issue( 2024-12-17T23:28:36.2852401Z  args.github_token, args.github_issue_repo, args.github_issue 2024-12-17T23:28:36.2853103Z  ) 2024-12-17T23:28:36.2853635Z  2024-12-17T23:28:36.2854109Z  username = get_potential_pr_author( 2024-12-17T23:28:36.2854708Z  args.github_token, 2024-12-17T23:28:36.2855390Z  args.github_repo, 2024-12-17T23:28:36.2855976Z  args.github_actor, 2024-12-17T23:28:36.2856546Z  args.github_ref_type, 2024-12-17T23:28:36.2857208Z  args.github_branch, 2024-12-17T23:28:36.2857769Z  ) 2024-12-17T23:28:36.2858210Z  2024-12-17T23:28:36.2859135Z  is_canary = args.github_repo == "pytorch/pytorch-canary" 2024-12-17T23:28:36.2859883Z  2024-12-17T23:28:36.2860355Z  runner_label_prefix = get_runner_prefix( 2024-12-17T23:28:36.2861065Z  rollout_state, 2024-12-17T23:28:36.2861686Z  (args.github_issue_owner, username), 2024-12-17T23:28:36.2862277Z  args.github_branch, 2024-12-17T23:28:36.2862942Z  args.eligible_experiments, 2024-12-17T23:28:36.2863550Z  is_canary, 2024-12-17T23:28:36.2864034Z  ) 2024-12-17T23:28:36.2864541Z  2024-12-17T23:28:36.2865017Z  except Exception as e: 2024-12-17T23:28:36.2865731Z  log.error( 2024-12-17T23:28:36.2866594Z  f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}" 2024-12-17T23:28:36.2867432Z  ) 2024-12-17T23:28:36.2867846Z  2024-12-17T23:28:36.2868523Z  set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix) 2024-12-17T23:28:36.2869244Z  2024-12-17T23:28:36.2869645Z  2024-12-17T23:28:36.2870138Z if __name__ == "__main__": 2024-12-17T23:28:36.2870672Z  main() 2024-12-17T23:28:36.2871095Z  2024-12-17T23:28:36.2871587Z EOF 2024-12-17T23:28:36.2872269Z  2024-12-17T23:28:36.2873026Z cat runner_determinator.py 2024-12-17T23:28:36.3303008Z shell: /usr/bin/bash -e {0} 2024-12-17T23:28:36.3303960Z env: 2024-12-17T23:28:36.3304745Z GITHUB_TOKEN: *** 2024-12-17T23:28:36.3305225Z ISSUE_NUMBER: 5132 2024-12-17T23:28:36.3305775Z TRIGGERING_ACTOR: malfet 2024-12-17T23:28:36.3306302Z ISSUE_OWNER: 2024-12-17T23:28:36.3306761Z CHECK_EXPERIMENTS: 2024-12-17T23:28:36.3307271Z PR_NUMBER: 2024-12-17T23:28:36.3307746Z ##[endgroup] 2024-12-17T23:28:36.3544450Z # flake8: noqa: G004 2024-12-17T23:28:36.3544839Z 2024-12-17T23:28:36.3545289Z # Note: Copies of this script in runner_determinator.py and _runner-determinator.yml 2024-12-17T23:28:36.3546378Z # must be kept in sync. You can do it easily by running the following command: 2024-12-17T23:28:36.3547227Z # python .github/scripts/update_runner_determinator.py 2024-12-17T23:28:36.3547714Z 2024-12-17T23:28:36.3547885Z """ 2024-12-17T23:28:36.3548571Z This runner determinator is used to determine which set of runners to run a 2024-12-17T23:28:36.3549496Z GitHub job on. It uses the first comment of a GitHub issue (by default 2024-12-17T23:28:36.3550391Z https://github.com/pytorch/test-infra/issues/5132) to define the configuration 2024-12-17T23:28:36.3551303Z of which runners should be used to run which job. 2024-12-17T23:28:36.3551730Z 2024-12-17T23:28:36.3552146Z The configuration has two parts, the settings and a list of opted-in users, 2024-12-17T23:28:36.3553275Z separated by a line containing "---". If the line is not present, the 2024-12-17T23:28:36.3554227Z settings are considered to be empty with only the second part, the user 2024-12-17T23:28:36.3554991Z list, defined. 2024-12-17T23:28:36.3555247Z 2024-12-17T23:28:36.3555621Z The first part is a YAML block that defines the rollout settings. This can be 2024-12-17T23:28:36.3556952Z used to define any settings that are needed to determine which runners to use. 2024-12-17T23:28:36.3557856Z It's fields are defined by the RolloutSettings class below. 2024-12-17T23:28:36.3558299Z 2024-12-17T23:28:36.3558721Z The second part is a list of users who are explicitly opted in to the LF fleet. 2024-12-17T23:28:36.3559637Z The user list is also a comma separated list of additional features or 2024-12-17T23:28:36.3560428Z experiments which the user could be opted in to. 2024-12-17T23:28:36.3560835Z 2024-12-17T23:28:36.3561098Z The user list has the following rules: 2024-12-17T23:28:36.3561436Z 2024-12-17T23:28:36.3561824Z - Users are GitHub usernames, which must start with the @ prefix 2024-12-17T23:28:36.3562727Z - Each user is also a comma-separated list of features/experiments to enable 2024-12-17T23:28:36.3563536Z - A "#" prefix opts the user out of all experiments 2024-12-17T23:28:36.3563918Z 2024-12-17T23:28:36.3564231Z Example config: 2024-12-17T23:28:36.3564703Z # A list of experiments that can be opted into. 2024-12-17T23:28:36.3565424Z # This defines the behavior they'll induce when opted into. 2024-12-17T23:28:36.3566154Z # Expected syntax is: 2024-12-17T23:28:36.3566832Z # [experiment_name]: # Name of the experiment. Also used for the label prefix. 2024-12-17T23:28:36.3567826Z # rollout_perc: [int] # % of workflows to run with this experiment when users are not opted in. 2024-12-17T23:28:36.3568404Z 2024-12-17T23:28:36.3568877Z experiments: 2024-12-17T23:28:36.3569329Z lf: 2024-12-17T23:28:36.3569763Z rollout_percent: 25 2024-12-17T23:28:36.3570366Z all_branches: false 2024-12-17T23:28:36.3570831Z default: true 2024-12-17T23:28:36.3571314Z --- 2024-12-17T23:28:36.3571515Z 2024-12-17T23:28:36.3571821Z # Opt-ins: 2024-12-17T23:28:36.3572414Z # Users can opt into the LF fleet by adding their GitHub username to this list 2024-12-17T23:28:36.3573303Z # and specifying experiments to enable in a comma-separated list. 2024-12-17T23:28:36.3574176Z # To always opt out of an experiment, prefix it with a "-". 2024-12-17T23:28:36.3574834Z # Experiments should be from the above list. 2024-12-17T23:28:36.3575248Z 2024-12-17T23:28:36.3575441Z @User1,-lf,split_build 2024-12-17T23:28:36.3576016Z @User2,lf 2024-12-17T23:28:36.3576429Z @User3,split_build 2024-12-17T23:28:36.3576893Z """ 2024-12-17T23:28:36.3577081Z 2024-12-17T23:28:36.3577391Z import json 2024-12-17T23:28:36.3577790Z import logging 2024-12-17T23:28:36.3578252Z import os 2024-12-17T23:28:36.3579035Z import random 2024-12-17T23:28:36.3579469Z import re 2024-12-17T23:28:36.3579916Z import sys 2024-12-17T23:28:36.3580441Z from argparse import ArgumentParser 2024-12-17T23:28:36.3580982Z from functools import lru_cache 2024-12-17T23:28:36.3581531Z from logging import LogRecord 2024-12-17T23:28:36.3582317Z from typing import Any, Dict, FrozenSet, Iterable, List, NamedTuple, Set, Tuple 2024-12-17T23:28:36.3583075Z from urllib.request import Request, urlopen 2024-12-17T23:28:36.3583497Z 2024-12-17T23:28:36.3583675Z import yaml 2024-12-17T23:28:36.3584178Z from github import Auth, Github 2024-12-17T23:28:36.3584675Z from github.Issue import Issue 2024-12-17T23:28:36.3585052Z 2024-12-17T23:28:36.3585059Z 2024-12-17T23:28:36.3585281Z DEFAULT_LABEL_PREFIX = "" # use meta runners 2024-12-17T23:28:36.3586231Z WORKFLOW_LABEL_LF = "lf." # use runners from the linux foundation 2024-12-17T23:28:36.3587278Z WORKFLOW_LABEL_LF_CANARY = "lf.c." # use canary runners from the linux foundation 2024-12-17T23:28:36.3587832Z 2024-12-17T23:28:36.3588070Z GITHUB_OUTPUT = os.getenv("GITHUB_OUTPUT", "") 2024-12-17T23:28:36.3588968Z GH_OUTPUT_KEY_AMI = "runner-ami" 2024-12-17T23:28:36.3589551Z GH_OUTPUT_KEY_LABEL_TYPE = "label-type" 2024-12-17T23:28:36.3590105Z OPT_OUT_LABEL = "no-runner-experiments" 2024-12-17T23:28:36.3590538Z 2024-12-17T23:28:36.3590755Z SETTING_EXPERIMENTS = "experiments" 2024-12-17T23:28:36.3591109Z 2024-12-17T23:28:36.3591344Z LF_FLEET_EXPERIMENT = "lf" 2024-12-17T23:28:36.3591820Z CANARY_FLEET_SUFFIX = ".c" 2024-12-17T23:28:36.3592428Z 2024-12-17T23:28:36.3592435Z 2024-12-17T23:28:36.3592661Z class Experiment(NamedTuple): 2024-12-17T23:28:36.3593225Z rollout_perc: float = ( 2024-12-17T23:28:36.3593868Z 0 # Percentage of workflows to experiment on when user is not opted-in. 2024-12-17T23:28:36.3594659Z ) 2024-12-17T23:28:36.3595090Z all_branches: bool = ( 2024-12-17T23:28:36.3595732Z False # If True, the experiment is also enabled on the exception branches 2024-12-17T23:28:36.3596544Z ) 2024-12-17T23:28:36.3596955Z default: bool = ( 2024-12-17T23:28:36.3597534Z True # If True, the experiment is enabled by default for all queries 2024-12-17T23:28:36.3598288Z ) 2024-12-17T23:28:36.3598491Z 2024-12-17T23:28:36.3598715Z # Add more fields as needed 2024-12-17T23:28:36.3599018Z 2024-12-17T23:28:36.3599024Z 2024-12-17T23:28:36.3599238Z class Settings(NamedTuple): 2024-12-17T23:28:36.3599789Z """ 2024-12-17T23:28:36.3600302Z Settings for the experiments that can be opted into. 2024-12-17T23:28:36.3600899Z """ 2024-12-17T23:28:36.3601179Z 2024-12-17T23:28:36.3601413Z experiments: Dict[str, Experiment] = {} 2024-12-17T23:28:36.3601811Z 2024-12-17T23:28:36.3601817Z 2024-12-17T23:28:36.3602036Z class ColorFormatter(logging.Formatter): 2024-12-17T23:28:36.3602704Z """Color codes the log messages based on the log level""" 2024-12-17T23:28:36.3603354Z 2024-12-17T23:28:36.3603562Z COLORS = { 2024-12-17T23:28:36.3604026Z "WARNING": "\033[33m", # Yellow 2024-12-17T23:28:36.3604597Z "ERROR": "\033[31m", # Red 2024-12-17T23:28:36.3605196Z "CRITICAL": "\033[31m", # Red 2024-12-17T23:28:36.3605742Z "INFO": "\033[0m", # Reset 2024-12-17T23:28:36.3606264Z "DEBUG": "\033[0m", # Reset 2024-12-17T23:28:36.3606878Z } 2024-12-17T23:28:36.3607132Z 2024-12-17T23:28:36.3607374Z def format(self, record: LogRecord) -> str: 2024-12-17T23:28:36.3608143Z log_color = self.COLORS.get(record.levelname, "\033[0m") # Default to reset 2024-12-17T23:28:36.3608988Z record.msg = f"{log_color}{record.msg}\033[0m" 2024-12-17T23:28:36.3609624Z return super().format(record) 2024-12-17T23:28:36.3609987Z 2024-12-17T23:28:36.3609994Z 2024-12-17T23:28:36.3610183Z handler = logging.StreamHandler() 2024-12-17T23:28:36.3611007Z handler.setFormatter(ColorFormatter(fmt="%(levelname)-8s: %(message)s")) 2024-12-17T23:28:36.3611596Z 2024-12-17T23:28:36.3611877Z log = logging.getLogger(os.path.basename(__file__)) 2024-12-17T23:28:36.3612480Z log.addHandler(handler) 2024-12-17T23:28:36.3613054Z log.setLevel(logging.INFO) 2024-12-17T23:28:36.3613359Z 2024-12-17T23:28:36.3613367Z 2024-12-17T23:28:36.3613623Z def set_github_output(key: str, value: str) -> None: 2024-12-17T23:28:36.3614213Z """ 2024-12-17T23:28:36.3614864Z Defines outputs of the github action that invokes this script 2024-12-17T23:28:36.3615493Z """ 2024-12-17T23:28:36.3615899Z if not GITHUB_OUTPUT: 2024-12-17T23:28:36.3617070Z # See https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ for deprecation notice 2024-12-17T23:28:36.3618160Z log.warning( 2024-12-17T23:28:36.3619254Z "No env var found for GITHUB_OUTPUT, you must be running this code locally. Falling back to the deprecated print method." 2024-12-17T23:28:36.3620334Z ) 2024-12-17T23:28:36.3630161Z print(f"::set-output name={key}::{value}") 2024-12-17T23:28:36.3630773Z return 2024-12-17T23:28:36.3631181Z 2024-12-17T23:28:36.3631636Z with open(GITHUB_OUTPUT, "a") as f: 2024-12-17T23:28:36.3632277Z log.info(f"Setting output: {key}='{value}'") 2024-12-17T23:28:36.3632845Z f.write(f"{key}={value}\n") 2024-12-17T23:28:36.3633327Z 2024-12-17T23:28:36.3633335Z 2024-12-17T23:28:36.3633647Z def _str_comma_separated_to_set(value: str) -> FrozenSet[str]: 2024-12-17T23:28:36.3634322Z return frozenset( 2024-12-17T23:28:36.3634935Z filter(lambda itm: itm != "", map(str.strip, value.strip(" \n\t").split(","))) 2024-12-17T23:28:36.3635770Z ) 2024-12-17T23:28:36.3635978Z 2024-12-17T23:28:36.3635985Z 2024-12-17T23:28:36.3636216Z def parse_args() -> Any: 2024-12-17T23:28:36.3636765Z parser = ArgumentParser("Get dynamic rollout settings") 2024-12-17T23:28:36.3637751Z parser.add_argument("--github-token", type=str, required=True, help="GitHub token") 2024-12-17T23:28:36.3638553Z parser.add_argument( 2024-12-17T23:28:36.3639117Z "--github-issue-repo", 2024-12-17T23:28:36.3639611Z type=str, 2024-12-17T23:28:36.3640073Z required=False, 2024-12-17T23:28:36.3640655Z default="pytorch/test-infra", 2024-12-17T23:28:36.3641230Z help="GitHub repo to get the issue", 2024-12-17T23:28:36.3641785Z ) 2024-12-17T23:28:36.3642172Z parser.add_argument( 2024-12-17T23:28:36.3642752Z "--github-repo", 2024-12-17T23:28:36.3643235Z type=str, 2024-12-17T23:28:36.3643646Z required=True, 2024-12-17T23:28:36.3644228Z help="GitHub repo where CI is running", 2024-12-17T23:28:36.3644801Z ) 2024-12-17T23:28:36.3645185Z parser.add_argument( 2024-12-17T23:28:36.3645903Z "--github-issue", type=int, required=True, help="GitHub issue number" 2024-12-17T23:28:36.3646613Z ) 2024-12-17T23:28:36.3646991Z parser.add_argument( 2024-12-17T23:28:36.3647882Z "--github-actor", type=str, required=True, help="GitHub triggering_actor" 2024-12-17T23:28:36.3648630Z ) 2024-12-17T23:28:36.3648996Z parser.add_argument( 2024-12-17T23:28:36.3649773Z "--github-issue-owner", type=str, required=True, help="GitHub issue owner" 2024-12-17T23:28:36.3650516Z ) 2024-12-17T23:28:36.3650886Z parser.add_argument( 2024-12-17T23:28:36.3651649Z "--github-branch", type=str, required=True, help="Current GitHub branch or tag" 2024-12-17T23:28:36.3652398Z ) 2024-12-17T23:28:36.3652763Z parser.add_argument( 2024-12-17T23:28:36.3653351Z "--github-ref-type", 2024-12-17T23:28:36.3653875Z type=str, 2024-12-17T23:28:36.3654271Z required=True, 2024-12-17T23:28:36.3654896Z help="Current GitHub ref type, branch or tag", 2024-12-17T23:28:36.3655487Z ) 2024-12-17T23:28:36.3655852Z parser.add_argument( 2024-12-17T23:28:36.3656471Z "--eligible-experiments", 2024-12-17T23:28:36.3657024Z type=_str_comma_separated_to_set, 2024-12-17T23:28:36.3657538Z required=False, 2024-12-17T23:28:36.3658101Z default="", 2024-12-17T23:28:36.3659303Z help="comma separated list of experiments to check, if omitted all experiments marked with default=True are checked", 2024-12-17T23:28:36.3660230Z ) 2024-12-17T23:28:36.3660773Z parser.add_argument( 2024-12-17T23:28:36.3661272Z "--pr-number", 2024-12-17T23:28:36.3661684Z type=str, 2024-12-17T23:28:36.3662235Z required=False, 2024-12-17T23:28:36.3662705Z default="", 2024-12-17T23:28:36.3663172Z help="the optional PR number where this is run", 2024-12-17T23:28:36.3663876Z ) 2024-12-17T23:28:36.3664082Z 2024-12-17T23:28:36.3664315Z return parser.parse_args() 2024-12-17T23:28:36.3664631Z 2024-12-17T23:28:36.3664637Z 2024-12-17T23:28:36.3664976Z def get_gh_client(github_token: str) -> Github: 2024-12-17T23:28:36.3665571Z auth = Auth.Token(github_token) 2024-12-17T23:28:36.3666118Z return Github(auth=auth) 2024-12-17T23:28:36.3666447Z 2024-12-17T23:28:36.3666454Z 2024-12-17T23:28:36.3666831Z def get_issue(gh: Github, repo: str, issue_num: int) -> Issue: 2024-12-17T23:28:36.3667672Z repo = gh.get_repo(repo) 2024-12-17T23:28:36.3668250Z return repo.get_issue(number=issue_num) 2024-12-17T23:28:36.3668610Z 2024-12-17T23:28:36.3668616Z 2024-12-17T23:28:36.3668909Z def get_potential_pr_author( 2024-12-17T23:28:36.3669573Z github_token: str, repo: str, username: str, ref_type: str, ref_name: str 2024-12-17T23:28:36.3670298Z ) -> str: 2024-12-17T23:28:36.3670904Z # If the trigger was a new tag added by a bot, this is a ciflow case 2024-12-17T23:28:36.3671720Z # Fetch the actual username from the original PR. The PR number is 2024-12-17T23:28:36.3672501Z # embedded in the tag name: ciflow// 2024-12-17T23:28:36.3672930Z 2024-12-17T23:28:36.3673195Z gh = get_gh_client(github_token) 2024-12-17T23:28:36.3673548Z 2024-12-17T23:28:36.3673877Z if username == "pytorch-bot[bot]" and ref_type == "tag": 2024-12-17T23:28:36.3674500Z split_tag = ref_name.split("/") 2024-12-17T23:28:36.3675097Z if ( 2024-12-17T23:28:36.3675573Z len(split_tag) == 3 2024-12-17T23:28:36.3676066Z and split_tag[0] == "ciflow" 2024-12-17T23:28:36.3676683Z and split_tag[2].isnumeric() 2024-12-17T23:28:36.3677253Z ): 2024-12-17T23:28:36.3677653Z pr_number = split_tag[2] 2024-12-17T23:28:36.3678225Z try: 2024-12-17T23:28:36.3678736Z repository = gh.get_repo(repo) 2024-12-17T23:28:36.3679351Z pull = repository.get_pull(number=int(pr_number)) 2024-12-17T23:28:36.3680027Z except Exception as e: 2024-12-17T23:28:36.3680618Z raise Exception( # noqa: TRY002 2024-12-17T23:28:36.3681278Z f"issue with pull request {pr_number} from repo {repository}" 2024-12-17T23:28:36.3681997Z ) from e 2024-12-17T23:28:36.3682535Z return pull.user.login 2024-12-17T23:28:36.3683251Z # In all other cases, return the original input username 2024-12-17T23:28:36.3683965Z return username 2024-12-17T23:28:36.3684241Z 2024-12-17T23:28:36.3684247Z 2024-12-17T23:28:36.3684513Z def is_exception_branch(branch: str) -> bool: 2024-12-17T23:28:36.3685053Z """ 2024-12-17T23:28:36.3685782Z Branches that get opted out of experiments by default, until they're explicitly enabled. 2024-12-17T23:28:36.3686604Z """ 2024-12-17T23:28:36.3687153Z return branch.split("/")[0] in {"main", "nightly", "release", "landchecks"} 2024-12-17T23:28:36.3687678Z 2024-12-17T23:28:36.3687685Z 2024-12-17T23:28:36.3688066Z def load_yaml(yaml_text: str) -> Any: 2024-12-17T23:28:36.3688632Z try: 2024-12-17T23:28:36.3689090Z data = yaml.safe_load(yaml_text) 2024-12-17T23:28:36.3689647Z return data 2024-12-17T23:28:36.3690129Z except yaml.YAMLError: 2024-12-17T23:28:36.3690671Z log.exception("Error loading YAML") 2024-12-17T23:28:36.3691239Z raise 2024-12-17T23:28:36.3691511Z 2024-12-17T23:28:36.3691517Z 2024-12-17T23:28:36.3691924Z def extract_settings_user_opt_in_from_text(rollout_state: str) -> Tuple[str, str]: 2024-12-17T23:28:36.3692709Z """ 2024-12-17T23:28:36.3693371Z Extracts the text with settings, if any, and the opted in users from the rollout state. 2024-12-17T23:28:36.3694000Z 2024-12-17T23:28:36.3694343Z If the issue body contains "---" then the text above that is the settings 2024-12-17T23:28:36.3695127Z and the text below is the list of opted in users. 2024-12-17T23:28:36.3695509Z 2024-12-17T23:28:36.3695935Z If it doesn't contain "---" then the settings are empty and the rest is the users. 2024-12-17T23:28:36.3696680Z """ 2024-12-17T23:28:36.3697181Z rollout_state_parts = rollout_state.split("---") 2024-12-17T23:28:36.3697901Z if len(rollout_state_parts) >= 2: 2024-12-17T23:28:36.3698526Z return rollout_state_parts[0], rollout_state_parts[1] 2024-12-17T23:28:36.3699687Z else: 2024-12-17T23:28:36.3700222Z return "", rollout_state 2024-12-17T23:28:36.3700539Z 2024-12-17T23:28:36.3700545Z 2024-12-17T23:28:36.3700776Z class UserOptins(Dict[str, List[str]]): 2024-12-17T23:28:36.3701509Z """ 2024-12-17T23:28:36.3702145Z Dictionary of users with a list of features they have opted into 2024-12-17T23:28:36.3702794Z """ 2024-12-17T23:28:36.3703036Z 2024-12-17T23:28:36.3703042Z 2024-12-17T23:28:36.3703376Z def parse_user_opt_in_from_text(user_optin_text: str) -> UserOptins: 2024-12-17T23:28:36.3704139Z """ 2024-12-17T23:28:36.3704837Z Parse the user opt-in text into a key value pair of username and the list of features they have opted into 2024-12-17T23:28:36.3705535Z 2024-12-17T23:28:36.3706126Z Users are GitHub usernames with the @ prefix. Each user is also a comma-separated list of features/experiments to enable. 2024-12-17T23:28:36.3707266Z - Example line: "@User1,lf,split_build" 2024-12-17T23:28:36.3707992Z - A "#" prefix indicates the user is opted out of all experiments 2024-12-17T23:28:36.3708459Z 2024-12-17T23:28:36.3708465Z 2024-12-17T23:28:36.3708632Z """ 2024-12-17T23:28:36.3709134Z optins = UserOptins() 2024-12-17T23:28:36.3709662Z for user in user_optin_text.split("\n"): 2024-12-17T23:28:36.3710216Z user = user.strip("\r\n\t -") 2024-12-17T23:28:36.3710880Z if not user or not user.startswith("@"): 2024-12-17T23:28:36.3711467Z # Not a valid user. Skip 2024-12-17T23:28:36.3711955Z continue 2024-12-17T23:28:36.3712304Z 2024-12-17T23:28:36.3712493Z if user: 2024-12-17T23:28:36.3712979Z usr_name = user.split(",")[0].strip("@") 2024-12-17T23:28:36.3713697Z optins[usr_name] = [exp.strip(" ") for exp in user.split(",")[1:]] 2024-12-17T23:28:36.3714257Z 2024-12-17T23:28:36.3714453Z return optins 2024-12-17T23:28:36.3714697Z 2024-12-17T23:28:36.3714703Z 2024-12-17T23:28:36.3715019Z def is_valid_experiment_name(experiment_name: str) -> bool: 2024-12-17T23:28:36.3715779Z """ 2024-12-17T23:28:36.3716310Z Check if the experiment name is valid. 2024-12-17T23:28:36.3716893Z A valid name: 2024-12-17T23:28:36.3717529Z - Contains only alphanumeric characters and the special characters "_" & "-" 2024-12-17T23:28:36.3718536Z - The special characters "_" & "-" shouldn't be the first or last characters 2024-12-17T23:28:36.3719287Z - Cannot contain spaces 2024-12-17T23:28:36.3719768Z """ 2024-12-17T23:28:36.3720035Z 2024-12-17T23:28:36.3720318Z valid_char_regex = r"^[a-zA-Z0-9]([\w-]*[a-zA-Z0-9])?$" 2024-12-17T23:28:36.3721056Z valid = bool(re.match(valid_char_regex, experiment_name)) 2024-12-17T23:28:36.3721494Z 2024-12-17T23:28:36.3721687Z if valid: 2024-12-17T23:28:36.3722150Z return True 2024-12-17T23:28:36.3722442Z 2024-12-17T23:28:36.3722616Z log.error( 2024-12-17T23:28:36.3724026Z f"Invalid experiment name: {experiment_name}. Experiment names should only contain alphanumeric characters, '_', and '-'. They cannot contain spaces, and the special characters '_' and '-' cannot be the first or last characters." 2024-12-17T23:28:36.3725584Z ) 2024-12-17T23:28:36.3726020Z return False 2024-12-17T23:28:36.3726261Z 2024-12-17T23:28:36.3726267Z 2024-12-17T23:28:36.3726593Z def parse_settings_from_text(settings_text: str) -> Settings: 2024-12-17T23:28:36.3727339Z """ 2024-12-17T23:28:36.3727923Z Parse the experiments from the issue body into a list of ExperimentSettings 2024-12-17T23:28:36.3728646Z """ 2024-12-17T23:28:36.3729139Z try: 2024-12-17T23:28:36.3729537Z if settings_text: 2024-12-17T23:28:36.3730269Z # Escape the backtick as well so that we can have the settings in a code block on the GH issue 2024-12-17T23:28:36.3731168Z # for easy reading 2024-12-17T23:28:36.3731936Z # Note: Using ascii for the backtick so that the cat step in _runner-determinator.yml doesn't choke on 2024-12-17T23:28:36.3732822Z # the backtick character in shell commands. 2024-12-17T23:28:36.3733558Z backtick = chr(96) # backtick character 2024-12-17T23:28:36.3734216Z settings_text = settings_text.strip(f"\r\n\t{backtick} ") 2024-12-17T23:28:36.3735046Z settings = load_yaml(settings_text) 2024-12-17T23:28:36.3735474Z 2024-12-17T23:28:36.3735921Z # For now we just load experiments. We can expand this if/when we add more settings 2024-12-17T23:28:36.3736650Z experiments = {} 2024-12-17T23:28:36.3736995Z 2024-12-17T23:28:36.3737333Z for exp_name, exp_settings in settings.get(SETTING_EXPERIMENTS).items(): 2024-12-17T23:28:36.3738178Z if not is_valid_experiment_name(exp_name): 2024-12-17T23:28:36.3739539Z # Exclude invalid experiments from the list. We log an error, but don't raise an exception so that other experiments can still be processed. 2024-12-17T23:28:36.3740578Z continue 2024-12-17T23:28:36.3740992Z 2024-12-17T23:28:36.3741197Z valid_settings = {} 2024-12-17T23:28:36.3741778Z for setting in exp_settings: 2024-12-17T23:28:36.3742340Z if setting not in Experiment._fields: 2024-12-17T23:28:36.3743011Z log.warning( 2024-12-17T23:28:36.3743759Z f"Unexpected setting in experiment: {setting} = {exp_settings[setting]}" 2024-12-17T23:28:36.3744449Z ) 2024-12-17T23:28:36.3745004Z else: 2024-12-17T23:28:36.3745569Z valid_settings[setting] = exp_settings[setting] 2024-12-17T23:28:36.3745981Z 2024-12-17T23:28:36.3746237Z experiments[exp_name] = Experiment(**valid_settings) 2024-12-17T23:28:36.3746978Z return Settings(experiments) 2024-12-17T23:28:36.3747378Z 2024-12-17T23:28:36.3747561Z except Exception: 2024-12-17T23:28:36.3748120Z log.exception("Failed to parse settings") 2024-12-17T23:28:36.3748522Z 2024-12-17T23:28:36.3748878Z return Settings() 2024-12-17T23:28:36.3749191Z 2024-12-17T23:28:36.3749197Z 2024-12-17T23:28:36.3749457Z def parse_settings(rollout_state: str) -> Settings: 2024-12-17T23:28:36.3750118Z """ 2024-12-17T23:28:36.3750588Z Parse settings, if any, from the rollout state. 2024-12-17T23:28:36.3751022Z 2024-12-17T23:28:36.3751365Z If the issue body contains "---" then the text above that is the settings 2024-12-17T23:28:36.3752192Z and the text below is the list of opted in users. 2024-12-17T23:28:36.3752621Z 2024-12-17T23:28:36.3753016Z If it doesn't contain "---" then the settings are empty and the default values are used. 2024-12-17T23:28:36.3753775Z """ 2024-12-17T23:28:36.3754439Z settings_text, _ = extract_settings_user_opt_in_from_text(rollout_state) 2024-12-17T23:28:36.3755256Z return parse_settings_from_text(settings_text) 2024-12-17T23:28:36.3755649Z 2024-12-17T23:28:36.3755655Z 2024-12-17T23:28:36.3755911Z def parse_users(rollout_state: str) -> UserOptins: 2024-12-17T23:28:36.3756547Z """ 2024-12-17T23:28:36.3757010Z Parse users from the rollout state. 2024-12-17T23:28:36.3757358Z 2024-12-17T23:28:36.3757536Z """ 2024-12-17T23:28:36.3758160Z _, users_text = extract_settings_user_opt_in_from_text(rollout_state) 2024-12-17T23:28:36.3758934Z return parse_user_opt_in_from_text(users_text) 2024-12-17T23:28:36.3759325Z 2024-12-17T23:28:36.3759331Z 2024-12-17T23:28:36.3759731Z def is_user_opted_in(user: str, user_optins: UserOptins, experiment_name: str) -> bool: 2024-12-17T23:28:36.3760556Z """ 2024-12-17T23:28:36.3761029Z Check if a user is opted into an experiment 2024-12-17T23:28:36.3761562Z """ 2024-12-17T23:28:36.3762109Z return experiment_name in user_optins.get(user, []) 2024-12-17T23:28:36.3762543Z 2024-12-17T23:28:36.3762585Z 2024-12-17T23:28:36.3762990Z def is_user_opted_out(user: str, user_optins: UserOptins, experiment_name: str) -> bool: 2024-12-17T23:28:36.3763771Z """ 2024-12-17T23:28:36.3764281Z Check if a user explicitly opted out of an experiment 2024-12-17T23:28:36.3764910Z """ 2024-12-17T23:28:36.3765467Z # if the experiment is prefixed with a "-", then it's an opt-out 2024-12-17T23:28:36.3766363Z experiment_optout = "-" + experiment_name 2024-12-17T23:28:36.3767045Z if experiment_optout not in user_optins.get(user, []): 2024-12-17T23:28:36.3767687Z return False 2024-12-17T23:28:36.3767927Z 2024-12-17T23:28:36.3768260Z if is_user_opted_in(user, user_optins, experiment_name): 2024-12-17T23:28:36.3768904Z log.warning( 2024-12-17T23:28:36.3769728Z f"User {user} is opted into experiment {experiment_name}, but also opted out of it. Defaulting to opting out" 2024-12-17T23:28:36.3770646Z ) 2024-12-17T23:28:36.3770908Z 2024-12-17T23:28:36.3771102Z return True 2024-12-17T23:28:36.3771337Z 2024-12-17T23:28:36.3771343Z 2024-12-17T23:28:36.3771558Z def get_runner_prefix( 2024-12-17T23:28:36.3772046Z rollout_state: str, 2024-12-17T23:28:36.3772577Z workflow_requestors: Iterable[str], 2024-12-17T23:28:36.3773132Z branch: str, 2024-12-17T23:28:36.3773669Z eligible_experiments: FrozenSet[str] = frozenset(), 2024-12-17T23:28:36.3774332Z is_canary: bool = False, 2024-12-17T23:28:36.3774827Z ) -> str: 2024-12-17T23:28:36.3775290Z settings = parse_settings(rollout_state) 2024-12-17T23:28:36.3775928Z user_optins = parse_users(rollout_state) 2024-12-17T23:28:36.3776292Z 2024-12-17T23:28:36.3776524Z fleet_prefix = "" 2024-12-17T23:28:36.3777003Z prefixes = [] 2024-12-17T23:28:36.3777683Z for experiment_name, experiment_settings in settings.experiments.items(): 2024-12-17T23:28:36.3778623Z if not experiment_settings.all_branches and is_exception_branch(branch): 2024-12-17T23:28:36.3779760Z log.info( 2024-12-17T23:28:36.3780433Z f"Branch {branch} is an exception branch. Not enabling experiment {experiment_name}." 2024-12-17T23:28:36.3781207Z ) 2024-12-17T23:28:36.3781891Z continue 2024-12-17T23:28:36.3782170Z 2024-12-17T23:28:36.3782370Z if eligible_experiments: 2024-12-17T23:28:36.3782959Z if experiment_name not in eligible_experiments: 2024-12-17T23:28:36.3783718Z exp_list = ", ".join(eligible_experiments) 2024-12-17T23:28:36.3784279Z log.info( 2024-12-17T23:28:36.3785093Z f"Skipping experiment '{experiment_name}', as it is not in the eligible_experiments list: {exp_list}" 2024-12-17T23:28:36.3785996Z ) 2024-12-17T23:28:36.3786407Z continue 2024-12-17T23:28:36.3786929Z elif not experiment_settings.default: 2024-12-17T23:28:36.3787553Z log.info( 2024-12-17T23:28:36.3788290Z f"Skipping experiment '{experiment_name}', as it is not a default experiment" 2024-12-17T23:28:36.3789091Z ) 2024-12-17T23:28:36.3789528Z continue 2024-12-17T23:28:36.3789782Z 2024-12-17T23:28:36.3790076Z # Is any workflow_requestor opted out to this experiment? 2024-12-17T23:28:36.3790776Z opted_out_users = [ 2024-12-17T23:28:36.3791280Z requestor 2024-12-17T23:28:36.3791772Z for requestor in workflow_requestors 2024-12-17T23:28:36.3792513Z if is_user_opted_out(requestor, user_optins, experiment_name) 2024-12-17T23:28:36.3793183Z ] 2024-12-17T23:28:36.3793411Z 2024-12-17T23:28:36.3793630Z if opted_out_users: 2024-12-17T23:28:36.3794126Z log.info( 2024-12-17T23:28:36.3794802Z f"{', '.join(opted_out_users)} have opted out of experiment {experiment_name}." 2024-12-17T23:28:36.3795516Z ) 2024-12-17T23:28:36.3795946Z continue 2024-12-17T23:28:36.3796241Z 2024-12-17T23:28:36.3796536Z # Is any workflow_requestor opted in to this experiment? 2024-12-17T23:28:36.3797167Z opted_in_users = [ 2024-12-17T23:28:36.3797660Z requestor 2024-12-17T23:28:36.3798191Z for requestor in workflow_requestors 2024-12-17T23:28:36.3798888Z if is_user_opted_in(requestor, user_optins, experiment_name) 2024-12-17T23:28:36.3799548Z ] 2024-12-17T23:28:36.3799982Z 2024-12-17T23:28:36.3800171Z enabled = False 2024-12-17T23:28:36.3800645Z if opted_in_users: 2024-12-17T23:28:36.3801145Z log.info( 2024-12-17T23:28:36.3801809Z f"{', '.join(opted_in_users)} have opted into experiment {experiment_name}." 2024-12-17T23:28:36.3802514Z ) 2024-12-17T23:28:36.3802969Z enabled = True 2024-12-17T23:28:36.3803301Z 2024-12-17T23:28:36.3803520Z elif experiment_settings.rollout_perc: 2024-12-17T23:28:36.3804352Z # If no user is opted in, then we randomly enable the experiment based on the rollout percentage 2024-12-17T23:28:36.3805307Z if random.uniform(0, 100) <= experiment_settings.rollout_perc: 2024-12-17T23:28:36.3806013Z log.info( 2024-12-17T23:28:36.3806929Z f"Based on rollout percentage of {experiment_settings.rollout_perc}%, enabling experiment {experiment_name}." 2024-12-17T23:28:36.3807932Z ) 2024-12-17T23:28:36.3808354Z enabled = True 2024-12-17T23:28:36.3808684Z 2024-12-17T23:28:36.3808872Z if enabled: 2024-12-17T23:28:36.3809399Z label = experiment_name 2024-12-17T23:28:36.3809996Z if experiment_name == LF_FLEET_EXPERIMENT: 2024-12-17T23:28:36.3810837Z # We give some special treatment to the "lf" experiment since determines the fleet we use 2024-12-17T23:28:36.3811767Z # - If it's enabled, then we always list it's prefix first 2024-12-17T23:28:36.3812506Z # - If we're in the canary branch, then we append ".c" to the lf prefix 2024-12-17T23:28:36.3813201Z if is_canary: 2024-12-17T23:28:36.3813783Z label += CANARY_FLEET_SUFFIX 2024-12-17T23:28:36.3814354Z fleet_prefix = label 2024-12-17T23:28:36.3815012Z else: 2024-12-17T23:28:36.3815667Z prefixes.append(label) 2024-12-17T23:28:36.3816016Z 2024-12-17T23:28:36.3816221Z if len(prefixes) > 1: 2024-12-17T23:28:36.3816715Z log.error( 2024-12-17T23:28:36.3817806Z f"Only a fleet and one other experiment can be enabled for a job at any time. Enabling {prefixes[0]} and ignoring the rest, which are {', '.join(prefixes[1:])}" 2024-12-17T23:28:36.3819535Z ) 2024-12-17T23:28:36.3819995Z prefixes = prefixes[:1] 2024-12-17T23:28:36.3820405Z 2024-12-17T23:28:36.3820621Z # Fleet always comes first 2024-12-17T23:28:36.3821120Z if fleet_prefix: 2024-12-17T23:28:36.3821608Z prefixes.insert(0, fleet_prefix) 2024-12-17T23:28:36.3822061Z 2024-12-17T23:28:36.3822341Z return ".".join(prefixes) + "." if prefixes else "" 2024-12-17T23:28:36.3822746Z 2024-12-17T23:28:36.3822754Z 2024-12-17T23:28:36.3823218Z def get_rollout_state_from_issue(github_token: str, repo: str, issue_num: int) -> str: 2024-12-17T23:28:36.3823988Z """ 2024-12-17T23:28:36.3824684Z Gets the first comment of the issue, which contains the desired rollout state. 2024-12-17T23:28:36.3825230Z 2024-12-17T23:28:36.3825646Z The default issue we use - https://github.com/pytorch/test-infra/issues/5132 2024-12-17T23:28:36.3826339Z """ 2024-12-17T23:28:36.3827202Z gh = get_gh_client(github_token) 2024-12-17T23:28:36.3828063Z issue = get_issue(gh, repo, issue_num) 2024-12-17T23:28:36.3829120Z return str(issue.get_comments()[0].body.strip("\n\t ")) 2024-12-17T23:28:36.3829713Z 2024-12-17T23:28:36.3829719Z 2024-12-17T23:28:36.3830133Z def download_json(url: str, headers: Dict[str, str], num_retries: int = 3) -> Any: 2024-12-17T23:28:36.3830907Z for _ in range(num_retries): 2024-12-17T23:28:36.3831408Z try: 2024-12-17T23:28:36.3831951Z req = Request(url=url, headers=headers) 2024-12-17T23:28:36.3832635Z content = urlopen(req, timeout=5).read().decode("utf-8") 2024-12-17T23:28:36.3833311Z return json.loads(content) 2024-12-17T23:28:36.3833915Z except Exception as e: 2024-12-17T23:28:36.3834490Z log.warning(f"Could not download {url}: {e}") 2024-12-17T23:28:36.3835097Z 2024-12-17T23:28:36.3835488Z log.warning(f"All {num_retries} retries exhausted, downloading {url} failed") 2024-12-17T23:28:36.3836267Z return {} 2024-12-17T23:28:36.3836532Z 2024-12-17T23:28:36.3836538Z 2024-12-17T23:28:36.3836725Z @lru_cache(maxsize=None) 2024-12-17T23:28:36.3837435Z def get_pr_info(github_repo: str, github_token: str, pr_number: int) -> Dict[str, Any]: 2024-12-17T23:28:36.3838245Z """ 2024-12-17T23:28:36.3838697Z Dynamically get PR information 2024-12-17T23:28:36.3839228Z """ 2024-12-17T23:28:36.3839797Z github_api = f"https://api.github.com/repos/{github_repo}" 2024-12-17T23:28:36.3840464Z headers = { 2024-12-17T23:28:36.3840944Z "Accept": "application/vnd.github.v3+json", 2024-12-17T23:28:36.3841620Z "Authorization": f"token {github_token}", 2024-12-17T23:28:36.3842209Z } 2024-12-17T23:28:36.3842654Z json_response: Dict[str, Any] = download_json( 2024-12-17T23:28:36.3843331Z url=f"{github_api}/issues/{pr_number}", 2024-12-17T23:28:36.3843932Z headers=headers, 2024-12-17T23:28:36.3844388Z ) 2024-12-17T23:28:36.3844656Z 2024-12-17T23:28:36.3844860Z if not json_response: 2024-12-17T23:28:36.3845474Z log.warning(f"Failed to get the labels for #{pr_number}") 2024-12-17T23:28:36.3846126Z return {} 2024-12-17T23:28:36.3846425Z 2024-12-17T23:28:36.3846677Z return json_response 2024-12-17T23:28:36.3846961Z 2024-12-17T23:28:36.3846967Z 2024-12-17T23:28:36.3847350Z def get_labels(github_repo: str, github_token: str, pr_number: int) -> Set[str]: 2024-12-17T23:28:36.3848076Z """ 2024-12-17T23:28:36.3848819Z Dynamically get the latest list of labels from the pull request 2024-12-17T23:28:36.3849465Z """ 2024-12-17T23:28:36.3850119Z pr_info = get_pr_info(github_repo, github_token, pr_number) 2024-12-17T23:28:36.3850884Z return { 2024-12-17T23:28:36.3851457Z label.get("name") for label in pr_info.get("labels", []) if label.get("name") 2024-12-17T23:28:36.3852166Z } 2024-12-17T23:28:36.3852450Z 2024-12-17T23:28:36.3852456Z 2024-12-17T23:28:36.3852687Z def main() -> None: 2024-12-17T23:28:36.3853130Z args = parse_args() 2024-12-17T23:28:36.3853434Z 2024-12-17T23:28:36.3853658Z runner_label_prefix = DEFAULT_LABEL_PREFIX 2024-12-17T23:28:36.3854100Z 2024-12-17T23:28:36.3854346Z # Check if the PR is opt-out 2024-12-17T23:28:36.3854836Z if args.pr_number: 2024-12-17T23:28:36.3855678Z labels = get_labels(args.github_repo, args.github_token, int(args.pr_number)) 2024-12-17T23:28:36.3856539Z if OPT_OUT_LABEL in labels: 2024-12-17T23:28:36.3857042Z log.info( 2024-12-17T23:28:36.3857761Z f"Opt-out runner determinator because #{args.pr_number} has {OPT_OUT_LABEL} label" 2024-12-17T23:28:36.3858792Z ) 2024-12-17T23:28:36.3859526Z set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix) 2024-12-17T23:28:36.3860176Z sys.exit() 2024-12-17T23:28:36.3860582Z 2024-12-17T23:28:36.3860756Z try: 2024-12-17T23:28:36.3861262Z rollout_state = get_rollout_state_from_issue( 2024-12-17T23:28:36.3861937Z args.github_token, args.github_issue_repo, args.github_issue 2024-12-17T23:28:36.3862675Z ) 2024-12-17T23:28:36.3862885Z 2024-12-17T23:28:36.3863142Z username = get_potential_pr_author( 2024-12-17T23:28:36.3863664Z args.github_token, 2024-12-17T23:28:36.3864258Z args.github_repo, 2024-12-17T23:28:36.3864786Z args.github_actor, 2024-12-17T23:28:36.3865260Z args.github_ref_type, 2024-12-17T23:28:36.3865870Z args.github_branch, 2024-12-17T23:28:36.3866381Z ) 2024-12-17T23:28:36.3866592Z 2024-12-17T23:28:36.3866867Z is_canary = args.github_repo == "pytorch/pytorch-canary" 2024-12-17T23:28:36.3867422Z 2024-12-17T23:28:36.3867642Z runner_label_prefix = get_runner_prefix( 2024-12-17T23:28:36.3868248Z rollout_state, 2024-12-17T23:28:36.3868906Z (args.github_issue_owner, username), 2024-12-17T23:28:36.3869589Z args.github_branch, 2024-12-17T23:28:36.3870120Z args.eligible_experiments, 2024-12-17T23:28:36.3870607Z is_canary, 2024-12-17T23:28:36.3871150Z ) 2024-12-17T23:28:36.3871362Z 2024-12-17T23:28:36.3871587Z except Exception as e: 2024-12-17T23:28:36.3872023Z log.error( 2024-12-17T23:28:36.3872805Z f"Failed to get issue. Defaulting to Meta runners and no experiments. Exception: {e}" 2024-12-17T23:28:36.3873581Z ) 2024-12-17T23:28:36.3873790Z 2024-12-17T23:28:36.3874200Z set_github_output(GH_OUTPUT_KEY_LABEL_TYPE, runner_label_prefix) 2024-12-17T23:28:36.3874703Z 2024-12-17T23:28:36.3874711Z 2024-12-17T23:28:36.3874897Z if __name__ == "__main__": 2024-12-17T23:28:36.3875384Z main() 2024-12-17T23:28:36.3875596Z 2024-12-17T23:28:36.3967477Z ##[group]Run python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0 2024-12-17T23:28:36.3968400Z python3 -m pip install urllib3==1.26.18 PyGithub==2.3.0 2024-12-17T23:28:36.4026153Z shell: /usr/bin/bash -e {0} 2024-12-17T23:28:36.4026768Z env: 2024-12-17T23:28:36.4027418Z GITHUB_TOKEN: *** 2024-12-17T23:28:36.4027922Z ISSUE_NUMBER: 5132 2024-12-17T23:28:36.4028454Z TRIGGERING_ACTOR: malfet 2024-12-17T23:28:36.4028929Z ISSUE_OWNER: 2024-12-17T23:28:36.4029398Z CHECK_EXPERIMENTS: 2024-12-17T23:28:36.4029916Z PR_NUMBER: 2024-12-17T23:28:36.4030345Z ##[endgroup] 2024-12-17T23:28:36.8334199Z Defaulting to user installation because normal site-packages is not writeable 2024-12-17T23:28:37.2619657Z Collecting urllib3==1.26.18 2024-12-17T23:28:37.3216943Z Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB) 2024-12-17T23:28:37.3493602Z ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.8/143.8 KB 7.9 MB/s eta 0:00:00 2024-12-17T23:28:37.3786912Z Collecting PyGithub==2.3.0 2024-12-17T23:28:37.3861418Z Downloading PyGithub-2.3.0-py3-none-any.whl (354 kB) 2024-12-17T23:28:37.4085543Z ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 354.4/354.4 KB 16.9 MB/s eta 0:00:00 2024-12-17T23:28:37.4664914Z Collecting pynacl>=1.4.0 2024-12-17T23:28:37.4739505Z Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB) 2024-12-17T23:28:37.5104581Z ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 856.7/856.7 KB 24.5 MB/s eta 0:00:00 2024-12-17T23:28:37.5424052Z Collecting typing-extensions>=4.0.0 2024-12-17T23:28:37.5496234Z Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB) 2024-12-17T23:28:37.5536001Z Requirement already satisfied: requests>=2.14.0 in /usr/lib/python3/dist-packages (from PyGithub==2.3.0) (2.25.1) 2024-12-17T23:28:37.5871788Z Collecting pyjwt[crypto]>=2.4.0 2024-12-17T23:28:37.5951206Z Downloading PyJWT-2.10.1-py3-none-any.whl (22 kB) 2024-12-17T23:28:37.6225805Z Collecting Deprecated 2024-12-17T23:28:37.6297725Z Downloading Deprecated-1.2.15-py2.py3-none-any.whl (9.9 kB) 2024-12-17T23:28:37.6426463Z Requirement already satisfied: cryptography>=3.4.0 in /usr/lib/python3/dist-packages (from pyjwt[crypto]>=2.4.0->PyGithub==2.3.0) (3.4.8) 2024-12-17T23:28:37.8756758Z Collecting cffi>=1.4.1 2024-12-17T23:28:37.8832107Z Downloading cffi-1.17.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (446 kB) 2024-12-17T23:28:37.9000802Z ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 446.2/446.2 KB 29.6 MB/s eta 0:00:00 2024-12-17T23:28:38.1257750Z Collecting wrapt<2,>=1.10 2024-12-17T23:28:38.1334759Z Downloading wrapt-1.17.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB) 2024-12-17T23:28:38.1375444Z ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 82.7/82.7 KB 32.1 MB/s eta 0:00:00 2024-12-17T23:28:38.1566192Z Collecting pycparser 2024-12-17T23:28:38.1637402Z Downloading pycparser-2.22-py3-none-any.whl (117 kB) 2024-12-17T23:28:38.1688762Z ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 KB 32.8 MB/s eta 0:00:00 2024-12-17T23:28:38.3419159Z Installing collected packages: wrapt, urllib3, typing-extensions, pyjwt, pycparser, Deprecated, cffi, pynacl, PyGithub 2024-12-17T23:28:38.8106183Z Successfully installed Deprecated-1.2.15 PyGithub-2.3.0 cffi-1.17.1 pycparser-2.22 pyjwt-2.10.1 pynacl-1.5.0 typing-extensions-4.12.2 urllib3-1.26.18 wrapt-1.17.0 2024-12-17T23:28:38.8823266Z ##[group]Run curr_branch="release/2.6" 2024-12-17T23:28:38.8823938Z curr_branch="release/2.6" 2024-12-17T23:28:38.8824349Z curr_ref_type="branch" 2024-12-17T23:28:38.8824807Z echo "Current branch is '$curr_branch'" 2024-12-17T23:28:38.8825192Z  2024-12-17T23:28:38.8825538Z python3 runner_determinator.py \ 2024-12-17T23:28:38.8826011Z  --github-token "$GITHUB_TOKEN" \ 2024-12-17T23:28:38.8826414Z  --github-issue "$ISSUE_NUMBER" \ 2024-12-17T23:28:38.8826830Z  --github-branch "$curr_branch" \ 2024-12-17T23:28:38.8827288Z  --github-actor "$TRIGGERING_ACTOR" \ 2024-12-17T23:28:38.8827771Z  --github-issue-owner "$ISSUE_OWNER" \ 2024-12-17T23:28:38.8828154Z  --github-ref-type "$curr_ref_type" \ 2024-12-17T23:28:38.8828627Z  --github-repo "$GITHUB_REPOSITORY" \ 2024-12-17T23:28:38.8829113Z  --eligible-experiments "$CHECK_EXPERIMENTS" \ 2024-12-17T23:28:38.8829624Z  --pr-number "${PR_NUMBER}" 2024-12-17T23:28:38.8887638Z shell: /usr/bin/bash -e {0} 2024-12-17T23:28:38.8888132Z env: 2024-12-17T23:28:38.8888925Z GITHUB_TOKEN: *** 2024-12-17T23:28:38.8889236Z ISSUE_NUMBER: 5132 2024-12-17T23:28:38.8889652Z TRIGGERING_ACTOR: malfet 2024-12-17T23:28:38.8890011Z ISSUE_OWNER: 2024-12-17T23:28:38.8890285Z CHECK_EXPERIMENTS: 2024-12-17T23:28:38.8890688Z PR_NUMBER: 2024-12-17T23:28:38.8891010Z ##[endgroup] 2024-12-17T23:28:38.8967084Z Current branch is 'release/2.6' 2024-12-17T23:28:40.1385632Z INFO : Based on rollout percentage of 95%, enabling experiment lf. 2024-12-17T23:28:40.1386750Z INFO : Skipping experiment 'awsa100', as it is not a default experiment 2024-12-17T23:28:40.1387598Z INFO : Setting output: label-type='lf.' 2024-12-17T23:28:40.1671556Z Evaluate and set job outputs 2024-12-17T23:28:40.1678318Z Set output 'label-type' 2024-12-17T23:28:40.1680284Z Cleaning up orphan processes