Skip to main content
Version: Preview

Job API Reference

Customer-hosted cloud features

note

Some features are only available on customer-hosted clouds. Reach out to preview-help@anyscale.com for info.

Job Models

JobConfig

Configuration options for a job.

Fields

  • name (str | None): Name of the job. Multiple jobs can be submitted with the same name.
  • image_uri (str | None): URI of an existing image. Exclusive with containerfile.
  • containerfile (str | None): The file path to a containerfile that will be built into an image before running the workload. Exclusive with image_uri.
  • compute_config (ComputeConfig | Dict | str | None): The name of an existing registered compute config or an inlined ComputeConfig object.
  • working_dir (str | None): Directory that will be used as the working directory for the application. If a local directory is provided, it will be uploaded to cloud storage automatically. When running inside a workspace, this defaults to the current working directory ('.').
  • excludes (List[str] | None): A list of file path globs that will be excluded when uploading local files for working_dir.
  • requirements (str | List[str] | None): A list of requirements or a path to a requirements.txt file for the workload. When running inside a workspace, this defaults to the workspace-tracked requirements.
  • env_vars (Dict[str, str] | None): A dictionary of environment variables that will be set for the workload.
  • entrypoint (str): Command that will be run to execute the job, e.g., python main.py.
  • max_retries (int): Maximum number of times the job will be retried before being marked failed. Defaults to 1.

Python Methods

def __init__(self, **fields) -> JobConfig
"""Construct a model with the provided field values set."""

def options(self, **fields) -> JobConfig
"""Return a copy of the model with the provided field values overwritten."""

def to_dict(self) -> Dict[str, Any]
"""Return a dictionary representation of the model."""

Examples

name: my-job
entrypoint: python main.py
# An inline dictionary can also be provided.
compute_config: my-compute-config:1
# A containerfile path can also be provided.
image_uri: anyscale/image/my-image:1

Job CLI

anyscale job submit

Usage

anyscale job submit [OPTIONS] [ENTRYPOINT]...

Submit a job.

The job config can be specified in one of the following ways:

  • Job config file can be specified as a single positional argument. E.g. anyscale job submit config.yaml.

  • Job config can also be specified with command-line arguments. In this case, the entrypoint should be specified as the positional arguments starting with --. Other arguments can be specified with command-line flags. E.g.

    • anyscale job submit -- python main.py: submit a job with the entrypoint python main.py.

    • anyscale job submit --name my-job -- python main.py: submit a job with the name my-job and the entrypoint python main.py.

  • [Experimental] If you want to specify a config file and override some arguments with the commmand-line flags, use the --config-file flag. E.g.

    • anyscale job submit --config-file config.yaml: submit a job with the config in config.yaml.

    • anyscale job submit --config-file config.yaml -- python main.py: submit a job with the config in config.yaml and override the entrypoint with python main.py.

By default, this command submits the job asynchronously and exits. To wait for the job to complete, use the --wait flag.

Options

  • -n/--name: Name of the job.
  • -f/--follow: DEPRECATED: use --wait instead.
  • -w/--wait: Block this CLI command and print logs until the job finishes.
  • --config-file: Path to a YAML config file to use for this job. Command-line flags will overwrite values read from the file.
  • --image-uri: Container image to use for this job. When running in a workspace, this defaults to the image of the workspace.
  • --containerfile: Path to a containerfile to build the image to use for the job. This is exclusive with --image-uri.
  • -e/--env: Environment variables to set for the job. The format is 'key=value'. This argument can be specified multiple times.
  • --working-dir: Path to a local directory that will be the working directory for the job. The files in the directory will be automatically uploaded to cloud storage. When running in a workspace, this defaults to the current working directory.

Job SDK

anyscale.job.submit

Submit a job.

Arguments

  • config (JobConfig): The config options defining the job.

Returns: str

Examples

import anyscale
from anyscale.job.models import JobConfig

anyscale.job.submit(
JobConfig(
name="my-job",
entrypoint="python main.py",
working_dir=".",
),
)