Day 10: How to use Ray Runtime Environment Dependencies

Continuing with our #30DaysOfRay journey, Day 10 explores the developer’s environment dependencies. These dependencies include environment variables, small files on your laptop, persisted local models or models on a public cloud’s object store, and Python packages not part of the cluster image yet needed to run Python or model code on a Ray cluster.

On Day 9, we alluded to environment variables as part of a runtime environment. In this article, we elaborate on them and additional runtime dependencies such as Python modules, models, or config or data files.

For a production environment, you’ll want all your Python packages as part of the final container image and any models accessible via a model registry or a public cloud object store, along with meta-data files if needed as part of your runtime environment.

As recommended in the documentation, for dynamic environments (e.g., for development and experimentation), we recommend using runtime environments since it’s easy to iterate and alter when required.

Let’s consider a runtime environment we want to use with our Python script. In this script, we want to specify the following:

1. Environment variables that Ray tasks will access on the cluster
2. Local directory housing files with some meta-data and persisted models, albeit, in production, this should be stored in model store or public cloud bucket
3. List of packages or Python modules as dependencies

Fig. 1 Runtime environment dictionary

Using Pip or Conda dependencies

Flexibility is as vital as ease of use. Ray offers a choice to create a virtualenv or conda environment on the cluster. To replicate the above as a conda runtime environment on the cluster, simply use the following dictionary definition:

Fig 2. The runtime dictionary definition for conda dependencies

For conda dependencies, Ray will create a new conda environment, activate it, and install all the packages listed, along with installing Ray afresh since it does not inherit any packages in the newly activated conda environment. By contrast, when using pip, as we did in our example, Ray will create a virtualenv, inherit the base packages, and only install packages or modules listed.

Whether using conda or virtualenv, Ray caches the runtime dependencies on the cluster, so any repeated execution of your Ray application on the same cluster uses the cached runtime environment (see Fig. 5).

Accessing variables and artifacts in a Ray task

Two things must happen to access any of the environment variables or artifacts accessible via the uploaded local directory. First, a runtime_env argument is specified per task in its @ray.remote decorator as @ray.remote(runtime_env=my_runtime_env) or ray.init(runtime_en=my_runtime_env). The former is specific and applies only to a Ray task.

Second, the working directory is uploaded and unzipped, and all packages are installed, running in its conda or virtualenv environment. In contrast, the latter is cluster-wide for all tasks and actors, as shown below.

Fig. 3 Runtime environment dictionary specified as an argument for cluster-wide access to all tasks and actors.

The Ray task code below shows how to access the environment variables and artifacts uploaded to the cluster from a local working directory. The local directory “feature_files” is zipped and uploaded, and its URI is stored in the GCS. Note there is a 100 MiB limit for uploading local working directories.

Fig. 4 Ray tasks accessing environment variables and local directory artifacts and files uploaded to the cluster.

Finally, when we run this script, the output below indicates that the Ray task accessed all environment variables. Ray zipped and uploaded the local working directory and unzipped it into the node’s local directory, where the Ray task could access the artifacts. And Ray installed all the packages in a virtualenv.

Fig. 5 Output from the run showing feature_file zipped and uploaded onto GCS and the model output.

To sum up, runtime environments are a convenient and flexible way to test your Python code or models locally on your laptop and then scale to a Ray cluster. And to scale it to a cluster, you can employ Ray runtime environment dependencies to replicate your local runtime environment. Not only can you specify per Ray job, per-task, or per-actor runtime environments. You can do so for cluster-wide runtime environments where all tasks and actors have access to their artifacts, packages, and environment variables.

The full source code for this example is available here, and peruse the detailed documentation on Ray’s runtime environments dependencies.