sliderule-cluster

sliderule-cluster is a command line tool for submitting requests to a SlideRule web service cluster. It can be used to check on the health and identity of a cluster, query cluster defaults, search for science data granules (CMR/Earthdata), and run any of SlideRule’s dataframe-based processing APIs (e.g. atl06x, atl03x) from a shell or script.

Quick Start¶

# Which cluster am I talking to?
sliderule-cluster whoami

# What version is it running?
sliderule-cluster version

# Run an API request using a parameters file; prints the path of the output file
sliderule-cluster run atl06x --parms request.json

where request.json might look like:

{
  "poly": [
    {"lon": -108.3, "lat": 38.8},
    {"lon": -107.8, "lat": 38.8},
    {"lon": -107.8, "lat": 39.2},
    {"lon": -108.3, "lat": 39.2},
    {"lon": -108.3, "lat": 38.8}
  ],
  "t0": "2023-06-01T00:00:00Z",
  "t1": "2023-07-01T00:00:00Z"
}

Usage¶

sliderule-cluster <command> [options]

Run sliderule-cluster --help for a list of commands, or sliderule-cluster <command> --help for the options of a specific command.

Common Options¶

The following options are accepted by every command. They are attached to each subcommand, so they must appear after the command name:

Command Reference¶

whoami¶

sliderule-cluster whoami [common options]

Asks the cluster to identify itself. Useful as a quick connectivity check and to confirm that --domain, --cluster, and --user_service are resolving to the service you expect.

Examples:

# get the internal name of the public cluster
sliderule-cluster whoami

# get the internal name of the cluster running at sliderule.testsliderule.org
sliderule-cluster whoami --domain testsliderule.org --cluster sliderule

status¶

sliderule-cluster status [common options]

Reports the registration status of the cluster with the discovery service. When connected to a non-public service (for example when using --user_service), the request is scoped to that service.

Examples:

# get the number of nodes registered on the public cluster
sliderule-cluster status

# get the number of nodes registered to the user's private service
sliderule-cluster status --user_service

version¶

sliderule-cluster version [common options]

Returns version information for the cluster (and the client used to reach it).

Examples:

# get version of public cluster
sliderule-cluster version

# write version of public cluster to the file "version.json"
sliderule-cluster version --result version.json

defaults¶

sliderule-cluster defaults [common options]

Returns the cluster’s default request parameter values. This is a helpful reference when deciding which parameters you actually need to set in a request.

Examples:

# return defaults for public cluster
sliderule-cluster defaults

earthdata¶

sliderule-cluster earthdata [options] [common options]

Queries the Earthdata catalog (CMR) through SlideRule to find the science data resources (granules) that match an area of interest and time range. Use it to preview what a run request will process.

Only one area-of-interest option is used. If more than one is given, the precedence is --poly, then --geojson, then --bbox. Options that are not supplied are omitted from the request so that cluster defaults apply.

Examples:

# Granules over a bounding box in June 2023
sliderule-cluster earthdata \
    --short_name ATL03 \
    --bbox -108.3 38.8 -107.8 39.2 \
    --t0 2023-06-01T00:00:00Z --t1 2023-07-01T00:00:00Z

# Only resources whose names match a pattern, with metadata
sliderule-cluster earthdata --geojson aoi.geojson --name_filter "ATL03_2023" --with_meta

run¶

sliderule-cluster run <api> --parms <file> [common options]

Executes a dataframe-based API request against the cluster and delivers the result as an output file (GeoParquet by default).

Request parameters¶

The JSON in the parameters file is the same set of parameters that would be passed to the endpoint through any other SlideRule client. Refer to the API reference for the parameters supported by each endpoint. Use sliderule-cluster defaults to see the cluster’s defaults.

Output handling¶

Where the result ends up depends on the output block of your request parameters:

On completion the tool prints the location of the output file (a local path or a remote URL).

Examples:

# Default output: a GeoParquet file in the temp directory
sliderule-cluster run atl06p --parms request.json

# Save the printed output location to a file for use by a script
sliderule-cluster run atl06p --parms request.json --result output_path.txt

Additional Topics¶

Error Handling¶

By default, errors are caught and reported as a single line:

Unhandled error: <message>

Pass --verbose to raise the full exception and traceback instead, which is the best first step when troubleshooting.

Saving Results with --result¶

--result <file> writes the command’s result string to a file. For run this is the output file’s path or URL, which makes it convenient for chaining commands in shell scripts:

sliderule-cluster run atl06p --parms request.json --result out.txt
OUT=$(cat out.txt)

Environment and Connectivity Notes¶

• The tool connects to --cluster at --domain; the defaults target the public SlideRule service.

• whoami and status talk to the discovery service, and do not retry.

• Long-running run requests stream results back from the cluster; use --verbose to see progress messages.