Skip to content

Shell Reference#

Configuration#

config.bashrc#

Configuration file for building and running nwm-rte.

Description

This file is source'd by multiple shell scripts within nwm-rte, and its variables control various aspects of the installation / setup / build as well as the runtime environment itself (e.g. mount paths).

Choices of Component Packages:

For each component package (nwm-msw-mgr, nwm-fcst-mgr, etc), you can choose from an empty string (""), or "LOCAL", or any other string.

  • If an empty string "" is provided, the package will be skipped (will not be installed).

  • If "LOCAL" is provided, the package will install from the code currently on disk.

  • If any other string is provided, the package will install from GitHub, and the string must be a valid tag, branch, or commit.

Environment Variables
  • GH_ORG:

    GitHub Organization Name

  • NO_CACHE:

    Docker cache directive passed to docker build call. Choose from: ["--no-cache", ""]

  • STAGE:

    The stage of the docker build sequence to stop at. Choose from: ["ngen_rte_base", "ngen_rte_eval_verf"]. See Dockerfile.rte for more information

  • INSTALL_DEBUGGERS:

    Passed to ./install_debuggers.sh, causes pip package debugpy and OS package gdb to be installed. Choose from: ["NO", "YES"]

  • NGEN_LOG_TO_RTE:

    Causes ngen's log files to be written to host path ./logs/ngen/, mounted in the container at /ngen-app/logs_rte/ngen/, rather than to the realization's directory. Choose from: ["NO", "YES"]

  • RTE_EWTS_ENABLED:

    Switch for optional nwm-ewts integration. When enabled, EWTS is used for RTE logging, and modules' logs are treated as data via parsing of status Payloads and LogParts. Choose from: ["NO", "YES"]

    Component Packages#

  • REPO_TAG_FCST_MGR:

    Source for nwm-fcst-mgr.

  • REPO_TAG_MSW_MGR:

    Source for nwm-msw-mgr.

  • REPO_TAG_CAL_MGR:

    Source for nwm-cal-mgr.

  • REPO_TAG_REGION_MGR:

    Source for nwm-region-mgr.

  • REPO_TAG_DATA_ASSIM_ENGINE:

    Source for nwm-data-assimilation.

  • REPO_TAG_NGEN_FORCING:

    (Optional). Source for re-installing ngen-forcing Python package. Note that this does not affect the sourcing of the base image.

  • REPO_TAG_EWTS:

    (Optional). Source for re-installing the nwm-ewts Python package. Note that this does not affect the sourcing of the base image.

  • REPO_TAG_EVAL:

    Source for nwm-eval-mgr.

  • REPO_TAG_ECF_TASK_MGR:

    Source for ecf_task_mgr Python package.

    Sourcing of ngen Base#

  • NGEN_SOURCE_MODE:

    Affects the path of building, the choice of the base ngen image. Choose from: ["ghcr", "existing_local_tag", "build_from_local", "build_from_remote"]

    • "ghcr" causes the ngen base image to be sourced from an existing GitHub Container Registry image that has already been published to the ngen GitHub repository.

    • "existing_local_tag" causes the ngen base image to be sourced from an existing local image tag.

    • "build_from_local" and "build_from_remote" cause the ngen base image to be built from the bottom up (build ngen-forcing, then build ngen, then add the RTE layers). The former uses the existing state of the codebase currently on disk for ngen-forcing and for ngen, and the latter uses the provided GH refs for sourcing the codebases of ngen-forcing and ngen.

  • NGEN_BASE__REMOTE_GHCR_TAG:

    (Only used when NGEN_SOURCE_MODE="ghcr"). GHCR image tag to use, e.g. "latest" or a commit hash.

  • NGEN_BASE__EXISTING_LOCAL_TAG:

    (Only used when NGEN_SOURCE_MODE="existing_local_tag"). Choose any existing local image tag.

  • FORCING_BASE_REMOTE_TAG:

    Only used when NGEN_SOURCE_MODE="build_from_remote". Source of the ngen-forcing codebase. Choose any GitHub tag or branch.

  • NGEN_BASE_REMOTE_TAG:

    Only used when NGEN_SOURCE_MODE="build_from_remote". Source of the ngen codebase. Choose any GitHub tag or branch.

    Target Image Name#

  • TARGET_IMAGE_NAME:

    Freeform name for output image tag that is built.

    Misc#

  • THIS_SCRIPTS_GRANDPARENT_DIR:

    The grandparent directory of this current file. When using REPOS_COMMON_ROOT__HOST=${THIS_SCRIPTS_GRANDPARENT_DIR}, then the other NWM repos are assumed to be siblings of the nwm-rte repo.

  • REPOS_COMMON_ROOT__HOST:

    • ./setup_clone_repos.sh sets up this local directory and clones sibling repos into here. ./setup_data.sh downloads data into here.

    • ./ngen_rte_build.sh uses this to find ngen source code when NGEN_SOURCE_MODE="build_from_local"

    • ./run.sh mounts various subdirectories and files from this local directory into the container during runtime.

    • Choices for this variable:

      • A typical choice for this is ${THIS_SCRIPTS_GRANDPARENT_DIR}, which is equivalent to "${HOME}/${GH_ORG,,}" if you run this from "${HOME}/${GH_ORG,,}/nwm-rte" but another location such as "${HOME}/${GH_ORG,,}__rte" could be used if wanting to isolate the RTE from other work.
      • Using ${THIS_SCRIPTS_GRANDPARENT_DIR} guarantees that the setup scripts (./setup_data.sh and ./setup_clone_repos.sh) will copy data and clone repos into the same locations where the build script and run script will look for them.

    Mounts for data#

  • RUN_NGEN_ROOT__HOST:

    Root of the working data associated with running realizations (dirs for realizations and for intermediary data).

  • MNT__RUN_NGEN__HOST:

    Alias for RUN_NGEN_ROOT__HOST. Host path mounted by the container.

  • MNT__RUN_NGEN__CONTAINER:

    path within the container to which the host disk path described by MNT__RUN_NGEN__HOST is mounted.

    Mounts for regionalization data#

  • MNT__NWM_REGION_MGR__INPUT_DATA:

    Host path to input data from nwm-region-mgr repository for regionalization workflows. Mounted by the container.

    Installed regionalization results#

  • INSTALLED_REGIONALIZATION_RESULTS:

    Results of regionalization, baked into the MSWM Python package, which can be used for testing the regionalized form of ngen forecasts. Not resolved eagerly here (most scripts sourcing this file don't need it). Call resolve_installed_regionalization_results from scripts that do need this (only after building the image). Remote data sources for setup_data.sh and setup_data_one_gage.sh

  • SOURCE_BUCKET_DEV:

    Name of cloud bucket (no s3:// prefix in the string), used by setup_data.sh and setup_data_one_gage.sh

  • SOURCE_PREFIX_ROOT:

    Prefix root for data sourced from SOURCE_BUCKET_DEV, used by setup_data.sh and setup_data_one_gage.sh

  • EDFS_API_VERSION:

    For EDFS API URL for nwm-rte setup scripts. Used to construct URL path component. Not used during actual runtime. Test gage and associated domain. NOTE if updating these, may also need to update bin_mounted/ngen_rte/consts.py EDFS_URL_TEST / EDFS_URL_OE Base URLs for the Icefabric EDFS API Should match ngen_rte/consts.py's TEST_EDFS_URL / OE_EDFS_URL Logging functions File operation functions

Caveats

REPO_TAG_NGEN_FORCING and REPO_TAG_EWTS reinstall only the respective Python packages on top of the existing ngen base image. They do not rebuild the base image or incorporate changes to C++ code, Dockerfiles, or other base-image build instructions. These optional reinstalls occur before the component manager packages are installed. They write new source metadata files at /ngen-app/git-info/ngen-forcing_git_info.json and /ngen-app/git-info/nwm-ewts_git_info.json, respectively. Neither reinstall updates the existing /ngen-app/ngen-bmi-forcing_git_info.json export inherited from the base image. That inherited file continues to describe the ngen-forcing and nwm-ewts versions that were included when the base image was built.

Notes

For Pattern VAR_NAME=${VAR_NAME:-"some_string"}, values set with this pattern use the existing value if it already exists, e.g. as an OS env var, otherwise they default to "some_string".

Setup#

setup_clone_repos.sh#

Clone various NWM repositories.

Usage 
./setup_clone_repos.sh ssh
./setup_clone_repos.sh https shallow
Description

Clone repos from GitHub. Includes CLI option for using SSH (for r+w, requires credentials) or HTTPS (for read-only, no credentials required). Includes CLI option for a shallow (depth 1) clone or full clone. It will not alter the state of existing repos on your disk (for each clone, it skips if the folder already exists on disk).

Uses OS env var REPOS_COMMON_ROOT__HOST from config.bashrc.

Has 2 positional arguments and 0 named arguments.

Options
  • SSH_OR_HTTPS:

    Required. Specify to clone via SSH (for read-write) or via HTTPS (for read-only). Choices: ["ssh", "https"]

  • SHALLOW_CLONE:

    Optional. Specify to perform a shallow (--depth 1) clone or a deep clone. Choices: ["full", "shallow"]. Default: "full"

setup_data.sh#

Download various global input data, as well as specific input data for one example gage.

Usage 
./setup_data.sh 
./setup_data.sh -a 'oe'
./setup_data.sh -a 'test' -r
./setup_data.sh -a 'oe' -r
Description

Download input data from S3 and from APIs. Not needed for the "default" gage at the "default" realization, since those inputs are included in the repository. Calls ./setup_data_one_gage.sh for the default gage ID.

Uses various OS env vars from config.bashrc.

Has 0 positional arguments and 2 named arguments.

Options
  • -a, --api-environment:

    For EDFS API URL for nwm-rte setup scripts. Used to construct URL subdomain. Not used during ngen runtime. See -e, --environment in the ngen runtime CLI scripts. EDFS base URL set in config.bashrc for use in setup_data_one_gage.sh. Default: 'test'. Common choices: ['test', 'oe'].

  • -r, --regionalization:

    If provided, extra data will be downloaded for the regionalization workflow. Should be ran after cloning the nwm-region-mgr repository.

setup_data_one_gage.sh#

Download input data for one gage.

Usage 
./setup_data_one_gage.sh "01121330" "CONUS" "test"
Description

Download retrospective streamflow data from S3 and observed streamflow data from EDFS for one USGS gage. Not needed for the "default" gage at the "default" realization, since those inputs are included in the repository.

Uses various OS env vars from config.bashrc.

Has 3 positional arguments and 0 named arguments.

Options
  • GAGE_ID:

    The USGS gage ID to be downloaded, e.g. "01121330"

  • DOMAIN:

    The domain that the gage ID resides in, e.g. "CONUS"

  • EDFS_API_ENVIRONMENT:

    For EDFS API URL for nwm-rte setup scripts. Used to construct URL subdomain. Not used during ngen runtime. See -e, --environment in the ngen runtime CLI scripts. Default: 'test'. Choices: ['test', 'oe'].

Build#

ngen_rte_build.sh#

Build the ngen runtime environment Docker image.

Usage 
./ngen_rte_build.sh
Description

Build the ngen runtime environment Docker image, applying various OS env vars from config.bashrc.

Can build the ngen-forcing and ngen base images if not wishing to source ngen from an existing GHCR image or an existing local image.

Components can be installed from GitHub, or from local source code, or skipped.

The image built is tagged using env var TARGET_IMAGE_NAME (typically from config.bashrc).

When the script exits, it echoes the image name and provides a command that can be used to enter the container interactively.

Alternatively, the user can run ./run.sh after the build is complete.

Has 0 positional arguments and 0 named arguments.

Requirements:

If configured to install packages from local sources instead of from GitHub, then this script assumes this is ran from a sibling directory of those repositories (already cloned, checked out, pulled). ./setup_clone_repos.sh can be used to clone the relevant repos.

Run pytest Without Dev Container#

See also: Steps to Run: pytest

run_test_pytest.sh#

Runs an ephemeral docker container to execute python -m pytest and quit. An alternative to running pytest inside an interactive Dev Container.

Usage 
From `nwm-rte` repository root:
./run_test_pytest.sh
Notes

Uses OS env var TARGET_IMAGE_NAME if it is set, otherwise default to "ngen_rte_ghcr".

Enter the Container Interactively#

run.sh#

Script and sourceable function for running the ngen runtime environment Docker image.

Usage 
./run.sh 
source ./run.sh && docker_run python "/path/to/script.py" --arg_for_script
Description

Run the ngen runtime environment Docker image. Requires that the image has already been built.

The function docker_run is sourced by and called by other scripts in the repository.

When run as a script, causes the user to enter the container shell interactively.

Various OS env vars are applied from config.bashrc. Notably TARGET_IMAGE_NAME is the image that is sourced (must have already been built) and launched as a container.

Various mount paths are also applied, some defined in config.bashrc and others defined in ./run.sh itself. * Input data paths * Output data paths from previous runs * Scripts from ./bin_mounted/ * Cached files from ./devcontainer/tmp/

This script has 0 positional arguments and 0 named arguments. See the description for function docker_run, which is typically sourced by other scripts and ran externally.

Requirements:

The ngen runtime environment image (defined by TARGET_IMAGE_NAME) has already been built.

The various repositories and input data needed to run are available and mountable (see ./setup_clone_repos.sh, ./setup_data.sh, ./setup_data_one_gage.sh).

Functions

docker_run#

Start the ngen runtime environment container with various mounts applied. Either run in an interactive terminal, or run a specific command and exit. If arguments are provided, then the 1st argument is the entrypoint command, and all subsequent arguments are arguments for that entrypoint command. See usage examples in ./run_default.sh, etc.

Arguments
  • entrypoint:

    If "/bin/sh", then start an interactive session. Else, run the provided command and provide the extra arguments as the arguments to that command.

  • *:

    Any number of extra arguments, provided to the entrypoint command.

Run Forecasts and Calibrations#

run_default.sh#

Example commands for running "default" realizations. See CLI args for run_default.py.

Usage 
./run_default.sh
Description

Source ./run.sh to call its docker_run command for running "default" realizations. It requires that the ngen runtime environment image has already been built using ./ngen_rte_build.sh.

Various OS env vars are applied from config.bashrc. Notably TARGET_IMAGE_NAME is the image that is sourced (must have already been built) and launched as a container. Various data mount paths are also applied from config.bashrc.

Has 0 positional arguments and 0 named arguments.

Requirements:

The ngen runtime environment image (defined by TARGET_IMAGE_NAME) has already been built.

The various repositories and input data needed to run are available and mountable (see ./setup_clone_repos.sh, ./setup_data.sh, ./setup_data_one_gage.sh).

run_calib.sh#

Example commands for running "calibration" realizations. See CLI args for run_calibration.py.

Usage 
./run_calib.sh
Description

Source ./run.sh to call its docker_run command for running "calibration" realizations. It requires that the ngen runtime environment image has already been built using ./ngen_rte_build.sh.

Various OS env vars are applied from config.bashrc. Notably TARGET_IMAGE_NAME is the image that is sourced (must have already been built) and launched as a container. Various data mount paths are also applied from config.bashrc.

Has 0 positional arguments and 0 named arguments.

Requirements:

The ngen runtime environment image (defined by TARGET_IMAGE_NAME) has already been built.

The various repositories and input data needed to run are available and mountable (see ./setup_clone_repos.sh, ./setup_data.sh, ./setup_data_one_gage.sh).

run_fcst.sh#

Example commands for running "forecast" realizations. See CLI args for run_forecast.py.

Usage 
./run_fcst.sh
Description

Source ./run.sh to call its docker_run command for running "forecast" realizations. It requires that the ngen runtime environment image has already been built using ./ngen_rte_build.sh.

Various OS env vars are applied from config.bashrc. Notably TARGET_IMAGE_NAME is the image that is sourced (must have already been built) and launched as a container. Various data mount paths are also applied from config.bashrc.

Requirements:

The ngen runtime environment image (defined by TARGET_IMAGE_NAME) has already been built.

The various repositories and input data needed to run are available and mountable (see ./setup_clone_repos.sh, ./setup_data.sh, ./setup_data_one_gage.sh).

A "calibration" realization has already been ran for the gage of interest, and its output is available at the expected location.

This script has 1 positional arguments and 0 named arguments.

Options
  • fcst_run_name:

    Optional. Default="fcst_run". Specify to choose the name of the forecast run.

run_suite.sh#

Example commands for running realizations of various types. See CLI args for run_forecast.py, run_default.py, run_calibration.py, and run_tests.py.

Usage 
./run_suite.sh
Description

Source ./run.sh to call its docker_run command for running realizations of various types ("default", "calibration", and "forecast"). It requires that the ngen runtime environment image has already been built using ./ngen_rte_build.sh.

Various OS env vars are applied from config.bashrc. Notably TARGET_IMAGE_NAME is the image that is sourced (must have already been built) and launched as a container. Various data mount paths are also applied from config.bashrc.

Has 0 positional arguments and 0 named arguments.

Requirements:

The ngen runtime environment image (defined by TARGET_IMAGE_NAME) has already been built.

The various repositories and input data needed to run are available and mountable (see ./setup_clone_repos.sh, ./setup_data.sh, ./setup_data_one_gage.sh).

run_test_formulations.sh#

Example commands for running a series of "forecast" realizations via run_tests.py with various forcing configurations, optionally preceded by building and running "calibration" realizations first. See CLI args for run_tests.py.

Usage 
./run_test_formulations.sh
Description

Source ./run.sh to call its docker_run command for running run_tests.py. It requires that the ngen runtime environment image has already been built using ./ngen_rte_build.sh.

When realizations fail, this program does not halt, but rather moves to the next configuration type in the list, with the goal of "trying" many different realization configurations in one call.

The status of each configuration's build step and run step is reported and written to a json file at the end.

The Python script run_tests.py includes options for stopping realizations mid-way through their run, rather than waiting for them to complete.

Various OS env vars are applied from config.bashrc. Notably TARGET_IMAGE_NAME is the image that is sourced (must have already been built) and launched as a container. Various data mount paths are also applied from config.bashrc.

Requirements:

The ngen runtime environment image (defined by TARGET_IMAGE_NAME) has already been built.

The various repositories and input data needed to run are available and mountable (see ./setup_clone_repos.sh, ./setup_data.sh, ./setup_data_one_gage.sh).

This script has 1 positional arguments and 0 named arguments.

Options
  • fcst_run_name:

    Optional. Default="fcst_run". Specify to choose the name of the forecast run.

run_region_standalone.sh#

Example commands for running "regionalization" realizations. See CLI args for run_regionalization_standalone.py.

Usage 
./run_region_standalone.sh
Description

Source ./run.sh to call its docker_run command for running "regionalization" realizations. It requires that the ngen runtime environment image has already been built using ./ngen_rte_build.sh.

Various OS env vars are applied from config.bashrc. Notably TARGET_IMAGE_NAME is the image that is sourced (must have already been built) and launched as a container. Various data mount paths are also applied from config.bashrc.

Has 0 positional arguments and 0 named arguments.

Requirements:

The ngen runtime environment image (defined by TARGET_IMAGE_NAME) has already been built.

The various repositories and input data needed to run are available and mountable (see ./setup_clone_repos.sh, ./setup_data.sh, ./setup_data_one_gage.sh).

Run Output Postprocessing#

For complete workflow documentation, see the nwm-data-assimilation repository.

run_output_postprocess.sh#

Example commands for running the "output_postprocess" (overall_netcdf_workflow). See CLI args for run_output_postprocess.py.

Usage 
./run_output_postprocess.sh
Description

Source ./run.sh to call its docker_run command for running NetCDF output postprocess workflows. It requires that the ngen runtime environment image has already been built using ./ngen_rte_build.sh.

Various OS env vars are applied from config.bashrc. Notably TARGET_IMAGE_NAME is the image that is sourced (must have already been built) and launched as a container. Various data mount paths are also applied from config.bashrc.

Has 0 positional arguments and 0 named arguments.

Requirements:

The ngen runtime environment image (defined by TARGET_IMAGE_NAME) has already been built.

The various repositories and input data needed to run are available and mountable (see ./setup_clone_repos.sh, ./setup_data.sh, ./setup_data_one_gage.sh).

run_output_mosaic.sh#

Example commands for running the "output_mosaic" workflow.. See CLI args for run_output_mosaic.py.

Usage 
./run_output_mosaic.sh
Description

Source ./run.sh to call its docker_run command for running NetCDF output mosaicing workflows for combining gridded output files. It requires that the ngen runtime environment image has already been built using ./ngen_rte_build.sh.

Various OS env vars are applied from config.bashrc. Notably TARGET_IMAGE_NAME is the image that is sourced (must have already been built) and launched as a container. Various data mount paths are also applied from config.bashrc.

Has 0 positional arguments and 0 named arguments.

Requirements:

The ngen runtime environment image (defined by TARGET_IMAGE_NAME) has already been built.

The various repositories and input data needed to run are available and mountable (see ./setup_clone_repos.sh, ./setup_data.sh, ./setup_data_one_gage.sh).

Coastal Forcing#

For complete coastal workflow documentation, see the nwm-coastal repository.

make_coastal_forcing.sh#

Generate gridded atmospheric forcing files for coastal models.

Usage 
./bin_mounted/ngen_rte/coastal/make_coastal_forcing.sh 
./bin_mounted/ngen_rte/coastal/make_coastal_forcing.sh "coastal_test"
Description

Run ngen_rte.coastal.make_coastal_forcing in the RTE container. The calls below provide examples for operational cycles, retrospective windows, reusable regrid weights, and supported domains.

See the nwm-coastal repository, including scripts/setup_data_coastal.sh, for complete coastal workflow and input-data setup documentation.

Has 0 required positional arguments and 1 optional positional argument.

Options
  • RUN_NAME:

    Optional prefix used for run names in the example calls. Default: "coastal".

Additional Utilities#

install_debuggers.sh#

Install debugger packages. Used by Dockerfile.rte.

Usage 
./install_debuggers.sh YES
./install_debuggers.sh NO
Description

Install debugger packages: python package debugpy and OS package gdb. Used by Dockerfile.rte. Optionally skip installation (noop). The choice of noop is to abstract the flow control away into a parameter to work around the non-branching nature of Docker build flow control.

Has 1 positional argument and 0 named arguments.

Options
  • install_debuggers:

    Choose from: ["YES", "NO"]. If "NO", this is a noop.

install_package.sh#

Install a Python package, with options, and write git info json. Used by Dockerfile.rte

Usage 
./install_package.sh python nwm-region-mgr development "[parreg]"
Description

Install a Python package, with options for the python environment, local vs remote, tag, and "extras". Used by Dockerfile.rte.

Also call add_git_info.py to dump the package's git info to a json file.

Treats ngen-forcing reinstall as special since it necessarily is already installed in the base image, so if it is specified here, it receives a --force-reinstall flag.

Has 7 positional arguments and 0 named arguments.

Options
  • python_exe:

    The python executable to use for installation

  • gh_org:

    The GitHub organization name

  • repo_name:

    The name of the repository to be installed

  • distribution:

    The name of the distribution to be installed (often the same as the repository name, but not always).

  • repo_remote_tag:

    If "LOCAL", then install from local code. If empty string, skip installation. If other string, install that ref (tag, branch, or commit) from GitHub.

  • extras:

    Optional extras to include in the pip install directive, e.g. pip install package_name[extras]

  • repo_subdirectory:

    Optional subdirectory within the repo to install from, e.g. for pip install package_name @ git+https://github.com/org/repo@tag#subdirectory=subdir

Run Regionalization Workflows#

For more information, see the nwm-region-mgr repository.

run_region.sh#

Script to run NWM regionalization, NGEN simulation, and evaluation workflows using an RTE Docker container.

Usage 
<root-path-to-script>/run_region.sh [OPTIONS]
Description

Takes no positional arguments. Workflow steps are specified using named command-line arguments.

Options
  • -p, --parreg:

    Run parameter regionalization (includes formulation)

  • -f, --formreg:

    Run formulation regionalization only

  • -n, --ngen:

    Run NGEN simulation

  • -e, --eval:

    Run evaluation

  • -c, --config-dirDIR:

    (default: "./configs") Set config directory

  • -r, --rte-pathPATH:

    (default: "$(realpath .)/rte_scripts") Set path to the directory containing RTE scripts

  • -i, --image:

    (default: ghcr.io/ngwpc/nwm-rte) Set Docker image to use for running the workflows

  • -t, --image-tagTAG:

    (default: "latest") Set Docker image tag

  • --pull-image:

    (optional switch) Pull the latest Docker image before running

  • -d, --delete-runtime-dir:

    (optional switch) Delete runtime directory after completion

  • -h, --help:

    Show this message and exit

Examples

Run parameter regionalization

 
#Sample config files can be found in nwm-region-mgr repo under configs directory
./nwm-rte/run_region.sh --parreg
./nwm-rte/run_region.sh --parreg -c configs
./nwm-rte/run_region.sh -p -c configs
./nwm-rte/run_region.sh -p -c ~/ngwpc/nwm-region-mgr/configs

Run formulation regionalization only

 
./nwm-rte/run_region.sh --formreg -c configs

Run NGEN simulation

 
./nwm-rte/run_region.sh --ngen -c configs

Run evaluation

 
./nwm-rte/run_region.sh --eval -c configs

Run multiple steps

 
./nwm-rte/run_region.sh --parreg --ngen -c configs

Run with run_regionalization.py at a different location (default: ./rte_scripts/run_regionalization.py)

 
./nwm-rte/run_region.sh --ngen -c configs -r ~/repos/nwm-rte/bin_mounted/ngen_rte
(where -r specifies the path to the directory containing RTE scripts)

Run with different Docker image & tag

 
./nwm-rte/run_region.sh --ngen -c configs -i ghcr.io/noaa-owp/nwm-rte -t pr-20-build

Run with pulling the latest Docker image before running

 
./nwm-rte/run_region.sh --ngen -c configs --pull-image

Delete runtime directory after completion (default: keep for debugging)

 
./nwm-rte/run_region.sh -c configs -p -n -e -d

sbatch_run_region.sh#

Submits the regionalization workflow to SLURM using sbatch.

Usage 
./sbatch_run_region.sh [CONFIG_DIR] [WORKFLOW_OPTIONS] [OPTIONS]
Description

Submit the regionalization workflow to SLURM using sbatch.

Options
  • CONFIG_DIR:

    (optional, default: "configs") Directory containing configuration YAML files

  • WORKFLOW_OPTIONS:

    (optional, default: "parreg") Workflow steps to run: parreg, formreg, ngen, or eval

  • -n, --dry-run:

    (optional switch) Print the generated SLURM script instead of submitting

  • -i, --imageIMAGE:

    (optional, default: "ghcr.io/ngwpc/nwm-rte") Docker image to use for the RTE

  • -t, --image-tagTAG:

    (optional, default: "latest") Docker image tag to use for the RTE

  • -s, --scriptRUN_REGION_SCRIPT:

    (optional, default: "/ngen-app/nwm-rte/run_region.sh") Path to the regionalization script to run inside the Docker container

  • -w, --walltimeWALLTIME:

    (optional, default: "48:00:00") Walltime for the SLURM job

  • -p, --pull-image:

    (optional switch) Pull the latest Docker image before running

  • -d, --delete-runtime-dir:

    (optional switch) Delete runtime directory after completion

  • -h, --help:

    Display this help message and exit