Output Directory Structure#
The outputs generated by nwm-eval-mgr are stored in file_paths.output_dir, organized in a set of five
sub-directories correpsonding to the five steps:
output_dir/
├── <dataset_name>/ # output from the "fetch_fcst_data" step: forecast/simulation data organized by dataset name
├── usgs/ # output from the "fetch_obs_data" step: usgs streamflow observations
├── joined/ # output from the "pair_data" step: paired forecast and observation data
├── metrics/ # output from the "compute_metrics" step: evaluation metrics computed from the paired datasets
└── plots/ # output from the "plot_metrics" step: evaluation plots generated from the computed metrics
The specific files in each sub-directory depend on the configurations. Below are sample output directory structures for each supported forecast data source, as defined in the corresponding sample configuration files.
ngenCERF#
Sample output directory structure for nwm_forecast.data_source = ngenCERF, as defined in sample configuration
file config_ngencerf.yaml.
usgs_01123000/
├── joined/
│ ├── noah_cfes.ngen.short_range.joined.parquet
├── metrics/
│ ├── noah_cfes.ngen.short_range.metrics.parquet
├── noah_cfes/
│ ├── short_range/
│ │ ├── 20221201T00.parquet
├── plots/
│ ├── short_range/
│ │ ├── barchart/
│ │ │ ├── barchart_all_metrics_01123000.png
│ │ ├── metric_table/
│ │ │ ├── metric_table_lead_h1-18_01123000.png
│ │ ├── time_series/
│ │ │ ├── time_series_T0_20221201T00.png
├── usgs/
│ ├── 2022-12-01_2022-12-03.parquet
├── nwm_verf_config_expanded.yaml
├── verification.log
hindcast#
Sample output directory structure for nwm_forecast.data_source = hindcast, as defined in sample configuration
file config_hindcast.yaml.
usgs_01123000_hindcast/
├── joined/
│ ├── noah_cfes.ngen.short_range.joined.parquet
│ ├── noah_sac.ngen.short_range.joined.parquet
├── metrics/
│ ├── noah_cfes.ngen.short_range.metrics.csv
│ ├── noah_cfes.ngen.short_range.metrics.parquet
│ ├── noah_sac.ngen.short_range.metrics.csv
│ ├── noah_sac.ngen.short_range.metrics.parquet
├── noah_cfes/
│ ├── short_range/
│ │ ├── 20250820T00.parquet
│ │ ├── 20250820T06.parquet
│ │ ├── 20250820T12.parquet
│ │ ├── 20250820T18.parquet
│ │ ├── 20250821T00.parquet
│ │ └── ... (5 more files)
├── noah_sac/
│ ├── short_range/
│ │ ├── 20250820T00.parquet
│ │ ├── 20250820T06.parquet
│ │ ├── 20250820T12.parquet
│ │ ├── 20250820T18.parquet
│ │ ├── 20250821T00.parquet
│ │ └── ... (5 more files)
├── plots/
│ ├── short_range/
│ │ ├── barchart/
│ │ │ ├── barchart_CORR_01123000.png
│ │ │ ├── barchart_KGE_01123000.png
│ │ │ ├── barchart_NNSE_01123000.png
│ │ │ ├── barchart_NSE_01123000.png
│ │ │ ├── barchart_PBIAS_01123000.png
│ │ ├── metric_table/
│ │ │ ├── metric_table_lead_h1-18_01123000.png
│ │ │ ├── metric_table_lead_h1-5_01123000.png
│ │ │ ├── metric_table_lead_h10_01123000.png
│ │ │ ├── metric_table_lead_h1_01123000.png
│ │ ├── time_series/
│ │ │ ├── time_series_T0_20250820T00.png
│ │ │ ├── time_series_T0_20250821T00.png
│ │ │ ├── time_series_T0_20250822T00.png
│ │ │ ├── time_series_lead_h1.png
│ │ │ ├── time_series_lead_h12.png
│ │ │ └── ... (2 more files)
├── usgs/
│ ├── 2025-08-20_2025-08-24.parquet
├── nwm_verf_config_expanded.yaml
├── verification.log
ngenSim#
Sample output directory structure for nwm_forecast.data_source = ngenSIM, as defined in sample configuration
file config_ngensim.yaml.
Note for large-scale analyses that include hundreds of locations, the paired data is broken into multiple location
groups to avoid memory issues.The number of location groups is calculated based on pair_data.group_size and
the total number of locations. The groups are combined in the compute_metrics step to produce a single set
of metrics for the entire domain. The same applies to the generated plots in the plot_metrics step.
vpu_03S/
├── gower/
│ ├── ngen_simulation/
│ │ ├── 20121001T03-20121001T10.parquet
├── joined/
│ ├── gower.ngen.ngen_simulation.joined.group0.parquet
│ ├── gower.ngen.ngen_simulation.joined.group1.parquet
│ ├── kmeans.ngen.ngen_simulation.joined.group0.parquet
│ ├── kmeans.ngen.ngen_simulation.joined.group1.parquet
├── kmeans/
│ ├── ngen_simulation/
│ │ ├── 20121001T03-20121001T10.parquet
├── metrics/
│ ├── gower.ngen.ngen_simulation.metrics.parquet
│ ├── kmeans.ngen.ngen_simulation.metrics.parquet
├── plots/
│ ├── ngen_simulation/
│ │ ├── boxplot/
│ │ │ ├── boxplot_CORR.png
│ │ │ ├── boxplot_KGE.png
│ │ │ ├── boxplot_NNSE.png
│ │ │ ├── boxplot_NSE.png
│ │ ├── histogram/
│ │ │ ├── hist_CORR.png
│ │ │ ├── hist_KGE.png
│ │ │ ├── hist_NNSE.png
│ │ │ ├── hist_NSE.png
│ │ ├── spatial_map/
│ │ │ ├── map_CORR_gower.png
│ │ │ ├── map_CORR_kmeans.png
│ │ │ ├── map_KGE_gower.png
│ │ │ ├── map_KGE_kmeans.png
│ │ │ ├── map_NNSE_gower.png
│ │ │ └── ... (3 more files)
├── usgs/
│ ├── 2012-10-01_2012-10-03.parquet
├── nwm_verf_config_expanded.yaml
├── verification.log
GCS#
Sample output directory structure for nwm_forecast.data_source = GCS, as defined in sample configuration
file config-gcs-yaml.
Note in this case, there is an extra sub-directory “nwm30”, where the raw NWM forecast data retrieved from GCS are stored in JSON format in the sub-directory “zarr”, and the processed data for specific locations are stored in the sub-directory “time_series” in Parquet format. The sub-directories corresponding to the datasets (e.g., v3_oct and v3_sep) contain symbolic links to the processed data in “time_series”.
calib_basin_group1/
├── joined/
│ ├── v3_oct.nwm30.short_range.joined.parquet
│ ├── v3_sep.nwm30.short_range.joined.parquet
├── metrics/
│ ├── v3_oct.nwm30.short_range.metrics.parquet
│ ├── v3_sep.nwm30.short_range.metrics.parquet
├── nwm30/
│ ├── timeseries/
│ │ ├── short_range/
│ │ │ ├── 20240901T00.parquet
│ │ │ ├── 20240901T01.parquet
│ │ │ ├── 20240901T02.parquet
│ │ │ ├── 20240901T03.parquet
│ │ │ ├── 20240901T04.parquet
│ │ │ └── ... (1435 more files)
│ ├── zarr/
│ │ ├── short_range/
│ │ │ ├── nwm.20240901.nwm.t00z.short_range.channel_rt.f001.conus.nc.json
│ │ │ ├── nwm.20240901.nwm.t00z.short_range.channel_rt.f002.conus.nc.json
│ │ │ ├── nwm.20240901.nwm.t00z.short_range.channel_rt.f003.conus.nc.json
│ │ │ ├── nwm.20240901.nwm.t00z.short_range.channel_rt.f004.conus.nc.json
│ │ │ ├── nwm.20240901.nwm.t00z.short_range.channel_rt.f005.conus.nc.json
│ │ │ └── ... (25915 more files)
├── plots/
│ ├── short_range/
│ │ ├── boxplot/
│ │ │ ├── boxplot_CORR.png
│ │ │ ├── boxplot_KGE.png
│ │ │ ├── boxplot_NNSE.png
│ │ │ ├── boxplot_NSE.png
│ │ ├── histogram/
│ │ │ ├── hist_CORR_h1-5.png
│ │ │ ├── hist_CORR_h1.png
│ │ │ ├── hist_CORR_h10.png
│ │ │ ├── hist_CORR_h11-18.png
│ │ │ ├── hist_CORR_h15.png
│ │ │ └── ... (31 more files)
│ │ ├── spatial_map/
│ │ │ ├── map_CORR_h1-5_v3_oct.png
│ │ │ ├── map_CORR_h1-5_v3_sep.png
│ │ │ ├── map_CORR_h10_v3_oct.png
│ │ │ ├── map_CORR_h10_v3_sep.png
│ │ │ ├── map_CORR_h11-18_v3_oct.png
│ │ │ └── ... (67 more files)
├── usgs/
│ ├── 2024-09-01.parquet
│ ├── 2024-09-02.parquet
│ ├── 2024-09-03.parquet
│ ├── 2024-09-04.parquet
│ ├── 2024-09-05.parquet
│ └── ... (58 more files)
├── v3_oct/
│ ├── short_range/
│ │ ├── 20241001T00.parquet
│ │ ├── 20241001T01.parquet
│ │ ├── 20241001T02.parquet
│ │ ├── 20241001T03.parquet
│ │ ├── 20241001T04.parquet
│ │ └── ... (715 more files)
├── v3_sep/
│ ├── short_range/
│ │ ├── 20240901T00.parquet
│ │ ├── 20240901T01.parquet
│ │ ├── 20240901T02.parquet
│ │ ├── 20240901T03.parquet
│ │ ├── 20240901T04.parquet
│ │ └── ... (715 more files)
├── nwm_verf_config_expanded.yaml
├── verification.log