.. _output-tree:
Output Directory Structure
==========================

The outputs generated by nwm-eval-mgr are stored in ``file_paths.output_dir``, organized in a set of five 
sub-directories correpsonding to the five steps:

.. code-block:: text

    output_dir/
    ├── <dataset_name>/ # output from the "fetch_fcst_data" step: forecast/simulation data organized by dataset name
    ├── usgs/  # output from the "fetch_obs_data" step: usgs streamflow observations 
    ├── joined/  # output from the "pair_data" step: paired forecast and observation data
    ├── metrics/  # output from the "compute_metrics" step: evaluation metrics computed from the paired datasets
    └── plots/  # output from the "plot_metrics" step: evaluation plots generated from the computed metrics


The specific files in each sub-directory depend on the configurations. Below are sample output directory structures 
for each supported forecast data source, as defined in the corresponding sample configuration files.

ngenCERF
--------------------------
Sample output directory structure for ``nwm_forecast.data_source = ngenCERF``, as defined in sample configuration 
file :ref:`config-ngencerf-yaml`.

.. literalinclude:: ngencerf_output_tree.txt
   :language: text

hindcast
--------------------------
Sample output directory structure for ``nwm_forecast.data_source = hindcast``, as defined in sample configuration 
file :ref:`config-hindcast-yaml`.

.. literalinclude:: hindcast_output_tree.txt
    :language: text

ngenSim
--------------------------
Sample output directory structure for ``nwm_forecast.data_source = ngenSIM``, as defined in sample configuration 
file :ref:`config-ngensim-yaml`.

Note for large-scale analyses that include hundreds of locations, the paired data is broken into multiple location 
groups to avoid memory issues.The number of location groups is calculated based on ``pair_data.group_size`` and 
the total number of locations. The groups are combined in the ``compute_metrics`` step to produce a single set 
of metrics for the entire domain. The same applies to the generated plots in the ``plot_metrics`` step.

.. literalinclude:: ngensim_output_tree.txt
    :language: text

GCS
--------------------------
Sample output directory structure for ``nwm_forecast.data_source = GCS``, as defined in sample configuration 
file :ref:`config-gcs-yaml`.

Note in this case, there is an extra sub-directory "nwm30", where the raw NWM forecast data retrieved from GCS are 
stored in JSON format in the sub-directory "zarr", and the processed data for specific locations are stored in the 
sub-directory "time_series" in Parquet format. The sub-directories corresponding to the datasets (e.g., v3_oct and 
v3_sep) contain symbolic links to the processed data in "time_series".

.. literalinclude:: gcs_output_tree.txt
    :language: text