lw_pipeline.Output_Manager#

class lw_pipeline.Output_Manager(config, step_id, step_description='', output_registry=None)[source]#

Bases: object

Manage output saving with consistent paths, metadata, and overwrite handling.

This class provides a unified interface for saving various output types (figures, tables, MNE objects, etc.) with automatic: - Path generation (BIDS-compliant or custom) - Sidecar JSON creation with provenance metadata - Overwrite checking with multiple modes - Performance profiling (optional) - Step ID prefixing for output names

Parameters:

  • config (Config) – Configuration object containing output settings.

  • step_id (str) – Short identifier for the pipeline step (e.g., “01”, “02”).

  • step_description (str, optional) – Description of the pipeline step for metadata.

config#

Configuration object.

Type:

Config

step_id#

Step identifier used for prefixing output names.

Type:

str

step_description#

Step description for metadata.

Type:

str

__init__(config, step_id, step_description='', output_registry=None)[source]#

Initialize the Output_Manager.

Methods

__init__(config, step_id[, ...])

Initialize the Output_Manager.

get_output_path(name[, suffix, extension, ...])

Get output path without saving (useful for checking or manual saves).

save_dataframe(df, name[, format, suffix, ...])

Save a pandas DataFrame as CSV, TSV, or Excel.

save_figure(fig, name[, format, suffix, ...])

Save a matplotlib figure.

save_generic(obj, name, save_func[, suffix, ...])

Save any output type with consistent metadata and overwrite handling.

save_json(data, name[, suffix, metadata, ...])

Save data as JSON file.

save_mne_object(obj, name[, suffix, ...])

Save an MNE object (Raw, Epochs, Evoked, etc.).

save_numpy(arr, name[, format, suffix, ...])

Save a numpy array.

save_text(text, name[, suffix, metadata, ...])

Save text content to a file.

set_registry(output_registry)

Set the output registry for accessing registered output defaults.
get_output_path(name, suffix=None, extension=None, custom_dir=None, bids_path=None, **bids_params)[source]#

Get output path without saving (useful for checking or manual saves).

Parameters:

  • name (str) – Output name (will be prefixed with step_id).

  • suffix (str, optional) – BIDS suffix.

  • extension (str, optional) – File extension.

  • use_bids_structure (bool, optional) – Use BIDS directory structure. Default is False.

  • custom_dir (str or Path, optional) – Custom output directory.

  • bids_path (BIDSPath, optional) – Base BIDSPath object (required when use_bids_structure=True).

  • **bids_params (dict) – BIDS parameters for modifying bids_path.

Returns:

Output file path.

Return type:

Path

save_dataframe(df, name, format='tsv', suffix=None, metadata=None, source_file=None, **kwargs)[source]#

Save a pandas DataFrame as CSV, TSV, or Excel.

Parameters:

  • df (pandas.DataFrame) – DataFrame to save.

  • name (str) – Output name (will be prefixed with step_id).

  • format (str, optional) – Format: “csv”, “tsv”, or “xlsx”. Default is “csv”.

  • suffix (str, optional) – BIDS suffix. If None, uses registered default or “table”.

  • metadata (dict, optional) – Custom metadata for sidecar.

  • source_file (str or Path, optional) – Source file for ifnewer comparison.

  • **kwargs (dict) – Additional arguments (BIDS params + pandas to_csv/to_excel params). Merged with registered defaults (explicit params take precedence).

Returns:

Path to saved file.

Return type:

Path

save_figure(fig, name, format=None, suffix=None, metadata=None, source_file=None, **kwargs)[source]#

Save a matplotlib figure.

Parameters:

  • fig (matplotlib.figure.Figure) – Figure to save.

  • name (str) – Output name (will be prefixed with step_id).

  • format (str, optional) – Figure format (e.g., “pdf”, “png”, “svg”). If None, detected from extension in kwargs or registered defaults or defaults to “pdf”.

  • suffix (str, optional) – BIDS suffix. If None, uses registered default or “plot”.

  • metadata (dict, optional) – Custom metadata for sidecar.

  • source_file (str or Path, optional) – Source file for ifnewer comparison.

  • **kwargs (dict) – Additional arguments (BIDS params + savefig params like dpi, bbox_inches). Merged with registered defaults (explicit params take precedence).

Returns:

Path to saved figure.

Return type:

Path

save_generic(obj, name, save_func, suffix=None, extension=None, custom_dir=None, metadata=None, source_file=None, bids_path=None, **kwargs)[source]#

Save any output type with consistent metadata and overwrite handling.

Parameters:

  • obj (object) – Object to save.

  • name (str) – Output name (will be prefixed with step_id).

  • save_func (callable) – Function to call for saving (e.g., fig.savefig, df.to_csv).

  • suffix (str, optional) – BIDS suffix.

  • extension (str, optional) – File extension.

  • custom_dir (str or Path, optional) – Custom output directory.

  • metadata (dict, optional) – Custom metadata for sidecar.

  • source_file (str or Path, optional) – Source file for ifnewer comparison.

  • bids_path (BIDSPath, optional) – Base BIDSPath object (required when use_bids_structure=True).

  • **kwargs (dict) – Additional arguments passed to save_func and BIDS parameters.

Returns:

Path to saved file.

Return type:

Path

save_json(data, name, suffix='data', metadata=None, source_file=None, **kwargs)[source]#

Save data as JSON file.

Parameters:

  • data (dict or list) – Data to save as JSON.

  • name (str) – Output name (will be prefixed with step_id).

  • suffix (str, optional) – BIDS suffix. Default is “data”.

  • metadata (dict, optional) – Custom metadata for sidecar (note: data itself is JSON, sidecar is additional metadata).

  • source_file (str or Path, optional) – Source file for ifnewer comparison.

  • **kwargs (dict) – Additional arguments (BIDS params + json.dump params like indent).

Returns:

Path to saved file.

Return type:

Path

save_mne_object(obj, name, suffix=None, metadata=None, source_file=None, **kwargs)[source]#

Save an MNE object (Raw, Epochs, Evoked, etc.).

Parameters:

  • obj (mne.BaseRaw, mne.BaseEpochs, mne.Evoked, etc.) – MNE object to save.

  • name (str) – Output name (will be prefixed with step_id).

  • suffix (str, optional) – BIDS suffix (e.g., “eeg”, “epochs”). If None, auto-detected.

  • metadata (dict, optional) – Custom metadata for sidecar.

  • source_file (str or Path, optional) – Source file for ifnewer comparison.

  • **kwargs (dict) – Additional arguments (BIDS params + MNE save params like overwrite).

Returns:

Path to saved file.

Return type:

Path

save_numpy(arr, name, format='npy', suffix='array', metadata=None, source_file=None, **kwargs)[source]#

Save a numpy array.

Parameters:

  • arr (numpy.ndarray) – Array to save.

  • name (str) – Output name (will be prefixed with step_id).

  • format (str, optional) – Format: “npy” (binary) or “txt” (text). Default is “npy”.

  • suffix (str, optional) – BIDS suffix. Default is “array”.

  • metadata (dict, optional) – Custom metadata for sidecar.

  • source_file (str or Path, optional) – Source file for ifnewer comparison.

  • **kwargs (dict) – Additional arguments (BIDS params + numpy save params).

Returns:

Path to saved file.

Return type:

Path

save_text(text, name, suffix='log', metadata=None, source_file=None, **kwargs)[source]#

Save text content to a file.

Parameters:

  • text (str) – Text content to save.

  • name (str) – Output name (will be prefixed with step_id).

  • suffix (str, optional) – BIDS suffix. Default is “log”.

  • metadata (dict, optional) – Custom metadata for sidecar.

  • source_file (str or Path, optional) – Source file for ifnewer comparison.

  • **kwargs (dict) – Additional arguments (BIDS params + extension).

Returns:

Path to saved file.

Return type:

Path

set_registry(output_registry)[source]#

Set the output registry for accessing registered output defaults.

This is called by Pipeline_Step after both Output_Manager and Output_Registry are created to avoid circular dependencies.

Parameters:

output_registry (Output_Registry) – The registry containing registered output metadata.