pyLOM.NN#

Subpackages#

Module contents#

pyLOM.NN.plotSnapshot(mesh, dset, vars=[], idim=0, instant=0, **kwargs)[source]#

Plot using pyVista

pyLOM.NN.plotModalErrorBars(error: ndarray)[source]#

Do a barplot of a 1D array of errors, where each element is the error associated in the prediction of a mode.

pyLOM.NN.plotTimeSeries(time: ndarray, truth: ndarray, pred: ndarray, std: ndarray = None)[source]#

Function to plot the comparison between the truth and predicted N temporal series.

class pyLOM.NN.Pipeline(train_dataset, valid_dataset=None, test_dataset=None, model=None, training_params: Dict = None, optimizer: OptunaOptimizer = None, model_class=None, evaluators: List = [])[source]#

Bases: object

Pipeline class to train and evaluate models. To optimize a model, provide an optimizer and model class. To train a model with fixed parameters, provide a model and training parameters.

Parameters:

  • train_dataset – The training dataset.

  • valid_dataset (optional) – The validation dataset. Default is None.

  • test_dataset (optional) – The test dataset. Default is None.

  • model (Model, optional) – The model to train. Default is None. If optimizer and model_class are provided, this is not used.

  • training_params (Dict, optional) – The parameters for training the model. Default is None. If optimizer and model_class are provided, this is not used.

  • optimizer (OptunaOptimizer, optional) – The optimizer to use for optimization. Default is None.

  • model_class (Model, optional) – The model class to use for optimization. Default is None.

  • evaluators (List, optional) – The evaluators to use for evaluating the model. Default is [].

Raises:

AssertionError – If neither model and training_params nor optimizer and model_class are provided.

property model#

Get the trained model.

run() → Any[source]#

Run the pipeline, this will train the model and return the output of the model’s fit method. If optuna is used, the model will be trained with the optimized parameters.

Returns:

The output of the model’s fit method.

Return type:

model_output (Any)

class pyLOM.NN.Dataset(variables_out: Tuple, mesh_shape: Tuple = (1,), variables_in: ndarray = None, parameters: List[List[float]] = None, combine_parameters_with_cartesian_prod: bool = False, inputs_scaler=None, outputs_scaler=None, snapshots_by_column=True)[source]#

Bases: Dataset

Dataset class to be used with PyTorch. It can be used with both mesh and point data. It is useful convert the pyLOM.Dataset to a PyTorch dataset and train neural networks with results from CFD simulations.

Example

>>> original_dataset = pyLOM.Dataset.load(path)
>>> input_scaler = pyLOM.NN.MinMaxScaler()
>>> output_scaler = pyLOM.NN.MinMaxScaler()
>>> dataset = pyLOM.NN.Dataset(
...     variables_out=(original_dataset["CP"],),
...     variables_in=original_dataset.xyz,
...     parameters=[[*zip(original_dataset.get_variable('AoA'), original_dataset.get_variable('Mach'))]], # to have each Mach and AoA pair just once. To have all possibnle combinations, use [original_dataset.get_variable('AoA'), original_dataset.get_variable("Mach")]
...     inputs_scaler=input_scaler,
...     outputs_scaler=output_scaler,
... )
Parameters:

  • variables_out (Tuple) – Tuple of variables to be used as output. Each variable should be a 2d numpy array or torch tensor. If only one variable wants to be provided, it should be passed as a tuple with one element. E.g. (variable,).

  • mesh_shape (Tuple) – Shape of the mesh. If not mesh is used and the data is considered as points, leave this as default. Default is (1,).

  • variables_in (np.ndarray) – Input variables. Default is None.

  • parameters (List[List[float]]) – List of parameters to be used as input. All possible combinations of these parameters, i.e. its cartesian product, will appear along with variables_in. If there is only one inner list and its elements are tuples, they will be treated as a single element for the cartesiand produt, which is useful when the combination of the parameters was predefined. Default is None.

  • inputs_scaler (MinMaxScaler) – Scaler to scale the input variables. If the scaler is not fitted, it will be fitted. Default is None.

  • outputs_scaler (MinMaxScaler) – Scaler to scale the output variables. If the scaler is not fitted, it will be fitted. Default is None.

  • snapshots_by_column (bool) – If the snapshots from variables_out are stored by column. The snapshots on pyLOM.Dataset`s have this format. Default is ``True`.

property shape#
property num_parameters#
__getitem__(idx: int | slice)[source]#

Return the input data and the output data for a given index as a tuple. If there is no input data, only the output data will be returned. If parameters are used, the parameters will be concatenated with the input data at the end.

Parameters:

idx (int, slice) – Index of the data to be returned.

__setitem__(idx: int | slice, value: Tuple)[source]#

Set the input data and the output data for a given index. If there is no input data, only the output data will be set. If parameters are used, the parameters will be concatenated with the input data at the end.

Parameters:

  • idx (int, slice) – Index of the data to be set.

  • value (Tuple) – Tuple with the input data and the output data. If there is no input data, the tuple should have only one element.

__add__(other)[source]#

Concatenate two datasets. The datasets must have the same number of input coordinates and parameters.

Parameters:

other (Dataset) – Dataset to be concatenated with.

concatenate(other)[source]#

Alias for the __add__ method.

Parameters:

other (Dataset) – Dataset to be concatenated with.

crop(*args)[source]#

Crop the dataset to a desired shape. The cropping currently works for 2D and 3D meshes.

Parameters:

args (Tuple) – Desired shape of the mesh. If the mesh is 2D, the shape should be a tuple with two elements. If the mesh is 3D, the shape should be a tuple with three elements.

pad(*args)[source]#

Pad the dataset to a desired shape. The padding currently works for 2D and 3D meshes.

Parameters:

args (Tuple) – Desired shape of the mesh. If the mesh is 2D, the shape should be a tuple with two elements. If the mesh is 3D, the shape should be a tuple with three elements.

get_splits(sizes: Sequence[int], shuffle: bool = True, return_views: bool = True, generator: Generator | None = default_generator)[source]#

Randomly split the dataset into non-overlapping new datasets of given lengths.

If a list of fractions that sum up to 1 is given, the lengths will be computed automatically as floor(frac * len(self)) for each fraction provided.

After computing the lengths, if there are any remainders, 1 count will be distributed in round-robin fashion to the lengths until there are no remainders left.

Optionally fix the generator for reproducible results.

Parameters:

  • sizes (sequence) – lengths or fractions of splits to be produced.

  • shuffle (bool) – whether to shuffle the data before splitting. Default: True.

  • return_views (bool) – If True, the splits will be returned as views of the original dataset in form of torch.utils.data.Subset.

  • False (If) – True.

  • Default (the splits will be returned as new Dataset instances.) – True.

  • Warning – If return_views is True, the original dataset must be kept alive, otherwise the views will point to invalid memory.

  • parameters (Be careful when using return_views=False with variables_in and)

  • higher. (the memory usage will be)

  • generator (Generator) – Generator used for the random permutation. Default: default_generator.

Returns:

List with the splits of the dataset.

Return type:

List[Dataset | Subset]

get_splits_by_parameters(sizes: Sequence[int], shuffle: bool = True, return_views: bool = True, generator: Generator | None = default_generator)[source]#

Split the dataset into non-overlapping new datasets with diferent sets of parameters of given length.

If a list of fractions that sum up to 1 is given, the lengths will be computed automatically as floor(frac * len(self.parameters)) for each fraction provided.

After computing the lengths, if there are any remainders, 1 count will be distributed in round-robin fashion to the lengths until there are no remainders left.

Optionally fix the generator for reproducible results.

Parameters:

  • sizes (sequence) – lengths or fractions of splits to be produced

  • shuffle (bool) – whether to shuffle the data before splitting. Default: True

  • return_views (bool) – If True, the splits will be returned as views of the original dataset in form of torch.utils.data.Subset.

  • False (If) – True.

  • Default (the splits will be returned as new Dataset instances.) – True.

  • Warning – If return_views is True, the original dataset must be kept alive, otherwise the views will point to invalid memory.

  • generator (Generator) – Generator used for the random permutation. Default: default_generator.

Returns:

List with the splits of the dataset.

Return type:

List[Dataset | Subset]

classmethod load(file_path, field_names: List[str], variables_names: List[str] = ['all'], add_mesh_coordinates: bool = True, **kwargs)[source]#

Create a Dataset from a saved pyLOM.Dataset in one of its formats.

Parameters:

  • file_path (str) – Path to the HDF5 file.

  • variables_out_names (List[str]) – Names of the fields to be used as output. E.g. ["CP"].

  • add_variables (bool) – Whether to add the variables as input variables. Default is True.

  • variables_names (List[str]) – Names of the variables from pyLOM.Dataset.varnames to be used as input. If ["all"] is passed, all variables will be used. Default is ["all"].

  • kwargs – Additional arguments to be passed to the pyLOM.NN.Dataset constructor.

Returns:

Dataset created from the saved pyLOM.Dataset.

Return type:

Dataset

Example

>>> dataset = pyLOM.NN.Dataset.load(
...     file_path,
...     field_names=["CP"],
...     add_variables=True,
...     add_mesh_coordinates=True,
...     inputs_scaler=inputs_scaler,
...     outputs_scaler=outputs_scaler,
... )
map(function: Callable, fn_kwargs: dict = {}, batched: bool = False, batch_size: int = 1000)[source]#

Apply a function to the dataset.

Parameters:

  • function (Callable) –

    Function to be applied to the dataset with one of the following signatures:

    • function (inputs: torch.Tensor, outputs: torch.Tensor, **kwargs) -> Tuple[torch.Tensor, torch.Tensor] if variables_in exists. Here inputs is the input data and outputs is the output data that __getitem__ returns, so inputs will include the parameters if they exist.

    • function (outputs: torch.Tensor, **kwargs) -> torch.Tensor if variables_in does not exist.

    If batched is False, the tensors will have only one element in the first dimension.

  • fn_kwargs (Dict) – Additional keyword arguments to be passed to the function.

  • batched (bool) – If True, the function will be applied to the dataset in batches. Default is False.

  • batch_size (int) – Size of the batch. Default is 1000.

Returns:

A reference to the dataset with th e function applied.

Return type:

Dataset

filter(function: Callable, fn_kwargs: dict = {}, batched: bool = False, batch_size: int = 1000, return_views: bool = True)[source]#

Filter the dataset using a function.

Parameters:

  • function (Callable) –

    Function to be applied to the dataset with one of the following signatures:

    • function (inputs: torch.Tensor, outputs: torch.Tensor, **kwargs) -> bool if variables_in exists. Here inputs is the input data and outputs is the output data that __getitem__ returns, so inputs will include the parameters if they exist.

    • function (outputs: torch.Tensor, **kwargs) -> bool if variables_in does not exist.

    If batched is True, the function should return a list of booleans.

  • fn_kwargs (Dict) – Additional keyword arguments to be passed to the function.

  • batched (bool) – If True, the function will be applied to the dataset in batches. Default is False.

  • batch_size (int) – Size of the batch. Default is 1000.

  • return_views (bool) – If True, the filtered dataset will be returned as a view of the original dataset in form of torch.utils.data.Subset. Default is True.

  • Warning – If return_views is True, the original dataset must be kept alive, otherwise the views will point to invalid memory.

  • parameters (Be careful when using return_views=False with variables_in and)

  • higher. (the memory usage will be)

Returns:

Subset of the dataset that passed the filter.

Return type:

Subset | Dataset

remove_column(column_idx: int, from_variables_out: bool)[source]#

Remove a column from the dataset.

Parameters:

  • column_idx (int) – Index of the column to be removed.

  • from_variables_out (bool) – If True, the column will be removed from the output variables. If False, the column will be removed from the input variables, if column_idx is greater than the number of columns on variables_in, the column will be removed from parameters.

class pyLOM.NN.MinMaxScaler(feature_range=(0, 1), column=False)[source]#

Bases: object

Min-max scaling to scale variables to a desired range. The formula is given by:

\[X_{scaled} = \frac{X - X_{min}}{X_{max} - X_{min}} * (feature\_range_{max} - feature\_range_{min}) + feature\_range_{min}\]
Parameters:

  • feature_range (Tuple) – Desired range of transformed data. Default is (0, 1).

  • column (bool, optional) – Scale over the column space or the row space (default False)

property is_fitted#
fit(variables: List[ndarray | tensor] | ndarray | tensor)[source]#

Compute the min and max values of each variable. :param variables: List of variables to be fitted. The variables should be numpy arrays or torch tensors. :type variables: List :param A numpy array or torch tensor can be passed directly and each column will be considered as a variable to be scaled.:

transform(variables: List[ndarray | tensor] | ndarray | tensor) → List[ndarray | tensor][source]#

Scale variables to the range defined on feature_range using min-max scaling. :param variables: List of variables to be scaled. The variables should be numpy arrays or torch tensors. :type variables: List :param A numpy array or torch tensor can be passed directly and each column will be considered as a variable to be scaled.:

Returns:

List of scaled variables.

Return type:

scaled_variables

fit_transform(variables: List[ndarray | tensor]) → List[ndarray | tensor][source]#

Fit and transform the variables using min-max scaling. :param variables: List of variables to be fitted and scaled. The variables should be numpy arrays or torch tensors. :type variables: List :param A numpy array or torch tensor can be passed directly and each column will be considered as a variable to be scaled.:

Returns:

List of scaled variables.

Return type:

scaled_variables

inverse_transform(variables: List[ndarray | tensor]) → List[ndarray | tensor][source]#

Inverse scale variables that have been scaled using min-max scaling. :param variables: List of variables to be inverse scaled. The variables should be numpy arrays or torch tensors. :type variables: List :param A numpy array or torch tensor can be passed directly and each column will be considered as a variable to be scaled.:

Returns:

List of inverse scaled variables.

Return type:

inverse_scaled_variables

save(filepath: str) → None[source]#

Save the fitted scaler parameters to a JSON file.

Parameters:

filepath (str) – Path where the scaler parameters will be saved

static load(filepath: str) → MinMaxScaler[source]#

Load a saved MinMaxScaler from a JSON file.

Parameters:

filepath (str) – Path to the saved scaler parameters

Returns:

A new MinMaxScaler instance with the loaded parameters

Return type:

MinMaxScaler

pyLOM.NN.select_device(device: str = DEVICE)[source]#

Select the device to be used for the training.

Parameters:

device (str) – Device to be used. Default is cuda if available, otherwise cpu.

class pyLOM.NN.betaLinearScheduler(start_value, end_value, start_epoch, warmup)[source]#

Bases: object

Linear scheduler for beta parameter in the loss function of the Autoencoders.

Parameters:

  • start_value (float) – initial value of beta

  • end_value (float) – final value of beta

  • warmup (int) – number of epochs to reach final value

getBeta(epoch)[source]#

Get the value of beta for a given epoch.

Parameters:

epoch (int) – current epoch

pyLOM.NN.create_results_folder(RESUDIR: str, verbose: bool = True)[source]#

Create a folder to store the results of the neural network training.

Parameters:

  • RESUDIR (str) – Path to the folder to be created.

  • verbose (bool) – If True, print messages to the console. Default is True.

pyLOM.NN.set_seed(seed: int = 42, deterministic: bool = True)[source]#
class pyLOM.NN.OptunaOptimizer(optimization_params: Dict, n_trials: int = 100, direction: str = 'minimize', pruner: BasePruner = None, save_dir: str = None)[source]#

Bases: object

Parameters:

  • optimization_params (Dict) – A dictionary containing the parameters to optimize.

  • n_trials (int) – The number of trials to run. Default is 100.

  • direction (str) – The direction to optimize. Can be ‘minimize’ or ‘maximize’. Default is 'minimize'.

  • pruner (optuna.pruners.BasePruner) – The pruner to use. Default is None.

  • save_dir (str) – The directory to save the best parameters. Default is None.

property optimization_params: Dict#

Get the optimization parameters.

optimize(objective_function: Callable[[Trial], float]) → Dict[source]#

Optimize a model given an objective function.

Parameters:

objective_function (Callable) – The objective function to optimize. The function should take a optuna.Trial object as input and return a float.

Returns:

The best parameters obtained from the optimization.

Return type:

Dict

class pyLOM.NN.RegressionEvaluator(tolerance: float = 1e-4)[source]#

Bases: object

Evaluator class for regression tasks. Includes methods to calculate the mean squared error (MSE), mean absolute error (MAE), mean relative error (MRE), quantiles of the absolute errors, L2 error, and R-squared.

Parameters:

tolerance (float) – Tolerance level to consider values close to zero for MRE calculation (default: 1e-4).

property tolerance: float#
mean_squared_error(y_true: ndarray, y_pred: ndarray) → float[source]#

Compute the mean squared error (MSE) between the true values and the predicted values.

Parameters:
Returns:

The mean squared error.

Return type:

float

mean_absolute_error(y_true, y_pred)[source]#

Compute the mean absolute error (MAE) between the true values and the predicted values.

Parameters:
Returns:

The mean absolute error.

Return type:

float

mean_relative_error(y_pred: ndarray, y_true: ndarray) → float[source]#

Compute the mean relative error (MRE) between the true values and the predicted values, adding a tolerance level to consider values close to zero.

Parameters:

  • y_true (numpy.ndarray) – The true values.

  • y_pred (numpy.ndarray) – The predicted values.

  • tolerance (float) – Tolerance level to consider values close to zero. Default is 1e-4.

Returns:

The mean relative error excluding cases where y_true is close to zero.

Return type:

float

ae_q(y_pred: ndarray, y_true: ndarray, quantile: int) → float[source]#

Calculate the quantile of the absolute errors between the true and predicted values.

Parameters:

  • y_true (numpy.ndarray) – The true values.

  • y_pred (numpy.ndarray) – The predicted values.

  • quantile (int) – The quantile to calculate. Must be between 0 and 100.

Returns:

The quantile of the absolute errors.

Return type:

float

l2_error(y_pred: ndarray, y_true: ndarray) → float[source]#

Calculate the L2 error between the true and predicted values.

Parameters:
Returns:

The L2 error.

Return type:

float

R2(y_true: ndarray, y_pred: ndarray) → float[source]#

Calculate the R-squared (coefficient of determination) for a set of true and predicted values.

Parameters:
Returns:

The R-squared value.

Return type:

float

print_metrics()[source]#

Print the calculated regression metrics.

__call__(y_true: ndarray, y_pred: ndarray) → dict[source]#

Calculate multiple regression metrics between the true and predicted values.

Parameters:

  • y_true (numpy.ndarray) – An array-like object containing the true values.

  • y_pred (numpy.ndarray) – An array-like object containing the predicted values.

Returns:

A dictionary containing the calculated regression metrics.

Return type:

dict

class pyLOM.NN.EarlyStopper(patience: int = 1, min_delta: float = 0)[source]#

Bases: object

Early stopper callback.

Parameters:

  • patience (int) – Number of epochs to wait before stopping the training. Default: 1.

  • min_delta (float) – Minimum change in the monitored quantity to qualify as an improvement. Default: 0.

early_stop(validation_loss: float, prev_train: float, train: float) → bool[source]#

Early stopper routine. The training will stop if the validation loss does not improve for a number of epochs.

Parameters:

  • validation_loss (float) – Validation loss.

  • prev_train (float) – Previous training loss.

  • train (float) – Current training loss.

Returns:

True if early stopping is triggered, False otherwise.

Return type:

bool

class pyLOM.NN.MLP(input_size: int, output_size: int, n_layers: int, hidden_size: int, p_dropouts: float = 0.0, activation: Module = torch.nn.functional.relu, device: device = DEVICE, seed: int = None, **kwargs: Dict)[source]#

Bases: Module

Multi-layer perceptron model for regression tasks. The model is based on the PyTorch library torch.nn (detailed documentation can be found at https://pytorch.org/docs/stable/nn.html).

Parameters:

  • input_size (int) – Number of input features.

  • output_size (int) – Number of output features.

  • n_layers (int) – Number of hidden layers.

  • hidden_size (int) – Number of neurons in each hidden layer.

  • p_dropouts (float, optional) – Dropout probability for the hidden layers (default: 0.0).

  • checkpoint_file (str, optional) – Path to a checkpoint file to load the model from (default: None).

  • activation (torch.nn.Module, optional) – Activation function to use (default: torch.nn.functional.relu).

  • device (torch.device, optional) – Device to use (default: torch.device("cpu")).

  • seed (int, optional) – Seed for reproducibility (default: None).

  • kwargs – Additional keyword arguments.

forward(x)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

fit(train_dataset: Dataset, eval_dataset: Dataset = None, epochs: int = 100, lr: float = 0.001, lr_gamma: float = 1, lr_scheduler_step: int = 1, loss_fn: Module = torch.nn.MSELoss(), optimizer_class: Optimizer = torch.optim.Adam, scheduler_class: LRScheduler = torch.optim.lr_scheduler.StepLR, print_rate_batch: int = 0, print_rate_epoch: int = 1, **kwargs) → Dict[str, List[float]][source]#

Fit the model to the training data. If eval_set is provided, the model will be evaluated on this set after each epoch.

Parameters:

  • train_dataset (torch.utils.data.Dataset) – Training dataset to fit the model.

  • eval_dataset (torch.utils.data.Dataset) – Evaluation dataset to evaluate the model after each epoch (default: None).

  • epochs (int, optional) – Number of epochs to train the model (default: 100).

  • lr (float, optional) – Learning rate for the optimizer (default: 0.001).

  • lr_gamma (float, optional) – Multiplicative factor of learning rate decay (default: 1).

  • lr_scheduler_step (int, optional) – Number of epochs to decay the learning rate (default: 1).

  • loss_fn (torch.nn.Module, optional) – Loss function to optimize (default: torch.nn.MSELoss()).

  • optimizer_class (torch.optim.Optimizer, optional) – Optimizer class to use (default: torch.optim.Adam).

  • scheduler_class (torch.optim.lr_scheduler._LRScheduler, optional) – Learning rate scheduler class to use. If None, no scheduler will be used (default: torch.optim.lr_scheduler.StepLR).

  • print_rate_batch (int, optional) – Print loss every print_rate_batch batches (default: 1). If set to 0, no print will be done.

  • print_rate_epoch (int, optional) – Print loss every print_rate_epoch epochs (default: 1). If set to 0, no print will be done.

  • kwargs (dict, optional) –

    Additional keyword arguments to pass to the DataLoader. Can be used to set the parameters of the DataLoader (see PyTorch documentation at https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader):

    • batch_size (int, optional): Batch size (default: 32).

    • shuffle (bool, optional): Shuffle the data (default: True).

    • num_workers (int, optional): Number of workers to use (default: 0).

    • pin_memory (bool, optional): Pin memory (default: True).

Returns:

Dictionary containing the training and evaluation losses.

Return type:

Dict[str, List[float]]

predict(X: Dataset, return_targets: bool = False, **kwargs)[source]#

Predict the target values for the input data. The dataset is loaded to a DataLoader with the provided keyword arguments. The model is set to evaluation mode and the predictions are made using the input data. To make a prediction from a torch tensor, use the __call__ method directly.

Parameters:

  • X (torch.utils.data.Dataset) – The dataset whose target values are to be predicted using the input data.

  • rescale_output (bool) – Whether to rescale the output with the scaler of the dataset (default: True).

  • kwargs (dict, optional) –

    Additional keyword arguments to pass to the DataLoader. Can be used to set the parameters of the DataLoader (see PyTorch documentation at https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader):

    • batch_size (int, optional): Batch size (default: 256).

    • shuffle (bool, optional): Shuffle the data (default: False).

    • num_workers (int, optional): Number of workers to use (default: 0).

    • pin_memory (bool, optional): Pin memory (default: True).

Returns:

The predictions and the true target values.

Return type:

Tuple [np.ndarray, np.ndarray]

save(path: str)[source]#

Save the model to a checkpoint file.

Parameters:

  • path (str) – Path to save the model. It can be either a path to a directory or a file name.

  • directory (If it is a)

  • trained. (the model will be saved with a filename that includes the number of epochs)

classmethod load(path: str, device: device = DEVICE)[source]#

Load the model from a checkpoint file. Does not require the model to be instantiated.

Parameters:

  • path (str) – Path to the file to load the model from.

  • device (torch.device, optional) – Device to use (default: torch.device("cpu")).

Returns:

The loaded model.

Return type:

Model (MLP)

classmethod create_optimized_model(train_dataset: Dataset, eval_dataset: Dataset, optuna_optimizer: OptunaOptimizer, **kwargs) → Tuple[Module, Dict][source]#

Create an optimized model using Optuna. The model is trained on the training dataset and evaluated on the validation dataset.

Parameters:
Returns:

The optimized model and the optimization parameters.

Return type:

Tuple [MLP, Dict]

Example

>>> from pyLOM.NN import MLP, OptunaOptimizer
>>> # Split the dataset
>>> train_dataset, eval_dataset = dataset.get_splits([0.8, 0.2])
>>>
>>> # Define the optimization parameters
>>> optimization_params = {
>>>     "lr": (0.00001, 0.01), # optimizable parameter
>>>     "epochs": 50, # fixed parameter
>>>     "n_layers": (1, 4),
>>>     "batch_size": (128, 512),
>>>     "hidden_size": (200, 400),
>>>     "p_dropouts": (0.1, 0.5),
>>>     "num_workers": 0,
>>>     'print_rate_epoch': 5
>>> }
>>>
>>> # Define the optimizer
>>> optimizer = OptunaOptimizer(
>>>     optimization_params=optimization_params,
>>>     n_trials=5,
>>>     direction="minimize",
>>>     pruner=optuna.pruners.MedianPruner(n_startup_trials=5, n_warmup_steps=5, interval_steps=1),
>>>     save_dir=None,
>>> )
>>>
>>> # Create the optimized model
>>> model, optimization_params = MLP.create_optimized_model(train_dataset, eval_dataset, optimizer)
>>>
>>> # Fit the model
>>> model.fit(train_dataset, eval_dataset, **optimization_params)
class pyLOM.NN.KAN(input_size: int, output_size: int, n_layers: int, hidden_size: int, layer_type, model_name: str = 'KAN', p_dropouts: float = 0.0, device: device = DEVICE, verbose: bool = True, **layer_kwargs)[source]#

Bases: Module

KAN (Kolmogorov-Arnold Network) model for regression tasks. This model is based on https://arxiv.org/abs/2404.19756, inspired by the Kolmogorov-Arnold representation theorem.

Parameters:

  • input_size (int) – The number of input features.

  • output_size (int) – The number of output features.

  • n_layers (int) – The number of hidden layers.

  • hidden_size (int) – The number of neurons in the hidden layers.

  • layer_type (nn.Module) – The type of layer to use in the model. It can be one of the following: JacobiLayer, ChebyshevLayer.

  • model_name (str) – The name of the model.

  • p_dropouts (float, Optional) – The dropout probability (default: 0.0).

  • device (torch.device, Optional) – The device where the model is loaded (default: gpu if available).

  • verbose (bool, Optional) – Whether to print the model information (default: True).

  • **layer_kwargs – Additional keyword arguments to pass to the layer type. For example, the order of the Taylor series or the degree of the Chebyshev polynomial.

forward(x)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

fit(train_dataset: Dataset, eval_dataset: Dataset, batch_size: int = 32, epochs: int = 100, lr: float = 0.001, optimizer_class=optim.Adam, scheduler_type='StepLR', opti_kwargs={}, lr_kwargs={}, print_eval_rate: int = 2, loss_fn=nn.MSELoss(), save_logs_path=None, verbose: bool = True, max_norm_grad=float('inf'), **kwargs)[source]#

Train the model using the provided training dataset. The model is trained using the Adam optimizer with the provided learning rate and learning rate decay factor.

Parameters:

  • train_dataset (torch.utils.data.Dataset) – The training dataset.

  • eval_dataset (torch.utils.data.Dataset) – The evaluation dataset.

  • batch_size (int) – The batch size. (default: 32).

  • epochs (int) – The number of epochs to train the model. (default: 100).

  • lr (float) – The learning rate for the Adam optimizer. (default: 0.001).

  • optimizer_class (torch.optim, Optional) – The optimizer to use. Available all optimizers from PyTorch except AdaDelta. (default: optim.Adam).

  • scheduler_type (str, opcional) –

    Scheduler type to adjust the learning rate dynamically. (default: "StepLR"). Available options:

    • ”StepLR”: Reduce the learning rate by a factor every step_size batches.

    • ”ReduceLROnPlateau”: Reduces the learning rate when a metric has stopped improving.

    • ”OneCycleLR”: Adjust the learning rate in a single cycle of the training.

  • lr_kwargs (dict, opcional) –

    Dictionary containing the specific parameters for the learning rate scheduler. (default: {}). Some examples are:

    • StepLR: {“step_size”: int, “gamma”: float}.

    • ReduceLROnPlateau: {“mode”: str, “factor”: float, “patience”: int}.

    • OneCycleLR: {“anneal_strategy”: str, “div_factor”: float}.

  • opti_kwargs (dict, Optional) – Additional keyword arguments to pass to the optimizer (default: {}).

  • print_eval_rate (int, Optional) – The model will be evaluated every print_eval_rate epochs and the losses will be printed. If set to 0, nothing will be printed (default: 2).

  • loss_fn (torch.nn.Module, Optional) – The loss function (default: nn.MSELoss()).

  • save_logs_path (str, Optional) – Path to save the training and evaluation losses (default: None).

  • verbose (bool, Optional) – Whether to print the training information (default: True).

  • max_norm_grad (float, Optional) – The maximum norm of the gradients (default: float('inf')).

  • kwargs (dict, Optional) –

    Additional keyword arguments to pass to the DataLoader. Can be used to set the parameters of the DataLoader (see PyTorch documentation at https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader):

    • batch_size (int, Optional): Batch size (default: 32).

    • shuffle (bool, Optional): Shuffle the data (default: True).

    • num_workers (int, Optional): Number of workers to use (default: 0).

    • pin_memory (bool, Optional): Pin memory (default: True).

predict(X: Dataset, return_targets: bool = False, **kwargs)[source]#

Predict the target values for the input data. The dataset is loaded into a DataLoader with the provided keyword arguments. The model is set to evaluation mode and the predictions are made using the input data. The output can be rescaled using the dataset scaler.

Parameters:

  • X (torch.utils.data.Dataset) – The dataset whose target values are to be predicted using the input data.

  • rescale_output (bool) – Whether to rescale the output with the scaler of the dataset (default: True).

  • kwargs (dict, Optional) –

    Additional keyword arguments to pass to the DataLoader. Can be used to set the parameters of the DataLoader (see PyTorch documentation at https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader):

    • batch_size (int, Optional): Batch size (default: 32).

    • shuffle (bool, Optional): Shuffle the data (default: True).

    • num_workers (int, Optional): Number of workers to use (default: 0).

    • pin_memory (bool, Optional): Pin memory (default: True).

Returns:

The predictions and the true target values.

Return type:

Tuple[np.ndarray, np.ndarray]

save(path: str, save_only_model: bool = False)[source]#

Save the model to a checkpoint file.

Parameters:

  • path (str) – Path to save the model. It can be either a path to a directory or a file name.

  • directory (If it is a)

  • trained. (the model will be saved with a filename that includes the number of epochs)

  • save_only_model (bool, Optional) – Whether to only save the model, or also the optimizer and scheduler. Note that when this is true, you won’t be able to resume training from checkpoint.(default: False).

classmethod load(path: str, device: device = torch.device('cpu'))[source]#

Loads a model from a checkpoint file.

Parameters:

  • path (str) – Path to the checkpoint file.

  • device (torch.device) – Device where the model is loaded (default: cpu).

Returns:

The loaded KAN model with the trained weights.

Return type:

model (KAN)

classmethod create_optimized_model(train_dataset: Dataset, eval_dataset: Dataset, optuna_optimizer: OptunaOptimizer, **kwargs) → Tuple[Module, Dict][source]#

Create an optimized KAN model using Optuna. The model is trained on the training dataset and the metric to optimize is computed with the evaluation dataset. If the parameters from the optimizer are a tuple, the function will optimize the parameter. If the parameter is a single value, it will be fixed during optimization.

Parameters:
Returns:

The optimized model and the optimization parameters.

Return type:

Tuple [KAN, Dict]

Example

>>> from pyLOM.NN import KAN, OptunaOptimizer
>>> # Split the dataset
>>> train_dataset, eval_dataset = dataset.get_splits([0.8, 0.2])
>>>
>>> # Define the optimization parameters
>>> optimization_params = {
>>>     "lr": (0.00001, 0.1),
>>>     "batch_size": (10, 64),
>>>     "hidden_size": (10, 40), # optimizable parameter
>>>     "n_layers": (1, 4),
>>>     "print_eval_rate": 2,
>>>     "epochs": 10, # non-optimizable parameter
>>>    "lr_kwargs":{
>>>        "gamma": (0.95, 0.99),
>>>        "step_size": 7000
>>>    },
>>>     "model_name": "kan_test_optuna",
>>>     'device': device,
>>>     "layer_type": (pyLOM.NN.ChebyshevLayer, pyLOM.NN.JacobiLayer),
>>>     "layer_kwargs": {
>>>         "degree": (3, 10),
>>>     },
>>> }
>>>
>>> # Define the optimizer
>>> optimizer = OptunaOptimizer(
>>>     optimization_params=optimization_params,
>>>     n_trials=5,
>>>     direction="minimize",
>>>     pruner=optuna.pruners.MedianPruner(n_startup_trials=5, n_warmup_steps=5, interval_steps=1),
>>>     save_dir=None,
>>> )
>>>
>>> # Create the optimized model
>>> model, optimization_params = KAN.create_optimized_model(train_dataset, eval_dataset, optimizer)
>>>
>>> # Fit the model
>>> model.fit(train_dataset, eval_dataset, **optimization_params)
class pyLOM.NN.ChebyshevLayer(input_dim, output_dim, degree, **kwargs)[source]#

Bases: Module

Chebyshev layer for KAN model.

Parameters:

  • input_dim (int) – The number of input features.

  • output_dim (int) – The number of output features.

  • degree (int) – The degree of the Chebyshev polynomial.

forward(x)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class pyLOM.NN.JacobiLayer(input_dim, output_dim, degree, a=1.0, b=1.0)[source]#

Bases: Module

Jacobi layer for KAN model.

Parameters:

  • input_dim (int) – The number of input features.

  • output_dim (int) – The number of output features.

  • degree (int) – The degree of the Jacobi polynomial.

  • a (float, Optional) – The first parameter of the Jacobi polynomial (default: 1.0).

  • b (float, Optional) – The second parameter of the Jacobi polynomial (default: 1.0).

forward(x)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class pyLOM.NN.Autoencoder(latent_dim: int, in_shape: tuple, input_channels: int, encoder: Module, decoder: Module, device: device = DEVICE)[source]#

Bases: Module

Autoencoder class for neural network module. The model is based on the PyTorch.

Parameters:

  • latent_dim (int) – Dimension of the latent space.

  • in_shape (tuple) – Shape of the input data.

  • input_channels (int) – Number of input channels.

  • encoder (torch.nn.Module) – Encoder model.

  • decoder (torch.nn.Module) – Decoder model.

  • device (str) – Device to run the model. Default is ‘cuda’ if available, otherwise ‘cpu’.

forward(x)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

fit(train_dataset: Dataset, eval_dataset: Dataset = None, epochs: int = 100, callback=None, lr: float = 1e-3, BASEDIR: str = './', reduction: str = 'mean', lr_decay: float = 0.999, batch_size: int = 32, shuffle: bool = True, num_workers: int = 0, pin_memory: bool = True)[source]#

Train the autoencoder model. The logs are stored in the directory specified by BASEDIR with tensorboard format.

Parameters:

  • train_dataset (torch.utils.data.Dataset) – Training dataset.

  • eval_dataset (torch.utils.data.Dataset) – Evaluation dataset.

  • epochs (int) – Number of epochs to train the model. Default is 100.

  • callback – Callback object. Default is None.

  • lr (float) – Learning rate. Default is 1e-3.

  • BASEDIR (str) – Directory to save the model. Default is "./".

  • reduction (str) – Reduction method for the loss function. Default is "mean".

  • lr_decay (float) – Learning rate decay. Default is 0.999.

  • batch_size (int) – Batch size. Default is 32.

  • shuffle (bool) – Whether to shuffle the dataset or not. Default is True.

  • num_workers (int) – Number of workers for the Dataloader. Default is 0.

  • pin_memory (bool) – Pin memory for Dataloader. Default is True.

reconstruct(dataset: Dataset)[source]#

Reconstruct the dataset using the trained autoencoder model. It prints the energy, mean, and fluctuation of the reconstructed dataset.

Parameters:

dataset (torch.utils.data.Dataset) – Dataset to reconstruct.

Returns:

Reconstructed dataset.

Return type:

np.ndarray

latent_space(dataset: Dataset)[source]#

Compute the latent space of the elements of a given dataset.

Parameters:

dataset (torch.utils.data.Dataset) – Dataset to compute the latent space.

Returns:

Latent space of the dataset elements.

Return type:

np.ndarray

decode(z)[source]#

Decode the latent space to the original space.

Parameters:

z (np.ndarray) – Element of the latent space.

Returns:

Decoded latent space.

Return type:

np.ndarray

class pyLOM.NN.VariationalAutoencoder(latent_dim, in_shape, input_channels, encoder, decoder, device=DEVICE)[source]#

Bases: Autoencoder

Variational Autoencoder class for neural network module. The model is based on the PyTorch.

Parameters:

  • latent_dim (int) – Dimension of the latent space.

  • in_shape (tuple) – Shape of the input data.

  • input_channels (int) – Number of input channels.

  • encoder (torch.nn.Module) – Encoder model.

  • decoder (torch.nn.Module) – Decoder model.

  • device (str) – Device to run the model. Default is ‘cuda’ if available, otherwise ‘cpu’.

forward(x)[source]#

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

fit(train_dataset, eval_dataset=None, betasch=None, epochs=1000, callback=None, lr=1e-4, BASEDIR='./', batch_size=32, shuffle=True, num_workers=0, pin_memory=True)[source]#

Train the variational autoencoder model. The logs are stored in the directory specified by BASEDIR with tensorboard format.

Parameters:

  • train_dataset (torch.utils.data.Dataset) – Training dataset.

  • eval_dataset (torch.utils.data.Dataset) – Evaluation dataset.

  • epochs (int) – Number of epochs to train the model. Default is 100.

  • callback – Callback object to change the value of beta during training. Default is None.

  • lr (float) – Learning rate. Default is 1e-3.

  • BASEDIR (str) – Directory to save the model. Default is "./".

  • reduction (str) – Reduction method for the loss function. Default is "mean".

  • lr_decay (float) – Learning rate decay. Default is 0.999.

  • batch_size (int) – Batch size. Default is 32.

  • shuffle (bool) – Whether to shuffle the dataset or not. Default is True.

  • num_workers (int) – Number of workers for the Dataloader. Default is 0.

  • pin_memory (bool) – Pin memory for Dataloader. Default is True.

reconstruct(dataset)[source]#

Reconstruct the dataset using the trained variational autoencoder model. It prints the energy, mean, and fluctuation of the reconstructed dataset.

Parameters:

dataset (torch.utils.data.Dataset) – Dataset to reconstruct.

Returns:

Reconstructed dataset.

Return type:

np.ndarray

correlation(dataset)[source]#

Compute the correlation between the latent variables of the given dataset.

Parameters:

dataset (torch.utils.data.Dataset) – Dataset to compute the correlation.

Returns:

Correlation between the latent variables.

Return type:

np.ndarray

modes()[source]#

Compute the modes of the latent space.

Returns:

Modes of the latent space.

Return type:

np.ndarray

latent_space(dataset)[source]#

Compute the latent space of the elements of a given dataset.

Parameters:

dataset (torch.utils.data.Dataset) – Dataset to compute the latent space.

Returns:

Latent space of the dataset elements.

Return type:

np.ndarray

fine_tune(train_dataset, shape_, eval_dataset=None, epochs=1000, callback=None, lr=1e-4, BASEDIR='./', **dataloader_params)[source]#
decode(z)[source]#

Decode a latent space element to the original space.

Parameters:

z (np.ndarray) – Element of the latent space.

Returns:

Decoded latent space.

Return type:

np.ndarray

class pyLOM.NN.Encoder2D(nlayers: int, latent_dim: int, nh: int, nw: int, input_channels: int, filter_channels: int, kernel_size: int, padding: int, activation_funcs: list, nlinear: int, batch_norm: bool = False, stride: int = 2, dropout: float = 0, vae: bool = False)[source]#

Bases: Module

Encoder2D class for the 2D Convolutional Autoencoder.

Parameters:

  • nlayers (int) – Number of layers in the encoder.

  • latent_dim (int) – Latent dimension of the encoder.

  • nh (int) – Height of the input mesh/image.

  • nw (int) – Width of the input mesh/image.

  • input_channels (int) – Number of input channels.

  • filter_channels (int) – Number of filter channels.

  • kernel_size (int) – Kernel size for the convolutional layers.

  • padding (int) – Padding for the convolutional layers.

  • activation_funcs (list) – List of activation functions.

  • nlinear (int) – Number of neurons in the linear layer.

  • batch_norm (bool) – Whether to use batch normalization. Default is False.

  • stride (int) – Stride for the convolutional layers. Default is 2.

  • dropout (float) – Dropout probability. Default is 0.

  • vae (bool) – Wheather the encoder is going to be used on a VAE or not. Default is False.

forward(x: Tensor)[source]#

Do a forward evaluation of the data.

Parameters:

x (torch.Tensor) – input data to the neural network.

Returns:

Prediction of the neural network.

Return type:

(torch.Tensor)

class pyLOM.NN.Decoder2D(nlayers: int, latent_dim: int, nh: int, nw: int, input_channels: int, filter_channels: int, kernel_size: int, padding: int, activation_funcs: list, nlinear: int, batch_norm: bool = False, stride: int = 2, dropout: float = 0)[source]#

Bases: Module

Decoder2D class for the 2D Convolutional Autoencoder.

Parameters:

  • nlayers (int) – Number of layers in the encoder.

  • latent_dim (int) – Latent dimension of the encoder.

  • nh (int) – Height of the input mesh/image.

  • nw (int) – Width of the input mesh/image.

  • input_channels (int) – Number of input channels.

  • filter_channels (int) – Number of filter channels.

  • kernel_size (int) – Kernel size for the convolutional layers.

  • padding (int) – Padding for the convolutional layers.

  • activation_funcs (list) – List of activation functions.

  • nlinear (int) – Number of neurons in the linear layer.

  • batch_norm (bool) – Whether to use batch normalization. Default is False.

  • stride (int) – Stride for the convolutional layers. Default is 2.

  • dropout (float) – Dropout probability. Default is 0.

forward(x: Tensor)[source]#

Do a forward evaluation of the data.

Parameters:

x (torch.Tensor) – input data to the neural network.

Returns:

Prediction of the neural network.

Return type:

(torch.Tensor)

class pyLOM.NN.Encoder3D(nlayers: int, latent_dim: int, nx: int, ny: int, nz: int, input_channels: int, filter_channels: int, kernel_size: int, padding: int, activation_funcs: list, nlinear: int, batch_norm: bool = False, stride: int = 2, dropout: float = 0, vae: bool = False)[source]#

Bases: Module

Encoder3D class for the 3D Convolutional Encoder.

Parameters:

  • nlayers (int) – Number of layers in the encoder.

  • latent_dim (int) – Latent dimension of the encoder.

  • nx (int) – Height of the input mesh/image.

  • ny (int) – Width of the input mesh/image.

  • nz (int) – Depth of the input mesh/image.

  • input_channels (int) – Number of input channels.

  • filter_channels (int) – Number of filter channels.

  • kernel_size (int) – Kernel size for the convolutional layers.

  • padding (int) – Padding for the convolutional layers.

  • activation_funcs (list) – List of activation functions.

  • nlinear (int) – Number of neurons in the linear layer.

  • batch_norm (bool) – Whether to use batch normalization. Default is False.

  • stride (int) – Stride for the convolutional layers. Default is 2.

  • dropout (float) – Dropout probability. Default is 0.

  • vae (bool) – Wheather the encoder is going to be used on a VAE or not. Default is False.

forward(x: Tensor)[source]#

Do a forward evaluation of the data.

Parameters:

x (torch.Tensor) – input data to the neural network.

Returns:

Prediction of the neural network.

Return type:

(torch.Tensor)

class pyLOM.NN.Decoder3D(nlayers: int, latent_dim: int, nx: int, ny: int, nz: int, input_channels: int, filter_channels: int, kernel_size: int, padding: int, activation_funcs: list, nlinear: int, batch_norm: bool = False, stride: int = 2, dropout: float = 0)[source]#

Bases: Module

Dencoder3D class for the 3D Convolutional Autoencoder.

Parameters:

  • nlayers (int) – Number of layers in the encoder.

  • latent_dim (int) – Latent dimension of the encoder.

  • nx (int) – Height of the input mesh/image.

  • ny (int) – Width of the input mesh/image.

  • nz (int) – Depth of the input mesh/image.

  • input_channels (int) – Number of input channels.

  • filter_channels (int) – Number of filter channels.

  • kernel_size (int) – Kernel size for the convolutional layers.

  • padding (int) – Padding for the convolutional layers.

  • activation_funcs (list) – List of activation functions.

  • nlinear (int) – Number of neurons in the linear layer.

  • batch_norm (bool) – Whether to use batch normalization. Default is False.

  • stride (int) – Stride for the convolutional layers. Default is 2.

  • dropout (float) – Dropout probability. Default is 0.

forward(x: Tensor)[source]#

Do a forward evaluation of the data.

Parameters:

x (torch.Tensor) – input data to the neural network.

Returns:

Prediction of the neural network.

Return type:

(torch.Tensor)

class pyLOM.NN.ShallowDecoder(output_size: int, hidden_size: int, decoder_sizes: list, dropout: float)[source]#

Bases: Module

Decoder used for the SHRED architecture.

Parameters:

  • output_size (int) – Number of POD modes to predict.

  • hidden_size (int) – Dimension of the LSTM hidden layers. decoder_sizes (list): Integer list of the decoder layer sizes.

  • dropout (float) – Dropout probability for the decoder.

forward(output: Tensor)[source]#

Do a forward evaluation of the data.

Parameters:

x (torch.Tensor) – input data to the neural network.

Returns:

Prediction of the neural network.

Return type:

(torch.Tensor)

class pyLOM.NN.PINN(neural_net, device)[source]#

Bases: ABC

This class represents a Physics-Informed Neural Network (PINN) model. It is an abstract class that needs to be subclassed to implement the pde_loss method. That method should compute the residual from the partial differential equation (PDE) and then compute the loss from it (usually by squaring the residual).

Parameters:

  • neural_net (torch.nn.Module) – A neural network model that implements torch.nn.Module.

  • device (str) – The device to run the model on (e.g., ‘cpu’, ‘cuda’).

Variables:

  • device (str) – The device the model is running on.

  • model (torch.nn.Module) – The neural network model.

__call__(x)[source]#

Forward pass of the PINN model.

Parameters:

x (torch.Tensor) – The input tensor with the PDE input parameters.

Returns:

The output tensor, i.e. the solution for the PDE on x.

Return type:

torch.Tensor

bc_data_loss(pred, y, boundary_conditions, use_bfloat16=False)[source]#

Computes the loss from boundary conditions and data.

Parameters:

  • pred (torch.Tensor) – The predicted output tensor.

  • y (torch.Tensor) – The target tensor.

  • boundary_conditions (List[BoundaryCondition]) – The list of boundary conditions.

  • use_bfloat16 (bool, optional) – Whether to use bfloat16 precision. Defaults to False.

Returns:

The list of loss tensors.

Return type:

List[torch.Tensor]

compute_loss(x, y, boundary_conditions, use_bfloat16=False)[source]#

Computes the total loss for training.

Parameters:

  • x (torch.Tensor) – The input tensor.

  • y (torch.Tensor) – The target tensor.

  • boundary_conditions (List[BoundaryCondition]) – The list of boundary conditions.

  • use_bfloat16 (bool, optional) – Whether to use bfloat16 precision. Defaults to False.

Returns:

The list of loss tensors.

Return type:

List[torch.Tensor]

fit(train_dataset: Dataset, optimizer_class=torch.optim.Adam, optimizer_params={}, lr_scheduler_class=None, lr_scheduler_params={}, epochs=1000, boundary_conditions=[], update_logs_steps=1, loaded_logs=None, batch_size=None, eval_dataset: Dataset = None, use_bfloat16=False, **kwargs)[source]#

Trains the PINN model.

Parameters:

  • train_dataset (Dataset) – The training dataset. If the dataset returns a tuple, the first element is the input and the second element is the target. If not, the PINN is trained without simulation data.

  • optimizer_class (torch.optim.Optimizer, optional) – The optimizer class. Defaults to torch.optim.Adam.

  • optimizer_params (dict, optional) – The optimizer parameters. Defaults to {}.

  • lr_scheduler_class (torch.optim.lr_scheduler._LRScheduler, optional) – The learning rate scheduler class. Defaults to None.

  • lr_scheduler_params (dict, optional) – The learning rate scheduler parameters. Defaults to {}.

  • epochs (int, optional) – The number of epochs to train for. Defaults to 1000.

  • boundary_conditions (List[BoundaryCondition], optional) – The list of boundary conditions. Defaults to [].

  • update_logs_steps (int, optional) – The interval for updating the progress. Defaults to 100.

  • loaded_logs (dict, optional) – Loaded training logs to be used as initial logs. Defaults to None.

  • batch_size (int, optional) – The batch size. If none, the batch size will be equal to the number of collocation points given on train_dataset. Defaults to None.

  • eval_dataset (BaseDataset, optional) – The evaluation dataset. Defaults to None.

  • use_bfloat16 (bool, optional) – Whether to use bfloat16 precision. Defaults to False.

  • **kwargs – Additional keyword arguments.

Returns:

The training logs.

Return type:

dict

predict(X: Dataset, **kwargs) → ndarray[source]#

Predicts for the input dataset.

Parameters:

X (Dataset) – The input dataset.

Returns:

The predictions of the model.

Return type:

np.ndarray

__repr__()[source]#

Returns a string representation of the PINN model.

Returns:

The string representation.

Return type:

str

plot_training_logs(logs)[source]#

Plots the training logs.

Parameters:

logs (dict) – The training logs.

abstract pde_loss(pred, *input_variables)[source]#

Computes the loss from the partial differential equation (PDE).

Parameters:

  • pred (torch.Tensor) – The predicted output tensor.

  • *input_variables (torch.Tensor) – The input variables for the PDE. e.g. x, y, t.

Returns:

The loss tensor.

Return type:

torch.Tensor

save(path)[source]#

Saves the model to a file using torchscript.

Parameters:

path (str) – The path to save the model.

classmethod load(path, device='cpu')[source]#

Loads the model from a file.

Parameters:

  • path (str) – The path to load the model.

  • neural_net (torch.nn.Module) – The neural network model.

  • device (str, optional) – The device to run the model on. Defaults to ‘cpu’.

Returns:

The loaded PINN model.

Return type:

PINN

class pyLOM.NN.BurgersPINN(neural_net, device, viscosity=0.01)[source]#

Bases: PINN

This class represents a Physics-Informed Neural Network (PINN) model for the Burgers’ equation. The model predictions have 1 column, the velocity field \(u\).

\[\frac{\partial u}{\partial t} + u\frac{\partial u}{\partial x} = \nu\frac{\partial^2u}{\partial x^2}\]
Parameters:

  • neural_net (torch.nn.Module) – The neural network model.

  • device (str) – The device to run the model on (e.g., ‘cpu’, ‘cuda’).

  • viscosity (float) – The viscosity coefficient.

pde_loss(pred, *input_variables)[source]#

Computes the loss from the partial differential equation (PDE).

Parameters:

  • pred (torch.Tensor) – The predicted output tensor.

  • *input_variables (torch.Tensor) – The input variables for the PDE. e.g. x, y, t.

Returns:

The loss tensor.

Return type:

torch.Tensor

class pyLOM.NN.Euler2DPINN(neural_net, device)[source]#

Bases: PINN

This class represents a Physics-Informed Neural Network (PINN) model for the 2D Euler equations. The model predictions have 4 columns, the density \(\rho\), the velocity field \((u, v)\) and the total energy \(E\) fields.

\[\begin{split}\begin{align*} \frac{\partial \rho}{\partial t} + \frac{\partial (\rho u)}{\partial x} + \frac{\partial (\rho v)}{\partial y} &= 0, \\ \frac{\partial (\rho u)}{\partial t} + \frac{\partial (\rho u^2 + p)}{\partial x} + \frac{\partial (\rho uv)}{\partial y} &= 0, \\ \frac{\partial (\rho v)}{\partial t} + \frac{\partial (\rho uv)}{\partial x} + \frac{\partial (\rho v^2 + p)}{\partial y} &= 0, \\ \frac{\partial (\rho E)}{\partial t} + \frac{\partial (u(\rho E + p))}{\partial x} + \frac{\partial (v(\rho E + p))}{\partial y} &= 0. \end{align*}\end{split}\]
Parameters:

  • neural_net (torch.nn.Module) – The neural network model.

  • device (str) – The device to run the model on (e.g., ‘cpu’, ‘cuda’).

GAMMA = 1.4#
pde_loss(pred, *input_variables)[source]#

Computes the loss from the partial differential equation (PDE).

Parameters:

  • pred (torch.Tensor) – The predicted output tensor.

  • *input_variables (torch.Tensor) – The input variables for the PDE. e.g. x, y, t.

Returns:

The loss tensor.

Return type:

torch.Tensor

class pyLOM.NN.NavierStokesIncompressible2DPINN(neural_net, device, Re=50)[source]#

Bases: PINN

This class represents a Physics-Informed Neural Network (PINN) model for the incompressible steady 2D Navier-Stokes equations. The model predictions have 3 columns, the velocity field \((u, v)\) and the pressure \(p\) fields.

\[\begin{split}\begin{align*} u \frac{\partial u}{\partial x} + v \frac{\partial u}{\partial y} + \frac{\partial p}{\partial x} - \frac{1}{Re} \left( \frac{\partial^2 u}{\partial x^2} + \frac{\partial^2 u}{\partial y^2} \right) &= 0, \\ u \frac{\partial v}{\partial x} + v \frac{\partial v}{\partial y} + \frac{\partial p}{\partial y} - \frac{1}{Re} \left( \frac{\partial^2 v}{\partial x^2} + \frac{\partial^2 v}{\partial y^2} \right) &= 0, \\ \frac{\partial u}{\partial x} + \frac{\partial v}{\partial y} &= 0. \end{align*}\end{split}\]
Parameters:

  • neural_net (torch.nn.Module) – The neural network model.

  • device (str) – The device to run the model on (e.g., ‘cpu’, ‘cuda’).

  • Re (float) – The Reynolds number.

pde_loss(pred, x, y)[source]#

Computes the loss from the partial differential equation (PDE).

Parameters:

  • pred (torch.Tensor) – The predicted output tensor.

  • *input_variables (torch.Tensor) – The input variables for the PDE. e.g. x, y, t.

Returns:

The loss tensor.

Return type:

torch.Tensor

class pyLOM.NN.BoundaryCondition(points)[source]#

Bases: ABC

Abstract base class for defining boundary conditions. You need to implement the loss method to use a custom boundary condition.

Parameters:

points (Tensor) – The points where the boundary condition is defined.

Variables:

points (Tensor) – The points where the boundary condition is defined.

abstract loss(pred)[source]#

Computes the loss for the given prediction.

Parameters:

pred (Tensor) – The predicted values on the points where the boundary condition is defined.

Returns:

The loss value.

Return type:

Tensor

property points#
class pyLOM.NN.SHRED(output_size: int, device: device, total_sensors: int, hidden_size: int = 64, hidden_layers: int = 2, decoder_sizes: list = [350, 400], input_size: int = 3, dropout: int = 0.1, nconfigs: int = 1, compile: bool = False, seed: int = -1)[source]#

Bases: Module

Shallow recurrent decoder (SHRED) architecture. For more information on the theoretical background of the architecture check the following reference

Williams, J. P., Zahn, O., & Kutz, J. N. (2023). Sensing with shallow recurrent decoder networks. arXiv preprint arXiv:2301.12011.

The model is based on the PyTorch library torch.nn (detailed documentation can be found at https://pytorch.org/docs/stable/nn.html).

In this implementation we assume that the output are always the POD coefficients of the full dataset.

Parameters:

  • output_size (int) – Number of POD modes. device (torch.device): Device to use. total_sensors (int): Total number of sensors that will be used to ensamble the different configurations.

  • hidden_size (int, optional) – Dimension of the LSTM hidden layers (default: 64). hidden_layers (int, optional): Number of LSTM hidden layers (default: 2). decoder_sizes (list, optional): Integer list of the decoder layer sizes (default: [350, 400]). input_size (int, optional): Number of sensor signals used as input (default: 3).

  • dropouts (float, optional) – Dropout probability for the decoder (default: 0.1).

  • nconfigs (int, optional) – Number of configurations to train SHRED on (default: 1).

  • compile (bool, optional) – Flag to compile the model (default: False).

  • seed (int, optional) – Seed for reproducibility (default: -1).

forward(x: Tensor)[source]#

Do a forward evaluation of the data.

Parameters:

x (torch.Tensor) – input data to the neural network.

Returns:

Prediction of the neural network.

Return type:

(torch.Tensor)

freeze()[source]#

Freeze the model parameters to set it on inference mode.

unfreeze()[source]#

Unfreeze the model parameters to set it on training mode.

fit(train_dataset: Dataset, valid_dataset: Dataset, batch_size: int = 64, epochs: int = 4000, optim: Optimizer = torch.optim.Adam, lr: float = 1e-3, reduction: str = 'mean', verbose: bool = False, patience: int = 5, mod_scale: Tensor = None)[source]#

Fit of the SHRED model.

Parameters:

  • train_dataset (torch.utils.data.Dataset) – training dataset.

  • valid_dataset (torch.utils.data.Dataset) – validation dataset.

  • batch_size (int, optional) – length of each training batch (default: 64).

  • epochs (int, optional) – number of epochs to extend the training (default: 4000).

  • optim (torch.optim, optional) – optimizer used (default: torch.optim.Adam).

  • lr (float, optional) – learning rate (default: 0.001).

  • verbose (bool, optional) – define level of explicity on the output (default: False).

  • patience (int, optional) – epochs without improvements on the validation loss before stopping the training (default to 5).

save(path: str, scaler_path: str, podscale_path: str, sensors: array)[source]#

Save a SHRED configuration to a .pth file.

Parameters:

  • path (str) – where the model will be saved.

  • scaler_path (str) – path to the scaler used to scale the sensor data.

  • podscale_path (str) – path to the scaler used for the POD coefficients.

  • sensors (np.array) – IDs of the sensors used for the current SHRED configuration.

pyLOM.NN.tanh()[source]#
pyLOM.NN.relu()[source]#
pyLOM.NN.elu()[source]#
pyLOM.NN.sigmoid()[source]#
pyLOM.NN.leakyRelu()[source]#
pyLOM.NN.silu()[source]#
On this page
Edit on GitHub

This Page