Skip to content

Back to Tutorials

The software model of OpenNN

This tutorial describes the current high-level C++ software model of OpenNN. The library is divided into dependency-ordered modules for core utilities, networks, datasets, training, model selection, evaluation, response optimization and ready-made models.

The diagrams use the Unified Modeling Language (UML) to show the most important public classes and their relationships. Internal helper types are omitted so that the model remains readable.

Contents:

  1. Core classes
  2. Associations
  3. Compositions
  4. Configured and specialized classes
  5. Members and methods

1. Core classes

The main classes cover the complete model lifecycle, from loading data to evaluating and optimizing a trained model.

%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    class Dataset
    class Network
    class Training
    class ModelSelection
    class Evaluation
    class ResponseOptimization
  • Dataset: abstract data interface for samples, variables, roles, shapes and batches.
  • Network: owns and executes a directed graph of layers on CPU or CUDA.
  • Training: combines a loss and an optimizer to train a network with a dataset.
  • ModelSelection: searches for useful inputs and an appropriate hidden-layer size.
  • Evaluation: evaluates a trained model on testing samples.
  • ResponseOptimization: searches for model inputs that satisfy constraints and optimize one or more outputs.

2. Associations

These classes collaborate through non-owning references. Ownership of datasets and networks remains with the application.

Training and selection

%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    direction LR
    Training --> Network
    Training --> Dataset
    ModelSelection --> Training

Evaluation and response optimization

%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    direction LR
    Evaluation --> Network
    Evaluation --> Dataset
    ResponseOptimization --> Network
  • Training points to the Network and Dataset used during training.
  • ModelSelection repeatedly invokes its Training to compare candidates.
  • Evaluation combines network outputs with the corresponding testing targets.
  • ResponseOptimization evaluates the network while exploring feasible input values.

3. Compositions

Dataset

Dataset defines the common batching and metadata interface. Concrete datasets implement storage and preprocessing for each data modality.

%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    direction TB
    class Dataset
    class TabularDataset
    class ImageDataset
    class TextDataset
    class YoloDataset
    Dataset <|-- TabularDataset
    Dataset <|-- ImageDataset
    Dataset <|-- TextDataset
    Dataset <|-- YoloDataset
  • TabularDataset: delimited tabular files, variables, missing values, correlations, scaling and forecasting windows with past inputs and future targets.
  • ImageDataset: image folders and image preprocessing.
  • YoloDataset: object-detection images, boxes and class labels.
  • TextDataset: text classification, paired input and target sequences, next-token blocks from a continuous corpus, and token and attention-mask inputs for BERT models.

Network

A Network owns Layer objects and records the source layer indices for every connection. This supports sequential models, residual paths and multi-input graphs.

%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    direction TB
    class Layer
    class Scaling
    class Unscaling
    class Clamping
    class Dense
    class Activation
    class Recurrent
    class LSTM
    Layer <|-- Scaling
    Scaling <|-- Unscaling
    Layer <|-- Clamping
    Layer <|-- Dense
    Layer <|-- Activation
    Layer <|-- Recurrent
    Layer <|-- LSTM
%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    direction TB
    class Layer
    class Convolutional
    class Pooling
    class Pooling3d
    class Flatten
    class Normalization3d
    class Upsampling
    class Concatenation
    class Addition
    class C2PSA
    class Detection
    class DetectionV8
    class NonMaxSuppression
    Layer <|-- Convolutional
    Layer <|-- Pooling
    Layer <|-- Pooling3d
    Layer <|-- Flatten
    Layer <|-- Normalization3d
    Layer <|-- Upsampling
    Layer <|-- Concatenation
    Layer <|-- Addition
    Layer <|-- C2PSA
    Layer <|-- Detection
    Layer <|-- DetectionV8
    Layer <|-- NonMaxSuppression
%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    direction TB
    class Layer
    class Tokenizer
    class Embedding
    class MultiHeadAttention
    class GroupedQueryAttention
    Layer <|-- Tokenizer
    Layer <|-- Embedding
    Layer <|-- MultiHeadAttention
    Layer <|-- GroupedQueryAttention

The current catalogue no longer contains DenseRelu, ConvolutionalRelu or Bounding classes. Activations are configured in their corresponding layers, and output limits are implemented by Clamping. The name Bounding is accepted only as a legacy loading alias.

Ready-made models

The classes in opennn/models build common architectures on top of Network.

%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    direction TB
    class Network
    class ApproximationNetwork
    class ClassificationNetwork
    class ForecastingNetwork
    class ForecastingLstmNetwork
    class Autoencoder
    class ImageClassificationNetwork
    class ResNet
    class Yolo
    Network <|-- ApproximationNetwork
    Network <|-- ClassificationNetwork
    Network <|-- ForecastingNetwork
    Network <|-- ForecastingLstmNetwork
    Network <|-- Autoencoder
    Network <|-- ImageClassificationNetwork
    Network <|-- ResNet
    Network <|-- Yolo
%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    direction TB
    class Network
    class TextClassificationNetwork
    class Transformer
    class TextGenerationNetwork
    class Qwen3
    class Bert
    class BertForSequenceClassification
    Network <|-- TextClassificationNetwork
    Network <|-- Transformer
    Network <|-- TextGenerationNetwork
    Network <|-- Qwen3
    Network <|-- Bert
    Network <|-- BertForSequenceClassification

Training

Training owns one Loss and one Optimizer. Both point to the same network and dataset used by the training loop.

%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    direction LR
    class Training {
        -unique_ptr~Loss~ loss
        -unique_ptr~Optimizer~ optimizer
        -Network* network
        -Dataset* dataset
        +train() TrainingResult
    }
    class Loss
    class Optimizer
    class Network
    class Dataset
    Training *-- Loss : owns
    Training *-- Optimizer : owns
    Training --> Network : uses
    Training --> Dataset : uses

ModelSelection

ModelSelection owns a GrowingNeurons object and an optional input-selection strategy. All selection algorithms share the common SelectionAlgorithm configuration.

%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    direction TB
    class ModelSelection
    class SelectionAlgorithm
    class InputSelection
    class GrowingInputs
    class GeneticAlgorithm
    class GrowingNeurons
    SelectionAlgorithm <|-- InputSelection
    SelectionAlgorithm <|-- GrowingNeurons
    InputSelection <|-- GrowingInputs
    InputSelection <|-- GeneticAlgorithm
    ModelSelection *-- InputSelection : owns selected strategy
    ModelSelection *-- GrowingNeurons : owns

The former separate NeuronSelection hierarchy no longer exists; GrowingNeurons performs neuron selection directly.

4. Configured and specialized classes

Loss

Loss is a single class configured with an Error enum. It also supports L1, L2 or no regularization.

%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    class Loss {
        +MeanSquaredError
        +MeanAbsoluteError
        +NormalizedSquaredError
        +WeightedSquaredError
        +CrossEntropy
        +CrossEntropy3d
        +MinkowskiError
        +Yolo
        +set_error(Error)
        +set_regularization(Regularization)
    }

Optimizer

The optimizer registry exposes four named implementations. The base Optimizer also contains the shared training loop, stopping criteria, batching and callbacks.

%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    direction TB
    class Optimizer
    class Adam
    class QuasiNewton
    class SGD
    class LevenbergMarquardt
    Optimizer <|-- Adam
    Optimizer <|-- QuasiNewton
    Optimizer <|-- SGD
    Optimizer <|-- LevenbergMarquardt

InputSelection

InputSelection is the abstract feature-selection interface. Its current implementations are GrowingInputs and GeneticAlgorithm. Neuron selection is performed directly by GrowingNeurons.

5. Members and methods

The following diagrams summarize representative public methods. They are intentionally shorter than the complete headers.

Dataset and TabularDataset

%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    class Dataset {
        +split_samples(training, validation, testing)
        +get_features_number(role) Index
        +get_feature_names(role) vector~string~
        +get_input_shape() Shape
        +get_target_shape() Shape
        +fill_batch(Batch, indices, features)
    }
    class TabularDataset {
        +set(path, separator, has_header, has_ids)
        +scale_features(role) vector~Descriptives~
        +unscale_features(role, descriptives)
        +read_csv()
    }
    Dataset <|-- TabularDataset

Network

%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    class Network {
        +add_layer(unique_ptr~Layer~, source_indices) Index
        +compile(Device)
        +get_layer(index_or_name) Layer
        +get_source_layers() vector
        +calculate_outputs(Tensor) MatrixR
        +save(path)
        +load(path)
    }

Training

%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    class Training {
        +Training(Network*, Dataset*)
        +set_loss(name)
        +set_optimization_algorithm(name)
        +get_loss() Loss*
        +get_optimization_algorithm() Optimizer*
        +train() TrainingResult
        +save(path)
        +load(path)
    }

ModelSelection

%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    class ModelSelection {
        +ModelSelection(Training*)
        +perform_input_selection() InputSelectionResult
        +perform_neurons_selection() NeuronsSelectionResult
        +get_input_selection_name() string
        +save(path)
        +load(path)
    }

Evaluation

%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    class Evaluation {
        +Evaluation(Network*, Dataset*)
        +calculate_error_data() Tensor3
        +perform_goodness_of_fit_analysis() results
        +calculate_confusion(threshold) MatrixI
        +perform_roc_analysis() RocAnalysis
        +calculate_reconstruction_errors(role) VectorR
    }

ResponseOptimization

%%{init: {"look":"classic","theme":"base","themeVariables":{"background":"#ffffff","primaryColor":"#ffffff","primaryTextColor":"#111111","primaryBorderColor":"#111111","lineColor":"#111111","secondaryColor":"#ffffff","tertiaryColor":"#ffffff","edgeLabelBackground":"#ffffff","fontFamily":"Outfit, Arial, sans-serif"},"class":{"hideEmptyMembersBox":true}}}%%
classDiagram
    class ResponseOptimization {
        +ResponseOptimization(Network*)
        +add_objective(expression, sense, value)
        +add_constraint(expression, condition, values)
        +set_iterations_number(Index)
        +perform_response_optimization() MatrixR
        +get_advised_point(results) pair
    }
    class DomainContraction
    class GeneticResponse
    ResponseOptimization <|-- DomainContraction
    ResponseOptimization <|-- GeneticResponse

References