‹ Back to Tutorials

OpenNN in 6 steps

This tutorial shows the six main steps for building a classification model with the current OpenNN API. The example classifies Iris plants from four measurements and uses the maintained Iris plant example in the OpenNN repository.

The objective is to obtain a model that classifies new samples correctly and generalizes beyond the data used for training.

For the underlying concepts, see the neural network tutorial from Neural Designer.

Contents:

  1. Data set
  2. Neural network
  3. Training strategy
  4. Model selection
  5. Testing analysis
  6. Model deployment

1. Data set

The first step is to load the data. This example uses iris_plant_original.csv, a semicolon-separated file with a header row and 150 samples.

The file contains four numeric input variables and one categorical target:

  • sepal_lenght
  • sepal_width
  • petal_lenght
  • petal_width
  • iris_class

Use TabularDataset for CSV data:

TabularDataset dataset(
    "../data/iris_plant/iris_plant_original.csv",
    ";",
    true,
    false);

OpenNN detects the variable types, assigns the last variable as the target and expands its three categories into three target features. Therefore, the data set contains five variables and seven model features: four inputs and three targets.

The constructor also divides the samples randomly into training, validation and testing subsets using the default 60%, 20% and 20% ratios.

Obtain the input and target dimensions as follows:

const Index inputs_number = dataset.get_features_number("Input");
const Index targets_number = dataset.get_features_number("Target");

There is no need to scale or modify the data manually. During training, OpenNN calculates the scaling statistics from the training subset and configures the network’s scaling layer automatically.

For more information, see the data set tutorial.

2. Neural network

The second step is to define the model architecture. A tabular classification network contains:

  • A scaling layer.
  • One or more dense hidden layers.
  • A final dense layer with Softmax for multiclass classification or Sigmoid for binary classification.

ClassificationNetwork creates and compiles these layers automatically. We start with three hidden neurons because the next step will evaluate different network sizes:

const Index initial_neurons_number = 3;

ClassificationNetwork neural_network(
    {inputs_number},
    {initial_neurons_number},
    {targets_number});

For more information, see the neural network tutorial.

3. Training strategy

The training strategy combines a loss function and an optimization algorithm. For this multiclass problem, ClassificationNetwork selects cross-entropy as the default loss.

The default optimizer for tabular classification is the quasi-Newton method. In this example, we select adaptive moment estimation (Adam) and configure its stopping parameters:

TrainingStrategy training_strategy(&neural_network, &dataset);

training_strategy.set_optimization_algorithm(
    "AdaptiveMomentEstimation");

auto* adam = dynamic_cast<AdaptiveMomentEstimation*>(
    training_strategy.get_optimization_algorithm());

adam->set_maximum_epochs(500);
adam->set_display_period(100);

The model selection step below trains the candidate networks and leaves the selected model fitted. If model selection is not required, train the current architecture directly with training_strategy.train().

For more information, see the training strategy tutorial.

4. Model selection

Model selection searches for an architecture that balances fit and generalization. The growing-neurons algorithm trains networks with different hidden-layer sizes and compares their validation errors.

ModelSelection model_selection(&training_strategy);
model_selection.perform_neurons_selection();

When the search finishes, OpenNN restores the selected architecture and its trained parameters. This step is optional for a fixed architecture.

For more information, see the model selection tutorial.

5. Testing analysis

Testing analysis evaluates the final model with samples that were not used for training or validation. There is no need to scale or unscale the data manually; the network applies its scaling layer during inference.

For multiclass classification, calculate the confusion matrix:

TestingAnalysis testing_analysis(&neural_network, &dataset);

const MatrixI confusion = testing_analysis.calculate_confusion();
cout << "Confusion matrix:\n" << confusion << endl;

Rows represent actual classes and columns represent predicted classes. The diagonal contains correct classifications, while the final row and column contain totals.

For more information, see the testing analysis tutorial.

6. Model deployment

Once the model is trained, use calculate_outputs to obtain predictions for new samples. The Iris network returns one probability for each class:

MatrixR inputs(1, 4);
inputs << 5.1, 3.5, 1.4, 0.2;

const MatrixR outputs = neural_network.calculate_outputs(inputs);
cout << "Class probabilities: " << outputs << endl;

The three output columns correspond to iris_setosa, iris_versicolor and iris_virginica.

Save the complete OpenNN model as JSON, or export a standalone expression for deployment without the OpenNN library:

neural_network.save("iris_model.json");

const ModelExpression model_expression(&neural_network);

model_expression.save(
    "iris_model.c",
    ModelExpression::ProgrammingLanguage::C);

model_expression.save(
    "iris_model_tables.c",
    ModelExpression::ProgrammingLanguage::CEmbedded);

model_expression.save(
    "iris_model.py",
    ModelExpression::ProgrammingLanguage::Python);

ModelExpression can also generate JavaScript and PHP.

Complete program

The following program combines the six steps above:

#include "opennn/core/configuration.h"
#include "opennn/dataset/tabular_dataset.h"
#include "opennn/model_selection/model_selection.h"
#include "opennn/models/models.h"
#include "opennn/neural_network/model_expression.h"
#include "opennn/testing_analysis/testing_analysis.h"
#include "opennn/training_strategy/adaptive_moment_estimation.h"
#include "opennn/training_strategy/training_strategy.h"

using namespace opennn;

int main()
{
    try
    {
        Configuration::instance().set(Device::CPU, Type::FP32);

        // 1. Data set
        TabularDataset dataset(
            "../data/iris_plant/iris_plant_original.csv",
            ";",
            true,
            false);

        const Index inputs_number =
            dataset.get_features_number("Input");
        const Index targets_number =
            dataset.get_features_number("Target");

        // 2. Neural network
        const Index initial_neurons_number = 3;

        ClassificationNetwork neural_network(
            {inputs_number},
            {initial_neurons_number},
            {targets_number});

        // 3. Training strategy
        TrainingStrategy training_strategy(
            &neural_network,
            &dataset);

        training_strategy.set_optimization_algorithm(
            "AdaptiveMomentEstimation");

        auto* adam = dynamic_cast<AdaptiveMomentEstimation*>(
            training_strategy.get_optimization_algorithm());

        adam->set_maximum_epochs(500);
        adam->set_display_period(100);

        // 4. Model selection
        ModelSelection model_selection(&training_strategy);
        model_selection.perform_neurons_selection();

        // 5. Testing analysis
        TestingAnalysis testing_analysis(
            &neural_network,
            &dataset);

        const MatrixI confusion =
            testing_analysis.calculate_confusion();

        cout << "Confusion matrix:\n"
             << confusion << endl;

        // 6. Model deployment
        MatrixR inputs(1, 4);
        inputs << 5.1, 3.5, 1.4, 0.2;

        const MatrixR outputs =
            neural_network.calculate_outputs(inputs);

        cout << "Class probabilities: "
             << outputs << endl;

        neural_network.save("iris_model.json");

        const ModelExpression model_expression(
            &neural_network);

        model_expression.save(
            "iris_model.c",
            ModelExpression::ProgrammingLanguage::C);
        model_expression.save(
            "iris_model_tables.c",
            ModelExpression::ProgrammingLanguage::CEmbedded);
        model_expression.save(
            "iris_model.py",
            ModelExpression::ProgrammingLanguage::Python);

        return 0;
    }
    catch(const exception& exception)
    {
        cerr << exception.what() << endl;
        return 1;
    }
}