Estimate airline passengers using OpenNN

This example uses past months’ data to predict the maximum number of passengers an airline will have.

Contents:

  1. Application type.
  2. Data set.
  3. Neural network.
  4. Training strategy.
  5. Testing analysis.
  6. Model deployment.
  7. Full code.

1. Application type

The variable to be predicted is discrete (the number of passengers per month), so this is a forecasting project.

The primary goal is to model the passengers in the coming months, conditioned on the past ones.

2. Data set

The first step is to prepare the data set, which is the source of information for the forecasting problem.

The data source is the file airline_passengers.csv. It contains the data for this example in comma-separated values (CSV) format and can be loaded as:

TimeSeriesDataSet data_set("path_to_source/airine_passengers.csv", ';', true, false);

There are two columns and 144 rows. The variables in this problem are:

  • months: Unused.
  • passengers: Number of passengers in each month, used as input.

We need to preprocess the data for the forecasting model. This involves setting lag values and defining the number of steps ahead before transforming the data using:

const Index lags_number = 1;
const Index steps_ahead_number = 1;
data_set.set_lags_number(lags_number);
data_set.set_steps_ahead_number(steps_ahead_number);
data_set.transform_time_series();

To get the input and target variable number, we use the following command:

const Index input_variables_number = data_set.get_variables_number(DataSet::VariableUse::Input);
const Index target_variables_number = data_set.get_variables_number(DataSet::VariableUse::Target);

For more information on the data set methods, see the DataSet class.

3. Neural network

The second step is to choose the correct neural network architecture. For forecasting problems, it is usually composed of:

  • A scaling layer.
  • A long short-term memory layer.
  • A perceptron layer.
  • An unscaling layer.

The NeuralNetwork class is responsible for building the neural network and adequately organizing the layers of neurons using the following constructor. If you need more complex architectures, you should see the NeuralNetwork class.

NeuralNetwork neural_network(NeuralNetwork::ModelType::Forecasting,
   {input_variables_number}, {hidden_neurons_number}, {target_variables_number});

Therefore, we have already created a good-looking model. Thus, we proceed to the learning process with TrainingStrategy.

4. Training strategy

The third step is to set the training strategy, which is composed of:

  • Loss index.
  • Optimization algorithm.

Firstly, we construct the training strategy object

TrainingStrategy training_strategy(&neural_network, &data_set);

next, we set the error term

training_strategy.set_loss_method(TrainingStrategy::LossMethod::MEAN_SQUARED_ERROR);

and finally, the optimization algorithm

training_strategy.set_optimization_method(TrainingStrategy::OptimizationMethod::ADAPTIVE_MOMENT_ESTIMATION);

OpenNN builds the training strategy object by default using the quasi-Newton method as the optimization algorithm and normalized squared error as the loss method. We can now start the training process by using the command:

training_strategy.perform_training();

If we need to go further, OpenNN allows control of the optimization, for example:

AdaptiveMomentEstimation* adam = training_strategy.get_adaptive_moment_estimation();
adam->set_loss_goal(type(1.0e-3));
adam->set_maximum_epochs_number(10000);
adam->set_display_period(1000);

For more information on the training strategy methods, see the TrainingStrategy class.

5. Testing analysis

The fourth step is to evaluate our model. For that purpose, we need to use the TestingAnalysis class, whose goal is to validate the model’s generalization performance. Here, we compare the neural network outputs to the corresponding targets in the testing instances of the data set.
As in previous cases, we start by building the testing analysis object

TestingAnalysis testing_analysis(&neural_network, &data_set);

and perform the testing. In our case, we use linear regression:

testing_analysis.print_goodness_of_fit_analysis();

For more information about the testing analysis methods, see the TestingAnalysis class.

6. Model deployment

Once our model is complete, the neural network is prepared to predict outputs for the upcoming months. This process is called model deployment.

To generate predictions with new data, you can use:

neural_network.calculate_outputs();

For instance, the inputs can be calculated by:

Tensor<type, 2> inputs = data_set.get_data();

and for outputs, we can write

neural_network.calculate_outputs(inputs);

or save the model.

neural_network.save_expression(C, "../data/expression.txt");
neural_network.save_expression(Python, "../data/expression.py");

You can implement the model in Python, PHP, and so on.

7. Full code

Merging all steps, we obtain the following code:

// Data Set
TimeSeriesDataSet data_set("path_to_source/airine_passengers.csv", ';', true, false);
const Index lags_number = 1;
const Index steps_ahead_number = 1;
data_set.set_lags_number(lags_number);
data_set.set_steps_ahead_number(steps_ahead_number);
data_set.transform_time_series();
const Index input_variables_number = data_set.get_variables_number(DataSet::VariableUse::Input);
const Index target_variables_number = data_set.get_variables_number(DataSet::VariableUse::Target);
            
// Neural Network
const Index hidden_neurons_number = 3;
NeuralNetwork neural_network(NeuralNetwork::ModelType::Forecasting,
     {input_variables_number}, {hidden_neurons_number}, {target_variables_number});
            
// Training Strategy
TrainingStrategy training_strategy(&neural_network, &data_set);
training_strategy.set_loss_method(TrainingStrategy::LossMethod::MEAN_SQUARED_ERROR);
training_strategy.set_optimization_method(TrainingStrategy::OptimizationMethod::ADAPTIVE_MOMENT_ESTIMATION);
            
// Testing Analysis
TestingAnalysis testing_analysis(&neural_network, &data_set);
testing_analysis.print_goodness_of_fit_analysis();
            
// Model Deployment
Tensor<type, 2> inputs = data_set.get_data();
neural_network.calculate_outputs(inputs);
            
// Save results
opennn::NeuralNetwork::ProgrammingLanguage C;
opennn::NeuralNetwork::ProgrammingLanguage Python;
neural_network.save_expression(C, "../data/expression.txt");
neural_network.save_expression(Python, "../data/expression.py");

This code can be exported to your C++ project.

References:

  • Machine Learning Mastery Airline Passengers DataSet.
  • Fisher, R.A. «The use of multiple measurements in taxonomic problems» Annual Eugenics, 7, Part II, 179-188 (1936); also in «Contributions to Mathematical Statistics» (John Wiley, NY, 1950).