Estimate airline passengers using OpenNN

This example aims to predict the maximum number of passengers an airline will have, using past months’ data.

Contents:

  1. Application type.
  2. Data set.
  3. Neural network.
  4. Training strategy.
  5. Testing analysis.
  6. Model deployment.
  7. Full Code.

1. Application type

The variable to be predicted is discrete (number of passengers per month). Therefore, this is a forecasting project.

The primary goal here is to model the passengers in the coming months, conditioned on the past ones.

2. Data set

The first step is to prepare the data set, which is the source of information for the forecasting problem.

The data source is the file airline_passengers.csv. It contains the data for this example in comma-separated values (CSV) format and can be loaded as

DataSet data_set("path_to_source/airine_passengers.csv",',',true);

The number of columns is 2, and the number of rows is 144. The variables in this problem are:

  • months: Unused.
  • passengers: Number of passengers in each month, used as input.

We need to transform data to use the forecasting model. We set lags and steps ahead numbers and then transform data using:

const Index lags_number = 1;
const Index steps_ahead_number = 1;
data_set.set_lags_number(lags_number);
data_set.set_lags_number(steps_ahead_number);
data_set.transform_time_series();

To get the input and target variables numbers, we use the following command

const Index input_variables_number = data_set.get_input_variables_number();
const Index target_variables_number = data_set.get_target_variables_number();

For more information about the data set methods, see DataSet class.

3. Neural network

The second step is to choose the correct neural network architecture. For forecasting problems, it is usually composed by:

  • A scaling layer.
  • A long short-term memory layer.
  • A perceptron layer.
  • An unscaling layer.

The NeuralNetwork class is responsible for building the neural network and properly organizing the layers of neurons using the following constructor. If you need more complex architectures, you should see NeuralNetwork class.

NeuralNetwork neural_network(NeuralNetwork::Forecasting,
   {input_variables_number, hidden_neurons_number, target_variables_number});

Therefore, we have already created a good-looking model. Thus we proceed to the learning process with TrainingStrategy.

4. Training strategy

The third step is to set the training strategy, which is composed of:

  • Loss index.
  • Optimization algorithm.

Firstly, we construct the training strategy object

TrainingStrategy training_strategy(&neural_network, &data_set);

then, set the error term

training_strategy.set_loss_method(TrainingStrategy::MEAN_SQUARED_ERROR);

and finally the optimization algorithm

training_strategy.set_optimization_method(TrainingStrategy::ADAPTIVE_MOMENT_ESTIMATION);

OpenNN builds by default the training strategy object using the quasi-Newton method as the optimization algorithm and normalized squared error as the loss method. We can now start the training process by using the command

training_strategy.perform_training();

If we need to go further, OpenNN allows control of the optimization, for example

AdaptiveMomentEstimation* adam = training_strategy.get_adaptive_moment_estimation_pointer();
adam->set_loss_goal(type(1.0e-3));
adam->set_maximum_epochs_number(10000);
adam->set_display_period(1000);

For more information about the training strategy methods, see TrainingStrategy class.

5. Testing analysis

The fourth step is to evaluate our model. For that purpose, we need to use the testing analysis class, whose goal is to validate the model’s generalization performance. Here, we compare the neural network outputs to the corresponding targets in the testing instances of the data set.
As in the previous cases, we start by building the testing analysis object

TestingAnalysis testing_analysis(&neural_network, &data_set);

and perform the testing, in our case we use linear regression

const Tensor<TestingAnalysis::LinearRegressionAnalysis, 1> linear_regression_analysis
   = testing_analysis.perform_linear_regression_analysis();
linear_regression_analysis(0).print();

For more information about the testing analysis methods, see TestingAnalysis class.

6. Model deployment

Once our model is completed, the neural network is ready to predict outputs for incoming months. This process is called model deployment.

To generate predictions with new data, you can use

neural_network.calculate_outputs();

For instance, the inputs can be calculated by:

Tensor<type, 2> inputs = data_set.get_data();

and for outputs, we can write

neural_network.calculate_outputs(inputs);

or save the model.

neural_network.save_expression_c("../data/expression.txt");
neural_network.save_expression_python("../data/expression.txt");

The model can be implemented in python, php, … .

7. Full Code

Joining all steps, we obtain the following code:

// Data Set
DataSet data_set("path_to_source/airine_passengers.csv",',',true);
const Index lags_number = 1;
const Index steps_ahead_number = 1;
data_set.set_lags_number(lags_number);
data_set.set_lags_number(steps_ahead_number);
data_set.transform_time_series();
const Index input_variables_number = data_set.get_input_variables_number();
const Index target_variables_number = data_set.get_target_variables_number();
            
// Neural Network
const Index hidden_neurons_number = 3;
NeuralNetwork neural_network(NeuralNetwork::Forecasting,
     {input_variables_number, hidden_neurons_number, target_variables_number});
            
// Training Strategy
TrainingStrategy training_strategy(&neural_network, &data_set);
training_strategy.set_loss_method(TrainingStrategy::MEAN_SQUARED_ERROR);
training_strategy.set_optimization_method(TrainingStrategy::ADAPTIVE_MOMENT_ESTIMATION);
            
// Testing Analysis
TestingAnalysis testing_analysis(&neural_network, &data_set);
const Tensor<TestingAnalysis::LinearRegressionAnalysis, 1> linear_regression_analysis
   = testing_analysis.perform_linear_regression_analysis();
linear_regression_analysis(0).print();
            
// Model Deployment
Tensor<type, 2> inputs = data_set.get_data();
neural_network.calculate_outputs(inputs);
            
// Save results
neural_network.save_expression_c("../data/expression.txt");
neural_network.save_expression_python("../data/expression.txt");

This code can be exported to your C++ project.

References:

  • Machine Learning Mastery Airline Passengers DataSet.
  • Fisher,R.A. «The use of multiple measurements in taxonomic problems» Annual Eugenics, 7, Part II, 179-188 (1936); also in «Contributions to Mathematical Statistics» (John Wiley, NY, 1950).