‹ Back to Tutorials

The model selection class

ModelSelection searches for a network size and an input subset that generalize well to unseen data. It repeatedly trains candidate models and compares their validation error.

The current facade uses growing-neurons and growing-inputs selection. The individual algorithm classes provide additional controls, cross-validation and genetic input selection.

Contents:

  1. Prepare validation data
  2. Create model selection
  3. Select the number of neurons
  4. Select input variables
  5. Use genetic input selection
  6. Use cross-validation
  7. Inspect the result
  8. Save the configuration

1. Prepare validation data

Model selection must compare candidates on data that are not used to fit their parameters. Create training, validation and testing subsets before running it:

dataset.split_samples(
    0.6f, 0.2f, 0.2f, true);

TrainingStrategy training_strategy(
    &neural_network,
    &dataset);

Training fits each candidate, validation drives selection, and testing remains untouched for the final evaluation. Ordered observations should use a sequential split instead of random shuffling.

2. Create model selection

The facade keeps a non-owning pointer to the training strategy:

ModelSelection model_selection(
    &training_strategy);

Its default neuron algorithm is GrowingNeurons and its default input algorithm is GrowingInputs. Use the facade for the standard workflow, or construct an algorithm directly when its search bounds must be changed.

3. Select the number of neurons

The standard call grows the first hidden dense layer, trains each candidate and installs the best architecture and parameters:

NeuronsSelectionResult result =
    model_selection.perform_neurons_selection();

cout << result.optimal_neurons_number
     << '\n';

For explicit limits, use GrowingNeurons directly:

GrowingNeurons selection(&training_strategy);

selection.set_minimum_neurons(2);
selection.set_maximum_neurons(32);
selection.set_neurons_increment(2);
selection.set_trials_number(3);
selection.set_warm_start(true);

NeuronsSelectionResult result =
    selection.perform_neurons_selection();

Multiple trials reduce sensitivity to random initialization. Warm starts reuse compatible parameters when the hidden layer grows.

4. Select input variables

Growing-inputs selection starts with the most useful variables and adds candidates while validation performance improves:

InputsSelectionResult result =
    model_selection.perform_input_selection();

To control its range directly:

GrowingInputs selection(&training_strategy);

selection.set_minimum_inputs_number(1);
selection.set_maximum_inputs_number(12);
selection.set_trials_number(3);
selection.set_warm_start(true);

InputsSelectionResult result =
    selection.perform_input_selection();

The selected input roles, network input variables and compatible parameters are installed together, so the data set and network remain synchronized.

5. Use genetic input selection

GeneticAlgorithm explores combinations that a greedy growing search can miss. It is useful when interactions between inputs are important:

GeneticAlgorithm selection(&training_strategy);

selection.set_minimum_inputs_number(2);
selection.set_maximum_inputs_number(20);
selection.set_individuals_number(40);
selection.set_mutation_rate(0.05f);
selection.set_elitism_size(4);
selection.set_maximum_epochs(50);

InputsSelectionResult result =
    selection.perform_input_selection();

A larger population explores more subsets but requires more training runs. Keep the testing subset outside this search.

6. Use cross-validation

All selection algorithms inherit common controls for trials, folds and stopping criteria:

selection.set_folds_number(5);
selection.set_validation_error_goal(1.0e-3f);
selection.set_maximum_validation_failures(10);
selection.set_maximum_time(3600.0f);

With more than one fold, candidates are trained and evaluated across fold partitions, their validation errors are aggregated, and the winning structure is refitted on the development data. Use one fold when an explicit validation subset is already representative.

7. Inspect the result

Neuron-selection results contain the tested neuron counts, training and validation histories, optimal parameters, stopping condition and elapsed time. Input-selection results additionally expose the chosen variable names and indices:

result.print();

for(const string& name :
    result.optimal_input_variable_names)
{
    cout << name << '\n';
}

After selection, train the installed model once more if project policy requires a fresh final fit, then evaluate it only on the testing subset.

8. Save the configuration

The facade and the individual algorithms support JSON persistence:

model_selection.save("model_selection.json");

ModelSelection restored_selection(
    &training_strategy);
restored_selection.load("model_selection.json");

Save the data set and neural network separately to preserve the selected roles, architecture and parameters.

References

Continue with the testing analysis class tutorial.