LossIndex Class Referenceabstract

This abstract class represents the concept of loss index composed of an error term and a regularization term. More...

#include <loss_index.h>

Inheritance diagram for LossIndex:
CrossEntropyError MeanSquaredError MinkowskiError NormalizedSquaredError SumSquaredError WeightedSquaredError

Public Types

enum class  RegularizationMethod { L1 , L2 , NoRegularization }
 Enumeration of available regularization methods. More...
 

Public Member Functions

 LossIndex ()
 
 LossIndex (NeuralNetwork *, DataSet *)
 
virtual ~LossIndex ()
 Destructor. More...
 
NeuralNetworkget_neural_network_pointer () const
 Returns a pointer to the neural network object associated to the error term. More...
 
DataSetget_data_set_pointer () const
 Returns a pointer to the data set object associated to the error term. More...
 
const type & get_regularization_weight () const
 Returns regularization weight. More...
 
const bool & get_display () const
 
bool has_neural_network () const
 
bool has_data_set () const
 
RegularizationMethod get_regularization_method () const
 Returns the regularization method. More...
 
void set ()
 
void set (NeuralNetwork *)
 
void set (DataSet *)
 
void set (NeuralNetwork *, DataSet *)
 
void set (const LossIndex &)
 
void set_threads_number (const int &)
 
void set_neural_network_pointer (NeuralNetwork *)
 
virtual void set_data_set_pointer (DataSet *)
 Sets a new data set on which the error term is to be measured. More...
 
void set_default ()
 Sets the members of the error term to their default values: More...
 
void set_regularization_method (const RegularizationMethod &)
 
void set_regularization_method (const string &)
 
void set_regularization_weight (const type &)
 
void set_display (const bool &)
 
virtual void set_normalization_coefficient ()
 
bool has_selection () const
 Returns true if there are selection samples and false otherwise. More...
 
type calculate_eta () const
 
type calculate_h (const type &) const
 
Tensor< type, 1 > calculate_gradient_numerical_differentiation ()
 
Tensor< type, 2 > calculate_jacobian_numerical_differentiation ()
 
void calculate_errors (const DataSetBatch &, const NeuralNetworkForwardPropagation &, LossIndexBackPropagation &) const
 
virtual void calculate_error (const DataSetBatch &, const NeuralNetworkForwardPropagation &, LossIndexBackPropagation &) const =0
 
virtual void calculate_output_delta (const DataSetBatch &, NeuralNetworkForwardPropagation &, LossIndexBackPropagation &) const =0
 
void calculate_layers_delta (const DataSetBatch &, NeuralNetworkForwardPropagation &, LossIndexBackPropagation &) const
 
void calculate_error_gradient (const DataSetBatch &, const NeuralNetworkForwardPropagation &, LossIndexBackPropagation &) const
 
void back_propagate (const DataSetBatch &, NeuralNetworkForwardPropagation &, LossIndexBackPropagation &) const
 
void calculate_errors_lm (const DataSetBatch &, const NeuralNetworkForwardPropagation &, LossIndexBackPropagationLM &) const
 
virtual void calculate_squared_errors_lm (const DataSetBatch &, const NeuralNetworkForwardPropagation &, LossIndexBackPropagationLM &) const
 
virtual void calculate_error_lm (const DataSetBatch &, const NeuralNetworkForwardPropagation &, LossIndexBackPropagationLM &) const
 
virtual void calculate_output_delta_lm (const DataSetBatch &, NeuralNetworkForwardPropagation &, LossIndexBackPropagationLM &) const
 
void calculate_layers_delta_lm (const DataSetBatch &, NeuralNetworkForwardPropagation &, LossIndexBackPropagationLM &) const
 
virtual void calculate_error_gradient_lm (const DataSetBatch &, LossIndexBackPropagationLM &) const
 
void calculate_squared_errors_jacobian_lm (const DataSetBatch &, NeuralNetworkForwardPropagation &, LossIndexBackPropagationLM &) const
 
virtual void calculate_error_hessian_lm (const DataSetBatch &, LossIndexBackPropagationLM &) const
 
void back_propagate_lm (const DataSetBatch &, NeuralNetworkForwardPropagation &, LossIndexBackPropagationLM &) const
 
type calculate_regularization (const Tensor< type, 1 > &) const
 
void calculate_regularization_gradient (const Tensor< type, 1 > &, Tensor< type, 1 > &) const
 
void calculate_regularization_hessian (const Tensor< type, 1 > &, Tensor< type, 2 > &) const
 
void from_XML (const tinyxml2::XMLDocument &)
 
virtual void write_XML (tinyxml2::XMLPrinter &) const
 
void regularization_from_XML (const tinyxml2::XMLDocument &)
 
void write_regularization_XML (tinyxml2::XMLPrinter &) const
 
virtual string get_error_type () const
 Returns a string with the default type of error term, "USER_PERFORMANCE_TERM". More...
 
virtual string get_error_type_text () const
 Returns a string with the default type of error term in text format, "USER_PERFORMANCE_TERM". More...
 
string write_regularization_method () const
 
void check () const
 

Protected Attributes

ThreadPool * thread_pool = nullptr
 
ThreadPoolDevice * thread_pool_device = nullptr
 
NeuralNetworkneural_network_pointer = nullptr
 Pointer to a neural network object. More...
 
DataSetdata_set_pointer = nullptr
 Pointer to a data set object. More...
 
RegularizationMethod regularization_method = RegularizationMethod::L2
 Pointer to a regularization method object. More...
 
type regularization_weight = static_cast<type>(0.01)
 Regularization weight value. More...
 
bool display = true
 Display messages to screen. More...
 
const Eigen::array< IndexPair< Index >, 1 > AT_B = {IndexPair<Index>(0, 0)}
 
const Eigen::array< IndexPair< Index >, 1 > A_B = {IndexPair<Index>(1, 0)}
 
const Eigen::array< IndexPair< Index >, 2 > SSE = {IndexPair<Index>(0, 0), IndexPair<Index>(1, 1)}
 
const Eigen::array< int, 1 > rows_sum = {Eigen::array<int, 1>({1})}
 

Detailed Description

This abstract class represents the concept of loss index composed of an error term and a regularization term.

The error term could be:

  • Cross Entropy Error.
  • Mean Squared Error.
  • Minkowski Error.
  • Normalized Squared Error.
  • Sum Squared Error.
  • Weighted Squared Error.

Definition at line 48 of file loss_index.h.

Member Enumeration Documentation

◆ RegularizationMethod

enum class RegularizationMethod
strong

Enumeration of available regularization methods.

Definition at line 67 of file loss_index.h.

Constructor & Destructor Documentation

◆ LossIndex() [1/2]

LossIndex ( )
explicit

Default constructor. It creates a default error term object, with all pointers initialized to nullptr. It also initializes all the rest of class members to their default values.

Definition at line 18 of file loss_index.cpp.

◆ LossIndex() [2/2]

LossIndex ( NeuralNetwork new_neural_network_pointer,
DataSet new_data_set_pointer 
)
explicit

Neural network and data set constructor. It creates a error term object associated to a neural network and to be measured on a data set. The rest of pointers are initialized to nullptr. It also initializes all the rest of class members to their default values.

Parameters
new_neural_network_pointerPointer to a neural network object.
new_data_set_pointerPointer to a data set object.

Definition at line 33 of file loss_index.cpp.

◆ ~LossIndex()

~LossIndex ( )
virtual

Destructor.

Definition at line 43 of file loss_index.cpp.

Member Function Documentation

◆ back_propagate()

void back_propagate ( const DataSetBatch batch,
NeuralNetworkForwardPropagation forward_propagation,
LossIndexBackPropagation back_propagation 
) const

Definition at line 419 of file loss_index.cpp.

◆ back_propagate_lm()

void back_propagate_lm ( const DataSetBatch batch,
NeuralNetworkForwardPropagation forward_propagation,
LossIndexBackPropagationLM loss_index_back_propagation_lm 
) const

This method calculates the second order loss. It is used for optimization of parameters during training. Returns a second order terms loss structure, which contains the values and the Hessian of the error terms function.

Todo:
Update method.

Definition at line 457 of file loss_index.cpp.

◆ calculate_error()

virtual void calculate_error ( const DataSetBatch ,
const NeuralNetworkForwardPropagation ,
LossIndexBackPropagation  
) const
pure virtual

◆ calculate_error_gradient()

void calculate_error_gradient ( const DataSetBatch batch,
const NeuralNetworkForwardPropagation forward_propagation,
LossIndexBackPropagation back_propagation 
) const

Definition at line 760 of file loss_index.cpp.

◆ calculate_error_gradient_lm()

void calculate_error_gradient_lm ( const DataSetBatch batch,
LossIndexBackPropagationLM loss_index_back_propagation_lm 
) const
virtual

Definition at line 598 of file loss_index.cpp.

◆ calculate_error_hessian_lm()

virtual void calculate_error_hessian_lm ( const DataSetBatch ,
LossIndexBackPropagationLM  
) const
inlinevirtual

Definition at line 216 of file loss_index.h.

◆ calculate_error_lm()

virtual void calculate_error_lm ( const DataSetBatch ,
const NeuralNetworkForwardPropagation ,
LossIndexBackPropagationLM  
) const
inlinevirtual

Definition at line 197 of file loss_index.h.

◆ calculate_errors()

void calculate_errors ( const DataSetBatch batch,
const NeuralNetworkForwardPropagation forward_propagation,
LossIndexBackPropagation back_propagation 
) const

Definition at line 321 of file loss_index.cpp.

◆ calculate_errors_lm()

void calculate_errors_lm ( const DataSetBatch batch,
const NeuralNetworkForwardPropagation neural_network_forward_propagation,
LossIndexBackPropagationLM loss_index_back_propagation 
) const

Definition at line 366 of file loss_index.cpp.

◆ calculate_eta()

type calculate_eta ( ) const

Definition at line 1146 of file loss_index.cpp.

◆ calculate_gradient_numerical_differentiation()

Tensor< type, 1 > calculate_gradient_numerical_differentiation ( )

Definition at line 1019 of file loss_index.cpp.

◆ calculate_h()

type calculate_h ( const type &  x) const

Calculates a proper step size for computing the derivatives, as a function of the inputs point value.

Parameters
xInput value.

Definition at line 1157 of file loss_index.cpp.

◆ calculate_jacobian_numerical_differentiation()

Tensor< type, 2 > calculate_jacobian_numerical_differentiation ( )

Definition at line 1079 of file loss_index.cpp.

◆ calculate_layers_delta()

void calculate_layers_delta ( const DataSetBatch batch,
NeuralNetworkForwardPropagation forward_propagation,
LossIndexBackPropagation back_propagation 
) const

Definition at line 703 of file loss_index.cpp.

◆ calculate_layers_delta_lm()

void calculate_layers_delta_lm ( const DataSetBatch batch,
NeuralNetworkForwardPropagation forward_propagation,
LossIndexBackPropagationLM back_propagation 
) const

Definition at line 732 of file loss_index.cpp.

◆ calculate_output_delta_lm()

virtual void calculate_output_delta_lm ( const DataSetBatch ,
NeuralNetworkForwardPropagation ,
LossIndexBackPropagationLM  
) const
inlinevirtual

Definition at line 201 of file loss_index.h.

◆ calculate_regularization()

type calculate_regularization ( const Tensor< type, 1 > &  parameters) const

It calculates the regularization term using through the use of parameters. Returns the regularization evaluation, according to the respective regularization type used in the loss index expression.

Parameters
parametersvector with the parameters to get the regularization term.

Definition at line 648 of file loss_index.cpp.

◆ calculate_regularization_gradient()

void calculate_regularization_gradient ( const Tensor< type, 1 > &  parameters,
Tensor< type, 1 > &  regularization_gradient 
) const

It calculates the regularization term using the gradient method. Returns the gradient of the regularization, according to the regularization type. That gradient is the vector of partial derivatives of the regularization with respect to the parameters. The size is thus the number of parameters.

Parameters
parametersvector with the parameters to get the regularization term.
regularization_gradientGradient of the regularization.

Definition at line 670 of file loss_index.cpp.

◆ calculate_regularization_hessian()

void calculate_regularization_hessian ( const Tensor< type, 1 > &  parameters,
Tensor< type, 2 > &  regularization_hessian 
) const

It calculate the regularization term using the Hessian. Returns the Hessian of the regularization, according to the regularization type. That Hessian is the matrix of second partial derivatives of the regularization with respect to the parameters. That matrix is symmetric, with size the number of parameters.

Parameters
parametersvector with the parameters to get the regularization term.
regularization_hessianHessian of the regularization.

Definition at line 690 of file loss_index.cpp.

◆ calculate_squared_errors_jacobian_lm()

void calculate_squared_errors_jacobian_lm ( const DataSetBatch batch,
NeuralNetworkForwardPropagation forward_propagation,
LossIndexBackPropagationLM loss_index_back_propagation_lm 
) const

Calculates the Jacobian matrix of the error terms from layers. Returns the Jacobian of the error terms function, according to the objective type used in the loss index expression. Note that this function is only defined when the objective can be expressed as a sum of squared terms. The Jacobian elements are the partial derivatives of a single term with respect to a single parameter. The number of rows in the Jacobian matrix are the number of parameters, and the number of columns the number of terms composing the objective.

Parameters
batchDataSetBatch of data set that contains the inputs and targets to be trained.
forward_propagationNeuralNetwork class structure that saves the necessary parameters for forward propagation.
loss_index_back_propagation_lmLoss index class structure that saves the necessary parameters for back propagation.

Definition at line 508 of file loss_index.cpp.

◆ calculate_squared_errors_lm()

void calculate_squared_errors_lm ( const DataSetBatch ,
const NeuralNetworkForwardPropagation ,
LossIndexBackPropagationLM loss_index_back_propagation_lm 
) const
virtual

Definition at line 411 of file loss_index.cpp.

◆ check()

void check ( ) const

Checks that there is a neural network associated to the error term. If some of the above conditions is not hold, the method throws an exception.

Definition at line 295 of file loss_index.cpp.

◆ from_XML()

void from_XML ( const tinyxml2::XMLDocument document)

Loads a default error term from a XML document.

Parameters
documentTinyXML document containing the error term members.

Definition at line 982 of file loss_index.cpp.

◆ get_data_set_pointer()

DataSet * get_data_set_pointer ( ) const
inline

Returns a pointer to the data set object associated to the error term.

Definition at line 93 of file loss_index.h.

◆ get_display()

const bool & get_display ( ) const

Returns true if messages from this class can be displayed on the screen, or false if messages from this class can't be displayed on the screen.

Definition at line 61 of file loss_index.cpp.

◆ get_error_type()

string get_error_type ( ) const
virtual

Returns a string with the default type of error term, "USER_PERFORMANCE_TERM".

Reimplemented in CrossEntropyError, MeanSquaredError, MinkowskiError, NormalizedSquaredError, SumSquaredError, and WeightedSquaredError.

Definition at line 608 of file loss_index.cpp.

◆ get_error_type_text()

string get_error_type_text ( ) const
virtual

Returns a string with the default type of error term in text format, "USER_PERFORMANCE_TERM".

Reimplemented in CrossEntropyError, MeanSquaredError, MinkowskiError, NormalizedSquaredError, SumSquaredError, and WeightedSquaredError.

Definition at line 616 of file loss_index.cpp.

◆ get_neural_network_pointer()

NeuralNetwork * get_neural_network_pointer ( ) const
inline

Returns a pointer to the neural network object associated to the error term.

Definition at line 71 of file loss_index.h.

◆ get_regularization_method()

LossIndex::RegularizationMethod get_regularization_method ( ) const

Returns the regularization method.

Definition at line 101 of file loss_index.cpp.

◆ get_regularization_weight()

const type & get_regularization_weight ( ) const

Returns regularization weight.

Definition at line 52 of file loss_index.cpp.

◆ has_data_set()

bool has_data_set ( ) const

Returns true if this error term object has a data set pointer associated, and false otherwise.

Definition at line 86 of file loss_index.cpp.

◆ has_neural_network()

bool has_neural_network ( ) const

Returns true if this error term object has a neural nework class pointer associated, and false otherwise

Definition at line 70 of file loss_index.cpp.

◆ has_selection()

bool has_selection ( ) const

Returns true if there are selection samples and false otherwise.

Definition at line 279 of file loss_index.cpp.

◆ regularization_from_XML()

void regularization_from_XML ( const tinyxml2::XMLDocument document)

Definition at line 894 of file loss_index.cpp.

◆ set() [1/5]

void set ( )

Sets all the member pointers to nullptr(neural network, data set). It also initializes all the rest of class members to their default values.

Definition at line 110 of file loss_index.cpp.

◆ set() [2/5]

void set ( const LossIndex other_error_term)

Sets to this error term object the members of another error term object.

Parameters
other_error_termError term to be copied.

Definition at line 163 of file loss_index.cpp.

◆ set() [3/5]

void set ( DataSet new_data_set_pointer)

Sets all the member pointers to nullptr, but the data set, which set to a given pointer. It also initializes all the rest of class members to their default values.

Parameters
new_data_set_pointerPointer to a data set object.

Definition at line 136 of file loss_index.cpp.

◆ set() [4/5]

void set ( NeuralNetwork new_neural_network_pointer)

Sets all the member pointers to nullptr, but the neural network, which set to a given pointer. It also initializes all the rest of class members to their default values.

Parameters
new_neural_network_pointerPointer to a neural network object.

Definition at line 123 of file loss_index.cpp.

◆ set() [5/5]

void set ( NeuralNetwork new_neural_network_pointer,
DataSet new_data_set_pointer 
)

Sets new neural network and data set pointers. Finally, it initializes all the rest of class members to their default values.

Parameters
new_neural_network_pointerPointer to a neural network object.
new_data_set_pointerPointer to a data set object.

Definition at line 150 of file loss_index.cpp.

◆ set_data_set_pointer()

void set_data_set_pointer ( DataSet new_data_set_pointer)
virtual

Sets a new data set on which the error term is to be measured.

Reimplemented in WeightedSquaredError, and NormalizedSquaredError.

Definition at line 196 of file loss_index.cpp.

◆ set_default()

void set_default ( )

Sets the members of the error term to their default values:

Definition at line 204 of file loss_index.cpp.

◆ set_display()

void set_display ( const bool &  new_display)

Sets a new display value. If it is set to true messages from this class are to be displayed on the screen; if it is set to false messages from this class are not to be displayed on the screen.

Parameters
new_displayDisplay value.

Definition at line 271 of file loss_index.cpp.

◆ set_neural_network_pointer()

void set_neural_network_pointer ( NeuralNetwork new_neural_network_pointer)

Sets a pointer to a neural network object which is to be associated to the error term.

Parameters
new_neural_network_pointerPointer to a neural network object to be associated to the error term.

Definition at line 188 of file loss_index.cpp.

◆ set_normalization_coefficient()

virtual void set_normalization_coefficient ( )
inlinevirtual

Reimplemented in NormalizedSquaredError, and WeightedSquaredError.

Definition at line 148 of file loss_index.h.

◆ set_regularization_method() [1/2]

void set_regularization_method ( const RegularizationMethod new_regularization_method)

Sets the object with the regularization method.

Parameters
new_regularization_methodString with method.

Definition at line 251 of file loss_index.cpp.

◆ set_regularization_method() [2/2]

void set_regularization_method ( const string &  new_regularization_method)

Sets the object with the regularization method.

Parameters
new_regularization_methodString with method.

Definition at line 221 of file loss_index.cpp.

◆ set_regularization_weight()

void set_regularization_weight ( const type &  new_regularization_weight)

Sets the object with the regularization weights.

Parameters
new_regularization_weightNew regularization weight.

Definition at line 260 of file loss_index.cpp.

◆ set_threads_number()

void set_threads_number ( const int &  new_threads_number)

Definition at line 175 of file loss_index.cpp.

◆ write_regularization_method()

string write_regularization_method ( ) const

Returns a string with the regularization information of the error term. It will be used by the training strategy to monitor the training process.

Definition at line 625 of file loss_index.cpp.

◆ write_regularization_XML()

void write_regularization_XML ( tinyxml2::XMLPrinter file_stream) const

Definition at line 931 of file loss_index.cpp.

◆ write_XML()

void write_XML ( tinyxml2::XMLPrinter file_stream) const
virtual

Serializes a default error term object into a XML document of the TinyXML library without keep the DOM tree in memory. See the OpenNN manual for more information about the format of this document.

Reimplemented in CrossEntropyError, MeanSquaredError, MinkowskiError, NormalizedSquaredError, SumSquaredError, and WeightedSquaredError.

Definition at line 884 of file loss_index.cpp.

Member Data Documentation

◆ A_B

const Eigen::array<IndexPair<Index>, 1> A_B = {IndexPair<Index>(1, 0)}
protected

Definition at line 274 of file loss_index.h.

◆ AT_B

const Eigen::array<IndexPair<Index>, 1> AT_B = {IndexPair<Index>(0, 0)}
protected

Definition at line 273 of file loss_index.h.

◆ data_set_pointer

DataSet* data_set_pointer = nullptr
protected

Pointer to a data set object.

Definition at line 259 of file loss_index.h.

◆ display

bool display = true
protected

Display messages to screen.

Definition at line 271 of file loss_index.h.

◆ neural_network_pointer

NeuralNetwork* neural_network_pointer = nullptr
protected

Pointer to a neural network object.

Definition at line 255 of file loss_index.h.

◆ regularization_method

RegularizationMethod regularization_method = RegularizationMethod::L2
protected

Pointer to a regularization method object.

Definition at line 263 of file loss_index.h.

◆ regularization_weight

type regularization_weight = static_cast<type>(0.01)
protected

Regularization weight value.

Definition at line 267 of file loss_index.h.

◆ rows_sum

const Eigen::array<int, 1> rows_sum = {Eigen::array<int, 1>({1})}
protected

Definition at line 278 of file loss_index.h.

◆ SSE

const Eigen::array<IndexPair<Index>, 2> SSE = {IndexPair<Index>(0, 0), IndexPair<Index>(1, 1)}
protected

Definition at line 276 of file loss_index.h.

◆ thread_pool

ThreadPool* thread_pool = nullptr
protected

Definition at line 250 of file loss_index.h.

◆ thread_pool_device

ThreadPoolDevice* thread_pool_device = nullptr
protected

Definition at line 251 of file loss_index.h.


The documentation for this class was generated from the following files: