Vectors, matrix and tensors are the basic elements that store and allows OpennNN to store and manage all the information that will be later analyzed by the neural network.

In this tutorial, we will learn about the vector, matrix and tensor templates and how OpenNN allows you to easily work with them.

Contents:

The Vector class is a template, which means that it can be applied to different types. That is, we can create a Vector or int numbers, MyClass objects, etc.

The Vector in OpenNN is derived from the vector in the Standard Template Library.

The OpenNN Vector derives from the std vector and it does not include any additional member.

Multiple constructors are defined in the Vector class, where the different constructors take different parameters.

The easiest way of creating a vector object is by means of the default constructor, wich builds a vector of size zero. For example, in order to construct an empty Vector of int numbers we use the following.

Vector<int> v;

The following sentence constructs a Vector of 3 double numbers.

Vector<double> v(3);

If we want to construct Vector of 5 bool variables and initialize all the elements to false, we can use

Vector<bool> v(5, false);

It is also possible to construct an object of the Vector class and at the same time load its members from a file. In order to do that we can do

Vector<int> v("Vector.dat");

The file `Vector.dat' contains a first row with the size of the vector and an aditional row for each element of the vector.

The following sentence constructs a Vector which is a copy of another Vector,

Vector<MyClass> v(3);

Vector<MyClass> w(v);

A very useful way to create a vector object is to through a list:

Vector<int> v({1, 2, 3});

Vector<string> w({"one","two","three"});

A very useful way to create a vector object is to through a list:

Vector<int> v({1, 2, 3});

Vector<string> w({"one","two","three"});

The Vector class also implements different types of operators for assignment, reference, arithmetic or comparison.

The assignment operator copies a vector into another vector,

Vector<int> v;

Vector<int> w = v;

The following sentence constructs a vector and sets the values of their elements using the reference operator. Note that indexing goes from 0 to n-1, where n is the Vector size.

Vector<double> v(3);

v[0] = 1.0;

v[1] = 2.0;

v[2] = 3.0;

Sum, difference, product and quotient operators are included in the Vector class to perform arithmetic operations with a scalar or another Vector. Note that the arithmetic operators with another Vector require that they have the same sizes.

The following sentence uses the vector-scalar sum operator, which adds a given value to all the elements in the vector

Vector<double> v(3, 1.0);

Vector<double> w = v + 3.1415926;

An example of the use of the vector-vector multiplication operator is given below. It realizes the element-wise multiplication between the two vectors.

Vector<double> v(3, 1.2);

Vector<double> w(3, 3.4);

Vector<double> x = v*w;

Assignment by sum, difference, product or quotient with a scalar or another Vector is also possible by using the arithmetic and assignment operators. If another Vector is to be used, it must have the same size.

For instance, to assign by difference with a scalar, we might do

Vector<int> v(3, 2);

v -= 1;

In order to assign by quotation with another Vector, we can write

Vector<double> v(3, 2.0);

Vector<double> w(3, 0.5);

v /= w;

Equality and relational operators are also implemented here. They can be used with a scalar or another Vector. For the last case the same sizes are assumed. An example of the equal and not equal to operators with a scalar is

Vector<bool> v(5, false);

bool is_equal = (v == false);

bool is_not_equal = (v != false);

The less, less or equal, greater and greatar or equal than operator with another Vector can be used as follows,

Vector<double> v(5, 2.3);

Vector<double> w(5, 3.2);

bool is_less = (v < w);

bool is_less_equal = (v <= w);

bool is_greater = (v > w);

bool is_greater_equal = (v >= w);

Get and set methods for each member of this class are implemented to exchange information among objects.

The method size() returns the size of a Vector.

Vector<int> v({1, 2, 3, 4, 5});

size_t size = v.size();

On the other hand, the method set() sets a new size to a Vector. Note that the element values of that Vector are lost.

Vector<bool> v(3);

v.set(6);

If we want to initialize a vector at random we can use the randomize_uniform() or randomize_normal() methods. For the first method, a minimum and a maximum value must be specified while for the second mehtod

Vector<double> v(5);

Vector<double> w(3);

v.randomize_uniform(-1,1);

w.randomize_normal(0,1);

To get the first or last elements of a vector the following methods are used:

Vector<int> v({1, 2, 3, 4, 5, 6, 7, 8, 9});

Vector<int> first_5 = v.get_first(5);

Vector<int> last_5 = v.get_last(5);

The Vector class also includes some mathematical methods which can be useful in the development of neural networks algorithms and applications.

The calculate_L1_norm() method calculates the norm of the vector, <

Vector<double> v(5, 3.1415927);

double norm = v.calculate_L1_norm();

In order to calculate the dot product between this Vector and another Vector we can do

Vector<double> v(3, 2.0);

Vector<double> w(3, 5.0);

double dot = v.dot(w);

We can calculate the mean or the standard deviation values of the elements in a Vector by using the calculate_mean() and calculate_standard_deviation() methods, respectively. For instance

Vector<double> v({1.2, 2.5, 3.6, 2.4});

double mean = v.calculate_mean();

double standard_deviation = v.calculate_standard_deviation();

Finally, utility methods for serialization or loading and saving the class members to a file are also included. In order to obtain a std::string representation of a Vector object we can make the following.

Vector<bool> v(1, false);

Vector<std::string> vector_string = v.to_string_vector();

To save a Vector object to a file we can do

Vector<int> v(2, 0);

v.save("Vector.dat", ';');

The save method only needs the path and the separator.

If we want to load a Vector object from a data file we could write

Vector<int> v;

v.load("Vector.dat");

As it happens with the Vector class, the Matrix class is also a template. Therefore, a Matrix of any type can be created.

The Matrix class has three members:

- The number of rows.
- The number of columns.
- The header.

That members are private. Private members can be accessed only within methods of the class itself.

The Matrix class also implements multiple constructors, with different parameters.

The default constructor creates a matrix of integers with zero rows and zero columns.

Matrix<int> m;

In order to construct an empty Matrix with a specified number of rows and columns we use

Matrix<int> m(2, 3);

We can specify the number of rows and columns and initialize the Matrix elements at the same time by doing

Matrix<double> m(1, 5, 0.0);

To build a Matrix object by loading its members from a data file the following constructor is used,

Matrix<double> m("Matrix.dat");

A matrix data file contains the matrix elements arranged in rows and columns. For instance, the next data will correspond to a Matrix of zeros with 2 rows and 3 columns.

0 0 0

0 0 0

The copy constructor builds an object which is a copy of another object.

Matrix<bool> a(3, 5);

Matrix<bool> b(a);

One of the most convenient ways to initialize a matrix object is through a Vector List:

Matrix<double> m({Vector<double>({1, 2, 3}),Vector<double>({1, 2, 3})},{"a","b"});

The first argument of m is a list of vectors that correspond to the columns of the matrix. The second argument concerns to the header.

The Matrix class also implements the assignment operator.

Matrix<double> a(2, 1);

Matrix<double> b = a;

Below there is an usage example of the reference operator here. Note that row indexing goes from 0 to rows_number-1 and column indexing goes from 0 to columns_number-1.

Matrix<int> m(2, 2);

m(0, 0) = 1;

m(1, 1) = 1;

m(0, 1) = 0;

m(1, 0) = 0;

The use of the arithmetic operators for the Matrix class are very similar to those for the Vector class. The following sentence uses the scalar difference operator,

Matrix<double> a(5, 7, 2.5);

Matrix<double> b = a + 0.1;

Also, using the arithmetic and assignment operators with the Matrix class is similar than with the Vector class. For instance, to assign by sum with another Matrix we can write

Matrix<double> a(1, 2, 1.0);

Matrix<double> b(1, 2, 0.5);

a += b;

The not equal to operator with another Matrix can be used in the following way,

Matrix<string> a(1, 1, "hello");

Matrix<string> b(1, 1, "good bye");

bool is_not_equal_to = (a != b);

The use of the greater than operator with a scalar is listed below

Matrix<double> a(2, 3, 0.0);

bool is_greater_than = (a < 1.0);

As it happens for the Vector class, the Matrix class implements get and set methods for all the members. The get_rows_number() and get_columns_number() methods are very useful.

Matrix<int> m(4, 2);

size_t rows_number = m.get_rows_number();

size_t columns_number = m.get_columns_number();

In order to set a new number of rows or columns to a Matrix object, the set_rows_number() or set_columns_number() methods are used.

size_t rows_number = m.get_rows_number();

size_t columns_number = m.get_columns_number();

A Matrix can be initialized with a given value, at random with an uniform distribution or at random with a normal distribution,

Matrix<double> m(4, 2);

m.initialize(0.0);

m.randomize_uniform(-0.2, 0.4);

m.randomize_normal(-1.0, 0.25);

A set of mathematical methods are also implemented for convenience. For instance, the dot() method computes the dot product of this Matrix with a Vector or with another Matrix.

Matrix<double> m(4, 2, 1.0);

Vector<double> v(4, 2.0);

Vector<double> dot_product = m.dot(v);

Finally, string serializing, printing, saving or loading utility methods are also implemented. For example, the use of the print() method or print_preview() is as follows.

Matrix<bool> m(1, 3, false);

m.print();

m.print_preview();

The Tensor class has one member:

- The dimensions.

This member is private, which means that it can only be accessed within methods of the class itself.

In the same way as for the rest of the classes, a Tensor can be constructed in several ways.

The default constructor creates a zero dimensions tensor and it can be called as follows

Tensor<double> t;

A tensor can also be construcuted with a vector that specifies the number of dimensions of the vector and their values, which are specified in a vector. We can also initialize the tensor to a value. An example of a four dimensions tensor is shown below

Vector<size_t> dimensions({4, 2});

Tensor<double> t(dimensions);

Tensor<double> t(dimensions, 5.0);

It is also possible to construct a tensor with 1, 2, 3 or 4 dimensions. The next tensor will have 3 dimensions whose values are 2, 3 and 6. If we wanted a tensor with other number of dimensions, we just have to specify their values in the constructor.

Tensor<double> t(2, 3, 6);

Finally, a tensor can be constructed by copying a matrix. This will generate a tensor of two dimensions

Matrix<double> m(3, 2, 1.0);

Tensor<double> t(m);

The assignment operator is also available for tensors. It copies a tensor into a new one

Vector<size_t> dimensions({4, 2, 6});

Tensor<double> t(dimensions, 5.0);

Tensor<double> s = t;

The sum, difference, product and quotient with a scalar work in in the same way as for the vector and matrix classes. The next sentence multiplies every element of a tensor by a number

Vector<size_t> dimensions({4, 2, 6});

Tensor<double> t(dimensions, 5.0);

t = t * 5.0;

Similarly, these arithmetic operators can be applied to another tensor. Note that, in this case, the two tensors must have the same size

Vector<size_t> dimensions({4, 2, 6});

Tensor<double> t(dimensions, 2.0);

Tensor<double> s(dimensions, 5.0);

Tensor<double> l = t + s;

Comparing two tensors or a tensor with a scalar is also possible with the equal to, greater or lower than operators

Vector<size_t> dimensions({4, 2, 6});

Tensor<double> t(dimensions, 5.0);

Tensor<double> s(dimensions, 5.0);

bool is_equal = (t == s);

The Tensor class contains several methods that allow to get and set the members. For instance, the number of dimensions of a tensor can be obtained as follows

Vector<size_t> dimensions({4, 2, 6});

size_t dimensions_number = t.get_dimensions_number();

It is also possible to get all the dimensions arranged in a vector as follows

Tensor<double> t(4, 2, 6);

Vector<size_t> dimensions = t.get_dimensions();

Once a tensor has been constructed, it can be initialized to a given value or randomly. For the second method, the initialization could follow a uniform distribution or a normal distribution. An example of each case is shown below for a four dimensions tensor

Tensor<double> t(4, 2, 6, 3);

t.initialize(2.0);

t.randomize_uniform(-1.0, 1.0);

t.randomize_normal(0.0, 1.0);