Vector, Matrix and Tensor templates

Vectors, matrix and tensors are the basic elements that store and allows OpennNN to store and manage all the information that will be later analyzed by the neural network.

In this tutorial, we will learn about the vector, matrix and tensor templates and how OpenNN allows you to easily work with them.

Contents:

1. The Vector template

The Vector class is a template, which means that it can be applied to different types. That is, we can create a Vector or int numbers, MyClass objects, etc.

The Vector in OpenNN is derived from the vector in the Standard Template Library.

Members

The OpenNN Vector derives from the std vector and it does not include any additional member.

Constructors

Multiple constructors are defined in the Vector class, where the different constructors take different parameters.

The easiest way of creating a vector object is by means of the default constructor, wich builds a vector of size zero. For example, in order to construct an empty Vector of int numbers we use the following.

`Vector<int> v;`

The following sentence constructs a Vector of 3 double numbers.

`Vector<double> v(3);`

If we want to construct Vector of 5 bool variables and initialize all the elements to false, we can use

`Vector<bool> v(5, false);`

It is also possible to construct an object of the Vector class and at the same time load its members from a file. In order to do that we can do

`Vector<int> v("Vector.dat");`

The file `Vector.dat' contains a first row with the size of the vector and an aditional row for each element of the vector.

The following sentence constructs a Vector which is a copy of another Vector,

`Vector<MyClass> v(3);Vector<MyClass> w(v);`

A very useful way to create a vector object is to through a list:

`Vector<int> v({1, 2, 3});Vector<string> w({"one","two","three"});`

A very useful way to create a vector object is to through a list:

`Vector<int> v({1, 2, 3});Vector<string> w({"one","two","three"});`

Operators

The Vector class also implements different types of operators for assignment, reference, arithmetic or comparison.

The assignment operator copies a vector into another vector,

`Vector<int> v;Vector<int> w = v;`

The following sentence constructs a vector and sets the values of their elements using the reference operator. Note that indexing goes from 0 to n-1, where n is the Vector size.

`Vector<double> v(3);v[0] = 1.0;v[1] = 2.0;v[2] = 3.0;`

Sum, difference, product and quotient operators are included in the Vector class to perform arithmetic operations with a scalar or another Vector. Note that the arithmetic operators with another Vector require that they have the same sizes.

The following sentence uses the vector-scalar sum operator, which adds a given value to all the elements in the vector

`Vector<double> v(3, 1.0);Vector<double> w = v + 3.1415926;`

An example of the use of the vector-vector multiplication operator is given below. It realizes the element-wise multiplication between the two vectors.

`Vector<double> v(3, 1.2);Vector<double> w(3, 3.4);Vector<double> x = v*w;`

Assignment by sum, difference, product or quotient with a scalar or another Vector is also possible by using the arithmetic and assignment operators. If another Vector is to be used, it must have the same size.

For instance, to assign by difference with a scalar, we might do

`Vector<int> v(3, 2);v -= 1;`

In order to assign by quotation with another Vector, we can write

`Vector<double> v(3, 2.0);Vector<double> w(3, 0.5);v /= w;`

Equality and relational operators are also implemented here. They can be used with a scalar or another Vector. For the last case the same sizes are assumed. An example of the equal and not equal to operators with a scalar is

`Vector<bool> v(5, false);bool is_equal = (v == false);bool is_not_equal = (v != false);`

The less, less or equal, greater and greatar or equal than operator with another Vector can be used as follows,

`Vector<double> v(5, 2.3);Vector<double> w(5, 3.2);bool is_less = (v < w);bool is_less_equal = (v <= w);bool is_greater = (v > w);bool is_greater_equal = (v >= w);`

Methods

Get and set methods for each member of this class are implemented to exchange information among objects.

The method size() returns the size of a Vector.

`Vector<int> v({1, 2, 3, 4, 5});size_t size = v.size();`

On the other hand, the method set() sets a new size to a Vector. Note that the element values of that Vector are lost.

`Vector<bool> v(3);v.set(6);`

If we want to initialize a vector at random we can use the randomize_uniform() or randomize_normal() methods. For the first method, a minimum and a maximum value must be specified while for the second mehtod

`Vector<double> v(5);Vector<double> w(3);v.randomize_uniform(-1,1);w.randomize_normal(0,1);`

To get the first or last elements of a vector the following methods are used:

`Vector<int> v({1, 2, 3, 4, 5, 6, 7, 8, 9});Vector<int> first_5 = v.get_first(5);Vector<int> last_5 = v.get_last(5);`

The Vector class also includes some mathematical methods which can be useful in the development of neural networks algorithms and applications.

The calculate_L1_norm() method calculates the norm of the vector, <

`Vector<double> v(5, 3.1415927);double norm = v.calculate_L1_norm();`

In order to calculate the dot product between this Vector and another Vector we can do

`Vector<double> v(3, 2.0);Vector<double> w(3, 5.0);double dot = v.dot(w);`

We can calculate the mean or the standard deviation values of the elements in a Vector by using the calculate_mean() and calculate_standard_deviation() methods, respectively. For instance

`Vector<double> v({1.2, 2.5, 3.6, 2.4});double mean = v.calculate_mean();double standard_deviation = v.calculate_standard_deviation();`

Finally, utility methods for serialization or loading and saving the class members to a file are also included. In order to obtain a std::string representation of a Vector object we can make the following.

`Vector<bool> v(1, false);Vector<std::string> vector_string = v.to_string_vector();`

To save a Vector object to a file we can do

`Vector<int> v(2, 0);v.save("Vector.dat", ';');`

The save method only needs the path and the separator.

If we want to load a Vector object from a data file we could write

`Vector<int> v;v.load("Vector.dat");`

2. The Matrix template

As it happens with the Vector class, the Matrix class is also a template. Therefore, a Matrix of any type can be created.

Members

The Matrix class has three members:

• The number of rows.
• The number of columns.

That members are private. Private members can be accessed only within methods of the class itself.

Constructors

The Matrix class also implements multiple constructors, with different parameters.

The default constructor creates a matrix of integers with zero rows and zero columns.

`Matrix<int> m;`

In order to construct an empty Matrix with a specified number of rows and columns we use

`Matrix<int> m(2, 3);`

We can specify the number of rows and columns and initialize the Matrix elements at the same time by doing

```Matrix<double> m(1, 5, 0.0);
```

To build a Matrix object by loading its members from a data file the following constructor is used,

`Matrix<double> m("Matrix.dat");`

A matrix data file contains the matrix elements arranged in rows and columns. For instance, the next data will correspond to a Matrix of zeros with 2 rows and 3 columns.

`0 0 00 0 0`

The copy constructor builds an object which is a copy of another object.

`Matrix<bool> a(3, 5); Matrix<bool> b(a);`

One of the most convenient ways to initialize a matrix object is through a Vector List:

` Matrix<double> m({Vector<double>({1, 2, 3}),Vector<double>({1, 2, 3})},{"a","b"});`

The first argument of m is a list of vectors that correspond to the columns of the matrix. The second argument concerns to the header.

Operators

The Matrix class also implements the assignment operator.

`Matrix<double> a(2, 1);Matrix<double> b = a;`

Below there is an usage example of the reference operator here. Note that row indexing goes from 0 to rows_number-1 and column indexing goes from 0 to columns_number-1.

`Matrix<int> m(2, 2);m(0, 0) = 1;m(1, 1) = 1;m(0, 1) = 0;m(1, 0) = 0;`

The use of the arithmetic operators for the Matrix class are very similar to those for the Vector class. The following sentence uses the scalar difference operator,

`Matrix<double> a(5, 7, 2.5);Matrix<double> b = a + 0.1;`

Also, using the arithmetic and assignment operators with the Matrix class is similar than with the Vector class. For instance, to assign by sum with another Matrix we can write

`Matrix<double> a(1, 2, 1.0);Matrix<double> b(1, 2, 0.5);a += b;`

The not equal to operator with another Matrix can be used in the following way,

`Matrix<string> a(1, 1, "hello");Matrix<string> b(1, 1, "good bye");bool is_not_equal_to = (a != b);`

The use of the greater than operator with a scalar is listed below

`Matrix<double> a(2, 3, 0.0);bool is_greater_than = (a < 1.0);`

Methods

As it happens for the Vector class, the Matrix class implements get and set methods for all the members. The get_rows_number() and get_columns_number() methods are very useful.

`Matrix<int> m(4, 2);size_t rows_number = m.get_rows_number();size_t columns_number = m.get_columns_number();`

In order to set a new number of rows or columns to a Matrix object, the set_rows_number() or set_columns_number() methods are used.

`Matrix<int> m(4, 2);size_t rows_number = m.get_rows_number();size_t columns_number = m.get_columns_number();`

A Matrix can be initialized with a given value, at random with an uniform distribution or at random with a normal distribution,

`Matrix<double> m(4, 2);m.initialize(0.0);m.randomize_uniform(-0.2, 0.4);m.randomize_normal(-1.0, 0.25);`

A set of mathematical methods are also implemented for convenience. For instance, the dot() method computes the dot product of this Matrix with a Vector or with another Matrix.

`Matrix<double> m(4, 2, 1.0);Vector<double> v(4, 2.0);Vector<double> dot_product = m.dot(v);`

Finally, string serializing, printing, saving or loading utility methods are also implemented. For example, the use of the print() method or print_preview() is as follows.

`Matrix<bool> m(1, 3, false);m.print();m.print_preview();`

3. The Tensor template

Members

The Tensor class has one member:

• The dimensions.

This member is private, which means that it can only be accessed within methods of the class itself.

Constructors

In the same way as for the rest of the classes, a Tensor can be constructed in several ways.

The default constructor creates a zero dimensions tensor and it can be called as follows

`Tensor<double> t;`

A tensor can also be construcuted with a vector that specifies the number of dimensions of the vector and their values, which are specified in a vector. We can also initialize the tensor to a value. An example of a four dimensions tensor is shown below

`Vector<size_t> dimensions({4, 2});Tensor<double> t(dimensions);Tensor<double> t(dimensions, 5.0);`

It is also possible to construct a tensor with 1, 2, 3 or 4 dimensions. The next tensor will have 3 dimensions whose values are 2, 3 and 6. If we wanted a tensor with other number of dimensions, we just have to specify their values in the constructor.

`Tensor<double> t(2, 3, 6);`

Finally, a tensor can be constructed by copying a matrix. This will generate a tensor of two dimensions

`Matrix<double> m(3, 2, 1.0);Tensor<double> t(m);`

Operators

The assignment operator is also available for tensors. It copies a tensor into a new one

`Vector<size_t> dimensions({4, 2, 6});Tensor<double> t(dimensions, 5.0);Tensor<double> s = t;`

The sum, difference, product and quotient with a scalar work in in the same way as for the vector and matrix classes. The next sentence multiplies every element of a tensor by a number

`Vector<size_t> dimensions({4, 2, 6});Tensor<double> t(dimensions, 5.0);t = t * 5.0;`

Similarly, these arithmetic operators can be applied to another tensor. Note that, in this case, the two tensors must have the same size

`Vector<size_t> dimensions({4, 2, 6});Tensor<double> t(dimensions, 2.0);Tensor<double> s(dimensions, 5.0);Tensor<double> l = t + s;`

Comparing two tensors or a tensor with a scalar is also possible with the equal to, greater or lower than operators

`Vector<size_t> dimensions({4, 2, 6});Tensor<double> t(dimensions, 5.0);Tensor<double> s(dimensions, 5.0);bool is_equal = (t == s);`

Methods

The Tensor class contains several methods that allow to get and set the members. For instance, the number of dimensions of a tensor can be obtained as follows

`Vector<size_t> dimensions({4, 2, 6});size_t dimensions_number = t.get_dimensions_number();`

It is also possible to get all the dimensions arranged in a vector as follows

`Tensor<double> t(4, 2, 6);Vector<size_t> dimensions = t.get_dimensions();`

Once a tensor has been constructed, it can be initialized to a given value or randomly. For the second method, the initialization could follow a uniform distribution or a normal distribution. An example of each case is shown below for a four dimensions tensor

`Tensor<double> t(4, 2, 6, 3);t.initialize(2.0);t.randomize_uniform(-1.0, 1.0);t.randomize_normal(0.0, 1.0);`