Vector, Matrix and Tensor templates

Tensors are the essential elements that allow OpennNN to store and manage all the information that the neural network will later analyze.

This tutorial will teach about the tensor templates and how OpenNN allows you to work with them quickly.

Contents:

  1. The Vector template
  2. The Matrix template
  3. The Tensor template

1. The Vector template

The Vector class is a template, which means that it can be applied to different types. That is, we can create a Vector or int numbers, MyClass objects, etc.

The Vector in OpenNN is derived from the vector in the Standard Template Library.

Members

The OpenNN Vector derives from the std vector, and it does not include any additional members.

Constructors

Multiple constructors are defined in the Vector class, where the different constructors take different parameters.

The easiest way to create a vector object is to use the default constructor, which builds a vector of size zero. For example, to construct an empty Vector of int numbers, we use the following.

Vector<int> v;

The following sentence constructs a Vector of 3 double numbers.

Vector<double> v(3);

If we want to construct a Vector of 5 bool variables and initialize all the elements to false, we can use

Vector<bool> v(5, false);

It is also possible to construct a Vector class object while also loading its members from a file. To do that, we can do

Vector<int> v("Vector.dat");

The file `Vector.dat' contains a first row with the size of the vector and an additional row for each element of the vector.

The following sentence constructs a Vector, which is a copy of another Vector,

Vector<MyClass> v(3);
Vector<MyClass> w(v);

A useful way to create a vector object is to through a list:

Vector<int> v({1, 2, 3});
Vector<string> w({"one","two","three"});

A useful way to create a vector object is to through a list:

Vector<int> v({1, 2, 3});
Vector<string> w({"one","two","three"});

Operators

The Vector class also implements different types of operators for assignment, reference, arithmetic, or comparison.

The assignment operator copies a vector into another vector,

Vector<int> v;
Vector<int> w = v;

The following sentence constructs a vector and sets the values of their elements using the reference operator. Note that indexing goes from 0 to n-1, where n is the Vector size.

Vector<double> v(3);
v[0] = 1.0;
v[1] = 2.0;
v[2] = 3.0;

The Vector class includes sum, difference, product and quotient operators to perform arithmetic operations with a scalar or another Vector. Note that the arithmetic operators with another Vector require that they have the exact sizes.

The following sentence uses the vector-scalar sum operator, which adds a given value to all the elements in the vector

Vector<double> v(3, 1.0);
Vector<double> w = v + 3.1415926;

An example of the use of the vector-vector multiplication operator is given below. It realizes the element-wise multiplication between the two vectors.

Vector<double> v(3, 1.2);
Vector<double> w(3, 3.4);
Vector<double> x = v*w;

Assignment by sum, difference, product, or quotient with a scalar or another Vector is also possible using arithmetic and assignment operators. If another Vector is to be used, it must have the same size.

For instance, to assign by difference with a scalar, we might do

Vector<int> v(3, 2);
v -= 1;

To assign by quotation with another Vector, we can write

Vector<double> v(3, 2.0);
Vector<double> w(3, 0.5);
v /= w;

Equality and relational operators are also implemented here. They can be used with a scalar or another Vector. For the last case, the exact sizes are assumed. An example of the equal and not equal to operators with a scalar is

Vector<bool> v(5, false);
bool is_equal = (v == false);
bool is_not_equal = (v != false);

The less, less or equal, greater and greater or equal than an operator with another Vector can be used as follows,

Vector<double> v(5, 2.3);
Vector<double> w(5, 3.2);
bool is_less = (v < w);
bool is_less_equal = (v <= w);
bool is_greater = (v > w);
bool is_greater_equal = (v >= w);

Methods

Get and set methods for each member of this class are implemented to exchange information among objects.

The method size() returns the size of a Vector.

Vector<int> v({1, 2, 3, 4, 5});
size_t size = v.size();

On the other hand, the method set() sets a new size to a Vector. Note that the element values of that vector are lost.

Vector<bool> v(3);
v.set(6);

If we want to initialize a vector at random, we can use the randomize_uniform() or randomize_normal() methods. For the first method, a minimum and a maximum value must be specified, while for the second method

Vector<double> v(5);
Vector<double> w(3);
v.randomize_uniform(-1,1);
w.randomize_normal(0,1);

To get the first or last elements of a vector, the following methods are used:

Vector<int> v({1, 2, 3, 4, 5, 6, 7, 8, 9});
Vector<int> first_5 = v.get_first(5);
Vector<int> last_5 = v.get_last(5);

The Vector class also includes some mathematical methods that can be useful in developing neural networks algorithms and applications.

The calculate_L1_norm() method calculates the norm of the vector, <

Vector<double> v(5, 3.1415927);
double norm = v.calculate_L1_norm();

To calculate the dot product between this vector and another Vector, we can do

Vector<double> v(3, 2.0);
Vector<double> w(3, 5.0);
double dot = v.dot(w);

We can calculate the mean or the standard deviation values of the elements in a Vector by using the calculate_mean() and calculate_standard_deviation() methods, respectively. For instance

Vector<double> v({1.2, 2.5, 3.6, 2.4});
double mean = v.calculate_mean();
double standard_deviation = v.calculate_standard_deviation();

Finally, utility methods for serialization, loading, and saving the class members to a file are also included. To obtain a std::string representation of a Vector object, we can make the following.

Vector<bool> v(1, false);
Vector<std::string> vector_string = v.to_string_vector();

To save a Vector object to a file we can do

Vector<int> v(2, 0);
v.save("Vector.dat", ';');

The save method only needs the path and the separator.

If we want to load a Vector object from a data file, we could write

Vector<int> v;
v.load("Vector.dat");

2. The Matrix template

As it happens with the Vector class, the Matrix class is also a template. Therefore, a Matrix of any type can be created.

Members

The Matrix class has three members:

Those members are private. Private members can be accessed only within the methods of the class itself.

Constructors

The Matrix class also implements multiple constructors with different parameters.

The default constructor creates a matrix of integers with zero rows and zero columns.

Matrix<int> m;

To construct an empty Matrix with a specified number of rows and columns, we use

Matrix<int> m(2, 3);

We can specify the number of rows and columns and initialize the Matrix elements at the same time by doing

Matrix<double> m(1, 5, 0.0);
		

To build a Matrix object by loading its members from a data file, the following constructor is used

Matrix<double> m("Matrix.dat");

A matrix data file contains the matrix elements arranged in rows and columns. For instance, the following data will correspond to a Matrix of zeros with two rows and three columns.

0 0 0
0 0 0

The copy constructor builds an object which is a copy of another object.

Matrix<bool> a(3, 5); 
Matrix<bool> b(a);

One of the most convenient ways to initialize a matrix object is through a Vector List:

 Matrix<double> m({Vector<double>({1, 2, 3}),Vector<double>({1, 2, 3})},{"a","b"});

The first argument of m is a list of vectors that correspond to the columns of the matrix. The second argument concerns to the header.

Operators

The Matrix class also implements the assignment operator.

Matrix<double> a(2, 1);
Matrix<double> b = a;

Below there is a usage example of the reference operator here. Note that row indexing goes from 0 to rows_number-1, and column indexing goes from 0 to columns_number-1.

Matrix<int> m(2, 2);
m(0, 0) = 1;
m(1, 1) = 1;
m(0, 1) = 0;
m(1, 0) = 0;

The arithmetic operators for the Matrix class are very similar to those for the Vector class. The following sentence uses the scalar difference operator,

Matrix<double> a(5, 7, 2.5);
Matrix<double> b = a + 0.1;

Also, using the arithmetic and assignment operators with the Matrix class is similar to the Vector class. For instance, to assign by sum with another Matrix, we can write

Matrix<double> a(1, 2, 1.0);
Matrix<double> b(1, 2, 0.5);
a += b;

The not equal to the operator with another Matrix can be used in the following way,

Matrix<string> a(1, 1, "hello");
Matrix<string> b(1, 1, "good bye");
bool is_not_equal_to = (a != b);

The use of the greater than an operator with a scalar is listed below

Matrix<double> a(2, 3, 0.0);
bool is_greater_than = (a < 1.0);

Methods

As it happens for the Vector class, the Matrix class implements get and set methods for all the members. The get_rows_number() and get_columns_number() methods are very useful.

Matrix<int> m(4, 2);
size_t rows_number = m.get_rows_number();
size_t columns_number = m.get_columns_number();

In order to set a new number of rows or columns to a Matrix object, the set_rows_number() or set_columns_number() methods are used.

Matrix<int> m(4, 2);
size_t rows_number = m.get_rows_number();
size_t columns_number = m.get_columns_number();

A Matrix can be initialized with a given value, at random with a uniform distribution or at random with a normal distribution,

Matrix<double> m(4, 2);
m.initialize(0.0);
m.randomize_uniform(-0.2, 0.4);
m.randomize_normal(-1.0, 0.25);

A set of mathematical methods are also implemented for convenience. For instance, the dot() method computes the dot product of this matrix with a Vector or with another Matrix.

Matrix<double> m(4, 2, 1.0);
Vector<double> v(4, 2.0);
Vector<double> dot_product = m.dot(v);

Finally, string serializing, printing, saving or loading utility methods are also implemented. For example, the use of the print() method or print_preview() is as follows.

Matrix<bool> m(1, 3, false);
m.print();
m.print_preview();

3. The Tensor template

Members

The Tensor class has one member:

This member is private, which can only be accessed within the class's methods.

Constructors

In the same way, as for the rest of the classes, a Tensor can be constructed in several ways.

The default constructor creates a zero dimensions tensor, and it can be called as follows

Tensor<double> t;

A tensor can also be constructed with a vector that specifies the number of dimensions of the vector and their values, which are specified in a vector. We can also initialize the tensor to a value. An example of a four dimensions tensor is shown below

Vector<size_t> dimensions({4, 2});
Tensor<double> t(dimensions);
Tensor<double> t(dimensions, 5.0);

It is also possible to construct a tensor with 1, 2, 3, or 4 dimensions. For example, the following tensor will have three dimensions whose values are 2, 3, and 6. If we wanted a tensor with other dimensions, we have to specify their values in the constructor.

Tensor<double> t(2, 3, 6);

Finally, a tensor can be constructed by copying a matrix. This will generate a tensor of two dimensions

Matrix<double> m(3, 2, 1.0);
Tensor<double> t(m);

Operators

The assignment operator is also available for tensors. It copies a tensor into a new one

Vector<size_t> dimensions({4, 2, 6});
Tensor<double> t(dimensions, 5.0);
Tensor<double> s = t;

The sum, difference, product, and quotient with a scalar work in the same way as the vector and matrix classes. The next sentence multiplies every element of a tensor by a number

Vector<size_t> dimensions({4, 2, 6});
Tensor<double> t(dimensions, 5.0);
t = t * 5.0;

Similarly, these arithmetic operators can be applied to another tensor. Note that, in this case, the two tensors must have the same size

Vector<size_t> dimensions({4, 2, 6});
Tensor<double> t(dimensions, 2.0);
Tensor<double> s(dimensions, 5.0);
Tensor<double> l = t + s;

Comparing two tensors or a tensor with a scalar is also possible with the equal to, greater or lower than operators

Vector<size_t> dimensions({4, 2, 6});
Tensor<double> t(dimensions, 5.0);
Tensor<double> s(dimensions, 5.0);
bool is_equal = (t == s);

Methods

The Tensor class contains several methods that allow to get and set the members. For instance, the number of dimensions of a tensor can be obtained as follows

Vector<size_t> dimensions({4, 2, 6});
size_t dimensions_number = t.get_dimensions_number();

It is also possible to get all the dimensions arranged in a vector as follows

Tensor<double> t(4, 2, 6);
Vector<size_t> dimensions = t.get_dimensions();

Once a tensor has been constructed, it can be initialized to a given value or randomly. For the second method, the initialization could follow a uniform distribution or a normal distribution. An example of each case is shown below for a four dimensions tensor

Tensor<double> t(4, 2, 6, 3);
t.initialize(2.0);
t.randomize_uniform(-1.0, 1.0);
t.randomize_normal(0.0, 1.0);

Bibliography