Tensor Class using Expression Templates in C++

This class is designed to show the capabilities of C++ template metaprogramming and is not intended for production use. While it has the potential to be very efficient, it can also contribute to very long compile times and unreadable error messages.

See the full code at https://github.com/Nik4053/Templated-Cpp-Tensor

After finding myself writing the same tensor class over and over again, I decided to create a reusable C++ tensor class. This class is designed to be simple and easy to use, while still being efficient. The class is implemented using the C++ standard library and is header-only, meaning that it can be included in any C++ project without the need for any additional libraries.

What first started as a simple class, quickly turned into a project to explore the capabilities of C++ template metaprogramming. I started with a simple tensor class with a template parameter for the data type, then extended it to also include the number of dimensions as a template parameter. And finally, I added support for arbitrary tensor shapes using variadic templates.

Tensor<float> tensor;           // first version  <dtype>
Tensor<float, 3> tensor;        // second version <dtype,RANK>
Tensor<float, 2, 2, 2> tensor;  // final version  <dtype,SHAPE>

The Tensor class is fully templated and supports arbitrary tensor shapes using variadic templates. It allows you to create tensors of any data type and any number of dimensions, offering a high degree of flexibility. What sets it apart from other tensor libraries is its extensive use of expression templates which allow for intuitive, readable syntax and efficient data manipulation. Importantly, the class itself does not allocate or manage memory; instead, it operates as a lightweight wrapper around existing data structures by referencing external memory through a pointer.

Because the class does not maintain any internal state or parameters, each instance is considered trivial by the compiler. This allows the compiler to completely optimize away the class, ensuring zero overhead when used alongside existing data containers.

The arithmetic operations are implemented using expression templates, enabling efficient, lazy evaluation of expressions without the need for temporary objects. For example, an expressions like `D = A + B / C` is evaluated in a single pass, with element values computed only when accessed.

The whole Tensor project is header only, so you can simply include the header file in your project and use the Tensor class without any additional dependencies.

I wanted a simple storage class that allows me to easily access the data and perform basic operations such as addition, subtraction, multiplication, and division. The class itself does therefore not contain any data, but instead contains a pointer to the data. This also allows me to easily create sub-tensors by simply creating a new tensor object with a different pointer to the same data.

See the full code: https://github.com/Nik4053/Templated-Cpp-Tensor/blob/master/tensor.hpp.

// basic tensor definition
template <typename T, std::size_t... DIMS>
class Tensor;
 
// specialization for 1 dimensions
template <typename T, std::size_t DIM>
class Tensor<T, DIM> {
public:
    T* data;
    constexpr static size_t SIZE = DIM;
    constexpr static size_t RANK = 1;
    Tensor(T* data) : data(data) {}
};

// specialization for N dimensions
template <typename T, std::size_t DIM, std::size_t... DIMS>
class Tensor<T,DIM,DIMS...> {
public:
    T* data;
    constexpr static size_t SIZE = DIM * (DIMS * ...);
    constexpr static size_t RANK = 1 + sizeof...(DIMS);
    Tensor(T *data) : data(data) {}
};

Example of the operations this tensor allows:

Tensor<float, 4,3,2> tensor;                    // create a tensor with 3 dimensions
tensor[0][0][0] = 5;                            // write to tensor
tensor(0,0,0) = 5;                              // write to tensor
tensor += 1;                                    // add 1 to all elements
tensor += tensor;                               // element-wise tensor addition
Tensor<float, 3,2> sub = tensor[0];             // reference to sub-tensor
Tensor<float, 4*3*2> flat = tensor.flatten();   // flatten the tensor
cout << tensor << endl;                         // print the tensor

I also added some helper functions to make the class more user-friendly. See the full example code https://github.com/Nik4053/Templated-Cpp-Tensor/blob/master/main.cpp.

Example of the new operations this tensor allows:

Tensor<float, 4,3,2> A,B,C,D;                         // create a tensor with 3 dimensions
D = A + sin(B / C);                                   // add tensors
D = A + B + 4.2f;                                     // use constants
auto expr = A + B / C.max();                          // create an expression.
float value = expr[0];                                // evaluate the expression at a specific index 
value = expr[A.idx(1,1,1)];                           // evaluate the expression at a specific index

Tensor<float, 4,3,1> E;
E = A + B / C;                                        // ERROR: SIZE does not match

Tensor<float, 4*3*2> F;
F = A + B / C;  									  // Works: SIZE matches