Basic linear algebra in torchTT
This notebook is an introduction into the basic linar algebra operations that can be perfromed using the torchtt package. The basic operations such as +,-,*,@,norm,dot product can be performed between torchtt.TT instances without computing the full format by computing the TT cores of the result. One exception is the elementwise division between TT objects. For this, no explicit form of the resulting TT cores can be derived and therefore optimization techniques have to be employed (see
the notebook fast_tt_operations.ipynb).
Imports
[1]:
import torch as tn
try:
import torchtt as tntt
except:
print('Installing torchTT...')
%pip install git+https://github.com/ion-g-ion/torchTT
import torchtt as tntt
We will create a couple of tensors for the opperations that follow
[2]:
N = [10,10,10,10]
o = tntt.ones(N)
x = tntt.randn(N,[1,4,4,4,1])
y = tntt.TT(tn.reshape(tn.arange(N[0]*N[1]*N[2]*N[3], dtype = tn.float64),N))
A = tntt.randn([(n,n) for n in N],[1,2,3,4,1])
B = tntt.randn([(n,n) for n in N],[1,2,3,4,1])
Addition
The TT class has the “+” operator implemeted. It performs the addition between TT objects (must have compatible shape and type) and it returns a TT object. One can also add scalars to a TT object (float/int/torch.tensor with 1d).
The TT rank of the result is the sum of the ranks of the inputs. This is usually an overshoot and rounding can decrease the rank while maintaining the accuracy.
Here are a few examples:
[3]:
z = x+y
print(z)
# adding scalars is also possible
z = 1+x+1.0
z = z+tn.tensor(1.0)
# it works for the TT amtrices too
M = A+A+1
print(M)
TT with sizes and ranks:
N = [10, 10, 10, 10]
R = [1, 6, 6, 6, 1]
Device: cpu, dtype: torch.float64
#entries 840 compression 0.084
TT-matrix with sizes and ranks:
M = [10, 10, 10, 10]
N = [10, 10, 10, 10]
R = [1, 5, 7, 9, 1]
Device: cpu, dtype: torch.float64
#entries 11200 compression 0.000112
Broadcasting is also available and is similar to the PyTorch broadcasting. Tensors in the TT-format can be added even if their shapes are different. The rule is that the number of dimensions of the first operand must be greater or equal to the number of dimensions of the second operand. In the following example a (4,5) tensor is added to a (2,3,4,5) tensor:
[4]:
xx = tntt.random([2,3,4,5],[1,2,3,4,1])
yy = tntt.random([4,5],[1,2,1])
print(xx+yy)
TT with sizes and ranks:
N = [2, 3, 4, 5]
R = [1, 3, 4, 6, 1]
Device: cpu, dtype: torch.float64
#entries 168 compression 1.4
The mode sizes should match starting from the end or the mode size of the second tensor can be 1:
[5]:
xx = tntt.random([2,3,4,5],[1,2,3,4,1])
yy = tntt.random([1,1,4,5],[1,2,2,2,1])
print(xx+yy)
TT with sizes and ranks:
N = [2, 3, 4, 5]
R = [1, 4, 5, 6, 1]
Device: cpu, dtype: torch.float64
#entries 218 compression 1.8166666666666667
Subtraction
The “-” operator is also implemented in the torchtt.TT class. It can be used similarily to “+” between 2 torchtt.TT objects and between a torchtt.TT and a scalar. It can also be used as a negation.
[6]:
v = x-y-1-0.5
C = A-B-3.14
w = -x+x
print(tn.linalg.norm(w.full()))
tensor(1.2886e-14, dtype=torch.float64)
Broadcasting is available for the “-” operation as well.
Multiplication (elementwise)
One can perform the elementwise multiplication \(\mathsf{u}_{i_1...i_d} = \mathsf{x}_{i_1...i_d} \mathsf{y}_{i_1...i_d}\) between 2 tensors in the TT format without goin to full format. The main issues of this is that the rank of the result is the product of the ranks of the input TT tensors.
[7]:
u = x*y
print(u)
M2 = A*A
TT with sizes and ranks:
N = [10, 10, 10, 10]
R = [1, 8, 8, 8, 1]
Device: cpu, dtype: torch.float64
#entries 1440 compression 0.144
Broadcasting is available for the “*” operation as well.
Matrix vector product and matrix matrix product
TT matrix and TT tensor: \((\mathsf{Ax})_{i_1...i_d} = \sum\limits_{j_1...j_d}\mathsf{A}_{i_1...i_d,j_1...j_d} \mathsf{x}_{j_1...j_d}\)
TT matrix and TT matrix: \((\mathsf{AB})_{i_1...i_d,k_1...k_d} = \sum\limits_{j_1...j_d}\mathsf{A}_{i_1...i_d,j_1...j_d} \mathsf{B}_{j_1...j_d,k_1...k_d}\)
[8]:
print(A@x)
print(A@B)
print(A@B@x)
TT with sizes and ranks:
N = [10, 10, 10, 10]
R = [1, 8, 12, 16, 1]
Device: cpu, dtype: torch.float64
#entries 3120 compression 0.312
TT-matrix with sizes and ranks:
M = [10, 10, 10, 10]
N = [10, 10, 10, 10]
R = [1, 4, 9, 16, 1]
Device: cpu, dtype: torch.float64
#entries 20000 compression 0.0002
TT with sizes and ranks:
N = [10, 10, 10, 10]
R = [1, 16, 36, 64, 1]
Device: cpu, dtype: torch.float64
#entries 29600 compression 2.96
Multiplication can be performed between a TT operator and a full tensor (in torch.tensor format) the result in this case is a full tn.tensor
[9]:
print(A@tn.rand(A.N, dtype = tn.float64))
tensor([[[[-9.0336e+01, 3.8735e+01, -1.7435e+02, ..., -7.9007e+01,
2.6077e+01, -8.5886e+01],
[ 2.3355e+01, 4.3107e+01, 1.6970e+01, ..., 3.8010e+01,
-2.3604e+00, -4.4812e+01],
[-3.6112e+00, 1.2135e+01, -3.2300e+01, ..., -8.5324e+01,
-3.3324e+01, -7.1913e+00],
...,
[-5.4962e+01, 3.2071e+01, -5.6565e+01, ..., -6.5829e+00,
4.0358e+01, -3.1699e+01],
[-7.7103e+01, 5.8233e+01, -4.7107e+01, ..., -1.3475e+01,
-1.5326e+01, 2.2327e+00],
[-3.4774e+01, -6.1789e+01, -3.3708e+01, ..., -1.2907e+02,
-3.0087e+01, 2.4054e+01]],
[[-3.9436e+01, 1.9067e+01, -5.9823e+01, ..., -3.1155e+01,
-7.2907e-01, -5.0418e+01],
[ 4.4996e+01, 4.0265e+01, 1.8788e+01, ..., 9.0693e+00,
9.4273e+00, 5.4921e+00],
[-4.4252e+01, 9.7456e+00, -3.5214e+01, ..., -5.9046e+01,
1.0192e+01, -2.5947e+01],
...,
[-4.1946e+01, 1.8857e+00, 8.5353e-01, ..., 9.2885e+00,
4.0946e+01, 9.3712e+00],
[-6.5372e+00, -3.2586e+00, -1.6214e+01, ..., 3.5030e+01,
-1.6823e+01, 3.6858e+00],
[-4.8552e+01, -2.1148e+01, -4.7406e+01, ..., -1.1090e+02,
-6.4636e+00, 2.2849e+00]],
[[-4.5780e+01, -5.9659e+01, -4.2840e+01, ..., -7.1812e+01,
-3.8369e+01, -1.0003e+01],
[-3.4740e+01, -1.7994e+00, 3.0631e+01, ..., -4.2637e+01,
-5.4735e+01, -1.2876e+01],
[-1.4367e+01, 3.7647e+01, -3.9752e+01, ..., -5.2452e+01,
3.8651e+00, -2.3805e+01],
...,
[-2.7425e+01, 6.1218e+00, -4.9695e+01, ..., -4.6290e+01,
2.9006e+00, -4.2650e+01],
[-3.8827e+01, 2.6728e+01, 2.7179e+00, ..., -3.0048e+01,
-2.9350e+01, -4.1165e+00],
[-1.9126e+01, 1.1838e+01, -3.6201e+01, ..., -7.4385e+01,
1.9851e+01, -1.7571e+00]],
...,
[[ 8.6367e+01, -5.2682e+01, 1.3576e+02, ..., 1.1491e+02,
1.2565e+00, 6.5262e+01],
[ 2.4114e+01, -2.8229e+01, 9.9388e+01, ..., 5.6911e+01,
-3.5485e+01, 7.9717e+01],
[ 2.7809e+01, 1.7318e+01, -4.0279e+01, ..., -2.5222e+01,
-1.3807e+01, -3.0460e+01],
...,
[ 9.4533e+01, -7.4195e+01, 8.9131e+01, ..., 4.6406e+01,
-1.2505e+00, 8.6010e+01],
[ 9.3450e+01, 1.7071e+00, 6.5482e+01, ..., 6.8799e+01,
5.5319e+00, 4.0768e+01],
[ 1.6282e+01, 1.5140e+01, 3.7880e+01, ..., 5.2319e+01,
1.6073e+01, -5.0414e+01]],
[[ 3.2072e+01, 7.8254e+01, -4.8531e+00, ..., 3.0940e+01,
3.7199e+01, -5.5023e+01],
[ 7.8391e+01, -3.1667e+01, 2.2846e+01, ..., 9.6551e+01,
1.5065e+01, -1.6883e+01],
[ 3.9680e+01, -3.5254e+01, 1.4237e+01, ..., 4.3960e+01,
6.5710e+00, 3.8968e+01],
...,
[ 7.0491e+01, 3.9545e+01, 1.1331e+02, ..., 1.2315e+02,
-1.1225e+01, 1.7326e+01],
[ 2.6792e+01, -5.2023e+00, 5.9619e+00, ..., 1.2766e+01,
2.1583e+01, -2.4202e+01],
[ 6.1479e+00, -7.2089e+00, 4.2920e+01, ..., 3.0608e+01,
-1.0941e+01, 7.5194e+01]],
[[ 1.8515e+01, 2.0816e+00, 6.3453e+01, ..., 4.1497e+01,
2.1168e+01, 1.9736e+01],
[ 3.2334e+01, -2.9361e+00, 1.5455e+01, ..., 1.3765e+01,
9.5458e+00, 3.0400e+01],
[ 2.9168e+01, 1.3335e+01, 2.4453e+00, ..., 7.6992e+01,
-8.4443e+00, 1.3621e+01],
...,
[ 4.0129e+01, -3.7545e+01, 8.0538e+01, ..., 1.6958e+01,
-2.6002e+01, 3.6947e+01],
[ 8.3283e+01, -2.3386e+01, 6.9030e+00, ..., 6.6849e+01,
2.1997e+01, -2.1561e+00],
[ 1.4499e+01, 2.4019e+01, 1.8095e+01, ..., 1.1089e+01,
5.5190e-01, -3.6762e+00]]],
[[[-2.4022e+02, 1.6229e+02, -3.1450e+02, ..., -2.4727e+02,
4.3311e+01, -2.2495e+02],
[ 7.3594e+01, 4.6484e+01, -6.8606e+00, ..., 1.0347e+02,
2.2257e+01, -1.2515e+02],
[-1.3524e+01, -7.1460e+01, -1.0080e+02, ..., -5.7797e+01,
1.5620e+01, -3.9373e+01],
...,
[-1.5934e+02, 4.3413e+01, -5.0763e+01, ..., -7.4011e+01,
4.5191e+01, -1.7630e+01],
[-1.2594e+02, 9.6428e+01, -1.2858e+02, ..., -5.5950e+01,
-2.2809e+01, -3.5007e+00],
[-1.3663e+02, -1.4914e+02, -1.3422e+02, ..., -3.1168e+02,
1.8925e+01, 5.5882e+01]],
[[-8.2157e+01, 8.3188e+01, -1.4397e+02, ..., -1.6779e+02,
5.7117e+01, -6.0355e+01],
[ 7.3107e+01, 7.2466e+01, 1.8701e+01, ..., 7.6782e+01,
2.6922e+01, -2.2192e+01],
[-7.1024e+01, -2.5625e+01, -1.0611e+02, ..., -8.0653e+01,
4.0332e+01, -1.0061e+02],
...,
[-7.9073e+01, 2.4635e+01, -1.5324e+01, ..., 5.1124e+01,
1.1534e+02, -4.1479e+00],
[-1.8970e+01, 5.6004e+01, -4.6634e+00, ..., 4.0291e+01,
2.4633e+01, -9.5834e+00],
[-1.8408e+02, -9.8227e+01, -1.3419e+02, ..., -2.7394e+02,
1.3958e+01, -2.8962e+01]],
[[-9.8755e+01, -4.2617e+01, -3.8412e+01, ..., -1.4031e+02,
-4.3511e+01, -5.9902e+00],
[-5.8251e+01, 2.1478e+01, -3.9696e+01, ..., -4.2405e+01,
-6.3586e+01, -6.1868e+01],
[-2.6820e+01, 3.0859e+01, -1.2515e+02, ..., -1.5107e+02,
7.7468e+01, -5.1443e+01],
...,
[-3.5113e+01, -1.4932e+01, -7.7752e+01, ..., -6.7806e+01,
-3.9891e-01, -2.7286e+01],
[-1.1701e+02, 9.0316e+00, -2.2038e+01, ..., -7.7615e+01,
-5.8392e+01, 1.2142e+00],
[-4.0441e+01, -4.0592e+01, -5.5038e+01, ..., -1.2792e+02,
6.9558e+01, -3.7734e+01]],
...,
[[ 2.6176e+02, -1.5193e+02, 2.7416e+02, ..., 4.2529e+02,
-1.8232e+01, 1.3747e+02],
[-6.6050e+00, -8.1392e+01, 1.9762e+02, ..., 1.4524e+02,
-4.2343e+01, 1.6039e+02],
[ 7.0599e+00, -2.4597e+01, -3.2661e+01, ..., -7.4313e+01,
-2.6198e+01, -3.3784e+01],
...,
[ 1.9954e+02, -1.8723e+02, 1.5943e+02, ..., 1.9447e+02,
7.6098e+00, 1.8052e+02],
[ 1.7503e+02, -4.2121e+01, 1.7813e+02, ..., 1.6963e+02,
-3.7964e+00, 1.0084e+02],
[ 8.6096e+01, 9.6020e+01, 7.1147e+01, ..., 2.1962e+02,
2.5732e+01, -6.1041e+01]],
[[-1.0166e+01, 9.8634e+01, -6.4270e+01, ..., 7.3475e+01,
5.8113e+01, -7.2119e+01],
[ 1.6708e+02, -8.7255e+01, 7.9768e+01, ..., 2.5727e+02,
7.3005e+01, -1.1333e+02],
[ 9.7859e+01, -8.9022e+01, 2.4388e+01, ..., 6.9609e+01,
4.2142e+01, 1.1598e+02],
...,
[ 8.2152e+01, 5.3069e+01, 1.5030e+02, ..., 1.7084e+02,
-4.0333e+01, 8.3724e+01],
[ 3.8977e+01, -2.3550e+01, 3.1332e+00, ..., 5.6498e-01,
4.8194e+01, -4.0779e+01],
[ 5.5875e+01, -5.2880e+01, 1.0705e+02, ..., 6.9793e+01,
-1.3446e+01, 1.6090e+02]],
[[ 1.1589e+02, -1.2178e+02, 1.5790e+02, ..., 1.8099e+02,
1.3197e+01, 1.3700e+02],
[ 2.4259e+01, -8.8157e+01, 7.3578e+01, ..., 2.6399e+01,
-2.6976e+01, 6.3308e+01],
[ 3.5561e+01, 6.1328e+01, -6.1055e+00, ..., 5.0529e+01,
2.3431e+00, 3.4296e+01],
...,
[ 4.5575e+01, -2.9263e+01, 1.3433e+02, ..., 1.5979e+02,
2.7270e+01, 9.3876e+01],
[ 9.0797e+01, -3.6977e+01, 1.0318e+02, ..., 1.2985e+02,
5.0159e+01, 5.8107e+00],
[ 2.3471e+01, 3.9935e+01, 7.4660e+01, ..., 1.2977e+02,
-4.3921e+00, -7.2764e+01]]],
[[[ 3.4539e+00, -3.2747e+01, 1.0378e+01, ..., 6.0111e+01,
-7.6981e+00, 4.4321e+00],
[ 6.4590e+00, 1.8677e+01, -3.4701e+01, ..., -4.6370e+00,
2.6487e+01, 1.5253e+01],
[-4.1352e+00, 2.1456e+01, 4.0266e+01, ..., -1.5811e+01,
-4.3291e+01, -1.7733e+01],
...,
[ 5.4283e-01, 4.7112e+01, -2.3356e+01, ..., 3.6716e+01,
8.3522e+00, 2.1407e+01],
[ 4.2667e+01, -1.3936e+01, 3.0583e+01, ..., -5.3482e+00,
-5.5786e+01, 4.6558e+01],
[ 3.4250e+01, 3.8930e+01, 9.8798e+00, ..., 1.3517e+01,
-4.3862e+01, -1.4307e+00]],
[[ 1.3119e+01, -1.7008e+00, -2.0519e+01, ..., 4.8856e+01,
4.3680e+00, 1.7078e+00],
[-3.1038e+01, -1.0111e+01, -1.6139e+01, ..., -4.3589e+01,
-3.4131e+01, 1.5077e+01],
[-4.5673e+01, 2.9120e+01, -2.4420e+01, ..., -1.1546e+01,
-1.0594e+01, -4.4285e+01],
...,
[-3.5209e+01, -8.4563e+00, -2.4930e+01, ..., 9.4604e+00,
-1.7927e+01, 3.2778e+01],
[-2.5357e+01, -4.7520e-01, 9.9449e+00, ..., 1.2607e+00,
-4.0820e+01, 1.4085e+01],
[-1.0506e+01, 5.1851e+01, -1.7098e+01, ..., -2.0520e+01,
-1.2894e+01, -7.5569e+00]],
[[ 9.3360e-01, 2.4413e+01, -3.3156e+01, ..., -1.1251e+01,
-2.1151e-01, -2.0984e+01],
[ 1.3689e+01, 8.7482e+00, -3.7587e-01, ..., 5.0707e+01,
1.4182e+01, 8.0174e+00],
[-1.7838e+01, -1.7044e+00, 4.6386e+01, ..., 2.0462e+00,
-6.3244e+00, 9.2274e+00],
...,
[-2.8003e+01, 4.5057e+01, -2.3001e+01, ..., 1.0435e+01,
-1.5379e+01, -1.7186e+01],
[ 2.3118e+01, -3.8642e+00, 1.3782e+01, ..., -1.5497e+01,
-3.0299e+01, 1.3557e+01],
[-6.3324e+00, 1.7746e+01, 3.5038e-01, ..., -2.4961e+01,
-4.6291e+01, 3.9833e+01]],
...,
[[-3.6726e+01, 4.3705e+01, -1.1070e+01, ..., -6.3560e+01,
1.1650e+01, -1.0167e+01],
[-1.2660e+01, 2.4615e+01, 1.8539e+00, ..., 9.2855e+00,
1.2418e+01, 2.9615e-01],
[-7.2583e-01, -2.2651e+01, 2.8144e+01, ..., 4.7637e+01,
2.7412e+01, 1.0306e+00],
...,
[ 1.4849e+01, -4.8912e+00, -1.2807e+00, ..., -6.3923e+01,
-2.8621e+00, -1.7403e+00],
[ 2.5342e+01, -1.1739e+00, -5.9065e+01, ..., -2.8554e+01,
2.4754e+01, -1.4417e+00],
[-3.3877e+01, -7.4966e+00, 3.6425e+01, ..., -2.9935e+01,
5.3740e+01, 5.3962e+00]],
[[ 1.8536e+00, -9.8497e+00, -5.0451e+01, ..., -3.7866e+01,
2.7900e+01, -2.9742e+01],
[-2.0754e+01, 6.1178e+00, -1.2085e+01, ..., -5.4726e+01,
-2.5642e+01, 2.9796e+01],
[-5.5465e+01, -1.3338e+01, 2.9821e+01, ..., 1.8006e+01,
-2.7168e+01, -3.0888e+01],
...,
[-3.5879e+01, -1.1731e+01, -3.3895e+01, ..., -7.7998e+00,
1.6984e+01, -2.3270e+01],
[-1.5226e+00, 1.5980e+00, -2.4704e+01, ..., -4.7079e+00,
-2.0495e+01, -1.4318e+01],
[-3.4782e+01, -2.6274e+01, 1.7482e+01, ..., -2.2773e+00,
1.7259e+01, -6.7678e+00]],
[[ 1.4576e+01, -2.0423e+01, 9.4999e+00, ..., -2.1781e+01,
-1.1684e+00, 1.0864e+01],
[-2.5944e+01, 1.2426e+01, 1.0059e+01, ..., -3.4439e+01,
-2.7995e+01, 2.5194e+01],
[-1.3664e+01, 3.5247e+01, -3.8775e+01, ..., -7.1500e+00,
-5.0406e+00, -3.0295e+01],
...,
[ 4.1231e+01, -6.1245e+01, 3.3903e+01, ..., -7.7990e+00,
-1.8715e+01, 1.3175e+01],
[ 2.1708e+01, 8.6143e-01, 1.2356e+01, ..., -2.5873e+00,
-4.8002e+00, -6.1696e+00],
[-2.0847e+01, -1.9883e+01, -1.8419e+01, ..., -2.0467e+01,
3.7883e+01, 2.4044e+00]]],
...,
[[[ 1.0748e+02, -7.2122e+01, 1.3551e+02, ..., 1.6363e+02,
1.8931e+01, 1.2062e+02],
[-3.3196e+00, -2.2706e+01, 1.2110e+01, ..., -7.3953e+01,
-2.9993e+01, 1.0675e+02],
[-1.7142e+01, -1.1629e+01, -1.3210e+00, ..., 1.0106e+02,
-2.3343e+00, 2.9722e+01],
...,
[ 5.8103e+01, -7.8864e+01, 1.1623e+01, ..., -2.2979e+01,
-1.6199e+01, 4.6508e+01],
[ 1.1799e+02, -5.6350e+01, 6.0863e+01, ..., 4.6869e+01,
-2.9982e+01, 5.6229e+01],
[ 1.2306e+01, 1.0886e+02, 1.0321e+02, ..., 1.2138e+02,
3.6995e+01, -3.2777e+01]],
[[ 3.3638e+01, 2.0919e+01, 3.4553e+01, ..., 4.4204e+01,
2.1582e+01, 8.0658e+01],
[-5.6507e+01, -7.3086e+00, 5.5295e+00, ..., -8.0799e+01,
-5.1410e+01, 4.5172e+01],
[ 2.0131e+01, 5.2097e+00, 3.7685e+01, ..., 5.4664e+01,
8.9931e+00, 4.1686e+01],
...,
[ 3.3545e+01, 8.2185e+00, -3.8327e+01, ..., -2.4999e+00,
1.5480e+00, -8.3272e+00],
[-7.7327e+00, 2.9600e+01, 1.1527e+01, ..., -2.9038e+01,
1.6962e+01, -5.3533e+00],
[ 2.7120e+01, 3.0416e+01, 5.5686e+01, ..., 1.2408e+02,
4.3356e+01, -1.4464e+01]],
[[ 4.3603e+01, 4.5653e+01, 6.0292e+01, ..., 1.2309e+02,
-2.6492e+01, 3.1107e+01],
[ 3.1909e+01, -1.7344e+01, 2.1317e+01, ..., 3.7350e+01,
4.7700e+00, -2.8143e+01],
[ 8.2565e+01, 7.0683e+00, 2.4160e+00, ..., 6.8252e+01,
-3.1279e+01, 7.1086e+01],
...,
[ 2.1805e+01, -4.1180e+01, 5.3703e+01, ..., 5.0885e+01,
3.4740e+01, -2.5292e+01],
[-4.1773e+00, -8.4064e+00, 2.0604e+01, ..., -3.1339e+01,
1.8223e+01, -3.9139e+01],
[ 6.6420e+01, -7.4840e-01, 1.2615e+01, ..., 1.2134e+02,
-2.9389e+01, 6.5730e+00]],
...,
[[-1.5352e+02, 7.5583e+01, -1.4236e+02, ..., -1.6657e+02,
-2.6482e+01, -9.1400e+01],
[-1.9031e+01, 1.8980e+01, -1.0989e+02, ..., -2.4633e+01,
6.0567e+01, -1.5525e+02],
[ 2.5270e+01, 3.0965e+01, 3.4032e+01, ..., 8.1346e+01,
-9.8969e+00, 5.0818e+01],
...,
[-7.5174e+01, 5.0123e+01, -1.0797e+02, ..., -1.4463e+02,
1.0978e+00, -1.1853e+02],
[-7.0633e+01, 1.7892e+01, -1.0446e+02, ..., -1.1946e+02,
-1.2195e+01, -3.4817e+01],
[-3.7722e+00, -2.5571e+01, -5.9867e+01, ..., -4.2092e+01,
-5.3032e+01, 3.1166e+01]],
[[-1.6721e+01, -5.4891e+01, -8.2700e+00, ..., -1.2021e+01,
5.9421e+00, 4.5415e+01],
[-1.4001e+02, 3.1065e+00, -4.3161e+01, ..., -1.3798e+02,
1.6024e+01, 8.0678e+00],
[-6.2776e+01, 7.0387e+01, -4.5143e+01, ..., -7.0801e+01,
-4.5359e+01, -1.4998e+01],
...,
[-4.2993e+01, -3.9058e+01, -1.7660e+01, ..., -1.9196e+02,
-1.9034e+01, -2.2821e+01],
[-1.9296e+01, 3.8725e+00, -9.1081e+00, ..., -3.0161e+01,
-3.1672e+00, 2.7650e+01],
[-1.0329e+01, 6.1210e+01, -2.2374e+01, ..., -5.8960e+01,
1.2945e+01, -6.3073e+01]],
[[-1.0454e+02, 8.2529e+01, -1.0529e+02, ..., -6.9498e+01,
3.2333e+01, -5.9178e+01],
[-5.0263e+01, 1.5628e+01, -4.6340e+01, ..., -2.9271e+01,
1.8440e+01, -9.4662e+01],
[-4.6384e+01, -5.7964e+01, 5.3290e+01, ..., -1.9655e+01,
-8.3480e+01, 1.8070e+00],
...,
[-6.1460e+01, 7.0727e+01, -4.4120e+01, ..., -5.7566e+01,
2.1793e+01, -2.6782e+01],
[-7.8522e+01, 4.1006e+01, -4.1478e+01, ..., -3.5857e+01,
3.6457e+01, -4.6735e+01],
[-2.6630e+01, 1.2915e+01, -6.4067e+01, ..., -7.3969e+01,
-3.0295e+01, -2.0153e+01]]],
[[[-1.0362e+02, 4.5637e+01, -1.5429e+02, ..., -2.4141e+02,
-1.7087e+01, -8.6095e+01],
[-7.0729e-01, -7.8352e+00, -2.1153e+01, ..., 2.9007e+01,
-3.6190e+01, -1.1806e+02],
[-8.8775e+01, -2.4630e+01, -1.1357e+02, ..., -1.4904e+02,
-3.8545e+01, -3.2019e+01],
...,
[ 1.0043e+01, 1.5611e+01, -5.4129e+01, ..., -6.8732e+01,
-1.9782e+01, -1.0784e+01],
[-4.2558e+01, -4.7914e+01, -1.0801e+02, ..., -3.1816e+01,
-8.7409e+00, 6.7515e+00],
[-1.2705e+02, -4.6924e+01, -1.9457e+02, ..., -1.9311e+02,
7.7214e+00, 7.0954e+00]],
[[-4.0353e+01, 1.0240e+01, 9.1488e+00, ..., -3.2745e+01,
1.7030e+01, -1.6100e+01],
[ 3.0825e+01, -4.4573e+01, -1.4032e+00, ..., 2.0023e+00,
-3.8758e+01, 2.0510e+01],
[ 1.4995e+01, -5.5227e+01, 3.4455e+01, ..., 2.0871e+00,
-4.5736e+00, 2.2461e+01],
...,
[-2.7023e+01, 4.0920e+01, 2.1315e+00, ..., 7.3368e+01,
1.0145e+01, -1.0103e+01],
[-3.4454e+01, -7.4488e+00, -4.2404e+00, ..., -3.5435e+00,
3.5652e+00, -9.8863e+00],
[ 1.6629e+01, -3.4402e+01, -2.8591e+00, ..., 1.8625e+01,
4.3699e+01, 4.0020e+01]],
[[ 2.3920e+01, -6.4632e+01, 9.6837e+01, ..., -6.0481e+00,
-3.9358e+01, 7.5874e+01],
[-7.4976e+01, 1.2975e+01, -7.8148e+01, ..., -1.1346e+02,
-5.7074e+01, 3.5919e+01],
[-4.7693e+01, 6.0078e+00, -7.0760e+01, ..., -9.4188e+01,
2.2596e+01, -6.6485e+01],
...,
[ 3.5868e+01, -5.1946e+01, 1.9563e+01, ..., -3.7215e+01,
2.1692e+01, 5.2212e+01],
[ 2.2078e+01, -5.3687e+01, 1.8090e+01, ..., 1.7484e+01,
-1.3288e+01, 5.7175e+01],
[-4.1650e+01, 7.6845e+00, -5.8877e+01, ..., -3.8248e+01,
3.3241e+01, -1.0090e+02]],
...,
[[ 1.0406e+02, 4.5779e+00, 1.9528e+01, ..., 9.4902e+01,
-4.7327e+01, 1.9806e+01],
[-6.4107e-01, 1.9781e+01, 8.1636e+01, ..., 1.1129e+01,
-1.1554e+01, 8.0511e+01],
[-5.0904e+01, -2.3051e+01, -1.3511e+02, ..., -1.2289e+02,
-4.7452e+00, -6.5629e+01],
...,
[ 1.9940e+01, -6.2615e+00, 7.8518e+01, ..., 9.1174e+01,
6.4382e+01, 5.6504e+01],
[ 6.4160e+01, 1.9338e+01, 5.5032e+01, ..., 8.3636e+01,
1.9806e+01, 6.0478e+01],
[-3.1008e+01, 2.1072e+01, -9.2494e+01, ..., -6.7746e+01,
-2.8840e+01, -2.6479e+01]],
[[ 1.0009e+02, -1.0375e+02, 1.4626e+02, ..., 1.4162e+02,
-8.1808e+01, 5.4329e+01],
[ 8.4638e+01, -7.4944e+01, 1.6649e+02, ..., 1.0271e+02,
-4.5463e+01, 1.0746e+02],
[ 2.2580e+01, -7.3847e+01, -1.6349e+01, ..., 2.2575e+01,
3.1484e+01, 3.7637e+01],
...,
[ 4.8909e+01, -6.4524e+01, 1.1345e+02, ..., 1.7076e+02,
-3.4632e+01, 8.6226e+01],
[ 7.4945e+01, -2.2892e+01, 1.0654e+02, ..., 9.1755e+01,
-1.1313e+01, 6.7887e+01],
[ 3.9525e+01, -2.5273e+00, 4.4794e+01, ..., 1.0132e+02,
1.6300e+01, 2.4174e+01]],
[[-1.7807e+01, 4.4067e+01, -3.7952e+01, ..., 4.8095e+01,
-1.5301e+00, -4.9389e+01],
[ 3.6842e+01, -1.8586e+01, -2.0799e+01, ..., -8.2640e+00,
3.0608e+01, 1.9010e+01],
[ 6.1288e+01, -9.8545e+01, 1.1785e+02, ..., 1.0136e+01,
1.0743e+01, 4.5468e+01],
...,
[-5.4218e+01, 1.3983e+01, -3.2653e+01, ..., -1.4787e+00,
-3.3644e+01, -1.8941e+01],
[-6.9085e+01, 3.5651e+01, -3.8423e+00, ..., -8.5973e+00,
-1.5549e+01, 7.0765e+00],
[ 1.9282e+01, -5.4757e+01, 3.7907e+01, ..., 1.0108e+02,
9.7316e+00, 3.2104e+01]]],
[[[-1.8460e+01, 5.9338e+00, -3.9348e+01, ..., 1.7278e+01,
6.4247e+01, -5.5950e+00],
[-9.7661e+00, 1.8798e+01, -5.9344e+01, ..., -1.8917e+01,
4.4518e+01, 5.0530e+00],
[-4.4971e+00, 3.4785e+01, 5.1154e+01, ..., 1.5462e+00,
-1.0503e+01, -3.5153e+01],
...,
[-3.0752e+01, 1.4098e+01, -4.5861e+00, ..., 1.2329e+01,
-4.3033e+01, 1.1641e+01],
[-1.1146e+00, 5.1451e+01, 2.6147e+01, ..., 3.8092e+01,
1.7767e-02, -6.7235e+00],
[ 2.9412e+01, 3.8293e+01, -1.5556e+01, ..., -1.7411e+01,
7.1936e+01, -4.1500e+01]],
[[-4.2373e+00, 6.2969e+01, -8.3228e+00, ..., 4.9279e+00,
5.9329e+00, -3.8541e+00],
[-1.3915e+01, 3.4167e+01, -1.4150e+01, ..., 5.7126e+00,
3.6345e-01, -3.8542e+01],
[ 1.9086e+01, 5.3793e+00, -1.6980e+01, ..., 4.0052e+01,
-4.4645e+00, 1.9094e+01],
...,
[-1.8508e+01, -1.9354e+01, -2.5103e+01, ..., -9.1884e+01,
-2.4259e+01, -1.5919e+01],
[-4.6876e+00, 9.0047e-01, -1.6324e+01, ..., -2.3734e+00,
2.7124e+00, -1.2720e+01],
[-3.3499e+01, 3.9628e+01, 9.0529e+00, ..., -4.1568e+01,
-1.1719e+01, -2.3408e+01]],
[[ 1.0876e+01, 4.3385e+01, 2.6335e+00, ..., 1.8348e+00,
2.3903e+01, 2.1180e+01],
[-1.9446e+01, 3.3384e+01, -1.5151e+01, ..., -8.6684e+00,
7.4284e+00, -6.2286e+00],
[ 9.6792e+00, 6.0227e+01, 1.7371e+01, ..., -1.2454e+01,
4.9995e+01, 1.3553e+01],
...,
[ 8.0322e+00, 1.4658e+00, 2.3550e+00, ..., 2.4562e+01,
-2.0568e+01, -3.2709e+00],
[ 1.9442e+00, -1.8387e+01, 4.7736e+01, ..., 2.9166e+01,
-2.5101e+01, 3.9218e+00],
[-7.9830e+00, 3.1662e+01, 1.7983e+01, ..., -8.2206e+00,
2.6943e+01, -4.2212e+01]],
...,
[[ 3.2115e+01, -4.0239e+01, -2.7550e+00, ..., 3.7927e+01,
5.0370e+01, 9.1153e+00],
[ 1.1850e+01, -2.7978e+01, 3.8883e+01, ..., 1.8269e+01,
-1.4239e+01, 1.8116e+01],
[ 1.3395e+01, -1.0648e+01, -4.1921e+00, ..., -5.4046e+01,
2.0442e+01, 4.3137e+00],
...,
[ 1.7122e+01, -1.4179e+01, 2.7223e+01, ..., 8.1335e+01,
-2.0505e+01, 3.7946e+01],
[ 2.5938e+00, -1.8541e+01, 5.1562e+01, ..., -6.2957e+00,
-2.2272e+01, 2.5197e+01],
[-3.9724e+01, 2.0131e+01, 8.5557e-01, ..., -8.9338e+00,
1.1503e+01, 3.7376e+01]],
[[ 1.1152e+02, 3.9287e+01, -8.6246e+00, ..., 7.7151e+01,
9.5945e+01, 1.2983e+01],
[ 2.1957e+01, 9.7303e+00, -1.9517e-01, ..., 3.3867e+01,
-2.1911e+01, 6.1381e+01],
[-3.7790e+01, -4.9791e+01, -8.1265e-01, ..., -1.2081e+00,
-6.7745e+01, -3.8157e+01],
...,
[ 3.6969e+01, -2.9686e+01, 1.9623e+01, ..., 6.0836e+01,
4.4620e+01, 4.8730e+01],
[ 1.0805e+01, 1.4358e+01, 5.7966e+00, ..., 2.5286e+00,
3.2827e+01, -1.3775e+01],
[-2.7824e+00, -4.9491e+01, 2.1349e+01, ..., 7.0141e+01,
-4.2625e+01, 1.5289e+00]],
[[ 3.9041e+01, -7.7993e+00, 1.1389e+01, ..., -3.1484e+01,
7.5961e+00, 2.3051e+00],
[ 2.2387e+01, 3.5499e+01, 2.0807e+01, ..., 3.6273e+01,
6.2103e+00, -1.8743e+01],
[ 2.7219e+01, -2.7371e+01, -4.6367e+01, ..., 1.0128e+01,
-2.8173e+01, 3.6750e+01],
...,
[ 1.0475e+01, -2.0231e+01, 1.0387e+01, ..., 1.3276e+01,
4.2836e+01, -1.1328e+01],
[ 1.6066e+01, -2.5348e+01, -2.1298e+01, ..., -2.7432e+01,
1.3164e+01, 9.6544e-01],
[ 6.3008e+00, -3.2814e+01, 9.3471e+00, ..., -1.3216e+01,
-1.7597e+01, 3.6267e+01]]]], dtype=torch.float64)
Kronecker product
For computing the Kronecker product one can either use the “**” operator or the method torchtt.kron().
[10]:
print(x**y)
print(A**A)
TT with sizes and ranks:
N = [10, 10, 10, 10, 10, 10, 10, 10]
R = [1, 4, 4, 4, 1, 2, 2, 2, 1]
Device: cpu, dtype: torch.float64
#entries 520 compression 5.2e-06
TT-matrix with sizes and ranks:
M = [10, 10, 10, 10, 10, 10, 10, 10]
N = [10, 10, 10, 10, 10, 10, 10, 10]
R = [1, 2, 3, 4, 1, 2, 3, 4, 1]
Device: cpu, dtype: torch.float64
#entries 4800 compression 4.8e-13
Norm
Frobenius norm of a tensor \(||\mathsf{x}||_F^2 = \sum\limits_{i_1,...,i_d} \mathsf{x}_{i_1...i_d}\) can be directly domputed from a TT decomposition.
[11]:
print(y.norm())
print(A.norm())
tensor(577306.9677, dtype=torch.float64)
tensor(10317.9978, dtype=torch.float64)
Dot product and summing along modes
One can sum alonf dimensions in torchtt. The function is torchtt.TT.sum() and can be used without arguments to sum along all dimensions, returning a scalar:
[12]:
print('sum() result ', y.sum())
print('Must be equal to ', tn.sum(y.full()))
sum() result tensor(49995000.0000, dtype=torch.float64)
Must be equal to tensor(49995000.0000, dtype=torch.float64)
If a list of modes is additionally provided, the summing will be performed along the given modes and a torchtt.TT object is returned.
[13]:
print(x.sum(1))
print(x.sum([0,1,3]))
print(A.sum([1,2]))
TT with sizes and ranks:
N = [10, 10, 10]
R = [1, 4, 4, 1]
Device: cpu, dtype: torch.float64
#entries 240 compression 0.24
TT with sizes and ranks:
N = [10]
R = [1, 1]
Device: cpu, dtype: torch.float64
#entries 10 compression 1.0
TT-matrix with sizes and ranks:
M = [10, 10]
N = [10, 10]
R = [1, 2, 1]
Device: cpu, dtype: torch.float64
#entries 400 compression 0.04
Dot product between 2 tensors is also possible using the function tortchtt.dot().
[14]:
print(tntt.dot(y,y))
tensor(3.3328e+11, dtype=torch.float64)
Dot product can be performed between 2 tensors of different mode lengths. The modes alonnd the dot product is performed must be equal. And they are given as a list of integers as an additional argument. The modes given are relative to the first tensor. The returned value is a torchtt.TT instance.
[15]:
t1 = tntt.randn([4,5,6,7,8,9],[1,2,4,4,4,4,1])
t2 = tntt.randn([5,7,9],[1,3,3,1])
print(tntt.dot(t1,t2,[1,3,5]))
TT with sizes and ranks:
N = [4, 6, 8]
R = [1, 2, 12, 1]
Device: cpu, dtype: torch.float64
#entries 248 compression 1.2916666666666667
Reshaping
Given a tensor in the TT format, one can reshape it similarily as in pytorch or numpy. The method is torchtt.reshape() and it taks as argument a torchtt.TT object, the new shape, the relative accuracy epsilon and a maximum rank. The last 2 are optional. The method also performs rounding up to the desired accuracy.
[16]:
q = tntt.TT(tn.reshape(tn.arange(2*3*4*5*7*3, dtype = tn.float64),[2,3,4,5,7,3]))
# perform a series of reshapes
w = tntt.reshape(q,[12,10,21])
print(w)
w = tntt.reshape(w,[360,7])
print(w)
w = tntt.reshape(w,[2,3,4,5,7,3])
print('Error ',(w-q).norm()/q.norm())
TT with sizes and ranks:
N = [12, 10, 21]
R = [1, 4, 2, 1]
Device: cpu, dtype: torch.float64
#entries 170 compression 0.06746031746031746
TT with sizes and ranks:
N = [360, 7]
R = [1, 4, 1]
Device: cpu, dtype: torch.float64
#entries 1468 compression 0.5825396825396826
Error tensor(1.8163e-15, dtype=torch.float64)
Reshape works also for TT matrices. However there are some restrictions such as the merging or spliting of the dimensions must happen within the same core for both row/column indices.
[17]:
A = tntt.randn([(4,8),(6,4),(5,6),(8,8)],[1,2,3,2,1])
B = tntt.reshape(A,[(2,4),(6,4),(10,12),(8,8)])
print(B)
B = tntt.reshape(B,[(60,32),(16,48)])
print(B)
B = tntt.reshape(B,[(4,8),(6,4),(5,6),(8,8)])
print('Error ',(B-A).norm()/A.norm())
# this will not work: tntt.reshape(A,[(24,4),(5,16),(8,24)])
TT-matrix with sizes and ranks:
M = [2, 6, 10, 8]
N = [4, 4, 12, 8]
R = [1, 8, 12, 2, 1]
Device: cpu, dtype: torch.float64
#entries 5376 compression 0.0036458333333333334
TT-matrix with sizes and ranks:
M = [60, 16]
N = [32, 48]
R = [1, 24, 1]
Device: cpu, dtype: torch.float64
#entries 64512 compression 0.04375
Error tensor(6.5821e-15, dtype=torch.float64)