Note
Run this notebook in Google Colab or see the source on GitHub.
Random tensors in the TT format
torchtt.randn(N, R, var) returns a random tensor in the tensor-train (TT) format with shape N and TT rank R. The entries of the TT cores are independent and normally distributed, and they are scaled so that every entry of the full tensor has mean 0 and variance var. An entry of the full tensor is a sum of products of the core entries, so it is only approximately normally distributed.
This notebook checks the variance for a few shapes and ranks by computing the full tensors.
Imports:
[1]:
import torch as tn
try:
import torchtt as tntt
except:
print('Installing torchTT...')
%pip install git+https://github.com/ion-g-ion/torchTT
import torchtt as tntt
Variance 1
A 5-dimensional tensor of shape \(30 \times \cdots \times 30\) with TT rank \((1,8,16,16,8,1)\):
[2]:
x = tntt.randn([30]*5, [1,8,16,16,8,1])
x_full = x.full()
print('Var = ', tn.std(x_full).numpy()**2, ' (has to be comparable to 1.0)')
Var = 0.9535599014767242 (has to be comparable to 1.0)
Variance 4
The same shape and rank with var = 4.0:
[3]:
x = tntt.randn([30]*5, [1,8,16,16,8,1], var = 4.0)
x_full = x.full()
print('Var = ', tn.std(x_full).numpy()**2, ' (has to be comparable to 4.0)')
Var = 5.3232008276129 (has to be comparable to 4.0)
Variance 0.01
The same shape and rank with var = 0.01:
[4]:
x = tntt.randn([30]*5, [1,8,16,16,8,1], var = 0.01)
x_full = x.full()
print('Var = ', tn.std(x_full).numpy()**2, ' (has to be comparable to 0.01)')
Var = 0.010857722210434208 (has to be comparable to 0.01)
Longer train
A 7-dimensional tensor of shape \(10 \times \cdots \times 10\) with all inner ranks equal to 4:
[5]:
x = tntt.randn([10]*7, [1,4,4,4,4,4,4,1], var = 1.0)
x_full = x.full()
print('Var = ', tn.std(x_full).numpy()**2, ' (has to be comparable to 1.0)')
Var = 0.7857122980334933 (has to be comparable to 1.0)
Remark
The variance of the entries equals var in expectation, but the empirical variance of a single tensor fluctuates around it. All entries of a TT tensor are built from the same cores, so they are not independent, and the fluctuations are larger than for a dense tensor with independent entries.