torchtt package
Subpackages
Submodules
torchtt.cpp module
Module for the C++ backend.
torchtt.errors module
Contains the errors used in the torchtt package.
- exception torchtt.errors.IncompatibleTypes[source]
Bases:
ExceptionThe function arguments are not compatible.
Usually means that a TT matrix was passed as argument instead of a TT tensor (or viceversa).
- exception torchtt.errors.InvalidArguments[source]
Bases:
ExceptionThe arguments are not valid.
The arguments passed are not of valid type.
torchtt.grad module
Adds automatic differentiation (AD) functionality to torchtt.
The TT cores are torch.Tensor objects, so derivatives with respect to them are computed by the PyTorch autograd engine.
The functions of this module are shortcuts for it and there is no other difference:
The following two computations give the same gradients:
import torch
import torchtt
x = torchtt.randn([4, 5, 6], [1, 2, 2, 1])
# with torchtt.grad
torchtt.grad.watch(x)
g = torchtt.grad.grad(x.norm(), x)
# with PyTorch
cores = [c.detach().requires_grad_(True) for c in x.cores]
g = torch.autograd.grad(torchtt.TT(cores).norm(), cores)
Use torch.autograd.grad() directly when the gradients of successive calls must not add up, when higher derivatives are needed (create_graph=True) or when the cores are computed from other tensors that require gradients (they are not leaf tensors, and grad() returns None for them).
The layers in torchtt.nn store their cores as torch.nn.Parameter and are trained with the PyTorch optimizers, without this module.
The gradient with respect to the cores is not the gradient with respect to the tensor.
For optimization on the manifold of TT tensors with fixed rank use torchtt.manifold.riemannian_gradient(), which returns the gradient with respect to the tensor projected on the tangent space, as a TT tensor.
What can be differentiated.
The operations that keep the ranks fixed are compositions of PyTorch operations and can be differentiated:
+, -, the multiplication with a scalar, the exact products * and @ (without rounding), the Kronecker product **, torchtt.TT.full(), torchtt.TT.norm(), torchtt.TT.sum(), indexing and slicing, torchtt.TT.apply_mask(), torchtt.dot() and torchtt.bilinear_form().
The operations that choose the ranks from the data (with truncated SVDs or iterations) are not supported:
the TT decomposition torchtt.TT(full_tensor), torchtt.TT.round(), torchtt.TT.to_qtt(), torchtt.reshape(), torchtt.permute(), the division /, the products torchtt.matvec(), torchtt.matmat() and torchtt.hadamard(), and the cross interpolation of torchtt.interpolate.
Most of them raise an error if the cores require gradients; the others (e.g. method='swap' in the products) can return NaN gradients.
- torchtt.grad.grad(val, tens, core_indices=None)[source]
Compute the gradient w.r.t. the cores of the given TT-tensor (or TT-matrix). The cores must be watched with
watch()beforevalis computed.It calls
val.backward()and returnsc.gradfor the cores, therefore:the gradients are added to the ones of previous calls (set
c.grad = Nonefor the cores to start again from zero),valcan be differentiated only once (the recorded graph is freed).
For the first call the result is the same as
torch.autograd.grad(val, tens.cores), which has none of these side effects.- Parameters:
val (torch.tensor) – Scalar tensor that has to be differentiated.
tens (torchtt.TT) – The given tensor.
core_indices (list[int], optional) – The list of cores to construct the gradient. If None is provided, all the cores are watched. Defaults to None.
- Returns:
the list of cores representing the derivative of the expression w.r.t the tensor.
- Return type:
list[torch.tensor]
- torchtt.grad.grad_list(val, tensors, all_in_one=True)[source]
Compute the gradient w.r.t. the cores of several given TT-tensors (or TT-oeprators). Watch must be called on all of them beforehand. Like
grad(), it callsval.backward(), so the gradients are added to the ones of previous calls.- Parameters:
val (torch.tensor) – scalar tensor to be differentiated.
tensors (list[torch.TT]) – the tensors with respect to which the differentiation is made.
all_in_one (bool, optional) – Put all the cores in one list or create a list of lists with the cores. Defaults to True.
- Returns:
the resulting derivatives.
- Return type:
list[list[torchtt.TT]]
- torchtt.grad.unwatch(tens)[source]
Cancel the autograd graph recording. The flag
requires_gradof the cores is set to False. The gradients stored inc.gradfrom previous calls ofgrad()are kept.- Parameters:
tens (torchtt.TT) – the tensor.
- torchtt.grad.watch(tens, core_indices=None)[source]
Watch the TT-cores of a given tensor. Necessary for autograd.
The flag
requires_gradof the cores is set in place, so PyTorch records every following operation with the tensor. Call it before computing the value that is differentiated. It is the same astens.cores[i].requires_grad_(True)for the watched cores. The cores must be leaf tensors (not computed from other tensors that require gradients), otherwisegrad()returns None for them.Examples
x = torchtt.randn([4, 5, 6], [1, 2, 2, 1]) torchtt.grad.watch(x) val = (x * x).sum() grad_cores = torchtt.grad.grad(val, x)
- Parameters:
tens (torchtt.TT) – the TT-object to be watched.
core_indices (list[int], optional) – The list of cores to be watched. If None is provided, all the cores are watched. Defaults to None.
- torchtt.grad.watch_list(tensors)[source]
Watch the TT-cores for amultiple tensors givgen in a list. Necessary for autograd. Same as
watch()for every tensor of the list.- Parameters:
tensors (list[torchtt.TT]) – the list of tensors to be wtched for autograd.
torchtt.interpolate module
Implements cross approximation methods in the TT format [Oseledets and Tyrtyshnikov, 2010], based on the density matrix renormalization group (DMRG) [Savostyanov and Oseledets, 2011] and the alternating minimal energy (AMEn) [Dolgov and Savostyanov, 2014] schemes.
- class torchtt.interpolate.AmenCrossCallbacks(function, eval_mv, x, N, dtype, device)[source]
Bases:
AmenCallbacks
- torchtt.interpolate.dmrg_cross(function, N, eps=1e-09, nswp=10, x_start=None, kick=2, dtype=torch.float64, device=None, eval_vect=True, rmax=9223372036854775807, verbose=False, callback=None)[source]
Approximate a tensor in the TT format given that the individual entries are given using a function. The function is given as a function handle taking as arguments a matrix of integer indices.
Example
func = lambda I: 1/(2+I[:,0]+I[:,1]+I[:,2]+I[:,3]).to(dtype=torch.float64) N = [20]*4 x = torchtt.interpolate.dmrg_cross(func, N, eps = 1e-7)
- Parameters:
function (Callable) – function handle.
N (list[int]) – the shape of the tensor.
eps (float, optional) – the relative accuracy. Defaults to 1e-9.
nswp (int, optional) – number of iterations. Defaults to 20.
x_start (torchtt.TT, optional) – initial approximation of the output tensor (None coresponds to random initialization). Defaults to None.
kick (int, optional) – enrichment rank. Defaults to 2.
dtype (torch.dtype, optional) – the dtype of the result. Defaults to tn.float64.
device (torch.device, optional) – the device where the approximation will be stored. Defaults to None.
eval_vect (bool, optional) – not yet implemented. Defaults to True.
rmax (int, optional) – the maximum rank. Defaults to the maximum possible integer.
verbose (bool, optional) – display debug information to the console. Defaults to False.
callback (Callable, optional) – optional hook invoked at the end of every sweep as
callback(tt, sweep, error), wherettis the current approximation (torchtt.TT),sweepis the 0-based sweep index (int) anderroris the convergence metric for that sweep (float). If it returnsFalsethe sweeping is stopped early; any other return value continues. Useful for logging or custom stopping criteria. Defaults to None.
- Returns:
the result.
- Return type:
- torchtt.interpolate.function_interpolate(function, x, eps=1e-09, start_tens=None, nswp=20, kick=2, kick2=0, dtype=torch.float64, rmax=9223372036854775807, method='dmrg', verbose=False, callback=None)[source]
Interpolate a function using tensor train cross approximation [Oseledets and Tyrtyshnikov, 2010, Savostyanov and Oseledets, 2011].
Prefer trying
method='amen'when function evaluations are expensive or mode sizes are large; considermethod='dmrg'when evaluations are cheap and mode sizes are small. See Choosing an interpolation method for the tradeoffs and convergence guidance.- Parameters:
function (Callable) – Function to interpolate.
x (torchtt.TT or list[torchtt.TT]) – The points at which to evaluate the function.
eps (float, optional) – The desired relative error. Defaults to 1e-9.
start_tens (torchtt.TT, optional) – Initial tensor train approximation. Defaults to None.
nswp (int, optional) – Number of sweeps. Defaults to 20.
kick (int, optional) – Rank enrichment. Defaults to 2.
kick2 (int, optional) – Secondary rank enrichment (meant for amen method). Defaults to 0.
dtype (torch.dtype, optional) – The datatype of the result. Defaults to tn.float64.
rmax (int, optional) – Maximum allowed rank. Defaults to sys.maxsize.
method (str, optional) – Method to use: ‘dmrg’ (density matrix renormalization group) or ‘amen’ (alternating minimal energy). Defaults to ‘dmrg’.
verbose (bool, optional) – If True, display information. Defaults to False.
callback (Callable, optional) – optional hook invoked at the end of every sweep as
callback(tt, sweep, error), wherettis the current approximation (torchtt.TT),sweepis the 0-based sweep index (int) anderroris the convergence metric for that sweep (float). If it returnsFalsethe sweeping is stopped early; any other return value continues. Useful for logging or custom stopping criteria. Defaults to None.
- Raises:
ValueError – If the method is not ‘dmrg’ or ‘amen’.
- Returns:
The interpolated tensor.
- Return type:
torchtt.manifold module
Manifold gradient module: Riemannian gradients and tangent-space projections on the manifold of TT tensors with fixed rank [Steinlechner, 2016].
- torchtt.manifold.riemannian_gradient(x, func)[source]
Compute the Riemannian gradient using automatic differentiation (AD).
- Parameters:
x (torchtt.TT) – the point on the manifold where the gradient is computed.
func ([type]) – function that has to be differentiated. The function takes as only argument torchtt.TT instances.
- Returns:
the gradient projected on the tangent space of x.
- Return type:
- torchtt.manifold.riemannian_projection(Xspace, z)[source]
Project the tensor z onto the tangent space defined at xspace
- Parameters:
Xspace (torchtt.TT) – the target where the tensor should be projected.
z (torchtt.TT) – the tensor that should be projected.
- Raises:
IncompatibleTypes – Both must be of same type.
- Returns:
the projection.
- Return type:
torchtt.methods module
Methods for the products that truncate the rank of the result: torchtt.matvec(), torchtt.matmat() and torchtt.hadamard().
Each function computes an approximation of the product followed by rounding, e.g. (A @ x).round(eps, rmax).
The algorithm is chosen with the method argument, which is either the name of the method or an instance of one of the classes below.
Use an instance to change the parameters of the method:
import torchtt as tntt
y = tntt.matvec(A, x, eps=1e-10) # default method ('dmrg')
y = tntt.matvec(A, x, eps=1e-10, method='direct') # other method, default parameters
y = tntt.matvec(A, x, eps=1e-10, method=tntt.methods.DMRG(nswp=40)) # other parameters
The methods and the products they implement:
Method |
|
|
|
Use it when |
|---|---|---|---|---|
|
yes |
yes (default) |
yes |
the product rank |
|
yes (default) |
no |
yes (default) |
the product rank is large and the result has a much smaller rank. |
|
yes |
yes |
no |
the product of TT matrices has a large product rank and a result of small rank (real tensors only). |
|
yes |
yes |
yes |
a non-iterative and deterministic algorithm is needed. |
See Efficient products for a longer discussion.
- class torchtt.methods.AMEn(nswp: int = 22, kickrank: int = 4, kick2: int = 0)[source]
Bases:
objectAlternating minimal energy (AMEn) method [Dolgov and Savostyanov, 2014].
Updates one core of the result at a time and enriches it with an approximation of the residual, without forming the exact product. Like
DMRG, it pays off when the result has a much smaller rank than the product rank, and it is the only such method for the product of TT matrices. For the matrix-vector productDMRGis usually faster, since the cost of AMEn grows faster with the ranks of the operands. Without an initial guess it starts from rank 1 and the rank grows by at mostkickrank + kick2per sweep. A result with larger rank than1 + nswp * (kickrank + kick2)is therefore not reached and the returned tensor is not accurate: increasenswporkickrank, or pass aninitialguess.Implements the matrix-vector product and the product of TT matrices, for real tensors only.
- Parameters:
nswp (int, optional) – maximum number of sweeps. Defaults to 22.
kickrank (int, optional) – rank enrichment with the residual after each local update. Defaults to 4.
kick2 (int, optional) – additional rank enrichment with random vectors. Defaults to 0.
- Raises:
InvalidArguments – nswp must be a positive integer.
InvalidArguments – kickrank and kick2 must be non-negative integers.
- kick2: int = 0
- kickrank: int = 4
- matmat(A, B, eps, rmax, initial, verbose)[source]
Product of two TT matrices. Called by
torchtt.matmat(), which also checks the arguments.- Parameters:
A (torchtt.TT) – the first TT matrix.
B (torchtt.TT) – the second TT matrix.
eps (float) – relative accuracy.
rmax (int) – maximum rank.
initial (None | torchtt.TT) – initial guess of the result (None means a zero TT matrix of rank 1).
verbose (bool) – print information about the sweeps.
- Raises:
IncompatibleTypes – AMEn does not support complex tensors.
- Returns:
the result.
- Return type:
- matvec(A, x, eps, rmax, initial, verbose)[source]
Matrix-vector product. Called by
torchtt.matvec(), which also checks the arguments.- Parameters:
A (torchtt.TT) – the TT matrix.
x (torchtt.TT) – the TT tensor.
eps (float) – relative accuracy.
rmax (int) – maximum rank.
initial (None | torchtt.TT) – initial guess of the result (None means a zero tensor of rank 1).
verbose (bool) – print information about the sweeps.
- Raises:
IncompatibleTypes – AMEn does not support complex tensors.
- Returns:
the result.
- Return type:
- nswp: int = 22
- class torchtt.methods.DMRG(nswp: int = 20, kickrank: int = 4, use_cpp: bool = True)[source]
Bases:
objectDensity matrix renormalization group (DMRG) method [Oseledets, 2011].
Optimizes two neighbouring cores of the result at a time, without forming the exact product. The cost depends on the rank of the result and not on the product of the ranks of the operands, so it pays off when the result has a much smaller rank than the product rank. If the result cannot be compressed,
Directis faster. The iteration starts from a random tensor (or from theinitialguess) and stops when the cores change by less thanepsor afternswpsweeps. The ranks of the result can exceed the smallest ones for the accuracyepsby up tokickrank; usetorchtt.TT.round()if this matters.Implements the matrix-vector product (with a C++ implementation) and the elementwise product.
- Parameters:
nswp (int, optional) – maximum number of sweeps. Defaults to 20.
kickrank (int, optional) – rank enrichment with random vectors after each local update. Defaults to 4.
use_cpp (bool, optional) – use the C++ implementation if available (only for the matrix-vector product). It is up to 2x faster for small ranks and about as fast as Python for larger ones. Defaults to True.
- Raises:
InvalidArguments – nswp must be a positive integer.
InvalidArguments – kickrank must be a non-negative integer.
- hadamard(x, y, eps, rmax, initial, verbose)[source]
Elementwise product. Called by
torchtt.hadamard(), which also checks the arguments. TT matrices are multiplied as TT tensors with the mode sizesM[k]*N[k].- Parameters:
x (torchtt.TT) – the first operand.
y (torchtt.TT) – the second operand.
eps (float) – relative accuracy.
rmax (int) – maximum rank.
initial (None | torchtt.TT) – initial guess of the result (None means random). It is not modified.
verbose (bool) – print information about the sweeps.
- Returns:
the result.
- Return type:
- kickrank: int = 4
- matvec(A, x, eps, rmax, initial, verbose)[source]
Matrix-vector product. Called by
torchtt.matvec(), which also checks the arguments.- Parameters:
A (torchtt.TT) – the TT matrix.
x (torchtt.TT) – the TT tensor.
eps (float) – relative accuracy.
rmax (int) – maximum rank.
initial (None | torchtt.TT) – initial guess of the result (None means random). It is not modified.
verbose (bool) – print information about the sweeps.
- Returns:
the result.
- Return type:
- nswp: int = 20
- use_cpp: bool = True
- class torchtt.methods.Direct[source]
Bases:
objectCompute the exact product and round it with
torchtt.TT.round()[Oseledets, 2011].The rank of the exact product is the product of the ranks of the operands and the rounding costs \(\mathcal{O}(dn(r_ar_b)^3)\). It is the best choice when the product rank \(r_ar_b\) is moderate (up to a few hundred) or when the result cannot be compressed much, since no iterations are needed. Its cost and memory become large for large product ranks; use
DMRGorAMEnif the result has a much smaller rank. The result is deterministic and its rank is the smallest one for the accuracyeps.Implements all the products and does not use an initial guess.
- hadamard(x, y, eps, rmax, initial, verbose)[source]
Elementwise product. Called by
torchtt.hadamard(), which also checks the arguments.- Parameters:
x (torchtt.TT) – the first operand.
y (torchtt.TT) – the second operand.
eps (float) – relative accuracy.
rmax (int) – maximum rank.
initial (None) – not used, must be None.
verbose (bool) – not used.
- Returns:
the result.
- Return type:
- matmat(A, B, eps, rmax, initial, verbose)[source]
Product of two TT matrices. Called by
torchtt.matmat(), which also checks the arguments.- Parameters:
A (torchtt.TT) – the first TT matrix.
B (torchtt.TT) – the second TT matrix.
eps (float) – relative accuracy.
rmax (int) – maximum rank.
initial (None) – not used, must be None.
verbose (bool) – not used.
- Returns:
the result.
- Return type:
- matvec(A, x, eps, rmax, initial, verbose)[source]
Matrix-vector product. Called by
torchtt.matvec(), which also checks the arguments.- Parameters:
A (torchtt.TT) – the TT matrix.
x (torchtt.TT) – the TT tensor.
eps (float) – relative accuracy.
rmax (int) – maximum rank.
initial (None) – not used, must be None.
verbose (bool) – not used.
- Returns:
the result.
- Return type:
- class torchtt.methods.Swap[source]
Bases:
objectProduct by swapping neighbouring cores [Michailidis et al., 2025].
The cores of the first operand are moved through the cores of the second one with \(d(d-1)/2\) swaps of neighbouring cores, each truncated with an SVD. There are no iterations and no initial guess, so the result is deterministic. Every swap is truncated with the relative accuracy
eps, so the error of the result can be larger thaneps. Like forDMRG, the cost depends mostly on the ranks of the result, but it grows quadratically with the number of dimensions and quickly with the mode sizes, and it is usually slower thanDMRG. When the intermediate tensors have large ranks (e.g. for smooth functions in the QTT format), the ranks of the result can be much larger than needed.Implements all the products and does not use an initial guess.
- hadamard(x, y, eps, rmax, initial, verbose)[source]
Elementwise product. Called by
torchtt.hadamard(), which also checks the arguments.- Parameters:
x (torchtt.TT) – the first operand.
y (torchtt.TT) – the second operand.
eps (float) – relative accuracy of each swap.
rmax (int) – maximum rank.
initial (None) – not used, must be None.
verbose (bool) – not used.
- Returns:
the result.
- Return type:
- matmat(A, B, eps, rmax, initial, verbose)[source]
Product of two TT matrices. Called by
torchtt.matmat(), which also checks the arguments.- Parameters:
A (torchtt.TT) – the first TT matrix.
B (torchtt.TT) – the second TT matrix.
eps (float) – relative accuracy of each swap.
rmax (int) – maximum rank.
initial (None) – not used, must be None.
verbose (bool) – not used.
- Returns:
the result.
- Return type:
- matvec(A, x, eps, rmax, initial, verbose)[source]
Matrix-vector product. Called by
torchtt.matvec(), which also checks the arguments.- Parameters:
A (torchtt.TT) – the TT matrix.
x (torchtt.TT) – the TT tensor.
eps (float) – relative accuracy of each swap.
rmax (int) – maximum rank.
initial (None) – not used, must be None.
verbose (bool) – not used.
- Returns:
the result.
- Return type:
torchtt.nn module
Implements a basic TT layer for constructing deep TT networks.
- class torchtt.nn.AffineTransform(dim)[source]
Bases:
TransformAffine transformation block: z = R(theta) diag(exp(a)) x + b
Where R(theta) is a rotation matrix parameterized by Givens angles. det J = prod(exp(a))
- forward(x, params)[source]
Applies the transformation. :param x: Input tensor of shape (…, dim). :type x: torch.Tensor :param params: Parameter tensor of shape (…, input_requirement). :type params: torch.Tensor
- Returns:
Transformed tensor. det_jac (torch.Tensor or float): Determinant of the Jacobian.
- Return type:
z (torch.Tensor)
- class torchtt.nn.ComposedTransform(transforms)[source]
Bases:
TransformChains multiple transformations.
- forward(x, params)[source]
Applies the transformation. :param x: Input tensor of shape (…, dim). :type x: torch.Tensor :param params: Parameter tensor of shape (…, input_requirement). :type params: torch.Tensor
- Returns:
Transformed tensor. det_jac (torch.Tensor or float): Determinant of the Jacobian.
- Return type:
z (torch.Tensor)
- class torchtt.nn.CompressedTTLayer(N_in, N_out, R_layer, R_output, activation=<built-in method relu of type object>, bias=True, dtype=torch.float32)[source]
Bases:
ModuleA Tensor Train (TT) neural network layer that operates directly on TT objects and applies nonlinear activation between TT cores during multiplication.
This layer is inspired by nonlinear tensor-train formats for deep neural networks [Wang et al., 2021]. Instead of computing the full dense tensor, it performs a fast matrix-vector-like multiplication of the layer’s TT-matrix (TTM) weights with the input TT object. The layer natively applies a nonlinear activation and an optional bias to the intermediate core representations during the contraction sweep. The intermediate representations are orthogonalized and truncated to maintain the compression, ensuring the output TT object’s ranks are strictly bounded by
R_output.- forward(x)[source]
Forward pass for the CompressedTTLayer.
Computes the operation by multiplying the input TT with the layer’s TT-matrix (TTM) weights using a right-to-left sweep. During this sweep, the intermediate bias is added and the activation is applied to the core before the core is orthogonalized and truncated (using a singular value decomposition, SVD) to strictly enforce the per-bond rank limits given by
R_output.- Parameters:
x (torchtt.TT) – The input Tensor Train object.
- Returns:
The output Tensor Train object representing the nonlinearly compressed forward pass, with ranks bounded by
R_output.- Return type:
- class torchtt.nn.LinearLayerTT(size_in, size_out, rank, dtype=torch.float32, initializer='He')[source]
Bases:
ModuleBasic class for TT layers. See Tensorizing Neural Networks for a detailed description. It can be used similarily to any layer from torch.nn. The output of the layer is \(\mathcal{LTT}(\mathsf{x}) =\mathsf{Wx}+\mathsf{b}\), where the tensor operator \(\mathsf{W}\) is represented in the TT format (with a fixed prescribed rank).
- forward(x)[source]
Computes the output of the layer for the given input.
Supports trailing dimensiond broadcasting. If the input of the layer is set to
[M1,...,Md]and a tensor od shape[...,M1,...,Md]is provided then the multiplication is performed along the last d dimensions.- Parameters:
x (torch.tensor) – input of the layer.
- Returns:
output of the layer.
- Return type:
torch.tensor
- class torchtt.nn.Rank1Shear(dim, degree=2, clip=None)[source]
Bases:
TransformVolume-preserving rank-1 nonlinear shear: z = x + u * g(v^T x + c)
To ensure volume preservation (det J = 1), u is projected orthogonally to v such that v^T u = 0. The scalar function g(t) is a polynomial of degree D with no constant term: g(t) = sum_{p=1}^D alpha_p t^p. If clipping is applied, the displacement is squashed to prevent numerical overflow: g_clip(t) = clip * tanh(g(t) / clip).
- forward(x, params)[source]
Applies the transformation. :param x: Input tensor of shape (…, dim). :type x: torch.Tensor :param params: Parameter tensor of shape (…, input_requirement). :type params: torch.Tensor
- Returns:
Transformed tensor. det_jac (torch.Tensor or float): Determinant of the Jacobian.
- Return type:
z (torch.Tensor)
- class torchtt.nn.SinhArcsinhWarp(dim)[source]
Bases:
TransformElementwise sinh-arcsinh warp: z_i = sinh(e^{s_i} asinh(x_i) + b_i)
- forward(x, params)[source]
Applies the transformation. :param x: Input tensor of shape (…, dim). :type x: torch.Tensor :param params: Parameter tensor of shape (…, input_requirement). :type params: torch.Tensor
- Returns:
Transformed tensor. det_jac (torch.Tensor or float): Determinant of the Jacobian.
- Return type:
z (torch.Tensor)
- class torchtt.nn.TTDensityLayer(N, R, basis, transform=None, dtype=torch.float32)[source]
Bases:
ModuleA TT Density Layer evaluating \(p(x) = p_{\mathrm{ref}}(T(x)) \, |\det J_T(x)|\).
- forward(tts, x)[source]
Define the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Moduleinstance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
- input_requireemnt()
- property integration_weights
- class torchtt.nn.Transform(dim)[source]
Bases:
ModuleBase class for diffeomorphism transformations used in TTDensityLayer.
- forward(x, params)[source]
Applies the transformation. :param x: Input tensor of shape (…, dim). :type x: torch.Tensor :param params: Parameter tensor of shape (…, input_requirement). :type params: torch.Tensor
- Returns:
Transformed tensor. det_jac (torch.Tensor or float): Determinant of the Jacobian.
- Return type:
z (torch.Tensor)
- class torchtt.nn.TriangularPolyShear(dim, degree)[source]
Bases:
TransformKnothe-Rosenblatt-style triangular polynomial shear: z_i = x_i + sum_{j>i} q_{ij}(x_j)
Each q_{ij}(t) is a polynomial of degree D_poly with no constant term: q_{ij}(t) = sum_{p=1}^{D_poly} beta_{ij,p} t^p. det J = 1.
- forward(x, params)[source]
Applies the transformation. :param x: Input tensor of shape (…, dim). :type x: torch.Tensor :param params: Parameter tensor of shape (…, input_requirement). :type params: torch.Tensor
- Returns:
Transformed tensor. det_jac (torch.Tensor or float): Determinant of the Jacobian.
- Return type:
z (torch.Tensor)
- class torchtt.nn.TriangularShear(dim)[source]
Bases:
TransformUnit upper-triangular linear shear (completes the affine family). det J = 1.
- forward(x, params)[source]
Applies the transformation. :param x: Input tensor of shape (…, dim). :type x: torch.Tensor :param params: Parameter tensor of shape (…, input_requirement). :type params: torch.Tensor
- Returns:
Transformed tensor. det_jac (torch.Tensor or float): Determinant of the Jacobian.
- Return type:
z (torch.Tensor)
torchtt.solvers module
System solvers in the TT format, based on the alternating minimal energy (AMEn) method [Dolgov and Savostyanov, 2014].
- torchtt.solvers.amen_solve(A, b, nswp=22, x0=None, eps=1e-10, rmax=32768, max_full=256, kickrank=4, kick2=0, trunc_norm='res', local_solver=1, local_iterations=40, resets=2, verbose=False, preconditioner=None, use_cpp=True, band_diagonal=-1, use_single_precision=False)[source]
Solve a multilinear system \(\mathsf{Ax} = \mathsf{b}\) in the Tensor Train format.
This method implements the alternating minimal energy (AMEn) algorithm from Sergey V Dolgov, Dmitry V Savostyanov, Alternating minimal energy methods for linear systems in higher dimensions.
Example
import torchtt A = torchtt.random([(4,4),(5,5),(6,6)],[1,2,3,1]) # create random matrix x = torchtt.random([4,5,6],[1,2,3,1]) # invent a random solution b = A @ x # compute the rhs xx = torchtt.solvers.amen_solve(A,b) # solve print((xx-x).norm()/x.norm()) # error
- Parameters:
A (torchtt.TT) – the system matrix in TT.
b (torchtt.TT) – the right hand side in TT.
nswp (int, optional) – number of sweeps. Defaults to 22.
x0 (torchtt.TT, optional) – initial guess. If None is provided the initial guess is a ones tensor. Defaults to None.
eps (float, optional) – relative residual. Defaults to 1e-10.
rmax (int | list[int], optional) – maximum rank, or one maximum per rank index. Defaults to 32768.
max_full (int, optional) – local systems strictly smaller than this are solved directly, larger ones with the iterative local solver. Defaults to 256.
kickrank (int, optional) – rank enrichment. Defaults to 4.
kick2 (int, optional) – number of random vectors added to the rank enrichment, in addition to the
kickrankvectors computed from the residual. Defaults to 0.trunc_norm (str, optional) – truncation of the local solution.
'res'keeps the smallest rank whose local residual stays below the tolerance,'fro'truncates the singular values in the Frobenius norm. Defaults to ‘res’.local_solver (int, optional) – choose local iterative solver: 1 for GMRES (generalized minimal residual) and 2 for BiCGSTAB (biconjugate gradient stabilized). Defaults to 1.
local_iterations (int, optional) – number of iterations of the local solver: GMRES iterations between two restarts, or BiCGSTAB iterations. Defaults to 40.
resets (int, optional) – number of GMRES restarts. Defaults to 2.
verbose (bool, optional) – choose whether to display or not additional information during the runtime. Defaults to False.
preconditioner (string, optional) – Choose the preconditioner for the local system. Possible values are None, ‘c’ (block Jacobi with one block per pair of rank indices) and ‘r’ (block Jacobi with one block per left rank index). No preconditioner is used if None is provided. Defaults to None.
use_cpp (bool, optional) – use the C++ implementation of AMEn if available. It has the same accuracy and is 3-7x faster on CPU, most for small TT cores. It supports only real tensors,
local_solver=1,trunc_norm='res',band_diagonal=-1and an integerrmax; other calls run the Python implementation. Defaults to True.band_diagonal (int, optional) – bandwidth of the TT cores of the matrix if they are band diagonal, -1 for dense cores. Defaults to -1.
use_single_precision (bool, optional) – solve the local systems with the iterative solver in single precision. The Python implementation also computes the local residuals in single precision, the C++ implementation computes them in the precision of the input. Defaults to False.
- Raises:
InvalidArguments – A and b must be TT instances.
InvalidArguments – Invalid preconditioner.
InvalidArguments – Invalid trunc_norm.
InvalidArguments – Solver not implemented.
IncompatibleTypes – A must be TT-matrix and b must be vector.
ShapeMismatch – A is not quadratic.
ShapeMismatch – Dimension mismatch.
- Returns:
the approximation of the solution in TT format.
- Return type:
Module contents
Provides Tensor-Train (TT) decomposition using pytorch as backend.
Contains routines for computing the TT decomposition and all the basisc linear algebra in the TT format. Additionally, GPU support can be used thanks to the pytorch backend. It also has linear solvers in TT and cross approximation as well as automatic differentiation.
- class torchtt.TT(source, shape=None, eps=1e-10, rmax=9223372036854775807)[source]
Bases:
object- property M
Return the “row” shape in case of TT matrices.
- Raises:
IncompatibleTypes – The field is_ttm is defined only for TT matrices.
- Returns:
the shape.
- Return type:
list[int]
- property N
Return the shape of a tensor or the “column” shape of a TT operator.
- Returns:
the shape.
- Return type:
list[int]
- property R
The rank of the TT decomposition. It’s length should be
len(R)==len(N)+1.- Returns:
the rank.
- Return type:
list[int]
- apply_mask(indices)[source]
Evaluate the tensor on the given index list.
Examples
x = torchtt.random([10,12,14],[1,4,5,1]) indices = torch.tensor([[0,0,0],[1,2,3],[1,1,1]]) val = x.apply_mask(indices)
- Parameters:
indices (list[list[int]]) – the index list where the tensor should be evaluated. Length is M.
- Returns:
the values of the tensor
- Return type:
torch.tensor
- clone()[source]
Clones the torchtt.TT instance. Similar to torch.tensor.clone().
- Returns:
the cloned TT object.
- Return type:
- conj()[source]
Return the complex conjugate of a tensor in TT format.
- Returns:
the complex conjugated tensor.
- Return type:
- cuda(device=None)[source]
Return a torchtt.TT object on the CUDA device by cloning all the cores on the GPU.
- Parameters:
device (torch.device, optional) – The CUDA device (None for CPU). Defaults to None.
- Returns:
The TT-object. The TT-cores are on CUDA.
- Return type:
- detach()[source]
Detaches the TT tensor. Similar to
torch.tensor.detach().- Returns:
the detached tensor.
- Return type:
- fast_matvec(other, eps=1e-12, initial=None, nswp=20, verb=False, use_cpp=True)[source]
Fast matrix vector multiplication A@x using density matrix renormalization group (DMRG) iterations [Oseledets, 2011]. Faster than traditional matvec + rounding.
Deprecated since version 0.6.0: Use
torchtt.TT.matvec()ortorchtt.matvec()withmethod='dmrg'(ormethod=torchtt.methods.DMRG(nswp=...)).- Parameters:
other (torchtt.TT) – the TT tensor.
eps (float, optional) – relative accuracy for DMRG. Defaults to 1e-12.
initial (None|torchtt.TT, optional) – an approximation of the product (None means random initial guess). Defaults to None.
nswp (int, optional) – number of DMRG iterations. Defaults to 20.
verb (bool, optional) – show info for debug. Defaults to False.
use_cpp (bool, optional) – use the C++ implementation if available. Defaults to True.
- Raises:
InvalidArguments – Second operand has to be TT object.
IncompatibleTypes – First operand should be a TT matrix and second a TT vector.
- Returns:
the result.
- Return type:
- full()[source]
Return the full tensor. In case of a TT matrix (TTM), the result has the shape
M1 x M2 x ... x Md x N1 x N2 x ... x Nd.- Returns:
the full tensor.
- Return type:
torch.tensor
- hadamard(other, eps=1e-12, rmax=9223372036854775807, method='dmrg', initial=None, verbose=False)[source]
Elementwise product with rank truncation,
(self * other).round(eps, rmax), without forming the exact product. Same astorchtt.hadamard()withselfas the first operand (see there how to choose the method).- Parameters:
other (torchtt.TT) – the second operand.
eps (float, optional) – relative accuracy. Defaults to 1e-12.
rmax (int, optional) – maximum rank of the result. Defaults to the maximum possible integer.
method (str | torchtt.methods.DMRG | torchtt.methods.Direct | torchtt.methods.Swap, optional) – the method, given by name or as an instance to change its parameters. Defaults to ‘dmrg’.
initial (torchtt.TT, optional) – initial guess of the result (only for
'dmrg'). Defaults to None.verbose (bool, optional) – print information about the iterations. Defaults to False.
- Returns:
the result.
- Return type:
- is_cuda()[source]
Return True if the tensor is on GPU.
- Returns:
Is the torchtt.TT on GPU or not.
- Return type:
bool
- property is_ttm
Check whether the instance is a TT operator or not.
- Returns:
the flag.
- Return type:
bool
- matmat(other, eps=1e-12, rmax=9223372036854775807, method='direct', initial=None, verbose=False)[source]
Product of two TT matrices with rank truncation,
(self @ other).round(eps, rmax). Same astorchtt.matmat()withselfas the first TT matrix (see there how to choose the method).- Parameters:
other (torchtt.TT) – the second TT matrix.
eps (float, optional) – relative accuracy. Defaults to 1e-12.
rmax (int, optional) – maximum rank of the result. Defaults to the maximum possible integer.
method (str | torchtt.methods.Direct | torchtt.methods.AMEn | torchtt.methods.Swap, optional) – the method, given by name or as an instance to change its parameters. Defaults to ‘direct’.
initial (torchtt.TT, optional) – initial guess of the result (only for
'amen'). Defaults to None.verbose (bool, optional) – print information about the iterations. Defaults to False.
- Returns:
the result.
- Return type:
- matvec(other, eps=1e-12, rmax=9223372036854775807, method='dmrg', initial=None, verbose=False)[source]
Matrix-vector product with rank truncation,
(self @ other).round(eps, rmax), without forming the exact product. Same astorchtt.matvec()withselfas the TT matrix (see there how to choose the method).Examples
y = A.matvec(x, eps=1e-10) y = A.matvec(x, eps=1e-10, method='direct') y = A.matvec(x, eps=1e-10, method=torchtt.methods.DMRG(nswp=40))
- Parameters:
other (torchtt.TT) – the TT tensor.
eps (float, optional) – relative accuracy. Defaults to 1e-12.
rmax (int, optional) – maximum rank of the result. Defaults to the maximum possible integer.
method (str | torchtt.methods.DMRG | torchtt.methods.Direct | torchtt.methods.AMEn | torchtt.methods.Swap, optional) – the method, given by name or as an instance to change its parameters. Defaults to ‘dmrg’.
initial (torchtt.TT, optional) – initial guess of the result (only for
'dmrg'and'amen'). Defaults to None.verbose (bool, optional) – print information about the iterations. Defaults to False.
- Returns:
the result.
- Return type:
- mprod(factor_matrices, mode)[source]
n-mode product.
- Parameters:
factor_matrices (torch.tensor or list[torch.tensor]) – either a single matrix is directly provided or a list of matrices for product along multiple modes.
mode (int or list[int]) – the mode for the product. If factor_matrices is a torch.tensor then mode is an integer and the multiplication will be performed along a single mode. If factor_matrices is a list, the mode has to be list[int] of equal size.
- Raises:
InvalidArguments – Invalid arguments.
ShapeMismatch – The n-th mode of the tensor must be equal with the 2nd mode of the matrix.
IncompatibleTypes – n-model product works only with TT-tensors and not TT matrices.
- Returns:
the result
- Return type:
- norm(squared=False)[source]
Computes the frobenius norm of a TT object.
- Parameters:
squared (bool, optional) – returns the square of the norm if True. Defaults to False.
- Returns:
the norm.
- Return type:
torch.tensor
- numpy()[source]
Return the full tensor as a numpy.array. In case of a TT matrix (TTM), the result has the shape
M1 x M2 x ... x Md x N1 x N2 x ... x Nd. If it is involved in an automatic differentiation (AD) graph, an error will occur.- Returns:
the full tensor in numpy.
- Return type:
numpy.array
- qtt_to_tens(original_shape)[source]
Transform a tensor back from the quantized tensor-train (QTT) format (see
to_qtt()).- Parameters:
original_shape (list) – the original shape.
- Raises:
InvalidArguments – Original shape must be a list.
ShapeMismatch – Mode sizes do not match.
- Returns:
the folded tensor.
- Return type:
- reduce_dims(exclude=[])[source]
Reduces the size 1 modes of the TT-object. At least one mode should be larger than 1.
- Parameters:
exclude (list, optional) – Indices to exclude. Defaults to [].
- round(eps=1e-12, rmax=9223372036854775807)[source]
Implements the rounding operations within a given tolerance epsilon [Oseledets, 2011]. The maximum rank is also provided.
- Parameters:
eps (float, optional) – the relative accuracy. Defaults to 1e-12.
rmax (int, optional) – the maximum rank. Defaults to the maximum possible integer.
- Returns:
the result.
- Return type:
- set_core(k, core)[source]
Replaces the k-th TT core. This can change the mode size of the TT object.
- Parameters:
k (int) – the core index.
core (torch.tensor) – _description_
- Raises:
InvalidArguments – The given core must match the the ranks and the dimensionality.
InvalidArguments – The index of the core mst match the dimensionality.
- sum(index=None)[source]
Contracts a tensor in the TT format along the given indices and retuyrns the resulting tensor in the TT format. If no index list is given, the sum over all indices is performed.
Examples
a = torchtt.ones([3,4,5,6,7]) print(a.sum()) print(a.sum([0,2,4])) print(a.sum([1,2])) print(a.sum([0,1,2,3,4]))
- Parameters:
index (int | list[int] | None, optional) – the indices along which the summation is performed. None selects all of them. Defaults to None.
- Raises:
InvalidArguments – Invalid index.
- Returns:
the result.
- Return type:
torchtt.TT/torch.tensor
- t()[source]
Returns the transpose of a given TT matrix.
- Returns:
the transpose.
- Return type:
- Raises:
InvalidArguments – Has to be TT matrix.
- to(device=None, dtype=None)[source]
Moves the TT instance to the given device with the given dtype.
- Parameters:
device (torch.device, optional) – The desired device. If None is provided, the device is not changed. Defaults to None.
dtype (torch.dtype, optional) – The desired dtype (torch.float64, torch.float32,…). If None is provided the dtype is not changed. Defaults to None.
- to_qtt(eps=1e-12, mode_size=2, rmax=9223372036854775807)[source]
Converts a tensor to the quantized tensor-train (QTT) format [Khoromskij, 2011, Oseledets, 2010]: the TT format applied to the tensor reshaped so that every mode has size
mode_size(2 by default). The shape changes as N1 x N2 x … x Nd -> mode_size x mode_size x … x mode_size, so every mode size must be a power ofmode_size. The tensor in QTT can be converted back using the qtt_to_tens() method.Examples
x = torchtt.random([16,8,64,128],[1,2,10,12,1]) x_qtt = x.to_qtt() print(x_qtt) xf = x_qtt.qtt_to_tens(x.N) # a TT-rounding is recommended.
- Parameters:
eps (float,optional) – the accuracy. Defaults to 1e-12.
mode_size (int, optional) – the size of the modes. Defaults to 2.
rmax (int) – the maximum rank. Defaults to the maximum possible integer.
- Raises:
ShapeMismatch – Only quadratic TTM can be tranformed to QTT.
ShapeMismatch – Reshaping error: check if the dimensions are powers of the desired mode size.
- Returns:
the resulting reshaped tensor.
- Return type:
- torchtt.amen_mm(A, B, nswp=22, X0=None, eps=1e-10, rmax=1024, kickrank=4, kick2=0, verbose=False)[source]
Perform the product of two TT matrices (TTM) using the alternating minimal energy (AMEn) method [Dolgov and Savostyanov, 2014]. Suited when the operators have high ranks, but the result is expected to be low rank.
Deprecated since version 0.6.0: Use
torchtt.matmat()withmethod='amen'(ormethod=torchtt.methods.AMEn(nswp=..., kickrank=..., kick2=...)).- Parameters:
A (torchtt.TT) – the first TTM.
B (torchtt.TT) – the second TTM.
nswp (int, optional) – number of sweeps. Defaults to 22.
X0 (torchtt.TT, optional) – initial guess (None means no initial guess). Defaults to None.
eps (float, optional) – realtive tolerance. Defaults to 1e-10.
rmax (int, optional) – maximum rank. Defaults to 1024.
kickrank (int, optional) – kickrank. Defaults to 4.
kick2 (int, optional) – number of random vectors added to the rank enrichment, in addition to the
kickrankvectors computed from the residual. Defaults to 0.verbose (bool, optional) – show debug info. Defaults to False.
- Returns:
the result.
- Return type:
- torchtt.amen_mv(A, b, nswp=22, x0=None, eps=1e-10, rmax=1024, kickrank=4, kick2=0, verbose=False, use_cpp=True)[source]
Compute the matrix vector product between a TT matrix (TTM) and a TT tensor using the alternating minimal energy (AMEn) method [Dolgov and Savostyanov, 2014]. Suited when the output is expected to be low rank.
Deprecated since version 0.6.0: Use
torchtt.matvec()withmethod='amen'(ormethod=torchtt.methods.AMEn(nswp=..., kickrank=..., kick2=...)).- Parameters:
A (torchtt.TT) – the matrix in TT.
b (torchtt.TT) – the tensor TT.
nswp (int, optional) – number of sweeps. Defaults to 22.
x0 (torchtt.TT, optional) – initial guess. In None is provided the initial guess is a ones tensor. Defaults to None.
eps (float, optional) – relative residual. Defaults to 1e-10.
rmax (int, optional) – maximum rank. Defaults to 1024.
kickrank (int, optional) – rank enrichment. Defaults to 4.
kick2 (int, optional) – number of random vectors added to the rank enrichment, in addition to the
kickrankvectors computed from the residual. Defaults to 0.verbose (bool, optional) – choose whether to display or not additional information during the runtime. Defaults to False.
use_cpp (bool, optional) – not used (there is no C++ implementation). Defaults to True.
- Raises:
InvalidArguments – A and b must be TT instances.
IncompatibleTypes – A must be TT-matrix and b must be vector.
ShapeMismatch – Dimension mismatch.
- Returns:
the approximation of the solution in TT format.
- Return type:
- torchtt.bilinear_form(x, A, y)[source]
Computes the bilinear form x^T A y for TT tensors:
- Parameters:
x (torchtt.TT) – the tensors.
A (torchtt.TT) – the tensors (must be TT matrix).
y (torchtt.TT) – the tensors.
- Raises:
InvalidArguments – Inputs must be torchtt.TT instances.
IncompatibleTypes – x and y must be TT tensors and A must be TT matrix.
ShapeMismatch – Check the shapes. Required is x.N == A.M and y.N == A.N.
- Returns:
the result of the bilienar form as tensor with 1 element.
- Return type:
torch.tensor
- torchtt.cat(tensors, dim=0)[source]
Concatenate tensors in the TT format along a given dimension dim. Only works for TT tensors and not TT matrices.
Examples
import torchtt import torch a1 = torchtt.randn((3,4,2,6,7), [1,2,3,4,2,1]) a2 = torchtt.randn((3,4,8,6,7), [1,3,1,7,5,1]) a3 = torchtt.randn((3,4,15,6,7), [1,3,10,2,4,1]) a = torchtt.cat((a1,a2,a3),2) af = torch.cat((a1.full(), a2.full(), print(torch.linalg.norm(a.full()-af))
- Parameters:
tensors (tuple[TT]) – the tensors to be concatenated. Their mode sizes must match for all modex except the concatenating dimension.
dim (int, optional) – The dimension to be concatenated after. Defaults to 0.
- Raises:
InvalidArguments – Not implemented for tensor matrices.
InvalidArguments – The mode sizes must be the same on the nonconcatenated dimensions for all the provided tensors.
InvalidArguments – The tensors must have the same number of dimensions.
- Returns:
the result.
- Return type:
- torchtt.diag(input)[source]
Creates diagonal TT matrix from TT tensor or extracts the diagonal of a TT matrix:
If a TT matrix is provided the result is a TT tensor representing the diagonal :math:` mathsf{x}_{i_1…i_d} = mathsf{A}_{i_1…i_d,i_1…i_d} `
If a TT tensor is provided the result is a diagonal TT matrix with the entries :math:` mathsf{A}_{i_1…i_d,j_1…j_d} = mathsf{x}_{i_1…i_d} delta_{i_1}^{j_1} cdots delta_{i_d}^{j_d} `
- Parameters:
input (TT) – the input.
- Raises:
InvalidArguments – Input must be a torchtt.TT instance.
- Returns:
the result.
- Return type:
- torchtt.dmrg_hadamard(x, y, z0=None, nswp=20, eps=1e-12, rmax=32768, kickrank=4, verb=False, use_cpp=True)[source]
Perform fast elementwise multiplication z = x * y in the TT format using the density matrix renormalization group (DMRG) algorithm [Oseledets, 2011].
Deprecated since version 0.6.0: Use
torchtt.hadamard()withmethod='dmrg'(ormethod=torchtt.methods.DMRG(nswp=..., kickrank=...)).- Parameters:
x (TT) – TT tensor
y (TT) – TT tensor
z0 (TT, optional) – initial guess of the result (if None is provided a random tensor is generated as a guess). Defaults to None.
nswp (int, optional) – numebr of sweeps. Defaults to 20.
eps (float, optional) – relative accuracy. Defaults to 1e-12.
rmax (int, optional) – maximum rank. Defaults to 32768.
kickrank (int, optional) – kickrank. Defaults to 4.
verb (bool, optional) – show debug info. Defaults to False.
use_cpp (bool, optional) – not used (there is no C++ implementation). Defaults to True.
- Returns:
the result.
- Return type:
- torchtt.dot(a, b, axis=None)[source]
Computes the dot product between 2 tensors in TT format. If both a and b have identical mode sizes the result is the dot product. If a and b have inequal mode sizes, the function perform index contraction. The number of dimensions of a must be greater or equal as b. The modes of the tensor a along which the index contraction with b is performed are given in axis. For the compelx case (a,b) = b^H . a.
Examples
a = torchtt.randn([3,4,5,6,7],[1,2,2,2,2,1]) b = torchtt.randn([3,4,5,6,7],[1,2,2,2,2,1]) c = torchtt.randn([3,5,6],[1,2,2,1]) print(torchtt.dot(a,b)) print(torchtt.dot(a,c,[0,2,3]))
- Parameters:
a (torchtt.TT) – the first tensor.
b (torchtt.TT) – the second tensor.
axis (list[int], optional) – the mode indices for index contraction. Defaults to None.
- Raises:
InvalidArguments – Both operands should be TT instances.
NotImplementedError – Operation not implemented for TT-matrices.
ShapeMismatch – Operands are not the same size.
ShapeMismatch – Number of the modes of the first tensor must be equal with the second.
- Returns:
the result. If no axis index is provided the result is a scalar otherwise a torchtt.TT object.
- Return type:
float or torchtt.TT
- torchtt.elementwise_divide(x, y, eps=1e-12, starting_tensor=None, nswp=50, kick=4, local_iterations=40, resets=2, preconditioner=None, verbose=False)[source]
Perform the elemntwise division x/y of two tensors in the TT format using the alternating minimal energy (AMEn) method [Dolgov and Savostyanov, 2014]. Use this method if different AMEn arguments are needed. This method does not check the validity of the inputs.
- Parameters:
x (torchtt.TT or scalar) – first tensor (can also be scalar of type float, int, torch.tensor with shape (1)).
y (torchtt.TT) – second tensor.
eps (float, optional) – relative acccuracy. Defaults to 1e-12.
starting_tensor (torchtt.TT or None, optional) – initial guess of the result (None for random initial guess). Defaults to None.
nswp (int, optional) – number of iterations. Defaults to 50.
kick (int, optional) – size of rank enrichment. Defaults to 4.
local_iterations (int, optional) – the number of iterations for the local iterative solver. Defaults to 40.
resets (int, optional) – the number of restarts in the GMRES (generalized minimal residual) solver. Defaults to 2.
preconditioner (string, optional) – Use preconditioner for the local solver (possible vaules None, ‘c’). Defaults to None.
verbose (bool, optional) – display debug info. Defaults to False.
- Returns:
the result
- Return type:
- torchtt.eye(shape, dtype=torch.float64, device=None)[source]
Construct the TT decomposition of a multidimensional identity matrix. all the TT ranks are 1.
- Parameters:
shape (list[int]) – the shape.
dtype (torch.dtype, optional) – the dtype of the returned tensor. Defaults to tn.float64.
device (torch.device, optional) – the device where the TT cores are created (None means CPU). Defaults to None.
- Returns:
the one tensor.
- Return type:
- torchtt.fast_hadammard(tt_a, tt_b, eps=1e-10, rmax=None)[source]
Performs the elementwise multiplication between two TTs or TT matrices (TTMs) and rounds the result. Equivalent to (tt_a * tt_b).round(eps). Method described in [Michailidis et al., 2025].
Deprecated since version 0.6.0: Use
torchtt.hadamard()withmethod='swap'.- Parameters:
tt_a (torchtt.TT) – first operand.
tt_b (torchtt.TT) – second operand.
eps (float, optional) – relative tolerance. Defaults to 1e-10.
rmax (int, optional) – maximum rank. Defaults to None.
- Returns:
the result.
- Return type:
- torchtt.fast_mm(tt_a, tt_b, eps=1e-10, rmax=None)[source]
Performs the matmat product between two TT matrices (TTMs). Equivalent to (tt_a @ tt_b).round(eps). Method described in [Michailidis et al., 2025].
Deprecated since version 0.6.0: Use
torchtt.matmat()withmethod='swap'.- Parameters:
tt_a (torchtt.TT) – the first operand. Must be a TTM.
tt_b (torchtt.TT) – the second operand. Must be TTM.
eps (float, optional) – Relative tolerance. Defaults to 1e-10.
rmax (int, optional) – maximum rank. Defaults to None.
- Returns:
the result. This is a TTM.
- Return type:
- torchtt.fast_mv(tt_a, tt_b, eps=1e-10, rmax=None)[source]
Performs the matvec product between a TT matrix (TTM) and a TT tensor. Equivalent to (tt_a @ tt_b).round(eps). Method described in [Michailidis et al., 2025].
Deprecated since version 0.6.0: Use
torchtt.matvec()withmethod='swap'.- Parameters:
tt_a (torchtt.TT) – the first operand. Must be a TTM.
tt_b (torchtt.TT) – the second operand. Must be TT.
eps (float, optional) – Relative tolerance. Defaults to 1e-10.
rmax (int, optional) – maximum rank. Defaults to None.
- Returns:
the result. This is a TT.
- Return type:
- torchtt.hadamard(x, y, eps=1e-12, rmax=9223372036854775807, method='dmrg', initial=None, verbose=False)[source]
Elementwise (Hadamard) product of two TT tensors or two TT matrices with rank truncation. Computes an approximation of
(x * y).round(eps, rmax), which can be much cheaper than forming the exact productx * y(its rank is the product of the ranks of the operands).Choosing the method (see
torchtt.methodsand Efficient products):'dmrg'(default): best when the product rankrank(x) * rank(y)is large and the result has a much smaller rank.'direct': formsx * yand rounds it. Faster when the product rank is moderate (up to a few hundred) or when the result cannot be compressed much.'swap': no iterations and deterministic, usually slower than'dmrg'.
Examples
import torchtt as tntt x = tntt.randn([4] * 8, [1] + 7 * [3] + [1]) y = tntt.randn([4] * 8, [1] + 7 * [4] + [1]) z = tntt.hadamard(x, y, eps=1e-10) # DMRG z = tntt.hadamard(x, y, eps=1e-10, method='direct') # (x * y).round(1e-10)
- Parameters:
x (torchtt.TT) – the first operand.
y (torchtt.TT) – the second operand.
eps (float, optional) – relative accuracy. Defaults to 1e-12.
rmax (int, optional) – maximum rank of the result. Defaults to the maximum possible integer.
method (str | torchtt.methods.DMRG | torchtt.methods.Direct | torchtt.methods.Swap, optional) – the method, given by name (
'dmrg','direct'or'swap') or as an instance to change its parameters. Defaults to ‘dmrg’.initial (torchtt.TT, optional) – initial guess of the result (only for
'dmrg'). None means that the method chooses it. Defaults to None.verbose (bool, optional) – print information about the iterations (only for
'dmrg'). Defaults to False.
- Raises:
InvalidArguments – The operands must be torchtt.TT instances.
IncompatibleTypes – The operands must be both TT tensors or both TT matrices.
ShapeMismatch – The operands must have the same shape.
ShapeMismatch – The initial guess must have the shape of the operands.
InvalidArguments – The method is unknown, does not implement this product or does not use an initial guess.
- Returns:
the result.
- Return type:
- torchtt.kron(first, second)[source]
Computes the tensor Kronecker product. If None is provided as input the reult is the other tensor. If A is N_1 x … x N_d and B is M_1 x … x M_p, then kron(A,B) is N_1 x … x N_d x M_1 x … x M_p
- Parameters:
first (torchtt.TT | None) – first argument.
second (torchtt.TT | None) – second argument.
- Raises:
IncompatibleTypes – Incompatible data types (make sure both are either TT-matrices or TT-tensors).
InvalidArguments – Invalid arguments.
- Returns:
the result.
- Return type:
- torchtt.load(path)[source]
Load a torchtt.TT object from a file.
Examples
import torchtt #generate a TT object A = torchtt.randn([10,20,30,40,4,5],[1,6,5,4,3,2,1]) # save the TT object torchtt.save(A,"./test.TT") # load the TT object B = torchtt.load("./test.TT") # the loaded should be the same print((A-B).norm()/A.norm())
- Parameters:
path (str) – the file name.
- Returns:
the tensor.
- Return type:
- torchtt.matmat(A, B, eps=1e-12, rmax=9223372036854775807, method='direct', initial=None, verbose=False)[source]
Product of two TT matrices with rank truncation. Computes an approximation of
(A @ B).round(eps, rmax).Choosing the method (see
torchtt.methodsand Efficient products):'direct'(default): formsA @ Band rounds it. Best when the product rankrank(A) * rank(B)is moderate (up to a few hundred) or when the result cannot be compressed much.'amen': best when the product rank is large and the result has a much smaller rank (real tensors only). The rank grows by at mostkickrankper sweep, so give aninitialguess or increasenswpif the result has a large rank.'swap': no iterations and deterministic, usually slower than'direct'and'amen'.
Examples
import torchtt as tntt A = tntt.randn([(4, 4)] * 8, [1] + 7 * [3] + [1]) B = tntt.randn([(4, 4)] * 8, [1] + 7 * [4] + [1]) C = tntt.matmat(A, B, eps=1e-10) # (A @ B).round(1e-10) C = tntt.matmat(A, B, eps=1e-10, method=tntt.methods.AMEn(kickrank=8)) # AMEn
- Parameters:
A (torchtt.TT) – the first TT matrix.
B (torchtt.TT) – the second TT matrix.
eps (float, optional) – relative accuracy. Defaults to 1e-12.
rmax (int, optional) – maximum rank of the result. Defaults to the maximum possible integer.
method (str | torchtt.methods.Direct | torchtt.methods.AMEn | torchtt.methods.Swap, optional) – the method, given by name (
'direct','amen'or'swap') or as an instance to change its parameters. Defaults to ‘direct’.initial (torchtt.TT, optional) – initial guess of the result (only for
'amen'). None means that the method chooses it. Defaults to None.verbose (bool, optional) – print information about the iterations (only for
'amen'). Defaults to False.
- Raises:
InvalidArguments – The operands must be torchtt.TT instances.
IncompatibleTypes – Both operands must be TT matrices.
ShapeMismatch – The column shape of the first TT matrix must be the row shape of the second.
ShapeMismatch – The initial guess must be a TT matrix with the shape of the result.
InvalidArguments – The method is unknown, does not implement this product or does not use an initial guess.
- Returns:
the result.
- Return type:
- torchtt.matvec(A, x, eps=1e-12, rmax=9223372036854775807, method='dmrg', initial=None, verbose=False)[source]
Matrix-vector product between a TT matrix and a TT tensor with rank truncation. Computes an approximation of
(A @ x).round(eps, rmax), which can be much cheaper than forming the exact productA @ x(its rank is the product of the ranks of the operands).Choosing the method (see
torchtt.methodsand Efficient products):'dmrg'(default): best when the product rankrank(A) * rank(x)is large and the result has a much smaller rank. Uses the C++ implementation if available.'direct': formsA @ xand rounds it. Faster when the product rank is moderate (up to a few hundred) or when the result cannot be compressed much.'amen': alternative to'dmrg'for real tensors, usually slower. The rank grows by at mostkickrankper sweep, so give aninitialguess or increasenswpif the result has a large rank.'swap': no iterations and deterministic, usually slower than'dmrg'.
Examples
import torchtt as tntt A = tntt.randn([(4, 4)] * 8, [1] + 7 * [3] + [1]) x = tntt.randn([4] * 8, [1] + 7 * [4] + [1]) y = tntt.matvec(A, x, eps=1e-10) # DMRG y = tntt.matvec(A, x, eps=1e-10, method='direct') # (A @ x).round(1e-10) y = tntt.matvec(A, x, eps=1e-10, method=tntt.methods.DMRG(nswp=40)) # DMRG with 40 sweeps y = tntt.matvec(A, x, eps=1e-10, method='amen', initial=y) # AMEn with an initial guess
- Parameters:
A (torchtt.TT) – the TT matrix.
x (torchtt.TT) – the TT tensor.
eps (float, optional) – relative accuracy. Defaults to 1e-12.
rmax (int, optional) – maximum rank of the result. Defaults to the maximum possible integer.
method (str | torchtt.methods.DMRG | torchtt.methods.Direct | torchtt.methods.AMEn | torchtt.methods.Swap, optional) – the method, given by name (
'dmrg','direct','amen'or'swap') or as an instance to change its parameters. Defaults to ‘dmrg’.initial (torchtt.TT, optional) – initial guess of the result (only for
'dmrg'and'amen'). None means that the method chooses it. Defaults to None.verbose (bool, optional) – print information about the iterations (only for
'dmrg'and'amen'). Defaults to False.
- Raises:
InvalidArguments – The operands must be torchtt.TT instances.
IncompatibleTypes – The first operand must be a TT matrix and the second a TT tensor.
ShapeMismatch – The column shape of the TT matrix must be the shape of the TT tensor.
ShapeMismatch – The initial guess must be a TT tensor with the row shape of the TT matrix.
InvalidArguments – The method is unknown or does not use an initial guess.
- Returns:
the result.
- Return type:
- torchtt.meshgrid(vectors)[source]
Creates a meshgrid of torchtt.TT objects. Similar to numpy.meshgrid or torch.meshgrid. The input is a list of d torch.tensor vectors of sizes N_1, … ,N_d The result is a list of torchtt.TT instances of shapes N1 x … x Nd.
- Parameters:
vectors (list[torch.tensor]) – the vectors (1d tensors).
- Returns:
the resulting meshgrid.
- Return type:
list[TT]
- torchtt.numel(tensor)[source]
Return the number of entries needed to store the TT cores for the given tensor.
- Parameters:
tensor (torchtt.TT) – the TT representation of the tensor.
- Returns:
number of floats stored for the TT decomposition.
- Return type:
int
- torchtt.ones(shape, dtype=torch.float64, device=None)[source]
Construct a tensor that contains only ones. the shape can be a list of ints or a list of tuples of ints. The second case creates a TT matrix.
- Parameters:
shape (list[int] or list[tuple[int]]) – the shape.
dtype (torch.dtype, optional) – the dtype of the returned tensor. Defaults to tn.float64.
device (torch.device, optional) – the device where the TT cores are created (None means CPU). Defaults to None.
- Raises:
InvalidArguments – Shape must be a list.
- Returns:
the one tensor.
- Return type:
- torchtt.pad(tensor, padding, value=0.0)[source]
Pad a tensor in the TT format. The padding argument is a tuple of tuples ((b1, a1), (b2, a2), … , (bd, ad)). Each dimension is padded with bk at the beginning and ak at the end. The padding value is constant and is given as the argument value. In case of a TT operator, duiagual padding is performed. On the diagonal, the provided value is inserted.
- Parameters:
tensor (TT) – the tensor to be padded.
padding (tuple(tuple(int))) – the paddings.
value (float, optional) – the value to pad. Defaults to 0.0.
- Raises:
InvalidArguments – The number of paddings should not exceed the number of dimensions of the tensor.
- Returns:
the result.
- Return type:
- torchtt.permute(input, dims, eps=1e-12)[source]
Permutes the dimensions of the tensor. Works similarily to
torch.permute. Works like a bubble sort for both TT tensors and TT matrices.Examples
x_tt = torchtt.random([5,6,7,8,9],[1,2,3,4,2,1]) xp_tt = torchtt.permute(x_tt, [4,3,2,1,0], 1e-10) print(xp_tt) # the shape of this tensor should be [9,8,7,6,5]
- Parameters:
input (torchtt.TT) – the input tensor.
dims (list[int]) – the order of the indices in the new tensor.
eps (float, optional) – the relative accuracy of the decomposition. Defaults to 1e-12.
- Raises:
InvalidArguments – The input must be a TT tensor dims must be a list of integers or a tple of integers.
ShapeMismatch – dims must be the length of the number of dimensions.
InvalidArguments – Duplicate dims are not allowed.
InvalidArguments – Dims should only contain integers from 0 to d-1.
- Returns:
the resulting tensor.
- Return type:
- torchtt.randn(N, R, var=1.0, dtype=torch.float64, device=None)[source]
A torchtt.TT tensor of shape N = [N1 x … x Nd] and rank R is returned. The entries of the full tensor are almost normally distributed with the variance var.
- Parameters:
N (list[int]) – the shape.
R (list[int]) – the rank.
var (float, optional) – the variance. Defaults to 1.0.
dtype (torch.dtype, optional) – the dtype of the returned tensor. Defaults to tn.float64.
device (torch.device, optional) – the device where the TT cores are created (None means CPU). Defaults to None.
- Returns:
the result.
- Return type:
- torchtt.random(N, R, dtype=torch.float64, device=None)[source]
Returns a tensor of shape N with random cores of rank R. Each core is a normal distributed with mean 0 and variance 1. Check also the method torchtt.randn()for better random tensors in the TT format.
- Parameters:
N (list[int] or list[tuple[int]]) – the shape of the tensor. If the elements are tuples of integers, we deal with a TT-matrix.
R (list[int] or int) – can be a list if the exact rank is specified or an integer if the maximum rank is secified.
dtype (torch.dtype, optional) – the dtype of the returned tensor. Defaults to tn.float64.
device (torch.device, optional) – the device where the TT cores are created (None means CPU). Defaults to None.
- Raises:
InvalidArguments – Check if N and R are right.
- Returns:
the result.
- Return type:
- torchtt.rank1TT(elements)[source]
Compute the rank 1 TT from a list of vectors (or matrices).
- Parameters:
elements (list[torch.tensor]) – the list of vectors (or matrices in case a TT matrix should be created).
- Returns:
the resulting TT object.
- Return type:
- torchtt.reshape(tens, shape, eps=1e-16, rmax=9223372036854775807)[source]
Reshapes a torchtt.TT tensor in the TT format. A rounding is also performed.
- Parameters:
tens (torchtt.TT) – the input tensor.
shape (list[int] or list[tuple[int]]) – the desired shape. In the case of a TT operator the shape has to be given as list of tuples of ints [(M1,N1),…,(Md,Nd)].
eps (float, optional) – relative accuracy. Defaults to 1e-16.
rmax (int, optional) – maximum rank. Defaults to the maximum possible integer.
- Raises:
ShapeMismatch – The product of modes should remain equal. Check the given shape.
- Returns:
the resulting tensor.
- Return type:
- torchtt.save(tensor, path)[source]
Save a torchtt.TT object in a file.
Examples
import torchtt #generate a TT object A = torchtt.randn([10,20,30,40,4,5],[1,6,5,4,3,2,1]) # save the TT object torchtt.save(A,"./test.TT") # load the TT object B = torchtt.load("./test.TT") # the loaded should be the same print((A-B).norm()/A.norm())
- Parameters:
tensor (torchtt.TT) – the tensor to be saved.
path (str) – the file name.
- Raises:
InvalidArguments – First argument must be a torchtt.TT instance.
- torchtt.shape_mn_to_tuple(M, N)[source]
Convert the shape of a TT matrix (TTM) from row/column format to tuple format.
- Parameters:
M (list[int]) – row shapes.
N (list[int]) – column shapes.
- Returns:
shape.
- Return type:
list[tuple[int]]
- torchtt.shape_tuple_to_mn(shape)[source]
Convert the shape of a TT matrix (TTM) from tuple format to row and column shapes.
- Parameters:
shape (list[tuple[int]]) – shape.
- Returns:
still the shape.
- Return type:
tuple[list[int],list[int]]
- torchtt.zeros(shape, dtype=torch.float64, device=None)[source]
Construct a tensor that contains only zeros. the shape can be a list of ints or a list of tuples of ints. The second case creates a TT matrix.
- Parameters:
shape (list[int] | list[tuple[int]]) – the shape.
dtype (torch.dtype, optional) – the dtype of the returned tensor. Defaults to tn.float64.
device (torch.device, optional) – the device where the TT cores are created (None means CPU). Defaults to None.
- Raises:
InvalidArguments – Shape must be a list.
- Returns:
the zero tensor.
- Return type: