torch¶

The torch package contains data structures for multi-dimensional tensors and defines mathematical operations over these tensors. Additionally, it provides many utilities for efficient serializing of Tensors and arbitrary types, and other useful utilities.

It has a CUDA counterpart, that enables you to run your tensor computations on an NVIDIA GPU with compute capability >= 3.0.

Tensors¶

torch.is_tensor(obj)[source]¶

Returns True if obj is a PyTorch tensor.

Note that this function is simply doing isinstance(obj, Tensor). Using that isinstance check is better for typechecking with mypy, and more explicit - so it’s recommended to use that instead of is_tensor.

Parameters: obj (Object) – Object to test

torch.is_storage(obj)[source]¶

Returns True if obj is a PyTorch storage object.

Parameters: obj (Object) – Object to test

torch.set_default_dtype(d)[source]¶

Sets the default floating point dtype to d. This dtype is:

The inferred dtype for python floats in torch.tensor().
Used to infer dtype for python complex numbers. The default complex dtype is set to torch.complex128 if default floating point dtype is torch.float64, otherwise it’s set to torch.complex64

The default floating point dtype is initially torch.float32.

Parameters: d (torch.dtype) – the floating point dtype to make the default

Example

>>> # initial default for floating point is torch.float32
>>> torch.tensor([1.2, 3]).dtype
torch.float32
>>> # initial default for floating point is torch.complex64
>>> torch.tensor([1.2, 3j]).dtype
torch.complex64
>>> torch.set_default_dtype(torch.float64)
>>> torch.tensor([1.2, 3]).dtype    # a new floating point tensor
torch.float64
>>> torch.tensor([1.2, 3j]).dtype   # a new complex tensor
torch.complex128

torch.get_default_dtype() → torch.dtype ¶

Get the current default floating point torch.dtype.

Example:

>>> torch.get_default_dtype()  # initial default for floating point is torch.float32
torch.float32
>>> torch.set_default_dtype(torch.float64)
>>> torch.get_default_dtype()  # default is now changed to torch.float64
torch.float64
>>> torch.set_default_tensor_type(torch.FloatTensor)  # setting tensor type also affects this
>>> torch.get_default_dtype()  # changed to torch.float32, the dtype for torch.FloatTensor
torch.float32

torch.set_default_tensor_type(t)[source]¶

Sets the default torch.Tensor type to floating point tensor type t. This type will also be used as default floating point type for type inference in torch.tensor().

The default floating point tensor type is initially torch.FloatTensor.

Parameters: t (type or string) – the floating point tensor type or its name

Example:

>>> torch.tensor([1.2, 3]).dtype    # initial default for floating point is torch.float32
torch.float32
>>> torch.set_default_tensor_type(torch.DoubleTensor)
>>> torch.tensor([1.2, 3]).dtype    # a new floating point tensor
torch.float64

torch.numel(input) → int ¶

Returns the total number of elements in the input tensor.

Parameters: input (Tensor) – the input tensor.

Example:

>>> a = torch.randn(1, 2, 3, 4, 5)
>>> torch.numel(a)
120
>>> a = torch.zeros(4,4)
>>> torch.numel(a)
16

torch.set_printoptions(precision=None, threshold=None, edgeitems=None, linewidth=None, profile=None, sci_mode=None)[source]¶

Set options for printing. Items shamelessly taken from NumPy

Parameters

precision – Number of digits of precision for floating point output (default = 4).
threshold – Total number of array elements which trigger summarization rather than full repr (default = 1000).
edgeitems – Number of array items in summary at beginning and end of each dimension (default = 3).
linewidth – The number of characters per line for the purpose of inserting line breaks (default = 80). Thresholded matrices will ignore this parameter.
profile – Sane defaults for pretty printing. Can override with any of the above options. (any one of default, short, full)
sci_mode – Enable (True) or disable (False) scientific notation. If None (default) is specified, the value is defined by torch._tensor_str._Formatter. This value is automatically chosen by the framework.

torch.set_flush_denormal(mode) → bool ¶

Disables denormal floating numbers on CPU.

Returns True if your system supports flushing denormal numbers and it successfully configures flush denormal mode. set_flush_denormal() is only supported on x86 architectures supporting SSE3.

Parameters: mode (bool) – Controls whether to enable flush denormal mode or not

Example:

>>> torch.set_flush_denormal(True)
True
>>> torch.tensor([1e-323], dtype=torch.float64)
tensor([ 0.], dtype=torch.float64)
>>> torch.set_flush_denormal(False)
True
>>> torch.tensor([1e-323], dtype=torch.float64)
tensor(9.88131e-324 *
       [ 1.0000], dtype=torch.float64)

Creation Ops¶

Note

Random sampling creation ops are listed under Random sampling and include: torch.rand() torch.rand_like() torch.randn() torch.randn_like() torch.randint() torch.randint_like() torch.randperm() You may also use torch.empty() with the In-place random sampling methods to create torch.Tensor s with values sampled from a broader range of distributions.

torch.tensor(data, *, dtype=None, device=None, requires_grad=False, pin_memory=False) → Tensor ¶

Constructs a tensor with data.

Warning

torch.tensor() always copies data. If you have a Tensor data and want to avoid a copy, use torch.Tensor.requires_grad_() or torch.Tensor.detach(). If you have a NumPy ndarray and want to avoid a copy, use torch.as_tensor().

Warning

When data is a tensor x, torch.tensor() reads out ‘the data’ from whatever it is passed, and constructs a leaf variable. Therefore torch.tensor(x) is equivalent to x.clone().detach() and torch.tensor(x, requires_grad=True) is equivalent to x.clone().detach().requires_grad_(True). The equivalents using clone() and detach() are recommended.

Parameters

data (array_like) – Initial data for the tensor. Can be a list, tuple, NumPy ndarray, scalar, and other types.

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, infers data type from data.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.
pin_memory (bool, optional) – If set, returned tensor would be allocated in the pinned memory. Works only for CPU tensors. Default: False.

Example:

>>> torch.tensor([[0.1, 1.2], [2.2, 3.1], [4.9, 5.2]])
tensor([[ 0.1000,  1.2000],
        [ 2.2000,  3.1000],
        [ 4.9000,  5.2000]])

>>> torch.tensor([0, 1])  # Type inference on data
tensor([ 0,  1])

>>> torch.tensor([[0.11111, 0.222222, 0.3333333]],
...              dtype=torch.float64,
...              device=torch.device('cuda:0'))  # creates a torch.cuda.DoubleTensor
tensor([[ 0.1111,  0.2222,  0.3333]], dtype=torch.float64, device='cuda:0')

>>> torch.tensor(3.14159)  # Create a scalar (zero-dimensional tensor)
tensor(3.1416)

>>> torch.tensor([])  # Create an empty tensor (of size (0,))
tensor([])

torch.sparse_coo_tensor(indices, values, size=None, *, dtype=None, device=None, requires_grad=False) → Tensor ¶

Constructs a sparse tensor in COO(rdinate) format with specified values at the given indices.

Note

This function returns an uncoalesced tensor.

Parameters

indices (array_like) – Initial data for the tensor. Can be a list, tuple, NumPy ndarray, scalar, and other types. Will be cast to a torch.LongTensor internally. The indices are the coordinates of the non-zero values in the matrix, and thus should be two-dimensional where the first dimension is the number of tensor dimensions and the second dimension is the number of non-zero values.
values (array_like) – Initial values for the tensor. Can be a list, tuple, NumPy ndarray, scalar, and other types.
size (list, tuple, or torch.Size, optional) – Size of the sparse tensor. If not provided the size will be inferred as the minimum size big enough to hold all non-zero elements.

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, infers data type from values.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>>> i = torch.tensor([[0, 1, 1],
...                   [2, 0, 2]])
>>> v = torch.tensor([3, 4, 5], dtype=torch.float32)
>>> torch.sparse_coo_tensor(i, v, [2, 4])
tensor(indices=tensor([[0, 1, 1],
                       [2, 0, 2]]),
       values=tensor([3., 4., 5.]),
       size=(2, 4), nnz=3, layout=torch.sparse_coo)

>>> torch.sparse_coo_tensor(i, v)  # Shape inference
tensor(indices=tensor([[0, 1, 1],
                       [2, 0, 2]]),
       values=tensor([3., 4., 5.]),
       size=(2, 3), nnz=3, layout=torch.sparse_coo)

>>> torch.sparse_coo_tensor(i, v, [2, 4],
...                         dtype=torch.float64,
...                         device=torch.device('cuda:0'))
tensor(indices=tensor([[0, 1, 1],
                       [2, 0, 2]]),
       values=tensor([3., 4., 5.]),
       device='cuda:0', size=(2, 4), nnz=3, dtype=torch.float64,
       layout=torch.sparse_coo)

# Create an empty sparse tensor with the following invariants:
#   1. sparse_dim + dense_dim = len(SparseTensor.shape)
#   2. SparseTensor._indices().shape = (sparse_dim, nnz)
#   3. SparseTensor._values().shape = (nnz, SparseTensor.shape[sparse_dim:])
#
# For instance, to create an empty sparse tensor with nnz = 0, dense_dim = 0 and
# sparse_dim = 1 (hence indices is a 2D tensor of shape = (1, 0))
>>> S = torch.sparse_coo_tensor(torch.empty([1, 0]), [], [1])
tensor(indices=tensor([], size=(1, 0)),
       values=tensor([], size=(0,)),
       size=(1,), nnz=0, layout=torch.sparse_coo)

# and to create an empty sparse tensor with nnz = 0, dense_dim = 1 and
# sparse_dim = 1
>>> S = torch.sparse_coo_tensor(torch.empty([1, 0]), torch.empty([0, 2]), [1, 2])
tensor(indices=tensor([], size=(1, 0)),
       values=tensor([], size=(0, 2)),
       size=(1, 2), nnz=0, layout=torch.sparse_coo)

torch.as_tensor(data, dtype=None, device=None) → Tensor ¶

Convert the data into a torch.Tensor. If the data is already a Tensor with the same dtype and device, no copy will be performed, otherwise a new Tensor will be returned with computational graph retained if data Tensor has requires_grad=True. Similarly, if the data is an ndarray of the corresponding dtype and the device is the cpu, no copy will be performed.

Parameters

data (array_like) – Initial data for the tensor. Can be a list, tuple, NumPy ndarray, scalar, and other types.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, infers data type from data.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.

Example:

>>> a = numpy.array([1, 2, 3])
>>> t = torch.as_tensor(a)
>>> t
tensor([ 1,  2,  3])
>>> t[0] = -1
>>> a
array([-1,  2,  3])

>>> a = numpy.array([1, 2, 3])
>>> t = torch.as_tensor(a, device=torch.device('cuda'))
>>> t
tensor([ 1,  2,  3])
>>> t[0] = -1
>>> a
array([1,  2,  3])

torch.from_numpy(ndarray) → Tensor ¶

Creates a Tensor from a numpy.ndarray.

The returned tensor and ndarray share the same memory. Modifications to the tensor will be reflected in the ndarray and vice versa. The returned tensor is not resizable.

It currently accepts ndarray with dtypes of numpy.float64, numpy.float32, numpy.float16, numpy.complex64, numpy.complex128, numpy.int64, numpy.int32, numpy.int16, numpy.int8, numpy.uint8, and numpy.bool.

Example:

>>> a = numpy.array([1, 2, 3])
>>> t = torch.from_numpy(a)
>>> t
tensor([ 1,  2,  3])
>>> t[0] = -1
>>> a
array([-1,  2,  3])

torch.zeros(*size, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor ¶

Returns a tensor filled with the scalar value 0, with the shape defined by the variable argument size.

Parameters

size (int...) – a sequence of integers defining the shape of the output tensor. Can be a variable number of arguments or a collection like a list or tuple.

Keyword Arguments

out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()).
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>>> torch.zeros(2, 3)
tensor([[ 0.,  0.,  0.],
        [ 0.,  0.,  0.]])

>>> torch.zeros(5)
tensor([ 0.,  0.,  0.,  0.,  0.])

torch.zeros_like(input, *, dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format) → Tensor ¶

Returns a tensor filled with the scalar value 0, with the same size as input. torch.zeros_like(input) is equivalent to torch.zeros(input.size(), dtype=input.dtype, layout=input.layout, device=input.device).

Warning

As of 0.4, this function does not support an out keyword. As an alternative, the old torch.zeros_like(input, out=output) is equivalent to torch.zeros(input.size(), out=output).

Parameters

input (Tensor) – the size of input will determine size of the output tensor.

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned Tensor. Default: if None, defaults to the dtype of input.
layout (torch.layout, optional) – the desired layout of returned tensor. Default: if None, defaults to the layout of input.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, defaults to the device of input.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.
memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.

Example:

>>> input = torch.empty(2, 3)
>>> torch.zeros_like(input)
tensor([[ 0.,  0.,  0.],
        [ 0.,  0.,  0.]])

torch.ones(*size, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor ¶

Returns a tensor filled with the scalar value 1, with the shape defined by the variable argument size.

Parameters

size (int...) – a sequence of integers defining the shape of the output tensor. Can be a variable number of arguments or a collection like a list or tuple.

Keyword Arguments

out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()).
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>>> torch.ones(2, 3)
tensor([[ 1.,  1.,  1.],
        [ 1.,  1.,  1.]])

>>> torch.ones(5)
tensor([ 1.,  1.,  1.,  1.,  1.])

torch.ones_like(input, *, dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format) → Tensor ¶

Returns a tensor filled with the scalar value 1, with the same size as input. torch.ones_like(input) is equivalent to torch.ones(input.size(), dtype=input.dtype, layout=input.layout, device=input.device).

Warning

As of 0.4, this function does not support an out keyword. As an alternative, the old torch.ones_like(input, out=output) is equivalent to torch.ones(input.size(), out=output).

Parameters

input (Tensor) – the size of input will determine size of the output tensor.

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned Tensor. Default: if None, defaults to the dtype of input.
layout (torch.layout, optional) – the desired layout of returned tensor. Default: if None, defaults to the layout of input.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, defaults to the device of input.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.
memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.

Example:

>>> input = torch.empty(2, 3)
>>> torch.ones_like(input)
tensor([[ 1.,  1.,  1.],
        [ 1.,  1.,  1.]])

torch.arange(start=0, end, step=1, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor ¶

Returns a 1-D tensor of size \(\left\lceil \frac{\text{end} - \text{start}}{\text{step}} \right\rceil\) with values from the interval [start, end) taken with common difference step beginning from start.

Note that non-integer step is subject to floating point rounding errors when comparing against end; to avoid inconsistency, we advise adding a small epsilon to end in such cases.

\[\text{out}_{{i+1}} = \text{out}_{i} + \text{step} \]

Parameters

start (Number) – the starting value for the set of points. Default: 0.
end (Number) – the ending value for the set of points
step (Number) – the gap between each pair of adjacent points. Default: 1.

Keyword Arguments

out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()). If dtype is not given, infer the data type from the other input arguments. If any of start, end, or stop are floating-point, the dtype is inferred to be the default dtype, see get_default_dtype(). Otherwise, the dtype is inferred to be torch.int64.
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>>> torch.arange(5)
tensor([ 0,  1,  2,  3,  4])
>>> torch.arange(1, 4)
tensor([ 1,  2,  3])
>>> torch.arange(1, 2.5, 0.5)
tensor([ 1.0000,  1.5000,  2.0000])

torch.range(start=0, end, step=1, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor ¶

Returns a 1-D tensor of size \(\left\lfloor \frac{\text{end} - \text{start}}{\text{step}} \right\rfloor + 1\) with values from start to end with step step. Step is the gap between two values in the tensor.

\[\text{out}_{i+1} = \text{out}_i + \text{step}. \]

Warning

This function is deprecated and will be removed in a future release because its behavior is inconsistent with Python’s range builtin. Instead, use torch.arange(), which produces values in [start, end).

Parameters

start (float) – the starting value for the set of points. Default: 0.
end (float) – the ending value for the set of points
step (float) – the gap between each pair of adjacent points. Default: 1.

Keyword Arguments

out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()). If dtype is not given, infer the data type from the other input arguments. If any of start, end, or stop are floating-point, the dtype is inferred to be the default dtype, see get_default_dtype(). Otherwise, the dtype is inferred to be torch.int64.
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>>> torch.range(1, 4)
tensor([ 1.,  2.,  3.,  4.])
>>> torch.range(1, 4, 0.5)
tensor([ 1.0000,  1.5000,  2.0000,  2.5000,  3.0000,  3.5000,  4.0000])

torch.linspace(start, end, steps, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor ¶

Creates a one-dimensional tensor of size steps whose values are evenly spaced from start to end, inclusive. That is, the value are:

\[(\text{start}, \text{start} + \frac{\text{end} - \text{start}}{\text{steps} - 1}, \ldots, \text{start} + (\text{steps} - 2) * \frac{\text{end} - \text{start}}{\text{steps} - 1}, \text{end}) \]

Warning

Not providing a value for steps is deprecated. For backwards compatibility, not providing a value for steps will create a tensor with 100 elements. Note that this behavior is not reflected in the documented function signature and should not be relied on. In a future PyTorch release, failing to provide a value for steps will throw a runtime error.

Parameters

start (float) – the starting value for the set of points
end (float) – the ending value for the set of points
steps (int) – size of the constructed tensor

Keyword Arguments

out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()).
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>>> torch.linspace(3, 10, steps=5)
tensor([  3.0000,   4.7500,   6.5000,   8.2500,  10.0000])
>>> torch.linspace(-10, 10, steps=5)
tensor([-10.,  -5.,   0.,   5.,  10.])
>>> torch.linspace(start=-10, end=10, steps=5)
tensor([-10.,  -5.,   0.,   5.,  10.])
>>> torch.linspace(start=-10, end=10, steps=1)
tensor([-10.])

torch.logspace(start, end, steps, base=10.0, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor ¶

Creates a one-dimensional tensor of size steps whose values are evenly spaced from \({{\text{{base}}}}^{{\text{{start}}}}\) to \({{\text{{base}}}}^{{\text{{end}}}}\), inclusive, on a logarithmic scale with base base. That is, the values are:

\[(\text{base}^{\text{start}}, \text{base}^{(\text{start} + \frac{\text{end} - \text{start}}{ \text{steps} - 1})}, \ldots, \text{base}^{(\text{start} + (\text{steps} - 2) * \frac{\text{end} - \text{start}}{ \text{steps} - 1})}, \text{base}^{\text{end}}) \]

Warning

Not providing a value for steps is deprecated. For backwards compatibility, not providing a value for steps will create a tensor with 100 elements. Note that this behavior is not reflected in the documented function signature and should not be relied on. In a future PyTorch release, failing to provide a value for steps will throw a runtime error.

Parameters

start (float) – the starting value for the set of points
end (float) – the ending value for the set of points
steps (int) – size of the constructed tensor
base (float, optional) – base of the logarithm function. Default: 10.0.

Keyword Arguments

out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()).
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>>> torch.logspace(start=-10, end=10, steps=5)
tensor([ 1.0000e-10,  1.0000e-05,  1.0000e+00,  1.0000e+05,  1.0000e+10])
>>> torch.logspace(start=0.1, end=1.0, steps=5)
tensor([  1.2589,   2.1135,   3.5481,   5.9566,  10.0000])
>>> torch.logspace(start=0.1, end=1.0, steps=1)
tensor([1.2589])
>>> torch.logspace(start=2, end=2, steps=1, base=2)
tensor([4.0])

torch.eye(n, m=None, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor ¶

Returns a 2-D tensor with ones on the diagonal and zeros elsewhere.

Parameters

n (int) – the number of rows
m (int, optional) – the number of columns with default being n

Keyword Arguments

out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()).
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Returns

A 2-D tensor with ones on the diagonal and zeros elsewhere

Return type

Tensor

Example:

>>> torch.eye(3)
tensor([[ 1.,  0.,  0.],
        [ 0.,  1.,  0.],
        [ 0.,  0.,  1.]])

torch.empty(*size, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False, pin_memory=False) → Tensor ¶

Returns a tensor filled with uninitialized data. The shape of the tensor is defined by the variable argument size.

Parameters

size (int...) – a sequence of integers defining the shape of the output tensor. Can be a variable number of arguments or a collection like a list or tuple.

Keyword Arguments

out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()).
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.
pin_memory (bool, optional) – If set, returned tensor would be allocated in the pinned memory. Works only for CPU tensors. Default: False.
memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.contiguous_format.

Example:

>>> torch.empty(2, 3)
tensor(1.00000e-08 *
       [[ 6.3984,  0.0000,  0.0000],
        [ 0.0000,  0.0000,  0.0000]])

torch.empty_like(input, *, dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format) → Tensor ¶

Returns an uninitialized tensor with the same size as input. torch.empty_like(input) is equivalent to torch.empty(input.size(), dtype=input.dtype, layout=input.layout, device=input.device).

Parameters

input (Tensor) – the size of input will determine size of the output tensor.

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned Tensor. Default: if None, defaults to the dtype of input.
layout (torch.layout, optional) – the desired layout of returned tensor. Default: if None, defaults to the layout of input.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, defaults to the device of input.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.
memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.

Example:

>>> torch.empty((2,3), dtype=torch.int64)
tensor([[ 9.4064e+13,  2.8000e+01,  9.3493e+13],
        [ 7.5751e+18,  7.1428e+18,  7.5955e+18]])

torch.full(size, fill_value, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor ¶

Creates a tensor of size size filled with fill_value. The tensor’s dtype is inferred from fill_value.

Parameters

size (int...) – a list, tuple, or torch.Size of integers defining the shape of the output tensor.
fill_value (Scalar) – the value to fill the output tensor with.

Keyword Arguments

out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()).
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>>> torch.full((2, 3), 3.141592)
tensor([[ 3.1416,  3.1416,  3.1416],
        [ 3.1416,  3.1416,  3.1416]])

torch.full_like(input, fill_value, \*, dtype=None, layout=torch.strided, device=None, requires_grad=False, memory_format=torch.preserve_format) → Tensor ¶

Returns a tensor with the same size as input filled with fill_value. torch.full_like(input, fill_value) is equivalent to torch.full(input.size(), fill_value, dtype=input.dtype, layout=input.layout, device=input.device).

Parameters

input (Tensor) – the size of input will determine size of the output tensor.
fill_value – the number to fill the output tensor with.

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned Tensor. Default: if None, defaults to the dtype of input.
layout (torch.layout, optional) – the desired layout of returned tensor. Default: if None, defaults to the layout of input.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, defaults to the device of input.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.
memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.

Indexing, Slicing, Joining, Mutating Ops¶

torch.cat(tensors, dim=0, *, out=None) → Tensor ¶

Concatenates the given sequence of seq tensors in the given dimension. All tensors must either have the same shape (except in the concatenating dimension) or be empty.

torch.cat() can be seen as an inverse operation for torch.split() and torch.chunk().

torch.cat() can be best understood via examples.

Parameters

tensors (sequence of Tensors) – any python sequence of tensors of the same type. Non-empty tensors provided must have the same shape, except in the cat dimension.
dim (int, optional) – the dimension over which the tensors are concatenated

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> x = torch.randn(2, 3)
>>> x
tensor([[ 0.6580, -1.0969, -0.4614],
        [-0.1034, -0.5790,  0.1497]])
>>> torch.cat((x, x, x), 0)
tensor([[ 0.6580, -1.0969, -0.4614],
        [-0.1034, -0.5790,  0.1497],
        [ 0.6580, -1.0969, -0.4614],
        [-0.1034, -0.5790,  0.1497],
        [ 0.6580, -1.0969, -0.4614],
        [-0.1034, -0.5790,  0.1497]])
>>> torch.cat((x, x, x), 1)
tensor([[ 0.6580, -1.0969, -0.4614,  0.6580, -1.0969, -0.4614,  0.6580,
         -1.0969, -0.4614],
        [-0.1034, -0.5790,  0.1497, -0.1034, -0.5790,  0.1497, -0.1034,
         -0.5790,  0.1497]])

torch.chunk(input, chunks, dim=0) → List of Tensors¶

Splits a tensor into a specific number of chunks. Each chunk is a view of the input tensor.

Last chunk will be smaller if the tensor size along the given dimension dim is not divisible by chunks.

Parameters

input (Tensor) – the tensor to split
chunks (int) – number of chunks to return
dim (int) – dimension along which to split the tensor

torch.gather(input, dim, index, *, sparse_grad=False, out=None) → Tensor ¶

Gathers values along an axis specified by dim.

For a 3-D tensor the output is specified by:

out[i][j][k] = input[index[i][j][k]][j][k]  # if dim == 0
out[i][j][k] = input[i][index[i][j][k]][k]  # if dim == 1
out[i][j][k] = input[i][j][index[i][j][k]]  # if dim == 2

input and index must have the same number of dimensions. It is also required that index.size(d) <= input.size(d) for all dimensions d != dim. out will have the same shape as index. Note that input and index do not broadcast against each other.

Parameters

input (Tensor) – the source tensor
dim (int) – the axis along which to index
index (LongTensor) – the indices of elements to gather

Keyword Arguments

sparse_grad (bool, optional) – If True, gradient w.r.t. input will be a sparse tensor.
out (Tensor, optional) – the destination tensor

Example:

>>> t = torch.tensor([[1, 2], [3, 4]])
>>> torch.gather(t, 1, torch.tensor([[0, 0], [1, 0]]))
tensor([[ 1,  1],
        [ 4,  3]])

torch.index_select(input, dim, index, *, out=None) → Tensor ¶

Returns a new tensor which indexes the input tensor along dimension dim using the entries in index which is a LongTensor.

The returned tensor has the same number of dimensions as the original tensor (input). The dimth dimension has the same size as the length of index; other dimensions have the same size as in the original tensor.

Note

The returned tensor does not use the same storage as the original tensor. If out has a different shape than expected, we silently change it to the correct shape, reallocating the underlying storage if necessary.

Parameters

input (Tensor) – the input tensor.
dim (int) – the dimension in which we index
index (IntTensor or LongTensor) – the 1-D tensor containing the indices to index

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> x = torch.randn(3, 4)
>>> x
tensor([[ 0.1427,  0.0231, -0.5414, -1.0009],
        [-0.4664,  0.2647, -0.1228, -1.1068],
        [-1.1734, -0.6571,  0.7230, -0.6004]])
>>> indices = torch.tensor([0, 2])
>>> torch.index_select(x, 0, indices)
tensor([[ 0.1427,  0.0231, -0.5414, -1.0009],
        [-1.1734, -0.6571,  0.7230, -0.6004]])
>>> torch.index_select(x, 1, indices)
tensor([[ 0.1427, -0.5414],
        [-0.4664, -0.1228],
        [-1.1734,  0.7230]])

torch.masked_select(input, mask, *, out=None) → Tensor ¶

Returns a new 1-D tensor which indexes the input tensor according to the boolean mask mask which is a BoolTensor.

The shapes of the mask tensor and the input tensor don’t need to match, but they must be broadcastable.

Note

The returned tensor does not use the same storage as the original tensor

Parameters

input (Tensor) – the input tensor.
mask (BoolTensor) – the tensor containing the binary mask to index with

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> x = torch.randn(3, 4)
>>> x
tensor([[ 0.3552, -2.3825, -0.8297,  0.3477],
        [-1.2035,  1.2252,  0.5002,  0.6248],
        [ 0.1307, -2.0608,  0.1244,  2.0139]])
>>> mask = x.ge(0.5)
>>> mask
tensor([[False, False, False, False],
        [False, True, True, True],
        [False, False, False, True]])
>>> torch.masked_select(x, mask)
tensor([ 1.2252,  0.5002,  0.6248,  2.0139])

torch.narrow(input, dim, start, length) → Tensor ¶

Returns a new tensor that is a narrowed version of input tensor. The dimension dim is input from start to start + length. The returned tensor and input tensor share the same underlying storage.

Parameters

input (Tensor) – the tensor to narrow
dim (int) – the dimension along which to narrow
start (int) – the starting dimension
length (int) – the distance to the ending dimension

Example:

>>> x = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
>>> torch.narrow(x, 0, 0, 2)
tensor([[ 1,  2,  3],
        [ 4,  5,  6]])
>>> torch.narrow(x, 1, 1, 2)
tensor([[ 2,  3],
        [ 5,  6],
        [ 8,  9]])

torch.nonzero(input, *, out=None, as_tuple=False) → LongTensor or tuple of LongTensors¶

Note

torch.nonzero(..., as_tuple=False) (default) returns a 2-D tensor where each row is the index for a nonzero value.

torch.nonzero(..., as_tuple=True) returns a tuple of 1-D index tensors, allowing for advanced indexing, so x[x.nonzero(as_tuple=True)] gives all nonzero values of tensor x. Of the returned tuple, each index tensor contains nonzero indices for a certain dimension.

See below for more details on the two behaviors.

When input is on CUDA, torch.nonzero() causes host-device synchronization.

When as_tuple is ``False`` (default):

Returns a tensor containing the indices of all non-zero elements of input. Each row in the result contains the indices of a non-zero element in input. The result is sorted lexicographically, with the last index changing the fastest (C-style).

If input has \(n\) dimensions, then the resulting indices tensor out is of size \((z \times n)\), where \(z\) is the total number of non-zero elements in the input tensor.

When as_tuple is ``True``:

Returns a tuple of 1-D tensors, one for each dimension in input, each containing the indices (in that dimension) of all non-zero elements of input .

If input has \(n\) dimensions, then the resulting tuple contains \(n\) tensors of size \(z\), where \(z\) is the total number of non-zero elements in the input tensor.

As a special case, when input has zero dimensions and a nonzero scalar value, it is treated as a one-dimensional tensor with one element.

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (LongTensor, optional) – the output tensor containing indices
Returns: If as_tuple is False, the output tensor containing indices. If as_tuple is True, one 1-D tensor for each dimension, containing the indices of each nonzero element along that dimension.
Return type: LongTensor or tuple of LongTensor

Example:

>>> torch.nonzero(torch.tensor([1, 1, 1, 0, 1]))
tensor([[ 0],
        [ 1],
        [ 2],
        [ 4]])
>>> torch.nonzero(torch.tensor([[0.6, 0.0, 0.0, 0.0],
...                             [0.0, 0.4, 0.0, 0.0],
...                             [0.0, 0.0, 1.2, 0.0],
...                             [0.0, 0.0, 0.0,-0.4]]))
tensor([[ 0,  0],
        [ 1,  1],
        [ 2,  2],
        [ 3,  3]])
>>> torch.nonzero(torch.tensor([1, 1, 1, 0, 1]), as_tuple=True)
(tensor([0, 1, 2, 4]),)
>>> torch.nonzero(torch.tensor([[0.6, 0.0, 0.0, 0.0],
...                             [0.0, 0.4, 0.0, 0.0],
...                             [0.0, 0.0, 1.2, 0.0],
...                             [0.0, 0.0, 0.0,-0.4]]), as_tuple=True)
(tensor([0, 1, 2, 3]), tensor([0, 1, 2, 3]))
>>> torch.nonzero(torch.tensor(5), as_tuple=True)
(tensor([0]),)

torch.reshape(input, shape) → Tensor ¶

Returns a tensor with the same data and number of elements as input, but with the specified shape. When possible, the returned tensor will be a view of input. Otherwise, it will be a copy. Contiguous inputs and inputs with compatible strides can be reshaped without copying, but you should not depend on the copying vs. viewing behavior.

See torch.Tensor.view() on when it is possible to return a view.

A single dimension may be -1, in which case it’s inferred from the remaining dimensions and the number of elements in input.

Parameters

input (Tensor) – the tensor to be reshaped
shape (tuple of python:ints) – the new shape

Example:

>>> a = torch.arange(4.)
>>> torch.reshape(a, (2, 2))
tensor([[ 0.,  1.],
        [ 2.,  3.]])
>>> b = torch.tensor([[0, 1], [2, 3]])
>>> torch.reshape(b, (-1,))
tensor([ 0,  1,  2,  3])

torch.split(tensor, split_size_or_sections, dim=0)[source]¶

Splits the tensor into chunks. Each chunk is a view of the original tensor.

If split_size_or_sections is an integer type, then tensor will be split into equally sized chunks (if possible). Last chunk will be smaller if the tensor size along the given dimension dim is not divisible by split_size.

If split_size_or_sections is a list, then tensor will be split into len(split_size_or_sections) chunks with sizes in dim according to split_size_or_sections.

Parameters

tensor (Tensor) – tensor to split.
split_size_or_sections (int) or (list(int)) – size of a single chunk or list of sizes for each chunk
dim (int) – dimension along which to split the tensor.

Example::

>>> a = torch.arange(10).reshape(5,2)
>>> a
tensor([[0, 1],
        [2, 3],
        [4, 5],
        [6, 7],
        [8, 9]])
>>> torch.split(a, 2)
(tensor([[0, 1],
         [2, 3]]),
 tensor([[4, 5],
         [6, 7]]),
 tensor([[8, 9]]))
>>> torch.split(a, [1,4])
(tensor([[0, 1]]),
 tensor([[2, 3],
         [4, 5],
         [6, 7],
         [8, 9]]))

torch.squeeze(input, dim=None, *, out=None) → Tensor ¶

Returns a tensor with all the dimensions of input of size 1 removed.

For example, if input is of shape: \((A \times 1 \times B \times C \times 1 \times D)\) then the out tensor will be of shape: \((A \times B \times C \times D)\).

When dim is given, a squeeze operation is done only in the given dimension. If input is of shape: \((A \times 1 \times B)\), squeeze(input, 0) leaves the tensor unchanged, but squeeze(input, 1) will squeeze the tensor to the shape \((A \times B)\).

Note

The returned tensor shares the storage with the input tensor, so changing the contents of one will change the contents of the other.

Warning

If the tensor has a batch dimension of size 1, then squeeze(input) will also remove the batch dimension, which can lead to unexpected errors.

Parameters

input (Tensor) – the input tensor.
dim (int, optional) – if given, the input will be squeezed only in this dimension

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> x = torch.zeros(2, 1, 2, 1, 2)
>>> x.size()
torch.Size([2, 1, 2, 1, 2])
>>> y = torch.squeeze(x)
>>> y.size()
torch.Size([2, 2, 2])
>>> y = torch.squeeze(x, 0)
>>> y.size()
torch.Size([2, 1, 2, 1, 2])
>>> y = torch.squeeze(x, 1)
>>> y.size()
torch.Size([2, 2, 1, 2])

torch.stack(tensors, dim=0, *, out=None) → Tensor ¶

Concatenates a sequence of tensors along a new dimension.

All tensors need to be of the same size.

Parameters

tensors (sequence of Tensors) – sequence of tensors to concatenate
dim (int) – dimension to insert. Has to be between 0 and the number of dimensions of concatenated tensors (inclusive)

Keyword Arguments

out (Tensor, optional) – the output tensor.

torch.t(input) → Tensor ¶

Expects input to be <= 2-D tensor and transposes dimensions 0 and 1.

0-D and 1-D tensors are returned as is. When input is a 2-D tensor this is equivalent to transpose(input, 0, 1).

Parameters: input (Tensor) – the input tensor.

Example:

>>> x = torch.randn(())
>>> x
tensor(0.1995)
>>> torch.t(x)
tensor(0.1995)
>>> x = torch.randn(3)
>>> x
tensor([ 2.4320, -0.4608,  0.7702])
>>> torch.t(x)
tensor([ 2.4320, -0.4608,  0.7702])
>>> x = torch.randn(2, 3)
>>> x
tensor([[ 0.4875,  0.9158, -0.5872],
        [ 0.3938, -0.6929,  0.6932]])
>>> torch.t(x)
tensor([[ 0.4875,  0.3938],
        [ 0.9158, -0.6929],
        [-0.5872,  0.6932]])

torch.take(input, index) → Tensor ¶

Returns a new tensor with the elements of input at the given indices. The input tensor is treated as if it were viewed as a 1-D tensor. The result takes the same shape as the indices.

Parameters

input (Tensor) – the input tensor.
indices (LongTensor) – the indices into tensor

Example:

>>> src = torch.tensor([[4, 3, 5],
...                     [6, 7, 8]])
>>> torch.take(src, torch.tensor([0, 2, 5]))
tensor([ 4,  5,  8])

torch.transpose(input, dim0, dim1) → Tensor ¶

Returns a tensor that is a transposed version of input. The given dimensions dim0 and dim1 are swapped.

The resulting out tensor shares its underlying storage with the input tensor, so changing the content of one would change the content of the other.

Parameters

input (Tensor) – the input tensor.
dim0 (int) – the first dimension to be transposed
dim1 (int) – the second dimension to be transposed

Example:

>>> x = torch.randn(2, 3)
>>> x
tensor([[ 1.0028, -0.9893,  0.5809],
        [-0.1669,  0.7299,  0.4942]])
>>> torch.transpose(x, 0, 1)
tensor([[ 1.0028, -0.1669],
        [-0.9893,  0.7299],
        [ 0.5809,  0.4942]])

torch.unbind(input, dim=0) → seq¶

Removes a tensor dimension.

Returns a tuple of all slices along a given dimension, already without it.

Parameters

input (Tensor) – the tensor to unbind
dim (int) – dimension to remove

Example:

>>> torch.unbind(torch.tensor([[1, 2, 3],
>>>                            [4, 5, 6],
>>>                            [7, 8, 9]]))
(tensor([1, 2, 3]), tensor([4, 5, 6]), tensor([7, 8, 9]))

torch.unsqueeze(input, dim) → Tensor ¶

Returns a new tensor with a dimension of size one inserted at the specified position.

The returned tensor shares the same underlying data with this tensor.

A dim value within the range [-input.dim() - 1, input.dim() + 1) can be used. Negative dim will correspond to unsqueeze() applied at dim = dim + input.dim() + 1.

Parameters

input (Tensor) – the input tensor.
dim (int) – the index at which to insert the singleton dimension

Example:

>>> x = torch.tensor([1, 2, 3, 4])
>>> torch.unsqueeze(x, 0)
tensor([[ 1,  2,  3,  4]])
>>> torch.unsqueeze(x, 1)
tensor([[ 1],
        [ 2],
        [ 3],
        [ 4]])

torch.where(condition, x, y) → Tensor ¶

Return a tensor of elements selected from either x or y, depending on condition.

The operation is defined as:

\[\text{out}_i = \begin{cases} \text{x}_i & \text{if } \text{condition}_i \\ \text{y}_i & \text{otherwise} \\ \end{cases} \]

Note

The tensors condition, x, y must be broadcastable.

Note

Currently valid scalar and tensor combination are 1. Scalar of floating dtype and torch.double 2. Scalar of integral dtype and torch.long 3. Scalar of complex dtype and torch.complex128

Parameters

condition (BoolTensor) – When True (nonzero), yield x, otherwise yield y
x (Tensor or Scalar) – value (if :attr:x is a scalar) or values selected at indices where condition is True
y (Tensor or Scalar) – value (if :attr:x is a scalar) or values selected at indices where condition is False

Returns

A tensor of shape equal to the broadcasted shape of condition, x, y

Return type

Tensor

Example:

>>> x = torch.randn(3, 2)
>>> y = torch.ones(3, 2)
>>> x
tensor([[-0.4620,  0.3139],
        [ 0.3898, -0.7197],
        [ 0.0478, -0.1657]])
>>> torch.where(x > 0, x, y)
tensor([[ 1.0000,  0.3139],
        [ 0.3898,  1.0000],
        [ 0.0478,  1.0000]])
>>> x = torch.randn(2, 2, dtype=torch.double)
>>> x
tensor([[ 1.0779,  0.0383],
        [-0.8785, -1.1089]], dtype=torch.float64)
>>> torch.where(x > 0, x, 0.)
tensor([[1.0779, 0.0383],
        [0.0000, 0.0000]], dtype=torch.float64)

torch.where(condition) → tuple of LongTensor¶

torch.where(condition) is identical to torch.nonzero(condition, as_tuple=True).

Note

Random sampling¶

torch.manual_seed(seed) → torch._C.Generator[source]¶

Sets the seed for generating random numbers. Returns a torch.Generator object.

Parameters: seed (int) – The desired seed. Value must be within the inclusive range [-0x8000_0000_0000_0000, 0xffff_ffff_ffff_ffff]. Otherwise, a RuntimeError is raised. Negative inputs are remapped to positive values with the formula 0xffff_ffff_ffff_ffff + seed.

torch.initial_seed() → int [source]¶: Returns the initial seed for generating random numbers as a Python long.

torch.get_rng_state() → torch.Tensor [source]¶: Returns the random number generator state as a torch.ByteTensor.

torch.set_rng_state(new_state) → None [source]¶

Sets the random number generator state.

Parameters: new_state (torch.ByteTensor) – The desired state

torch.default_generator = <torch._C.Generator object>¶

torch.bernoulli(input, *, generator=None, out=None) → Tensor ¶

Draws binary random numbers (0 or 1) from a Bernoulli distribution.

The input tensor should be a tensor containing probabilities to be used for drawing the binary random number. Hence, all values in input have to be in the range: \(0 \leq \text{input}_i \leq 1\).

The \(\text{i}^{th}\) element of the output tensor will draw a value \(1\) according to the \(\text{i}^{th}\) probability value given in input.

\[\text{out}_{i} \sim \mathrm{Bernoulli}(p = \text{input}_{i}) \]

The returned out tensor only has values 0 or 1 and is of the same shape as input.

out can have integral dtype, but input must have floating point dtype.

Parameters

input (Tensor) – the input tensor of probability values for the Bernoulli distribution

Keyword Arguments

generator (torch.Generator, optional) – a pseudorandom number generator for sampling
out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.empty(3, 3).uniform_(0, 1)  # generate a uniform random matrix with range [0, 1]
>>> a
tensor([[ 0.1737,  0.0950,  0.3609],
        [ 0.7148,  0.0289,  0.2676],
        [ 0.9456,  0.8937,  0.7202]])
>>> torch.bernoulli(a)
tensor([[ 1.,  0.,  0.],
        [ 0.,  0.,  0.],
        [ 1.,  1.,  1.]])

>>> a = torch.ones(3, 3) # probability of drawing "1" is 1
>>> torch.bernoulli(a)
tensor([[ 1.,  1.,  1.],
        [ 1.,  1.,  1.],
        [ 1.,  1.,  1.]])
>>> a = torch.zeros(3, 3) # probability of drawing "1" is 0
>>> torch.bernoulli(a)
tensor([[ 0.,  0.,  0.],
        [ 0.,  0.,  0.],
        [ 0.,  0.,  0.]])

torch.multinomial(input, num_samples, replacement=False, *, generator=None, out=None) → LongTensor¶

Returns a tensor where each row contains num_samples indices sampled from the multinomial probability distribution located in the corresponding row of tensor input.

Note

The rows of input do not need to sum to one (in which case we use the values as weights), but must be non-negative, finite and have a non-zero sum.

Indices are ordered from left to right according to when each was sampled (first samples are placed in first column).

If input is a vector, out is a vector of size num_samples.

If input is a matrix with m rows, out is an matrix of shape \((m \times \text{num\_samples})\).

If replacement is True, samples are drawn with replacement.

If not, they are drawn without replacement, which means that when a sample index is drawn for a row, it cannot be drawn again for that row.

Note

When drawn without replacement, num_samples must be lower than number of non-zero elements in input (or the min number of non-zero elements in each row of input if it is a matrix).

Parameters

input (Tensor) – the input tensor containing probabilities
num_samples (int) – number of samples to draw
replacement (bool, optional) – whether to draw with replacement or not

Keyword Arguments

generator (torch.Generator, optional) – a pseudorandom number generator for sampling
out (Tensor, optional) – the output tensor.

Example:

>>> weights = torch.tensor([0, 10, 3, 0], dtype=torch.float) # create a tensor of weights
>>> torch.multinomial(weights, 2)
tensor([1, 2])
>>> torch.multinomial(weights, 4) # ERROR!
RuntimeError: invalid argument 2: invalid multinomial distribution (with replacement=False,
not enough non-negative category to sample) at ../aten/src/TH/generic/THTensorRandom.cpp:320
>>> torch.multinomial(weights, 4, replacement=True)
tensor([ 2,  1,  1,  1])

torch.normal(mean, std, *, generator=None, out=None) → Tensor ¶

Returns a tensor of random numbers drawn from separate normal distributions whose mean and standard deviation are given.

The mean is a tensor with the mean of each output element’s normal distribution

The std is a tensor with the standard deviation of each output element’s normal distribution

The shapes of mean and std don’t need to match, but the total number of elements in each tensor need to be the same.

Note

When the shapes do not match, the shape of mean is used as the shape for the returned output tensor

Parameters

mean (Tensor) – the tensor of per-element means
std (Tensor) – the tensor of per-element standard deviations

Keyword Arguments

generator (torch.Generator, optional) – a pseudorandom number generator for sampling
out (Tensor, optional) – the output tensor.

Example:

>>> torch.normal(mean=torch.arange(1., 11.), std=torch.arange(1, 0, -0.1))
tensor([  1.0425,   3.5672,   2.7969,   4.2925,   4.7229,   6.2134,
          8.0505,   8.1408,   9.0563,  10.0566])

torch.normal(mean=0.0, std, *, out=None) → Tensor ¶

Similar to the function above, but the means are shared among all drawn elements.

Parameters

mean (float, optional) – the mean for all distributions
std (Tensor) – the tensor of per-element standard deviations

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> torch.normal(mean=0.5, std=torch.arange(1., 6.))
tensor([-1.2793, -1.0732, -2.0687,  5.1177, -1.2303])

torch.normal(mean, std=1.0, *, out=None) → Tensor ¶

Similar to the function above, but the standard-deviations are shared among all drawn elements.

Parameters

mean (Tensor) – the tensor of per-element means
std (float, optional) – the standard deviation for all distributions

Keyword Arguments

out (Tensor, optional) – the output tensor

Example:

>>> torch.normal(mean=torch.arange(1., 6.))
tensor([ 1.1552,  2.6148,  2.6535,  5.8318,  4.2361])

torch.normal(mean, std, size, *, out=None) → Tensor ¶

Similar to the function above, but the means and standard deviations are shared among all drawn elements. The resulting tensor has size given by size.

Parameters

mean (float) – the mean for all distributions
std (float) – the standard deviation for all distributions
size (int...) – a sequence of integers defining the shape of the output tensor.

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> torch.normal(2, 3, size=(1, 4))
tensor([[-1.3987, -1.9544,  3.6048,  0.7909]])

torch.rand(*size, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor ¶

Returns a tensor filled with random numbers from a uniform distribution on the interval \([0, 1)\)

The shape of the tensor is defined by the variable argument size.

Parameters

size (int...) – a sequence of integers defining the shape of the output tensor. Can be a variable number of arguments or a collection like a list or tuple.

Keyword Arguments

out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()).
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>>> torch.rand(4)
tensor([ 0.5204,  0.2503,  0.3525,  0.5673])
>>> torch.rand(2, 3)
tensor([[ 0.8237,  0.5781,  0.6879],
        [ 0.3816,  0.7249,  0.0998]])

torch.rand_like(input, *, dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format) → Tensor ¶

Returns a tensor with the same size as input that is filled with random numbers from a uniform distribution on the interval \([0, 1)\). torch.rand_like(input) is equivalent to torch.rand(input.size(), dtype=input.dtype, layout=input.layout, device=input.device).

Parameters

input (Tensor) – the size of input will determine size of the output tensor.

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned Tensor. Default: if None, defaults to the dtype of input.
layout (torch.layout, optional) – the desired layout of returned tensor. Default: if None, defaults to the layout of input.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, defaults to the device of input.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.
memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.

torch.randint(low=0, high, size, \*, generator=None, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor ¶

Returns a tensor filled with random integers generated uniformly between low (inclusive) and high (exclusive).

The shape of the tensor is defined by the variable argument size.

Note

With the global dtype default (torch.float32), this function returns a tensor with dtype torch.int64.

Parameters

low (int, optional) – Lowest integer to be drawn from the distribution. Default: 0.
high (int) – One above the highest integer to be drawn from the distribution.
size (tuple) – a tuple defining the shape of the output tensor.

Keyword Arguments

generator (torch.Generator, optional) – a pseudorandom number generator for sampling
out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()).
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>>> torch.randint(3, 5, (3,))
tensor([4, 3, 4])


>>> torch.randint(10, (2, 2))
tensor([[0, 2],
        [5, 5]])


>>> torch.randint(3, 10, (2, 2))
tensor([[4, 5],
        [6, 7]])

torch.randint_like(input, low=0, high, \*, dtype=None, layout=torch.strided, device=None, requires_grad=False, memory_format=torch.preserve_format) → Tensor ¶

Returns a tensor with the same shape as Tensor input filled with random integers generated uniformly between low (inclusive) and high (exclusive).

Parameters

input (Tensor) – the size of input will determine size of the output tensor.
low (int, optional) – Lowest integer to be drawn from the distribution. Default: 0.
high (int) – One above the highest integer to be drawn from the distribution.

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned Tensor. Default: if None, defaults to the dtype of input.
layout (torch.layout, optional) – the desired layout of returned tensor. Default: if None, defaults to the layout of input.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, defaults to the device of input.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.
memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.

torch.randn(*size, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor ¶

Returns a tensor filled with random numbers from a normal distribution with mean 0 and variance 1 (also called the standard normal distribution).

\[\text{out}_{i} \sim \mathcal{N}(0, 1) \]

The shape of the tensor is defined by the variable argument size.

Parameters

size (int...) – a sequence of integers defining the shape of the output tensor. Can be a variable number of arguments or a collection like a list or tuple.

Keyword Arguments

out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()).
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Example:

>>> torch.randn(4)
tensor([-2.1436,  0.9966,  2.3426, -0.6366])
>>> torch.randn(2, 3)
tensor([[ 1.5954,  2.8929, -1.0923],
        [ 1.1719, -0.4709, -0.1996]])

torch.randn_like(input, *, dtype=None, layout=None, device=None, requires_grad=False, memory_format=torch.preserve_format) → Tensor ¶

Returns a tensor with the same size as input that is filled with random numbers from a normal distribution with mean 0 and variance 1. torch.randn_like(input) is equivalent to torch.randn(input.size(), dtype=input.dtype, layout=input.layout, device=input.device).

Parameters

input (Tensor) – the size of input will determine size of the output tensor.

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned Tensor. Default: if None, defaults to the dtype of input.
layout (torch.layout, optional) – the desired layout of returned tensor. Default: if None, defaults to the layout of input.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, defaults to the device of input.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.
memory_format (torch.memory_format, optional) – the desired memory format of returned Tensor. Default: torch.preserve_format.

torch.randperm(n, *, generator=None, out=None, dtype=torch.int64, layout=torch.strided, device=None, requires_grad=False, pin_memory=False) → Tensor ¶

Returns a random permutation of integers from 0 to n - 1.

Parameters

n (int) – the upper bound (exclusive)

Keyword Arguments

generator (torch.Generator, optional) – a pseudorandom number generator for sampling
out (Tensor, optional) – the output tensor.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: torch.int64.
layout (torch.layout, optional) – the desired layout of returned Tensor. Default: torch.strided.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.
pin_memory (bool, optional) – If set, returned tensor would be allocated in the pinned memory. Works only for CPU tensors. Default: False.

Example:

>>> torch.randperm(4)
tensor([2, 1, 0, 3])

In-place random sampling¶

There are a few more in-place random sampling functions defined on Tensors as well. Click through to refer to their documentation:

torch.Tensor.bernoulli_() - in-place version of torch.bernoulli()
torch.Tensor.cauchy_() - numbers drawn from the Cauchy distribution
torch.Tensor.exponential_() - numbers drawn from the exponential distribution
torch.Tensor.geometric_() - elements drawn from the geometric distribution
torch.Tensor.log_normal_() - samples from the log-normal distribution
torch.Tensor.normal_() - in-place version of torch.normal()
torch.Tensor.random_() - numbers sampled from the discrete uniform distribution
torch.Tensor.uniform_() - numbers sampled from the continuous uniform distribution

Serialization¶

torch.save(obj, f: Union[str, os.PathLike, BinaryIO, IO[bytes]], pickle_module=<module 'pickle' from '/Users/brandongreen/opt/anaconda3/envs/pytorch/lib/python3.6/pickle.py'>, pickle_protocol=2, _use_new_zipfile_serialization=True) → None [source]¶

Saves an object to a disk file.

Parallelism¶

torch.get_num_threads() → int ¶: Returns the number of threads used for parallelizing CPU operations

torch.set_num_threads(int)¶: Sets the number of threads used for intraop parallelism on CPU.

Warning

To ensure that the correct number of threads is used, set_num_threads must be called before running eager, JIT or autograd code.

Locally disabling gradient computation¶

The context managers torch.no_grad(), torch.enable_grad(), and torch.set_grad_enabled() are helpful for locally disabling and enabling gradient computation. See Locally disabling gradient computation for more details on their usage.

Examples:

>>> x = torch.zeros(1, requires_grad=True)
>>> with torch.no_grad():
...     y = x * 2
>>> y.requires_grad
False

>>> is_train = False
>>> with torch.set_grad_enabled(is_train):
...     y = x * 2
>>> y.requires_grad
False

>>> torch.set_grad_enabled(True)  # this can also be used as a function
>>> y = x * 2
>>> y.requires_grad
True

>>> torch.set_grad_enabled(False)
>>> y = x * 2
>>> y.requires_grad
False

Math operations¶

Pointwise Ops¶

torch.abs(input, *, out=None) → Tensor ¶

Computes the absolute value of each element in input.

\[\text{out}_{i} = |\text{input}_{i}| \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> torch.abs(torch.tensor([-1, -2, 3]))
tensor([ 1,  2,  3])

torch.acos(input, *, out=None) → Tensor ¶

Computes the inverse cosine of each element in input.

\[\text{out}_{i} = \cos^{-1}(\text{input}_{i}) \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4)
>>> a
tensor([ 0.3348, -0.5889,  0.2005, -0.1584])
>>> torch.acos(a)
tensor([ 1.2294,  2.2004,  1.3690,  1.7298])

torch.add(input, other, *, out=None)¶

Adds the scalar other to each element of the input input and returns a new resulting tensor.

\[\text{out} = \text{input} + \text{other} \]

If input is of type FloatTensor or DoubleTensor, other must be a real number, otherwise it should be an integer.

Parameters

input (Tensor) – the input tensor.
value (Number) – the number to be added to each element of input

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4)
>>> a
tensor([ 0.0202,  1.0985,  1.3506, -0.6056])
>>> torch.add(a, 20)
tensor([ 20.0202,  21.0985,  21.3506,  19.3944])

torch.add(input, other, *, alpha=1, out=None)¶

Each element of the tensor other is multiplied by the scalar alpha and added to each element of the tensor input. The resulting tensor is returned.

The shapes of input and other must be broadcastable.

\[\text{out} = \text{input} + \text{alpha} \times \text{other} \]

If other is of type FloatTensor or DoubleTensor, alpha must be a real number, otherwise it should be an integer.

Parameters

input (Tensor) – the first input tensor
other (Tensor) – the second input tensor

Keyword Arguments

alpha (Number) – the scalar multiplier for other
out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4)
>>> a
tensor([-0.9732, -0.3497,  0.6245,  0.4022])
>>> b = torch.randn(4, 1)
>>> b
tensor([[ 0.3743],
        [-1.7724],
        [-0.5811],
        [-0.8017]])
>>> torch.add(a, b, alpha=10)
tensor([[  2.7695,   3.3930,   4.3672,   4.1450],
        [-18.6971, -18.0736, -17.0994, -17.3216],
        [ -6.7845,  -6.1610,  -5.1868,  -5.4090],
        [ -8.9902,  -8.3667,  -7.3925,  -7.6147]])

torch.addcdiv(input, tensor1, tensor2, *, value=1, out=None) → Tensor ¶

Performs the element-wise division of tensor1 by tensor2, multiply the result by the scalar value and add it to input.

Warning

Integer division with addcdiv is no longer supported, and in a future release addcdiv will perform a true division of tensor1 and tensor2. The historic addcdiv behavior can be implemented as (input + value * torch.trunc(tensor1 / tensor2)).to(input.dtype) for integer inputs and as (input + value * tensor1 / tensor2) for float inputs. The future addcdiv behavior is just the latter implementation: (input + value * tensor1 / tensor2), for all dtypes.

\[\text{out}_i = \text{input}_i + \text{value} \times \frac{\text{tensor1}_i}{\text{tensor2}_i} \]

The shapes of input, tensor1, and tensor2 must be broadcastable.

For inputs of type FloatTensor or DoubleTensor, value must be a real number, otherwise an integer.

Parameters

input (Tensor) – the tensor to be added
tensor1 (Tensor) – the numerator tensor
tensor2 (Tensor) – the denominator tensor

Keyword Arguments

value (Number, optional) – multiplier for \(\text{tensor1} / \text{tensor2}\)
out (Tensor, optional) – the output tensor.

Example:

>>> t = torch.randn(1, 3)
>>> t1 = torch.randn(3, 1)
>>> t2 = torch.randn(1, 3)
>>> torch.addcdiv(t, t1, t2, value=0.1)
tensor([[-0.2312, -3.6496,  0.1312],
        [-1.0428,  3.4292, -0.1030],
        [-0.5369, -0.9829,  0.0430]])

torch.addcmul(input, tensor1, tensor2, *, value=1, out=None) → Tensor ¶

Performs the element-wise multiplication of tensor1 by tensor2, multiply the result by the scalar value and add it to input.

\[\text{out}_i = \text{input}_i + \text{value} \times \text{tensor1}_i \times \text{tensor2}_i \]

The shapes of tensor, tensor1, and tensor2 must be broadcastable.

For inputs of type FloatTensor or DoubleTensor, value must be a real number, otherwise an integer.

Parameters

input (Tensor) – the tensor to be added
tensor1 (Tensor) – the tensor to be multiplied
tensor2 (Tensor) – the tensor to be multiplied

Keyword Arguments

value (Number, optional) – multiplier for \(tensor1 .* tensor2\)
out (Tensor, optional) – the output tensor.

Example:

>>> t = torch.randn(1, 3)
>>> t1 = torch.randn(3, 1)
>>> t2 = torch.randn(1, 3)
>>> torch.addcmul(t, t1, t2, value=0.1)
tensor([[-0.8635, -0.6391,  1.6174],
        [-0.7617, -0.5879,  1.7388],
        [-0.8353, -0.6249,  1.6511]])

torch.asin(input, *, out=None) → Tensor ¶

Returns a new tensor with the arcsine of the elements of input.

\[\text{out}_{i} = \sin^{-1}(\text{input}_{i}) \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4)
>>> a
tensor([-0.5962,  1.4985, -0.4396,  1.4525])
>>> torch.asin(a)
tensor([-0.6387,     nan, -0.4552,     nan])

torch.atan(input, *, out=None) → Tensor ¶

Returns a new tensor with the arctangent of the elements of input.

\[\text{out}_{i} = \tan^{-1}(\text{input}_{i}) \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4)
>>> a
tensor([ 0.2341,  0.2539, -0.6256, -0.6448])
>>> torch.atan(a)
tensor([ 0.2299,  0.2487, -0.5591, -0.5727])

torch.atan2(input, other, *, out=None) → Tensor ¶

Element-wise arctangent of \(\text{input}_{i} / \text{other}_{i}\) with consideration of the quadrant. Returns a new tensor with the signed angles in radians between vector \((\text{other}_{i}, \text{input}_{i})\) and vector \((1, 0)\). (Note that \(\text{other}_{i}\), the second parameter, is the x-coordinate, while \(\text{input}_{i}\), the first parameter, is the y-coordinate.)

The shapes of input and other must be broadcastable.

Parameters

input (Tensor) – the first input tensor
other (Tensor) – the second input tensor

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4)
>>> a
tensor([ 0.9041,  0.0196, -0.3108, -2.4423])
>>> torch.atan2(a, torch.randn(4))
tensor([ 0.9833,  0.0811, -1.9743, -1.4151])

torch.ceil(input, *, out=None) → Tensor ¶

Returns a new tensor with the ceil of the elements of input, the smallest integer greater than or equal to each element.

\[\text{out}_{i} = \left\lceil \text{input}_{i} \right\rceil = \left\lfloor \text{input}_{i} \right\rfloor + 1 \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4)
>>> a
tensor([-0.6341, -1.4208, -1.0900,  0.5826])
>>> torch.ceil(a)
tensor([-0., -1., -1.,  1.])

torch.clamp(input, min, max, *, out=None) → Tensor ¶

Clamp all elements in input into the range [ min, max ]. Let min_value and max_value be min and max, respectively, this returns:

\[y_i = \min(\max(x_i, \text{min\_value}), \text{max\_value}) \]

Parameters

input (Tensor) – the input tensor.
min (Number) – lower-bound of the range to be clamped to
max (Number) – upper-bound of the range to be clamped to

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4)
>>> a
tensor([-1.7120,  0.1734, -0.0478, -0.0922])
>>> torch.clamp(a, min=-0.5, max=0.5)
tensor([-0.5000,  0.1734, -0.0478, -0.0922])

torch.clamp(input, *, min, out=None) → Tensor ¶

Clamps all elements in input to be larger or equal min.

Parameters

input (Tensor) – the input tensor.

Keyword Arguments

min (Number) – minimal value of each element in the output
out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4)
>>> a
tensor([-0.0299, -2.3184,  2.1593, -0.8883])
>>> torch.clamp(a, min=0.5)
tensor([ 0.5000,  0.5000,  2.1593,  0.5000])

torch.clamp(input, *, max, out=None) → Tensor ¶

Clamps all elements in input to be smaller or equal max.

Parameters

input (Tensor) – the input tensor.

Keyword Arguments

max (Number) – maximal value of each element in the output
out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4)
>>> a
tensor([ 0.7753, -0.4702, -0.4599,  1.1899])
>>> torch.clamp(a, max=0.5)
tensor([ 0.5000, -0.4702, -0.4599,  0.5000])

torch.cos(input, *, out=None) → Tensor ¶

Returns a new tensor with the cosine of the elements of input.

\[\text{out}_{i} = \cos(\text{input}_{i}) \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4)
>>> a
tensor([ 1.4309,  1.2706, -0.8562,  0.9796])
>>> torch.cos(a)
tensor([ 0.1395,  0.2957,  0.6553,  0.5574])

torch.cosh(input, *, out=None) → Tensor ¶

Returns a new tensor with the hyperbolic cosine of the elements of input.

\[\text{out}_{i} = \cosh(\text{input}_{i}) \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4)
>>> a
tensor([ 0.1632,  1.1835, -0.6979, -0.7325])
>>> torch.cosh(a)
tensor([ 1.0133,  1.7860,  1.2536,  1.2805])

Note

When input is on the CPU, the implementation of torch.cosh may use the Sleef library, which rounds very large results to infinity or negative infinity. See here for details.

torch.div(input, other, *, rounding_mode=None, out=None) → Tensor ¶

Divides each element of the input input by the corresponding element of other.

\[\text{out}_i = \frac{\text{input}_i}{\text{other}_i} \]

Note

By default, this performs a “true” division like Python 3. See the rounding_mode argument for floor division.

Supports broadcasting to a common shape, type promotion, and integer, float, and complex inputs. Always promotes integer types to the default scalar type.

Parameters

input (Tensor) – the dividend
other (Tensor or Number) – the divisor

Keyword Arguments

rounding_mode (str, optional) –
Type of rounding applied to the result:
- None - default behavior. Performs no rounding and, if both input and other are integer types, promotes the inputs to the default scalar type. Equivalent to true division in Python (the / operator) and NumPy’s np.true_divide.
- "trunc" - rounds the results of the division towards zero. Equivalent to C-style integer division.
- "floor" - rounds the results of the division down. Equivalent to floor division in Python (the // operator) and NumPy’s np.floor_divide.
out (Tensor, optional) – the output tensor.

Examples:

>>> x = torch.tensor([ 0.3810,  1.2774, -0.2972, -0.3719,  0.4637])
>>> torch.div(x, 0.5)
tensor([ 0.7620,  2.5548, -0.5944, -0.7438,  0.9274])

>>> a = torch.tensor([[-0.3711, -1.9353, -0.4605, -0.2917],
...                   [ 0.1815, -1.0111,  0.9805, -1.5923],
...                   [ 0.1062,  1.4581,  0.7759, -1.2344],
...                   [-0.1830, -0.0313,  1.1908, -1.4757]])
>>> b = torch.tensor([ 0.8032,  0.2930, -0.8113, -0.2308])
>>> torch.div(a, b)
tensor([[-0.4620, -6.6051,  0.5676,  1.2639],
        [ 0.2260, -3.4509, -1.2086,  6.8990],
        [ 0.1322,  4.9764, -0.9564,  5.3484],
        [-0.2278, -0.1068, -1.4678,  6.3938]])

>>> torch.div(a, b, rounding_mode='trunc')
tensor([[-0., -6.,  0.,  1.],
        [ 0., -3., -1.,  6.],
        [ 0.,  4., -0.,  5.],
        [-0., -0., -1.,  6.]])

>>> torch.div(a, b, rounding_mode='floor')
tensor([[-1., -7.,  0.,  1.],
        [ 0., -4., -2.,  6.],
        [ 0.,  4., -1.,  5.],
        [-1., -1., -2.,  6.]])

torch.digamma(input, *, out=None) → Tensor ¶

Computes the logarithmic derivative of the gamma function on input.

\[\psi(x) = \frac{d}{dx} \ln\left(\Gamma\left(x\right)\right) = \frac{\Gamma'(x)}{\Gamma(x)} \]

Parameters: input (Tensor) – the tensor to compute the digamma function on
Keyword Arguments: out (Tensor, optional) – the output tensor.

Note

This function is similar to SciPy’s scipy.special.digamma.

Note

From PyTorch 1.8 onwards, the digamma function returns -Inf for 0. Previously it returned NaN for 0.

Example:

>>> a = torch.tensor([1, 0.5])
>>> torch.digamma(a)
tensor([-0.5772, -1.9635])

torch.erf(input, *, out=None) → Tensor ¶

Computes the error function of each element. The error function is defined as follows:

\[\mathrm{erf}(x) = \frac{2}{\sqrt{\pi}} \int_{0}^{x} e^{-t^2} dt \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> torch.erf(torch.tensor([0, -1., 10.]))
tensor([ 0.0000, -0.8427,  1.0000])

torch.erfc(input, *, out=None) → Tensor ¶

Computes the complementary error function of each element of input. The complementary error function is defined as follows:

\[\mathrm{erfc}(x) = 1 - \frac{2}{\sqrt{\pi}} \int_{0}^{x} e^{-t^2} dt \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> torch.erfc(torch.tensor([0, -1., 10.]))
tensor([ 1.0000, 1.8427,  0.0000])

torch.erfinv(input, *, out=None) → Tensor ¶

Computes the inverse error function of each element of input. The inverse error function is defined in the range \((-1, 1)\) as:

\[\mathrm{erfinv}(\mathrm{erf}(x)) = x \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> torch.erfinv(torch.tensor([0, 0.5, -1.]))
tensor([ 0.0000,  0.4769,    -inf])

torch.exp(input, *, out=None) → Tensor ¶

Returns a new tensor with the exponential of the elements of the input tensor input.

\[y_{i} = e^{x_{i}} \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> torch.exp(torch.tensor([0, math.log(2.)]))
tensor([ 1.,  2.])

torch.expm1(input, *, out=None) → Tensor ¶

Returns a new tensor with the exponential of the elements minus 1 of input.

\[y_{i} = e^{x_{i}} - 1 \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> torch.expm1(torch.tensor([0, math.log(2.)]))
tensor([ 0.,  1.])

torch.floor(input, *, out=None) → Tensor ¶

Returns a new tensor with the floor of the elements of input, the largest integer less than or equal to each element.

\[\text{out}_{i} = \left\lfloor \text{input}_{i} \right\rfloor \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4)
>>> a
tensor([-0.8166,  1.5308, -0.2530, -0.2091])
>>> torch.floor(a)
tensor([-1.,  1., -1., -1.])

torch.fmod(input, other, *, out=None) → Tensor ¶

Computes the element-wise remainder of division.

The dividend and divisor may contain both for integer and floating point numbers. The remainder has the same sign as the dividend input.

Supports broadcasting to a common shape, type promotion, and integer and float inputs.

Note

When the divisor is zero, returns NaN for floating point dtypes on both CPU and GPU; raises RuntimeError for integer division by zero on CPU; Integer division by zero on GPU may return any value.

Parameters

input (Tensor) – the dividend
other (Tensor or Scalar) – the divisor

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> torch.fmod(torch.tensor([-3., -2, -1, 1, 2, 3]), 2)
tensor([-1., -0., -1.,  1.,  0.,  1.])
>>> torch.fmod(torch.tensor([1, 2, 3, 4, 5]), 1.5)
tensor([1.0000, 0.5000, 0.0000, 1.0000, 0.5000])

torch.frac(input, *, out=None) → Tensor ¶

Computes the fractional portion of each element in input.

\[\text{out}_{i} = \text{input}_{i} - \left\lfloor |\text{input}_{i}| \right\rfloor * \operatorname{sgn}(\text{input}_{i}) \]

Example:

>>> torch.frac(torch.tensor([1, 2.5, -3.2]))
tensor([ 0.0000,  0.5000, -0.2000])

torch.lerp(input, end, weight, *, out=None)¶

Does a linear interpolation of two tensors start (given by input) and end based on a scalar or tensor weight and returns the resulting out tensor.

\[\text{out}_i = \text{start}_i + \text{weight}_i \times (\text{end}_i - \text{start}_i) \]

The shapes of start and end must be broadcastable. If weight is a tensor, then the shapes of weight, start, and end must be broadcastable.

Parameters

input (Tensor) – the tensor with the starting points
end (Tensor) – the tensor with the ending points
weight (float or tensor) – the weight for the interpolation formula

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> start = torch.arange(1., 5.)
>>> end = torch.empty(4).fill_(10)
>>> start
tensor([ 1.,  2.,  3.,  4.])
>>> end
tensor([ 10.,  10.,  10.,  10.])
>>> torch.lerp(start, end, 0.5)
tensor([ 5.5000,  6.0000,  6.5000,  7.0000])
>>> torch.lerp(start, end, torch.full_like(start, 0.5))
tensor([ 5.5000,  6.0000,  6.5000,  7.0000])

torch.log(input, *, out=None) → Tensor ¶

Returns a new tensor with the natural logarithm of the elements of input.

\[y_{i} = \log_{e} (x_{i}) \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(5)
>>> a
tensor([-0.7168, -0.5471, -0.8933, -1.4428, -0.1190])
>>> torch.log(a)
tensor([ nan,  nan,  nan,  nan,  nan])

torch.log10(input, *, out=None) → Tensor ¶

Returns a new tensor with the logarithm to the base 10 of the elements of input.

\[y_{i} = \log_{10} (x_{i}) \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.rand(5)
>>> a
tensor([ 0.5224,  0.9354,  0.7257,  0.1301,  0.2251])


>>> torch.log10(a)
tensor([-0.2820, -0.0290, -0.1392, -0.8857, -0.6476])

torch.log1p(input, *, out=None) → Tensor ¶

Returns a new tensor with the natural logarithm of (1 + input).

\[y_i = \log_{e} (x_i + 1) \]

Note

This function is more accurate than torch.log() for small values of input

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(5)
>>> a
tensor([-1.0090, -0.9923,  1.0249, -0.5372,  0.2492])
>>> torch.log1p(a)
tensor([    nan, -4.8653,  0.7055, -0.7705,  0.2225])

torch.log2(input, *, out=None) → Tensor ¶

Returns a new tensor with the logarithm to the base 2 of the elements of input.

\[y_{i} = \log_{2} (x_{i}) \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.rand(5)
>>> a
tensor([ 0.8419,  0.8003,  0.9971,  0.5287,  0.0490])


>>> torch.log2(a)
tensor([-0.2483, -0.3213, -0.0042, -0.9196, -4.3504])

torch.mul(input, other, *, out=None)¶

Multiplies each element of the input input with the scalar other and returns a new resulting tensor.

\[\text{out}_i = \text{other} \times \text{input}_i \]

If input is of type FloatTensor or DoubleTensor, other should be a real number, otherwise it should be an integer

Parameters

{input} –
other (Number) – the number to be multiplied to each element of input

Keyword Arguments

{out} –

Example:

>>> a = torch.randn(3)
>>> a
tensor([ 0.2015, -0.4255,  2.6087])
>>> torch.mul(a, 100)
tensor([  20.1494,  -42.5491,  260.8663])

torch.mul(input, other, *, out=None)¶

Each element of the tensor input is multiplied by the corresponding element of the Tensor other. The resulting tensor is returned.

The shapes of input and other must be broadcastable.

\[\text{out}_i = \text{input}_i \times \text{other}_i \]

Parameters

input (Tensor) – the first multiplicand tensor
other (Tensor) – the second multiplicand tensor

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4, 1)
>>> a
tensor([[ 1.1207],
        [-0.3137],
        [ 0.0700],
        [ 0.8378]])
>>> b = torch.randn(1, 4)
>>> b
tensor([[ 0.5146,  0.1216, -0.5244,  2.2382]])
>>> torch.mul(a, b)
tensor([[ 0.5767,  0.1363, -0.5877,  2.5083],
        [-0.1614, -0.0382,  0.1645, -0.7021],
        [ 0.0360,  0.0085, -0.0367,  0.1567],
        [ 0.4312,  0.1019, -0.4394,  1.8753]])

torch.mvlgamma(input, p) → Tensor ¶

Computes the multivariate log-gamma function) with dimension \(p\) element-wise, given by

\[\log(\Gamma_{p}(a)) = C + \displaystyle \sum_{i=1}^{p} \log\left(\Gamma\left(a - \frac{i - 1}{2}\right)\right) \]

where \(C = \log(\pi) \times \frac{p (p - 1)}{4}\) and \(\Gamma(\cdot)\) is the Gamma function.

All elements must be greater than \(\frac{p - 1}{2}\), otherwise an error would be thrown.

Parameters

input (Tensor) – the tensor to compute the multivariate log-gamma function
p (int) – the number of dimensions

Example:

>>> a = torch.empty(2, 3).uniform_(1, 2)
>>> a
tensor([[1.6835, 1.8474, 1.1929],
        [1.0475, 1.7162, 1.4180]])
>>> torch.mvlgamma(a, 2)
tensor([[0.3928, 0.4007, 0.7586],
        [1.0311, 0.3901, 0.5049]])

torch.neg(input, *, out=None) → Tensor ¶

Returns a new tensor with the negative of the elements of input.

\[\text{out} = -1 \times \text{input} \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(5)
>>> a
tensor([ 0.0090, -0.2262, -0.0682, -0.2866,  0.3940])
>>> torch.neg(a)
tensor([-0.0090,  0.2262,  0.0682,  0.2866, -0.3940])

torch.pow(input, exponent, *, out=None) → Tensor ¶

Takes the power of each element in input with exponent and returns a tensor with the result.

exponent can be either a single float number or a Tensor with the same number of elements as input.

When exponent is a scalar value, the operation applied is:

\[\text{out}_i = x_i ^ \text{exponent} \]

When exponent is a tensor, the operation applied is:

\[\text{out}_i = x_i ^ {\text{exponent}_i} \]

When exponent is a tensor, the shapes of input and exponent must be broadcastable.

Parameters

input (Tensor) – the input tensor.
exponent (float or tensor) – the exponent value

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4)
>>> a
tensor([ 0.4331,  1.2475,  0.6834, -0.2791])
>>> torch.pow(a, 2)
tensor([ 0.1875,  1.5561,  0.4670,  0.0779])
>>> exp = torch.arange(1., 5.)

>>> a = torch.arange(1., 5.)
>>> a
tensor([ 1.,  2.,  3.,  4.])
>>> exp
tensor([ 1.,  2.,  3.,  4.])
>>> torch.pow(a, exp)
tensor([   1.,    4.,   27.,  256.])

torch.pow(self, exponent, *, out=None) → Tensor ¶

self is a scalar float value, and exponent is a tensor. The returned tensor out is of the same shape as exponent

The operation applied is:

\[\text{out}_i = \text{self} ^ {\text{exponent}_i} \]

Parameters

self (float) – the scalar base value for the power operation
exponent (Tensor) – the exponent tensor

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> exp = torch.arange(1., 5.)
>>> base = 2
>>> torch.pow(base, exp)
tensor([  2.,   4.,   8.,  16.])

torch.reciprocal(input, *, out=None) → Tensor ¶

Returns a new tensor with the reciprocal of the elements of input

Note

Unlike NumPy’s reciprocal, torch.reciprocal supports integral inputs. Integral inputs to reciprocal are automatically promoted to the default scalar type.

\[\text{out}_{i} = \frac{1}{\text{input}_{i}} \]

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4)
>>> a
tensor([-0.4595, -2.1219, -1.4314,  0.7298])
>>> torch.reciprocal(a)
tensor([-2.1763, -0.4713, -0.6986,  1.3702])

torch.remainder(input, other, *, out=None) → Tensor ¶

Computes the element-wise remainder of division.

The dividend and divisor may contain both for integer and floating point numbers. The remainder has the same sign as the divisor other.

Supports broadcasting to a common shape, type promotion, and integer and float inputs.

Note

Complex inputs are not supported. In some cases, it is not mathematically possible to satisfy the definition of a modulo operation with complex numbers. See torch.fmod() for how division by zero is handled.

Parameters

input (Tensor) – the dividend
other (Tensor or Scalar) – the divisor

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> torch.remainder(torch.tensor([-3., -2, -1, 1, 2, 3]), 2)
tensor([ 1.,  0.,  1.,  1.,  0.,  1.])
>>> torch.remainder(torch.tensor([1, 2, 3, 4, 5]), 1.5)
tensor([ 1.0000,  0.5000,  0.0000,  1.0000,  0.5000])

Reduction Ops¶

torch.argmax(input) → LongTensor¶

Returns the indices of the maximum value of all elements in the input tensor.

This is the second value returned by torch.max(). See its documentation for the exact semantics of this method.

Note

If there are multiple minimal values then the indices of the first minimal value are returned.

Parameters: input (Tensor) – the input tensor.

Example:

>>> a = torch.randn(4, 4)
>>> a
tensor([[ 1.3398,  0.2663, -0.2686,  0.2450],
        [-0.7401, -0.8805, -0.3402, -1.1936],
        [ 0.4907, -1.3948, -1.0691, -0.3132],
        [-1.6092,  0.5419, -0.2993,  0.3195]])
>>> torch.argmax(a)
tensor(0)

torch.argmax(input, dim, keepdim=False) → LongTensor¶

Returns the indices of the maximum values of a tensor across a dimension.

This is the second value returned by torch.max(). See its documentation for the exact semantics of this method.

Parameters

input (Tensor) – the input tensor.
dim (int) – the dimension to reduce. If None, the argmax of the flattened input is returned.
keepdim (bool) – whether the output tensor has dim retained or not. Ignored if dim=None.

Example:

>>> a = torch.randn(4, 4)
>>> a
tensor([[ 1.3398,  0.2663, -0.2686,  0.2450],
        [-0.7401, -0.8805, -0.3402, -1.1936],
        [ 0.4907, -1.3948, -1.0691, -0.3132],
        [-1.6092,  0.5419, -0.2993,  0.3195]])
>>> torch.argmax(a, dim=1)
tensor([ 0,  2,  0,  1])

torch.argmin(input, dim=None, keepdim=False) → LongTensor¶

Returns the indices of the minimum value(s) of the flattened tensor or along a dimension

This is the second value returned by torch.min(). See its documentation for the exact semantics of this method.

Note

If there are multiple minimal values then the indices of the first minimal value are returned.

Parameters

input (Tensor) – the input tensor.
dim (int) – the dimension to reduce. If None, the argmin of the flattened input is returned.
keepdim (bool) – whether the output tensor has dim retained or not. Ignored if dim=None.

Example:

>>> a = torch.randn(4, 4)
>>> a
tensor([[ 0.1139,  0.2254, -0.1381,  0.3687],
        [ 1.0100, -1.1975, -0.0102, -0.4732],
        [-0.9240,  0.1207, -0.7506, -1.0213],
        [ 1.7809, -1.2960,  0.9384,  0.1438]])
>>> torch.argmin(a)
tensor(13)
>>> torch.argmin(a, dim=1)
tensor([ 2,  1,  3,  1])
>>> torch.argmin(a, dim=1, keepdim=True)
tensor([[2],
        [1],
        [3],
        [1]])

torch.cumprod(input, dim, *, dtype=None, out=None) → Tensor ¶

Returns the cumulative product of elements of input in the dimension dim.

For example, if input is a vector of size N, the result will also be a vector of size N, with elements.

\[y_i = x_1 \times x_2\times x_3\times \dots \times x_i \]

Parameters

input (Tensor) – the input tensor.
dim (int) – the dimension to do the operation over

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype before the operation is performed. This is useful for preventing data type overflows. Default: None.
out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(10)
>>> a
tensor([ 0.6001,  0.2069, -0.1919,  0.9792,  0.6727,  1.0062,  0.4126,
        -0.2129, -0.4206,  0.1968])
>>> torch.cumprod(a, dim=0)
tensor([ 0.6001,  0.1241, -0.0238, -0.0233, -0.0157, -0.0158, -0.0065,
         0.0014, -0.0006, -0.0001])

>>> a[5] = 0.0
>>> torch.cumprod(a, dim=0)
tensor([ 0.6001,  0.1241, -0.0238, -0.0233, -0.0157, -0.0000, -0.0000,
         0.0000, -0.0000, -0.0000])

torch.cumsum(input, dim, *, dtype=None, out=None) → Tensor ¶

Returns the cumulative sum of elements of input in the dimension dim.

For example, if input is a vector of size N, the result will also be a vector of size N, with elements.

\[y_i = x_1 + x_2 + x_3 + \dots + x_i \]

Parameters

input (Tensor) – the input tensor.
dim (int) – the dimension to do the operation over

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype before the operation is performed. This is useful for preventing data type overflows. Default: None.
out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(10)
>>> a
tensor([-0.8286, -0.4890,  0.5155,  0.8443,  0.1865, -0.1752, -2.0595,
         0.1850, -1.1571, -0.4243])
>>> torch.cumsum(a, dim=0)
tensor([-0.8286, -1.3175, -0.8020,  0.0423,  0.2289,  0.0537, -2.0058,
        -1.8209, -2.9780, -3.4022])

torch.dist(input, other, p=2) → Tensor ¶

Returns the p-norm of (input - other)

The shapes of input and other must be broadcastable.

Parameters

input (Tensor) – the input tensor.
other (Tensor) – the Right-hand-side input tensor
p (float, optional) – the norm to be computed

Example:

>>> x = torch.randn(4)
>>> x
tensor([-1.5393, -0.8675,  0.5916,  1.6321])
>>> y = torch.randn(4)
>>> y
tensor([ 0.0967, -1.0511,  0.6295,  0.8360])
>>> torch.dist(x, y, 3.5)
tensor(1.6727)
>>> torch.dist(x, y, 3)
tensor(1.6973)
>>> torch.dist(x, y, 0)
tensor(inf)
>>> torch.dist(x, y, 1)
tensor(2.6537)

torch.logsumexp(input, dim, keepdim=False, *, out=None)¶

Returns the log of summed exponentials of each row of the input tensor in the given dimension dim. The computation is numerically stabilized.

For summation index \(j\) given by dim and other indices \(i\), the result is

\[\text{logsumexp}(x)_{i} = \log \sum_j \exp(x_{ij}) \]

If keepdim is True, the output tensor is of the same size as input except in the dimension(s) dim where it is of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in the output tensor having 1 (or len(dim)) fewer dimension(s).

Parameters

input (Tensor) – the input tensor.
dim (int or tuple of python:ints) – the dimension or dimensions to reduce.
keepdim (bool) – whether the output tensor has dim retained or not.

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example::

>>> a = torch.randn(3, 3)
>>> torch.logsumexp(a, 1)
tensor([ 0.8442,  1.4322,  0.8711])

torch.mean(input) → Tensor ¶

Returns the mean value of all elements in the input tensor.

Parameters: input (Tensor) – the input tensor.

Example:

>>> a = torch.randn(1, 3)
>>> a
tensor([[ 0.2294, -0.5481,  1.3288]])
>>> torch.mean(a)
tensor(0.3367)

torch.mean(input, dim, keepdim=False, *, out=None) → Tensor ¶

Returns the mean value of each row of the input tensor in the given dimension dim. If dim is a list of dimensions, reduce over all of them.

If keepdim is True, the output tensor is of the same size as input except in the dimension(s) dim where it is of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in the output tensor having 1 (or len(dim)) fewer dimension(s).

Parameters

input (Tensor) – the input tensor.
dim (int or tuple of python:ints) – the dimension or dimensions to reduce.
keepdim (bool) – whether the output tensor has dim retained or not.

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4, 4)
>>> a
tensor([[-0.3841,  0.6320,  0.4254, -0.7384],
        [-0.9644,  1.0131, -0.6549, -1.4279],
        [-0.2951, -1.3350, -0.7694,  0.5600],
        [ 1.0842, -0.9580,  0.3623,  0.2343]])
>>> torch.mean(a, 1)
tensor([-0.0163, -0.5085, -0.4599,  0.1807])
>>> torch.mean(a, 1, True)
tensor([[-0.0163],
        [-0.5085],
        [-0.4599],
        [ 0.1807]])

torch.median(input) → Tensor ¶

Returns the median of the values in input.

Note

The median is not unique for input tensors with an even number of elements. In this case the lower of the two medians is returned. To compute the mean of both medians, use torch.quantile() with q=0.5 instead.

Warning

This function produces deterministic (sub)gradients unlike median(dim=0)

Parameters: input (Tensor) – the input tensor.

Example:

>>> a = torch.randn(1, 3)
>>> a
tensor([[ 1.5219, -1.5212,  0.2202]])
>>> torch.median(a)
tensor(0.2202)

torch.median(input, dim=- 1, keepdim=False, *, out=None)¶

Returns a namedtuple (values, indices) where values contains the median of each row of input in the dimension dim, and indices contains the index of the median values found in the dimension dim.

By default, dim is the last dimension of the input tensor.

If keepdim is True, the output tensors are of the same size as input except in the dimension dim where they are of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in the outputs tensor having 1 fewer dimension than input.

Note

The median is not unique for input tensors with an even number of elements in the dimension dim. In this case the lower of the two medians is returned. To compute the mean of both medians in input, use torch.quantile() with q=0.5 instead.

Warning

indices does not necessarily contain the first occurrence of each median value found, unless it is unique. The exact implementation details are device-specific. Do not expect the same result when run on CPU and GPU in general. For the same reason do not expect the gradients to be deterministic.

Parameters

input (Tensor) – the input tensor.
dim (int) – the dimension to reduce.
keepdim (bool) – whether the output tensor has dim retained or not.

Keyword Arguments

out ((Tensor, Tensor), optional) – The first tensor will be populated with the median values and the second tensor, which must have dtype long, with their indices in the dimension dim of input.

Example:

>>> a = torch.randn(4, 5)
>>> a
tensor([[ 0.2505, -0.3982, -0.9948,  0.3518, -1.3131],
        [ 0.3180, -0.6993,  1.0436,  0.0438,  0.2270],
        [-0.2751,  0.7303,  0.2192,  0.3321,  0.2488],
        [ 1.0778, -1.9510,  0.7048,  0.4742, -0.7125]])
>>> torch.median(a, 1)
torch.return_types.median(values=tensor([-0.3982,  0.2270,  0.2488,  0.4742]), indices=tensor([1, 4, 4, 3]))

torch.mode(input, dim=- 1, keepdim=False, *, out=None)¶

Returns a namedtuple (values, indices) where values is the mode value of each row of the input tensor in the given dimension dim, i.e. a value which appears most often in that row, and indices is the index location of each mode value found.

By default, dim is the last dimension of the input tensor.

If keepdim is True, the output tensors are of the same size as input except in the dimension dim where they are of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in the output tensors having 1 fewer dimension than input.

Note

This function is not defined for torch.cuda.Tensor yet.

Parameters

input (Tensor) – the input tensor.
dim (int) – the dimension to reduce.
keepdim (bool) – whether the output tensor has dim retained or not.

Keyword Arguments

out (tuple, optional) – the result tuple of two output tensors (values, indices)

Example:

>>> a = torch.randint(10, (5,))
>>> a
tensor([6, 5, 1, 0, 2])
>>> b = a + (torch.randn(50, 1) * 5).long()
>>> torch.mode(b, 0)
torch.return_types.mode(values=tensor([6, 5, 1, 0, 2]), indices=tensor([2, 2, 2, 2, 2]))

torch.norm(input, p='fro', dim=None, keepdim=False, out=None, dtype=None)[source]¶

Returns the matrix norm or vector norm of a given tensor.

Warning

torch.norm is deprecated and may be removed in a future PyTorch release. Use torch.linalg.norm() instead, but note that torch.linalg.norm() has a different signature and slightly different behavior that is more consistent with NumPy’s numpy.linalg.norm.

Parameters

input (Tensor) – The input tensor. Its data type must be either a floating point or complex type. For complex inputs, the norm is calculated using the absolute value of each element. If the input is complex and neither dtype nor out is specified, the result’s data type will be the corresponding floating point type (e.g. float if input is complexfloat).
p (int, float, inf, -inf, 'fro', 'nuc', optional) –
the order of norm. Default: 'fro' The following norms can be calculated:

ord

matrix norm

vector norm

’fro’

Frobenius norm

–

‘nuc’

nuclear norm

–

Number

–

sum(abs(x)**ord)**(1./ord)

The vector norm can be calculated across any number of dimensions. The corresponding dimensions of input are flattened into one dimension, and the norm is calculated on the flattened dimension.

Frobenius norm produces the same result as p=2 in all cases except when dim is a list of three or more dims, in which case Frobenius norm throws an error.

Nuclear norm can only be calculated across exactly two dimensions.
dim (int, tuple of python:ints, list of python:ints, optional) – Specifies which dimension or dimensions of input to calculate the norm across. If dim is None, the norm will be calculated across all dimensions of input. If the norm type indicated by p does not support the specified number of dimensions, an error will occur.
keepdim (bool, optional) – whether the output tensors have dim retained or not. Ignored if dim = None and out = None. Default: False
out (Tensor, optional) – the output tensor. Ignored if dim = None and out = None.
dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to :attr:’dtype’ while performing the operation. Default: None.

Note

Even though p='fro' supports any number of dimensions, the true mathematical definition of Frobenius norm only applies to tensors with exactly two dimensions. torch.linalg.norm() with ord='fro' aligns with the mathematical definition, since it can only be applied across exactly two dimensions.

Example:

>>> import torch
>>> a = torch.arange(9, dtype= torch.float) - 4
>>> b = a.reshape((3, 3))
>>> torch.norm(a)
tensor(7.7460)
>>> torch.norm(b)
tensor(7.7460)
>>> torch.norm(a, float('inf'))
tensor(4.)
>>> torch.norm(b, float('inf'))
tensor(4.)
>>> c = torch.tensor([[ 1, 2, 3],[-1, 1, 4]] , dtype= torch.float)
>>> torch.norm(c, dim=0)
tensor([1.4142, 2.2361, 5.0000])
>>> torch.norm(c, dim=1)
tensor([3.7417, 4.2426])
>>> torch.norm(c, p=1, dim=1)
tensor([6., 6.])
>>> d = torch.arange(8, dtype= torch.float).reshape(2,2,2)
>>> torch.norm(d, dim=(1,2))
tensor([ 3.7417, 11.2250])
>>> torch.norm(d[0, :, :]), torch.norm(d[1, :, :])
(tensor(3.7417), tensor(11.2250))

torch.prod(input, *, dtype=None) → Tensor ¶

Returns the product of all elements in the input tensor.

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype before the operation is performed. This is useful for preventing data type overflows. Default: None.

Example:

>>> a = torch.randn(1, 3)
>>> a
tensor([[-0.8020,  0.5428, -1.5854]])
>>> torch.prod(a)
tensor(0.6902)

torch.prod(input, dim, keepdim=False, *, dtype=None) → Tensor ¶

Returns the product of each row of the input tensor in the given dimension dim.

If keepdim is True, the output tensor is of the same size as input except in the dimension dim where it is of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in the output tensor having 1 fewer dimension than input.

Parameters

input (Tensor) – the input tensor.
dim (int) – the dimension to reduce.
keepdim (bool) – whether the output tensor has dim retained or not.

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype before the operation is performed. This is useful for preventing data type overflows. Default: None.

Example:

>>> a = torch.randn(4, 2)
>>> a
tensor([[ 0.5261, -0.3837],
        [ 1.1857, -0.2498],
        [-1.1646,  0.0705],
        [ 1.1131, -1.0629]])
>>> torch.prod(a, 1)
tensor([-0.2018, -0.2962, -0.0821, -1.1831])

torch.std(input, unbiased=True) → Tensor ¶

Returns the standard-deviation of all elements in the input tensor.

If unbiased is False, then the standard-deviation will be calculated via the biased estimator. Otherwise, Bessel’s correction will be used.

Parameters

input (Tensor) – the input tensor.
unbiased (bool) – whether to use the unbiased estimation or not

Example:

>>> a = torch.randn(1, 3)
>>> a
tensor([[-0.8166, -1.3802, -0.3560]])
>>> torch.std(a)
tensor(0.5130)

torch.std(input, dim, unbiased=True, keepdim=False, *, out=None) → Tensor ¶

Returns the standard-deviation of each row of the input tensor in the dimension dim. If dim is a list of dimensions, reduce over all of them.

If keepdim is True, the output tensor is of the same size as input except in the dimension(s) dim where it is of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in the output tensor having 1 (or len(dim)) fewer dimension(s).

If unbiased is False, then the standard-deviation will be calculated via the biased estimator. Otherwise, Bessel’s correction will be used.

Parameters

input (Tensor) – the input tensor.
dim (int or tuple of python:ints) – the dimension or dimensions to reduce.
unbiased (bool) – whether to use the unbiased estimation or not
keepdim (bool) – whether the output tensor has dim retained or not.

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4, 4)
>>> a
tensor([[ 0.2035,  1.2959,  1.8101, -0.4644],
        [ 1.5027, -0.3270,  0.5905,  0.6538],
        [-1.5745,  1.3330, -0.5596, -0.6548],
        [ 0.1264, -0.5080,  1.6420,  0.1992]])
>>> torch.std(a, dim=1)
tensor([ 1.0311,  0.7477,  1.2204,  0.9087])

torch.sum(input, *, dtype=None) → Tensor ¶

Returns the sum of all elements in the input tensor.

Parameters: input (Tensor) – the input tensor.
Keyword Arguments: dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype before the operation is performed. This is useful for preventing data type overflows. Default: None.

Example:

>>> a = torch.randn(1, 3)
>>> a
tensor([[ 0.1133, -0.9567,  0.2958]])
>>> torch.sum(a)
tensor(-0.5475)

torch.sum(input, dim, keepdim=False, *, dtype=None) → Tensor ¶

Returns the sum of each row of the input tensor in the given dimension dim. If dim is a list of dimensions, reduce over all of them.

If keepdim is True, the output tensor is of the same size as input except in the dimension(s) dim where it is of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in the output tensor having 1 (or len(dim)) fewer dimension(s).

Parameters

input (Tensor) – the input tensor.
dim (int or tuple of python:ints) – the dimension or dimensions to reduce.
keepdim (bool) – whether the output tensor has dim retained or not.

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype before the operation is performed. This is useful for preventing data type overflows. Default: None.

Example:

>>> a = torch.randn(4, 4)
>>> a
tensor([[ 0.0569, -0.2475,  0.0737, -0.3429],
        [-0.2993,  0.9138,  0.9337, -1.6864],
        [ 0.1132,  0.7892, -0.1003,  0.5688],
        [ 0.3637, -0.9906, -0.4752, -1.5197]])
>>> torch.sum(a, 1)
tensor([-0.4598, -0.1381,  1.3708, -2.6217])
>>> b = torch.arange(4 * 5 * 6).view(4, 5, 6)
>>> torch.sum(b, (2, 1))
tensor([  435.,  1335.,  2235.,  3135.])

torch.unique(*args, **kwargs)¶

Returns the unique elements of the input tensor.

Note

This function is different from torch.unique_consecutive() in the sense that this function also eliminates non-consecutive duplicate values.

Note

Currently in the CUDA implementation and the CPU implementation when dim is specified, torch.unique always sort the tensor at the beginning regardless of the sort argument. Sorting could be slow, so if your input tensor is already sorted, it is recommended to use torch.unique_consecutive() which avoids the sorting.

Parameters

input (Tensor) – the input tensor
sorted (bool) – Whether to sort the unique elements in ascending order before returning as output.
return_inverse (bool) – Whether to also return the indices for where elements in the original input ended up in the returned unique list.
return_counts (bool) – Whether to also return the counts for each unique element.
dim (int) – the dimension to apply unique. If None, the unique of the flattened input is returned. default: None

Returns

A tensor or a tuple of tensors containing

output (Tensor): the output list of unique scalar elements.

inverse_indices (Tensor): (optional) if return_inverse is True, there will be an additional returned tensor (same shape as input) representing the indices for where elements in the original input map to in the output; otherwise, this function will only return a single tensor.

counts (Tensor): (optional) if return_counts is True, there will be an additional returned tensor (same shape as output or output.size(dim), if dim was specified) representing the number of occurrences for each unique value or tensor.

Return type

(Tensor, Tensor (optional), Tensor (optional))

Example:

>>> output = torch.unique(torch.tensor([1, 3, 2, 3], dtype=torch.long))
>>> output
tensor([ 2,  3,  1])

>>> output, inverse_indices = torch.unique(
...     torch.tensor([1, 3, 2, 3], dtype=torch.long), sorted=True, return_inverse=True)
>>> output
tensor([ 1,  2,  3])
>>> inverse_indices
tensor([ 0,  2,  1,  2])

>>> output, inverse_indices = torch.unique(
...     torch.tensor([[1, 3], [2, 3]], dtype=torch.long), sorted=True, return_inverse=True)
>>> output
tensor([ 1,  2,  3])
>>> inverse_indices
tensor([[ 0,  2],
        [ 1,  2]])

torch.var(input, unbiased=True) → Tensor ¶

Returns the variance of all elements in the input tensor.

If unbiased is False, then the variance will be calculated via the biased estimator. Otherwise, Bessel’s correction will be used.

Parameters

input (Tensor) – the input tensor.
unbiased (bool) – whether to use the unbiased estimation or not

Example:

>>> a = torch.randn(1, 3)
>>> a
tensor([[-0.3425, -1.2636, -0.4864]])
>>> torch.var(a)
tensor(0.2455)

torch.var(input, dim, unbiased=True, keepdim=False, *, out=None) → Tensor ¶

Returns the variance of each row of the input tensor in the given dimension dim.

If keepdim is True, the output tensor is of the same size as input except in the dimension(s) dim where it is of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in the output tensor having 1 (or len(dim)) fewer dimension(s).

If unbiased is False, then the variance will be calculated via the biased estimator. Otherwise, Bessel’s correction will be used.

Parameters

input (Tensor) – the input tensor.
dim (int or tuple of python:ints) – the dimension or dimensions to reduce.
unbiased (bool) – whether to use the unbiased estimation or not
keepdim (bool) – whether the output tensor has dim retained or not.

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4, 4)
>>> a
tensor([[-0.3567,  1.7385, -1.3042,  0.7423],
        [ 1.3436, -0.1015, -0.9834, -0.8438],
        [ 0.6056,  0.1089, -0.3112, -1.4085],
        [-0.7700,  0.6074, -0.1469,  0.7777]])
>>> torch.var(a, 1)
tensor([ 1.7444,  1.1363,  0.7356,  0.5112])

Comparison Ops¶

torch.allclose(input, other, rtol=1e-05, atol=1e-08, equal_nan=False) → bool ¶

This function checks if all input and other satisfy the condition:

\[\lvert \text{input} - \text{other} \rvert \leq \texttt{atol} + \texttt{rtol} \times \lvert \text{other} \rvert \]

elementwise, for all elements of input and other. The behaviour of this function is analogous to numpy.allclose

Parameters

input (Tensor) – first tensor to compare
other (Tensor) – second tensor to compare
atol (float, optional) – absolute tolerance. Default: 1e-08
rtol (float, optional) – relative tolerance. Default: 1e-05
equal_nan (bool, optional) – if True, then two NaN s will be considered equal. Default: False

Example:

>>> torch.allclose(torch.tensor([10000., 1e-07]), torch.tensor([10000.1, 1e-08]))
False
>>> torch.allclose(torch.tensor([10000., 1e-08]), torch.tensor([10000.1, 1e-09]))
True
>>> torch.allclose(torch.tensor([1.0, float('nan')]), torch.tensor([1.0, float('nan')]))
False
>>> torch.allclose(torch.tensor([1.0, float('nan')]), torch.tensor([1.0, float('nan')]), equal_nan=True)
True

torch.argsort(input, dim=- 1, descending=False) → LongTensor¶

Returns the indices that sort a tensor along a given dimension in ascending order by value.

This is the second value returned by torch.sort(). See its documentation for the exact semantics of this method.

Parameters

input (Tensor) – the input tensor.
dim (int, optional) – the dimension to sort along
descending (bool, optional) – controls the sorting order (ascending or descending)

Example:

>>> a = torch.randn(4, 4)
>>> a
tensor([[ 0.0785,  1.5267, -0.8521,  0.4065],
        [ 0.1598,  0.0788, -0.0745, -1.2700],
        [ 1.2208,  1.0722, -0.7064,  1.2564],
        [ 0.0669, -0.2318, -0.8229, -0.9280]])


>>> torch.argsort(a, dim=1)
tensor([[2, 0, 3, 1],
        [3, 2, 1, 0],
        [2, 1, 0, 3],
        [3, 2, 1, 0]])

torch.eq(input, other, *, out=None) → Tensor ¶

Computes element-wise equality

The second argument can be a number or a tensor whose shape is broadcastable with the first argument.

Parameters

input (Tensor) – the tensor to compare
other (Tensor or float) – the tensor or value to compare

Keyword Arguments

out (Tensor, optional) – the output tensor.

Returns

A boolean tensor that is True where input is equal to other and False elsewhere

Example:

>>> torch.eq(torch.tensor([[1, 2], [3, 4]]), torch.tensor([[1, 1], [4, 4]]))
tensor([[ True, False],
        [False, True]])

torch.equal(input, other) → bool ¶

True if two tensors have the same size and elements, False otherwise.

Example:

>>> torch.equal(torch.tensor([1, 2]), torch.tensor([1, 2]))
True

torch.ge(input, other, *, out=None) → Tensor ¶

Computes \(\text{input} \geq \text{other}\) element-wise.

The second argument can be a number or a tensor whose shape is broadcastable with the first argument.

Parameters

input (Tensor) – the tensor to compare
other (Tensor or float) – the tensor or value to compare

Keyword Arguments

out (Tensor, optional) – the output tensor.

Returns

A boolean tensor that is True where input is greater than or equal to other and False elsewhere

Example:

>>> torch.ge(torch.tensor([[1, 2], [3, 4]]), torch.tensor([[1, 1], [4, 4]]))
tensor([[True, True], [False, True]])

torch.gt(input, other, *, out=None) → Tensor ¶

Computes \(\text{input} > \text{other}\) element-wise.

The second argument can be a number or a tensor whose shape is broadcastable with the first argument.

Parameters

input (Tensor) – the tensor to compare
other (Tensor or float) – the tensor or value to compare

Keyword Arguments

out (Tensor, optional) – the output tensor.

Returns

A boolean tensor that is True where input is greater than other and False elsewhere

Example:

>>> torch.gt(torch.tensor([[1, 2], [3, 4]]), torch.tensor([[1, 1], [4, 4]]))
tensor([[False, True], [False, False]])

torch.isfinite(input) → Tensor ¶

Returns a new tensor with boolean elements representing if each element is finite or not.

Real values are finite when they are not NaN, negative infinity, or infinity. Complex values are finite when both their real and imaginary parts are finite.

Args:
input (Tensor): the input tensor.

Returns:
A boolean tensor that is True where input is finite and False elsewhere

Example:
>>> torch.isfinite(torch.tensor([1, float('inf'), 2, float('-inf'), float('nan')]))
tensor([True,  False,  True,  False,  False])

torch.isinf(input) → Tensor ¶

Tests if each element of input is infinite (positive or negative infinity) or not.

Note

Complex values are infinite when their real or imaginary part is infinite.

Args:: {input}
Returns:: A boolean tensor that is True where input is infinite and False elsewhere

Example:

>>> torch.isinf(torch.tensor([1, float('inf'), 2, float('-inf'), float('nan')]))
tensor([False,  True,  False,  True,  False])

torch.isnan(input) → Tensor ¶

Returns a new tensor with boolean elements representing if each element of input is NaN or not. Complex values are considered NaN when either their real and/or imaginary part is NaN.

Parameters: input (Tensor) – the input tensor.
Returns: A boolean tensor that is True where input is NaN and False elsewhere

Example:

>>> torch.isnan(torch.tensor([1, float('nan'), 2]))
tensor([False, True, False])

torch.kthvalue(input, k, dim=None, keepdim=False, *, out=None)¶

Returns a namedtuple (values, indices) where values is the k th smallest element of each row of the input tensor in the given dimension dim. And indices is the index location of each element found.

If dim is not given, the last dimension of the input is chosen.

If keepdim is True, both the values and indices tensors are the same size as input, except in the dimension dim where they are of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in both the values and indices tensors having 1 fewer dimension than the input tensor.

Note

When input is a CUDA tensor and there are multiple valid k th values, this function may nondeterministically return indices for any of them.

Parameters

input (Tensor) – the input tensor.
k (int) – k for the k-th smallest element
dim (int, optional) – the dimension to find the kth value along
keepdim (bool) – whether the output tensor has dim retained or not.

Keyword Arguments

out (tuple, optional) – the output tuple of (Tensor, LongTensor) can be optionally given to be used as output buffers

Example:

>>> x = torch.arange(1., 6.)
>>> x
tensor([ 1.,  2.,  3.,  4.,  5.])
>>> torch.kthvalue(x, 4)
torch.return_types.kthvalue(values=tensor(4.), indices=tensor(3))

>>> x=torch.arange(1.,7.).resize_(2,3)
>>> x
tensor([[ 1.,  2.,  3.],
        [ 4.,  5.,  6.]])
>>> torch.kthvalue(x, 2, 0, True)
torch.return_types.kthvalue(values=tensor([[4., 5., 6.]]), indices=tensor([[1, 1, 1]]))

torch.le(input, other, *, out=None) → Tensor ¶

Computes \(\text{input} \leq \text{other}\) element-wise.

The second argument can be a number or a tensor whose shape is broadcastable with the first argument.

Parameters

input (Tensor) – the tensor to compare
other (Tensor or Scalar) – the tensor or value to compare

Keyword Arguments

out (Tensor, optional) – the output tensor.

Returns

A boolean tensor that is True where input is less than or equal to other and False elsewhere

Example:

>>> torch.le(torch.tensor([[1, 2], [3, 4]]), torch.tensor([[1, 1], [4, 4]]))
tensor([[True, False], [True, True]])

torch.lt(input, other, *, out=None) → Tensor ¶

Computes \(\text{input} < \text{other}\) element-wise.

The second argument can be a number or a tensor whose shape is broadcastable with the first argument.

Parameters

input (Tensor) – the tensor to compare
other (Tensor or float) – the tensor or value to compare

Keyword Arguments

out (Tensor, optional) – the output tensor.

Returns

A boolean tensor that is True where input is less than other and False elsewhere

Example:

>>> torch.lt(torch.tensor([[1, 2], [3, 4]]), torch.tensor([[1, 1], [4, 4]]))
tensor([[False, False], [True, False]])

torch.max(input) → Tensor ¶

Returns the maximum value of all elements in the input tensor.

Warning

This function produces deterministic (sub)gradients unlike max(dim=0)

Parameters: input (Tensor) – the input tensor.

Example:

>>> a = torch.randn(1, 3)
>>> a
tensor([[ 0.6763,  0.7445, -2.2369]])
>>> torch.max(a)
tensor(0.7445)

torch.max(input, dim, keepdim=False, *, out=None)¶

Returns a namedtuple (values, indices) where values is the maximum value of each row of the input tensor in the given dimension dim. And indices is the index location of each maximum value found (argmax).

If keepdim is True, the output tensors are of the same size as input except in the dimension dim where they are of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in the output tensors having 1 fewer dimension than input.

Note

If there are multiple maximal values in a reduced row then the indices of the first maximal value are returned.

Parameters

input (Tensor) – the input tensor.
dim (int) – the dimension to reduce.
keepdim (bool) – whether the output tensor has dim retained or not. Default: False.

Keyword Arguments

out (tuple, optional) – the result tuple of two output tensors (max, max_indices)

Example:

>>> a = torch.randn(4, 4)
>>> a
tensor([[-1.2360, -0.2942, -0.1222,  0.8475],
        [ 1.1949, -1.1127, -2.2379, -0.6702],
        [ 1.5717, -0.9207,  0.1297, -1.8768],
        [-0.6172,  1.0036, -0.6060, -0.2432]])
>>> torch.max(a, 1)
torch.return_types.max(values=tensor([0.8475, 1.1949, 1.5717, 1.0036]), indices=tensor([3, 0, 0, 1]))

torch.max(input, other, *, out=None) → Tensor ¶

See torch.maximum().

torch.min(input) → Tensor ¶

Returns the minimum value of all elements in the input tensor.

Warning

This function produces deterministic (sub)gradients unlike min(dim=0)

Parameters: input (Tensor) – the input tensor.

Example:

>>> a = torch.randn(1, 3)
>>> a
tensor([[ 0.6750,  1.0857,  1.7197]])
>>> torch.min(a)
tensor(0.6750)

torch.min(input, dim, keepdim=False, *, out=None)¶

Returns a namedtuple (values, indices) where values is the minimum value of each row of the input tensor in the given dimension dim. And indices is the index location of each minimum value found (argmin).

If keepdim is True, the output tensors are of the same size as input except in the dimension dim where they are of size 1. Otherwise, dim is squeezed (see torch.squeeze()), resulting in the output tensors having 1 fewer dimension than input.

Note

If there are multiple minimal values in a reduced row then the indices of the first minimal value are returned.

Parameters

input (Tensor) – the input tensor.
dim (int) – the dimension to reduce.
keepdim (bool) – whether the output tensor has dim retained or not.

Keyword Arguments

out (tuple, optional) – the tuple of two output tensors (min, min_indices)

Example:

>>> a = torch.randn(4, 4)
>>> a
tensor([[-0.6248,  1.1334, -1.1899, -0.2803],
        [-1.4644, -0.2635, -0.3651,  0.6134],
        [ 0.2457,  0.0384,  1.0128,  0.7015],
        [-0.1153,  2.9849,  2.1458,  0.5788]])
>>> torch.min(a, 1)
torch.return_types.min(values=tensor([-1.1899, -1.4644,  0.0384, -0.1153]), indices=tensor([2, 0, 1, 0]))

torch.min(input, other, *, out=None) → Tensor ¶

See torch.minimum().

torch.ne(input, other, *, out=None) → Tensor ¶

Computes \(\text{input} \neq \text{other}\) element-wise.

The second argument can be a number or a tensor whose shape is broadcastable with the first argument.

Parameters

input (Tensor) – the tensor to compare
other (Tensor or float) – the tensor or value to compare

Keyword Arguments

out (Tensor, optional) – the output tensor.

Returns

A boolean tensor that is True where input is not equal to other and False elsewhere

Example:

>>> torch.ne(torch.tensor([[1, 2], [3, 4]]), torch.tensor([[1, 1], [4, 4]]))
tensor([[False, True], [True, False]])

torch.sort(input, dim=- 1, descending=False, *, out=None)¶

Sorts the elements of the input tensor along a given dimension in ascending order by value.

If dim is not given, the last dimension of the input is chosen.

If descending is True then the elements are sorted in descending order by value.

A namedtuple of (values, indices) is returned, where the values are the sorted values and indices are the indices of the elements in the original input tensor.

Parameters

input (Tensor) – the input tensor.
dim (int, optional) – the dimension to sort along
descending (bool, optional) – controls the sorting order (ascending or descending)

Keyword Arguments

out (tuple, optional) – the output tuple of (Tensor, LongTensor) that can be optionally given to be used as output buffers

Example:

>>> x = torch.randn(3, 4)
>>> sorted, indices = torch.sort(x)
>>> sorted
tensor([[-0.2162,  0.0608,  0.6719,  2.3332],
        [-0.5793,  0.0061,  0.6058,  0.9497],
        [-0.5071,  0.3343,  0.9553,  1.0960]])
>>> indices
tensor([[ 1,  0,  2,  3],
        [ 3,  1,  0,  2],
        [ 0,  3,  1,  2]])

>>> sorted, indices = torch.sort(x, 0)
>>> sorted
tensor([[-0.5071, -0.2162,  0.6719, -0.5793],
        [ 0.0608,  0.0061,  0.9497,  0.3343],
        [ 0.6058,  0.9553,  1.0960,  2.3332]])
>>> indices
tensor([[ 2,  0,  0,  1],
        [ 0,  1,  1,  2],
        [ 1,  2,  2,  0]])

torch.topk(input, k, dim=None, largest=True, sorted=True, *, out=None)¶

Returns the k largest elements of the given input tensor along a given dimension.

If dim is not given, the last dimension of the input is chosen.

If largest is False then the k smallest elements are returned.

A namedtuple of (values, indices) is returned, where the indices are the indices of the elements in the original input tensor.

The boolean option sorted if True, will make sure that the returned k elements are themselves sorted

Parameters

input (Tensor) – the input tensor.
k (int) – the k in “top-k”
dim (int, optional) – the dimension to sort along
largest (bool, optional) – controls whether to return largest or smallest elements
sorted (bool, optional) – controls whether to return the elements in sorted order

Keyword Arguments

out (tuple, optional) – the output tuple of (Tensor, LongTensor) that can be optionally given to be used as output buffers

Example:

>>> x = torch.arange(1., 6.)
>>> x
tensor([ 1.,  2.,  3.,  4.,  5.])
>>> torch.topk(x, 3)
torch.return_types.topk(values=tensor([5., 4., 3.]), indices=tensor([4, 3, 2]))

Spectral Ops¶

torch.fft()¶

torch.stft(input: torch.Tensor, n_fft: int, hop_length: Optional[int] = None, win_length: Optional[int] = None, window: Optional[torch.Tensor] = None, center: bool = True, pad_mode: str = 'reflect', normalized: bool = False, onesided: Optional[bool] = None, return_complex: Optional[bool] = None) → torch.Tensor [source]¶

Short-time Fourier transform (STFT).

Warning

From version 1.8.0, return_complex must always be given explicitly for real inputs and return_complex=False has been deprecated. Strongly prefer return_complex=True as in a future pytorch release, this function will only return complex tensors.

Note that torch.view_as_real() can be used to recover a real tensor with an extra last dimension for real and imaginary components.

The STFT computes the Fourier transform of short overlapping windows of the input. This giving frequency components of the signal as they change over time. The interface of this function is modeled after the librosa stft function.

Ignoring the optional batch dimension, this method computes the following expression:

\[X[m, \omega] = \sum_{k = 0}^{\text{win\_length-1}}% \text{window}[k]\ \text{input}[m \times \text{hop\_length} + k]\ % \exp\left(- j \frac{2 \pi \cdot \omega k}{\text{win\_length}}\right), \]

where \(m\) is the index of the sliding window, and \(\omega\) is the frequency that \(0 \leq \omega < \text{n\_fft}\). When onesided is the default value True,

input must be either a 1-D time sequence or a 2-D batch of time sequences.
If hop_length is None (default), it is treated as equal to floor(n_fft / 4).
If win_length is None (default), it is treated as equal to n_fft.
window can be a 1-D tensor of size win_length, e.g., from torch.hann_window(). If window is None (default), it is treated as if having \(1\) everywhere in the window. If \(\text{win\_length} < \text{n\_fft}\), window will be padded on both sides to length n_fft before being applied.
If center is True (default), input will be padded on both sides so that the \(t\)-th frame is centered at time \(t \times \text{hop\_length}\). Otherwise, the \(t\)-th frame begins at time \(t \times \text{hop\_length}\).
pad_mode determines the padding method used on input when center is True. See torch.nn.functional.pad() for all available options. Default is "reflect".
If onesided is True (default for real input), only values for \(\omega\) in \(\left[0, 1, 2, \dots, \left\lfloor \frac{\text{n\_fft}}{2} \right\rfloor + 1\right]\) are returned because the real-to-complex Fourier transform satisfies the conjugate symmetry, i.e., \(X[m, \omega] = X[m, \text{n\_fft} - \omega]^*\). Note if the input or window tensors are complex, then onesided output is not possible.
If normalized is True (default is False), the function returns the normalized STFT results, i.e., multiplied by \((\text{frame\_length})^{-0.5}\).
If return_complex is True (default if input is complex), the return is a input.dim() + 1 dimensional complex tensor. If False, the output is a input.dim() + 2 dimensional real tensor where the last dimension represents the real and imaginary components.

Returns either a complex tensor of size \((* \times N \times T)\) if return_complex is true, or a real tensor of size \((* \times N \times T \times 2)\). Where \(*\) is the optional batch size of input, \(N\) is the number of frequencies where STFT is applied and \(T\) is the total number of frames used.

Warning

This function changed signature at version 0.4.1. Calling with the previous signature may cause error or return incorrect result.

Parameters

input (Tensor) – the input tensor
n_fft (int) – size of Fourier transform
hop_length (int, optional) – the distance between neighboring sliding window frames. Default: None (treated as equal to floor(n_fft / 4))
win_length (int, optional) – the size of window frame and STFT filter. Default: None (treated as equal to n_fft)
window (Tensor, optional) – the optional window function. Default: None (treated as window of all \(1\) s)
center (bool, optional) – whether to pad input on both sides so that the \(t\)-th frame is centered at time \(t \times \text{hop\_length}\). Default: True
pad_mode (string, optional) – controls the padding method used when center is True. Default: "reflect"
normalized (bool, optional) – controls whether to return the normalized STFT results Default: False
onesided (bool, optional) – controls whether to return half of results to avoid redundancy for real inputs. Default: True for real input and window, False otherwise.
return_complex (bool, optional) – whether to return a complex tensor, or a real tensor with an extra last dimension for the real and imaginary components.

Returns

A tensor containing the STFT result with shape described above

Return type

Tensor

torch.bartlett_window(window_length, periodic=True, *, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor ¶

Bartlett window function.

\[w[n] = 1 - \left| \frac{2n}{N-1} - 1 \right| = \begin{cases} \frac{2n}{N - 1} & \text{if } 0 \leq n \leq \frac{N - 1}{2} \\ 2 - \frac{2n}{N - 1} & \text{if } \frac{N - 1}{2} < n < N \\ \end{cases}, \]

where \(N\) is the full window size.

The input window_length is a positive integer controlling the returned window size. periodic flag determines whether the returned window trims off the last duplicate value from the symmetric window and is ready to be used as a periodic window with functions like torch.stft(). Therefore, if periodic is true, the \(N\) in above formula is in fact \(\text{window\_length} + 1\). Also, we always have torch.bartlett_window(L, periodic=True) equal to torch.bartlett_window(L + 1, periodic=False)[:-1]).

Note

If window_length \(=1\), the returned window contains a single value 1.

Parameters

window_length (int) – the size of returned window
periodic (bool, optional) – If True, returns a window to be used as periodic function. If False, return a symmetric window.

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()). Only floating point types are supported.
layout (torch.layout, optional) – the desired layout of returned window tensor. Only torch.strided (dense layout) is supported.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Returns

A 1-D tensor of size \((\text{window\_length},)\) containing the window

Return type

Tensor

torch.blackman_window(window_length, periodic=True, *, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor ¶

Blackman window function.

\[w[n] = 0.42 - 0.5 \cos \left( \frac{2 \pi n}{N - 1} \right) + 0.08 \cos \left( \frac{4 \pi n}{N - 1} \right) \]

where \(N\) is the full window size.

The input window_length is a positive integer controlling the returned window size. periodic flag determines whether the returned window trims off the last duplicate value from the symmetric window and is ready to be used as a periodic window with functions like torch.stft(). Therefore, if periodic is true, the \(N\) in above formula is in fact \(\text{window\_length} + 1\). Also, we always have torch.blackman_window(L, periodic=True) equal to torch.blackman_window(L + 1, periodic=False)[:-1]).

Note

If window_length \(=1\), the returned window contains a single value 1.

Parameters

window_length (int) – the size of returned window
periodic (bool, optional) – If True, returns a window to be used as periodic function. If False, return a symmetric window.

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()). Only floating point types are supported.
layout (torch.layout, optional) – the desired layout of returned window tensor. Only torch.strided (dense layout) is supported.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Returns

A 1-D tensor of size \((\text{window\_length},)\) containing the window

Return type

Tensor

torch.hamming_window(window_length, periodic=True, alpha=0.54, beta=0.46, *, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor ¶

Hamming window function.

\[w[n] = \alpha - \beta\ \cos \left( \frac{2 \pi n}{N - 1} \right), \]

where \(N\) is the full window size.

The input window_length is a positive integer controlling the returned window size. periodic flag determines whether the returned window trims off the last duplicate value from the symmetric window and is ready to be used as a periodic window with functions like torch.stft(). Therefore, if periodic is true, the \(N\) in above formula is in fact \(\text{window\_length} + 1\). Also, we always have torch.hamming_window(L, periodic=True) equal to torch.hamming_window(L + 1, periodic=False)[:-1]).

Note

If window_length \(=1\), the returned window contains a single value 1.

Note

This is a generalized version of torch.hann_window().

Parameters

window_length (int) – the size of returned window
periodic (bool, optional) – If True, returns a window to be used as periodic function. If False, return a symmetric window.
alpha (float, optional) – The coefficient \(\alpha\) in the equation above
beta (float, optional) – The coefficient \(\beta\) in the equation above

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()). Only floating point types are supported.
layout (torch.layout, optional) – the desired layout of returned window tensor. Only torch.strided (dense layout) is supported.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Returns

A 1-D tensor of size \((\text{window\_length},)\) containing the window

Return type

Tensor

torch.hann_window(window_length, periodic=True, *, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor ¶

Hann window function.

\[w[n] = \frac{1}{2}\ \left[1 - \cos \left( \frac{2 \pi n}{N - 1} \right)\right] = \sin^2 \left( \frac{\pi n}{N - 1} \right), \]

where \(N\) is the full window size.

The input window_length is a positive integer controlling the returned window size. periodic flag determines whether the returned window trims off the last duplicate value from the symmetric window and is ready to be used as a periodic window with functions like torch.stft(). Therefore, if periodic is true, the \(N\) in above formula is in fact \(\text{window\_length} + 1\). Also, we always have torch.hann_window(L, periodic=True) equal to torch.hann_window(L + 1, periodic=False)[:-1]).

Note

If window_length \(=1\), the returned window contains a single value 1.

Parameters

window_length (int) – the size of returned window
periodic (bool, optional) – If True, returns a window to be used as periodic function. If False, return a symmetric window.

Keyword Arguments

dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()). Only floating point types are supported.
layout (torch.layout, optional) – the desired layout of returned window tensor. Only torch.strided (dense layout) is supported.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Returns

A 1-D tensor of size \((\text{window\_length},)\) containing the window

Return type

Tensor

Other Operations¶

torch.bincount(input, weights=None, minlength=0) → Tensor ¶

Count the frequency of each value in an array of non-negative ints.

The number of bins (size 1) is one larger than the largest value in input unless input is empty, in which case the result is a tensor of size 0. If minlength is specified, the number of bins is at least minlength and if input is empty, then the result is tensor of size minlength filled with zeros. If n is the value at position i, out[n] += weights[i] if weights is specified else out[n] += 1.

Note

This operation may produce nondeterministic gradients when given tensors on a CUDA device. See Reproducibility for more information.

Parameters

input (Tensor) – 1-d int tensor
weights (Tensor) – optional, weight for each value in the input tensor. Should be of same size as input tensor.
minlength (int) – optional, minimum number of bins. Should be non-negative.

Returns

a tensor of shape Size([max(input) + 1]) if input is non-empty, else Size(0)

Return type

output (Tensor)

Example:

>>> input = torch.randint(0, 8, (5,), dtype=torch.int64)
>>> weights = torch.linspace(0, 1, steps=5)
>>> input, weights
(tensor([4, 3, 6, 3, 4]),
 tensor([ 0.0000,  0.2500,  0.5000,  0.7500,  1.0000])

>>> torch.bincount(input)
tensor([0, 0, 0, 2, 2, 0, 1])

>>> input.bincount(weights)
tensor([0.0000, 0.0000, 0.0000, 1.0000, 1.0000, 0.0000, 0.5000])

torch.broadcast_tensors(*tensors) → List of Tensors[source]¶

Broadcasts the given tensors according to Broadcasting semantics.

Parameters: *tensors – any number of tensors of the same type

Warning

More than one element of a broadcasted tensor may refer to a single memory location. As a result, in-place operations (especially ones that are vectorized) may result in incorrect behavior. If you need to write to the tensors, please clone them first.

Example:

>>> x = torch.arange(3).view(1, 3)
>>> y = torch.arange(2).view(2, 1)
>>> a, b = torch.broadcast_tensors(x, y)
>>> a.size()
torch.Size([2, 3])
>>> a
tensor([[0, 1, 2],
        [0, 1, 2]])

torch.cross(input, other, dim=None, *, out=None) → Tensor ¶

Returns the cross product of vectors in dimension dim of input and other.

input and other must have the same size, and the size of their dim dimension should be 3.

If dim is not given, it defaults to the first dimension found with the size 3. Note that this might be unexpected.

Parameters

input (Tensor) – the input tensor.
other (Tensor) – the second input tensor
dim (int, optional) – the dimension to take the cross-product in.

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4, 3)
>>> a
tensor([[-0.3956,  1.1455,  1.6895],
        [-0.5849,  1.3672,  0.3599],
        [-1.1626,  0.7180, -0.0521],
        [-0.1339,  0.9902, -2.0225]])
>>> b = torch.randn(4, 3)
>>> b
tensor([[-0.0257, -1.4725, -1.2251],
        [-1.1479, -0.7005, -1.9757],
        [-1.3904,  0.3726, -1.1836],
        [-0.9688, -0.7153,  0.2159]])
>>> torch.cross(a, b, dim=1)
tensor([[ 1.0844, -0.5281,  0.6120],
        [-2.4490, -1.5687,  1.9792],
        [-0.8304, -1.3037,  0.5650],
        [-1.2329,  1.9883,  1.0551]])
>>> torch.cross(a, b)
tensor([[ 1.0844, -0.5281,  0.6120],
        [-2.4490, -1.5687,  1.9792],
        [-0.8304, -1.3037,  0.5650],
        [-1.2329,  1.9883,  1.0551]])

torch.diag(input, diagonal=0, *, out=None) → Tensor ¶

If input is a vector (1-D tensor), then returns a 2-D square tensor with the elements of input as the diagonal.
If input is a matrix (2-D tensor), then returns a 1-D tensor with the diagonal elements of input.

The argument diagonal controls which diagonal to consider:

If diagonal = 0, it is the main diagonal.
If diagonal > 0, it is above the main diagonal.
If diagonal < 0, it is below the main diagonal.

Parameters

input (Tensor) – the input tensor.
diagonal (int, optional) – the diagonal to consider

Keyword Arguments

out (Tensor, optional) – the output tensor.

BLAS and LAPACK Operations¶

torch.addbmm(input, batch1, batch2, *, beta=1, alpha=1, out=None) → Tensor ¶

Performs a batch matrix-matrix product of matrices stored in batch1 and batch2, with a reduced add step (all matrix multiplications get accumulated along the first dimension). input is added to the final result.

batch1 and batch2 must be 3-D tensors each containing the same number of matrices.

If batch1 is a \((b \times n \times m)\) tensor, batch2 is a \((b \times m \times p)\) tensor, input must be broadcastable with a \((n \times p)\) tensor and out will be a \((n \times p)\) tensor.

\[out = \beta\ \text{input} + \alpha\ (\sum_{i=0}^{b-1} \text{batch1}_i \mathbin{@} \text{batch2}_i) \]

If beta is 0, then input will be ignored, and nan and inf in it will not be propagated.

For inputs of type FloatTensor or DoubleTensor, arguments beta and alpha must be real numbers, otherwise they should be integers.

This operator supports TensorFloat32.

Parameters

batch1 (Tensor) – the first batch of matrices to be multiplied
batch2 (Tensor) – the second batch of matrices to be multiplied

Keyword Arguments

beta (Number, optional) – multiplier for input (\(\beta\))
input (Tensor) – matrix to be added
alpha (Number, optional) – multiplier for batch1 @ batch2 (\(\alpha\))
out (Tensor, optional) – the output tensor.

Example:

>>> M = torch.randn(3, 5)
>>> batch1 = torch.randn(10, 3, 4)
>>> batch2 = torch.randn(10, 4, 5)
>>> torch.addbmm(M, batch1, batch2)
tensor([[  6.6311,   0.0503,   6.9768, -12.0362,  -2.1653],
        [ -4.8185,  -1.4255,  -6.6760,   8.9453,   2.5743],
        [ -3.8202,   4.3691,   1.0943,  -1.1109,   5.4730]])

torch.addmm(input, mat1, mat2, *, beta=1, alpha=1, out=None) → Tensor ¶

Performs a matrix multiplication of the matrices mat1 and mat2. The matrix input is added to the final result.

If mat1 is a \((n \times m)\) tensor, mat2 is a \((m \times p)\) tensor, then input must be broadcastable with a \((n \times p)\) tensor and out will be a \((n \times p)\) tensor.

alpha and beta are scaling factors on matrix-vector product between mat1 and mat2 and the added matrix input respectively.

\[\text{out} = \beta\ \text{input} + \alpha\ (\text{mat1}_i \mathbin{@} \text{mat2}_i) \]

If beta is 0, then input will be ignored, and nan and inf in it will not be propagated.

For inputs of type FloatTensor or DoubleTensor, arguments beta and alpha must be real numbers, otherwise they should be integers.

This operator supports TensorFloat32.

Parameters

input (Tensor) – matrix to be added
mat1 (Tensor) – the first matrix to be matrix multiplied
mat2 (Tensor) – the second matrix to be matrix multiplied

Keyword Arguments

beta (Number, optional) – multiplier for input (\(\beta\))
alpha (Number, optional) – multiplier for \(mat1 @ mat2\) (\(\alpha\))
out (Tensor, optional) – the output tensor.

Example:

>>> M = torch.randn(2, 3)
>>> mat1 = torch.randn(2, 3)
>>> mat2 = torch.randn(3, 3)
>>> torch.addmm(M, mat1, mat2)
tensor([[-4.8716,  1.4671, -1.3746],
        [ 0.7573, -3.9555, -2.8681]])

torch.addmv(input, mat, vec, *, beta=1, alpha=1, out=None) → Tensor ¶

Performs a matrix-vector product of the matrix mat and the vector vec. The vector input is added to the final result.

If mat is a \((n \times m)\) tensor, vec is a 1-D tensor of size m, then input must be broadcastable with a 1-D tensor of size n and out will be 1-D tensor of size n.

alpha and beta are scaling factors on matrix-vector product between mat and vec and the added tensor input respectively.

\[\text{out} = \beta\ \text{input} + \alpha\ (\text{mat} \mathbin{@} \text{vec}) \]

If beta is 0, then input will be ignored, and nan and inf in it will not be propagated.

For inputs of type FloatTensor or DoubleTensor, arguments beta and alpha must be real numbers, otherwise they should be integers

Parameters

input (Tensor) – vector to be added
mat (Tensor) – matrix to be matrix multiplied
vec (Tensor) – vector to be matrix multiplied

Keyword Arguments

beta (Number, optional) – multiplier for input (\(\beta\))
alpha (Number, optional) – multiplier for \(mat @ vec\) (\(\alpha\))
out (Tensor, optional) – the output tensor.

Example:

>>> M = torch.randn(2)
>>> mat = torch.randn(2, 3)
>>> vec = torch.randn(3)
>>> torch.addmv(M, mat, vec)
tensor([-0.3768, -5.5565])

torch.addr(input, vec1, vec2, *, beta=1, alpha=1, out=None) → Tensor ¶

Performs the outer-product of vectors vec1 and vec2 and adds it to the matrix input.

Optional values beta and alpha are scaling factors on the outer product between vec1 and vec2 and the added matrix input respectively.

\[\text{out} = \beta\ \text{input} + \alpha\ (\text{vec1} \otimes \text{vec2}) \]

If beta is 0, then input will be ignored, and nan and inf in it will not be propagated.

If vec1 is a vector of size n and vec2 is a vector of size m, then input must be broadcastable with a matrix of size \((n \times m)\) and out will be a matrix of size \((n \times m)\).

Parameters

input (Tensor) – matrix to be added
vec1 (Tensor) – the first vector of the outer product
vec2 (Tensor) – the second vector of the outer product

Keyword Arguments

beta (Number, optional) – multiplier for input (\(\beta\))
alpha (Number, optional) – multiplier for \(\text{vec1} \otimes \text{vec2}\) (\(\alpha\))
out (Tensor, optional) – the output tensor.

Example:

>>> vec1 = torch.arange(1., 4.)
>>> vec2 = torch.arange(1., 3.)
>>> M = torch.zeros(3, 2)
>>> torch.addr(M, vec1, vec2)
tensor([[ 1.,  2.],
        [ 2.,  4.],
        [ 3.,  6.]])

torch.baddbmm(input, batch1, batch2, *, beta=1, alpha=1, out=None) → Tensor ¶

Performs a batch matrix-matrix product of matrices in batch1 and batch2. input is added to the final result.

batch1 and batch2 must be 3-D tensors each containing the same number of matrices.

If batch1 is a \((b \times n \times m)\) tensor, batch2 is a \((b \times m \times p)\) tensor, then input must be broadcastable with a \((b \times n \times p)\) tensor and out will be a \((b \times n \times p)\) tensor. Both alpha and beta mean the same as the scaling factors used in torch.addbmm().

\[\text{out}_i = \beta\ \text{input}_i + \alpha\ (\text{batch1}_i \mathbin{@} \text{batch2}_i) \]

If beta is 0, then input will be ignored, and nan and inf in it will not be propagated.

For inputs of type FloatTensor or DoubleTensor, arguments beta and alpha must be real numbers, otherwise they should be integers.

This operator supports TensorFloat32.

Parameters

input (Tensor) – the tensor to be added
batch1 (Tensor) – the first batch of matrices to be multiplied
batch2 (Tensor) – the second batch of matrices to be multiplied

Keyword Arguments

beta (Number, optional) – multiplier for input (\(\beta\))
alpha (Number, optional) – multiplier for \(\text{batch1} \mathbin{@} \text{batch2}\) (\(\alpha\))
out (Tensor, optional) – the output tensor.

Example:

>>> M = torch.randn(10, 3, 5)
>>> batch1 = torch.randn(10, 3, 4)
>>> batch2 = torch.randn(10, 4, 5)
>>> torch.baddbmm(M, batch1, batch2).size()
torch.Size([10, 3, 5])

torch.bmm(input, mat2, *, deterministic=False, out=None) → Tensor ¶

Performs a batch matrix-matrix product of matrices stored in input and mat2.

input and mat2 must be 3-D tensors each containing the same number of matrices.

If input is a \((b \times n \times m)\) tensor, mat2 is a \((b \times m \times p)\) tensor, out will be a \((b \times n \times p)\) tensor.

\[\text{out}_i = \text{input}_i \mathbin{@} \text{mat2}_i \]

This operator supports TensorFloat32.

Note

This function does not broadcast. For broadcasting matrix products, see torch.matmul().

Parameters

input (Tensor) – the first batch of matrices to be multiplied
mat2 (Tensor) – the second batch of matrices to be multiplied

Keyword Arguments

deterministic (bool, optional) – flag to choose between a faster non-deterministic calculation, or a slower deterministic calculation. This argument is only available for sparse-dense CUDA bmm. Default: False
out (Tensor, optional) – the output tensor.

Example:

>>> input = torch.randn(10, 3, 4)
>>> mat2 = torch.randn(10, 4, 5)
>>> res = torch.bmm(input, mat2)
>>> res.size()
torch.Size([10, 3, 5])

torch.chain_matmul(*matrices)[source]¶

Returns the matrix product of the \(N\) 2-D tensors. This product is efficiently computed using the matrix chain order algorithm which selects the order in which incurs the lowest cost in terms of arithmetic operations ([CLRS]). Note that since this is a function to compute the product, \(N\) needs to be greater than or equal to 2; if equal to 2 then a trivial matrix-matrix product is returned. If \(N\) is 1, then this is a no-op - the original matrix is returned as is.

Parameters: matrices (Tensors...) – a sequence of 2 or more 2-D tensors whose product is to be determined.
Returns: if the \(i^{th}\) tensor was of dimensions \(p_{i} \times p_{i + 1}\), then the product would be of dimensions \(p_{1} \times p_{N + 1}\).
Return type: Tensor

Example:

>>> a = torch.randn(3, 4)
>>> b = torch.randn(4, 5)
>>> c = torch.randn(5, 6)
>>> d = torch.randn(6, 7)
>>> torch.chain_matmul(a, b, c, d)
tensor([[ -2.3375,  -3.9790,  -4.1119,  -6.6577,   9.5609, -11.5095,  -3.2614],
        [ 21.4038,   3.3378,  -8.4982,  -5.2457, -10.2561,  -2.4684,   2.7163],
        [ -0.9647,  -5.8917,  -2.3213,  -5.2284,  12.8615, -12.2816,  -2.5095]])

torch.cholesky(input, upper=False, *, out=None) → Tensor ¶

Computes the Cholesky decomposition of a symmetric positive-definite matrix \(A\) or for batches of symmetric positive-definite matrices.

If upper is True, the returned matrix U is upper-triangular, and the decomposition has the form:

\[A = U^TU\]

If upper is False, the returned matrix L is lower-triangular, and the decomposition has the form:

\[A = LL^T\]

If upper is True, and \(A\) is a batch of symmetric positive-definite matrices, then the returned tensor will be composed of upper-triangular Cholesky factors of each of the individual matrices. Similarly, when upper is False, the returned tensor will be composed of lower-triangular Cholesky factors of each of the individual matrices.

Note

torch.linalg.cholesky() should be used over torch.cholesky when possible. Note however that torch.linalg.cholesky() does not yet support the upper parameter and instead always returns the lower triangular matrix.

Parameters

input (Tensor) – the input tensor \(A\) of size \((*, n, n)\) where * is zero or more batch dimensions consisting of symmetric positive-definite matrices.
upper (bool, optional) – flag that indicates whether to return a upper or lower triangular matrix. Default: False

Keyword Arguments

out (Tensor, optional) – the output matrix

Example:

>>> a = torch.randn(3, 3)
>>> a = torch.mm(a, a.t()) # make symmetric positive-definite
>>> l = torch.cholesky(a)
>>> a
tensor([[ 2.4112, -0.7486,  1.4551],
        [-0.7486,  1.3544,  0.1294],
        [ 1.4551,  0.1294,  1.6724]])
>>> l
tensor([[ 1.5528,  0.0000,  0.0000],
        [-0.4821,  1.0592,  0.0000],
        [ 0.9371,  0.5487,  0.7023]])
>>> torch.mm(l, l.t())
tensor([[ 2.4112, -0.7486,  1.4551],
        [-0.7486,  1.3544,  0.1294],
        [ 1.4551,  0.1294,  1.6724]])
>>> a = torch.randn(3, 2, 2)
>>> a = torch.matmul(a, a.transpose(-1, -2)) + 1e-03 # make symmetric positive-definite
>>> l = torch.cholesky(a)
>>> z = torch.matmul(l, l.transpose(-1, -2))
>>> torch.max(torch.abs(z - a)) # Max non-zero
tensor(2.3842e-07)

torch.dot(input, other, *, out=None) → Tensor ¶

Computes the dot product of two 1D tensors.

Note

Unlike NumPy’s dot, torch.dot intentionally only supports computing the dot product of two 1D tensors with the same number of elements.

Parameters

input (Tensor) – first tensor in the dot product, must be 1D.
other (Tensor) – second tensor in the dot product, must be 1D.

Keyword Arguments

{out} –

Example:

>>> torch.dot(torch.tensor([2, 3]), torch.tensor([2, 1]))
tensor(7)

torch.eig(input, eigenvectors=False, *, out=None)¶

Computes the eigenvalues and eigenvectors of a real square matrix.

Note

Since eigenvalues and eigenvectors might be complex, backward pass is supported only if eigenvalues and eigenvectors are all real valued.

When input is on CUDA, torch.eig() causes host-device synchronization.

Parameters

input (Tensor) – the square matrix of shape \((n \times n)\) for which the eigenvalues and eigenvectors will be computed
eigenvectors (bool) – True to compute both eigenvalues and eigenvectors; otherwise, only eigenvalues will be computed

Keyword Arguments

out (tuple, optional) – the output tensors

Returns

A namedtuple (eigenvalues, eigenvectors) containing

eigenvalues (Tensor): Shape \((n \times 2)\). Each row is an eigenvalue of input, where the first element is the real part and the second element is the imaginary part. The eigenvalues are not necessarily ordered.

eigenvectors (Tensor): If eigenvectors=False, it’s an empty tensor. Otherwise, this tensor of shape \((n \times n)\) can be used to compute normalized (unit length) eigenvectors of corresponding eigenvalues as follows. If the corresponding eigenvalues[j] is a real number, column eigenvectors[:, j] is the eigenvector corresponding to eigenvalues[j]. If the corresponding eigenvalues[j] and eigenvalues[j + 1] form a complex conjugate pair, then the true eigenvectors can be computed as \(\text{true eigenvector}[j] = eigenvectors[:, j] + i \times eigenvectors[:, j + 1]\), \(\text{true eigenvector}[j + 1] = eigenvectors[:, j] - i \times eigenvectors[:, j + 1]\).

Return type

(Tensor, Tensor)

Example:

Trivial example with a diagonal matrix. By default, only eigenvalues are computed:

>>> a = torch.diag(torch.tensor([1, 2, 3], dtype=torch.double))
>>> e, v = torch.eig(a)
>>> e
tensor([[1., 0.],
        [2., 0.],
        [3., 0.]], dtype=torch.float64)
>>> v
tensor([], dtype=torch.float64)

Compute also the eigenvectors:

>>> e, v = torch.eig(a, eigenvectors=True)
>>> e
tensor([[1., 0.],
        [2., 0.],
        [3., 0.]], dtype=torch.float64)
>>> v
tensor([[1., 0., 0.],
        [0., 1., 0.],
        [0., 0., 1.]], dtype=torch.float64)

torch.geqrf(input, *, out=None)¶

This is a low-level function for calling LAPACK directly. This function returns a namedtuple (a, tau) as defined in LAPACK documentation for geqrf .

You’ll generally want to use torch.qr() instead.

Computes a QR decomposition of input, but without constructing \(Q\) and \(R\) as explicit separate matrices.

Rather, this directly calls the underlying LAPACK function ?geqrf which produces a sequence of ‘elementary reflectors’.

See LAPACK documentation for geqrf for further details.

Parameters: input (Tensor) – the input matrix
Keyword Arguments: out (tuple, optional) – the output tuple of (Tensor, Tensor)

torch.ger(input, vec2, *, out=None) → Tensor ¶: Alias of torch.outer().

Warning

This function is deprecated and will be removed in a future PyTorch release. Use torch.outer() instead.

torch.inverse(input, *, out=None) → Tensor ¶

Takes the inverse of the square matrix input. input can be batches of 2D square tensors, in which case this function would return a tensor composed of individual inverses.

Supports real and complex input.

Note

torch.inverse() is deprecated. Please use torch.linalg.inv() instead.

Note

Irrespective of the original strides, the returned tensors will be transposed, i.e. with strides like input.contiguous().transpose(-2, -1).stride()

Parameters: input (Tensor) – the input tensor of size \((*, n, n)\) where * is zero or more batch dimensions
Keyword Arguments: out (Tensor, optional) – the output tensor.

Examples:

>>> x = torch.rand(4, 4)
>>> y = torch.inverse(x)
>>> z = torch.mm(x, y)
>>> z
tensor([[ 1.0000, -0.0000, -0.0000,  0.0000],
        [ 0.0000,  1.0000,  0.0000,  0.0000],
        [ 0.0000,  0.0000,  1.0000,  0.0000],
        [ 0.0000, -0.0000, -0.0000,  1.0000]])
>>> torch.max(torch.abs(z - torch.eye(4))) # Max non-zero
tensor(1.1921e-07)

>>> # Batched inverse example
>>> x = torch.randn(2, 3, 4, 4)
>>> y = torch.inverse(x)
>>> z = torch.matmul(x, y)
>>> torch.max(torch.abs(z - torch.eye(4).expand_as(x))) # Max non-zero
tensor(1.9073e-06)

>>> x = torch.rand(4, 4, dtype=torch.cdouble)
>>> y = torch.inverse(x)
>>> z = torch.mm(x, y)
>>> z
tensor([[ 1.0000e+00+0.0000e+00j, -1.3878e-16+3.4694e-16j,
        5.5511e-17-1.1102e-16j,  0.0000e+00-1.6653e-16j],
        [ 5.5511e-16-1.6653e-16j,  1.0000e+00+6.9389e-17j,
        2.2204e-16-1.1102e-16j, -2.2204e-16+1.1102e-16j],
        [ 3.8858e-16-1.2490e-16j,  2.7756e-17+3.4694e-17j,
        1.0000e+00+0.0000e+00j, -4.4409e-16+5.5511e-17j],
        [ 4.4409e-16+5.5511e-16j, -3.8858e-16+1.8041e-16j,
        2.2204e-16+0.0000e+00j,  1.0000e+00-3.4694e-16j]],
    dtype=torch.complex128)
>>> torch.max(torch.abs(z - torch.eye(4, dtype=torch.cdouble))) # Max non-zero
tensor(7.5107e-16, dtype=torch.float64)

torch.det(input) → Tensor ¶

Calculates determinant of a square matrix or batches of square matrices.

Note

torch.det() is deprecated. Please use torch.linalg.det() instead.

Note

Backward through \(det\) internally uses SVD results when input is not invertible. In this case, double backward through \(det\) will be unstable when input doesn’t have distinct singular values. See \(~torch.svd\) for details.

Parameters: input (Tensor) – the input tensor of size (*, n, n) where * is zero or more batch dimensions.

Example:

>>> A = torch.randn(3, 3)
>>> torch.det(A)
tensor(3.7641)

>>> A = torch.randn(3, 2, 2)
>>> A
tensor([[[ 0.9254, -0.6213],
         [-0.5787,  1.6843]],

        [[ 0.3242, -0.9665],
         [ 0.4539, -0.0887]],

        [[ 1.1336, -0.4025],
         [-0.7089,  0.9032]]])
>>> A.det()
tensor([1.1990, 0.4099, 0.7386])

torch.logdet(input) → Tensor ¶

Calculates log determinant of a square matrix or batches of square matrices.

Note

Result is -inf if input has zero log determinant, and is nan if input has negative determinant.

Note

Backward through logdet() internally uses SVD results when input is not invertible. In this case, double backward through logdet() will be unstable in when input doesn’t have distinct singular values. See svd() for details.

Parameters: input (Tensor) – the input tensor of size (*, n, n) where * is zero or more batch dimensions.

Example:

>>> A = torch.randn(3, 3)
>>> torch.det(A)
tensor(0.2611)
>>> torch.logdet(A)
tensor(-1.3430)
>>> A
tensor([[[ 0.9254, -0.6213],
         [-0.5787,  1.6843]],

        [[ 0.3242, -0.9665],
         [ 0.4539, -0.0887]],

        [[ 1.1336, -0.4025],
         [-0.7089,  0.9032]]])
>>> A.det()
tensor([1.1990, 0.4099, 0.7386])
>>> A.det().log()
tensor([ 0.1815, -0.8917, -0.3031])

torch.slogdet(input)¶

Calculates the sign and log absolute value of the determinant(s) of a square matrix or batches of square matrices.

Note

torch.slogdet() is deprecated. Please use torch.linalg.slogdet() instead.

Note

If input has zero determinant, this returns (0, -inf).

Note

Backward through slogdet() internally uses SVD results when input is not invertible. In this case, double backward through slogdet() will be unstable in when input doesn’t have distinct singular values. See svd() for details.

Parameters: input (Tensor) – the input tensor of size (*, n, n) where * is zero or more batch dimensions.
Returns: A namedtuple (sign, logabsdet) containing the sign of the determinant, and the log value of the absolute determinant.

Example:

>>> A = torch.randn(3, 3)
>>> A
tensor([[ 0.0032, -0.2239, -1.1219],
        [-0.6690,  0.1161,  0.4053],
        [-1.6218, -0.9273, -0.0082]])
>>> torch.det(A)
tensor(-0.7576)
>>> torch.logdet(A)
tensor(nan)
>>> torch.slogdet(A)
torch.return_types.slogdet(sign=tensor(-1.), logabsdet=tensor(-0.2776))

torch.matmul(input, other, *, out=None) → Tensor ¶

Matrix product of two tensors.

The behavior depends on the dimensionality of the tensors as follows:

If both tensors are 1-dimensional, the dot product (scalar) is returned.
If both arguments are 2-dimensional, the matrix-matrix product is returned.
If the first argument is 1-dimensional and the second argument is 2-dimensional, a 1 is prepended to its dimension for the purpose of the matrix multiply. After the matrix multiply, the prepended dimension is removed.
If the first argument is 2-dimensional and the second argument is 1-dimensional, the matrix-vector product is returned.
If both arguments are at least 1-dimensional and at least one argument is N-dimensional (where N > 2), then a batched matrix multiply is returned. If the first argument is 1-dimensional, a 1 is prepended to its dimension for the purpose of the batched matrix multiply and removed after. If the second argument is 1-dimensional, a 1 is appended to its dimension for the purpose of the batched matrix multiple and removed after. The non-matrix (i.e. batch) dimensions are broadcasted (and thus must be broadcastable). For example, if input is a \((j \times 1 \times n \times n)\) tensor and other is a \((k \times n \times n)\) tensor, out will be a \((j \times k \times n \times n)\) tensor.

Note that the broadcasting logic only looks at the batch dimensions when determining if the inputs are broadcastable, and not the matrix dimensions. For example, if input is a \((j \times 1 \times n \times m)\) tensor and other is a \((k \times m \times p)\) tensor, these inputs are valid for broadcasting even though the final two dimensions (i.e. the matrix dimensions) are different. out will be a \((j \times k \times n \times p)\) tensor.

This operator supports TensorFloat32.

Note

The 1-dimensional dot product version of this function does not support an out parameter.

Parameters

input (Tensor) – the first tensor to be multiplied
other (Tensor) – the second tensor to be multiplied

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> # vector x vector
>>> tensor1 = torch.randn(3)
>>> tensor2 = torch.randn(3)
>>> torch.matmul(tensor1, tensor2).size()
torch.Size([])
>>> # matrix x vector
>>> tensor1 = torch.randn(3, 4)
>>> tensor2 = torch.randn(4)
>>> torch.matmul(tensor1, tensor2).size()
torch.Size([3])
>>> # batched matrix x broadcasted vector
>>> tensor1 = torch.randn(10, 3, 4)
>>> tensor2 = torch.randn(4)
>>> torch.matmul(tensor1, tensor2).size()
torch.Size([10, 3])
>>> # batched matrix x batched matrix
>>> tensor1 = torch.randn(10, 3, 4)
>>> tensor2 = torch.randn(10, 4, 5)
>>> torch.matmul(tensor1, tensor2).size()
torch.Size([10, 3, 5])
>>> # batched matrix x broadcasted matrix
>>> tensor1 = torch.randn(10, 3, 4)
>>> tensor2 = torch.randn(4, 5)
>>> torch.matmul(tensor1, tensor2).size()
torch.Size([10, 3, 5])

torch.matrix_power(input, n) → Tensor ¶

Returns the matrix raised to the power n for square matrices. For batch of matrices, each individual matrix is raised to the power n.

If n is negative, then the inverse of the matrix (if invertible) is raised to the power n. For a batch of matrices, the batched inverse (if invertible) is raised to the power n. If n is 0, then an identity matrix is returned.

Parameters

input (Tensor) – the input tensor.
n (int) – the power to raise the matrix to

Example:

>>> a = torch.randn(2, 2, 2)
>>> a
tensor([[[-1.9975, -1.9610],
         [ 0.9592, -2.3364]],

        [[-1.2534, -1.3429],
         [ 0.4153, -1.4664]]])
>>> torch.matrix_power(a, 3)
tensor([[[  3.9392, -23.9916],
         [ 11.7357,  -0.2070]],

        [[  0.2468,  -6.7168],
         [  2.0774,  -0.8187]]])

torch.matrix_rank(input, tol=None, symmetric=False, *, out=None) → Tensor ¶

Returns the numerical rank of a 2-D tensor. The method to compute the matrix rank is done using SVD by default. If symmetric is True, then input is assumed to be symmetric, and the computation of the rank is done by obtaining the eigenvalues.

tol is the threshold below which the singular values (or the eigenvalues when symmetric is True) are considered to be 0. If tol is not specified, tol is set to S.max() * max(S.size()) * eps where S is the singular values (or the eigenvalues when symmetric is True), and eps is the epsilon value for the datatype of input.

Note

torch.matrix_rank() is deprecated. Please use torch.linalg.matrix_rank() instead. The parameter symmetric was renamed in torch.linalg.matrix_rank() to hermitian.

Parameters

input (Tensor) – the input 2-D tensor
tol (float, optional) – the tolerance value. Default: None
symmetric (bool, optional) – indicates whether input is symmetric. Default: False

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.eye(10)
>>> torch.matrix_rank(a)
tensor(10)
>>> b = torch.eye(10)
>>> b[0, 0] = 0
>>> torch.matrix_rank(b)
tensor(9)

torch.mm(input, mat2, *, out=None) → Tensor ¶

Performs a matrix multiplication of the matrices input and mat2.

If input is a \((n \times m)\) tensor, mat2 is a \((m \times p)\) tensor, out will be a \((n \times p)\) tensor.

Note

This function does not broadcast. For broadcasting matrix products, see torch.matmul().

Supports strided and sparse 2-D tensors as inputs, autograd with respect to strided inputs.

This operator supports TensorFloat32.

Parameters

input (Tensor) – the first matrix to be matrix multiplied
mat2 (Tensor) – the second matrix to be matrix multiplied

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> mat1 = torch.randn(2, 3)
>>> mat2 = torch.randn(3, 3)
>>> torch.mm(mat1, mat2)
tensor([[ 0.4851,  0.5037, -0.3633],
        [-0.0760, -3.6705,  2.4784]])

torch.mv(input, vec, *, out=None) → Tensor ¶

Performs a matrix-vector product of the matrix input and the vector vec.

If input is a \((n \times m)\) tensor, vec is a 1-D tensor of size \(m\), out will be 1-D of size \(n\).

Note

This function does not broadcast.

Parameters

input (Tensor) – matrix to be multiplied
vec (Tensor) – vector to be multiplied

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> mat = torch.randn(2, 3)
>>> vec = torch.randn(3)
>>> torch.mv(mat, vec)
tensor([ 1.0404, -0.6361])

torch.orgqr(input, input2) → Tensor ¶

Computes the orthogonal matrix Q of a QR factorization, from the (input, input2) tuple returned by torch.geqrf().

This directly calls the underlying LAPACK function ?orgqr. See LAPACK documentation for orgqr for further details.

Parameters

input (Tensor) – the a from torch.geqrf().
input2 (Tensor) – the tau from torch.geqrf().

torch.ormqr(input, input2, input3, left=True, transpose=False) → Tensor ¶

Multiplies mat (given by input3) by the orthogonal Q matrix of the QR factorization formed by torch.geqrf() that is represented by (a, tau) (given by (input, input2)).

This directly calls the underlying LAPACK function ?ormqr. See LAPACK documentation for ormqr for further details.

Parameters

input (Tensor) – the a from torch.geqrf().
input2 (Tensor) – the tau from torch.geqrf().
input3 (Tensor) – the matrix to be multiplied.

torch.pinverse(input, rcond=1e-15) → Tensor ¶

Calculates the pseudo-inverse (also known as the Moore-Penrose inverse) of a 2D tensor. Please look at Moore-Penrose inverse for more details

Note

torch.pinverse() is deprecated. Please use torch.linalg.pinv() instead which includes new parameters hermitian and out.

Note

This method is implemented using the Singular Value Decomposition.

Note

The pseudo-inverse is not necessarily a continuous function in the elements of the matrix [1]. Therefore, derivatives are not always existent, and exist for a constant rank only [2]. However, this method is backprop-able due to the implementation by using SVD results, and could be unstable. Double-backward will also be unstable due to the usage of SVD internally. See svd() for more details.

Note

Supports real and complex inputs. Batched version for complex inputs is only supported on the CPU.

Parameters

input (Tensor) – The input tensor of size \((*, m, n)\) where \(*\) is zero or more batch dimensions.
rcond (float, optional) – A floating point value to determine the cutoff for small singular values. Default: 1e-15.

Returns

The pseudo-inverse of input of dimensions \((*, n, m)\)

Example:

>>> input = torch.randn(3, 5)
>>> input
tensor([[ 0.5495,  0.0979, -1.4092, -0.1128,  0.4132],
        [-1.1143, -0.3662,  0.3042,  1.6374, -0.9294],
        [-0.3269, -0.5745, -0.0382, -0.5922, -0.6759]])
>>> torch.pinverse(input)
tensor([[ 0.0600, -0.1933, -0.2090],
        [-0.0903, -0.0817, -0.4752],
        [-0.7124, -0.1631, -0.2272],
        [ 0.1356,  0.3933, -0.5023],
        [-0.0308, -0.1725, -0.5216]])
>>> # Batched pinverse example
>>> a = torch.randn(2,6,3)
>>> b = torch.pinverse(a)
>>> torch.matmul(b, a)
tensor([[[ 1.0000e+00,  1.6391e-07, -1.1548e-07],
        [ 8.3121e-08,  1.0000e+00, -2.7567e-07],
        [ 3.5390e-08,  1.4901e-08,  1.0000e+00]],

        [[ 1.0000e+00, -8.9407e-08,  2.9802e-08],
        [-2.2352e-07,  1.0000e+00,  1.1921e-07],
        [ 0.0000e+00,  8.9407e-08,  1.0000e+00]]])

torch.qr(input, some=True, *, out=None)¶

Computes the QR decomposition of a matrix or a batch of matrices input, and returns a namedtuple (Q, R) of tensors such that \(\text{input} = Q R\) with \(Q\) being an orthogonal matrix or batch of orthogonal matrices and \(R\) being an upper triangular matrix or batch of upper triangular matrices.

If some is True, then this function returns the thin (reduced) QR factorization. Otherwise, if some is False, this function returns the complete QR factorization.

Warning

torch.qr is deprecated. Please use torch.linalg.qr() instead.

Differences with torch.linalg.qr:

torch.linalg.qr takes a string parameter mode instead of some:
- some=True is equivalent of mode='reduced': both are the default
- some=False is equivalent of mode='complete'.

Warning

If you plan to backpropagate through QR, note that the current backward implementation is only well-defined when the first \(\min(input.size(-1), input.size(-2))\) columns of input are linearly independent. This behavior will propably change once QR supports pivoting.

Note

This function uses LAPACK for CPU inputs and MAGMA for CUDA inputs, and may produce different (valid) decompositions on different device types or different platforms.

Parameters

input (Tensor) – the input tensor of size \((*, m, n)\) where * is zero or more batch dimensions consisting of matrices of dimension \(m \times n\).
some (bool, optional) –
Set to True for reduced QR decomposition and False for complete QR decomposition. If k = min(m, n) then:
- some=True : returns (Q, R) with dimensions (m, k), (k, n) (default)
- 'some=False': returns (Q, R) with dimensions (m, m), (m, n)

Keyword Arguments

out (tuple, optional) – tuple of Q and R tensors. The dimensions of Q and R are detailed in the description of some above.

Example:

>>> a = torch.tensor([[12., -51, 4], [6, 167, -68], [-4, 24, -41]])
>>> q, r = torch.qr(a)
>>> q
tensor([[-0.8571,  0.3943,  0.3314],
        [-0.4286, -0.9029, -0.0343],
        [ 0.2857, -0.1714,  0.9429]])
>>> r
tensor([[ -14.0000,  -21.0000,   14.0000],
        [   0.0000, -175.0000,   70.0000],
        [   0.0000,    0.0000,  -35.0000]])
>>> torch.mm(q, r).round()
tensor([[  12.,  -51.,    4.],
        [   6.,  167.,  -68.],
        [  -4.,   24.,  -41.]])
>>> torch.mm(q.t(), q).round()
tensor([[ 1.,  0.,  0.],
        [ 0.,  1., -0.],
        [ 0., -0.,  1.]])
>>> a = torch.randn(3, 4, 5)
>>> q, r = torch.qr(a, some=False)
>>> torch.allclose(torch.matmul(q, r), a)
True
>>> torch.allclose(torch.matmul(q.transpose(-2, -1), q), torch.eye(5))
True

torch.svd(input, some=True, compute_uv=True, *, out=None)¶

Computes the singular value decomposition of either a matrix or batch of matrices input. The singular value decomposition is represented as a namedtuple (U,S,V), such that input = U diag(S) Vᴴ, where Vᴴ is the transpose of V for the real-valued inputs, or the conjugate transpose of V for the complex-valued inputs. If input is a batch of tensors, then U, S, and V are also batched with the same batch dimensions as input.

If some is True (default), the method returns the reduced singular value decomposition i.e., if the last two dimensions of input are m and n, then the returned U and V matrices will contain only min(n, m) orthonormal columns.

If compute_uv is False, the returned U and V will be zero-filled matrices of shape (m × m) and (n × n) respectively, and the same device as input. The some argument has no effect when compute_uv is False.

Supports input of float, double, cfloat and cdouble data types. The dtypes of U and V are the same as input’s. S will always be real-valued, even if input is complex.

Warning

torch.svd() is deprecated. Please use torch.linalg.svd() instead, which is similar to NumPy’s numpy.linalg.svd.

Note

Differences with torch.linalg.svd():

some is the opposite of torch.linalg.svd()’s full_matricies. Note that default value for both is True, so the default behavior is effectively the opposite.
torch.svd() returns V, whereas torch.linalg.svd() returns Vᴴ.
If compute_uv=False, torch.svd() returns zero-filled tensors for U and Vh, whereas torch.linalg.svd() returns empty tensors.

Note

The singular values are returned in descending order. If input is a batch of matrices, then the singular values of each matrix in the batch is returned in descending order.

Note

The implementation of SVD on CPU uses the LAPACK routine ?gesdd (a divide-and-conquer algorithm) instead of ?gesvd for speed. Analogously, the SVD on GPU uses the cuSOLVER routines gesvdj and gesvdjBatched on CUDA 10.1.243 and later, and uses the MAGMA routine gesdd on earlier versions of CUDA.

Note

The returned matrix U will be transposed, i.e. with strides U.contiguous().transpose(-2, -1).stride().

Note

Gradients computed using U and V may be unstable if input is not full rank or has non-unique singular values.

Note

When some = False, the gradients on U[..., :, min(m, n):] and V[..., :, min(m, n):] will be ignored in backward as those vectors can be arbitrary bases of the subspaces.

Note

The S tensor can only be used to compute gradients if compute_uv is True.

Note

With the complex-valued input the backward operation works correctly only for gauge invariant loss functions. Please look at Gauge problem in AD for more details.

Note

Since U and V of an SVD is not unique, each vector can be multiplied by an arbitrary phase factor \(e^{i \phi}\) while the SVD result is still correct. Different platforms, like Numpy, or inputs on different device types, may produce different U and V tensors.

Parameters

input (Tensor) – the input tensor of size (*, m, n) where * is zero or more batch dimensions consisting of (m × n) matrices.
some (bool, optional) – controls whether to compute the reduced or full decomposition, and consequently the shape of returned U and V. Defaults to True.
compute_uv (bool, optional) – option whether to compute U and V or not. Defaults to True.

Keyword Arguments

out (tuple, optional) – the output tuple of tensors

Example:

>>> a = torch.randn(5, 3)
>>> a
tensor([[ 0.2364, -0.7752,  0.6372],
        [ 1.7201,  0.7394, -0.0504],
        [-0.3371, -1.0584,  0.5296],
        [ 0.3550, -0.4022,  1.5569],
        [ 0.2445, -0.0158,  1.1414]])
>>> u, s, v = torch.svd(a)
>>> u
tensor([[ 0.4027,  0.0287,  0.5434],
        [-0.1946,  0.8833,  0.3679],
        [ 0.4296, -0.2890,  0.5261],
        [ 0.6604,  0.2717, -0.2618],
        [ 0.4234,  0.2481, -0.4733]])
>>> s
tensor([2.3289, 2.0315, 0.7806])
>>> v
tensor([[-0.0199,  0.8766,  0.4809],
        [-0.5080,  0.4054, -0.7600],
        [ 0.8611,  0.2594, -0.4373]])
>>> torch.dist(a, torch.mm(torch.mm(u, torch.diag(s)), v.t()))
tensor(8.6531e-07)
>>> a_big = torch.randn(7, 5, 3)
>>> u, s, v = torch.svd(a_big)
>>> torch.dist(a_big, torch.matmul(torch.matmul(u, torch.diag_embed(s)), v.transpose(-2, -1)))
tensor(2.6503e-06)

torch.symeig(input, eigenvectors=False, upper=True, *, out=None)¶

This function returns eigenvalues and eigenvectors of a real symmetric matrix input or a batch of real symmetric matrices, represented by a namedtuple (eigenvalues, eigenvectors).

This function calculates all eigenvalues (and vectors) of input such that \(\text{input} = V \text{diag}(e) V^T\).

The boolean argument eigenvectors defines computation of both eigenvectors and eigenvalues or eigenvalues only.

If it is False, only eigenvalues are computed. If it is True, both eigenvalues and eigenvectors are computed.

Since the input matrix input is supposed to be symmetric, only the upper triangular portion is used by default.

If upper is False, then lower triangular portion is used.

Note

The eigenvalues are returned in ascending order. If input is a batch of matrices, then the eigenvalues of each matrix in the batch is returned in ascending order.

Note

Irrespective of the original strides, the returned matrix V will be transposed, i.e. with strides V.contiguous().transpose(-1, -2).stride().

Warning

Extra care needs to be taken when backward through outputs. Such operation is only stable when all eigenvalues are distinct and becomes less stable the smaller \(\min_{i \neq j} |\lambda_i - \lambda_j|\) is.

Parameters

input (Tensor) – the input tensor of size \((*, n, n)\) where * is zero or more batch dimensions consisting of symmetric matrices.
eigenvectors (bool, optional) – controls whether eigenvectors have to be computed
upper (boolean, optional) – controls whether to consider upper-triangular or lower-triangular region

Keyword Arguments

out (tuple, optional) – the output tuple of (Tensor, Tensor)

Returns

A namedtuple (eigenvalues, eigenvectors) containing

eigenvalues (Tensor): Shape \((*, m)\). The eigenvalues in ascending order.

eigenvectors (Tensor): Shape \((*, m, m)\). If eigenvectors=False, it’s an empty tensor. Otherwise, this tensor contains the orthonormal eigenvectors of the input.

Return type

(Tensor, Tensor)

Examples:

>>> a = torch.randn(5, 5)
>>> a = a + a.t()  # To make a symmetric
>>> a
tensor([[-5.7827,  4.4559, -0.2344, -1.7123, -1.8330],
        [ 4.4559,  1.4250, -2.8636, -3.2100, -0.1798],
        [-0.2344, -2.8636,  1.7112, -5.5785,  7.1988],
        [-1.7123, -3.2100, -5.5785, -2.6227,  3.1036],
        [-1.8330, -0.1798,  7.1988,  3.1036, -5.1453]])
>>> e, v = torch.symeig(a, eigenvectors=True)
>>> e
tensor([-13.7012,  -7.7497,  -2.3163,   5.2477,   8.1050])
>>> v
tensor([[ 0.1643,  0.9034, -0.0291,  0.3508,  0.1817],
        [-0.2417, -0.3071, -0.5081,  0.6534,  0.4026],
        [-0.5176,  0.1223, -0.0220,  0.3295, -0.7798],
        [-0.4850,  0.2695, -0.5773, -0.5840,  0.1337],
        [ 0.6415, -0.0447, -0.6381, -0.0193, -0.4230]])
>>> a_big = torch.randn(5, 2, 2)
>>> a_big = a_big + a_big.transpose(-2, -1)  # To make a_big symmetric
>>> e, v = a_big.symeig(eigenvectors=True)
>>> torch.allclose(torch.matmul(v, torch.matmul(e.diag_embed(), v.transpose(-2, -1))), a_big)
True

Utilities¶

torch.compiled_with_cxx11_abi()[source]¶: Returns whether PyTorch was built with _GLIBCXX_USE_CXX11_ABI=1

torch¶

Tensors¶

Creation Ops¶

Indexing, Slicing, Joining, Mutating Ops¶

Random sampling¶

In-place random sampling¶

Serialization¶

Parallelism¶

Locally disabling gradient computation¶

Math operations¶

Pointwise Ops¶

Reduction Ops¶

Comparison Ops¶

Spectral Ops¶

Other Operations¶

BLAS and LAPACK Operations¶

Utilities¶

Docs

Tutorials

Resources

ord	matrix norm	vector norm
’fro’	Frobenius norm	–
‘nuc’	nuclear norm	–
Number	–	sum(abs(x)ord)(1./ord)