CrypTensor¶

A CrypTensor is an encrypted torch tensor for secure computations.

CrypTen currently only supports secure MPC protocols (though we intend to add support for other advanced encryption protocols). Using the MPCTensor backend, a CrypTensor acts as torch tensor whose values are encrypted using the secure MPC protocol.

To create a cryptensor,

# Create torch tensor
x = torch.tensor([1.0, 2.0, 3.0])

# Encrypt x
x_enc = crypten.cryptensor(x)

We can decrypt x_enc by calling x_enc.get_plain_text().

Tensor Operations¶

CrypTensor``s provide various operations similar to ``torch tensors.

class crypten.cryptensor.CrypTensor(requires_grad=False)¶

Abstract implementation of encrypted tensor type. Every subclass of CrypTensor must implement the methods defined here. The actual tensor data should live in an instance attribute called _tensor. When implemented, the CrypTensor provides a full autograd implementation to the user.

adaptive_avg_pool2d(output_size)¶

Applies a 2D adaptive average pooling over an input signal composed of several input planes.

See AdaptiveAvgPool2d for details and output shape.

Parameters: output_size – the target output size (single integer or double-integer tuple)

adaptive_max_pool2d(output_size, return_indices=False)¶

Applies a 2D adaptive max pooling over an input signal composed of several input planes.

See AdaptiveMaxPool2d for details and output shape.

Parameters

output_size – the target output size (single integer or double-integer tuple)
return_indices – whether to return pooling indices. Default: False

add(tensor)¶

Adds tensor to this self.

Parameters: tensor – can be a torch tensor or a CrypTensor.

The shapes of self and tensor must be broadcastable.

For a scalar tensor,

\[\text{{out_i}} = \text{{input_i}} + \text{{tensor}}\]

add_(tensor)¶: Adds tensor to self (in-place) see add().

argmax(dim=None, keepdim=False, one_hot=False)¶

Returns the indices of the maximum value of all elements in self

If multiple values are equal to the maximum, ties will be broken (randomly). Note that this deviates from PyTorch’s implementation since PyTorch does not break ties randomly, but rather returns the lowest index of a maximal value.

If keepdim is True, the output tensor are of the same size as self except in the dimension dim where they are of size 1. Otherwise, dim is squeezed, resulting in the output tensors having 1 fewer dimension than self.

If one_hot is True, the output tensor will have the same size as the self and contain elements of value 1 on argmax indices (with random tiebreaking) and value 0 on other indices.

argmin(dim=None, keepdim=False, one_hot=False)¶

Returns the indices of the minimum value of all elements in the self

If multiple values are equal to the minimum, ties will be broken (randomly). Note that this deviates from PyTorch’s implementation since PyTorch does not break ties randomly, but rather returns the lowest index of a minimal value.

If keepdim is True, the output tensor are of the same size as self except in the dimension dim where they are of size 1. Otherwise, dim is squeezed, resulting in the output tensors having 1 fewer dimension than self.

If one_hot is True, the output tensor will have the same size as the self and contain elements of value 1 on argmin indices (with random tiebreaking) and value 0 on other indices.

avg_pool2d(kernel_size, stride=None, padding=0)¶

Perform an average pooling on each 2D matrix of the given tensor

Parameters: kernel_size (int or tuple) – pooling kernel size.

backward(grad_input=None, top_node=True)¶: Backpropagates gradient through the computation graph. The function only maintains the gradients in leaf nodes of the graph.

batchnorm(ctx, weight, bias, running_mean=None, running_var=None, training=False, eps=1e-05, momentum=0.1)¶: Batch normalization.

clone()¶: Returns a copy of the self tensor. The copy has the same size and data type as self.

Note

This function is recorded in the computation graph. Gradients propagating to the cloned tensor will propagate to the original tensor.

conv2d(*args, **kwargs)¶: 2D convolution.

copy_(other)¶: Copies value of other CrypTensor into this CrypTensor.

cos()¶: Computes the cosine of the input.

cumsum(dim)¶

Returns the cumulative sum of elements of self in the dimension dim

For example, if self is a vector of size N, the result will also be a vector of size N, with elements.

\[y_i = x_1 + x_2 + x_3 + \dots + x_i\]

Parameters: dim (int) – the dimension to do the operation over

detach()¶: Detaches tensor from the autograd graph, making it a leaf.

detach_()¶: Detaches tensor from the autograd graph (in-place), making it a leaf.

div(tensor)¶

Divides each element of self with the tensor and returns a new resulting tensor.

\[\text{out}_i = \frac{\text{input}_i}{\text{tensor}_i}\]

The shapes of self and tensor must be broadcastable.

Parameters: tensor (Tensor or float) – the tensor or value in the denominator.

div_(tensor)¶: Element-wise in-place divide by a tensor (see div()).

dot(tensor, weights=None)¶: Perform (weighted) inner product with plain or cipher text.

dropout(p=0.5, training=True, inplace=False)¶

Randomly zeroes some of the elements of the input tensor with probability p.

Parameters

p – probability of a channel to be zeroed. Default: 0.5
training – apply dropout if is True. Default: True
inplace – If set to True, operation is in-place (default=``False``).

dropout2d(p=0.5, training=True, inplace=False)¶

Randomly zero out entire channels (a channel is a 2D feature map, e.g., the \(j\)-th channel of the \(i\)-th sample in the batched input is a 2D tensor \(\text{input}[i, j]\)) of the input tensor). Each channel will be zeroed out independently on every forward call with probability p using samples from a Bernoulli distribution.

Parameters

p – probability of a channel to be zeroed. Default: 0.5
training – apply dropout if is True. Default: True
inplace – If set to True, operation is in-place (default=``False``).

dropout3d(p=0.5, training=True, inplace=False)¶

Randomly zero out entire channels (a channel is a 3D feature map, e.g., the \(j\)-th channel of the \(i\)-th sample in the batched input is a 3D tensor \(\text{input}[i, j]\)) of the input tensor). Each channel will be zeroed out independently on every forward call with probability p using samples from a Bernoulli distribution.

Parameters

p – probability of a channel to be zeroed. Default: 0.5
training – apply dropout if is True. Default: True
inplace – If set to True, operation is in-place (default=``False``)

static enable_grad()¶: Context manager that enables Crypten’s autograd.

eq(tensor)¶

Element-wise equality

The tensor argument can be a number or a tensor whose shape is broadcastable with self

Parameters: tensor (Tensor or float) – the tensor or value to compare.

exp()¶: Computes exponential function on the tensor.

expand(*sizes)¶

Returns a new view of self with singleton dimensions expanded to a larger size.

Passing -1 as the size for a dimension means not changing the size of that dimension.

Tensor can be also expanded to a larger number of dimensions, and the new ones will be appended at the front. For the new dimensions, the size cannot be set to -1.

Expanding a tensor does not allocate new memory, but only creates a new view on the existing tensor where a dimension of size one is expanded to a larger size by setting the stride to 0. Any dimension of size 1 can be expanded to an arbitrary value without allocating new memory.

flatten(start_dim=0, end_dim=-1)¶

Flattens a contiguous range of dims in a tensor.

Parameters

start_dim (int) – the first dim to flatten. Default is 0.
end_dim (int) – the last dim to flatten. Default is -1.

flip(input, dims)¶

Reverse the order of a n-D tensor along given axis in dims.

Parameters: dims (a list or tuple) – axis to flip on

gather(dim, index)¶

Gathers values along an axis specified by dim.

For a 3-D tensor the output is specified by:

out[i][j][k] = input[index[i][j][k]][j][k] # if dim == 0
out[i][j][k] = input[i][index[i][j][k]][k] # if dim == 1
out[i][j][k] = input[i][j][index[i][j][k]] # if dim == 2

ge(tensor)¶

Element-wise greater than or equal to

The tensor argument can be a number or a tensor whose shape is broadcastable with self

Parameters: tensor (Tensor or float) – the tensor or value to compare
Returns: an encrypted boolean valued tensor containing a True at each location where comparison is true

ger(tensor)¶: Compute outer product.

get_plain_text()¶: Decrypts the encrypted tensor.

gt(tensor)¶

Element-wise greater than

The tensor argument can be a number or a tensor whose shape is broadcastable with self.

Parameters: tensor (Tensor or float) – the tensor or value to compare.
Returns: an encrypted boolean valued tensor containing a True at each location where comparison is true

index_add(dim, index, tensor)¶

Accumulate the elements of tensor into self by adding to the indices in the order given in index

Example: if dim == 0 and index[i] == j, then the i-th row of tensor is added to the j-th row of self

Parameters

dim (int) – dimension along which to index
index (LongTensor) – indices of tensor to select from
tensor (MPCTensor or torch.Tensor) – containing values to add

index_add_(dim, index, tensor)¶

Accumulate the elements of tensor into self by adding to the indices in the order given in index

Example: if dim == 0 and index[i] == j, then the i-th row of tensor is added to the j-th row of self

Parameters

dim (int) – dimension along which to index
index (LongTensor) – indices of tensor to select from
tensor (MPCTensor or torch.Tensor) – containing values to add

index_select(dim, index)¶

Returns a new tensor which indexes the self tensor along dimension dim using the entries in index.

The returned tensor has the same number of dimensions as self The dimension dim has the same size as the length of index; other dimensions have the same size as in self.

le(tensor)¶

Element-wise less than or equal to

The tensor argument can be a number or a tensor whose shape is broadcastable with self.

Parameters: tensor (Tensor or float) – the tensor or value to compare.
Returns: an encrypted boolean valued tensor containing a True at each location where comparison is true

log()¶: Computes the natural logarithm of the tensor.

log_softmax(dim, **kwargs)¶: Applies a softmax of a tensor’s elements along a given dimension, followed by a logarithm.

lt(tensor)¶

Element-wise less than

The tensor argument can be a number or a tensor whose shape is broadcastable with self.

Parameters: tensor (Tensor or float) – the tensor or value to compare.
Returns: an encrypted boolean valued tensor containing a True at each location where comparison is true

matmul(tensor)¶

Performs matrix multiplication of self with tensor

The behavior depends on the dimensionality of the tensors as follows:

If both tensors are 1-dimensional, the dot product (scalar) is returned.
If both arguments are 2-dimensional, the matrix-matrix product is returned.
If the first argument is 1-dimensional and the second argument is 2-dimensional, a 1 is prepended to its dimension for the purpose of the matrix multiply. After the matrix multiply, the prepended dimension is removed.
If the first argument is 2-dimensional and the second argument is 1-dimensional, the matrix-vector product is returned.
If both arguments are at least 1-dimensional and at least one argument is N-dimensional (where N > 2), then a batched matrix multiply is returned. If the first argument is 1-dimensional, a 1 is prepended to its dimension for the purpose of the batched matrix multiply and removed after. If the second argument is 1-dimensional, a 1 is appended to its dimension for the purpose of the batched matrix multiple and removed after. The non-matrix (i.e. batch) dimensions are broadcasted (and thus must be broadcastable). For example, if self is a \((j \times 1 \times n \times m)\) tensor and tensor is a \((k \times m \times p)\) tensor, out will be an \((j \times k \times n \times p)\) tensor.

Parameters: tensor (Tensor) – the tensor to be multiplied

max(dim=None, keepdim=False, one_hot=False)¶

Returns the maximum value of all elements in self

If dim is specified, returns a tuple (values, indices) where values is the maximum value of each row of self in the given dimension dim. And indices is the result of an argmax() call with the same keyword arguments (dim, keepdim, and one_hot)

If keepdim is True, the output tensors are of the same size as self except in the dimension dim where they are of size 1. Otherwise, dim is squeezed, resulting in the output tensors having 1 fewer dimension than self.

max_pool2d(kernel_size, padding=None, stride=None, return_indices=False)¶

Applies a 2D max pooling over an input signal composed of several input planes.

If return_indices is True, this will return the one-hot max indices along with the outputs.

These indices will be returned as with dimensions equal to the max_pool2d output dimensions plus the kernel dimensions. This is because each returned index will be a one-hot kernel for each element of the output that corresponds to the maximal block element of the corresponding input block.

A max pool with output tensor of size \((i, j, k, l)\) with kernel size \(m\) and will return an index tensor of size \((i, j, k, l, m, m)\).

[ 0,  1,  2,  3]                    [[0, 0], [0, 0]]
[ 4,  5,  6,  7]         ->         [[0, 1], [0, 1]]
[ 8,  9, 10, 11]         ->         [[0, 0], [0, 0]]
[12, 13, 14, 15]                    [[0, 1], [0, 1]]

Note: This deviates from PyTorch’s implementation since PyTorch returns the index values for each element rather than a one-hot kernel. This deviation is useful for implementing _max_pool2d_backward later.

mean(dim=None)¶: Compute mean.

min(dim=None, keepdim=False, one_hot=False)¶

Returns the minimum value of all elements in self.

If dim is sepcified, returns a tuple (values, indices) where values is the minimum value of each row of self tin the given dimension dim. And indices is the result of an argmin() call with the same keyword arguments (dim, keepdim, and one_hot)

If keepdim is True, the output tensors are of the same size as self except in the dimension dim where they are of size 1. Otherwise, dim is squeezed, resulting in the output tensors having 1 fewer dimension than self.

mul(tensor)¶

Element-wise multiply with a tensor.

\[\text{out}_i = \text{tensor}_i \times \text{self}_i\]

Parameters: tensor (Tensor or float) – the tensor or value to multiply.

The shapes of self and tensor must be broadcastable.

mul_(tensor)¶: Element-wise multiply with a tensor in-place, see mul().

narrow(dim, start, length)¶: Returns a new tensor that is a narrowed version of self The dimension dim is input from start to start + length. The returned tensor and self share the same underlying storage.

ne(tensor)¶

Element-wise inequality

The tensor argument can be a number or a tensor whose shape is broadcastable with self

Parameters: tensor (Tensor or float) – the tensor or value to compare
Returns: an encrypted boolean tensor containing a True at each location where comparison is true.

neg()¶: Returns a new tensor with the negative of the elements of self.

\[\text{out} = -1 \times \text{input}\]

neg_()¶: Negative value of a tensor (in-place), see neg()

static new(*args, **kwargs)¶: Static method that creates a new CrypTensor of same type.

static no_grad()¶: Context manager that disables Crypten’s autograd.

norm(p='fro', dim=None, keepdim=False)¶

Computes the p-norm of the self (or along a dimension)

Parameters

p (str, int, or float) – specifying type of p-norm
dim (int) – optional dimension along which to compute p-norm
keepdim (bool) – whether the output tensor has dim retained or not

pad(pad, mode='constant', value=0)¶: Pads tensor with constant.

reciprocal()¶: Computes the reciprocal of the tensor.

relu()¶: Compute a Rectified Linear function on the input tensor.

repeat(*sizes)¶

Repeats self along the specified dimensions.

Unlike expand(), this function copies the tensor’s data.

Parameters

sizes (torch.Size or int...) – The number of times to repeat this
along each dimension (tensor) –

reshape(shape)¶

Returns a tensor with the same data and number of elements as self but with the specified shape.

Parameters: shape (tuple of ints or int...) – the desired shape

roll(shifts, dims=None)¶: Roll self along the given dimensions dims. Elements that are shifted beyond the last position are re-introduced at the first position. If a dimension is not specified, the tensor will be flattened before rolling and then restored to the original shape.

scatter_add(dim, index, other)¶

Adds all values from the other into self at the indices specified in index. This an out-of-place version of scatter_add_(). For each value in other, it is added to an index in self which is specified by its index in other for dimension != dim and by the corresponding value in index for dimension = dim.

Parameters

dim (int) – the axis along which to index
index (LongTensor) – the indices of elements to scatter and add, can be either empty or the same size of src. When empty, the operation returns identity.
other (Tensor) – the source elements to scatter and add

scatter_add_(dim, index, other)¶

Adds all values from the other into self at the indices specified in index. For each value in other, it is added to an index in self which is specified by its index in other for dimension != dim and by the corresponding value in index for dimension = dim.

Parameters

dim (int) – the axis along which to index
index (LongTensor) – the indices of elements to scatter and add, can be either empty or the same size of src. When empty, the operation returns identity.
other (Tensor) – the source elements to scatter and add

set(enc_tensor)¶: Sets self encrypted to enc_tensor in place

static set_grad_enabled(mode)¶: Enables (mode = True) or disables (mode = False) Crypten’s autograd.

shallow_copy()¶: Creates a shallow copy of the CrypTensor.

sigmoid(reciprocal_method='log')¶: Computes the sigmoid function on the input value sigmoid(x) = (1 + exp(-x))^{-1}

sin()¶: Computes the sine of the input.

softmax(dim, **kwargs)¶: Compute the softmax of a tensor’s elements along a given dimension

sqrt()¶: Computes the square root of self

square()¶: Computes the square of self

squeeze(dim=None)¶

Returns a tensor with all the dimensions of self of size 1 removed.

For example, if self is of shape: (A imes 1 imes B imes C imes 1 imes D)(A×1×B×C×1×D) then the returned tensor will be of shape: (A imes B imes C imes D)(A×B×C×D).

When dim is given, a squeeze() operation is done only in the given dimension. If self is of shape: (A imes 1 imes B)(A×1×B), squeeze(self, 0) leaves the tensor unchanged, but squeeze(self, 1) will squeeze the tensor to the shape (A imes B)(A×B).

sub(tensor)¶: Subtracts a tensor from self tensor. The shape of tensor must be broadcastable with the shape of self.

sub_(tensor)¶: Subtracts tensor from self (in-place), see sub()

sum(dim=None, keepdim=False)¶

Returns the sum of all elements in the self

If dim is a list of dimensions, reduce over all of them.

t()¶

Expects self to be <= 2D tensor and transposes dimensions 0 and 1.

0D and 1D tensors are returned as is and for 2D tensors this can be seen as a short-hand function for self.transpose(0, 1).

take(index, dimension=None)¶: Returns a new tensor with the elements of input at the given indices. When the dimension is None, self tensor is treated as if it were viewed as a 1D tensor, and the result takes the same shape as the indices. When the dimension is an integer, the result take entries of tensor along a dimension according to the index.

tanh(reciprocal_method='log')¶: Computes tanh from the sigmoid function: tanh(x) = 2 * sigmoid(2 * x) - 1

trace()¶: Returns the sum of the elements of the diagonal of self. self has to be a 2D tensor.

transpose(dim0, dim1)¶

Returns a tensor that is a transposed version of self The given dimensions dim0 and dim1 are swapped.

The resulting tensor shares it’s underlying storage with self, so changing the content of one would change the content of the other.

Parameters

dim0 (int) – the first dimension to be transposed
dim1 (int) – the second dimension to be transposed

unfold(dimension, size, step)¶

Returns a tensor which contains all slices of size size from self in the dimension dimension.

Step between two slices is given by step

If sizedim is the size of dimension for self, the size of dimension in the returned tensor will be (sizedim - size) / step + 1.

An additional dimension of size size is appended in the returned tensor.

Parameters

dimension (int) – dimension in which unfolding happens
size (int) – the size of each slice that is unfolded
step (int) – the step between each slice

unsqueeze(dim)¶

Returns a new tensor with a dimension of size one inserted at the specified position.

The returned tensor shares the same underlying data with self

A dim value within the range [-self.dim() - 1, self.dim() + 1) can be used. Negative dim will correspond to unsqueeze() applied at dim = dim + self.dim() + 1.

Parameters: dim (int) – the index at which to insert the singleton dimension

var(dim=None)¶: Compute variance.

view(*shape)¶

Returns a new encrypted tensor with the same data as the self tensor but of a different shape.

The returned tensor shares the same data and must have the same number of elements, but may have a different size. For a tensor to be viewed, the new view size must be compatible with its original size and stride, i.e., each new view dimension must either be a subspace of an original dimension, or only span across original dimensions \(d, d+1, \dots, d+k\) that satisfy the following contiguity-like condition that \(\forall i = 0, \dots, k-1\),

\[\text{stride}[i] = \text{stride}[i+1] \times \text{size}[i+1]\]

Parameters: shape (torch.Size or int...) – the desired

where(condition, y)¶

Selects elements from self or y based on condition

Parameters

condition (torch.bool or MPCTensor) – when True yield self, otherwise yield y
y (torch.tensor or CrypTensor) – values selected at indices where condition is False.

Returns: CrypTensor or torch.tensor

class crypten.cryptensor.CrypTensorMetaclass¶: Metaclass for CrypTensor that ensures autograd is invoked for calls to static methods such as crypten.cat and crypten.stack.

Autograd¶

CrypTensor``s support autograd similar to ``torch tensors. You can set the requires_grad attribute on a CrypTensor to record forward computations, and call backward() on a CrypTensor to perform backpropagation.

# Create an CrypTensor that supports autograd
x_enc = crypten.cryptensor(x_enc, requires_grad=True)
y_enc = x_enc.mul(2).sum()
y_enc.backward()

For an example of backpropagation using CrypTensor, please see Tutorial 7.

File I/O Utilities¶

CrypTen provides utilities for loading CrypTensors from files and saving CrypTensors to files.

crypten.load(f, preloaded=None, encrypted=False, dummy_model=None, src=0, load_closure=<function load>, **kwargs)¶: Loads an object saved with torch.save() or crypten.save_from_party(). Note: this function is deprecated; please use load_from_party instead.

crypten.save(obj, f, src=0, save_closure=<function save>, **kwargs)¶: Saves a CrypTensor or PyTorch tensor to a file. Note: this function is deprecated, please use save_from_party instead