PyTorch_《Tutorials》

本篇章主要记录的是官网提供的Tutorials和PyTorch官方文档(中文版)的重要内容，是Pytorch探索之旅的第2篇章啦！

Reference materials include:

  • PyTorch官方文档(中文版): 我感觉这个文档的好处在于非常详尽的给出相关函数和模块的细致化定义介绍
  • Tutorials：感觉官网的这个文档是从Pytorch的使用框架去简要介绍

1 Quickstart

1.1 Working with data

PyTorch has two primitives to work with data: torch.utils.data.DataLoader and torch.utils.data.Dataset. Dataset stores the samples and their corresponding labels, and DataLoader wraps an iterable around the Dataset. PyTorch offers domain-specific libraries such as TorchText, TorchVision, and TorchAudio, all of which include datasets. The torchvision.datasets module contains Dataset objects for many real-world vision data like CIFAR, COCO (full list here).
Every TorchVision Dataset includes two arguments: transform and target_transform to modify the samples and labels respectively.

1.2 Creating Models

To define a neural network in PyTorch, we create a class that inherits from nn.Module.

  • We define the layers of the network in the __init__ function
  • Specify how data will pass through the network in the forward function
  • To accelerate operations in the neural network, we move it to the GPU or MPS if available.

1.3 Optimizing the Model Parameters

To train a model, we need a loss function and an optimizer.

1.4 Saving Models

1
2
torch.save(model.state_dict(), "model.pth")
print("Saved PyTorch Model State to model.pth")

1.5 Loading Models

The process for loading a model includes re-creating the model structure and loading the state dictionary into it.

1
2
model = NeuralNetwork().to(device)
model.load_state_dict(torch.load("model.pth"))

2 Tensors

Tensors are similar to NumPy’s ndarrays, except that tensors can run on GPUs or other hardware accelerators.

2.1 Initializing a Tensor

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# 1. Directly from data
data = [[1, 2],[3, 4]]
x_data = torch.tensor(data)

# 2. From a NumPy array
np_array = np.array(data)
x_np = torch.from_numpy(np_array)

# 3. From another tensor
x_ones = torch.ones_like(x_data) # retains the properties of x_data
x_rand = torch.rand_like(x_data, dtype=torch.float) # overrides the datatype of x_data

# 4. With random or constant values
shape = (2,3,)
rand_tensor = torch.rand(shape)
ones_tensor = torch.ones(shape)
zeros_tensor = torch.zeros(shape)

2.2 Attributes of a Tensor

1
2
3
print(f"Shape of tensor: {tensor.shape}")
print(f"Datatype of tensor: {tensor.dtype}")
print(f"Device tensor is stored on: {tensor.device}")

2.3 Operations on Tensor

Over 100 tensor operations, including arithmetic, linear algebra, matrix manipulation (transposing, indexing, slicing), sampling and more are comprehensively described here.
By default, tensors are created on the CPU. We need to explicitly move tensors to the GPU using .to method (after checking for GPU availability). Keep in mind that copying large tensors across devices can be expensive in terms of time and memory!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# We move our tensor to the GPU if available
if torch.cuda.is_available():
    tensor = tensor.to("cuda")

# 1 Standard numpy-like indexing and slicing
tensor = torch.ones(4, 4)
print(f"First row: {tensor[0]}")
print(f"First column: {tensor[:, 0]}")
print(f"Last column: {tensor[..., -1]}")
tensor[:,1] = 0
print(tensor)
## 输出
First row: tensor([1., 1., 1., 1.])
First column: tensor([1., 1., 1., 1.])
Last column: tensor([1., 1., 1., 1.])
tensor([[1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.]])

# 2 Joining tensors
# - `torch.cat`: Concatenates the given sequence along an existing dimension.
# - `torch.stack`: Concatenates a sequence of tensors along a new dimension.

# 3 Single-element tensors 
# If you have a one-element tensor, for example by aggregating all values of a tensor into one value, you can convert it to a Python numerical value using `item()`

2.4 Bridge with NumPy

Tensors on the CPU and NumPy arrays can share their underlying memory locations, and changing one will change the other.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Tensor to NumPy array
n = t.numpy()
# NumPy array to Tensor
t = torch.from_numpy(n)

## Changes in the NumPy array reflects in the tensor.
n = np.ones(5)
t = torch.from_numpy(n)
np.add(n, 1, out=n)
print(f"t: {t}")
print(f"n: {n}")
### 输出
t: tensor([2., 2., 2., 2., 2.], dtype=torch.float64)
n: [2. 2. 2. 2. 2.]

3 Datasets and DataLoaders

3.1 Loading a Dataset

We can load the FashionMNIST Dataset with the following parameters:

  • root is the path where the train/test data is stored,
  • train specifies training or test dataset,
  • download=True downloads the data from the internet if it’s not available at root.
  • transform and target_transform specify the feature and label transformations
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    import torch
    from torch.utils.data import Dataset
    from torchvision import datasets
    from torchvision.transforms import ToTensor
    import matplotlib.pyplot as plt


    training_data = datasets.FashionMNIST(
        root="data",
        train=True,
        download=True,
        transform=ToTensor()
    )

    test_data = datasets.FashionMNIST(
        root="data",
        train=False,
        download=True,
        transform=ToTensor()
    )

3.2 Iterating and Visualizing the Dataset

We can index Datasets manually like a list: training_data[index]. We use matplotlib to visualize some samples in our training data.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
labels_map = {
    0: "T-Shirt",
    1: "Trouser",
    2: "Pullover",
    3: "Dress",
    4: "Coat",
    5: "Sandal",
    6: "Shirt",
    7: "Sneaker",
    8: "Bag",
    9: "Ankle Boot",
}
figure = plt.figure(figsize=(8, 8))
cols, rows = 3, 3
for i in range(1, cols * rows + 1):
    sample_idx = torch.randint(len(training_data), size=(1,)).item()
    img, label = training_data[sample_idx]
    figure.add_subplot(rows, cols, i)
    plt.title(labels_map[label])
    plt.axis("off")
    plt.imshow(img.squeeze(), cmap="gray")
plt.show()

3.3 Creating a Custom Dataset for your files

A custom Dataset class must implement three functions: __init__, __len__, and __getitem__.
The FashionMNIST images are stored in a directory img_dir, and their labels are stored separately in a CSV file annotations_file.
The labels.csv file looks like:

1
2
3
4
tshirt1.jpg, 0
tshirt2.jpg, 0
......
ankleboot999.jpg, 9

Code as fellow:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
import os
import pandas as pd
from torchvision.io import read_image

class CustomImageDataset(Dataset):
    # The __init__ function is run once when instantiating the Dataset object.
    def __init__(self, annotations_file, img_dir, transform=None, target_transform=None):
        self.img_labels = pd.read_csv(annotations_file)
        self.img_dir = img_dir
        self.transform = transform
        self.target_transform = target_transform

    # The __len__ function returns the number of samples in our dataset.
    def __len__(self):
        return len(self.img_labels)

    # The __getitem__ function loads and returns a sample from the dataset at the given index idx.
    def __getitem__(self, idx):
        # Based on the index, it identifies the image’s location on disk
        # converts that to a tensor using read_image
        # retrieves the corresponding label from the csv data in self.img_labels
        # calls the transform functions on them (if applicable)
        # and returns the tensor image and corresponding label in a tuple.
        img_path = os.path.join(self.img_dir, self.img_labels.iloc[idx, 0])
        image = read_image(img_path)
        label = self.img_labels.iloc[idx, 1]
        if self.transform:
            image = self.transform(image)
        if self.target_transform:
            label = self.target_transform(label)
        return image, label

3.4 Preparing your data for training with DataLoaders

The Dataset retrieves our dataset’s features and labels one sample at a time. While training a model, we typically want to pass samples in “minibatches”, reshuffle the data at every epoch to reduce model overfitting, and use Python’s multiprocessing to speed up data retrieval.

1
2
3
4
from torch.utils.data import DataLoader

train_dataloader = DataLoader(training_data, batch_size=64, shuffle=True)
test_dataloader = DataLoader(test_data, batch_size=64, shuffle=True)

3.5 Iterate through the DataLoader

We have loaded that dataset into the DataLoader and can iterate through the dataset as needed. Each iteration below returns a batch of train_features and train_labels (containing batch_size=64 features and labels respectively). Because we specified shuffle=True, after we iterate over all batches the data is shuffled (for finer-grained control over the data loading order, take a look at Samplers).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# Display image and label.
train_features, train_labels = next(iter(train_dataloader))
print(f"Feature batch shape: {train_features.size()}")
print(f"Labels batch shape: {train_labels.size()}")
img = train_features[0].squeeze()
label = train_labels[0]
plt.imshow(img, cmap="gray")
plt.show()
print(f"Label: {label}")

## 输出
## 一张图示
Feature batch shape: torch.Size([64, 1, 28, 28])
Labels batch shape: torch.Size([64])
Label: 5

4 Transformers

We use transforms to perform some manipulation of the data and make it suitable for training. All TorchVision datasets have two parameters transform to modify the features and target_transform to modify the labels that accept callables containing the transformation logic. More resource in torchvision.transforms module 

1
2
3
4
5
6
7
8
9
10
11
12
import torch
from torchvision import datasets
from torchvision.transforms import ToTensor, Lambda

ds = datasets.FashionMNIST(
    root="data",
    train=True,
    download=True,
    transform=ToTensor(),
    # Lambda transforms apply any user-defined lambda function. 
    target_transform=Lambda(lambda y: torch.zeros(10, dtype=torch.float).scatter_(0, torch.tensor(y), value=1))
)

5 Build Model

  • The nn.Flatten layer converts each 2D 28x28 image into a contiguous array of 784 pixel values.
  • The nn.Linear layer is a module that applies a linear transformation on the input using its stored weights and biases.
  • The nn.ReLU creates the complex mappings between the model’s inputs and outputs.
  • nn.Sequential is an ordered container of modules.
    1
    2
    3
    4
    5
    6
    7
    8
    seq_modules = nn.Sequential(
        flatten,
        layer1,
        nn.ReLU(),
        nn.Linear(20, 10)
    )
    input_image = torch.rand(3,28,28)
    logits = seq_modules(input_image)
  • nn.Softmax : The logits are scaled to values [0, 1] representing the model’s predicted probabilities for each class.

6 Automatic Differentiation

Autograd mechanics

7 Optimization Loop

Training a model is an iterative process; in each iteration the model makes a guess about the output, calculates the error in its guess (loss), collects the derivatives of the error with respect to its parameters (as we saw in the previous section), and optimizes these parameters using gradient descent.
We define the following hyperparameters for training:

  • Number of Epochs: the number times to iterate over the dataset
  • Batch Size: the number of data samples propagated through the network before the parameters are updated
  • Learning Rate: how much to update models parameters at each batch/epoch. Smaller values yield slow learning speed, while large values may result in unpredictable behavior during training

Each iteration of the optimization loop is called an epoch.And it consists of two main parts:

  • The Train Loop：iterate over the training dataset and try to converge to optimal parameters.
  • The Validation/Test Loop：iterate over the test dataset to check if model performance is improving.

Common loss functions include nn.MSELoss (Mean Square Error) for regression tasks, and nn.NLLLoss (Negative Log Likelihood) for classification. nn.CrossEntropyLoss combines nn.LogSoftmax and nn.NLLLoss.

Optimization is the process of adjusting model parameters to reduce model error in each training step.

8 Save, Load and Use Model

PyTorch models store the learned parameters in an internal state dictionary, called state_dict. These can be persisted via the torch.save method:

1
2
3
4
5
6
7
model = models.vgg16(weights='IMAGENET1K_V1')
torch.save(model.state_dict(), 'model_weights.pth')

# To load model weights, you need to create an instance of the same model first, and then load the parameters 
model.load_state_dict(torch.load('model_weights.pth'))
# be sure to call model.eval() method before inferencing to set the dropout and batch normalization layers to evaluation mode.
model.eval()

9 PACKAGE参考

9.1 torch

torch包含了多维张量的数据结构以及基于其上的多种数学操作。另外，它也提供了多种工具，其中一些可以更有效地对张量和任意类型进行序列化。

9.1.1 Tensor

  • torch.is_tensor(obj)
  • torch.is_storage(obj)
  • torch.numel(input): 返回input张量中的元素个数
  • torch.set_printoptions(precision=None, threshold=None, edgeitems=None, linewidth=None, profile=None)
    • 设置打印选项。
    • precision：浮点数输出的精度位数 (默认为8)
    • threshold：阈值，触发汇总显示而不是完全显示(repr)的数组元素的总数 （默认为1000）
    • edgeitems：汇总显示中，每维（轴）两端显示的项数（默认值为3）
    • linewidth：用于插入行间隔的每行字符数（默认为80）。Thresholded matricies will ignore this parameter.
    • profile：pretty打印的完全默认值。 可以覆盖上述所有选项 (默认为short, full)

9.1.2 创建操作

  • torch.eye(n, m=None, out=None): 返回一个nxn的2维张量，对角线位置全1，其它位置全0
  • torch.from_numpy(ndarray)
  • torch.linspace(start, end, steps=100, out=None): 返回一个1维张量，包含在区间start和end上均匀间隔的steps个点。 输出1维张量的长度为steps
  • torch.logspace(start, end, steps=100, out=None)
  • torch.ones(*sizes, out=None)
  • torch.rand(*sizes, out=None): 返回一个张量，包含了从区间[0,1)的均匀分布中抽取的一组随机数，形状由可变参数sizes 定义
  • torch.randn(*sizes, out=None): 返回一个张量，包含了从标准正态分布(均值为0，方差为 1，即高斯白噪声)中抽取一组随机数，形状由可变参数sizes定义
  • torch.randperm(n, out=None): 给定参数n，返回一个从0 到n -1 的随机整数排列
  • torch.arange(start, end, step=1, out=None): 包含从start到end，以step为步长的一组序列值(默认步长为1)
  • torch.zeros(*sizes, out=None)

9.1.3 索引,切片,连接,换位

  • torch.cat(inputs, dimension=0)
  • torch.chunk(tensor, chunks, dim=0): 在给定维度(轴)上将输入张量进行分块儿
  • torch.gather(input, dim, index, out=None): 沿给定轴dim，将输入索引张量index指定位置的值进行聚合
  • torch.index_select(input, dim, index, out=None): 沿着指定维度对输入进行切片，取index中指定的相应项(index为一个LongTensor)，然后返回到一个新的张量， 返回的张量与原始张量_Tensor_有相同的维度(在指定轴上)。注意： 返回的张量不与原始张量共享内存空间
  • torch.masked_select(input, mask, out=None)
  • torch.nonzero(input, out=None): 返回一个包含输入input中非零元素索引的张量
  • torch.split(tensor, split_size, dim=0): 将输入张量分割成相等形状的chunks（如果可分）
  • torch.squeeze(input, dim=None, out=None): 将输入张量形状中的1去除并返回
  • torch.stack(sequence, dim=0): 沿着一个新维度对输入张量序列进行连接。 序列中所有的张量都应该为相同形状
  • torch.t: 输入一个矩阵（2维张量），并转置0, 1维。 可以被视为函数transpose(input, 0, 1)的简写函数
  • torch.transpose(input, dim0, dim1, out=None): 返回输入矩阵input的转置。交换维度dim0和dim1。 输出张量与输入张量共享内存，所以改变其中一个会导致另外一个也被修改
  • torch.unbind: 移除指定维后，返回一个元组，包含了沿着指定维切片后的各个切片
  • torch.unsqueeze(input, dim, out=None): 返回一个新的张量，对输入的制定位置插入维度 1

9.1.4 随机抽样

  • torch.manual_seed(seed)
  • torch.initial_seed()
  • torch.get_rng_state()
  • torch.set_rng_state(new_state)
  • torch.default_generator = <torch._C.Generator object>
  • torch.bernoulli(input, out=None): 从伯努利分布中抽取二元随机数(0 或者 1)。输入张量须包含用于抽取上述二元随机值的概率
  • torch.multinomial(input, num_samples,replacement=False, out=None): 返回一个张量，每行包含从input相应行中定义的多项分布中抽取的num_samples个样本
  • torch.normal(means, std, out=None): 返回一个张量，包含从给定参数means,std的离散正态分布中抽取随机数

9.1.5 序列化

  • torch.save(obj, f, pickle_module=<module 'pickle' from '/home/jenkins/miniconda/lib/python3.5/pickle.py'>, pickle_protocol=2): 保存一个对象到一个硬盘文件上
  • torch.load

9.1.6 并行化

  • torch.get_num_threads()：获得用于并行化CPU操作的OpenMP线程数
  • torch.set_num_threads(): 设定用于并行化CPU操作的OpenMP线程数

9.1.7 数学操作

  • torch.abs(input, out=None): 计算输入张量的每个元素绝对值
  • torch.sin(input, out=None)
  • torch.sinh(input, out=None): 双曲正弦
  • torch.cos(input, out=None): 返回一个新张量，包含输入input张量每个元素的余弦
  • torch.cosh(input, out=None): 双曲余弦
  • torch.tan(input, out=None)
  • torch.tanh(input, out=None):双曲正切
  • torch.acos(input, out=None): 返回一个新张量，包含输入张量每个元素的反余弦
  • torch.asin(input, out=None): 返回一个新张量，包含输入input张量每个元素的反正弦函数
  • torch.atan(input, out=None): 返回一个新张量，包含输入input张量每个元素的反正切函数
  • torch.atan2(input1, input2, out=None): 返回一个新张量，包含两个输入张量input1和input2的反正切函数
  • torch.add(input, value, out=None)
  • torch.addcdiv(tensor, value=1, tensor1, tensor2, out=None): 用tensor2对tensor1逐元素相除，然后乘以标量值value 并加到tensor
  • torch.addcmul(tensor, value=1, tensor1, tensor2, out=None): 用tensor2对tensor1逐元素相乘，并对结果乘以标量值value然后加到tensor
  • torch.ceil(input, out=None): 天井函数，对输入input张量每个元素向上取整, 即取不小于每个元素的最小整数，并返回结果到输出
  • torch.floor(input, out=None): 床函数,返回一个新张量，包含输入input张量每个元素的floor，即不小于元素的最大整数
  • torch.round(input, out=None): 返回一个新张量，将输入input张量每个元素舍入到最近的整数
  • torch.sign(input, out=None): 符号函数：返回一个新张量，包含输入input张量每个元素的正负
  • torch.trunc(input, out=None): 返回一个新张量，包含输入input张量每个元素的截断值(标量x的截断值是最接近其的整数，其比x更接近零。简而言之，有符号数的小数部分被舍弃)
  • torch.clamp(input, min, max, out=None): 将输入input张量每个元素的夹紧到区间 [min,max]，并返回结果到一个新张量
  • torch.div(input, value, out=None): 将input逐元素除以标量值value，并返回结果到输出张量out
  • torch.exp(tensor, out=None): 返回一个新张量，包含输入input张量每个元素的指数
  • torch.fmod(input, divisor, out=None): 计算除法余数，其结果的符号与 input 相同
  • torch.remainder(input, divisor, out=None): 返回一个新张量，包含输入input张量每个元素的除法余数，其结果与 divisor 相同
  • torch.frac(tensor, out=None): 返回每个元素的分数部分
  • torch.lerp(start, end, weight, out=None): 对两个张量以start，end做线性插值， 将结果返回到输出张量。out = start +weight * (end - start)
  • torch.log(input, out=None): 计算input的自然对数
  • torch.log1p(input, out=None) : 计算input+1的自然对数
  • torch.mul(input, value, out=None): 用标量值value乘以输入input的每个元素，并返回一个新的结果张量
  • torch.neg(input, out=None): 返回一个新张量，包含输入input张量按元素取负
  • torch.pow(input, exponent, out=None): 对输入input的按元素求exponent次幂值，并返回结果张量
  • torch.reciprocal(input, out=None): 倒数
  • torch.rsqrt(input, out=None): 平方根倒数
  • torch.sqrt(input, out=None): 平方根
  • torch.sigmoid(input, out=None)

9.1.8 Reduction Ops(缩减操作)

  • torch.cumprod(input, dim, out=None): 返回输入沿指定维度的累积
  • torch.prod(input): 返回输入张量input 所有元素的积
  • torch.cumsum(input, dim, out=None): 返回输入沿指定维度的累和
  • torch.sum(input): 返回输入张量input 所有元素的和
  • torch.dist(input, other, p=2, out=None): 返回 (input - other) 的 p范数
  • torch.mean(input)
  • torch.median(input, dim=-1, values=None, indices=None): 返回输入张量给定维度每行的中位数，同时返回一个包含中位数的索引的LongTensor
  • torch.mode(input, dim=-1, values=None, indices=None): 返回给定维dim上，每行的众数值。 同时返回一个LongTensor，包含众数职的索引
  • torch.norm(input, p=2): 返回输入张量input 的p 范数
  • torch.std(input): 返回输入张量input 所有元素的标准差
  • torch.var(input): 返回输入张量所有元素的方差

9.1.8 Comparison Ops(比较操作)

  • torch.eq(input, other, out=None)：比较元素相等性, 返回一个torch.ByteTensor张量，包含了每个位置的比较结果(相等为1，不等为0 )
  • torch.equal(tensor1, tensor2): 如果两个张量有相同的形状和元素值，则返回True ，否则 False
  • torch.ge(input, other, out=None): 逐元素比较input和other，即是否 input>=other. 返回一个 torch.ByteTensor 张量，包含了每个位置的比较结果(是否 input >= other )
  • torch.gt(input, other, out=None): 逐元素比较input和other ， 即是否input>other. 如果两个张量有相同的形状和元素值，则返回True ，否则 False
  • torch.le(input, other, out=None): 逐元素比较input和other ， 即是否input<=other
  • torch.lt(input, other, out=None): 逐元素比较input和other ， 即是否 input < other
  • torch.ne(input, other, out=None): 逐元素比较input和other ， 即是否 input!=other
  • torch.kthvalue(input, k, dim=None, out=None): 取输入张量input指定维上第k 个最小值
  • torch.max()
  • torch.min(input)
  • torch.sort(input, dim=None, descending=False, out=None): 对输入张量input沿着指定维按升序排序
  • torch.topk(input, k, dim=None, largest=True, sorted=True, out=None): 沿给定dim维度返回输入张量input中 k 个最大值

9.1.9 Other Operations(其他操作)

  • torch.cross(input, other, dim=-1, out=None): 返回沿着维度dim上，两个张量input和other的向量积（叉积）
  • torch.diag(input, diagonal=0, out=None)
    • 如果输入是一个向量(1D 张量)，则返回一个以input为对角线元素的2D方阵
    • 如果输入是一个矩阵(2D 张量)，则返回一个包含input对角线元素的1D张量
  • torch.histc(input, bins=100, min=0, max=0, out=None): 计算输入张量的直方图. 如果min和max都为0, 则利用数据中的最大最小值作为边界
  • torch.renorm(input, p, dim, maxnorm, out=None): 返回一个张量，包含规范化后的各个子张量，使得沿着dim维划分的各子张量的p范数小于maxnorm
  • torch.trace(input): 返回输入2维矩阵对角线元素的和
  • torch.tril(input, k=0, out=None): 返回一个张量out，包含输入矩阵(2D张量)的下三角部分，out其余部分被设为0
  • torch.triu(input, k=0, out=None): 返回一个张量，包含输入矩阵(2D张量)的上三角部分，其余部分被设为0

9.1.10 BLAS and LAPACK Operations

  • torch.addbmm(beta=1, mat, alpha=1, batch1, batch2, out=None): 执行mat = beta * mat + alpha * batch1.bmm(batch2) 操作。
    • mat：二维张量（n, p）
    • batch1：三维张量（batch_size, n, m）
    • batch2：三维张量（batch_size, m, p）
  • torch.baddbmm(beta=1, mat, alpha=1, batch1, batch2, out=None): 执行 mat = beta * mat + alpha * torch.bmm(batch1, batch2) 操作
    • mat：三维张量（batch_size, n, p）
    • batch1：三维张量（batch_size, n, m）
    • batch2：三维张量（batch_size, m, p）
  • torch.bmm(batch1, batch2, out=None): 对存储在两个批batch1和batch2内的矩阵进行批矩阵乘操作
    • batch1：三维张量（batch_size, n, m）
    • batch2：三维张量（batch_size, m, p）
  • torch.mm(mat1, mat2, out=None): 执行二维矩阵乘法
  • torch.addmm(beta=1, mat, alpha=1, mat1, mat2, out=None): 适用于将批量矩阵乘法的结果与一个二维矩阵按比例相加
  • torch.mv(mat, vec, out=None): 对矩阵mat和向量vec进行相乘
  • torch.addmv(beta=1, tensor, alpha=1, mat, vec, out=None): 向量相乘
  • torch.addr(beta=1, mat, alpha=1, vec1, vec2, out=None)：张量积
  • torch.btrifact(A, info=None): 返回一个元组，包含LU 分解和pivots
  • torch.btrisolve(b, LU_data, LU_pivots): 返回线性方程组Ax=b的LU解
  • torch.dot(tensor1, tensor2)
  • torch.eig(a, eigenvectors=False, out=None): 计算实方阵a的特征值和特征向量
  • torch.gels(B, A, out=None): 对形如m×n的满秩矩阵a计算其最小二乘和最小范数问题的解
  • torch.geqrf(input, out=None): 直接调用LAPACK的底层函数
  • torch.ger(vec1, vec2, out=None): 计算两向量vec1,vec2的张量积
  • torch.gesv(B, A, out=None): 返回线性方程组AX=B的解
  • torch.inverse(input, out=None): 对方阵输入input取逆
  • torch.svd(input, some=True, out=None): 返回对形如n×m的实矩阵A进行奇异值分解的结果
  • torch.symeig(input, eigenvectors=False, upper=True, out=None): 返回实对称矩阵input的特征值和特征向量

9.2 torch.Tensor

torch.Tensor是一种包含单一数据类型元素的多维矩阵。

！注意： 会改变tensor的函数操作会用一个下划线后缀来标示。比如，torch.FloatTensor.abs_()会在原地计算绝对值，并返回改变后的tensor，而tensor.FloatTensor.abs()将会在一个新的tensor中计算结果。

9.3 torch.Storage

torch.Storage是一个单一数据类型的连续一维数组。

9.4 torch.nn

torch.nn

Tutorial > PyTorch
#PyTorch
PyTorch_《Tutorials》
http://example.com/2024/05/29/1_Pytorch_1/
Author
Xi Wang
Posted on
May 29, 2024
Licensed under