因為之前在項目中一直使用Tensorflow，最近需要處理NLP問題，對Pytorch框架還比較陌生，所以特地再學習一下pytorch在自然語言處理問題中的簡單使用，這里做一個記錄。

一、Pytorch基礎

首先，第一步是導入pytorch的一系列包

import torchimport torch.autograd as autograd #Autograd為Tensor所有操作提供自動求導方法import torch.nn as nnimport torch.nn.functional as Fimport torch.optim as optim

1）Tensor張量

a) 創建Tensors

#tensorx = torch.Tensor([[1,2,3],[4,5,6]])#size為2x3x4的隨機數隨機數x = torch.randn((2,3,4))

b) Tensors計算

x = torch.Tensor([[1,2],[3,4]])y = torch.Tensor([[5,6],[7,8]])z = x+y

c) Reshape Tensors

x = torch.randn(2,3,4)#拉直x = x.view(-1)#4*6維度x = x.view(4,6)

2）計算圖和自動微分

a) Variable變量

#將Tensor變為Variablex = autograd.Variable(torch.Tensor([1,2,3]),requires_grad = True)#將Variable變為Tensory = x.data

b) 反向梯度算法

x = autograd.Variable(torch.Tensor([1,2]),requires_grad=True)y = autograd.Variable(torch.Tensor([3,4]),requires_grad=True)z = x+y#求和s = z.sum()#反向梯度傳播s.backward()print(x.grad)

c) 線性映射

linear = nn.Linear(3,5) #三維線性映射到五維x = autograd.Variable(torch.randn(4,3))#輸出為（4,5）維y = linear(x)

d) 非線性映射（激活函數的使用）

x = autograd.Variable(torch.randn(5))#relu激活函數x_relu = F.relu(x)print(x_relu)x_soft = F.softmax(x)#softmax激活函數print(x_soft)print(x_soft.sum())

output:

Variable containing:-0.9347-0.9882 1.3801-0.1173 0.9317[torch.FloatTensor of size 5] Variable containing: 0.0481 0.0456 0.4867 0.1089 0.3108[torch.FloatTensor of size 5] Variable containing: 1[torch.FloatTensor of size 1] Variable containing:-3.0350-3.0885-0.7201-2.2176-1.1686[torch.FloatTensor of size 5]

二、Pytorch創建網絡

1) word embedding詞嵌入

通過nn.Embedding(m,n)實現，m表示所有的單詞數目，n表示詞嵌入的維度。

word_to_idx = {'hello':0,'world':1}embeds = nn.Embedding(2,5) #即兩個單詞，單詞的詞嵌入維度為5hello_idx = torch.LongTensor([word_to_idx['hello']])hello_idx = autograd.Variable(hello_idx)hello_embed = embeds(hello_idx)print(hello_embed)

output:

Variable containing:-0.6982 0.3909 -1.0760 -1.6215 0.4429[torch.FloatTensor of size 1x5]

2) N-Gram 語言模型

先介紹一下N-Gram語言模型，給定一個單詞序列 ，計算 ，其中 是序列的第 個單詞。

import torchimport torch.nn as nnimport torch.nn.functional as Fimport torch.autograd as autogradimport torch.optim as optim from six.moves import xrange

對句子進行分詞：

context_size = 2embed_dim = 10text_sequence = """When forty winters shall besiege thy brow,And dig deep trenches in thy beauty's field,Thy youth's proud livery so gazed on now,Will be a totter'd weed of small worth held:Then being asked, where all thy beauty lies,Where all the treasure of thy lusty days;To say, within thine own deep sunken eyes,Were an all-eating shame, and thriftless praise.How much more praise deserv'd thy beauty's use,If thou couldst answer 'This fair child of mineShall sum my count, and make my old excuse,'Proving his beauty by succession thine!This were to be new made when thou art old,And see thy blood warm when thou feel'st it cold.""".split()#分詞trigrams = [ ([text_sequence[i], text_sequence[i+1]], text_sequence[i+2]) for i in xrange(len(text_sequence) - 2) ]trigrams[:10]