English to German Translation using Seq2Seq Models In PyTorch

English to German Translation using Seq2Seq Models In PyTorch

Sequence to Sequence models, also referred to as encoder-decoder models, are a family of models that typically train 2 recurrent neural networks. The first RNN, the encoder, is trained to recieve input text and encode it sequentially. The second RNN, the decoder, receives that encoded sequence and performs a mapping to the text. This unique method of training 2 RNNs together was introduced by Cho et al. in https://arxiv.org/pdf/1406.1078v3.pdfand instantly gained popularity in NLP tasks where the input and output are explicit text pairs, such as translation and summarization.

In the following tutorial, we will explore how to create and train Seq2Seq models in PyTorch for English-German translation.

Overview:

• Imports and Data Loading
• Tokenization
• Creating Encoder RNN
• Creating Decoder RNN
• Setup and Training
• Evaluation

Imports and Data Loading

In [1]:

import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import DataLoader, TensorDataset

import numpy as np

import matplotlib.pyplot as plt

We use the Multi30k dataset, a

(continued...)

from Planet SciPy
read more