seq2seq code 구현 예시

개요

• 이하 구성은 batch train; encoder train; decoder train; 의 샘플코드를 작성
• 그림 참조
  
  

샘플코드

• 구현 순서: batch data iter -> encoder forward -> get hidden state -> decoder forward -> loss

• batch data: [batch size, max_seq_len]

  # 각 value는 voc_idx
  # max_seq_len는 encoder/decoder의 time step
  # shape: [batch size, max_seq_len]
  tensor([[24,  6, 22,  ...,  2,  2,  2],
  , [20,  1,  ...,  2,  2,  2],
  , [4, 15,  ...,  2,  2,  2], ...
  , [19,  7, 16,  ...,  2,  2,  2]
  , [21, 28, 24,  ...,  2,  2,  2]
  , [ 4, 18,  1,  ...,  2,  2,  2]])

• forward batch data

  # batch train
  for batch_x, batch_y in train_loader:
  	loss = train(batch_x, batch_y, encoder, decoder)
  
  # encoder part
  for e in range(max_seq_len_en):
  	encoder_output, encoder_hidden = \
  encoder(batch_x_transpose[e], batch_size, encoder_hidden)
  	encoder_outputs[ei] = encoder_output[0]
  
  # decoder part
  for d in range(max_seq_len_de):
  	decoder_output, decoder_hidden= \
  decoder(decoder_input, batch_size, decoder_hidden)
  
  	loss += criterion(decoder_output, batch_y_transpose[d])