Previous talks at the SCCS Colloquium

Yi-Han Hsieh: Second Order training for Natural Language Processing using Newton-CG Optimizer

SCCS Colloquium |


We present a Hessian-based second-order optimizer called Newton-CG to solve a Portuguese to English Machine Translation (MT) Task in the Natural Language Processing (NLP) field. We mainly focus on the comparison of the performance between Newton-CG and first-order optimizers Adam and Stochastic gradient descent(SGD) that both are widely used in deep learning tasks. In this task, We use the most dominant NMT model called Transformer.

Master's thesis talk. Yi-Han is advised by Felix Dietrich and Severin Reiz.