Skip to Content

Department of Cognitive Science

Seminar Abstract

Latent variables for neural machine translation. (CLaS-CCD Research Colloquium Series)

Speaker : Mr Philip Schulz, Mathematics and Computer Science, University of Amsterdam, The Netherlands.
Date : 20th of December 2017, 10:00AM until 11:00AM
Location : E6A, Level 3, Room 357, Macquarie University.

    Neural machine translation (NMT) models have matured over the past years and delivered great improvements compared to traditional phrase-based systems. Recently, new architectures like the transformer network have been proposed that change the way in which features for target word predictions are constructed. The underlying statistical model stays unchanged, however: it is a simple conditional language model which assumes that translations are invariably produced by the same process. This is an unrealistic assumption since we know that the identity of the translator, the domain of the text and other factors have an influence on the style and lexical choice of the translation. In this talk I present ongoing research on latent variable models for NMT. The idea is that these latent variables should capture the variation that is present in the translation data and thus lead to better translations during decoding. These models are implemented in the variational auto encoding (VAE) framework and face certain challenges that are inherent to this framework (e.g. the model might learn to simply ignore the latent variables). I first review the VAE framework and then show how to apply it to NMT. I then point out the difficulties we experienced when implementing those models and report initial experimental result.

    Further Information

    Seminars

    Who is Visiting

    Contact Details

    Telephone: (02) 9850 9599
    Fax : (02) 9850 6059
    Email : cogsci@mq.edu.au
    Web : www.cogsci.mq.edu.au