Google has launched Tensor2Tensor, a library that will help researchers train deep learning models for use in its TensorFlow framework.<\/p>\n
“T2T facilitates the creation of state-of-the art models for a variety of ML applications, such as translation, parsing, image captioning, and more,” said Google.<\/p>\n
“This release also includes a library of datasets and models, including the best models from a few recent papers to help kick-start your own DL research.”<\/p>\n
Models available include:<\/p>\n
The following results of a\u00a0standard WMT English-German translation task using previous state-of-the-art models, compared to Tensor2Tensor, were provided by Google.<\/p>\n
Its Transformer and SliceNet outperformed GNMT and GNMT+MOE.<\/p>\n
| \n Translation Model<\/b><\/span><\/div>\n<\/td>\n \n | Training time<\/b><\/span><\/div>\n<\/td>\n \n | BLEU (difference from baseline)<\/b><\/span><\/div>\n<\/td>\n<\/tr>\n Transformer<\/a> (T2T)<\/td>\n | \n | 3 days on 8 GPUs<\/div>\n<\/td>\n \n | 28.4 (+7.8)<\/b><\/div>\n<\/td>\n<\/tr>\n SliceNet<\/a> (T2T)<\/td>\n | \n | 6 days on 32 GPUs<\/div>\n<\/td>\n \n | 26.1 (+5.5)<\/strong><\/div>\n<\/td>\n<\/tr>\n \n | GNMT + Mixture of Experts <\/a><\/div>\n<\/td>\n \n | 1 day on 64 GPUs<\/div>\n<\/td>\n \n | 26.0 (+5.4)<\/strong><\/div>\n<\/td>\n<\/tr>\n ConvS2S<\/a><\/td>\n | \n | 18 days on 1 GPU<\/div>\n<\/td>\n \n | 25.1 (+4.5)<\/strong><\/div>\n<\/td>\n<\/tr>\n GNMT<\/a><\/td>\n | \n | 1 day on 96 GPUs<\/div>\n<\/td>\n \n | 24.6 (+4.0)<\/strong><\/div>\n<\/td>\n<\/tr>\n \n | ByteNet<\/a><\/div>\n<\/td>\n \n | 8 days on 32 GPUs<\/div>\n<\/td>\n \n | 23.8 (+3.2)<\/strong><\/div>\n<\/td>\n<\/tr>\n MOSES<\/a> (phrase-based baseline)<\/td>\n | \n | |