Thursday, 15 March 2018 08:45

Microsoft team uses AI to match human translation Featured


Researchers at Microsoft claim to have created the first machine translation system that is able to translate news articles from Chinese to English with the same level of accuracy as a human.

Personnel from the company's Asia and US labs said their system achieved human parity using a commonly used test set of news stories called newstest2017 developed by industry and academic partners.

External bilingual human evaluators compared the results achieved by the Microsoft researchers to two translations produced independently by humans.

One method used by the researchers is known as dual learning. Each time a sentence was translated from Chinese to English, it was sent back through the system to change it from English to Chinese.

Another method is called deliberation networks and is similar to the practice of editing and revising that writers do repeatedly in order to improve their own efforts.

ai chinese

Xuedong Huang, technical fellow in charge of Microsoft’s speech, natural language and machine translation efforts.

Xuedong Huang, a technical fellow in charge of Microsoft’s speech, natural language and machine translation efforts, said: "Hitting human parity in a machine translation task is a dream that all of us have had. We just didn’t realise we’d be able to hit it so soon.”

Arul Menezes, partner research manager of Microsoft’s machine translation team, said the researchers had tried to prove that their systems could achieve the same level of translation using a language pair — Chinese and English — for which there is a lot of data, on a test set that includes the more commonplace vocabulary of general interest news stories.

“Given the best-case situation as far as data and availability of resources goes, we wanted to find out if we could actually match the performance of a professional human translator,” he said.

“Much of our research is really inspired by how we humans do things,” said Tie-Yan Liu, a principal research manager with Microsoft Research Asia in Beijing. Liu leads a team that worked on this project.


Arul Menezes, partner research manager of Microsoft’s machine translation team.

Ming Zhou, assistant managing director of Microsoft Research Asia and head of a natural language processing group that worked on the project, said the group had also developed two new techniques that would improve the accuracy of their translations.

One technique, called joint training, is used to iteratively boost the English-to-Chinese and Chinese-to-English translation systems. It translates new English sentences into Chinese in order to obtain new sentence pairs which are then used to augment the training dataset used for the opposite direction – Chinese to English.

A second technique, agreement regularisation, generates a translation by reading in different direction in order to obtain a consensus translation.

Zhou said he expects these methods and techniques to be useful for improving machine translation in other languages and situations as well. He said they also could be used to make other AI breakthroughs beyond translation.

“This is an area where machine translation research can apply to the whole field of AI research,” he said.

“Machine translation is much more complex than a pure pattern recognition task,” Zhou said. “People can use different words to express the exact same thing, but you cannot necessarily say which one is better.”

Photos: courtesy Microsoft

Subscribe to ITWIRE UPDATE Newsletter here

Now’s the Time for 400G Migration

The optical fibre community is anxiously awaiting the benefits that 400G capacity per wavelength will bring to existing and future fibre optic networks.

Nearly every business wants to leverage the latest in digital offerings to remain competitive in their respective markets and to provide support for fast and ever-increasing demands for data capacity. 400G is the answer.

Initial challenges are associated with supporting such project and upgrades to fulfil the promise of higher-capacity transport.

The foundation of optical networking infrastructure includes coherent optical transceivers and digital signal processing (DSP), mux/demux, ROADM, and optical amplifiers, all of which must be able to support 400G capacity.

With today’s proprietary power-hungry and high cost transceivers and DSP, how is migration to 400G networks going to be a viable option?

PacketLight's next-generation standardised solutions may be the answer. Click below to read the full article.


WEBINAR PROMOTION ON ITWIRE: It's all about webinars

These days our customers Advertising & Marketing campaigns are mainly focussed on webinars.

If you wish to promote a Webinar we recommend at least a 2 week campaign prior to your event.

The iTWire campaign will include extensive adverts on our News Site and prominent Newsletter promotion and Promotional News & Editorial.

This coupled with the new capabilities 5G brings opens up huge opportunities for both network operators and enterprise organisations.

We have a Webinar Business Booster Pack and other supportive programs.

We look forward to discussing your campaign goals with you.


Sam Varghese

website statistics

Sam Varghese has been writing for iTWire since 2006, a year after the site came into existence. For nearly a decade thereafter, he wrote mostly about free and open source software, based on his own use of this genre of software. Since May 2016, he has been writing across many areas of technology. He has been a journalist for nearly 40 years in India (Indian Express and Deccan Herald), the UAE (Khaleej Times) and Australia (Daily Commercial News (now defunct) and The Age). His personal blog is titled Irregular Expression.

Share News tips for the iTWire Journalists? Your tip will be anonymous




Guest Opinion

Guest Interviews

Guest Reviews

Guest Research

Guest Research & Case Studies

Channel News