Google Translate: an AR ambassador

Real time translation with Augmented Reality

The smartphone application of google Translate is now capable of translating text directly from the camera.
This new capability greatly extends the scope of the application in the real world, where it can be used

The main improvement with respect to pure text translation is that now the user does not need to copy the whole text into the application, the translation is immediate and gets superimposed to the original text. This greatly reduces the effort required, making the user more prone to use it in everyday life, for instance to translate the menu of a foreign restaurant directly at the table, or read complex road signs while travelling.

The application is surprisingly realistic.
The translated text is superimposed to the original in a perfect way, and most often is the translation itself which is not realistic, rather than the representation.

Looking at the picture, the app was tested on the back of an italian paperback.
The most amazing piece is probably the title, which seems to be real, both from the syntactic and the visual point of view.
The same cannot be said for the paragraph. The user may be able to grasp the main concept, but the translation does not resemble even closely a proper English text.
The superimposition has been successful in that letters seems to have been actually printed on the paper and not digitally added, but almost each word is of a different size and under different perspective. Moreover, the system is extremely sensitive, and every movement of the camera changes the translation quite a bit.


The technology then has a lot of space for improvements. It makes sense however, to imagine application like this to reach augmented reality (AR) devices, such as glasses or lenses, that could provide stable real-time translation without any user input.

However, there are risks involved.
Once the translation is immediate, realistic and perfectly superimposed to the original text, the user may no longer be aware the text was changed by the device at all.
The loss of awareness of the user poses serious risks to freedom and culture.
The fact that something is written in some languages may induce the user to behave in different manners than if it was written in other languages.


Moreover, such a real-time translation, possibly also on audio, may foster the loss of multiculturalism, flattening everything to the level of our own language.
It is no secret that almost every culture resolves around one language, and most, if not all, the literature work worldwide is strictly dependent on the language it is written. Reading Dante’s Inferno in English is not the same as reading it in Italian, and vice versa for Shakespeare’s works.

Regardless of that, the fundamental issue with this kind of application is the translation itself.
The company who operates the translation can have a huge influence in political and economic choices by steering people’s opinion providing biased information when translating.

Overall, the Google Translate app is the first step towards moving translation into augmented reality.
Although there are still some technological improvements to be made, it is safe to say that the biggest challenge it faces is not technology, but rather ethics, keeping in mind that the final aim should be to improve humans and not computers.

Sept. 15th, 2017


Note: This article was written as an homework for course CS 491 – Virtual and Augmented Reality, held by professor Johnson at UIC.