Facebook AI can now use just one word to mimic text style from images

Facebook has announced their new AI project TextStyleBrush, a software capable of copying the style of handwritten or printed text in an image using only one word. Users can utilize this model to alter and replace text in photos.

TextStyleBrush presents a step up from previous AI systems that can only copy text from photos using well-defined tasks and multiple examples. While this system is still in the research stage, scientists hope this new AI can assist with creative personalized captions and messaging as well as photo-realistic linguistic translation in augmented reality (AR).

So far, AI-generated images have already been rapidly progressing, from visually reconstructing historical events to rendering a photo to look like a Renoir or Van Gogh painting. Now, with a software that can register all types of calligraphy and typography, TextStyleBrush also promises to interpret various kinds of rotations and transformations of text, such as curved characters and natural deformations between paper and pen.

This software operates in a similar manner to style brush tools in word processors, only adapted for text aesthetics in images. Perhaps most importantly, this tool is self-supervised, making it the most independent visual text rendering program yet. One such example could be TextStyleBrush replacing handwritten produce signs at a grocery stand. The original content from which the program learns is known as the source sample, while the associated style used by the AI according to that sample is called the source style.

In contrast to previous textual interpretation methods that use defined parameters such as target style supervision and typeface, TextStyleBrush considers all elements of an image, including both text and other visuals. The program then uses this overall text box appearance as a reference point for all related images, shedding the need to retrain every time the software encounters a similar example.

Notwithstanding, Facebook remains aware of the potential to use this type of technology for malicious activity, such as deepfake text attacks. For mitigating such attacks, Facebook will share research benchmarks with the Deepfake Detection Challenge data set to contribute to the broader knowledgebase of deepfakes by adding fake text to existing studies on fake faces.