Back in the Sixties, one of the hottest toys in history swept America. It was called Etch-A-Sketch, and its popularity was based on a now-laughably simple feature. It was a handheld small-laptop-sized device that allowed users to create crude images by turning two control knobs that drew horizontal, vertical and diagonal lines composed of aluminum particles sealed in a plastic case. It allowed experienced artists to compose simple and sometimes recognizable portraits. And it allowed inexperienced wannabe artists who could barely draw stick-figure characters to feel like masters of the genre by generating what, frankly, still looked pretty much like mush. But Etch-A-Sketch was fun, and it went on to sell 100 million units to this day.
Six decades later, researchers at the Chinese Academy of Sciences and City University of Hong Kong have come up with an invention that actually does what so many wishful enthusiasts imagined Etch-A-Sketch did all those years ago.
DeepFaceDrawing allows users to create stunningly lifelike portraits by inputting loose, non-professional, roughly drawn sketches. It requires no artistic skills and no programming experience.
The program relies on machine learning to fill in the spaces between basic components such as eyes, nose and mouth. A massive database of facial features provides detailed fill information and probability formulas help determine the likelihood of various details to be inserted.
There have been recent technologies that offered users the ability to create images from rudimentary drawings, such as pix2pixHD, Lines2FacePhoto and iSketchNFill. A comparison of DeepFaceDrawing to these programs reveals sometimes frightening results by the other programs. The comparisons show DeepFaceDrawing standing head and shoulders above the other sketch programs.
The researchers, led by Shu-Yu Chen, explained the difference between the programs in a report recently posted online:
“Recent deep image-to-image translation techniques allow fast generation of face images from freehand sketches. However, existing solutions tend to overfit to sketches, thus requiring professional sketches or even edge maps as input. To address this issue, our key idea is to implicitly model the shape space of plausible face images and synthesize a face image in this space to approximate an input sketch.”
“Our method essentially uses input sketches as soft constraints,” the report states, “and is thus able to produce high-quality face images even from rough and/or incomplete sketches.”
The team plans to demonstrate DeepFaceDrawing at this year’s SIGGRAPH July conference. The group, founded in 1969, holds annual conferences on the latest developments in computer graphics. This year’s event will be a virtual one, due to COVID-19.
DeepFaceDrawing has obvious entertainment appeal. But it holds great potential for use in law enforcement where fast, accurate sketches of suspects are invaluable. It holds potential for use in facial recognition apps, medical procedures and educational training as well.
The team said they will soon release their code.
This research follows a report last week that Turkish researchers at the Department of Computer Engineering of Istanbul Technical University used deep neural networks to accurately create people’s faces based solely on images of their ears.