Style Transfer Machine Learning Tests

Machine learning has made giant steps in past years thanks to the democratization of cloud computing infrastructure that allows it to train models on hundreds/thousands TPU at the same time. Mastering this technology becomes a mandatory qualification for any company willing to stay competitive and innovative.

Machine Learning Models

Machine learning splits up in different models categories, each of them being specialized into solving one type of problem. Most of this specialization is acquired through model’s architecture, dataset that could be so huge that they are sometimes referred as data lake’ and finally hyperparameters customization. One of the most available data for machine learning is pictures. Pictures machine learning models are so advanced that some people are turning their non visual datas like noise, mouse movement to images that could be ingested by those models to accomplish their classification tasks.

Today we will talk about style transfer machine learning algorithm. Style transfer is an algorithm that given two images will output a picture containing the content of one of the input while being stylized by the second one. Here is a couple of examples taken from the web to make this more visual.

Potential usage cases

Such technology could be applied in the cinema and video game industries in order to produce specific stylized textures, FX or simply post-process the whole movie to create a strong artistic identity looking like this oil painted feature film for a fraction of the cost.

It’s Play Time

Today I’ll share some pretty nice looking results of my personal experience using a toonifer style transfer tensorflow model on some VFX/CG trailers (to observe the temporally coherence noise) and finally on some randoms pictures.