Reading 28: A Neural Algorithm of Artistic Style

Citation

Gatys, Leon A., Alexander S. Ecker, and Matthias Bethge. "A neural algorithm of artistic style." arXiv preprint arXiv:1508.06576 (2015).

PDF

Summary

This paper presented a way to separate out representation of content and style of an image by using feature map from inner layers of a convolutional neural network. These representations are generated by starting from a white noise image and minimizing loss functions of style and content separately.  Loss functions for style and content are described in the paper. The CNN used was a pre-trained model on object detection (VGG, by Visual Geometry Group, University of Oxford).

Then a new image can be synthesized by combining the content representation from an input image and style representation from a famous artwork using a new loss function combining loss functions of style and content with two weight parameters. The resulting image has qualities of both the images which can be controlled by the weight parameters.

Discussion

This paper presented a novel way to produce artistic images from normal images. The major ideas in the paper were the loss functions for style and content of image (not understood completely). Synthesizing images from white noise by using a new loss function on a pre-trained CNN is the underlying principle used in this paper. The way the application works is:

1) Generate content representation of an input image.

2) Generate style representation of a famous artwork.

3) Generate new image by using weighted loss with content of 1) and style of 2).