Deep Learning and Fontli

25
Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve. DEEP LEARNING & FONTLI

Transcript of Deep Learning and Fontli

Page 1: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

DEEP LEARNING & FONTLI

Page 2: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

I would like to understand things better, but I don’t want to understand them perfectly.

- Douglas Hofstadter

Page 3: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

In Fluid Concepts and Creative Analogies, Douglas Hofstadter gives centre stage to analogy making as a foundation for intelligence

Hofstadter talks about derivative projects – one of which (“Letter Spirit”) is about analogy making in the typography space : the question being “if <word1> : <insert-stylized-word1> then <other-letters> : ?”.

In a sentence, if I show the system a particular stylized presentation of the letters “abcd”, how do you get it to construct other letters such as “wxyz” in the same style.

Page 4: Deep Learning and Fontli

Deep learning experiment with Fontli

Page 5: Deep Learning and Fontli

Fontli is a social network for typoholics. Capture typography around you and share it. Let the typetalk develop on Fontli.

http://www.fontli.com/
Page 6: Deep Learning and Fontli

OUR SEARCH FOR ANSWER …

Is it possible to take a image style and apply to another image?

Page 7: Deep Learning and Fontli

So we tried it with a bunch of images largely in three classes – type, patterns and semi-realistic pictures – with some surprisingly good and surprisingly crappy results.

TYPE PATTERNS SEMI-REALISTIC

Page 8: Deep Learning and Fontli

We wanted to try what “taking the style from an image” would do to a dead simple bland piece of text in black and

white – the “content”, shown below.

The image which was passed to neural-style as the “content” image in all the cases.

Page 9: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

DEEPCONVOLUTION

NEURALNETWORK

(DCNN)

The layers of a deep convolution neural network (DCNN) that are closer to the input tend to look at spatially local features

The layers deeper down tend to be concerned with higher level and even conceptual features

Page 10: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

The images were rendered using the 19-layer network which is the default in the neural-style tool

The amount of content has been kept small in these images deliberately – the proportion varying

between 5:1000 and 50:1000

Each image took about 10mins to generate on a g2.2xlarge instance on AWS

DCNN+

FONTLI

Page 11: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

THE RESULTS …

A top favourite. The swirly strokes of the pattern have been beautifully woven to form the letters amidst the chaos of flowy colours.

Page 12: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

THE RESULTS …

The tree pattern has lots of whirlies all over, which appears to be the reason the whirlieshave been used to express the letter shaping. Note that the “n” appears in bold and somewhat like the trunk of the tree.

Page 13: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

THE RESULTS …

The colours and texture of the mat pieces in the style image have been used pretty effectively to create the letters. Again, the “content” is set to low in this case. We’ve also lost some of the regularity of the whole pieces in the style image, which we might consider to be the “content” of the style image.

Page 14: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

THE RESULTS …

The colour palette and the texture of the colouring has transposed nicely on to the result image as expected.

Page 15: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

THE RESULTS …

This is interesting because the vertical strokes of the tree bark have been preserved more or less intact, while locally altering the lines to reveal the letters. The notches in the style image have been used to make the characters. See how the lines converge towards the dot of the “i”.

Page 16: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

THE RESULTS …

One of those exceptional cases where something interesting came out of using a typography image as the style. That said, I had no idea such an image would result –i.e. the result in unpredictable.

Page 17: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

THE RESULTS …

Another favourite. The strokes of the “kolam” pattern and the colourshave been used beautifully to shape the letters.

Page 18: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

THE RESULTS …

The thorny pattern seems to have been used as the “pencil” to draw out the letters this time.

Page 19: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

THE HEAD-SCRATCHERS …

Realistic photographs did absolutely nothing.

Also, since the ImageNet training set doesn’t attempt to teach typography to the neural net, we

also didn’t have much success trying to use the letter styles from an image and apply it to new text.

The results were mixed, with decent results some times, but head-scratchers at other times.

Page 20: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

THE HEAD-SCRATCHERS …

Using real-ishphotographs does absolutely nothing artistically interesting to the text! This turned out to be the crappiest of what we tried :)

Page 21: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

THE HEAD-SCRATCHERS …

We thought the sketchiness would transfer over to the lettering, but that didn’t happen. As noted, this area is a bit unpredictable. Maybe the sketching is quite far away from what we might find in photographs.

Page 22: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

LESSONS LEARNT

What we’ve seen in this quick experiment is that the net trained on real world photos does not quite have

the ability to read typography. This isn’t surprising since typography neither features strongly in the

collection nor have such images been labelled well. That said, we can indeed make some very interesting images that present lettering and we hope to include

this functionality in Fontli.

Page 23: Deep Learning and Fontli

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

Are you looking to solve complex problems using Deep Learning?Explore Deep Learning with Imaginea…

GET IN TOUCH WITH US >>

http://www.imaginea.com/contact-us/
Page 24: Deep Learning and Fontli

Pramati’s M&A’s of Leading products

Serving from 5 Global Locations

Over 200 Product Companies

Unique Products & Services

Agile Methodology

User-centric design

Open Source Contributions

Products built from conception-code-cash

Imaginea: Services Company with Product DNA

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.

Page 25: Deep Learning and Fontli

Disclaimer

This document may contain forward-looking statements concerning products and strategies. These statements are based on management's current expectations and actual results may differ materially from those projected, as a result of certain risks, uncertainties and assumptions, including but not limited to: the growth of the markets addressed by our products and our customers' products, the demand for and market acceptance of our products; our ability to successfully compete in the markets in which we do business; our ability to successfully address the cost structure of our offerings; the ability to develop and implement new technologies and to obtain protection for the related intellectual property; and our ability to realize financial and strategic benefits of past and future transactions. These forward-looking statements are made only as of the date indicated, and the company disclaims any obligation to update or revise the information contained in any forward-looking statements, whether as a result of new information, future events or otherwise.

All Trademarks and other registered marks belong to their respective owners.

Copyright © 2012-2015, Imaginea Technologies, Inc. and/or its affiliates. All rights reserved.

Credits

Images under Creative Commons Zero license.

Private and confidential. Copyright (C) 2016, Imaginea Technologies Inc. All rights reserve.


Multimodal Deep Learning (D4L4 Deep Learning for Speech and Language UPC 2017)

Multimodal Deep Learning (D4L4 Deep Learning for Speech and Language UPC 2017)

Deep Learning and HPC

Deep Learning and HPC

Deep Learning in real world @Deep Learning Tokyo

Deep Learning in real world @Deep Learning Tokyo

Artificial Intelligence, Machine Learning and Deep Learning

Artificial Intelligence, Machine Learning and Deep Learning

Deep Learning and Deep Fuzzy Networks Learning with Fuzzy Systems.pdf · Deep Learning •The first deep-learning model was proposed by Alexey Grigorevich Ivakhnenko and Lapa in 1965.

Deep Learning and Deep Fuzzy Networks Learning with Fuzzy Systems.pdf · Deep Learning •The first deep-learning model was proposed by Alexey Grigorevich Ivakhnenko and Lapa in 1965.

Learning Deep Learning

Learning Deep Learning

Lecture 3: Deep Learning Optimizations - GitHub Pagesuvadlc.github.io/lectures/nov2019/lecture3.pdf · Lecture 3: Deep Learning Optimizations Deep Learning @ UvA. UVA DEEP LEARNING

Lecture 3: Deep Learning Optimizations - GitHub Pagesuvadlc.github.io/lectures/nov2019/lecture3.pdf · Lecture 3: Deep Learning Optimizations Deep Learning @ UvA. UVA DEEP LEARNING

C++ and Deep Learning

C++ and Deep Learning

Deep Learning - Visualizing and Understanding ...vision.cse.psu.edu/people/chrisF/deep-learning/DL_pres.pdf · Deep Learning Visualizing and Understanding Convolutional Networks Christopher

Deep Learning - Visualizing and Understanding ...vision.cse.psu.edu/people/chrisF/deep-learning/DL_pres.pdf · Deep Learning Visualizing and Understanding Convolutional Networks Christopher

Lecture 3: Deeper into Deep Learning and Optimizations · 2019. 12. 14. · Lecture 3: Deeper into Deep Learning and Optimizations Deep Learning @ UvA. UVA DEEP LEARNING COURSE –EFSTRATIOS

Lecture 3: Deeper into Deep Learning and Optimizations · 2019. 12. 14. · Lecture 3: Deeper into Deep Learning and Optimizations Deep Learning @ UvA. UVA DEEP LEARNING COURSE –EFSTRATIOS

Globally Optimal Matrix Factorizations, Deep Learning and Beyondrvidal/talks/learning/Structured... · 2015-06-30 · Learning Deep Image Feature Hierarchies • Deep learning gives

Globally Optimal Matrix Factorizations, Deep Learning and Beyondrvidal/talks/learning/Structured... · 2015-06-30 · Learning Deep Image Feature Hierarchies • Deep learning gives

Deep Learning and Reinforcement Learning Workflows in A.I. · Deep Learning and Reinforcement Learning Workflows in A.I. Jon Cherrie Software Engineering Manager Deep Learning Toolbox.

Deep Learning and Reinforcement Learning Workflows in A.I. · Deep Learning and Reinforcement Learning Workflows in A.I. Jon Cherrie Software Engineering Manager Deep Learning Toolbox.

Deep Learning and Reinforcement Learning

Deep Learning and Reinforcement Learning

A Powerful, Flexible, and Intui5ve Deep Learning Frameworkon-demand.gputechconf.com/gtc/...hido-chainer-deep-learning-framework.pdf · A Powerful, Flexible, and Intui5ve Deep Learning

A Powerful, Flexible, and Intui5ve Deep Learning Frameworkon-demand.gputechconf.com/gtc/...hido-chainer-deep-learning-framework.pdf · A Powerful, Flexible, and Intui5ve Deep Learning

Scala and Deep Learning

Scala and Deep Learning

Demystifying Deep Learning - MathWorks · Demystifying Deep Learning. 2 What’s Deep Learning and why should I care? A practical approach to Deep Learning (for images) –Transfer

Demystifying Deep Learning - MathWorks · Demystifying Deep Learning. 2 What’s Deep Learning and why should I care? A practical approach to Deep Learning (for images) –Transfer

(Deep Learning and 3D vision) · 23 What is Deep Learning ? Deep learning performs end-end learning by learning features, representations and tasks directly from images, text and

(Deep Learning and 3D vision) · 23 What is Deep Learning ? Deep learning performs end-end learning by learning features, representations and tasks directly from images, text and

Angular and Deep Learning

Angular and Deep Learning

Introduction to Machine Learning and Deep Learning

Introduction to Machine Learning and Deep Learning

Deep Learning for Generic Object Detection: A Survey · deep learning, probabilistic models, autoencoders, manifold learning, and deep networks 17 Deep learning LeCun et al. (2015)

Deep Learning for Generic Object Detection: A Survey · deep learning, probabilistic models, autoencoders, manifold learning, and deep networks 17 Deep learning LeCun et al. (2015)