Generating art with guided deep dreaming.

One of the main benefits of the bat-country Python package for deep dreaming and visualization is its ease of use, extensibility, and customization.

And let me tell you, that customization really came in handy last Friday when the Google Research team released an update to their deep dream work, demonstrating a method to “guide” your input images to visualize the features of a target image.

The results were quite impressive — so I decided to port the functionality to bat-country.

Truth be told, it only took ~20 minutes from start-to-finish to get the code together. I honestly spent more time running the Python scripts to gather example images and updating the documentation than I did updating the codebase.

The secret to this quick turnaround is the extensibility of the

BatCountry

  class where nearly every function and every method can be overridden and extended.

Want to change how each image is pre-processed or post-processed? No problem. Define your own custom processor and pass it in. Want to change the objective function? Again, just define your own objective and you’re good to go.

In fact, defining your own custom objective function is the the exact route I took when extending 

bat-country

 . I simply defined a new objective function, allowing the step function to be further customized, and we’re done!

In the remainder of this blog post we’ll play around with the new bat-country update to perform guided dreaming — and even use it to generate our own art using guided deep dreaming!

Guided deep dreaming

Last Friday the Google Research team posted an update to their deep dream working demonstrating it was possible to guide your dreaming process by supplying a seed image. This method passes your input image through the network in a similar manner, but this time using your seed image to guide and influence the output.

Using

bat-country

 , it’s just as easy to perform guided dreaming as deep dreaming. Here’s some quick sample code:

# import the necessary packages
from batcountry import BatCountry
from PIL import Image
import numpy as np
import argparse

# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-b", "--base-model", required=True, help="base model path")
ap.add_argument("-l", "--layer", type=str, default="inception_4c/output",
        help="layer of CNN to use")
ap.add_argument("-i", "--image", required=True, help="path to base image")
ap.add_argument("-g", "--guide-image", required=True, help="path to guide image")
ap.add_argument("-o", "--output", required=True, help="path to output image")
args = ap.parse_args()

# we can't stop here...
bc = BatCountry(args.base_model)
features = bc.prepare_guide(Image.open(args.guide_image), end=args.layer)
image = bc.dream(np.float32(Image.open(args.image)), end=args.layer,
        iter_n=20, objective_fn=BatCountry.guided_objective,
        objective_features=features)
bc.cleanup()

# write the output image to file
result = Image.fromarray(np.uint8(image))
result.save(args.output)

If you don’t already have

bat-country

  installed, either pull down the code from the GitHub repo or use pip to install it on your system: 

pip install bat-country

 or 

pip install --upgrade bat-country

 . You’ll also need a working installation of Caffe.

What’s nice about this approach is that we can “guide” what the output image looks like. Here I use a seed image of Vincent van Gogh’s Starry Night and apply to an image of clouds:

Figure 1: An example of applying guided dreaming using Starry Night and a cloud image.

As you can see, the output cloud image after applying guided dreaming appears to mimic many of the brush strokes of Van Gogh’s painting.

Which got me thinking — what would happen if I took some well-known paintings from extremely famous artists such as Andy Warhol, MC Escher, Pablo Picasso, Jackson Pollock, and Vincent van Gogh and used them as inputs and guides to each other?

What would the results look like? Would the artistic style of each painting be transferred to the other?

In order to test this out, I collected images of the following pieces of work:

• Andy Warhol – Gun
• Andy Warhol – Marilyn Monroe
• Jackson Pollock – Energy Made Visibile
• MC Escher – Sky and Water I
• MC Escher – Mosaic II
• Pablo Picasso – Guernica
• Vincent van Gogh – Wheat Field with Cypresses

And then I passed each of them through the

demo_guided.py

  script detailed above.

Generating art with deep dreaming

Overall, the results look really fantastic. I’m especially pleased with how Wheat Field with Cypresses and Guernica turned out when used as input images and the others paintings as guided images.

Below you can find some of my favorite images:

Vincent van Gogh – Wheat Field with Cypresses

Figure 2: Vincent van Gogh’s Wheat Field with Cypresses guided using Warhol’s Marilyn Monroe. Notice how eyes are now present in the sky.

Figure 3: Vincent van Gogh’s Wheat Field with Cypresses guided using Jackson Pollock’s Energy Made Visible.

Figure 4: Vincent van Gogh’s Wheat Field with Cypresses guided using Picasso’s Guernica.

Pablo Picasso – Guernica

Figure 5: Pablo Picasso’s Guernica guided with MC Escher’s Mosaic II.

Figure 6: Pablo Picasso’s Guernica guided with Jackson Pollock’s Energy Made Visible.

Figure 7: Pablo Picasso’s Guernica guided with Andy Warhol’s Marilyn Monroe.

MC Escher – Mosaic II

Figure 8: MC Esher’s Mosaic II guided by Jackson Pollock’s Energy Made Visible.

Figure 9: MC Esher’s Mosaic II guided by Vincent van Gogh’s Wheat Field with Cypresses.

Andy Warhol – Marilyn Monroe

Figure 10: Andy Warhol’s Marilyn Monroe guided using MC Escher’s Sky and Water I.

Figure 11: Andy Warhol’s Marilyn Monroe guided using Picasso’s Guernica.

Summary

In this blog post I reviewed the updates to the bat-country package will allow for guided deep dreaming by supplying two images: an input image that will be passed through the network, and a seed image that the network will use to “guide” the output.

I then took the updated code and used it to generate art from famous works, such as Jackson Pollock’s Energy Made Visible, Pablo Picasso’s Guernica, and Vincent van Gogh’s Wheat Field with Cypresses.

Definitely consider installing the

bat-country

  package on your system and giving deep dreaming a try! It’s strangely addictive (and not to mention, a lot of fun) to play around and generate your own images.

Finally, if you’re interested in deep learning, deep dreaming, and computer vision, think about signing up for the PyImageSearch Newsletter by entering your email address in the form below. I send out regular updates on the PyImageSearch blog, each filled with actionable, real-world computer vision projects.

See you next time!

The post Generating art with guided deep dreaming. appeared first on PyImageSearch.

from PyImageSearch http://ift.tt/1HXwNOY
via IFTTT