Patrick McGuire: Montages with OpenCV

Monday, May 29, 2017

Montages with OpenCV

Today’s blog post is inspired by an email I received from PyImageSearch reader, Brian.

Brian asks:

Hi Adrian,

I’m really enjoying the PyImageSearch blog. I found your site a few days ago and I’ve been hooked on your tutorials ever since.

I followed your tutorial on building an image search engine, but instead of displaying the result images one-by-one (like you did), I want to display the top-20 results in a montage.

Is there a way to do with OpenCV?

Thanks,

Brian

Great question Brian, thanks for asking.

One of my favorite aspects of running the PyImageSearch blog is being able to chat with you, the reader, and discover the projects you’re working on.

It’s especially exciting when I can take questions or comments and turn them into blog posts — that way the entire PyImageSearch community is able to benefit from the answer.

Today we will learn how to build montages of images using OpenCV and the imutils package. A big thank you to Kyle Hounslow who contributed the

build_montages

function to imutils.

To learn more about building an image montage with OpenCV, just keep reading.

Looking for the source code to this post?
Jump right to the downloads section.

Montages with OpenCV

There are four primary pieces to today’s blog post.

In the first part, we’ll learn how to build a list of image paths from an image dataset residing on disk.

From there, we’ll use the

build_montages

function to take this list of images and create the actual montage.

Next, we’ll display the montage to our screen.

Finally, I’ll provide an example of using montages to display images with OpenCV.

To download the source code + example images to this blog post, be sure to use the “Downloads” section below.

Creating a montage with OpenCV

To get started, open up a new file, name it

montage_example.py

, and insert the following code:

# import the necessary packages
from imutils import build_montages
from imutils import paths
import argparse
import random
import cv2

Lines 2-6 import our required Python packages. Notice how

build_montages

is imported from the imutils package.

If you do not have

imutils

installed on your system (v0.4.3 as of this writing), then make sure you install/upgrade it via

pip

$ pip install --upgrade imutils

Note: If you are using Python virtual environments (as all of my OpenCV install tutorials do), make sure you use the

workon

command to access your virtual environment first and then install/upgrade

imutils

From there, we can parse our command line arguments:

# import the necessary packages
from imutils import build_montages
from imutils import paths
import argparse
import random
import cv2

# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-i", "--images", required=True,
        help="path to input directory of images")
ap.add_argument("-s", "--sample", type=int, default=21,
        help="# of images to sample")
args = vars(ap.parse_args())

Our script requires one command line argument, followed by a second optional one, each detailed below:

```
--images
```
: The path to your directory containing the images you want to build a montage out of.
```
--samples
```
: An optional command line argument that specifies the number of images to sample (we default this value to
```
21
```
total images).

Next, we can use the

--images

path to randomly select some input images:

# import the necessary packages
from imutils import build_montages
from imutils import paths
import argparse
import random
import cv2

# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-i", "--images", required=True,
        help="path to input directory of images")
ap.add_argument("-s", "--sample", type=int, default=21,
        help="# of images to sample")
args = vars(ap.parse_args())

# grab the paths to the images, then randomly select a sample of
# them
imagePaths = list(paths.list_images(args["images"]))
random.shuffle(imagePaths)
imagePaths = imagePaths[:args["sample"]]

To obtain a listing of all image paths inside the

--images

directory, we make a call to the

list_images

function (Line 18).

For the purpose of this exercise we randomly shuffle the image paths on Line 19, followed by taking a sample of these images to display to our screen (Line 20). The set of

imagePaths

returned by this sampling will be used to build our montage.

For your own applications you likely will not have to bother with randomly shuffling and selecting a set of image paths — you will already have your image paths.

In the context of Brian’s original question, he is looking to display the results of his image search engine.

The results therefore contain his image paths.

Again, keep in mind that we are simply demonstrating how to build a montage with OpenCV — how you actually use this example is entirely up to you.

Given our

imagePaths

, we are ready to build the montage:

# import the necessary packages
from imutils import build_montages
from imutils import paths
import argparse
import random
import cv2

# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-i", "--images", required=True,
        help="path to input directory of images")
ap.add_argument("-s", "--sample", type=int, default=21,
        help="# of images to sample")
args = vars(ap.parse_args())

# grab the paths to the images, then randomly select a sample of
# them
imagePaths = list(paths.list_images(args["images"]))
random.shuffle(imagePaths)
imagePaths = imagePaths[:args["sample"]]

# initialize the list of images
images = []

# loop over the list of image paths
for imagePath in imagePaths:
        # load the image and update the list of images
        image = cv2.imread(imagePath)
        images.append(image)

# construct the montages for the images
montages = build_montages(images, (128, 196), (7, 3))

On Line 23 we initialize our list of

images

We then loop through the

imagePaths

on Lines 26-29, loading each

image

from disk, and then appending the

image

to our

images

list.

To actually construct the montage, we make a call to the

build_montages

function on Line 32 — this is where all of the heavy lifting is done. If you’re curious about the internals of the

build_montages

method and what is going on under the hood, be sure to check out the source code implementation on GitHub.

The

build_montages

function requires three arguments:

```
image_list
```
: This parameter is a list of images loaded via OpenCV. In our case, we supply the
```
images
```
list built on Lines 26-29.
```
image_shape
```
: A tuple containing the width and height of each image in the montage. Here we indicate that all images in the montage will be resized to 129 x 196. Resizing every image in the montage to a fixed size is a requirement so we can properly allocate memory in the resulting NumPy array. Note: Empty space in the montage will be filled with black pixels.
```
montage_shape
```
: A second tuple, this one specifying the number of columns and rows in the montage. Here we indicate that our montage will have 7 columns (7 images wide) and 3 rows (3 images tall).

The

build_montages

method returns a list of montage images in NumPy array format.

If there are more images in the

images

list than the

montage_shape

can hold, a new montage is created for the extra

images

. This process is repeated until all

images

have been added to a montage. This process is identical to displaying search results over multiple pages.

Our final code block handles displaying the

montages

to our screen:

# import the necessary packages
from imutils import build_montages
from imutils import paths
import argparse
import random
import cv2

# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-i", "--images", required=True,
        help="path to input directory of images")
ap.add_argument("-s", "--sample", type=int, default=21,
        help="# of images to sample")
args = vars(ap.parse_args())

# grab the paths to the images, then randomly select a sample of
# them
imagePaths = list(paths.list_images(args["images"]))
random.shuffle(imagePaths)
imagePaths = imagePaths[:args["sample"]]

# initialize the list of images
images = []

# loop over the list of image paths
for imagePath in imagePaths:
        # load the image and update the list of images
        image = cv2.imread(imagePath)
        images.append(image)

# construct the montages for the images
montages = build_montages(images, (128, 196), (7, 3))

# loop over the montages and display each of them
for montage in montages:
        cv2.imshow("Montage", montage)
        cv2.waitKey(0)

On Line 35 we loop over each of the

montages

(again, similar to displaying N number of (faux) “search results” on a page).

Lines 36 and 37 then display the current

montage

to our screen. The

cv2.waitKey

call pauses execution of our script until we select the currently active window and press any key on our keyboard. This will cause the

for

loop to advance.

Once we reach the end of the

montages

list, the script exits.

Displaying the Montage

Approximately two years ago I was involved in a computer vision project that required me to build a simple image fashion search engine. To accomplish this, I built a simple web crawler to spider Nordstrom.com and download all the product images and associated meta data.

We are are going to use a tiny sample of this data today when demoing the

build_montages

function.

Once you’ve used the “Downloads” section below to download the source code + example images, you can execute the following command to see the results:

$ python montage_example.py --images nordstrom_sample

After executing the script you should see output similar to the following:

Figure 1: Building a montage with OpenCV and Python.

Note: The exact images that you see in the montage will vary from mine since we are randomly sampling from the input directory.

As we can see in Figure 1 above, we have three rows, each row containing seven images. Each image in the montage has been resized to a fixed size of 128 x 196 pixels.

In the context of Brian’s question at the top of this blog post, this montage could be search results from his image search engine algorithm.

As a second example, let’s increase the

--sample

such that we create multiple montages since all images will not fit in a three row, seven column format:

$ python montage_example.py --images nordstrom_sample --sample 33

Since 3 x 7 = 21, we know that sampling 33 images cannot possibly fit into a 21 image montage.

Luckily for us, the

build_montages

function realizes that there are too many images to fit into a single montage and thus creates two montages.

The first montage can be seen below with all 21 spaces in the montage occupied:

Figure 2: The first montage generated with OpenCV is completely filled.

The second montage holds the remaining 12 images that could not fit in the first montage:

Figure 3: The second montage displays images that could not fit in the first montage.

Notice how empty spaces in the montage are filled with black pixels.

Summary

In today’s blog post I demonstrated how to build a montage with OpenCV and Python to visualize a collection of images. This is a handy tool you can use in your own image processing projects, such as in Brian’s image search engine project detailed at the top of this blog post.

I would also like to take a second and give a big shoutout to Kyle Hounslow who contributed the

build_montages

function to the imutils package — thanks again Kyle!

In next week’s blog post I’ll demonstrate how to use this montage functionality in an actual application where we sort images in a dataset according to how “colorful” they are.

To be notified when this next blog post goes live, be sure to enter your email address in the form below.