written by
David El Malih

Deformable Image Registration using Deep Learning

Machine Learning Computer Vision 3 min read
Source: Radio Canada

This post explains how to perform deformable image registration using deep neural networks.

What is Image Registration?

Image registration is the process of transforming different sets of data into the same coordinate system. The spatial relationships between these images can be rigid (translations and rotations), affine (shears for example), homographies, or complex deformable models.

For rigid image registration, I recommend that you read the article written by J. Joslove and E. Kamoun. In the following, let us focus on deformable image registration, which is the most general use case.

Deformable registration of a source image to a target image is finding a pixel-wise displacement field such that, when applying it to the source image, it matches the target image.

deformable registration
Illustration of deformable registration of a donkey into a cat

How can I do deformable image registration?

There are several ways to do deformable image registration: optimal transport, MRFs, and other optimization-based techniques. In this article, I would like to describe the most recent technique and, surely, the simplest to implement: The Deep Learning-based one.

Here is a schema representation of the training process of a deep learning-based pipeline for deformable image registration:

Schema representing the training process of a deep registration pipeline

Step 1: Outputting the Displacement Field

Give a (Source, Target) pair of images to the registration network - which consists of a CNN (Convolutional Neural Network), often a UNet.

This CNN takes the pair of images as input and outputs a Displacement Field. The displacement field is simply a tensor that maps every single pixel (x, y) in the source image to a displacement vector (Δx, Δy).

Step 2: Computing the Warped Image

Once you have the displacement field, you can apply it to the Source image.

For that, for each pixel in the source image, you compute the target pixel (x’, y’) = (x+Δx, y+Δy). Then, you sample the source image with the target coordinates to create the warped image. This step is done in the Warping layer.

Step 3: Computing Similarity Warped ↔ Target

Now, we have to measure how good our registration is in order to give the network an optimization goal (a Loss Function).

There are 2 main functions that can measure the similarity between images:

  • The MSE (Mean Squared Error) Loss:
    The MSE is a measure of the similarity of two images in a pixel-wise way. If the two images are strictly the same, the MSE between the two images will be equal to 0.
    The MSE between two images I and J is defined by:
MSE value of two images I and J
  • The NCC (Normalized Cross-Correlation) Loss:
    The NCC is a measure of the similarity of two series as a function of the displacement of one relative to the other. It is similar in nature to the convolution of two functions.
    If W is a constant kernel of size K, the NCC between two images I and J is defined by:
NCC value of two images I and J

Step 4: Updating the weights

Once the similarity is computed, you can use backpropagation to update the weights of the CNN.

Finally, loop over your dataset to train the deformable image registration network.

Going Further

This article presents the key principles of deformable registration. I hope you enjoyed it! Image registration is a wide field that has led to the writing of several research papers.

If you want to go further in understanding and mastering deformable image registration, these links may be useful:

If you want to learn about rigid registration using OpenCV, please check out our article. Also, please check out and subscribe to our blog on Computer Vision and Data Engineering.

Are you looking for Image Recognition Experts? Don't hesitate to contact us!

Thanks to Etienne Bennequin and Bastien Ponchon.

Data Science Artificial intelligence computer vision Registration CNN UNet Optimal Transport Deep Learning
Get our Battle-Tested Tutorials Delivered Straight Into Your Inbox Every Week
Sign up for our newsletter