ERAN ASA
Electronics Engineer
Video Stabilization
This Project is my final project for BSc in Electronics
A detailed description can be found here (in Hebrew) and see Publication (down)
Here’s an overview of the algorithm:
In order to perform Image Stabilization, a geometric transformation is computed which maps the coordinates of one frame to the coordinates of another frame.
There are several types of geometric transformations, here we use the Homography transformation (Projective transforamtion).
The Homography transformation preserves straight lines, map lines to lines (meaning points that are collinear before the transformation remain collinear, though not parallel like in affine transformation), and has eight degrees of freedom.
The Algorithm based on Feature Point Matching, here is a flowchart of the algorithm:
Here is the input video - Unstabilized Video (The images and videos are courtesy of the Israel National Police, Technology Unit.)
we show the flow on this video.
Now, we will go over the stages and show detailed example, we will take reference image and different image in the video sequence and warp(stabilize) the different image to the reference image.
Corner Point Detection
The First step is to compute feature point, here we use the Harris Corner Point.
Right - the Frame Reference with the Harris Corner Point Detection (red-plus +), ~100 Points.
Left - Frame #100 of the video sequence.
Correlation
Each corner point from one frame correlated with corner points in the other frame.(in specific radius - for reduce in computational)
A matching is considered if the local correlation (local template/window around each corner point) score is high.
There are several ways to perform this measure of similarity between the local templates: SAD - Sum of Absolute Differences,
SSD - Sum of Squared Differences, NCC - Normalized Cross Correlation. we used the NCC.
In the figure we see the output of the correlation step:
Red lines are the result of Correlation step - Putative Matching - 71 points
RANSAC - RANdom SAmpling And Consensus
RANSAC is an iterative algorithm to estimate parameters of a mathematical model (in our case the Homograhy matrix) from a set of observed data (in our case the putative matching points) which contains outliers.
This Algorithm is robust if there is a little outliers relative to the inliers, can see in the figure below that almost all matching points are inliers but there is a few that are outliers.
Green lines are the inliers of the RANSAC algorithm and Red lines are the outliers
64 inliers
7 outliers
Homography - Estimate Homography Matrix
Now, we can estimate the homography matrix H by using all of the correspondence points labeled as inliers in the iteration that yielded the optimal homography.
This is the Homography Matrix we got: (close to be Euclidean Transformation with little rotation and large translation)
H =
0.9913 -0.0367 105.8630
0.0191 1.0179 321.9432
-0.0000 -0.0000 1.0068
Transformation
Once the homography matrix H is computed, it can be used to transform one image into the frame of reference of the other
image using a simple matrix multiplication.
the figure below shows how frame #100 was transformed to match the frame of reference :
The Final Result:
Video Mosaicing
Video Mosaicing is the process of stitching video frames together to form a comprehensive view of the scene,
here is result of Video Mosaicing of the same video from above:
Publication
Edmond Chalom, Eran Asa, and Elior Biton, "Measuring Image Similarity: An Overview of Some Useful Applications", IEEE Instrumentation and Measurement Magazine, Vol 16, No. 1, February 2013: 24-28 (pdf)