Deinterlacing

Introduction

In this guide, we're going to learn about deinterlacing and, more important, what solutions are there and which one are better or more suitable to our needs.

I'm sure you have all seen a movie on the computer or at the TV. You have surely noticed it's two main parts: the video and the audio.

The video part is built out of pictures that appear on the screen, with a precise frequency. This frequency of the pictures, which I will call from now on ==frames==, must be over a determined standard value, 24 frames per second. People watching video files with less than 24 frames per second are likely to notice fliker.

This small amount of frames may be enough for cinema but it's not enough for the rest of the applications such as the TV. If you would watch a football game, recorded at just 25 frames per second, the movement of the ball in the air would seem at least weird, it would seem the ball stands still in the air or jumps from one location to another.

Something had to be done to increase the number of frames shown on the screen but, unfortunately, in the 30's, when TV was invented, there simply was no technology powerful enough to accomplish this. Therefore, the inventors resorted to a nice trick: they've reduced the vertical height of the image to half of the original and interlaced (mixed together vertically) two frames together. This way, there are now 50 frames per second, which brings fluidity but the video resolution is a bit lower. After all, the height is only half the original.

In conclusion, a regular PAL video broadcast, 768 pixels wide, 576 pixels tall and 25 frames per second interlaced is actually a video broadcast 768 pixels wide, 288 pixels tall, with 50 frames each second. The TV or the video player on the computer, before showing the image on the screen, deinterlaces the video, splitting and processing each video frame in order to obtain the two original two frames.

So, what is deinterlacing?

It's the art of converting this interlaced sequence of video frames into a non-interlaced form, while preserving as much quality as possible.

Let's move on and see what kind of problems we may be faced with during the deinterlacing process.

Interlacing Problems

Let's see how the two frames - called from now on fields - are interlaced:

In reality, the second field is positioned directly under the first field. As you can see, the two fields alternate, forming the frame. The first field is the first frame and the second one is the second field is the second frame.

Here's an image that shows this better:

If you look at it, you can bearly see interlacing artifacts. In this case the interlacing is a good thing, it manages to bundle together two frames without noticeable loss. Let's look at the two fields that compose the frame:

The first field in the frame

The second field in the frame (click to enlarge)

Problems occur when the camera changes its location or when another scene starts.

This is best illustrated in the image below:

A transition between two scenes is a perfect example (click to enlarge)

The two fields that form the image belong to different scenes:

The first field in the frame

The second field in the frame

Because each field has it's own different time but they're mixed together into a single frame, it is not possible to deinterlace the frames, maintain 25 frames per second and keep all the information in the images, at least one of these parameters has to be sacrificed.

This is where different deinterlacing algorithms have a lot of work to do and this is where optimizations appear.

Interlacing problems

Another typical location where algorithms may have problems deinterlacing is a location where camera pans from left to right or from right to left:

A moving camera can cause problems when deinterlacing

The two fields that form the image belong to the same scene, but the camera moves a bit to the left:

The first field in the frame

The second field in the frame

This is a really serious problem when text scrolls horizontally, at the bottom of the image:

Mice teeth appear on the text in this image from a Romanian talkshow because the text scrolls from right to left

Vertical scrolling text is also affected and causes a fair amount of trouble:

The vertical scrolling credits text produces mice teeth, because of interlacing (click to enlarge)

Some digital cameras have problems recording and interlacing images, sometimes the color smears or bleeds. This is usually because the internal hardware places different colors on different fields or because some CCD sensors react slower to some of the colors in the image.

Because there are so many issues and things to consider, there hardly is a deinterlacing method that would work with any video material. One has decide which method to use considering lots of factors.

In the next pages, I'll try to explain some of the deinterlacing methods known.

Deinterlacing methods

Blending

Also called Average or Combine Fields, this method overlaps the two fields and blends them together.

The main advantages of this method are speed and simplicity. Any average computer can do it in real time, therefore it's suitable for capturing video and deinterlacing at the same time.

The blending method is very good at deinterlacing video sequences with little movement such as talk shows or presentations. However, consider using another method, if you have the time: when movement occurs, the image may become blurry or foggy, because of the blending.

Weave

This method is also called Do Nothing because is consists in simply showing both fields at once.

Naturally, processing is very fast because there is almost no processing performed, the video height remains the same but also the mice teeth.

The method is typically used when the video contains both interlaced and deinterlaced material, when some sequences don't have to be deinterlaced.

Discarding Fields

Also called Single Field Mode, this method consists of completely removing one field from the frame.

There is obviously less quality because half of the information is lost, however the image will still be sharper than using the blending method and the video will be easier to compress. In quiet scenes some sharpness is lost and the video will be less fluid, in some areas it will stutter.

Adaptive Deinterlacing

This method analyzes the video and determines when mice teeth should be blurred (usually on movement) or when preserving sharpness is appropriate.

The algorithm must be well optimized, otherwise good parts of the video can be removed by accident or the image can become very blurred at fast movement.

Bob

This technique simply separates the two fields and places them in two separate frames, effectively doubling the video's frame rate.

The height of the image is reduced by half yet the video is very fluid and sharp.

Because one of the fields is one scan line below the other field, there is a possibility for video artifacts to "jump", effect usually noticed on TV logos.

A very powerful computer to play a compressed video deinterlaced with this method: the video codec must be able to handle twice the number of original frames per second and resize the images to the original height at the same time.

This deinterlace method, combined with the Weave method, usually produce the highest quality results.

Resizing to a lower resolution

Well, it's not really a deinterlacing method - in fact any video processing application can do this - but it does guarantee that the interlacing effects disappear.

According to standards, a video that is exactly or less than 384x288 pixels can not be interlaced: resizing to this resolution or to a lower resolution (but preserving the aspect-ratio) assures a interlace-free video.

It's a very easy solution and the only negative effect is that a bit of quality is lost (usually the horizontal width is reduced by half).

Motion compensation

This last method is also the most complex: it is supposed to analyze the contents of each field and determine the objects inside the images, then to deinterlace the images full spatial resolution without loss of image resolution.

This method is very processor intensive, it takes about 40 seconds to deinterlace one second of video on an average computer using the only available application designed to do this: HiCon³², from Fraunhofer Labs, the inventors of the MP3 format.

The results are amazing, crisp, sharp images without signs of interlaced material, however, if you really need it, you're better using expensive hardware solutions.