Video Repairing

Jiaya Jia

Tai-Pang Wu Yu-Wing Tai

Chi-Keung Tang

Abstract ! This paper presents a complete system capable of synthesizing a large number of pixels that are missing due to occlusion or damage in an uncalibrated input video. These missing pixels may correspond to the static background or cyclicmotions of the captured scene. Our system employs user-assisted video layer segmentation, while themain processing in video repair is fully automatic. The input video is first decomposed into the color and illumination videos. The necessary temporal consistency is maintained by tensor voting in the spatio-temporal domain. Missing colors and illumination of the background are synthesized by applying image repairing. Finally, the occluded motions are inferred by spatio-temporal alignment of collected samples at multiple scales. We experimented our system with some difficult examples with variable illumination, where the capturing camera can be stationary or in motion.



    author = {Jiaya Jia and
Tai-Pang Wu and Yu-Wing Tai and Chi-Keung Tang},
    title = {
Video Repairing: Inference of Foreground and Background Under Severe Occlusion},
    booktitle = {CVPR},
    year = {2004},
    pages = {
I: 364-371}

    author = {Jiaya Jia and Yu-Wing Tai and Tai-Pang Wu and Chi-Keung Tang},
title = {Video Repairing Under Variable Illumination Using Cyclic Motions},
    journal = {
IEEE Transactions on Pattern Analysis and Machine Intelligence},
    volume = {28},
    year = {2006},
    number = {5},
    pages = {832-839}


In this project, we analyze the cyclic motions in videos and propose the approach to repair either occluded static background or motions.

Repairing static background with camera motions:

Input video Result video with the bridge removed and a keyed-in car


Repairing cyclic motions:


Input video Occluder removed Repaired video


Complex scene containing occluded multiple-layer cyclic motions and background:

It is noted that the shadow is removed in the background completion in this example.

Input video Repaired video with completed motions and background


Repairing video with varied luminance:

Input video Repaired video with varied luminance


Repairing video with varied luminance and moving camera:

Input video Repaired video with varied luminance and moving camera