Here are the results using our Mutual-Structure for Joint Filtering to restore the depth/noisy color images from Lu et al. 2014. 

Before restore the depth image, we applied bilateral filter to fill the depth image holes and a median filer with size [3 3] is employed to our final restore depth images.