This paper presents a restoration framework for correcting both photometric and geometric distortions in camerabased images of non-planar shaped documents to facilitate human perception and machine recognition. The photometric distortions, usually perceived as shading artifacts, are corrected by separating the shading image from the reflectance image through digital inpainting and surface fitting techniques. Meanwhile, the extracted shading image is also used to recover the document’s surface shape through a Shape-from-Shading (SFS) method with a generic formulation of the image irradiance under arbitrary illumination conditions. The recovered surface shape is then employed to correct the geometric distortions through a physicallybased flattening process. Results on real document images demonstrate the performance of each sub-task and the functionality of the whole framework. OCR results on restored images also show great improvements over the original distorted images. 1.

We present a restoration framework to reduce undesirable distortions in imaged documents. Our framework is based on two components: 1) an image inpainting procedure that can separate non-uniform illumination (and other) artifacts from the printed content; and 2) a Shape-from-Shading (SfS) formulation that can reconstruct the 3D shape of the document’s surface. Used either piecewise or in its entirety, this framework can correct a variety of distortions including shading, shadow, ink-bleed, show-through, perspective and geometric distortions, for both camera-imaged and flatbed-imaged documents. Our overall framework is described in detail. In addition, our SfS formulation can be easily modified to target various illumination conditions to suit different real-world applications. Results on images of synthetic and real documents demonstrate the effectiveness of our approach. OCR results are also used to gauge the performance of our approach.