Citation Context

...with linear filters is that edges are not preserved. Even though pixels near an edge are very close spatially their colors (or normals in our case) are not correlated. The idea of bilateral filtering =-=[26]-=- is to take into account not only domain weights (distance) but also range weights (color similarity). In [13], bilateral filtering was extended to RGBNs, using also normal similarity (see Figure 2.16...

Citation Context

...mple (Figure 3.2). As an alternative to the bilateral filter we could build the smooth normals using a low pass kernel. To assist the user in defining the editing region, we use a segmentation method =-=[35]-=- which was extended [13] to handle RGBN images. The procedure is fast and is well suited to interactive applications. It also takes into account the normal and color channels, using all available info...

Citation Context

...gular curve (C ′ (t) ̸= 0). We can always approximate discontinuities with high curvatures. Under this hypothesis, there is a tubular neighborhood T where the mapping defined by φ is a diffeomorphism =-=[25]-=-. In fact this holds if T does not intersect C’s medial axis M 1Matching profiles and deformations is left as an exercise for the reader. 26Figure 2.10: Custom height profiles along a path are transf...

Citation Context

...ted one at a time. In [28], the authors search for a best match between the neighborhood of the already synthesized texture and neighborhoods in the texture sample. There are also patch-based methods =-=[29]-=- [12] where the algorithm step consists of iteratively synthesizing small regions at a time. This approach leads to a direct transfer of local statistics, but it has the drawback that seams between pa...

Citation Context

...uild a custom parametrization in the region of interest, mapping the points to a chart where operations are performed. Resampling for both screen and editing charts is done with point-based rendering =-=[68]-=- methods. To update their point cloud storage, they either use an id-buffer to update the points or create a new sample point for each edited texel to avoid resampling. In this work, we present a poin...

Citation Context

... to obtain shape. Since SfS works on common photographs, it allows very high resolution models to be obtained with low high-frequency noise. One important example of SfS methods is photometric stereo =-=[10, 11]-=- in which multiple photographs are used (Figure 1.4). As detailed above, acquisition of normals is slightly more complex than usual photography, yet the power and flexibility of 3D models can be attai...

Citation Context

...ic details attribute should be stored? There are many possibilities. A first option would be storing 3D positions themselves per pixel, a technique called geometry images [3]. Second, in bump mapping =-=[4]-=-, a height map is used. This means beyond the base geometry, texture stores a displacement in the normal direction. The normal of this local height map is calculated and added to the base normal, the ...

Citation Context

...l mesh. However, what geometric details attribute should be stored? There are many possibilities. A first option would be storing 3D positions themselves per pixel, a technique called geometry images =-=[3]-=-. Second, in bump mapping [4], a height map is used. This means beyond the base geometry, texture stores a displacement in the normal direction. The normal of this local height map is calculated and a...

Citation Context

...eason, we cannot have a single mapping from the surface to texture space. Inspired by differential geometry, the graphics community has approached the texture problem as an atlas construction problem =-=[60]-=-. We chose this solution. For it to work, we additionally assume the topology of the surface is the same of the base surface. The fixed topology of the base surface lets us represent our surface using...

Citation Context

...rface. In addition to scalar methods, there are vector-based methods. Instead of storing a displacement in the base normal direction, these techniques store an arbitrary 3D vector [48]. Krishnamurthy =-=[54]-=- proposed to fit a dense polygon mesh with smooth B-splines and store vector displacements to recover the true surface. In our work, we assume the surface can be described by a scalar displacement in ...

Citation Context

...eveloped that map free-form user edit specifications to mesh deformations. Some methods pose the specifications as boundary conditions and use some form of Laplacian preservation to propagate changes =-=[39, 40, 41]-=-. Other methods build on this framework but handle details better, either encoding 53details in local frames [42] or searching for locally rigid transformations [43]. These approaches have limitation...

Citation Context

...multaneously, usually leading to large linear systems or optimization problems. But the main problem with meshes is their lack of regularity. On the contrary of irregular meshes, subdivision surfaces =-=[44]-=- have an irregular control mesh but are regular after a few subdivision steps. These surfaces easily represent smooth models with few control points and details with a large number of control points. ...

Citation Context

...ure usually in R 2 or R 3 . Parametric synthesis will target more general manifolds by working in the parametric domain (usually planar). There are also manifold methods which are non-parametric [30] =-=[31]-=-. These methods work directly in the manifold representation (usually a mesh) and use only local decisions. In [17, 12] the authors extend manifold techniques to work on RGBN images, using the normal ...

Citation Context

...ssing methods, but we defer an in depth discussion to the next chapter. Most previous methods have been restricted to working with color attributes or at most scalar displacements treated like colors =-=[55, 56, 57]-=-. 56Figure 4.1: The modeled surface consists in a smooth base surface b displaced by a scalar field h in its normal direction N b . 4.2 Scale decomposition The separation of macro and mesostructure i...

Citation Context

...e range scans have limited resolution and they have problems with high frequency noise. Acquiring normals directly is an alternative which can be accomplished with shape from shading (SfS) techniques =-=[9]-=-. These methods take a shaded image as input and aim at inverting the illumination equation to obtain shape. Since SfS works on common photographs, it allows very high resolution models to be obtained...

Citation Context

...ssing methods, but we defer an in depth discussion to the next chapter. Most previous methods have been restricted to working with color attributes or at most scalar displacements treated like colors =-=[55, 56, 57]-=-. 56Figure 4.1: The modeled surface consists in a smooth base surface b displaced by a scalar field h in its normal direction N b . 4.2 Scale decomposition The separation of macro and mesostructure i...

Citation Context

... texture usually in R 2 or R 3 . Parametric synthesis will target more general manifolds by working in the parametric domain (usually planar). There are also manifold methods which are non-parametric =-=[30]-=- [31]. These methods work directly in the manifold representation (usually a mesh) and use only local decisions. In [17, 12] the authors extend manifold techniques to work on RGBN images, using the no...

Citation Context

...nted bump-mapping. Both the surface and the displacements were stored, but he only used displacements for rendering. On the modeling side, scalar displacements have been used in the work of Lee et al =-=[52]-=-, which formally defines their model as a smooth subdivision surface together with a scalar displacement. Besides rendering, they present methods for compression, editing and animation in this hybrid ...

Citation Context

...earence similar to the original when rendered (Figure 1.3-right). The 5Figure 1.3: Normal maps can be used to enhance simplified meshes with little loss in quality. Images taken from [8]. authors of =-=[7]-=- use an appearence error metric to simplify a large mesh, storing color and normals in textures. This means pixels on a rendered image deviate little from the original color. In [8], attributes are tr...

Citation Context

... for understanding real models as they are easy to capture and easy to analyse. However, shape from shading has its limitations. Typically, the normals produced have a low frequency bias. Nehab et al =-=[14]-=- pointed out that a hybrid solution for geometry acquisition is necessary. They developed a method to integrate positions and normals. Their method unites the best of both worlds: the low frequencies ...

Citation Context

...tion to propagate changes [39, 40, 41]. Other methods build on this framework but handle details better, either encoding 53details in local frames [42] or searching for locally rigid transformations =-=[43]-=-. These approaches have limitations regarding the resolution of the models they work. The reason is they represent both macro and mesostructure in a triangle mesh, which is a non-hierarchical model. T...

Citation Context

...n and store position in the planar domain. They can render the mesh by decomposing this domain in microtriangles and rendering each one separately. Their approach has been extended to multiple charts =-=[51]-=- to reduce distortions. We follow the approaches based in a mesh together with a texture atlas. While any texture atlas can be used in our method, we have chosen to use a projective atlas [38] since i...

Citation Context

... to obtain shape. Since SfS works on common photographs, it allows very high resolution models to be obtained with low high-frequency noise. One important example of SfS methods is photometric stereo =-=[10, 11]-=- in which multiple photographs are used (Figure 1.4). As detailed above, acquisition of normals is slightly more complex than usual photography, yet the power and flexibility of 3D models can be attai...

Citation Context

...eveloped that map free-form user edit specifications to mesh deformations. Some methods pose the specifications as boundary conditions and use some form of Laplacian preservation to propagate changes =-=[39, 40, 41]-=-. Other methods build on this framework but handle details better, either encoding 53details in local frames [42] or searching for locally rigid transformations [43]. These approaches have limitation...

Citation Context

...y conditions and use some form of Laplacian preservation to propagate changes [39, 40, 41]. Other methods build on this framework but handle details better, either encoding 53details in local frames =-=[42]-=- or searching for locally rigid transformations [43]. These approaches have limitations regarding the resolution of the models they work. The reason is they represent both macro and mesostructure in a...

Citation Context

...ation details. There are many possible representations for surfaces in modeling, for example, polygon soup, parametric surfaces, subdivision surfaces. Ideally, we would like a manifold representation =-=[58, 59]-=-. It would let us work in a differentiable chart for any point on the surface. However, building such a representation is a hard problem. One of the main problems is that while in differential geometr...

Citation Context

...have used parametrizations for operations. Ying et al [47] propose a local walk around a vertex to build a local parametrization. This neighbourhood is then used for texture synthesis. Biermann et al =-=[48]-=- present a solution for transferring details between meshes. During an editing session, they parameterize both source and target surface subsets in a common planar domain. There is one problem with su...

Citation Context

...approach introduces a discontinuity on the middle of the image. In [61], the authors place seams in low visibility regions. Consequently, any discontinuities in texture will be seldom noticed. Piponi =-=[62]-=- overcomes the discontinuities by using localized overlapping charts along the seams. Actually, he only 66Figure 4.7: Editing with disjoint charts introduces discontinuities, also called seams. Examp...

Citation Context

...cy. It is common to view the modeled object in a multiresolution framework that allows lowering or increasing the level of detail of the model. With such a continuous multiscale representations, e.g. =-=[1]-=-, it is easy to simplify complex models for objects that are far from the camera for example. Such objects will contribute to few screen pixels and do not need to be rendered fully. 1One problem with...

Citation Context

...cessary, making the normal image returned from SfS an important data structure by itself. We shall use the term RGBN image to mean images that contain both color and normals per pixel. In Textureshop =-=[12]-=- the 6Figure 1.4: Photometric stereo uses multiple photographs are used. Images taken from [11] in which the authors acquired Michelangelo’s Pieta. authors recovered normals from photographs and then...

Citation Context

...lculate normals in a way that is equivalent to height extrusion. In their approach only shapes with simple geometry can be created and editing of normals is not supported. In Gradient Domain Painting =-=[20]-=-, the authors propose tools that change the 14gradient of color images, a GPU-multigrid integrator recovers the new image in real-time. Different gradient blending modes are shown. In our work, the e...

Citation Context

...tion to this problem is maintaining an irregular mesh as a base mesh, but 54parametrizing it to obtain a regular representation for details. Many solutions have been introduced to parametrize a mesh =-=[50]-=-. In particular, some methods involve first partitioning the mesh and parametrizing it into multiple charts to reduce distortions. Some works use the parametrization to synthesize attributes in textur...

Citation Context

...rent and so are their storage requirements. A multiresolution hierarchy which does not handle all scales the same way is thus necessary for rendering. It is common to use three different scale levels =-=[2]-=-. The macrostructure level is the coarsest one. It is usually represented as a triangle mesh or a spline surface. This level is the general shape of the model as would even be seen from a distance. Th...

Citation Context

...oy a full multiresolution decomposition, we only decompose the surface in two scales: macro and mesostructure. Some works on subdivision surfaces have used parametrizations for operations. Ying et al =-=[47]-=- propose a local walk around a vertex to build a local parametrization. This neighbourhood is then used for texture synthesis. Biermann et al [48] present a solution for transferring details between m...

Citation Context

...h resolution models can be loaded in the GPU, avoiding delays in the change from editing to viewing mode. In future work, we would like to use this multiresolution atlas for editing purposes directly =-=[71]-=-. For instance, when editing from a distance, texels are minified. Minified samples could have an editing semantics of only editing low scale version of the signals. Computationally, we would use spla...

Citation Context

...e update solutions. Figure 5.2: During a render to texture operation with texture mapped triangles, pixels that were not visible from the camera may be wrongly changed. not visible (Figure 5.2). Carr =-=[56]-=- proposed to solve this problem using a shadow buffer, feature still not included in OpenGL standard functionality. On the other hand, the id-buffer method, which we follow in our work, provides a sim...

Citation Context

...ient of color images, a GPU-multigrid integrator recovers the new image in real-time. Different gradient blending modes are shown. In our work, the entire pipeline is composed of local operations. In =-=[21]-=-, Taubin defines a specialized laplacian operator for filtering normals on a mesh. He then integrates this normal field to obtain filtered positions. Different smoothing filter for normal maps have be...

Citation Context

...camera or the light. 4.7.3 Filtering In this section, we apply the filtering method from Chapter 2 to the editing chart. A few methods have been proposed for normal filtering both in irregular meshes =-=[63, 64, 65, 66, 21]-=- and in volumetric representations [63]. Compared to irregular mesh based approaches, our parametric normal filtering method is not only more efficient but also allows for kernels of larger support, w...

Citation Context

...f the red and blue rectangles. Figure 5.13: An example of texture distortion caused by orthogonal charts. We show a measure of distortion by color coding the mapping derivative norm. Image taken from =-=[72]-=-. 109Figure 5.14: We only allow editing in a subset of the visible region, namely regions where the normal makes a small angle with the camera direction. In addition, we cannot edit near depth discon...

Citation Context

...ng filter for normal maps have been developed for mipmapping. In [22], the author smooths normals and shows how the shortening they introduce can be used to reduce aliasing of specular highlights. In =-=[23]-=-, the authors formalize normal map filtering using convolution between normal distribution functions and the BRDF. 2.2 Filtering The simple way to filter an RGBN is to consider each channel of a 6D (c...

Citation Context

...the texture appropriately we are easily led to a high dimensional NN problem which would be too slow to handle directly. Instead, the authors use an approximate nearest neighbors (ANN) data structure =-=[37]-=- that allows very fast queries. For further optimization, PCA is used for dimensionality reduction of the feature vector. As Zelinka notices, jump map synthesis works best for stochastic textures and ...

Citation Context

...eveloped that map free-form user edit specifications to mesh deformations. Some methods pose the specifications as boundary conditions and use some form of Laplacian preservation to propagate changes =-=[39, 40, 41]-=-. Other methods build on this framework but handle details better, either encoding 53details in local frames [42] or searching for locally rigid transformations [43]. These approaches have limitation...

Citation Context

...ile still respecting shape and shading. Unlike these works, in our method, both color and normals are synthesized on RGBN images. One way of looking at RGBN synthesis is as a quasi-stationary process =-=[32]-=-. Synthesis proceeds through the image but varies spatially depending on the normals. In [33], the authors capture high resolution normals of small patches of face skin and then use texture synthesis ...

Citation Context

...ons are easier to specify using normals. In particular, operations based in specifying shading are more naturally formulated with normal deformations maybe obtained from shape from shading algorithms =-=[77, 73]-=-. However, to leverage to power of normal operations in this context, we would need a method to change position to reflect changes in normals. In other words, we need a normal integration engine. A bi...

Citation Context

...images of pixols which contain colors and depth. Normal maps can be generated from the final models. For editing normals, they would first have to be converted to depth information outside ZBrush. In =-=[17, 12]-=- the authors recovered normals from photographs and then developed texture synthesis on the resulting RGBNs. This way the normals are used to guide local distortions in the synthesized texture. In Tex...

Citation Context

...mals are synthesized on RGBN images. One way of looking at RGBN synthesis is as a quasi-stationary process [32]. Synthesis proceeds through the image but varies spatially depending on the normals. In =-=[33]-=-, the authors capture high resolution normals of small patches of face skin and then use texture synthesis to replicate these tangent space normals in parametric space. Adding detail to faces is an im...

Citation Context

...ation details. There are many possible representations for surfaces in modeling, for example, polygon soup, parametric surfaces, subdivision surfaces. Ideally, we would like a manifold representation =-=[58, 59]-=-. It would let us work in a differentiable chart for any point on the surface. However, building such a representation is a hard problem. One of the main problems is that while in differential geometr...

Citation Context

...camera or the light. 4.7.3 Filtering In this section, we apply the filtering method from Chapter 2 to the editing chart. A few methods have been proposed for normal filtering both in irregular meshes =-=[63, 64, 65, 66, 21]-=- and in volumetric representations [63]. Compared to irregular mesh based approaches, our parametric normal filtering method is not only more efficient but also allows for kernels of larger support, w...

Citation Context

...important application of our method. While in their approach, a normal map is synthesized from scratch, we can respect existing frequency bands in the normal map. Procedural synthesis was explored in =-=[34]-=- where bump maps are created on the fly based on a normal density function. 3.3 Method While the problem of synthesizing color textures on top of RGBN models has been studied in [17], we propose a met...

Citation Context

...given detail on the surface, we will have to change all the representation coefficients across many scales. For example, placing a specified feature on any position on the surface may be non-trivial. =-=[49]-=-. A solution to this problem is maintaining an irregular mesh as a base mesh, but 54parametrizing it to obtain a regular representation for details. Many solutions have been introduced to parametrize...

Citation Context

...what you see is what you get (WYSIWYG). • Some operations are more naturally specified with normals, in particular, tangent plane operations, gradient domain operations [41] and shading-based editing =-=[73]-=-. 116• The editing chart allows image-based methods to be applied to processing normals in surfaces leading to fast geometry processing algorithms. Algorithms previously proposed for images can be di...

Citation Context

...ting for foreshortening distortions. Finally, in the combination step, we merge the synthesized normal details with the smooth RGBN image. To build the smooth normals we use the RGBN bilateral filter =-=[13]-=-. It takes into account both normal and color differences and respects edges. During splitting, we use a the filtering method of Chapter 2 to apply a high pass kernel on the texture sample (Figure 3.2...

Citation Context

...proximating the curve. In addition, it is possible but time consuming to simulate normal texture synthesis with the stamp operator (Figure 2.17). In the next chapter, we extend the work of Fang et al =-=[27, 12]-=- to synthesize normals. It allows more advanced editing of normals and color in large regions. The main limitation of the RGBN image is the impossibility of changing the view point. While in this chap...

Citation Context

...f acquisition and representation of normals. Figure 1.2: Object-space normal maps (left) have range in the unit sphere. Tangentspace maps (right) take values on a single hemisphere. Images taken from =-=[6]-=-. 41.2 Normal Maps Normal mapping stores the normal in a texture map. During rendering, this map will be consulted and used to calculate shading. Since these normals will be stored at a much higher r...

Citation Context

...aplacian operator for filtering normals on a mesh. He then integrates this normal field to obtain filtered positions. Different smoothing filter for normal maps have been developed for mipmapping. In =-=[22]-=-, the author smooths normals and shows how the shortening they introduce can be used to reduce aliasing of specular highlights. In [23], the authors formalize normal map filtering using convolution be...

Citation Context

... interesting to use photogrammetric texture mapping methods [74] to map photometric normal to pre-existing models. To make acquisition easier, an important advance would be the automatic registration =-=[75]-=- of mesh and RGBN images reducing the calibration burden. In addition, fusion of multiple RGBN images should enable not only more accurate surfaces but also larger and larger RGBN images. Dynamic meso...

Citation Context

...se a nonlinear operator to add details. All the methods described in this and the next chapter were implemented in an RGBN image manipulation program as described in a technical report by the authors =-=[15]-=-. 2.1 Related Work Toler-Franklin et al. [13] coined the term RGBN to refer to an array of pixels with associated color and normal channels. They have shown that many NPR rendering algorithms work in ...

Citation Context

...sferred normals between images. Poisson image editing [18] was used to merge normals seamlessly, followed by normalization, a process which does not guarantee a conservative normal field. Normalpaint =-=[19]-=- proposes a tool for creating normal maps directly, thus avoiding the need to model a high-res mesh in the traditional modeling pipeline. They calculate normals in a way that is equivalent to height e...

Citation Context

...d, in subsection 2.4.1, we use single sketches allowing creation of new details such as new features. In the next results, the warping fields were generated using the sketch-based method presented in =-=[24]-=-. Two kinds of sketches are used: boundary and internal sketches. While boundary sketches relate the borders of Ω and Λ, interior sketches specify the mapping in the interior of these regions. As can ...

Citation Context

...control which frequency bands we edit or replace. The next chapter presents in an abstract level our surface normal editing process and how the RGBN operations are integrated using a projective atlas =-=[38]-=-. 51sChapter 4 Chart Editing In the last chapters, we presented many operations with normal vectors using the RGBN image representation. The main limitation of RGBN image is that they are limited to a...

Citation Context

...camera or the light. 4.7.3 Filtering In this section, we apply the filtering method from Chapter 2 to the editing chart. A few methods have been proposed for normal filtering both in irregular meshes =-=[63, 64, 65, 66, 21]-=- and in volumetric representations [63]. Compared to irregular mesh based approaches, our parametric normal filtering method is not only more efficient but also allows for kernels of larger support, w...

Citation Context

...camera or the light. 4.7.3 Filtering In this section, we apply the filtering method from Chapter 2 to the editing chart. A few methods have been proposed for normal filtering both in irregular meshes =-=[63, 64, 65, 66, 21]-=- and in volumetric representations [63]. Compared to irregular mesh based approaches, our parametric normal filtering method is not only more efficient but also allows for kernels of larger support, w...

Citation Context

... depth along this ray. In other words, geometry, in this case depth, together with camera parameters let us calculate the parametrization function. In fact, we used a multiresolution projective atlas =-=[70]-=-. Using the charts, it is simple to build a multiresolution model: each image is downsampled multiple times. 105(a) Shaded (b) Id (c) Id Extension (d) Normal (e) Normal Extension Figure 5.10: The nos...

Citation Context

...s should enable not only more accurate surfaces but also larger and larger RGBN images. Dynamic mesostructure in the form of RGBN-t, i.e, videos with color and normal per pixel, are already available =-=[76]-=-. As a consequence, manipulation methods that handle time would be useful. Present 118capture techniques still require expensive hardware, therefore more practical capture methods are necessary for i...

Citation Context

...d, in Subsection 2.4.1, we use single sketches allowing creation of new details such as new features. In the next results, the warping fields were generated using the sketch-based method presented in =-=[24]-=-. Two kinds of sketches are used: boundary and internal sketches. While boundary sketches relate the borders of Ω and Λ, interior sketches specify the mapping in the interior of these regions. As can ...