Robotics Active Vision Group – Cinvestav

Shape analysis

For shape analysis we have focused efforts on the recovery of 3D facial shape from one or several 2D images. To this end, two approaches have been explored:

  • Statistical approaches

Here, a previously built model is learnt for variations of shape and texture. These models are usually constructed by maximizing covariance (finding principal components) of either shape and texture. We use the Basel face model, as full correspondences are also available for all of the vertexes in cylindrical coordinates. For many shape analysis problems (not necessarily face analysis), coupled variations provide a more suitable way for understanding the nature of data and the hidden language of the variables involved in the studied phenomena. Our most recent research involves predicting full 3D shape from 2D occluding contours. The main idea is illustrated in the following figure:

 

Selected publications on this topic (Some of this work has been in collaboration with the University of York, United Kingdom and the Center for Research in Mathematics, Mexico.)

Dalila Sánchez-Escobedo, Mario Castelán and William A. P. Smith. Statistical 3D face shape estimation from occluding contours. Computer Vision and Image Understanding, 142: 111-124 (2016)

Dalila Sánchez-Escobedo and Mario Castelán. 3D face shape prediction from a frontal image using cylindrical coordinates and partial least squares, Pattern Recognition Letters, 34(4): 389-399 (2013).

Mario Castelán and Johan Van Horebeek. Relating intensities with three-dimensional facial shape using partial least squares, IET Computer Vision, 3(2):60-73 (2009).

Mario Castelán, William A. P. Smith and Edwin R. Hancock. A coupled statistical model for face shape recovery from brightness images, IEEE Transactions on Image Processing, 16(4): 1139-1151 (2007).

  • Photometric approaches

In this case, shape analysis becomes a problem of inverting Lambert’s law, i.e. the relationship between the surface normal and the light source direction causing an observed irradiance. The original ill-posed problem is known as shape-from-shading, however, when more than one light source is considered, new constraints allow numerical solutions to be included in the family of photometic stereo methods. We have been interested in studying the reflectance properties of both faces and diverse materials. Our most recent research involves the development of a photometric stereo approach that is exclusive for face shape recovery, seeking for a coherent way to keeping facial proportions when estimating shape from several 2D images. Examples of facial shape recovery are shown in the following figure, where the recovered surface is imposed over an image of the profile picture of two subjects:

 

Selected publications on this topic:

Felipe Hernández-Rodríguez and Mario Castelán. A photometric sampling method for facial shape recovery.  Machine Vision and Applications,  27 (4): 483-497 (2016).

Mario Castelán, Elier Cruz-Pérez and Luz Abril Torres-Méndez. A Photometric sampling strategy for reflectance characterization and transference, Computación y Sistemas, 19 (2) : 255-272 (2015)

Jocelyn Miranda-Hernández, Mario Castelán and Luz Abril Torres-Méndez. Face colour synthesis using partial least squares and the luminance-a-b colour transform, IET Computer Vision, 6(4): 263-272 (2012).

Felipe Hernández Rodríguez and Mario Castelán. A method for improving consistency in photometric databases, Proc. British Machine Vision Conference, 1-1o (2012).