Robotics Active Vision Group – Cinvestav

Shape analysis and 3D reconstruction

Mario Castelán’s research

Back to Mario Castelán

My passion in research is the discovery of hidden relationships in data containing a large amount of variables in comparison with a small number of observations. This falls in the field of statistical analysis with an emphasis on dimensionality reduction for building a more suitable representation for the data. Besides other applications such as humanoid robotics, acoustics and more recently agricultural analysis, I have applied dimensionality reduction techniques in two main areas of computer vision: Shape Analysis and 3D reconstruction.


Shape analysis

For shape analysis I have focused efforts on the recovery of 3D facial shape from one or several 2D images. To this end, I have explored two approaches:

  • Statistical approaches

Here, a previously built model is learnt for variations of shape and texture. These models are usually constructed by maximizing covariance (finding principal components) of either shape and texture. I particularly like working with the Basel face model, as full correspondences are also available for all of the vertexes in cylindrical coordinates. For many shape analysis problems (not necessarily face analysis), I believe that coupled variations provide a more suitable way for understanding the nature of data and the hidden language of the variables involved in the studied phenomena. My most recent research involves predicting full 3D shape from 2D occluding contours. This was one of the PhD contribution of Dr. Dalila Sánchez-Escobedo. The main idea is illustrated in the following figure:

CVIU1

Selected publications on this topic (Some of this work has been in collaboration with the University of York, United Kingdom and the Center for Research in Mathematics, Mexico.)

Dalila Sánchez-Escobedo, Mario Castelán and William A. P. Smith. Statistical 3D face shape estimation from occluding contours. Computer Vision and Image Understanding, 142: 111-124 (2016)

Dalila Sánchez-Escobedo and Mario Castelán. 3D face shape prediction from a frontal image using cylindrical coordinates and partial least squares, Pattern Recognition Letters, 34(4): 389-399 (2013).

Mario Castelán and Johan Van Horebeek. Relating intensities with three-dimensional facial shape using partial least squares, IET Computer Vision, 3(2):60-73 (2009).

Mario Castelán, William A. P. Smith and Edwin R. Hancock. A coupled statistical model for face shape recovery from brightness images, IEEE Transactions on Image Processing, 16(4): 1139-1151 (2007).

  • Photometric approaches

In this case, shape analysis becomes a problem of inverting Lambert’s law, i.e. the relationship between the surface normal and the light source direction causing an observed irradiance. The original ill-posed problem is known as shape-from-shading, however, when more than one light source is considered, new constraints allow numerical solutions to be included in the family of photometic stereo methods. I have been interested in studying the reflectance properties of both faces and diverse materials.  My most recent research involves the development of a photometric stereo approach that is exclusive for face shape recovery, seeking for a coherent way to keeping facial proportions when estimating shape from several 2D images. This was one of the PhD contribution of my former student, Dr. Felipe Hernández-Rodríguez. Examples of facial shape recovery are shown in the following figure, where the recovered surface is imposed over an image of the profile picture of two subjects:

MVAP1

Selected publications on this topic:

Felipe Hernández-Rodríguez and Mario Castelán. A photometric sampling method for facial shape recovery.  Machine Vision and Applications,  27 (4): 483-497 (2016).

Mario Castelán, Elier Cruz-Pérez and Luz Abril Torres-Méndez. A Photometric sampling strategy for reflectance characterization and transference, Computación y Sistemas, 19 (2) : 255-272 (2015)

Jocelyn Miranda-Hernández, Mario Castelán and Luz Abril Torres-Méndez. Face colour synthesis using partial least squares and the luminance-a-b colour transform, IET Computer Vision, 6(4): 263-272 (2012).

Felipe Hernández Rodríguez and Mario Castelán. A method for improving consistency in photometric databases, Proc. British Machine Vision Conference, 1-1o (2012).


3D reconstruction

Although statistical analysis can also be applied in 3D face shape recovery,  I have additionally worked on the development of geometric approaches for the monocular-based localization of humanoid robots, finding applications in both navigation and object reconstruction tasks.  This is more related with the 3D reconstruction of the environment sensed by the robot. The main idea here is to profit on the accuracy of 3D-to-2D visual odometry strategies so that full 3D models of objects can be approximated from the march of a humanoid robot. It is also important to consider the closed-loop of the visual information with the linear and angular velocities of the robot. The following image shows some of the results corresponding to one of the contribution of my former PhD student, Dr. Pablo A. Martínez, for the application of 3D object reconstruction from a video sequence obtained by a humanoid robot. The task is to generate a set of views of a real world object while following a circular trajectory, so that healthy segmentations of the object can later feed a space carving algorithm for 3D reconstruction of the object with measuring dimensions matching those of object in the real world.

Humanoids_20141

Selected publications on this topic (Some of this work has been in collaboration with the Technical University of Catalonia, Spain and the Center for Research in Mathematics, Mexico.)

Pablo A. Martínez-González, Mario Castelán and Gustavo Arechavaleta. Vision based persistent localization of a humanoid robot for locomotion Tasks,  International Journal of Applied Mathematics and Computer Science,  26 (3)  (2016).

Pablo Arturo Martínez, David Varas, Mario Castelán, Margarita Camacho, Ferran Marques and Gustavo Arechavaleta. 3D shape reconstruction from a humanoid generated video sequence, Proc. IEEE-RAS International Conference on Humanoid Robots, 699-706 (2014).

Josafat Delfin, Oscar Mar, Jean-Bernard Hayet, Mario Castelán and Gustavo Arechavaleta. An active strategy for the simultaneous localization and reconstruction of a 3D object from a humanoid platform, Proc. IEEE-RAS International Conference on Humanoid Robots, 384-389 (2012).


Contact us

CINVESTAV, Robótica y Manufactura Avanzada

Contact us