You are here

Human Motion Adaptation by a distance-based approach

Team and supervisors
Department / Team: 
Team Web Site: 
https://team.inria.fr/mimetic/
PhD Director
Franck Multon
Co-director(s), co-supervisor(s)
Antonio Mucherino
Ludovic Hoyet
Contact(s)
NameEmail address
Antonio Mucherino
antonio.mucherino@irisa.fr
PhD subject
Abstract

Animated human-like characters are nowadays largely exploited in computer graphics, with applications from game industry to film production. The traditional representation of such characters is based on a main skeletal structure, consisting of bones and joints, and the classical approach for describing a motion is through the variations of joint angles over time. A common problem arising in this context is the design of new motions, edited by exploiting and adapting motions that were previously recorded from a given actor, for a new "target" character who may have a morphology that is quite different from the one of the original actor. This problem is known in computer graphics as the "retargeting" problem. One main challenge is the automatic transfer of existing constraints, such as the contacts with the virtual environment, and in-between body segments [1,2]. For example, automatically ensuring that the hand of a character still lies on the surface of his abdomen during the motion is not straightforward, especially when the morphology of the actor and the character to animate strongly differ. Various approaches have been proposed in the scientific literature for the retargeting problem, such as the ones based on inverse kinematics [3], interaction mesh [4], computer puppetry [5], and morphology independent skeletal representations [6], to name a few. However, despite the large efforts the scientific community has been devoting to this problem, an automatic and accurate solution method was not yet discovered, implying the necessity of manual editing, which is incompatible with real-time applications.

The PhD candidate will be involved in the research activity currently performed by the MimeTIC team at IRISA, which aims at developing a novel approach for automatically solving human motion adaptation issues for a wide range of situations. This approach consists in replacing the traditional joint-angles encoding by a new methodology that is based on the concept of distance; distances can represent body parts, as well as the relative motions between body segments, or even between a body segment and the environment. The motivation to explore such a distance-based approach comes from the fact that several constraints arising in the motions can be easily expressed in terms of distances. For example, a character hand touching a body part can be simply represented by a null distance between two points.

In order to explore such questions, we have recently been drawing methodologies from the field of Distance Geometry (DG). One of the main problems in DG consists in identifying the positions of the points of a given set, in a Euclidean space, by exploiting known inter-point distances [7]. Some of our most recent results have therefore been relying on DG to encode and solve the variations of relative distances between body parts to represent the motions [8], a representation allowing us to prevent the appearance of undesired contacts, as well as to preserve original ones. Our preliminary studies on simple character models have shown that such a distance-based approach is promising for overcoming these typical adaptation issues [9,10], and have great potentials to become the basis of more advanced methods for automatic real-time processing applications. As we only considered in our initial studies the traditional skeletal structure representing the human body, the simplicity of the model led to the definition of DG instances with continuous search domains, which are in general hard to explore exhaustively. Moreover, this simple model is not able to represent all body contacts involving the surface of the character. Therefore, even though at the current status this approach can adapt motions from a character to another having a different morphology, it is not able yet to deal with extremely different morphologies, mostly because of the limitations given by the currently considered representation.

The goal of this PhD is therefore to develop a more accurate distance-based character model. Unlike our previous work based on skeletal joint information, the objective is to rely on the surface information, i.e. the mesh structure of the character, which can give a quite precise representation of the character surface. The use of this more detailed character representation will lead to several contributions. In particular, the definition of the retargeting problem using DG instances with a discrete (and finite!) search domain will allow us to propose new solutions based on ad-hoc tools for combinatorial optimization, while improving the overall visual quality of retargeted motions. The PhD candidate will have to review the existing methods for DG and combinatorial optimization, and to adapt some selected methods for the retargeting problem. These optimization methods will be compared and (when relevant) combined with the state of the art in computer graphics, in order to deliver, at the end of the PhD Thesis, a software tool for retargeting to researchers and practitioners. An application in film industry will be investigated for validating the developed software tools.

Requirements for candidacy
- Strong C/C++ programming skills
- Solid background in mathematics
- Previous experience in character animation would be an advantage

Bibliography

1. F. Multon, R. Kulpa, B. Bideau, "MKM: a Global Framework for Animating Humans in Virtual Reality Applications", Presence: Teleoperators and Virtual Environments 17(1), 17–28, 2008.

2. Ch. Hecker, B. Raabe, R.W. Enslow, J. DeWeese, J. Maynard, K. van Prooijen, "Real-Time Motion Retargeting to Highly Varied User-Created Morphologies", Proceedings of ACM SIGGRAPH 2008, ACM Transactions on Graphics 27(3), 2008.

3. B. Le Callennec, R. Boulic, "Interactive Motion Deformation with Prioritized Constraints", Proceedings of the 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA04), 163–171, 2004.

4. E.S.L. Ho, T. Komura, C.-L. Tai, "Spatial Relationship Preserving Character Motion Adaptation", ACM Transactions on Graphics 29(4), article 33, 2010.

5. H.J. Shin, J. Lee, S.Y. Shin, M. Gleiche, "Computer Puppetry: An Importance-Based Approach", ACM Transactions on Graphics 20(2), 67–94, 2001.

6. R. Kulpa, F. Multon, B. Arnaldi, "Morphology-independent Representation of Motions for Interactive Human-Like Animation", Computer Graphics Forum 24(3), 343–351, 2005.

7. L. Liberti, C. Lavor, N. Maculan, A. Mucherino, "Euclidean Distance Geometry and Applications", SIAM Review 56(1), 3-69, 2014.

8. A. Mucherino, D.S. Gonçalves, "An Approach to Dynamical Distance Geometry", Lecture Notes in Computer Science 10589, F. Nielsen, F. Barbaresco (Eds.), Proceedings of Geometric Science of Information (GSI17), Paris, France, 821--829, 2017.

9. A. Bernardin, L. Hoyet, A. Mucherino, D.S. Gonçalves, F. Multon, "Normalized Euclidean Distance Matrices for Human Motion Retargeting", ACM Conference Proceedings, Motion in Games 2017 (MIG17), Barcelona, Spain, article 15, 2017.

10. A. Mucherino, D.S. Gonçalves, A. Bernardin, L. Hoyet, F. Multon, "A Distance-Based Approach for Human Posture Simulations", IEEE Conference Proceedings, Federated Conference on Computer Science and Information Systems (FedCSIS17), Workshop on Computational Optimization (WCO17), Prague, Czech Republic, 441-444, 2017.

Work start date: 
September 2018
Keywords: 
motion retargeting, animated characters, distance geometry, combinatorial and global optimization
Place: 
IRISA - Campus universitaire de Beaulieu, Rennes