Leveraging non-Euclidean spaces for learning on structured data

Context. Most of machine learning algorithms require input data to be embedded in an Euclidean space,
that is to say represented in a vectorial form as a set of floats. In many domains though, the data (e.g.
hierarchical, graph or cyclical data) can be decomposed into a set of entities that possess an intricate
internal structure usually in the form of one or more relations between them. There is thus a need to
embed those structured data into meaningful representations and to define a similarity between them.
Nevertheless, those data do not possess a Euclidean structure but rather a hierarchical or cyclical struc-
ture and, in that case, they cannot be embedded in a Euclidean space with low distortion [1, 2]. In
the recent years, several works have shown the superiority of non-Euclidean representations such as the
Riemannian geometry, whose choice determines how the relations between points are quantified. In this
context, hyperbolic spaces has shown to be well suited to cope with hierarchical data, with state-of-the-art
performances in graph [3] or word embedding [4] tasks, image segmentation [5] or few-shot learning [6].
On the other hand, spherical spaces successfully deal with the face recognition problem [7] or with points
on lying on the surface of the Earth [8]. Those Riemannian spaces are of constant positive curvature
(spherical) or negative curvature (hyperbolic), in comparison to the Euclidean space which is of zero
curvature. From a field with marginal interest in the ML community about 5 years ago (Google Scholar
indicates 56 results when searching for “Hyperbolic spaces”+”Machine Learning” in 2017), it is now a
booming research field (with almost 500 results for 2022).
Going beyond spaces of constant curvature, product spaces [9] combine several copies of Euclidean, hy-
perbolic and spherical spaces, with the assumption that the data have complicated structure, aiming at
getting the best of all worlds. While these spaces have shown promising results, the optimal combination
of the spaces is still chosen by trial-errors, which is not acceptable in large scale applications.

Scientific objectives and expected achievements. The objective of the PhD is to fully take advantage
of the Riemannian geometry to perform learning on structured data. While embedding data into an
appropriate space leads to enhanced learning performances, most of the existing works have focused on
designing dedicated architectures [10] or dedicated metrics [11], synchronized on Euclidean developments.
Very few of them focused on specific loss functions that fully take advantage of the geometry at hand:
as an example, standard losses such as the cross-entropy loss are used in structured spaces, with little
modifications w.r.t. the space (i.e. angular soft max [7] for spherical spaces or hierarchical soft max
[12] for hyperbolic spaces). They very often ignore some available explicit prior knowledge such as class
text description, labels, or class hierarchy. The intrinsic properties of the spaces such as permutation
or rotational invariances are not taken into account either. In addition, when comparing different views
of the same data (e.g. text/image or images taken from different captors in remote sensing), embedded
in different geometries, the comparison of those incomparable spaces is challenging. The question of a dedicated loss for product spaces is still an open problem.
Those challenging questions will be first tackled by considering Gromov-Wasserstein [13]-like losses, which
have been shown to be an effective metric for comparing graph-like data in Euclidean Spaces. In partic-
ular, it allows comparing data living in different spaces, and retains some important invariances such as
the translational or rotational invariance. It has also shown its interest in dealing with graph-structured
data [14].
From an applicative view point, remote sensing data, that are inherently hierarchically organized, multi-
modal, with spectral and correlated attributes, will serve as a playground to validate the proposed meth-
ods. The problem of few-shot learning will be specifically targeted to evaluate the ability of generalization
and the performance in low-dimensional output embeddings.

Outcomes. Outcomes of the PhD will lead to publications in the machine learning community. This
project follows developments performed during the Multiscale ANR project (ANR-18-CE23-0022, PI:
Laetitia Chapel) whose aim is to learn on multi-modal hierarchically-organized remote sensing data.
 

Previous significative works performed by the team. The PhD will build over very recent developments
performed by the Obelix team, which shows the interest of taking into account the hierarchical nature
of the data by designing a specific loss in Euclidian spaces ([15], best paper award) or the superiority
of hyperbolic embedding w.r.t. Euclidean one in a remote sensing few shot learning problem [16]. New
metrics to compare data lying in hyperbolic [17] or spherical [11] spaces have also been proposed by some
members of the team.