Introduction to Geometric Deep Learning

Geometric Deep Learning / Introduction

Facing challenges with high-dimensional, densely packed but limited data, and complex distributions? Geometric Deep Learning (GDL) offers a solution by enabling data scientists to grasp the true shape and distribution of data. This newsletter explores the diverse techniques and frameworks shaping the field of Geometric Deep Learning.

Geometric Deep Learning, World Models use complex math to overcome limitations of traditional deep learning. We make that math practical through hands-on tutorials.

---

1. Why this Matters

2. Key Takeaways

3. Limitations of Current Models

   Deep Learning

   Generative Models

4. What is Geometric Deep Learning

   Graph Neural Networks

   Topological Data Analysis

   Topological Deep Learning

   Data Manifolds & Groups

   Mesh & Grid Based Models

5. Graph Theory

   Transductive vs. Inductive Graph Networks

   Message Passing & Aggregation

6. Differential Geometry

   Vector, Tensor fields & Differential Forms

   Geodesic, Metric & Maps

   Lie Groups & Algebras

7. Topology

   Terminology

   Simplicial Complexes

   Cell Complexes

   Hypergraphs

   Laplacians

   Persistent Homology

8. Information Geometry

   Divergences

   Fisher Metric

   Closed-form Statistical Manifolds

9. Applications

10. Frameworks & Libraries

11. References

12. Q&A

13. Paper Review

👉 Patrick Nicolas is a 30-year software engineering veteran and consultant specializing in Geometric Deep Learning and World Models, author of Scala for Machine Learning, and writer of Geometric Learning in Python.

Why this Matters

Purpose: Introduction to Geometric Deep Learning and how it addresses the limitations of current machine learning models.

Audience: Anyone with some basic understandings of artificial intelligence and machine learning.

Value: Learning about Geometric Deep Learning and Graph Neural Networks unlocks the ability to model complex, structured, and relational data, enabling advanced problem-solving in areas like social networks, molecular science, design optimization, and transportation systems.

---

📚 The mathematics behind Geometric Deep Learning spans algebraic topology, differential geometry, graph theory, information geometry, and category theory. For an overview, see Demystifying Math of Geometric Deep Learning

---

Key Takeaways

• Geometric Deep Learning (GDL) tackles key limitations of traditional deep learning models, including overfitting, representation dependency, and interpretability challenges.

• GDL methods fall into several categories: Graph Neural Networks, Topological Data Analysis, data manifold models, and mesh/grid-based models.

• Applications leveraging GDL are beginning to emerge, supported by various Python libraries, with Geomstats and PyTorch Geometric being among the most widely used.

• This newsletter explores the diverse techniques and frameworks shaping the field of Geometric Deep Learning.

Limitations of Current Models

Deep Learning

Data scientists face challenges when building deep learning models that can be addressed by geometric, non-Euclidean representation of data. Those challenges are [ref 1]:

• High dimensionality: Models related to computer vision or images deal with high-dimensional data, such as images or videos, which can make training more difficult due to the curse of dimensionality.

• Availability of quality data: The quality and quantity of training data significantly affect the model's ability to generate realistic samples. Insufficient or biased data can lead to overfitting or poor generalization.

• Underfitting or overfitting: Balancing the model's ability to generalize well while avoiding overfitting to the training data is a critical challenge. Models that overfit may generate high-quality outputs that are too similar to the training data, lacking novelty.

• Embedding physics law or geometric constraints: Incorporating domain constraints such as boundary conditions or differential equations model s very challenging for high-dimensional data.

• Representation dependence: The performance of many learning algorithms is very sensitive to the choice of representation (i.e. z-normalization impact on predictors).

Generative Models

Generative modeling includes techniques such as auto-encoders, generative adversarial networks (GANs), Markov chains, transformers, and their various derivatives.

Creating generative models presents several specific challenges beyond plain vanilla deep learning models for data scientists and engineers, primarily due to the complexity of modeling and generating data that accurately reflects real-world distributions [ref 2]. The challenges that can be addressed with differential geometry include:

• Performance evaluation: Unlike supervised learning models, assessing the performance of generative models is not straightforward. Traditional metrics like accuracy do not apply, leading to the development of alternative metrics such as the Frechet Inception Distance (FID) or Inception Score, which have their limitations.

• Latent space interpretability: Understanding and interpreting the latent space of generative models, where the model learns a compressed representation of the data, can be challenging but is crucial for controlling and improving the generation process.

---

📌 There are other areas of machine learning which is affected by the restriction of Euclidean representation and would benefit from Geometric learning.

---

What is Geometric Deep Learning

Geometric Deep Learning has been introduced by M. Bronstein, J. Bruna, Y. LeCun, A. Szlam and P. Vandergheynst in 2017 [ref 3]. Readers can explore the topic further through a tutorial by M. Bronstein [ref 4].

Geometric Deep Learning (GDL) is a branch of machine learning dedicated to extending deep learning techniques to non-Euclidean data structures. Currently, there is no universally agreed-upon, well defined scope for Geometric Deep Learning, as its interpretation varies among authors.

---

📚 Articles

Taming PyTorch Geometric for Graph Neural Networks

Demystifying Graph Sampling & Walk Methods

---

Here's a somewhat arbitrary breakdown:

1. Graphs: Represent relationships between entities (nodes/vertices and edges), such as social networks, knowledge graphs, and molecular structures. Graph Neural Networks (GNNs) are widely used models that leverage graph-based representations.

   ---

   📚 Articles - Hands-on Geometric Deep Learning

   Taming PyTorch Geometric for Graph Neural Networks

   Demystifying Graph Sampling & Walk Methods

   Plug & Play Training for Graph Convolutional Networks

   Neighbors Matter: How Homophily Shapes Graph Neural Networks

   Revisiting Inductive Graph Neural Networks

   Graph Convolutional or SAGE Networks? Shootout

   Slimming the Graph Neural Network Footprint

   Demystifying the Math of Geometric Deep Learning - Graph Theory

   ---

2. Topological Domains: Encode higher-level relationships between entities in scientific datasets using structures like hypergraphs, simplicial complexes, or cell complexes. Topological Data Analysis (TDA) and Topological Deep Learning address limitations of Graph Neural Networks by capturing non-local properties and dependencies.

   ---

   📚 Articles - Hands-on Geometric Deep Learning

   From Nodes to Complexes: A Guide to Topological Deep Learning

   Exploring Simplicial Complexes for Deep Learning: Concepts to Code

   Demystifying the Math of Geometric Deep Learning - Topology

   ---

3. Manifolds: Utilize differential and Riemannian geometry to represent low-dimensional, continuous, curved spaces or embeddings within higher-dimensional Euclidean spaces. Manifold learning and manifold neural networks are designed to extract intrinsic data representations.

   ---

   📚 Articles - Hands-on Geometric Deep Learning

   Uniform Manifold Approximation & Projection

   Riemannian Manifolds: Foundational Concepts

   Riemannian Manifolds: Hands-on with Hypersphere

   Demystifying the Math of Geometric Deep Learning - Differential Geometry

   ---

4. Point Clouds & Meshes: Represent 3D data often processed through mesh-based or grid-based models, enabling effective analysis of spatial structures.

Graph Neural Networks

Data on manifolds can often be represented as a graph, where the manifold's local structure is approximated by connections between nearby points. GNNs and their variants (like Graph Convolutional Networks (GCNs)) extend neural networks to process data on non-Euclidean domains by leveraging the graph structure, which may approximate the underlying manifold [ref 5, 6, 7].

Types of Graph Neural Networks

• Graph Convolutional Networks (GCNs): GCNs generalize the concept of convolution from grids (e.g., images) to graphs. They aggregate information from a node's neighbors using normalized adjacency matrices and apply transformations to learn node embeddings.

• Graph Attention Networks (GATs): GATs use attention mechanisms to learn the importance of neighboring nodes dynamically. Each edge is assigned a learned weight during aggregation.

• GraphSAGE (Graph Sample and Aggregate):It learns node embeddings by sampling and aggregating features from a fixed-size neighborhood of each node, enabling scalable learning on large graphs.

• Graph Isomorphism Networks (GINs): GINs are designed to be as powerful as the Weisfeiler-Lehman (WL) graph isomorphism test, distinguishing graph structures more effectively.

• Spectral Graph Neural Networks (SGNN): These networks operate in the spectral domain using the graph Laplacian. They use eigenvectors of the Laplacian for convolution-like operations.

• Graph Pooling Networks: They summarize graph information into a smaller representation, similar to pooling in CNNs. They can be categorized into Global and hierarchical pooling

• Hyperbolic Graph Neural Networks: These networks operate in hyperbolic space, which is well-suited for representing hierarchical or tree-like graph structures.

• Dynamic Graph Neural Networks: These networks are designed to handle temporal graphs, where nodes and edges evolve over time.

• Relational Graph Convolutional Networks (R-GCNs):R-GCNs extend GCNs to handle heterogeneous graphs with different types of nodes and edges.

• Graph Transformers: They adapt the Transformer architecture to graph-structured data using attention mechanisms and global context.

• Graph Autoencoders: These are used for unsupervised learning on graphs, aiming to reconstruct graph structure and node features.

• Diffusion-Based GNNs: These networks use graph diffusion processes to propagate information.

PyTorch Geometric [ ref 8] is a widely used library for implementing, training, and evaluating Graph Neural Networks. This library will be discussed in a future article.

---

📌 Future articles will cover Graph Neural Network architecture and theory, along with hands-on applications implemented using PyTorch Geometric.

---

Topological Data Analysis

Topological Data Analysis (TDA) is a methodology that applies concepts from algebraic topology and computational geometry to analyze and extract meaningful patterns from complex datasets. It provides a geometric and topological perspective to study the shape and structure of data. TDA seeks to develop rigorous mathematical, statistical, and algorithmic techniques to infer, analyze, and leverage the intricate topological and geometric structures underlying data, often represented as point clouds in Euclidean or more general metric spaces. [ref 9].

The most common topological domains are

• Simplicial Complexes

• Cellular Complexes

• Hypergraphs

• Combinatorial Complexes

Features

• Understand data shape: TDA focuses on capturing the "shape" or structure of data, including patterns, clusters, loops, and voids that traditional methods might miss.

• Reduce dimension: It enables the extraction of features that summarize the global properties of high-dimensional data while preserving important topological characteristics.

• Discount noise: TDA methods are designed to distinguish signal from noise effectively, providing stable and meaningful insights even in noisy datasets.

• Quantify global properties: TDA quantifies global and intrinsic properties of datasets, such as connectedness, holes, or higher-dimensional features.

Benefits

• Insight into Complex Structures: TDA can reveal features like clusters, loops, and voids in data that are not immediately apparent using standard statistical or machine learning techniques.

• Versatility: It can be applied to any dataset, regardless of the domain, and works effectively on structured, unstructured, and graph-based data.

• Improved Noise Tolerance: Topological features are stable and robust to small perturbations in data, making TDA especially valuable for noisy or incomplete datasets.

• Enhancing Machine Learning: TDA features can be combined with machine learning models to improve predictions, clustering, and classification, especially in tasks where geometry and relationships between data points matter.

• Nonlinearity Detection: TDA captures nonlinear patterns and relationships that may be missed by linear methods like PCA or linear regression.

Recently, TDA has evolved into Topology Deep Learning that encompasses extracting high level abstraction and hierarchical representation from data. For instance, Topological neural networks enable the processing of data, for example via higher-order message-passing schemes on a topological space [ref 10].

---

📌 The most common data structures and algorithms for TDA as implemented in GUDHI library, with C++ implementation and Python interfaces [ref 11].

---

Topological Deep Learning

Topological Deep Learning (TDL) is an emerging area of machine learning that extends traditional deep learning and graph neural networks (GNNs) to operate on higher-order topological structures beyond simple graphs. While GNNs model pairwise relationships (nodes and edges), TDL leverages the mathematics of algebraic topology to capture multi-way interactions (e.g., triangles, tetrahedra) and topological invariants like holes, cycles, and cavities.

TDL is widely considered a subfield of Geometric Deep Learning (GDL). Deep Learning distinguished by the way it defines local neighborhoods.

In geometric models built on continuous manifolds, the neighborhood around a point is typically represented by a local Euclidean space—often formalized as a tangent space or the Lie algebra at the identity of the manifold.

The most common applications of TDL are:

• Molecular and protein modeling (capturing 3D binding pockets, loops).

• Mesh and 3D object analysis (e.g., shape classification, segmentation).

• Sensor and communication networks (detecting coverage holes and cycles).

• Neuroscience (studying brain connectivity and functional cycles).

• Point cloud data (capturing geometric and topological invariants).

• Complex social and financial networks (analyzing group interactions).

Data Manifolds & Groups

Overview

Machine learning models based on manifolds leverage differential geometry [ref 12]. A manifold is essentially a space that, around every point, looks like Euclidean space, created from a collection of maps (or charts) called an atlas, which belongs to Euclidean space. Differential (or smooth) manifolds have a tangent space at each point, consisting of vectors. Riemannian manifolds are a type of differential manifold equipped with a metric to measure curvature, gradient, and divergence [ref 13].

In deep learning, the manifolds of interest are typically Riemannian due to these properties.

It is important to keep in mind that the goal of any machine learning or deep learning model is to predict p(y) from p(y|x) for observed features y given latent features x.

\(p(y)=\int_{\Omega }^{} p(y|x).p(x)dx\)

The latent space x can be defined as a differential manifold embedding in the data space (number of features of the input data).

Definitions

A manifold is a topological space that, around any given point, closely resembles Euclidean space. Specifically, an n-dimensional manifold is a topological space where each point is part of a neighborhood that is homeomorphic to an open subset of n-dimensional Euclidean space. Examples of manifolds include one-dimensional circles, two-dimensional planes and spheres, and the four-dimensional space-time used in general relativity.

Smooth or Differential manifolds are types of manifolds with a local differential structure, allowing for definitions of vector fields or tensors that create a global differential tangent space. A Riemannian manifold is a differential manifold that comes with a metric tensor, providing a way to measure distances and angles.

An exponential map turns an infinitesimal move on the tangent space into a finite move on the manifold. in simple terms, given a tangent vector, v the exponential map consists of following the geodesic in the direction of v for a unit of time.

A logarithm map brings a nearby point back to the tangent plane to define the tangent vector. It consists of finding the geodesic between two points on a manifold and extract is tangent vector.

Curvature measures the extent to which a geometric object like a curve strays from being straight or how a surface diverges from being flat. It can also be defined as a vector that incorporates both the direction and the magnitude of the curve’s deviation.

A geodesic is the shortest path (arc) between two points in a Riemannian manifold. Practically, geodesics are curves on a surface that only turn as necessary to remain on the surface, without deviating sideways. They generalize the concept of a “straight line” from a plane to a surface, representing the shortest path between two points on that surface.

---

Studying data that reside on manifolds can often be done without the need for Riemannian Geometry, yet opting to perform data analysis on manifolds presents three key advantages:

• By analyzing data directly on its residing manifold, you can simplify the system by reducing its degrees of freedom. This simplification not only makes calculations easier but also results in findings that are more straightforward to understand and interpret.

• Understanding the specific manifold to which a dataset belongs enhances your comprehension of how the data evolves over time.

• Being aware of the manifold on which a dataset exists enhances your ability to predict future data points. This knowledge allows for more effective signal extraction from datasets that are either noisy or contain limited data points.

---

Lie groups play a crucial role in Geometric Deep Learning by modeling symmetries such as rotation, translation, and scaling. This enables non-linear models to generalize effectively for tasks like object detection and transformations in generative models. The Special Euclidean Group and Special Orthogonal Group are two examples of Lie groups.

In differential geometry, a Lie group is a mathematical structure that combines the properties of both a group and a smooth manifold. It allows for the application of both algebraic and geometric techniques.

---

Geomstats [ref 14] and PyTorch Geometric [ref 8] are among the most widely used Python libraries for machine learning on data manifolds.

---

📌 Future articles will cover topics such as application of differential geometry, deep learning models on manifolds along with hands-on applications and information geometry.

---

Mesh & Grid Based Models

These models are a special cases of Graph Neural Networks and Geometric Deep Learning applied to 3D spaces.

A mesh is a collection of vertices, edges, and faces that define the shape of a 3D object. Mesh-based learning focuses on directly processing these structures. A grid represents data in a structured, regular manner, such as a 2D/3D pixel or voxel grid.

Mesh-based learning models use triangular, quadrilateral, or polygonal meshes to represent surfaces. It preserves topology and local geometric structure and ideal for irregular surfaces, such as 3D scans, medical imaging, and CAD models [ref 15].

The common designs are

• GNN on meshes for which mesh is treated as a graph

• Spectral models that use the Laplacian of the mesh

• Geometric Deep Learning methods that apply message passing the mesh.

---

Meshes are often associated with simplicial manifolds and are studied as part of Discrete Differential Geometry.

Discrete differential geometry (DDG) studies combinatorial analogs of smooth geometric notions—curvature, geodesics, differential forms, connections—defined on discrete meshes/graphs/complexes so that computations are exact on the discrete object yet faithful to the smooth theory in the limit.

---

A grid represents data in a structured, regular manner, such as a 2D/3D pixel or voxel grid. It uses uniform grids and is suitable for applications where data is naturally aligned with a regular grid. A simplicial manifold is a simplicial object in the category of manifolds.

The common designs are

• Voxel-based Neural Networks converting 3D objects into voxel grids and applies 3D CNNs.

• Implicit Representations representing surfaces as signed distance functions

• Physics-Based Learning simulating physical interactions within a grid.

Graph Theory

Graph theory concerns mathematical graphs used to encode pairwise relations. A graph comprises vertices and edges; a digraph differs from an undirected graph in that each edge is oriented.

Transductive vs. Inductive Graph Networks

Graph Convolutional Networks (GCNs) were reviewed in [ref 16]. A straightforward way to characterize GraphSAGE is by contrasting it with GCNs: Graph Convolutional Networks are transductive models while GraphSAGE is an instance of a inductive model.

In a nutshell:

• Transductive GNNs learn using the entire target graph (including test nodes/edges) during training and predict labels for those specific nodes/edges. As a result, they tend to overfit the training set and struggle to generalize to unseen nodes, edges, or entirely new graphs. Their message passing and aggregation schemes are typically straightforward. Our PyTorch Geometric implementation of the Graph Convolutional Network follows this transductive setting.

• Inductive GNNs learn a function (message-passing/aggregation rules) that transfers to unseen nodes or entirely new graphs at test time. They learn from existing (training) and new, unseen nodes, links or graphs (inference). There is no need for these models to store and node/link embeddings. For example, the graph SAGE model that leverages PyTorch Geometric is strictly inductive.

Message Passing & Aggregation

Message passing and aggregation underpin GNNs; we summarize and illustrate them here

Fig. 1 Illustration Message-passing and Aggregation in Graph Neural Network

Fig. 2 Illustration of the 3 steps in processing messages through a graph neural layer

As shown in Figure 2, a neural layer processes the graph in three stages:

1. Message collection: Node 0 gathers information from its neighboring nodes.

2. Aggregation: The collected messages are combined using operations such as sum, mean, or pooling.

3. Feature update: The node’s own features are updated based on the aggregated information.

The generic message passing model is the most expressive design making it suitable for complex modeling such as dynamic systems, proteins generation at a high computational cost and memory consumption

\(\begin{matrix} \psi\left( h_{j}^{(k)} \right) = W. h_{j}^{(k)}+ b \ \ \ Trainable \ function \\ \mu_{ij}=\psi \left( h_{i}^{(k)}, h_{j}^{(k)} \right) \ \ \ \ Feature \ vector \ nodes \ i \ j\\ h_{i}^{(k+1)} =\Phi \left( \bigwedge_{ij} \left( h_{j}^{(k)}, \bigoplus_{j\in N_{i}}\psi\left( h_{i}^{(k)}, h_{j}^{(k)} \right) \right) \right) \\ \bigwedge: Concatenator \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \\ \bigoplus: Aggregator \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \end{matrix}\)

• µ_ij is a feature vector that describes the interaction of node i with node j.·

• N_i is the 1-hop neighborhood of i (excluding i)

• w_ij are unlearned weights, usually depending only on the local graph topology and which encode the connection strength between pairs of nodes.

Differential Geometry

Differential geometry studies smooth spaces (manifolds) by doing calculus on them, using tools like tangent spaces, vector fields, and differential forms. It formalizes intrinsic notions such as curvature, geodesics, and connections—foundations for modern physics and for algorithms in graphics, robotics, and machine learning on curved data [ref 17].

Vector, Tensor Fields & Differential Forms

A vector is an arrow pointing through space. A covector is an object that measures vectors moving through space. A covector is a measurement dual to that arrow. While a vector is visualized as an arrow, a covector is best visualized as a set of parallel, evenly spaced hyperplanes(lines in 2D, planes in 3D).

A vector field assigns a single vector to every point in a space. You can visualize it as an infinite collection of arrows, where each arrow’s base sits at a specific point, its direction points where something is moving or pulling, and its length represents intensity.

Tensors are multidimensional arrays of numbers that obey specific transformation laws when changing coordinate systems. They track complex, multi-directional relationships.

A tensor field is a generalization of a vector field. Instead of assigning a simple arrow to every point, it assigns a tensor to every point.

Intuitively, the directional derivative measures how a function changes as you move in a specific direction. The directional derivative

• is large and positive, the function is increasing quickly in that direction.

• is zero, the function doesn’t change in that direction.

• is negative, the function is decreasing in that direction.

Given a metric g, a covector ω is defined as a linear map

\(\begin{matrix} \omega: \ V \to \mathbb{R} \ \ \ \ \ \ \\ \omega: \ v \mapsto g(v, .) \end{matrix}\)

A vector fields v is defined as

\(v=v^{i}\frac{\partial }{\partial x^{i}} =\sum_{i=1}^{n}v^{i}\frac{\partial }{\partial x^{i}}\)

A differential 1-form α is defined as

\(\alpha = \alpha_{i}dx^{i}=\sum_{i=0}^{n}\alpha_{i}dx^{i} \ \ \ \ \ dx^{i}\left( \frac{\partial }{\partial x^i} \right) = \delta ^{i}_{j}=\begin{matrix} 1 \ \ if \ i=j \\ 0 \ \ if \ i\neq j \end{matrix}\)

Given a differentiable function f, a vector v in Euclidean space Rⁿ and a point x on manifold, the directional derivative in v direction at x is defined as:

\(\triangledown _{v} f(x)= v_{i}\frac{\partial f}{\partial x_{i}}(x) =\sum_{i=1}^{n}v_{i}\frac{\partial f}{\partial x_{i}}(x) \ with \ f: \mathbb{R}^{n} \rightarrow \mathbb{R}\)

The gradient can be expressed intrinsically using the Riemannian metric g

\(\left( \nabla f \right)^{i}=g^{ij}\sum_{j=0}^{n}\frac{\partial f}{\partial x^{j}}\)

Geodesic, Metric & Maps

An exponential map is a map from a subset of a tangent space of a Riemannian manifold while the logarithm map assign a tangent vector to a point on the manifold as illustrated below [ref 18, 19].

Fig. 3 Visualization of tangent plane, geodesic, exponential and logarithm maps on Hypersphere

The Riemannian metric g is an operator that assigns an inner product to the tangent space at every single point on the manifold. The Riemannian Metric is a smoothly varying, positive-definite inner product on each tangent space. It allows the calculation of length, angle and volume associated with vector fields.

In a Riemannian manifold, the metric can be used to

• Estimate kernel density

• Approximate the encoder function of an auto-encoder

• Represent the vector space defined by classes/labels in a classifier

We cannot directly compare vectors from two different tangent spaces because they live in entirely different vector spaces. The Levi-Civita connection solves this issue by providing a consistent rule for directional differentiation and parallel transport.

Christoffel symbols answer the question: How much does the basis vector of coordinate x(I) tilt or stretch into the direction of coordinate x_k when moving along x_i?

The Christoffel symbols are computed from metric tensor g expressed in a chosen coordinate system, and its partial derivatives with respect to those coordinates, using the Einstein summation

The geodesic is computed through a differential equation:

\(\nabla _{\frac{d \gamma}{dt}}\frac{d \gamma}{dt}=0\)

The formalism for the Christoffel symbols is

\(\Gamma_{ij}^{k}\)

• Index i: The direction x(i) you are moving along the manifold

• index j: The basis vector x(j) you are tracking

• index k: The component of the change

The Christoffel Symbol can be expressed, intrinsically using the Riemannian metric.

\(\Gamma^{k}_{ij}=\frac{1}{2}g^{kl}\left( \frac{\partial g_{jl}}{\partial x^{i}} + \frac{\partial g_{il}}{\partial x^{j}}-\frac{\partial g_{ij}}{\partial x^{l}}\right)\)

The geodesic differential equation can be expressed using the Christoffel symbols

\(\frac{d^{2}x^{k} }{dt^{2}}+\Gamma^{k}_{ij}\frac{d x^{i}}{dt}\frac{d x^{j}}{dt}\)

Given a differentiable function f on a domain a manifold M of dimension d is defined by:

\(\mathit{M}=f(\Omega) \ \ \ with \ f: \Omega \subset \mathbb{R}^{d}\rightarrow \mathbb{R}^{d}\)

Given a Riemannian manifold M with a metric tensor g, the geodesic length L of a continuously differentiable curve f: [a, b] -> M

\(L(f)=\int _a^b \sqrt {g_{f(t))} (\frac{\mathrm{d} f}{\mathrm{d} x}(t), \frac{\mathrm{d} f}{\mathrm{d} x}(t))}dt\)

Lie Groups & Algebras

Symmetry

Lie groups play a crucial role in Geometric Deep Learning by modeling symmetries such as rotation, translation, and scaling. This enables non-linear models to generalize effectively for tasks like object detection and transformations in generative models [ref 20, 21].

Symmetry is the organizing principle behind geometric deep learning. It tells you what should not change (invariance) or how things should change (equivariance) when you transform the input.

• Symmetry (Webster dictionary): The property of remaining invariant under certain changes (as of orientation in space, of the sign of the electric charge, of parity, or of the direction of time flow) —used of physical phenomena and of equations describing them

• Invariance: The output doesn’t change when the input is transformed. Example: a cat classifier still predicts “cat” for an upside-down photo.

• Equivariance: The output changes in a predictable way under the same transformation. Example: rotate an image, and the predicted segmentation mask rotates accordingly.

Lie groups are mathematical structures that combine the properties of groups and smooth manifolds, meaning they support both algebraic and differentiable operations. Named after the Norwegian mathematician Sophus Lie, these groups play a fundamental role in continuous symmetries, particularly in physics and geometry.

A Lie algebra is the tangent space at the identity element of a Lie group, equipped with a special operation called the Lie bracket, which encodes the infinitesimal structure of the group.

Key Properties:

• Continuity and Differentiability: Lie groups are differentiable manifolds, allowing the application of calculus.

• Group Structure: They satisfy closure, associativity, identity, and inverse properties.

• Infinitesimal Generators: The corresponding Lie algebra describes local (infinitesimal) symmetries, making them useful for transformations.

Examples of Lie Groups

General Linear Group of invertible matrices

The General Linear Group, denoted as GL(n,R) (or GL(n,C) for complex values) is the set of all invertible n×n matrices with real entries, equipped with matrix multiplication as the group operation. It is a Lie group because it forms a smooth differentiable manifold and supports continuous transformations

Symmetric Positive Definite Group

The Symmetric Positive Definite (SPD) Group, denoted as SPD(n), is the Lie group of all n×n real symmetric positive definite matrices. It plays a crucial role in statistics, machine learning, optimization, and Riemannian geometry.

Special Orthogonal Group

The Special Orthogonal Groups SO(2) (resp. SO(3)) are the Lie groups that represents rotations in 2-dimensional (resp. 3-dimensional) space. It is widely used in robotics, physics, computer vision, and geometric deep learning to describe rigid body rotations without scaling or reflection.

Special Euclidean Group

The Euclidean group is a subset of the broader affine transformation group. It contains the translational and orthogonal groups as subgroups. The Special Euclidean Group is a subgroup of Euclidean group. In robotics and physics, an element of SE(3) is often referred to as a rigid transformation, since it describes a rotation + translation without scaling or deformation

GL(n,R)

\(\begin{matrix} 2\ast 2 \ \ Inv \ Matrix \ on \ \mathbb{R} & GL(2, \mathbb{R}) =\left\{ \begin{bmatrix} a & b \\ c & d \end{bmatrix} \ a, b, c, d \in \mathbb{R}, \ \ ad-bc \neq 0 \right\} \\ 2\ast2 \ \ Inv \ Matrix \ on \ \mathbb{C} & GL(2, \mathbb{C})=\left\{ \begin{bmatrix} a & b \\ c & d \end{bmatrix} \ a, b, c, d \in \mathbb{C}, \ \ ad-bc \neq 0 \right\} \\ 3\ast3 \ \ Inv \ Matrix \ on \ \mathbb{R} & GL(3, \mathbb{R})=\left\{ \begin{bmatrix} a & b & c\\ d & e & f \\ g & h & i \end{bmatrix} \ a, b, c, d, e, f, g, h, i \in \mathbb{R}, \ det \neq 0 \right\} \\ \end{matrix}\)

SPD

\(SPD(n)=\left\{ S\in \mathbb{R}^{n\ast n} \ | \ \ S^{T}=S, \ [1] \ \ \ \ x^{T}Sx \gt 0, \ \ \forall x\neq 0 \ \ [2] \right\}\)

1. Specifies the symmetry condition

2. Ensures the positive definite condition

SO2, SO3

\(\begin{matrix} 2D \ Rotation & SO(2)=\left\{ R \in \mathbb{R}^{2 \ast 2} \ | \ R.R^{-1}=I_{d}, \ [1] \ \ \ det(R)=1 \ \ [2] \right\} \\ 3D \ Rotation & SO(3)=\left\{ A \in \mathbb{R}^{3 \ast 3} \ | \ R.R^{-1}=I_{d}, \ [1] \ det(R)=1 \ \ [2]\right\} \end{matrix}\)

1. Enforces orthogonally for which the 2 or 3 unit of rotations are mutually perpendicular

2. Ensures rotation and preserves orientation

SE3

\(SE(3)=\left\{ \begin{vmatrix} R & t \\ 0 & 1 \end{vmatrix} \in \mathbb{R}^{4 \ast 4} \ \ | \ R \in SO(3), t \in \mathbb{R}^{3} \right\}\)

Lie Algebra

A Lie algebra is a mathematical structure used to study continuous symmetries, consisting of a vector space equipped with a bilinear operation called the Lie bracket, which satisfies

• antisymmetry ([X,Y]=−[Y,X][X,Y]=−[Y,X])

• Jacobi identity ([X,[Y,Z]]+[Y,[Z,X]]+[Z,[X,Y]]=0[X,[Y,Z]]+[Y,[Z,X]]+[Z,[X,Y]]=0)

Topology

Topology describes shape properties unchanged by continuous deformations, provided no holes are opened or closed and no tearing or gluing occurs.

Terminology

• Betti number: A key topological invariant that count the number of certain types of topological features in a space. It provides a concise summary of the space’s shape and structure, especially when dealing with high-dimensional or complex datasets (dimension 0: Connected components, dimension 1: holes, loops, dimension 2: 3D holes and voids,)

• Category Theory: A branch of mathematics that deals with abstractions of mathematical structures and the relationships between them. It focuses on how entire structures relate and transform via morphism

• Cellular (or CW, Cell) Complex: A topological space obtained as a disjoint union of topological disks (cells), with each of these cells being homeomorphic to the interior of a Euclidean ball. It is a general and flexible structure used to build topological spaces from basic building blocks called cells, which can be points, lines, disks, and higher-dimensional analogs.

• Cochain (or k-Cochain): A signal, a function from a combinatorial complex of rank k to a real vector.

• Cochain Map: A map between cochain spaces that use incidence matrix as transformation.

• Cochain Space: The space of cochains.

• Combinatorial Complex: A structured collections of simple building blocks (like vertices, edges, triangles, etc.) that models the shape of data in a way that is amenable to algebraic and computational analysis.

• Filtration: A nested sequence of topological spaces (typically simplicial or cell complexes) built from data, where each space includes the previous one. Each of these topological spaces, represents the data’s topology at a certain scale or resolution.

• Hausdorff Space: Topological space for which points are separable and can be included into their open sets that do not intersect.

• Hypergraph: A generalization of a traditional graph where edges (called hyperedges) can connect any number of nodes, not just two. While edge of a graph models a binary relationship, a hypergraph models higher-order relationship among group of nodes

• Homeomorphism. A morphism between two topological spaces is a bijective map for which a function and its inverse are continuous.

• Homotopy: A fundamental idea in topology about continuously deforming one shape into another — like stretching, bending, or twisting — without tearing or gluing.

• Incidence Matrix: An encoding of relationship between a k-dimensional and (k-1) -dimensional simplices (nodes, edges, triangle, tetrahedron,..) in a simplicial complex.

• Persistent Homology: A method from TDA that captures the shape of data across multiple scales. It tracks topological features—such as connected components, loops, and voids—as they appear and disappear in a growing family of simplicial complexes.

• Point Cloud: A collection of data points defined in a 3D space. These points represent the external surface of an object. A point cloud has spatial coordinate (x, y, z) and internal representation but does not contain any information about how points are connected (edges, faces,).

• Pooling Operation: A pooling operation is a mechanism for reducing the complexity of topological input spaces (like graphs, simplicial complexes, or cell complexes) while preserving essential topological features.

• Poset: A Partially Ordered Set which pear in causal structures, stratified space and singular manifolds. A poset is a set P equipped with a partial order <= satisfying (Reflexivity, Anti-symmetry, Transitivity).

• Push-forward Operation: A way of transferring functions or signals defined on one topological domain to another, typically along a continuous or structure-preserving map. It’s rooted in differential geometry and algebraic topology.

• Relation: A subset of a set of vertices characterized by a cardinality (e.g., 2 for binary relation).

• Sheave: A mathematical tool that helps track local data on a space and describes how it glues together consistently across overlapping regions. Sheaves generalize persistent homology by allowing richer types of data.

• Simplex: A generalization of edge-bound shapes such as triangles, tetrahedron

• Simplicial Complex: A generalization of graphs that model higher-order relationships among data elements—not just pairwise (edges), but also triplets, quadruplets, and beyond (0-simplex: node, 1-simplex: edge, 2-simplex: Triangle, 3-simplex: Tetrahedron, ).

• Set-type Relation: A type of relation which existence is not implied by another relation within the graph (e.g., hypergraphs).

• Topological Data: A set of feature vectors supported on relations in a topological space.

• Topological Data Analysis (TDA): A framework that uses concepts from topology to study the shape and structure of data. It captures high-dimensional patterns, such as clusters, loops, and voids, by building simplicial complexes and computing topological invariants like Betti numbers. TDA is especially useful for revealing robust geometric features that persist across multiple scales in noisy or complex datasets.

• Topological Deep Learning (TDL). An approach that incorporates ideas from topology, particularly algebraic topology, into deep learning. It focuses on the intrinsic, global properties of data that remain unchanged under continuous deformations, like stretching or bending, rather than the local structure of data.

• Topological Group: A mathematical object that is simultaneously:

  • A group (an algebraic structure with an associative binary operation, identity, and inverses), and

  • A topological space (a set with a notion of continuity, open sets, etc.)

• Topological Space: This is a set X with a collection of subsets T, known last open sets of X that satisfy:

  • Empty Set belongs to T

  • X belongs to T

  • Union of open sets is open

  • Intersection of open sets is open

• Unspooling Operation: The counterpart to the pooling process. It reconstructs or expands a coarsened topological structure back to a finer-resolution space — typically to enable decoding, up-sampling, or reconstruction in a neural network.

• Vietoris-Rips: A filtration captures how topological features (like clusters, loops, voids) appear and disappear as the scale changes. This persistence is visualized in barcodes or persistence diagrams. The Vietoris–Rips filtration is one of the most common constructions in TDA for extracting topological features from a point cloud.

Simplicial Complexes

Simplicial Complex is the simplest form of topological complex that can be lifted from a graph.

Graphs and simplicial complexes are quintessential examples of non-Euclidean structures, though the reasoning varies slightly depending on whether you are looking at them combinatorially or geometrically.

This distinction is critical for how information is “passed” between elements:

• Graphs: Message passing happens only between vertices across edges.

• Simplicial Complexes: You can have message passing between edges (1-cells) or faces (2-cells). This allows the model to respect the homology of the data—identifying holes, voids, and higher-dimensional connections that a simple graph would miss.

Cell Complexes

A cell complex (like a simplicial or combinatorial complex) is built from cells of different dimensions:

• 0-cells: vertices

• 1-cells: edges

• 2-cells: faces

• 3-cells: volumes, etc.

A triangle can be seen as a 2D cell complex built from one 2-cell (the interior), three 1-cells (edges), and three 0-cells (vertices).

Hypergraphs

A hypergraph is a generalization of a graph where a hyperedge can connect any number of nodes or vertices. Similarly to Cell and Simplicial Complexes, a node and a hyperedge are said to be incident if the vertex is a member of the hyperedge.

Simplicial Complexes has more constraints (more structured) and therefore can be easily converted into a hypergraph.

Each simplex is then converted into a hyperedges as follow:

• edge → 2-hyperedge

• triangle → 3-hyperedge

• tetrahedron → 4-hyperedge

• …

One benefit of transforming a Hypergraph into a simplicial complex is the ability to leverage the established definitions and implementations of the Up, Down, and Hodge Laplacians available for simplicial complexes.

Laplacians

The Laplacian matrices generalize the graph Laplacian to higher-dimensional simplicial complexes. Their purposes are

• Generalization of the diffusion & message passing scheme

• Capture higher-order dependencies (Boundaries and Co-boundaries)

• Support convolution operations in Spectral Simplicial Neural Networks

• Encoding of topological features such holes or cycles.

The Upper-Laplacian (a.k.a. Up-Laplacian) is a key operator in simplicial homology and Hodge theory, defined for simplicial complexes to generalize the notion of graph Laplacians to higher-order structures.

The Down-Laplacian (a.k.a. Low-Laplacian) is similar to the Up-Laplacian except that the boundary operator projects k-simplices to (k-1) faces.

The Hodge Laplacian for simplicial complexes is a generalization of the graph Laplacian to higher-order structures like triangles, tetrahedra, etc. It plays a central role in topological signal processing, Topological Deep Learning, and Hodge theory.

Up-Laplacian

\(\Delta_{k}^{up}=\partial_{k+1}\partial_{k+1}^{T} \ \ \ \ \ \partial_{k+1} \ is Boundary \ operator: \ \ k+1 \rightarrow k\)

Down-Laplacian

It quantifies how k-simplex is influence by its lower-dimensional faces with the following mathematical definition:

\(\Delta_{k}^{down}=\partial_{k+1}\partial_{k}^{T} \ \ \ \ \ \partial_{k} \ is Boundary \ operator: \ \ k \rightarrow k-1\)

Hodge-Laplacian

For a simplicial complex, the Hodge Laplacian Δk at dimension k is defined as:

\(\Delta_{k} = \Delta_{k}^{up} + \Delta_{k}^{down} = \partial_{k+1}\partial_{k+1}^{T} +\partial_{k}\partial_{k}^{T} \)

Taking into account the existing constraints on the Up- and Down-Laplacian, the formulation of the Hodge-Laplacian can be simplified as follows:

Persistent Homology

Analyzing the overall geometric structure of data on a low-dimensional smooth manifold is notoriously difficult. Persistent homology simplifies this by providing a reliable way to distinguish real features from background noise—allowing us to detect loops, cycles, and singularities based purely on the data’s underlying distance relationships.

Persistent homology is the process of capturing the global geometric and topological structure of complex data and identifying and tracking features such as loops, cycles, and voids across multiple scales, as illustrated below:

The filtration method the most commonly used in Topological Data Analysis is the Vietoris-Rips filtration that can arbitrary be broken down into 3 steps

1. Growing Spheres: We place a ball of radius ε around each point in space (point cloud).

2. Building Complexes: We increase ε 0 → infinity, to expand these balls. When two balls intersect, a line segment (1-simplex) connects them. When three intersect, a triangle (2-simplex) forms, creating a simplicial complex.

3. Birth and Death: The topological features slowly appear and eventually get filled in.

   • Birth: The radius ε at which a specific topological hole first appears.

   • Death: The radius ε at which that same hole is filled in or merges into a larger component.

A chain complex C is a sequence of R modules (e.g. vector spaces) and linear maps ∂ called boundary operators, such as

\(\begin{matrix} ...\to C^{k+1} -[\partial^{k+1}] \to C^{k}-[\partial^{k}] \to C^{k-1} \to ...\\ \partial^{k}\circ \partial^{k+1} = 0 \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \end{matrix}\)

A cochain complex C is a sequence of R modules (e.g. vector spaces) and linear maps d called coboundary operators, such as

\(\begin{matrix} C^{0}\to ...\to C^{k-1} -[\textbf{d}^{k-1}] \to C^{k}-[\textbf{d}^{k}] \to C^{k} \to ...\\ \textbf{d}^{k+1}\circ \textbf{d}^{k} = 0 \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \end{matrix}\)

A filtration is defined as a sequence of topological Xi spaces of increasing size

\(F =X_{0} \subseteq X_{1} \subseteq \ ... \ \subseteq X_{n}\)

The dimension of the homology group is defined as

• k=0: Connected components

• k=1: Loops

• k=2: Cavities

• …

The k-th homology groups Hk for each of the topological space Xi and its induced linear map i associated to the inclusion.

\(\left\{ H_{k}(X_{0}), H_{k}(X_{1}), \ .... \ ,H_{k}(X_{h}) \right\} \ \ \ \ i_{p\to q}: \ H_{k}(X_{p})\to H_{k}(X_{q}) \ \ \ p \le q \)

Information Geometry

Information geometry applies differential-geometric tools to probability and statistics. It treats families of distributions as statistical manifolds—Riemannian manifolds whose points are probability laws—often equipped with a pair of conjugate affine connections; in many cases these structures are dually flat.

Divergences

Kullback–Leibler (KL) divergence—a.k.a. relative entropy—measures how one probability distribution departs from a reference/target distribution. It’s not a metric (it’s asymmetric and fails the triangle inequality).

Jensen–Shannon (JS) divergence—the symmetrized, KL-based measure—quantifies similarity between two distributions and has a closed form for Gaussian families.

Bregman divergence (or Bregman “distance”) is a difference measure generated by a strictly convex function. When points represent probability distributions (e.g., model parameters or datasets), it serves as a statistical divergence. A basic example is squared Euclidean distance.

Kullback–Leibler (KL)

Given a probability distribution P differs and a reference distribution Q over the same probability space, the Kullback-Leibler divergence is computed as

\(D_{KL}(P || Q)=\int_{-\infty }^{+\infty }p(x)log\left( \frac{p(x)}{q(x)} \right)\)

Jensen–Shannon (JS)

It measures the similarity between two probability distributions and its square root satisfies the triangle inequality. Given a probability distribution P differs and a reference distribution Q over the same probability space, the JS divergence is computed as

\(D_{JS}\left( P || Q \right)= \frac{1}{2}\left( D_{KL} (P || M)+D_{KL} (Q || M)\right) \ \ \ \ M=\frac{P+Q}{2}\)

Bregman

Given a continuously differentiable function F over a convex set, the Bregman divergence is computed as the difference between F(x) and the first-order of Taylor expansion at F(y)

\(\begin{matrix} F: \Omega \to \mathbb{R} \\ D_{F}=F(x)-F(y) -\left\langle \nabla F(y), x-y \right\rangle \end{matrix}\)

Fisher Metric

A Fisher-Riemann manifold is a differentiable manifold MM whose points correspond to probability distributions p(x∣θ) from a statistical model parameterized by θ∈Θ⊂Rn, and which is equipped with the Fisher information metric [ref 22].

The Fisher information metric or Fisher-Rao metric is a Riemannian metric defined on the parameter space of a family of probability distributions, constructed using the Fisher information matrix. It remains invariant under reparameterization and is the only Riemannian metric that aligns with the model’s notion of information about its parameters [ref 23].

From the Riemannian metric, g

\( g_{j k}(\theta)=\int_X \frac{\partial \log p(x, \theta)} {\partial \theta_j}\frac{\partial \log p(x, \theta)} {\partial \theta_k} p(x, \theta) d x \)

The Fisher information for a continuous probability density function p is defined as

\(I_{X}(\theta)=g(\theta)= \int_{X}^{.} \left( \frac{\partial \ log \ p(x|\theta)}{\partial \theta} \right)^2.p_{\theta}(x).dx \)

and for discrete distributions,

\(I_{X}(\theta)=\sum_{x \in X}^{}\left( \frac{\partial \ log \ p(x|\theta)}{\partial \theta} \right)^2 . p_{\theta}(x)\)

Under mild regularity conditions, the Fisher Information metric can be re-written

\(I_{X}(\theta)=g(\theta) = \int_{X}^{.} \frac{\partial^2 log \ p(x|\theta)}{\partial \theta^2}.p_{\theta}(x).dx\)

Closed-form Statistical Manifolds

Statistical manifolds are smooth families of probability distributions represented as geometric, parameterized spaces. The canonical Riemannian metric related to the distribution parameters is known as the Fisher-Rao metric or Fisher Information Matrix.

While the geometric properties of statistical manifolds —such as exponential and logarithm maps or distance—are often intractable, univariate distributions typically allow for closed-form or analytical expressions of these properties.

Exponential Distribution

Given a rate parameter θ > 0 the probability density function of the exponential distribution is

\(p(x|\theta)= \begin{matrix} \theta e^{-\theta x} & x > 0 \\ 0 \ \ \ \ \ \ \ & x \leq 0 \end{matrix} \)

The Riemannian metric associated with the exponential distribution with

\(g(\theta )= \frac{1}{\theta^{2}}\)

Geometric Distribution

Given the probability of success p at each trial, the probability of success after k trials is

\( p(X=k)=p(1-p)^{k-1} \ \ \ \ k \in 1, 2, ... \)

Fisher Information Metric

\(g(p)=\frac{1}{p^{2}(1-p)}\)

Poisson Distribution

Given a Poisson distribution with the expectation λ (rate parameter) events for a specific interval, the probability of k events in the same interval is

\(p(x=k|\lambda)=\frac{e^{-\lambda}\lambda^{k}}{k!} \ \ \ \ \ k \in \left\{0, 1, 2, .., \lambda \right\}\)

Fisher Information Metric

\(g(\lambda)=\frac{1}{\lambda} \)

Binomial Distribution with Fixed Draw

Given a probability distribution of success p in a sequence of n independent experiments.

\(p(X=k)=\binom{n}{k}p^{k}(1-p)^{n-k} \ \ \ \ k={0, 1, ... n} \ \ \ \ n \ constant \)

Fisher Information Metric

\(g(p)=\frac{n}{p(1-p)} \ \ \ \ \ n \ constant \)

Frameworks & Libraries

There are numerous open-source Python libraries available, with a variety of focuses not exclusively tied to machine learning or generative modeling:

• diffgeom: A PyPi project aimed at symbolic differential geometry. Detailed information is available at PyPi Diffgeom.

• SymPy: A more comprehensive library for symbolic mathematics, useful for studying topology and differential geometry. Documentation is accessible at SymPy

• Geomstats: Designed for performing computations and statistical analysis on nonlinear (Riemannian) manifolds (Github Geomstats).

• PyG: PyTorch Geometric is a library built on PyTorch dedicated to geometric deep learning and graph neural networks (Pytorch Geometric).

• PyRiemann: A package based on scikit-learn provides a high-level interface for processing and classification of multivariate data through the Riemannian geometry of symmetric positive definite matrices (pyRiemann: Machine learning for multivariate data with Riemannian geometry).

• PyManOpt: A library for optimization and automatic differentiation on Riemannian manifolds (PyManOpt).

• GUDHI: The library is a generic open source C++ library with a Python interface for Topological Data Analysis and Higher Dimensional Geometry Understanding (GUDHI - Geometry Understanding in Higher Dimensions).

• TopoX: TopoX are a set of Python packages for meeting diverse computational needs on topological domains. Topological domains are the natural mathematical structures representing relations between the components of a dataset (TopoX Documentation)

Applications

Classifying applications based on their association with specific models can be challenging, as multiple approaches may offer accurate representations and reliable predictions. Here are some examples:

• Community detection: Identify clusters or communities in social networks (Graph Neural Networks).

• Recommendations: Suggest products, services or connection using graph-based similarity (Graph Neural Networks).

• Drug Discovery: Predict molecular properties, drug-target interactions, or generate novel compounds (Graph Neural Networks).

• Protein Structure Prediction: Model amino acid interactions to predict 3D protein folding and stability (Graph Neural Networks).

• Gene Networks: Analyze gene regulatory interactions or detect disease pathways (Graph Neural Networks).

• 3D Shape Analysis: Analyze and process point clouds or 3D meshes (3D Mesh model),

• Action Recognition: Use graphs to model relationships between body joints or objects in video sequences (Graph Neural Networks).

• Fraud Detection: Detect fraudulent activities by modeling transaction networks (Graph Neural Networks).

• Portfolio Optimization: Represent and analyze stock correlations using graphs (Graph Neural Networks).

• Electronic Circuit Design: Optimize circuit layouts represented as graphs (Graph Neural Networks).

• Risk Management: Model credit risk and customer relationships in financial networks (Graph Neural Networks).

• Disease Prediction: Model patient similarities and interactions for personalized medicine (Graph Neural Networks).

• Patient Care Networks: Represent patient-doctor or hospital networks to optimize care pathways (Graph Neural Networks).

• Drug Repurposing: Identify new therapeutic uses for existing drugs using graph-based analysis (Graph Neural Networks).

• Physics-Informed Learning: Modeling physical systems using graphs and manifolds (Manifold learning).

• Disease Prediction: Model patient similarities and interactions for personalized medicine (Graph Neural Networks).

• Patient Care Networks: Represent patient-doctor or hospital networks to optimize care pathways (Graph Neural Networks).

• Drug Repurposing: Identify new therapeutic uses for existing drugs using graph-based analysis (Graph Neural Networks).

• Knowledge Graphs: Improve semantic search, question answering, and reasoning by embedding nodes and edges (Graph Neural Networks).

• Collaboration Networks: Model co-authorship or citation networks for analyzing research trends (Graph Neural Networks).

• Astronomy: Analyze large-scale cosmic networks and galaxy clusters (Graph Neural Networks).

• Power Grid Management: Monitor and optimize energy distribution networks (Graph Neural Networks).

• Text Classification: Represent relationships between words, sentences, or documents as graphs for classification (Graph Neural Networks).

• Trajectory Optimization: Manifold learning helps model and optimize robot trajectories, especially in constrained spaces like joints or environments (Manifold learning).

• Pose Estimation: Manifold models are used to represent rotations, translations, and orientations in 3D space (e.g., SO(3), SE(3) groups) (Manifold learning).

• 3D Shape Analysis: Used for shape recognition, segmentation, and alignment (Manifold learning).

• Physics Simulations: Manifolds represent physical systems like fluid dynamics or electromagnetic fields for improved simulation and understanding (Manifold learning).

• Manifold-based Data Augmentation: Generative models use learned manifolds to create augmented data samples that adhere to the original data distribution (Manifold learning).

• Brain Signal Analysis: EEG/MEG data is modeled on manifolds to uncover latent patterns related to neurological states (Manifold learning).

• Matrix Factorization: Manifold-based optimization is applied to problems like low-rank matrix approximation in recommendation systems (Manifold learning).

• Topology and Geometry Analysis: Persistent homology and topological data analysis rely on manifold representations for analyzing complex data structures (Topological data analysis).

• Manifold-Aware Transformers: Emerging methods adapt transformer architectures to manifold-structured data (Topological data analysis).

• Hyperbolic Embeddings: Effective for modeling hierarchical or tree-like data structures (e.g., taxonomies, organizational hierarchies) (Topological data analysis).

References

1. Limitations of Deep Neural Networks - S. Tsimenidis - 2020

2. Geometric and spectral limitations in generative adversarial networks - K. Mahyar - Rutgers University Libraries, 20201

3. Geometric deep learning: going beyond Euclidean data - M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, P. Vandergheynst - 2017

4. Geometric foundations of Deep Learning - M. Bronstein - Medium, 2021

5. A Practical Tutorial on Graph Neural Networks I. Ward, J. Joyner, C. Lickfold, Y. Guo, 2021

6. A Comprehensive Introduction to Graph Neural Networks - A. Awan - Datacamp, 2021

7. Graph Neural Networks: A gentle introduction A. Persson - YouTube, 2022

8. Pytorch Geometric - PyG Documentation, 2024

9. An introduction to Topological Data Analysis: fundamental and practical aspects for data scientist - F. Chazal, B. Michel - INRIA, FR, 2021

10. Position: Topological Deep Learning is the New Frontier for Relational Learning - T. Papamarkou et all, 2024

11. GUDHI - Geometry Understanding in Higher Dimensions - Documentation - INRIA, FR, 2024

12. Differential Geometric Approaches to Machine Learning - A. Pouplin, PhD Thesis - Technical University of Denmark, 2023

13. Differential geometry for generative modeling - S. Hauberg, 2025

14. Github geomstats Documentation, 2023

15. An Introduction to Deep Learning on Meshes - Practical course - R. Hanocka, H-T Liu, 2022

16. Taming PyTorch Geometric for Graph Neural Networks - Hands-on Geometric Deep Learning - 2025

17. Introduction to Differential Geometry - ETH Zurich

18. Differential Geometric Structures W. Poor - Dover Publications, New York 1981

19. Introduction to Smooth Manifolds J. Lee - Springer Science+Business media New York 2013

20. Introduction to Lie Groups and Lie algebras - A Kirillov. Jr - SUNNY at Stony Brook

21. Taming Symmetry: A Dive into Lie Groups with Python - Hands-on Geometric Deep Learning - 2025

22. Shape Your Models with the Fisher-Rao Metric - Hands-on Geometric Deep Learning - 2025

23. What is Fisher Information? - YouTube - Ian Collings 2022

Q & A

1. What are the two most widely used Python libraries for Geometric Deep Learning?

2. Can you name four different types of Graph Neural Networks?

3. What are the advantages of using Topological Data Analysis?

4. How does a differential (smooth) manifold differ from a Riemannian manifold?

5. Between mesh-based and grid-based learning models, which has the higher computational cost?

Answers

Paper Review

This section focuses on news and reviews of papers pertaining to geometric deep learning and its related disciplines.

Paper review Intrinsic and extrinsic deep learning on manifolds Y. Fang, i. Ohh, V. Gupta, L. Lin 2023

---

This paper introduces two types of general deep neural network architecture on manifolds:

Extrinsic deep neural network on manifolds

This architecture maintains the geometric characteristics of manifolds by employing an equivariant embeddings of a manifold into the Euclidean space.

This method is applied in regression models or Gaussian processes on manifolds, where the idea is to construct the neural network based on the manifold's representation after embedding, while still maintaining its geometric properties. By adopting this strategy, it becomes possible to utilize stochastic gradient descent and backpropagation techniques from Euclidean space. This results in enhanced accuracy compared to conventional machine learning algorithms like SVM, random forest, etc.

Intrinsic deep neural network on manifolds

The objective is to embed the inherent geometric nature of Riemannian manifolds using exponential and logarithmic maps. This framework, which projects localized points from a Riemannian manifold onto a single tangent space, proves beneficial when embeddings cannot be determined. Each localized tangent space (or chart) is mapped (via exp/log functions) onto a neural network. This architectural approach achieves higher accuracy compared to deep models in Euclidean space and the Extrinsic architecture.

These two frameworks are assessed based on their performance in

1. Classifying health-related simulated datasets on a spherical manifold

2. Dealing with symmetric semi-positive definite matrices.

Share the next topic you’d like me to tackle.