Hi 👋🏽! I am Bruno, an ML Systems Researcher at Huawei Research Switzerland. Previously, I was an ML Researcher at Microsoft Research Cambridge, and an HPC Engineer and PhD in Computational Neuroscience at EPFL. I focus on ML efficiency — from embedded inference to distributed training — across Transformer, MoE, diffusion, and agentic models.
In this space, I write about my projects and topics related to my fields of interest:
| 2026 |
Efficient Inference: Kernel Fusion for Mixture-of-Experts Expert Parallelism
|
|
Efficient Inference: Kernel Fusion for Megatron-LM Tensor Parallelism
|
| 2025 |
Efficient Inference: Faster attention (GQA, FlashAttention, sparse, linear and latent attention)
|
|
Efficient Inference: Prefill vs decode, roofline, KV cache, continuous/ragged batching, disaggregation
|
|
ML Architectures: Diffusion models, from DDPMs to DiTs
|
| 2024 |
Distributed Training: Sequence and context parallelism with Ulysses and Ring attention
|
|
Distributed Training: Variable sequence lengths, curriculum learning, adaptive batch size and LR
|
|
Distributed Training: Mixture-of-Experts and Expert Parallelism
|
| 2023 |
Distributed Training: Tensor parallelism
|
|
Distributed Training: Pipeline parallelism (1F1B, Zero Bubble, DualPipe)
|
|
Distributed Training: Data parallelism, sharding and CPU offloading
|
|
Efficient Inference: GPT in C++: LibTorch vs. PyTorch, TorchScript and torch.compile
|
|
ML Architectures: Building a GPT model in PyTorch from scratch
|
| 2020 |
ML Architectures: Learning from sequences with Encoder-Decoder, Transformers and BERT
|
| 2019 |
ML Architectures: Variational Autoencoders (VAEs) and Generative Adversarial Neural Networks (GANs)
|
|
Machine Learning: Variational Inference (ELBO, CAVI) and Gaussian Mixture Models
|
|
Machine Learning: Exponential Family of Distributions
|
| 2018 |
Machine Learning: Bayesian Linear Regression, Maximum Likelihood and Maximum-A-Posteriori
|
|
Machine Learning: Statistics for ML
|
|
Machine Learning: Mathematics for ML
|
|
Machine Learning: Deep Neural Networks, backpropagation, autodiff, dropout, CNNs and embeddings
|
| 2017 |
Machine Learning: Unsupervised Learning basics and Principal Component Analysis
|
|
Computational Neuroscience: Variable Timestep Simulation of the Electrical Activity of Neurons
|
|
Machine Learning: Closed-form Linear Regression and Matrix Factorization, and loss functions
|
| 2016 |
Computational Neuroscience: Numerical Resolution of the Electrical Activity of Detailed Neuron Models
|
|
Computational Neuroscience: The Leaky Integrate-and-Fire Neuron Model and The Brunel Network
|
| 2015 |
Distributed Algorithms: Distributed Load Balancing and Spatial Slicing
|
|
Distributed Algorithms: Distributed Matrix Transpose Algorithms
|
| 2014 |
Distributed Algorithms: Distributed Sorting Algorithms
|
I also keep a curated list of related books and other resources available online:
| book |
Deep Learning - Foundations and Concepts, Christopher M. Bishop, Hugh Bishop (pdf)
|
|
Pattern Recognition and Machine Learning, Christopher M. Bishop (pdf)
|
|
Model-Based Machine Learning, John Winn et al. (pdf)
|
|
Mathematics for Machine Learning, Marc Peter Deisenroth, A. Aldo Faisal, Cheng Soon Ong (pdf)
|
|
Neuronal Dynamics, Wulfram Gerstner et al. (online)
|
|
The Little Book of Deep Learning, François Fleuret (pdf)
|
|
Understanding Machine Learning: From Theory to Algorithms, Shai Shalev-Shwartz, Shai Ben-David (pdf)
|
|
Information Theory, Inference, and Learning Algorithms, David MacKay (pdf)
|
| blog |
The vLLM MoE Playbook: A Practical Guide to TP, DP, PP and Expert Parallelism (AMD)
|
|
How to scale your model (JAX ML)
|
|
The Ultra-Scale Playbook: Training LLMs on GPU Clusters (Hugging Face)
|
| univ |
Computing Gradients with Backpropagation - Automatic Differentiation (Princeton course COS-324) (pdf)
|
|
Statistics for Data Science (EPFL course MATH-413)
|
| misc |
The Matrix Cookbook
|
And finally, I maintain a list of relevant publications. Enjoy, and feel free to reach out with feedback, fixes, and questions 🚀!
