Hi 👋🏽! I am Bruno, an ML Researcher at Huawei Research. I focus on efficient Transformers, MoE, diffusion and agents, from embedded inference to distributed training. Previously, I was an ML Researcher at Microsoft Research, an HPC Engineer and PhD Computational Neuroscience at EPFL, and MSc Computing at Imperial.
In this space, I write about my projects and topics related to my fields of interest:
| 2026 |
Efficient Inference: Kernel Fusion for MoE Expert Parallelism
|
|
Efficient Inference: Kernel Fusion for Tensor Parallelism and MoE Expert Sharding
|
| 2025 |
Efficient Inference: Faster attention (GQA, FlashAttention, sparse, linear and latent attention)
|
|
Efficient Inference: Prefill vs decode, roofline, KV cache, batching and disaggregation
|
|
ML Architectures: Diffusion models, from DDPMs to DiTs
|
| 2024 |
Distributed Training: Sequence and context parallelism with Ulysses and Ring attention
|
|
Distributed Training: Variable sequence lengths, curriculum learning, adaptive batch size and LR
|
|
Distributed Training: Mixture-of-Experts and Expert Parallelism
|
| 2023 |
Distributed Training: Tensor parallelism like Megatron-LM
|
|
Distributed Training: Pipeline parallelism (1F1B, Zero Bubble, DualPipe)
|
|
Distributed Training: Data parallelism, sharding and CPU offloading
|
|
ML Architectures: Building a GPT model in PyTorch from scratch
|
| 2020 |
ML Architectures: Learning from sequences with Encoder-Decoder, Transformers and BERT
|
| 2019 |
ML Architectures: Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs)
|
|
Machine Learning: Variational Inference (ELBO, CAVI) and Gaussian Mixture Models
|
|
Machine Learning: Exponential Family of Distributions
|
| 2018 |
Machine Learning: Bayesian Linear Regression, Maximum Likelihood and Maximum a Posteriori
|
|
Machine Learning: Statistics for ML
|
|
Machine Learning: Dropout, CNNs, Skip Connections and Embeddings
|
|
Machine Learning: Deep Neural Networks, Backpropagation and Optimization
|
| 2017 |
Machine Learning: Unsupervised Learning methods
|
|
Machine Learning: Linear Regression and Matrix Factorization
|
|
Machine Learning: Algebra for ML
|
|
Computational Neuroscience: Variable Timestep Simulation of the Electrical Activity of Neurons
|
| 2016 |
Computational Neuroscience: Numerical Simulation of Detailed Neuron Models
|
|
Computational Neuroscience: The Leaky Integrate-and-Fire Neuron and the Brunel Network
|
| 2015 |
Distributed Algorithms: Distributed Load Balancing and Spatial Slicing
|
|
Distributed Algorithms: Distributed Matrix Transpose Algorithms
|
| 2014 |
Distributed Algorithms: Distributed Sorting Algorithms
|
|
Distributed Algorithms: Distributed Matrix Multiplication and Database Joins
|
I also keep a curated list of publications, books and other resources in the field:
| pubs |
Publication Summaries (continuously updated)
|
| book |
Deep Learning - Foundations and Concepts, Christopher M. Bishop, Hugh Bishop
|
|
Pattern Recognition and Machine Learning, Christopher M. Bishop
|
|
Model-Based Machine Learning, John Winn et al.
|
|
Mathematics for Machine Learning, Marc Peter Deisenroth, A. Aldo Faisal, Cheng Soon Ong
|
|
Neuronal Dynamics, Wulfram Gerstner et al.
|
|
The Little Book of Deep Learning, François Fleuret
|
|
Understanding Machine Learning: From Theory to Algorithms, S. Shalev-Shwartz, S. Ben-David
|
|
Information Theory, Inference, and Learning Algorithms, David MacKay
|
| blog |
The vLLM MoE Playbook: A Practical Guide to TP, DP, PP and Expert Parallelism (AMD)
|
|
How to scale your model (JAX ML)
|
|
The Ultra-Scale Playbook: Training LLMs on GPU Clusters (Hugging Face)
|
| univ |
Introduction to Machine Learning - Automatic Differentiation (Princeton COS 324)
|
|
Statistics for Data Science (EPFL MATH-413)
|
|
Machine Learning (EPFL CS-433)
|
| misc |
The Matrix Cookbook
|
Enjoy, and feel free to reach out with feedback, fixes, and questions 🚀!
