Dongyang Fan 范冬阳

PhD student · Machine Learning and Optimization Lab, EPFL

Hi, thanks for stopping by :) I'm Dongyang (pronounced Don-Young), a 4th-year PhD student at EPFL advised by Prof. Martin Jaggi.

I study LLMs from two angles: building them, so that they learn efficiently from data and can be developed and trained collaboratively across many parties; and measuring them, tracking frontier capabilities through evaluation and benchmarking.

I'm always open to new directions. If you'd like to chat or collaborate, feel free to email me!

Research Interests

Data

What makes training data valuable, and how do we use it well and responsibly?

Data-efficient pretraining through metadata conditioning
Synthetic data generation, e.g., for math reasoning
Compliant data use: respecting web-crawling opt-outs and pricing data contributions fairly

Evaluation

How do we track model capabilities and performance over time?

Hard, realistic benchmarks that stay informative as models improve
Real-life focus: multi-turn interactions and agentic behaviors
Analyzing model behavior beyond single leaderboard numbers

Modular & Collaborative Learning

How can models be built from specialized parts, and trained by many parties together?

Mixture-of-Experts: design choices, specialization, and routing
Decentralized and collaborative training across devices and data owners
Personalization without sharing raw data

News

05/2026I started my internship at NVIDIA 🇨🇭, working on LLM evaluation and analysis!
03/2026Our benchmark HalluHard is discussed on Radio France!
02/2026Our benchmark HalluHard is on arXiv! See our website for newly released results.
01/2026Our paper on data-efficient LLM pretraining is accepted at ICLR 2026 🇧🇷🎉
01/2026I started my internship at DatologyAI 🇺🇸, working on math synthetic data generation.
12/2025Attending NeurIPS 2025 in San Diego 🇺🇸, presenting our work on metadata-conditioned pretraining!

Publications

* equal contribution · WS = Workshop

Preprint 2026

HalluHard: A Hard Multi-Turn Hallucination Benchmark

Dongyang Fan*, Sebastien Delsad*, Nicolas Flammarion, Maksym Andriushchenko

arXiv code website

Hallucination Benchmark LLM Evaluation

ICLR 2026

Beyond URLs: Metadata Diversity and Position for Efficient LLM Pretraining

Dongyang Fan*, Diba Hashemi*, Sai Praneeth Karimireddy, Martin Jaggi

arXiv

LLM Pretraining Metadata Data Efficiency

ACL Main 2026

Apertus: Democratizing Open and Compliant LLMs for Global Language Environments

Apertus team (member of the pretraining team)

arXiv

Open LLMs Data Compliance Multilingual

COLM 2025 WS · Oral 🏆

TiMoE: Time-Aware Mixture of Language Experts

Robin Faro*, Dongyang Fan*, Tamar Alphaidze, Martin Jaggi

arXiv code slides

Mixture-of-Experts Temporal Modeling LLM Architecture

NeurIPS 2025

URLs Help, Topics Guide: Understanding Metadata Utility in LLM Training

Dongyang Fan, Vinko Sabolčec, Martin Jaggi

arXiv code

LLM Training Metadata Data Efficiency Steering

COLM 2025 · Oral 🏆

Can Performant LLMs Be Ethical? Quantifying the Impact of Web Crawling Opt-Outs

Dongyang Fan, Vinko Sabolčec, Matin Ansaripour, Ayush Kumar Tarun, Martin Jaggi, Antoine Bosselut, Imanol Schlag

arXiv code project page slides

Data Compliance Ethics Memorization Responsible AI

ICLR 2025 WS

Do Data Valuations Make Good Data Prices?

Dongyang Fan, Tyler J. Rotello, Sai Praneeth Karimireddy

arXiv

Data Valuation Game Theory Data Markets

ICML 2025

On-Device Collaborative Language Modeling via a Mixture of Generalists and Specialists

Dongyang Fan*, Bettina Messmer*, Nikita Doikov, Martin Jaggi

arXiv code

Collaborative Learning Mixture-of-Experts On-Device Federated Learning

ICLR 2024 WS

Towards an Empirical Understanding of MoE Design Choices

Dongyang Fan*, Bettina Messmer*, Martin Jaggi

arXiv

Mixture-of-Experts Architecture Design Expert Specialization

Personalized collaborative fine-tuning thumbnail

COLM 2024

Personalized Collaborative Fine-Tuning for On-Device Large Language Models

Nicolas Wagner, Dongyang Fan, Martin Jaggi

arXiv code

Collaborative Learning Personalization On-Device Fine-Tuning

AAAI 2024

Ghost Noise for Regularizing Deep Neural Networks

Atli Kosson, Dongyang Fan, Martin Jaggi

arXiv

Regularization Deep Learning Generalization

NeurIPS 2023

Collaborative Learning via Prediction Consensus

Dongyang Fan, Celestine Mendler-Dünner, Martin Jaggi

arXiv code poster

Collaborative Learning Consensus Knowledge Distillation

Academic Service

Reviewer for NeurIPS 2025 (Top Reviewer ⭐️), 2024; ICLR 2025, 2023; COLM 2025; and multiple workshops.
Supervision of student projects: supervised projects have led to a COLM paper and an oral presentation at a COLM workshop.

Miscellaneous

I like arts and culture. I'm also an outdoorsy person: I hike, ski, and sail, and I paint from my hiking trips 🌼