Julien Khlaut
Portrait of Julien Khlaut

Julien Khlaut

PhD in Machine Learning

Senior Machine Learning Researcher at Raidium, Paris. Foundation models for CT and MRI.

I build foundation models for radiology at Raidium, in Paris. Most of my work is pretraining vision and vision-language models on CT and MRI, on clusters of up to 256 H100 GPUs, and we release the weights on Hugging Face.

My latest model, Jolia, aligns 3D CT scans with the findings written in the radiology report, organ by organ. It won the CVPR 2026 3D-CT Foundation Model Challenge. Before that I was co-first author of Curia and worked on Curia-2, vision transformers trained with self-supervision on 200 million radiology images.

I also write a large part of the team's code. I am the main contributor to Raidium's internal ML library, I review most of the pull requests, and I built the experiment tracker that the R&D team (20+ people) uses every day.

I did my PhD (CIFRE) at Université Paris Cité on multimodal foundation models for radiology, supervised by Marc Sapoval and Tom Boeken. Before that I studied applied mathematics at École des Ponts ParisTech and machine learning in the MVA master at ENS Paris-Saclay.

If you want to talk about AI, don't hesitate to reach out! :)

Julien Khlaut during his PhD defense
My PhD defense, 2026.

Interests

These days I mostly work on self-supervised pretraining for medical images, and on aligning images with text at the level of individual organs and findings. I also like the engineering part a lot. Scaling a training run to hundreds of GPUs and building tools for the team is a big part of my week.

Outside radiology, I'm curious about any field with lots of noisy, high-dimensional data and few labels.

Publications

Jolia cross-attention maps highlighting organs on axial and coronal CT slices
Organ-level attention maps, axial and coronal (fig. 4)

Jolia: Concept-Level Vision-Language Alignment for 3D CT Contrastive Learningfirst author

J. Khlaut, C. Corbière, B. Callard, A. Prat, et al.

Winner of the CVPR 2026 3D-CT Foundation Model ChallengeAccepted at 3 workshops. Submitted to CVPR 2027.

arXivPDFweights

Abstract

Vision-language contrastive pretraining has become the dominant recipe for 3D medical foundation models, leveraging the large volumes of paired scans and reports produced in clinical practice. However, medical images usually span dozens of organs, and radiological reports are much longer than typical natural image captions and are composed of multiple structured sections. CLIP-style pretraining compresses this structure by encoding each modality into a single global token, at the risk of losing important details. We introduce ConQuer (Concept Queries), an image-text pretraining method that augments CLIP's global alignment with a set of localized alignments, one per concept. ConQuer splits the report into concept-specific sections and learns cross-attention queries that pool the matching image features without using any segmentation mask or spatial supervision. Contrastive learning is then applied independently for each concept. Concepts can be any unit of semantic localization; here, they are anatomical regions, one query per organ or gross body region. As a byproduct, each query learns attention maps focused on its concept, providing built-in spatial interpretability. We use ConQuer to train Jolia, a 3D CT foundation model on chest and abdominal CT. Jolia consistently outperforms a CLIP baseline on findings classification, report generation, and cross-center transfer, and sets a new state of the art across multiple public benchmarks. Jolia's weights are available at https://huggingface.co/raidium/Jolia

Tool catalog, function signatures and orchestrator pipeline of the agentic system
Tool catalog and orchestrator pipeline (fig. 1)

Can Agentic Systems Close the Gap in Radiology with Production AI Tools?

A. Charley, J. Khlaut, C. Corbière, P. Manceron, C. Dancette, L. Machado

arXiv preprint, 2026

arXivPDF

Abstract

Radiology AI has developed along two paths: specialized models that perform well on narrow tasks but cover few clinical findings, and generalist models that address a broader range of tasks but remain prone to hallucination and inaccurate measurements. Combining the two through agentic systems offers a route towards comprehensive radiology reporting, but published systems are tied to one modality or body part. We describe a radiology agentic framework that standardizes how an orchestrator calls modular tools to produce a structured report, and use it to measure what such a system can do with the tools from FDA-cleared radiology AI devices. Report generation is simulated on a large multi-modality report corpus, with the orchestrator given either a catalog of finding classes consolidated from the cleared devices, each available on the modalities it is cleared for, or an unlimited catalog in which any class can be requested, and planning either from the clinical indication or from the target report. Tool outputs are simulated from the reference report, so the comparison isolates tool availability and orchestration from tool error. With the cleared catalog and the indication alone, the orchestrator recovers under a third of the clinically actionable findings; recovery improves substantially with the unlimited catalog, and further when the plan is written from the target report. Attributing every missed finding to a cause shows that most are classes no cleared device offers for the study's modality, and that a third are classes or tools that were offered and not called. Adding the most frequently missing tools closes most of the gap to the unlimited catalog on held-out reports, whereas a smaller orchestrator with the unlimited catalog gives up the whole gain. Tool availability and effective orchestration are thus complementary constraints on comprehensive AI-assisted radiology reporting.

Graphical abstract of ClotIA
Graphical abstract (CC BY 4.0)

A Promptable 3D-CT Foundation Model-Based Approach for Pulmonary Embolism

K. Le Floch, E. Ferreres, J. Khlaut, A. Prat, L. Alberge, L. Machado, et al.

CardioVascular and Interventional Radiology, 2026

DOIPMC

Abstract

PURPOSE: Blood clot volume (BCV), defined as the total three-dimensional (3D) volume of the thrombus on computed tomography angiography (CTA), is an objective biomarker of pulmonary embolism (PE) severity whose clinical use is limited by time-consuming manual segmentation. This study evaluates ClotIA (Clot Interventional AI), a foundation model (FM)-based approach designed for rapid and interactive clot segmentation in PE. MATERIALS AND METHODS: RAPSv2, a foundation model derived from SAM2, was fine-tuned on a stratified sample of 309 patients from the RSPECT dataset (2020). Segmentation performance was evaluated using the Dice similarity coefficient (DSC) and compared to that of nnUNet (no-new-Net). The predicted BCV was correlated with imaging biomarkers of PE severity. RESULTS: ClotIA achieved a mean DSC of 0.83 ± 0.06 after guided refinement, compared to 0.79 ± 0.10 at baseline (p < 0.001) and 0.81 ± 0.13 for nnUNet (p < 0.001). The predicted BCV showed strong agreement with the reference volume (r = 0.995; mean bias + 0.12 mL) and was significantly correlated with RV/LV diameter ratio (r = 0.62, p < 0.001) and RV/LV volume ratio (r = 0.68, p < 0.001). CONCLUSION: ClotIA enables rapid and reproducible 3D quantification of pulmonary embolism thrombi, correlating with established severity markers and providing the necessary basis for translating emerging biomarkers into clinical practice.

Bar charts comparing pretrained and supervised models on four brain MRI tasks
Self-supervised pretraining improves generalization (fig. 1)

Towards Brain MRI Foundation Models for the Clinic: Findings from the FOMO25 Challenge

A. Munk, S. Cerri, V. Nersesjan, et al. (84 authors, incl. J. Khlaut)

arXiv preprint, 2026. FOMO25 challenge at MICCAI 2025.

arXivPDF

Abstract

Clinical deployment of automated brain MRI analysis faces a fundamental challenge: clinical data is heterogeneous and noisy, and high-quality labels are prohibitively costly to obtain. Self-supervised learning (SSL) can address this by leveraging the vast amounts of unlabeled data produced in clinical workflows to train robust \textit{foundation models} that adapt out-of-domain with minimal supervision. However, the development of foundation models for brain MRI has been limited by small pretraining datasets and in-domain benchmarking focused on high-quality, research-grade data. To address this gap, we organized the FOMO25 challenge as a satellite event at MICCAI 2025. FOMO25 provided participants with a large pretraining dataset, FOMO60K, and evaluated models on data sourced directly from clinical workflows in few-shot and out-of-domain settings. Tasks covered infarct classification, meningioma segmentation, and brain age regression, and considered both models trained on FOMO60K (method track) and any data (open track). Nineteen foundation models from sixteen teams were evaluated using a standardized containerized pipeline. Results show that (a) self-supervised pretraining improves generalization on clinical data under domain shift, with the strongest models trained \textit{out-of-domain} surpassing supervised baselines trained \textit{in-domain}. (b) No single pretraining objective benefits all tasks: MAE favors segmentation, hybrid reconstruction-contrastive objectives favor classification, and (c) strong performance was achieved by small pretrained models, and improvements from scaling model size and training duration did not yield reliable benefits.

Cosine similarity maps on knee MRI and abdominal CT from a single query patch
Dense feature similarity from a query patch (fig. 3)

Curia-2: Scaling Self-Supervised Learning for Radiology Foundation Models

A. Saporta, B. Callard, C. Dancette, J. Khlaut, C. Corbière, L. Butsanets, A. Prat, P. Manceron

arXiv preprint, 2026

arXivPDFweights

Abstract

The rapid growth of medical imaging has fueled the development of Foundation Models (FMs) to reduce the growing, unsustainable workload on radiologists. While recent FMs have shown the power of large-scale pre-training to CT and MRI analysis, there remains significant room to optimize how these models learn from complex radiological volumes. Building upon the Curia framework, this work introduces Curia-2, which significantly improves the original pre-training strategy and representation quality to better capture the specificities of radiological data. The proposed methodology enables scaling the architecture up to billion-parameter Vision Transformers, marking a first for multi-modal CT and MRI FMs. Furthermore, we formalize the evaluation of these models by extending and restructuring CuriaBench into two distinct tracks: a 2D track tailored for slice-based vision models and a 3D track for volumetric benchmarking. Our results demonstrate that Curia-2 outperforms all FMs on vision-focused tasks and fairs competitively to vision-language models on clinically complex tasks such as finding detection. Weights will be made publicly available to foster further research.

RadImageNet-VQA overview: anatomy, pathology, abnormality questions and captioning
Dataset overview (fig. 1)

RadImageNet-VQA: A Large-Scale CT and MRI Dataset for Radiologic Visual Question Answering

L. Butsanets, C. Corbière, J. Khlaut, P. Manceron, et al.

arXiv preprint, 2025

arXivPDF

Abstract

In this work, we introduce RadImageNet-VQA, a large-scale dataset designed to advance radiologic visual question answering (VQA) on CT and MRI exams. Existing medical VQA datasets are limited in scale, dominated by X-ray imaging or biomedical illustrations, and often prone to text-based shortcuts. RadImageNet-VQA is built from expert-curated annotations and provides 750K images paired with 7.5M question-answer samples. It covers three key tasks - abnormality detection, anatomy recognition, and pathology identification - spanning eight anatomical regions and 97 pathology categories, and supports open-ended, closed-ended, and multiple-choice questions. Extensive experiments show that state-of-the-art vision-language models still struggle with fine-grained pathology identification, particularly in open-ended settings and even after fine-tuning. Text-only analysis further reveals that model performance collapses to near-random without image inputs, confirming that RadImageNet-VQA is free from linguistic shortcuts. The full dataset and benchmark are publicly available at https://huggingface.co/datasets/raidium/RadImageNet-VQA.

Curia attention maps on brain CT and patch matching between CT and MRI
Attention maps and cross-modality matching (fig. 7)

Curia: A Multi-Modal Foundation Model for Radiologyco-first author

C. Dancette*, J. Khlaut*, A. Saporta, H. Philippe, E. Ferreres, B. Callard, et al.

*Equal contribution. Submitted to a medical journal, 2025

arXivPDFweights

Abstract

AI-assisted radiological interpretation is based on predominantly narrow, single-task models. This approach is impractical for covering the vast spectrum of imaging modalities, diseases, and radiological findings. Foundation models (FMs) hold the promise of broad generalization across modalities and in low-data settings. However, this potential has remained largely unrealized in radiology. We introduce Curia, a foundation model trained on the entire cross-sectional imaging output of a major hospital over several years, which to our knowledge is the largest such corpus of real-world data-encompassing 150,000 exams (130 TB). On a newly curated 19-task external validation benchmark, Curia accurately identifies organs, detects conditions like brain hemorrhages and myocardial infarctions, and predicts outcomes in tumor staging. Curia meets or surpasses the performance of radiologists and recent foundation models, and exhibits clinically significant emergent properties in cross-modality, and low-data regimes. To accelerate progress, we release our base model's weights at https://huggingface.co/raidium/curia.

Point, mask and box prompts on CT with ground truth and predicted masks
Prompts, ground truth and predicted masks (fig. 5)

RadSAM: Segmenting 3D Radiological Images with a 2D Promptable Modelfirst author

J. Khlaut, E. Ferreres, D. Tordjman, H. Philippe, et al.

arXiv preprint, 2025

arXivPDF

Abstract

Medical image segmentation is a crucial and time-consuming task in clinical care, where mask precision is extremely important. The Segment Anything Model (SAM) offers a promising approach, as it provides an interactive interface based on visual prompting and edition to refine an initial segmentation. This model has strong generalization capabilities, does not rely on predefined classes, and adapts to diverse objects; however, it is pre-trained on natural images and lacks the ability to process medical data effectively. In addition, this model is built for 2D images, whereas a whole medical domain is based on 3D images, such as CT and MRI. Recent adaptations of SAM for medical imaging are based on 2D models, thus requiring one prompt per slice to segment 3D objects, making the segmentation process tedious. They also lack important features such as editing. To bridge this gap, we propose RadSAM, a novel method for segmenting 3D objects with a 2D model from a single prompt. In practice, we train a 2D model using noisy masks as initial prompts, in addition to bounding boxes and points. We then use this novel prompt type with an iterative inference pipeline to reconstruct the 3D mask slice-by-slice. We introduce a benchmark to evaluate the model's ability to segment 3D objects in CT images from a single prompt and evaluate the models' out-of-domain transfer and edition capabilities. We demonstrate the effectiveness of our approach against state-of-the-art models on this benchmark using the AMOS abdominal organ segmentation dataset.

Radar plot of Dice scores per organ and example lesion segmentations
Dice per organ and example segmentations (fig. 2)

A Promptable CT Foundation Model for Solid Tumor Evaluation (ONCOPILOT)

L. Machado, L. Alberge, H. Philippe, E. Ferreres, J. Khlaut, et al.

npj Precision Oncology, 2025

arXivPubMed

Abstract

Carcinogenesis is a proteiform phenomenon, with tumors emerging in various locations and displaying complex, diverse shapes. At the crucial intersection of research and clinical practice, it demands precise and flexible assessment. However, current biomarkers, such as RECIST 1.1's long and short axis measurements, fall short of capturing this complexity, offering an approximate estimate of tumor burden and a simplistic representation of a more intricate process. Additionally, existing supervised AI models face challenges in addressing the variability in tumor presentations, limiting their clinical utility. These limitations arise from the scarcity of annotations and the models' focus on narrowly defined tasks. To address these challenges, we developed ONCOPILOT, an interactive radiological foundation model trained on approximately 7,500 CT scans covering the whole body, from both normal anatomy and a wide range of oncological cases. ONCOPILOT performs 3D tumor segmentation using visual prompts like point-click and bounding boxes, outperforming state-of-the-art models (e.g., nnUnet) and achieving radiologist-level accuracy in RECIST 1.1 measurements. The key advantage of this foundation model is its ability to surpass state-of-the-art performance while keeping the radiologist in the loop, a capability that previous models could not achieve. When radiologists interactively refine the segmentations, accuracy improves further. ONCOPILOT also accelerates measurement processes and reduces inter-reader variability, facilitating volumetric analysis and unlocking new biomarkers for deeper insights. This AI assistant is expected to enhance the precision of RECIST 1.1 measurements, unlock the potential of volumetric biomarkers, and improve patient stratification and clinical care, while seamlessly integrating into the radiological workflow.

Artificial Intelligence in Interventional Radiology: Current Concepts and Future Trends

A. Lesaunier, J. Khlaut, C. Dancette, L. Tselikas, B. Bonnet, T. Boeken

Diagnostic and Interventional Imaging, 2025

DOIPubMed

Abstract

While artificial intelligence (AI) is already well established in diagnostic radiology, it is beginning to make its mark in interventional radiology. AI has the potential to dramatically change the daily practice of interventional radiology at several levels. In the preoperative setting, recent advances in deep learning models, particularly foundation models, enable effective management of multimodality and increased autonomy through their ability to function minimally without supervision. Multimodality is at the heart of patient-tailored management and in interventional radiology, this translates into the development of innovative models for patient selection and outcome prediction. In the perioperative setting, AI is manifesting itself in applications that assist radiologists in image analysis and real-time decision making, thereby improving the efficiency, accuracy, and safety of interventions. In synergy with advances in robotic technologies, AI is laying the groundwork for an increased autonomy. From a research perspective, the development of artificial health data, such as AI-based data augmentation, offers an innovative solution to this central issue and promises to stimulate research in this area. This review aims to provide the medical community with the most important current and future applications of AI in interventional radiology.

Training pipeline: BioMedLM, continued pretraining on textbooks and generated cases, fine-tuning on ECN-QA
Training strategy (fig. 1)

Efficient Medical Question Answering with Knowledge-Augmented Question Generationfirst author

J. Khlaut, C. Dancette, E. Ferreres, A. Bennani, et al.

NAACL 2024 Clinical NLP Workshop

arXivPDF

Abstract

In the expanding field of language model applications, medical knowledge representation remains a significant challenge due to the specialized nature of the domain. Large language models, such as GPT-4, obtain reasonable scores on medical question answering tasks, but smaller models are far behind. In this work, we introduce a method to improve the proficiency of a small language model in the medical domain by employing a two-fold approach. We first fine-tune the model on a corpus of medical textbooks. Then, we use GPT-4 to generate questions similar to the downstream task, prompted with textbook knowledge, and use them to fine-tune the model. Additionally, we introduce ECN-QA, a novel medical question answering dataset containing ``progressive questions'' composed of related sequential questions. We show the benefits of our training strategy on this dataset. The study's findings highlight the potential of small language models in the medical domain when appropriately fine-tuned. The code and weights are available at https://github.com/raidium-med/MQG.

Promptable Foundation Model for Automatic Whole-Body RECIST Measurement

P. Hérent, H. Philippe, E. Ferreres, L. Machado, C. Dancette, J. Khlaut, et al.

Journal of Clinical Oncology (ASCO abstract), 2024

DOI

Abstract

Background: RECIST 1.1 is the gold standard for tumor evaluation in clinical trials and patient care. However, it faces challenges due to the subjective selection and measurement of lesions, notably because of inter-observer variability in the 20-30% range (1), which can potentially result in inaccuracies in therapeutic response classification (2). The main objective of this study is to assess the added value of a radiological foundation model to assist the radiologist to measure RECIST across different lesion topographies, using visual prompting. Methods: We use a promptable segmentation algorithm, based on a visual foundation model, pre-trained on various CT-scans and then trained on a subset of the Medical Segmentation Decathlon (MSD) dataset, covering 538 patients with pancreas, liver, colon and lung lesions. It outputs a 3D segmentation mask of the lesion, using a visual prompt, i.e. a 3D bounding box (BBox) as input. We then measure the capacity of the model to measure RECIST on the MSD validation set and on the KiTS validation set (99 samples), which features a new lesion type (kidney). To simulate inter-reader variability of up to 23%, we provide as inputs BBoxes centered on the lesions but with a varying error (15% variability: +10 pixels, 23% variability: +15 pixels). Results: The model is able to correct variability in BBox input: with all BBox error ranges, it beats the input variability (15% and 23% thresholds in each column) across many organ types, except in the pancreas when using large error BBoxes. Moreover, the model is able to generalize to new lesion types not seen during training, on an external dataset (KiTS - Kidney). Conclusions: Unlike existing supervised machine learning models dedicated to lesion detections in specific organs, our approach is organ and lesion agnostic and offers a more reliable, precise tumor assessment. This is a first step before allowing the longitudinal evaluation of tumors while safeguarding the clinical intuition necessary for selecting the right target lesions. Moreover, we believe that these methods will be crucial in advancing beyond RECIST 1.1, facilitating the identification of new prognostic biomarkers and proxies for tumor burden derived from previously underexplored radiomics features, ultimately refining the efficacy of clinical trial assessments and elevating the standard of patient care.

Experience

  • 2026 - presentSenior ML Researcher
    Raidium
  • 2023 - 2026ML Researcher, CIFRE PhD
    Raidium and Université Paris Cité
  • 2023ML Research Intern
    Raidium
  • 2022Data Scientist
    Warner Bros. Discovery
  • 2021Full-Stack Developer
    BAM

Education

  • 2023 - 2026PhD in Machine Learning
    Université Paris Cité
  • 2022 - 2023MSc MVA (Mathematics, Vision, Learning)
    ENS Paris-Saclay
  • 2019 - 2023Engineering degree, applied maths and computer science
    École des Ponts ParisTech
  • 2017 - 2019Classes préparatoires PCSI / PC*
    Collège Stanislas