France Job Openings

Inria

Internship: Combining Transformers and Normalizing Flows for Deep Surrogate Training

Saint-Martin-d'Hères

October 21, 2024

Le descriptif de l’offre ci-dessous est en Anglais

Type de contrat : Convention de stage

Niveau de diplôme exigé : Bac + 4 ou équivalent

Fonction : Stagiaire de la recherche

A propos du centre ou de la direction fonctionnelle

The Centre Inria de l’Université de Grenoble groups together almost 600 people in 23 research teams and 9 research support departments.
Staff is present on three campuses in Grenoble, in close collaboration with other research and higher education institutions (Université Grenoble Alpes, CNRS, CEA, INRAE, …), but also with key economic players in the area.
The Centre Inria de l’Université Grenoble Alpe is active in the fields of high-performance computing, verification and embedded systems, modeling of the environment at multiple levels, and data science and artificial intelligence. The center is a top-level scientific institute with an extensive network of international collaborations in Europe and the rest of the world.

Contexte et atouts du poste

The length of the internship is 4 months minimum and the start date is flexible, but need a 2 month delay before starting the interhsip due to administrative constraints.
The candidate will integrate the Datamove team located in the IMAG building on the campus of Saint Martin d’Heres (Univ. Grenoble Alpes) near Grenoble. The Data Move team is a friendly and stimulating environement gathering Professors, Researchers, Ph D and Master students. The city of Grenoble is a student friendly city surrounded by the alps mountains, offering a high quality of life and where you can experience all kinds of mountain related outdoors activities.
But there is also the possibility to pursue this internship being located at EDF R&D, Saclay, close to Paris. EDF is one fo the largest electricity supplier in Europe and their Saclay R&D EDF labs one of the largest industrial research center in France. EDF are long term collaborators actively involved in the development of Melissa and deep surrogate related investigation. EDF also brings industrial grade use-cases related to electrical machines (produced by Code_Carmel) and hydrological studies (produced by the open source code Open-Telemac).

Mission confiée

Context

Deep surrogates are deep neural networks trained from data produced by a numerical scientific simulation code, like fluids dynamics, weather forecast, molecular systems, etc. Deep surrogate are expected to be faster and smaller than the original simulation. There exists a wide variety of neural architectures used for deep surrogates, like U-Net, FNO, GNN, etc. Deep surrogate show different capabilities for generalization. Some are trained from a single simulation data, others from multiple simulation instances configured with different input parameters. A new trend is to train fundational models for scientific applications, leading to a neural network capable of supporting different types of simulations. Often these fundational models are based on a visual transformer architecture adapted for scientific data. The transformer architecture brings two key features for deep surrogates (1) the attention mechanism enables to capture correlations between simulation time steps; (2) the tokenization with positional encoding of input data into small data patches make the architecture more flexible to the resolution of the input data. In parallel, normalizing flow architectures, that project one known probability distribution to an other only partially known through data, have interesting properties (1) the are invertible and thus can be used for solving inverse problems; (2) they convey a measure of uncertainties through the learned probability distribution, a very important information for scientific computing.

Internship Goals

The goals of this internship is to investigate how effective the combination of transformer and normalizing flow architectures can be for training deep surrogates. We will consider as starting point several available architectures from the papers All-in-one simulation-based inference, Poseidon: Efficient Foundation Models for PDEs, Clima X: A foundation model for weather and climate that will be analyzed, tested and eventually combined. For the purpose of experiments, we will integrate these models into the Melissa framework developed in our team. Melissa enables to train deep surrogates on supercomputers directly from the running simulations while they produce data. Melissa enables to train very efficiently on significantly more data that the classical offline approaches that store output data from simulations to files and then read them back for training. Melissa also simply makes it easier to train deep surrogates by combing data production and training in a unified workflow.

Our Related Publications

Melissa DL x Breed: Towards Data-Efficient On-line Supervised Training of Multi-parametric Surrogates with Active Learning, SC AI4S 2024: https://hal.science/hal-04712480v1
Training Deep Surrogate Models with Large Scale Online Learning, ICML 2023: https://hal.science/hal-04102400v1
High Throughput Training of Deep Surrogates from Large Ensemble Runs, SC 2023, https://hal.science/hal-04213978v1
Deep Surrogate for Direct Time Fluid Dynamics, Neurips 2023 Thirty-fifth Workshop on Machine Learning and the Physical Sciences. https://hal.science/hal-03451432v2

Principales activités

See above

Compétences

This internship is open to students pursuing a master or an engineering degree. We expect candidates with basic probability, deep learning, physics and numerical simulation knowledges, good Python programming skills (numpy, basic pytorch). The main communication language is English.

Avantages

Subsidized meals
Partial reimbursement of public transport costs
Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
Possibility of teleworking (90 days / year) and flexible organization of working hours
Social, cultural and sports events and activities
Access to vocational training
Social security coverage under conditions

Rémunération

€4.35 per hour of actual presence at 1 January 2024.
About 590€ gross per month (internship allowance)

Informations générales

Thème/Domaine : Optimisation, apprentissage et méthodes statistiques
Calcul Scientifique (BAP E)
Ville : Saint Martin d'Hères
Centre Inria : Centre Inria de l'Université Grenoble Alpes
Date de prise de fonction souhaitée : 2025-02-01
Durée de contrat : 6 mois
Date limite pour postuler : 2024-11-21

Attention: Les candidatures doivent être déposées en ligne sur le site Inria. Le traitement des candidatures adressées par d'autres canaux n'est pas garanti.

Consignes pour postuler

CV + cover letter
Les candidatures doivent être déposées en ligne sur le site Inria.
Le traitement des candidatures adressées par d'autres canaux n'est pas garanti.

Sécurité défense :
Ce poste est susceptible d’être affecté dans une zone à régime restrictif (ZRR), telle que définie dans le décret n°2011-1425 relatif à la protection du potentiel scientifique et technique de la nation (PPST). L’autorisation d’accès à une zone est délivrée par le chef d’établissement, après avis ministériel favorable, tel que défini dans l’arrêté du 03 juillet 2012, relatif à la PPST. Un avis ministériel défavorable pour un poste affecté dans une ZRR aurait pour conséquence l’annulation du recrutement.
Politique de recrutement :
Dans le cadre de sa politique diversité, tous les postes Inria sont accessibles aux personnes en situation de handicap.

Contacts

Équipe Inria : DATAMOVE
Recruteur :
Raffin Bruno / bruno.raffin@inria.fr

L'essentiel pour réussir

This internship will open the doors to the domain of AI-for-Science, giving the candidate the opportunity to gain skills on advanced neural networks architectures, multi-GPU training on supercomputers, conduct of numerical experiments, scientific writing.

A propos d'Inria

Inria est l’institut national de recherche dédié aux sciences et technologies du numérique. Il emploie 2600 personnes. Ses 215 équipes-projets agiles, en général communes avec des partenaires académiques, impliquent plus de 3900 scientifiques pour relever les défis du numérique, souvent à l’interface d’autres disciplines. L’institut fait appel à de nombreux talents dans plus d’une quarantaine de métiers différents. 900 personnels d’appui à la recherche et à l’innovation contribuent à faire émerger et grandir des projets scientifiques ou entrepreneuriaux qui impactent le monde. Inria travaille avec de nombreuses entreprises et a accompagné la création de plus de 200 start-up. L'institut s'eﬀorce ainsi de répondre aux enjeux de la transformation numérique de la science, de la société et de l'économie.

New Job Alerts

W Electrique France SAS

Conseiller(ère) de beauté H/F

Le Chesnay

November 6, 2024

View Job Description