“Artificial intelligence for the Sciences” (AI4theSciences) is an innovative, interdisciplinary and intersectoral PhD programme, led by Université Paris Sciences et Lettres (PSL) and co-funded by the European Commission. Supported by the European innovation and research programme Horizon 2020-Marie Sklodowska-Curie Actions, AI4theSciences is uniquely shaped to train a new generation of researchers at the highest academic level in their main discipline (Physics, Engineering, Biology, Human and Social Sciences) and master the latest technologies in Artificial Intelligence and Machine Learning which apply in their own field. 26 doctoral students will join the PSL university's doctoral schools in 3 academic cohorts to carry out work on subjects suggested and defined by PSL's scientific community. The 2023 call will offer up to 4 PhD positions on 10 PhD research projects. The candidates will be recruited through HR processes of high standard, based on transparency, equal opportunities and excellence.

Context and Motivation for the Project

Humans have an innate ability to process language. This unique ability, linguists argue, results from a specific brain function: the recursive building of hierarchical structures (Hauser, Chomsky & Fitch, 2002). Specifically, a dedicated set of brain regions, known as the Language Network, would iteratively link the successive words of a sentence to build its latent syntactic structure, and ultimately produce compositional meaning. However, two major obstacles limit the discovery of the mechanics of the Language Network. First, linguistic models are based on symbolic representations and are thus difficult to compare to the vectorial representations of neuronal activity. Second, standard human brain recordings have a limited spatial resolution and cannot easily characterize the functions and representations of individual neurons or small neuronal populations.

Recent advances in neuroscience and Artificial Intelligence (AI) may now address this issue. In neuroscience, intracranial recordings can now be used to study language down to the single-neuron level, as we have recently shown (Lakretz et al, 2021). In AI, deep learning architectures trained on very large text corpora demonstrate near-human abilities on a variety of language tasks such as translation, question answering, and even dialogue (ChatGPT). These new language models are, like the human brain, based on vectorial representations, and thus provide new opportunities to understand the neural computations of natural language processing.

In the past two years, our respective teams have shown that such language representations from deep language models could be tracked, dissected and directly compared between brains and deep language models. The results, based on functional Magnetic Resonance Imaging, reveal systematic similarities between the functions of the Language Network in the brain, and those of deep language models. Furthermore, we have shown that it is possible to identify detailed neural mechanisms underlying sentence-structure processing in neural language models. This mechanistic understanding in the models leads to new predictions about human language processing, which we started to test in behavioral and neuro-imaging experiments using Magnetoencephalography.

B. Scientific Objectives, Methodology and Expected Results

In this research, we will combine these two breakthroughs in AI and neuroscience to understand how linguistic structures are represented in neuronal activity. For this, we will study the encoding of sentence structures in state-of-the-art neural language models, derive their predictions, and test them on a unique set of intracranial datasets that we are currently collecting. This dataset is unparalleled with respect to the various resolutions it will provide, from large neural populations down to the single-neuron level. To isolate sentence structures, we will focus on long-range grammatical agreement. Long-range agreements are considered one of the best windows into the processing of hierarchical structures in natural language. For example, in the sentence ‘The keys to the cabinet are on the table.’, the main subject 'keys' and the verb ‘are’ agree on their grammatical number (plural) despite their separation and the presence of an intervening noun ‘cabinet’, which carries the opposite grammatical number (singular). Long-range agreements therefore show that sentences have a latent hierarchical structure.

Objective 1 - Identify the neural mechanisms underlying long-range agreements in large language models: In previous work we identified the neural mechanism underlying long-range agreement in language models. However, this work focused on only a single neural architecture, a recurrent neural network (RNN), which is currently outperformed on many fronts by the newer models such as the Transformer (Vaswani et al., 2017). Our first objective is therefore to identify the mechanisms Transformers use to process long-range agreements. Our most recent study predicts that Transformers develop a similar neural mechanism to that in RNNs (Lakretz et al., 2022). However, it focused on the performance of the models without probing neural mechanisms.

Objective 2 - Test predictions from the models in humans: We will study sentence-structure processing in the human brain by testing predictions from the neural mechanisms identified in the models. For this, we will collect a large and unique dataset of intracranial data from humans reading isolated sentences. We will then compare these human neural data to the activations of artificial neurons in large language models.

Objective 3 - Quantify alignment between brains and models during sentence processing: In addition to localized mechanisms dedicated to long-range agreements, we will study larger scale processing of sentence structure in the human language network. For this, we will analyze publicly available datasets of fMRI and MEG, using machine-learning methods that we developed in previous work.

C. 3i Characteristics of the Thesis, Expected impact and Feasibility of the Project in 3 Years.

Interdisciplinary/International/Intersectoral - The research program uniquely interfaces theory and methods from neuroscience, artificial Intelligence and linguistics to identify neural mechanisms of fundamental computations in human language. Linguistic theory will guide the construction of sentence stimuli to probe hierarchical processing in both humans and models, neuroscientific analysis methods will be applied to both human and model neural data, recorded during language processing. Intracranial experiments will be conducted in collaborations with several neurosurgical centers around the world (Tel-Aviv; Marseille and Houston), in three languages - English, French and Hebrew. The thesis will therefore combine both computational and experimental aspects, and will benefit from unique technological facilities and expertise in both Deep Learning and intracranial recordings, and from extensive scientific exchanges with academic and non-academic leading labs around the world.

Expected impact - By linking multiple domains and by developing a neural-mechanistic understanding of the computations underlying a cognitive ability unique to our species, the thesis is expected to have a high scientific impact and to appear in high-profile journals.

Feasibility - Our previous work on human single-neuron and intracranial data (Lakretz et al., 2021; Woolnough et al., 2021), on language processing in AI models (Lakretz et al., 2019, 2021, 2022), suggests successful collaborations with the neurosurgical centers and AI research teams around the world. Simulation work will be conducted using the high-performance computing resource Jean Zay. The neurosurgical centers collect data at a rate of around 10 patients per year, and have ongoing umbrella ethical approvals for running experiments on language processing in humans. A successful pilot study has already been conducted in the Houston hospital center.

References:

Caucheteux, C., Gramfort, A., & King, J. R. (under embargo). Evidence of a predictive coding hierarchy in the human brain listening to speech, Nature Human Behavior.

Caucheteux, C., Gramfort, A., & King, J. R. (2021). Disentangling syntax and semantics in the brain with deep networks. ICML 2021.

Caucheteux, C., & King, J. R. (2022). Brains and algorithms partially converge in natural language processing. Communications biology, 5(1), 134.

Hauser, M. D., Chomsky, N., & Fitch, W. T. (2002). The faculty of language: what is it, who has it, and how did it evolve?. science, 298(5598), 1569-1579.

Lakretz, Y., Kruszewski, G., Desbordes, T., Hupkes, D., Dehaene, S., & Baroni, M. (2019). The emergence of number and syntax units in LSTM language models. NAACL 2019.

Lakretz, Y., Hupkes, D., Vergallito, A., Marelli, M., Baroni, M., & Dehaene, S. (2021). Mechanisms for handling nested dependencies in neural-network language models and humans. Cognition, 213, 104699.

Lakretz, Y., Ossmy, O., Friedmann, N., Mukamel, R., & Fried, I. (2021). Single-cell activity in human STG during perception of phonemes is organized according to manner of articulation. NeuroImage, 226, 117499.

Lakretz, Y., Desbordes, T., Hupkes, D., & Dehaene, S. (2022). Can Transformers Process Recursive Nested Constructions, Like Humans?. COLING, 2022.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. NIPS 2017.

Pasquiou, A., Lakretz, Y., Hale, J., Thirion, B., & Pallier, C. (2022). Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps. ICML 2022.

Woolnough, O., Donos, C., Rollo, P. S., Forseth, K. J., Lakretz, Y., Crone, N. E., ... & Tandon, N. (2021). Spatiotemporal dynamics of orthographic and lexical processing in the ventral visual pathway. Nature Human Behaviour, 5(3), 389-398.

Zacharopoulos, C. N., Dehaene, S., & Lakretz, Y. (2022). Disentangling Hierarchical and Sequential Computations during Sentence Processing. bioRxiv, 2022-07.

Thesis supervisors

Yair Lakretz, Laboratoire de Sciences Cognitives et Psycholinguistique (LSCP), ENS, Paris.

Jean-Rémi King, CNRS, Laboratoire des Systèmes Perceptifs (LSP), ENS, Paris. MetaAI, Paris.

PSL University PSL

Created in 2012, Université PSL is aiming at developing interdisciplinary training programmes and science projects of excellence within its members. Its 140 laboratories and 2,900 researchers carry out highlevel disciplinary research, both fundamental and applied, fostering a strong interdisciplinary approach. The scope of Université PSL covers all areas of knowledge and creation (Sciences, Humanities and Social Science, Engineering, the Arts). Its eleven component schools gather 17,000 students and have won more than 200 ERC. PSL has been ranked 36 in the 2020 Shanghai ranking (ARWU)