Grant PID2023149762NB-I00

This project intends to contribute to the debate on the applications of large language models (LLMs) in various scientific fields, while also proposing the integration of artificial intelligence in the processing of less-documented languages.

The project is expected to contribute to the debate on processing languages with scarce data, which is relevant to the efficiency and sustainability of LLMs. It is also expected to raise awareness of historical languages by placing them in the discussion of the new generative artificial

intelligence paradigm.

The objectives of this project are (i) to

implement a language model for Old English based on a natural language processing (NLP) library adjusted for morphosyntactic

annotation with Universal Dependencies (UD); and (ii) to automatically annotate a corpus of Old English prose consisting of 300,000 words

with Universal Dependencies. On the linguistic side, the methodology is based on data collection from a set of textual and lexicographical

sources and structuring them through tagsets of updated and widely distributed theoretical models.

On the computational side, the methodology is primarily based on trial and error. Different corpus sizes, pipeline configurations, and pretraining/retraining alternatives for the language model will be tested. LLMs and NLP libraries serve as the point of contact between the two methodological approaches.

The main milestones planned to be achieved include (i) adaptation of the UD annotation framework, inclusion of universal derivations, and

definition of enhanced dependencies; (ii) annotation standard; (iii) UD treebank for Old English; (iv) spaCy pipeline, including initial training

and component updates; (v) augmented training data for spaCy pipeline retraining; and (vi) testing without data in Gothic. Datasets and models will be freely distributed on GitHub and HuggingFace, as well as on the institutional respository of the University of La Rioja