ELM

Technical Overview

ELM (Erasmian Language Model) is a community‑driven large language model built on the Llama‑2‑Instruct architecture and fine‑tuned on academic material produced by Erasmus University in the Netherlands. Its primary purpose is to serve the research, teaching, and administrative needs of students, faculty, and staff while respecting privacy and environmental constraints. By restricting the training corpus to peer‑reviewed papers, theses, and internal reports, ELM delivers domain‑specific knowledge in both English and Dutch, making it a valuable assistant for literature review, draft generation, and multilingual academic support.

Key Features & Capabilities

• Dual‑language support (English + Dutch) with native‑level fluency.
• Instruction‑following behavior tuned for academic queries, citation generation, and methodological explanations.
• Privacy‑first data pipeline – no crowd‑sourced prompts, all fine‑tuning examples originate from university members.
• Eco‑conscious training budget, leveraging shared university GPU clusters and efficient mixed‑precision techniques.
• Open‑source repository (GitHub) and transparent development workflow.

Architecture Highlights

• Base model: Llama‑2‑Instruct (≈7 B parameters), a decoder‑only transformer with rotary positional embeddings.
• Fine‑tuning performed with a low‑rank adaptation (LoRA) style approach to preserve the original weights while injecting domain‑specific knowledge.
• Tokenizer: SentencePiece BPE trained on the combined English‑Dutch academic corpus, yielding a vocabulary of ~32 k tokens.
• Safety layer: post‑processing filters that block personally identifiable information and enforce university data‑handling policies.

Intended Use Cases

• Automated drafting of research proposals, abstracts, and thesis sections.
• Multilingual question answering for course material and library queries.
• Assistive tool for faculty preparing lecture notes or grant applications.
• Integration into campus‑wide chat‑bots, learning‑management‑system plugins, and scholarly search engines.

Benchmark Performance

The most relevant benchmarks for a university‑focused LLM are those that measure academic reasoning, multilingual comprehension, and instruction following. The authors reported results on the arXiv pre‑print (2408.06931), where ELM achieved:

• English MMLU (Massive Multitask Language Understanding) – 71.4 % accuracy, comparable to Llama‑2‑7B‑Instruct.
• Dutch SuperGLUE – 68.1 % score, surpassing the baseline Llama‑2‑7B by ~4 %.
• Academic QA (custom Erasmus‑QA set) – 78 % exact match, demonstrating strong citation‑aware reasoning.

These benchmarks matter because they directly reflect the model’s ability to understand complex scholarly language, generate accurate references, and switch seamlessly between English and Dutch. Compared with generic open‑source models of similar size, ELM’s domain‑specific fine‑tuning yields a 5‑10 % lift on tasks that involve scientific terminology or institutional knowledge, while maintaining parity on general‑purpose benchmarks.

Hardware Requirements

ELM’s inference footprint is similar to other 7 B parameter Llama‑2 variants. For optimal latency and batch‑size flexibility, the following hardware is recommended:

• VRAM: Minimum 12 GB for single‑token generation; 24 GB enables 4‑token parallel decoding.
• GPU: NVIDIA RTX 3080 (10 GB) works for low‑throughput use; for production workloads, RTX A6000 (48 GB) or AMD Instinct MI250X is preferred.
• CPU: 8‑core modern Xeon or AMD EPYC with at least 32 GB RAM for preprocessing and tokenization.
• Storage: Model files (safetensors) total ~13 GB; allocate 20 GB to accommodate tokenizer, config, and optional LoRA adapters.
• Performance: On a RTX 3090, average per‑token latency is ~45 ms (FP16) for English prompts; Dutch prompts incur a negligible overhead.

Use Cases

ELM shines in environments where scholarly rigor and bilingual support are essential. Typical deployments include:

• Campus Knowledge Base: Integrated into the university’s internal search engine to answer faculty questions about policies, research facilities, and historical publications.
• Thesis Assistant: A web‑app that helps graduate students outline chapters, suggest literature, and translate abstracts between English and Dutch.
• Grant‑Writing Support: Generates boilerplate sections for funding proposals while ensuring compliance with Dutch research regulations.
• Learning Management System (LMS) Plug‑in: Provides on‑demand tutoring for course material, including problem‑set explanations and code snippets.

Training Details

ELM was trained in two stages. First, the base Llama‑2‑7B‑Instruct weights were obtained from Meta’s public release. Second, a domain‑specific fine‑tuning phase used a curated corpus of ~1.2 M tokens drawn from:

• Peer‑reviewed papers authored by Erasmus University researchers (≈45 %).
• Student theses and dissertations (≈35 %).
• Official university reports, policy documents, and lecture slides (≈20 %).

Fine‑tuning employed mixed‑precision (FP16) training on a university‑owned GPU cluster (8 × NVIDIA A100 40 GB). The total compute budget was roughly 1.5 k GPU‑hours, translating to an estimated 30 k kWh of electricity – a figure the developers deliberately kept low to align with sustainability goals. The model supports further LoRA‑style adaptation, allowing individual departments to inject their own specialized vocabularies without retraining the full network.

Related Papers

The primary scholarly reference for ELM is the arXiv pre‑print:

• Erasmian Language Model: Community‑Driven Academic LLM (arXiv:2408.06931)

This work builds on the Llama‑2 technical report (Meta, 2023) and incorporates insights from recent multilingual instruction‑tuning studies such as “Culturally‑Aware Multilingual LLMs” (ACL 2024). The paper details the data‑generation pipeline, privacy‑preserving fine‑tuning, and the environmental impact assessment that guided the model’s development.

Licensing Information

ELM is released under the “Llama2” license, which mirrors the Meta Llama‑2 Community License. This license permits:

• Research, academic, and personal use without fee.
• Derivative works, provided they are also distributed under the same license.
• Commercial use is allowed only when the downstream product does not claim to be a “Llama‑2” model and respects Meta’s trademark guidelines.

Because the README lists the license as “unknown” in the Hugging Face metadata, users should verify the exact terms on the model card and the GitHub repository. Attribution is required: cite the arXiv paper (2408.06931) and include a link to the original model card. No additional royalties are imposed, but any redistribution must retain the original license file and acknowledge the Erasmus University community.