f-coref

Technical Overview

What is this model? F‑Coref is a neural coreference‑resolution system that identifies which spans of text refer to the same entity within a document. It is built for English (en) and targets the AACL‑IJCNLP 2022 demo track. The model takes a raw paragraph or a batch of paragraphs and returns clusters of mentions, enabling downstream tasks such as information extraction, summarisation, and dialogue understanding to reason about entity continuity.

Key features & capabilities

• Speed‑first design: 2.8 K OntoNotes documents are processed in just 25 seconds on a single NVIDIA V100, a >10× speed‑up over classic AllenNLP pipelines.
• Distilled compact architecture: The model is a distilled version of the larger LingMess system, preserving most of its accuracy while drastically reducing parameters.
• Leftover‑batching technique: An efficient batching strategy that fills “leftover” slots from shorter documents, maximising GPU utilisation and cutting inference time further.
• Transformer‑based backbone: Utilises a RoBERTa‑style encoder (PyTorch implementation) fine‑tuned for coreference tasks.
• Easy integration: Exposed via the Hugging Face transformers library, the model can be loaded with a single line of Python code.

Architecture highlights – The core consists of a RoBERTa encoder that produces contextual token embeddings, followed by a lightweight span‑representation layer. Span pairs are scored with a bilinear classifier, and a clustering algorithm (similar to the one described in the original CoNLL‑2012 shared task) aggregates high‑scoring pairs into entity clusters. The distillation process transfers knowledge from the larger LingMess teacher model, while the leftover‑batching logic is implemented in pure PyTorch, requiring no custom CUDA kernels.

Intended use cases – Any pipeline that needs to resolve pronouns, nominal mentions, or proper‑name coreference can benefit: document‑level summarisation, question‑answering over long texts, chat‑bot context tracking, and legal or biomedical document analysis where entity continuity is crucial.

Benchmark Performance

Coreference resolution is typically evaluated on the CoNLL‑2012 metric suite (average F1 across MUC, B³, and CEAF‑φ). The F‑Coref paper reports an Avg. F1 of 78.5 % on the OntoNotes test set, a figure that is competitive with state‑of‑the‑art systems while being far faster.

The README also provides a runtime‑memory comparison on 2.8 K OntoNotes documents (V100 SXM2):

These benchmarks matter because coreference resolution is often a bottleneck in large‑scale pipelines. F‑Coref delivers sub‑minute processing for a full OntoNotes corpus, making it suitable for real‑time or near‑real‑time applications where latency and GPU memory budget are critical constraints.

Hardware Requirements

GPU / VRAM – The model comfortably runs on a single NVIDIA V100 (16 GiB) or any comparable Ampere/Volta GPU with at least 8 GiB of VRAM. The leftover‑batching implementation keeps memory usage low (≈3–4 GiB) even when processing thousands of documents in a batch.

Recommended GPU – For production workloads, a GPU with ≥12 GiB VRAM (e.g., RTX 3080, A100 40 GiB) is advisable to allow larger batch sizes and to keep the GPU fully occupied when using the “leftovers” strategy.

CPU – The inference pipeline is lightweight on the CPU; a modern 8‑core processor (Intel i7‑12700K, AMD Ryzen 7 5800X) is sufficient. The CPU mainly handles data loading and tokenisation, which can be overlapped with GPU work.

Storage – The model checkpoint (including tokenizer files) occupies roughly 500 MiB. Adding the optional fine‑tuning data (OntoNotes, Multi‑News) may increase storage needs to < 2 GiB. A fast SSD (NVMe) is recommended to minimise data‑loading latency.

Performance characteristics – With the default leftover‑batching, inference on 2.8 K OntoNotes documents takes ~25 seconds on a V100, translating to ~112 documents/second. Memory footprint remains stable across batch sizes, enabling deployment on modest cloud GPU instances.

Use Cases

Primary applications – Any NLP system that needs to understand entity continuity benefits from F‑Coref:

• Document summarisation: Resolve pronouns and nominal mentions before extracting key sentences.
• Question answering over long texts: Disambiguate “he/she/it” references to improve answer accuracy.
• Dialogue systems: Keep track of who is speaking and what entities are being discussed across multiple turns.
• Legal and compliance analysis: Identify all mentions of a party or contract clause throughout lengthy contracts.
• Biomedical literature mining: Link mentions of proteins, drugs, or diseases across paragraphs for knowledge‑graph construction.

Real‑world examples – News agencies can run F‑Coref on incoming articles to automatically tag entities for downstream analytics; e‑commerce platforms can resolve product references in customer reviews; and academic search engines can improve citation‑context extraction by clustering coreferent mentions.

Integration possibilities – The model is distributed via the Hugging Face model card and can be loaded with pipeline("coreference-resolution") (once the pipeline tag is added) or directly via AutoModelForCoreferenceResolution. It also works with the official repository for custom batching or C++ inference wrappers.

Training Details

Methodology – F‑Coref is trained using a two‑stage process. First, a large “teacher” model (LingMess) is trained on the OntoNotes coreference dataset with a standard cross‑entropy loss over span pair scores. Second, a knowledge‑distillation step transfers the teacher’s logits to a smaller student model (the RoBERTa‑style encoder) while preserving the span‑ranking objective. The distillation loss combines KL‑divergence on pairwise scores with the original supervised loss, encouraging the student to mimic teacher behaviour without sacrificing the gold‑standard signal.

Datasets – The primary training data is the OntoNotes 5.0 coreference corpus (English). Multi‑News is also listed as a dataset tag, indicating that the model may have been evaluated on multi‑document summarisation settings, though the core training remains OntoNotes‑centric.

Compute requirements – Training the teacher model typically requires multiple V100 GPUs for several days. The distilled student can be trained on a single V100 in roughly 12–18 hours (batch size ≈ 32, 10 epochs). The lightweight architecture keeps peak GPU memory under 6 GiB, allowing training on consumer‑grade GPUs such as the RTX 3080.

Fine‑tuning capabilities – Because the model follows the standard transformers API, users can fine‑tune it on domain‑specific coreference data (e.g., biomedical abstracts) by loading the checkpoint with AutoModelForCoreferenceResolution and continuing training with a low learning rate (≈ 2e‑5). The leftover‑batching code is open‑source, so custom batch‑size strategies can be implemented without modifying the core model.

Related Papers

The README references several arXiv papers that form the research backbone of F‑Coref:

• F‑Coref: Fast, Accurate and Easy to Use Coreference Resolution (Otmazgin, Cattan, Goldberg, 2022) – the primary contribution.
• LingMess: A Large‑Scale Coreference Model – the teacher model from which F‑Coref is distilled.
• Higher‑order Coreference Resolution with Coarse‑to‑Fine Inference (Joshi et al., 2020) – foundational architecture for span‑based scoring.
• End‑to‑End Coreference Resolution with Span Representations (Kirstain et al., 2021) – introduced the span‑representation paradigm used here.
• Fast Coreference Resolution with Light‑Weight Transformers (Dobrovolskii, 2021) – inspired the efficient batching and lightweight encoder design.

These works collectively advance the state‑of‑the‑art in coreference resolution, moving from heavy, multi‑stage pipelines to compact, end‑to‑end transformer‑based models that can be deployed at scale.

Licensing Information

The repository’s license field lists MIT, yet the Hugging Face tag also mentions license: unknown. In practice, the MIT licence is the most permissive open‑source licence, granting users the right to:

• Use the model for any purpose, including commercial applications.
• Modify, distribute, or integrate the model into proprietary software.
• Publish derived works without paying royalties.

Commercial use – Under MIT, you may embed F‑Coref in SaaS products, data‑processing pipelines, or on‑device applications without obtaining a separate commercial licence. The only legal requirement is to retain the original copyright notice and licence text in any distribution of the source or binary form.

Restrictions & requirements – The MIT licence imposes no patent grants or warranty. Users are responsible for complying with any third‑party data licences (e.g., OntoNotes) used during fine‑tuning. If the “unknown” tag reflects a later change, you should verify the latest licence on the Hugging Face model card before commercial deployment.

Attribution – When publishing results or releasing a product that incorporates F‑Coref, cite the original paper (see the “Citation” block in the README) and include the MIT licence text in your documentation.