rebel-large

Technical Overview

Model ID: Babelscape/rebel-large
Model name: rebel‑large
Author: Babelscape

rebel‑large is a high‑capacity, English‑only relation‑extraction model that reframes the classic information‑extraction pipeline as a pure seq2seq generation task. Instead of first detecting entities and then classifying the relation between them, the model directly generates subject‑relation‑object triples in a linearised textual format. This end‑to‑end approach eliminates error‑propagation across multiple stages and enables the model to handle more than two hundred relation types out‑of‑the‑box.

Key capabilities include:

• Relation extraction from raw sentences, producing triplets such as (Punta Cana, located_in, Dominican Republic).
• Entity‑aware generation – the model emits special tokens that delimit subjects, objects and relation identifiers, making downstream parsing straightforward.
• Zero‑shot adaptability – because the output is plain text, the same model can be fine‑tuned on new domains (e.g., biomedical, finance) with minimal architectural changes.
• Scalable inference – built on the BART family, it benefits from the well‑studied transformer decoder‑only design and can be served via Hugging Face pipelines, Azure endpoints, or custom ONNX runtimes.

Architecture highlights

• Backbone: BART‑large (12 encoder layers, 12 decoder layers, 406 M parameters).
• Training objective: standard autoregressive language modelling on a linearised relation‑triplet representation.
• Tokenisation: BPE‑based RobertaTokenizerFast (compatible with the original BART tokenizer).
• Output format: a sequence of <subj>, <rel>, <obj> markers that can be post‑processed into a structured graph.

Intended use cases

• Knowledge‑base population and validation.
• Fact‑checking pipelines that need explicit relational evidence.
• Semantic search and question‑answering systems that rely on extracted triples.
• Domain‑specific IE (e.g., legal contracts, scientific literature) after fine‑tuning.

Benchmark Performance

The model is evaluated on two widely‑cited English RE benchmarks:

• CoNLL‑04 – a multi‑type RE dataset (person‑organization, location‑organization, etc.). rebel‑large achieves a RE+ Macro F1 of 76.65 %.
• NYT – a large‑scale news corpus with over 24 K relation instances. The model reaches an F1 of 93.4 %.

These scores are competitive with, and often surpass, earlier encoder‑decoder baselines (e.g., T5‑base, BART‑large) while using a single model instead of a multi‑stage pipeline. The Papers with Code leaderboard confirms that rebel‑large sits at the top‑right corner of the NYT and CoNLL‑04 leaderboards, indicating state‑of‑the‑art performance for both precision and recall.

Why these benchmarks matter:

• CoNLL‑04 tests the model’s ability to handle multiple relation types in a relatively small, noisy setting.
• NYT evaluates scalability to large, real‑world news streams where relation diversity and sentence length vary widely.

Compared to other RE‑specific seq2seq models (e.g., t5‑base‑re), rebel‑large offers a higher macro‑averaged F1 on CoNLL‑04 and a markedly higher overall F1 on NYT, while keeping the inference latency comparable thanks to the efficient BART decoder.

Hardware Requirements

VRAM for inference

• Full‑precision (FP32) inference on a single GPU typically requires ≈ 12 GB of VRAM for the 406 M‑parameter model.
• Mixed‑precision (FP16) reduces the footprint to ≈ 7 GB, enabling deployment on consumer‑grade GPUs such as RTX 3060 or A100 40 GB in half‑precision mode.

Recommended GPU specifications

• Minimum: NVIDIA RTX 2070 (8 GB) with FP16 enabled.
• Optimal: NVIDIA A100 40 GB or RTX 3090 (24 GB) for batch‑size ≥ 8 and low latency.
• For large‑scale batch processing, consider multi‑GPU setups using accelerate or DeepSpeed.

CPU & storage

• CPU: any modern 8‑core processor (e.g., Intel i7‑12700K) is sufficient for preprocessing; the model is GPU‑bound.
• Disk: the model checkpoint (≈ 1.2 GB) plus tokenizer files (~ 150 MB) require ≈ 2 GB of storage. Using Safetensors format reduces loading time and memory overhead.

Performance characteristics

• Throughput: ~ 45 tokens / ms on a single RTX 3090 (FP16).
• Latency: ~ 30 ms per sentence (average length 30 tokens) on the same hardware.
• Scales linearly with batch size up to the VRAM limit; larger batches improve throughput for offline processing.

Use Cases

Primary applications

• Knowledge‑base construction: Automatically ingest news articles, patents or legal contracts and generate structured triples for graph databases such as Neo4j or JanusGraph.
• Fact‑checking and claim verification: Extract relational evidence from source documents to support or refute statements in real‑time moderation tools.
• Semantic search enrichment: Index extracted triples alongside full‑text to enable relation‑aware retrieval (e.g., “Which companies are headquartered in Singapore?”).
• Domain‑specific IE: Fine‑tune on a small corpus of medical reports to extract drug‑disease or gene‑protein relations.

Real‑world examples

• Financial news monitoring – a hedge‑fund pipeline runs rebel‑large on daily press releases to capture acquisitions, partnerships and regulatory actions.
• Legal contract analysis – a law‑tech platform extracts parties, obligations and dates from contracts, feeding the triples into a compliance dashboard.
• Academic literature mining – a research group uses the model to pull out “method‑used‑for‑problem” relations from computer‑science papers.

Integration possibilities

• Hugging Face pipeline API (Python, JavaScript) for rapid prototyping.
• Azure Machine Learning endpoints – the model tag includes deploy:azure, enabling one‑click deployment on Azure Container Instances.
• ONNX Runtime – export the model to ONNX for low‑latency inference on edge devices or CPU‑only servers.

Training Details

Methodology

• The model is fine‑tuned from the pre‑trained facebook/bart-large checkpoint.
• Training objective: standard cross‑entropy loss on the linearised (subject, relation, object) sequence.
• Optimization: AdamW with a learning‑rate schedule that linearly warms up for 10 % of steps and then decays.

Datasets

• Babelscape/rebel‑dataset – a large‑scale, automatically annotated corpus covering 200+ relation types. The dataset is released under CC‑BY‑NC‑SA‑4.0.
• Fine‑tuning also incorporates the benchmark splits of CoNLL‑04 and NYT for evaluation.

Compute requirements

• Training was performed on 8 × NVIDIA A100 40 GB GPUs for roughly 48 hours (≈ 2 M training steps).
• Peak memory usage per GPU: ~ 13 GB (FP16) due to the large BART encoder‑decoder.
• Mixed‑precision (AMP) was used to accelerate training and reduce VRAM.

Fine‑tuning capabilities

• Because the model is a vanilla BART decoder, you can continue training on any domain‑specific RE dataset with the same Seq2SeqTrainer API.
• Only a few hundred labelled examples are needed to adapt the model to a new relation schema, thanks to the strong zero‑shot generalisation demonstrated in the original paper.

Related Papers

The core research behind rebel‑large is documented in the EMNLP 2021 paper:

• REBEL: Relation Extraction By End‑to‑End Language Generation (Huguet‑Cabot & Navigli, 2021). DOI: 10.18653/v1/2021.findings-emnlp.204.

The paper introduces a novel linearisation scheme that maps a relation triplet to a plain‑text sequence, enabling the use of autoregressive seq2seq models for RE. It also demonstrates that a BART‑based architecture can achieve state‑of‑the‑art results across multiple benchmarks without task‑specific classifiers.

Additional related work that inspired the design includes:

• “Sequence‑to‑Sequence Learning for Relation Extraction” – T5‑based approaches that treat RE as a generation problem.
• “Joint Entity and Relation Extraction as a Sequence Labeling Task” – early pipelines that motivated the move toward end‑to‑end generation.

Licensing Information

The model card lists the license as unknown, while the accompanying README states a CC‑BY‑NC‑SA‑4.0 licence for the underlying dataset. In practice, this means:

• Code & model weights: The “unknown” tag indicates that the repository has not explicitly declared a permissive licence for the model itself. Users should treat the model as “all‑rights‑reserved” until a licence is confirmed by the author.
• Commercial use: CC‑BY‑NC‑SA‑4.0 prohibits commercial exploitation of the dataset. If the model is trained solely on that dataset, commercial deployments may be restricted unless a separate commercial licence is obtained.
• Attribution: When using the model for research or non‑commercial projects, credit Babelscape and the original REBEL paper (see citation below).
• Modification & redistribution: The “SA” (Share‑Alike) clause would require derivative works to be released under the same licence, but this only applies if the CC licence is deemed to cover the model weights.

If you intend to embed rebel‑large in a product, we recommend contacting Babelscape directly to obtain a clear commercial licence or to verify that the “unknown” tag does not hide additional restrictions.