camembert-ner-with-dates

Technical Overview

camembert-ner-with-dates is a French‑language token‑classification model that extends the popular camembert‑ner architecture with a dedicated DATE label. Built on the CamemBERT‑base transformer (a RoBERTa‑style model pre‑trained on 138 GB of French text), it has been fine‑tuned for Named‑Entity Recognition (NER) on an enriched version of the wikiner_fr corpus, adding roughly 170 k sentences that contain explicit date expressions. The model therefore predicts the usual French NER categories (PER, ORG, LOC, MISC) **and** reliably extracts date spans such as “le 1er avril 1976” or “le 9 janvier 2015”.

Key features and capabilities

• French‑only NER with a specialized DATE tag.
• High precision and recall (≈ 0.93 F1) on standard NER metrics.
• Robust handling of informal text (chat, email) – the author reports ~83 % F1 on mixed‑style test data, outperforming generic date‑parsers (~70 %).
• Compatible with Hugging Face pipeline('ner') and AutoModelForTokenClassification.
• Exportable to ONNX and Safetensors for low‑latency inference.
• Ready for deployment on Azure (region US) and other cloud endpoints.

Architecture highlights

• Base: CamemBERT‑base (12 layers, 768 hidden size, 12 heads, ~110 M parameters).
• Head: token‑classification layer with 5 output labels (PER, ORG, LOC, MISC, DATE).
• Training: supervised fine‑tuning on the wikiner_fr dataset with added date annotations.

Intended use cases

• Automated extraction of entities and dates from French documents (contracts, news articles, social media).
• Pre‑processing step for downstream pipelines that need structured date objects (e.g., feeding results to dateparser).
• Enterprise knowledge‑graph construction for French‑speaking markets.

Benchmark Performance

The model is evaluated with the seqeval metric suite, which is the de‑facto standard for token‑level NER tasks. Global scores are:

• Precision: 0.928
• Recall: 0.928
• F1‑score: 0.928

Per‑entity results (support = number of true spans) are:

• LOC – Precision 0.929, Recall 0.932, F1 0.931 (9 510 instances)
• PER – Precision 0.952, Recall 0.965, F1 0.959 (9 399 instances)
• MISC – Precision 0.878, Recall 0.844, F1 0.860 (5 364 instances)
• ORG – Precision 0.848, Recall 0.883, F1 0.865 (2 299 instances)
• DATE – not directly reported (method of insertion), but an estimated F1 of ~90 % is mentioned.

These benchmarks are crucial because French NER models often suffer from limited tag sets; the addition of a reliable date tag directly addresses a common pain point in information‑extraction pipelines. Compared with the original camembert‑ner (which lacks a date label) and generic date‑parsers, this model delivers a measurable boost in both entity detection and date resolution.

Hardware Requirements

VRAM for inference

• GPU with at least 4 GB VRAM can run the model using a batch size of 1 (typical for real‑time APIs).
• For higher throughput (batch ≥ 8) or multi‑sentence documents, 8 GB VRAM is recommended.

Recommended GPU

• NVIDIA RTX 3060 (12 GB) or higher for comfortable latency.
• For cloud deployment, any recent NVIDIA T4, A10, or A100 instance will provide sub‑50 ms per‑sentence latency.

CPU requirements

• Modern x86‑64 CPUs (Intel i7‑9700K, AMD Ryzen 7 3700X) can handle low‑volume inference when GPU is unavailable, though latency will increase to ~300 ms per sentence.
• Multi‑core support (≥ 4 cores) helps with tokenization and post‑processing.

Storage

• Model files (weights + tokenizer) occupy roughly 500 MB (safetensors + config).
• ONNX export adds an additional ~300 MB, useful for edge devices.

Performance characteristics

• Inference speed: ~200 tokens / ms on a RTX 3060 (single‑batch).
• Scales linearly with batch size; multi‑GPU inference can be achieved via accelerate or torch.distributed.

Use Cases

Primary applications

• Legal document analysis – automatically locate contract parties (ORG, PER) and contract dates.
• News monitoring – extract event dates and entities from French press releases.
• Customer support automation – parse dates from user messages (e.g., “Je veux réserver le 12 mai”).

Real‑world examples

• Financial services: ingesting French earnings reports to populate a time‑series database of corporate actions.
• Healthcare: extracting appointment dates and patient identifiers from French medical notes.
• Travel industry: identifying travel dates and destination locations from booking emails.

Integration possibilities

• Hugging Face pipeline for quick API exposure.
• ONNX runtime for low‑latency micro‑service containers.
• Azure Machine Learning endpoint (model is marked deploy:azure).

Training Details

The model was fine‑tuned on an enriched wikiner_fr corpus containing approximately 170 k French sentences. Date entities were added by a custom preprocessing script that inserted a DATE tag for any token matching a regular‑expression date pattern. Training was performed with the AutoModelForTokenClassification class from the 🤗 Transformers library, using the seqeval metric for early stopping.

Compute

• GPU: 1 × NVIDIA V100 (16 GB) for 3–4 hours.
• Batch size: 32 sequences (max length 128 tokens).
• Optimizer: AdamW with learning rate 5e‑5, weight decay 0.01.
• Training steps: ~10 k updates.

Fine‑tuning capabilities

• Users can further fine‑tune on domain‑specific French corpora (e.g., legal contracts) by loading the model with AutoModelForTokenClassification and continuing training on a small labeled set.
• Because the model uses a standard Hugging Face tokeniser, any additional special tokens can be added without breaking compatibility.

Related Papers

The model builds on the original CamemBERT architecture, described in Conneau et al., “CamemBERT: a T5‑style French language model” (arXiv:1911.03894). The NER fine‑tuning follows the methodology of the BERT‑based NER paper by Devlin et al. (arXiv:1802.00988). The underlying dataset, wikiner_fr, is documented in the original GitHub repository. No additional arXiv or DOI links are provided in the README, but the above works form the research backbone of this model.

Licensing Information

The model card lists the license as unknown, while the accompanying README states a MIT license for the underlying dataset (wikiner_fr) and for the model’s distribution on Hugging Face. In practice, this means:

• Users may copy, modify, and redistribute the model weights and code without paying royalties.
• Commercial use is permitted under MIT terms, provided the original copyright notice is retained.
• Because the model’s license is not explicitly declared on the hub, organisations should verify the exact terms before embedding it in proprietary software.
• Attribution is required: “Model — camembert‑ner‑with‑dates, © Jean‑Baptiste, licensed under MIT (where applicable).”