ner-bert-base-cased-pt-lenerbr

Technical Overview

The ner‑bert‑base‑cased‑pt‑lenerbr model (model ID pierreguillou/ner-bert-base-cased-pt-lenerbr) is a Portuguese‑language Named Entity Recognition (NER) system built on top of the BERT‑base architecture. It has been fine‑tuned on the LeNER‑Br corpus, a collection of Brazilian legal texts annotated with legal‑domain entities such as Artigo, Lei, Jurisdição, and other statutory references. The model operates as a token‑classification pipeline, assigning an entity label to each token in an input sentence.

Key features and capabilities include:

• Portuguese‑only support (cased tokenizer) – preserving case‑sensitive legal terminology.
• Specialized legal vocabulary thanks to a prior language‑model fine‑tuning step on a masked‑language‑model objective.
• High‑precision entity extraction (precision ≈ 0.88) with strong recall (≈ 0.90) on the validation split.
• Compatibility with Hugging Face AutoModelForTokenClassification and AutoTokenizer for easy integration.
• Ready‑to‑deploy endpoints for Azure (tag deploy:azure) and other cloud services.

Architecture highlights:

• Base BERT (12‑layer, 768‑hidden, 12‑attention heads) – the same backbone as BERTimbau‑base.
• Fine‑tuned on a token‑classification head (linear layer) that maps the 768‑dimensional hidden states to the entity label space of LeNER‑Br.
• Trained on a single GPU in Google Colab, with early stopping that led to a modest over‑fit (loss ≈ 0.19).

Intended use cases revolve around the Brazilian legal sector: automated contract analysis, judicial opinion mining, statutory compliance checks, and any workflow that benefits from extracting structured legal entities from unstructured Portuguese text.

Benchmark Performance

The model’s evaluation was performed on the validation split of the LeNER‑Br dataset. The most relevant benchmarks for a legal‑domain NER system are F1‑score, Precision, Recall, and Accuracy. The reported numbers are:

• F1‑score: 0.8926
• Precision: 0.8810
• Recall: 0.9045
• Accuracy: 0.9759
• Loss (final): 0.1880

These metrics demonstrate that the model reliably identifies legal entities while maintaining a low false‑positive rate—critical for downstream legal analytics where precision is paramount. Compared to a baseline BERT‑base model that was not pre‑specialized on legal language (F1 ≈ 0.872), this version gains roughly 2 percentage points in F1, confirming the benefit of the intermediate masked‑language‑model fine‑tuning step. The larger counterpart (ner‑bert‑large‑cased‑pt‑lenerbr) pushes the F1 to 0.908, offering a trade‑off between size and accuracy.

Hardware Requirements

Because the model is based on BERT‑base (≈ 110 M parameters), inference can be performed on most modern GPUs with at least 4 GB of VRAM. For batch processing or low‑latency applications, a GPU with 8 GB VRAM (e.g., NVIDIA RTX 2070, RTX 3060, or A100) is recommended to comfortably hold the model weights and tokenized inputs.

• GPU: NVIDIA GeForce GTX 1060 6 GB or newer; CUDA 11+ for optimal transformer kernels.
• CPU: Any recent x86_64 or ARM64 CPU; 4 cores, 8 GB RAM is sufficient for single‑sentence inference.
• Storage: Model files occupy ~ 420 MB (weights + tokenizer). Keep at least 1 GB free for caching and temporary tensors.
• Performance: On a RTX 3060, latency per sentence (≈ 30 tokens) is ~ 30 ms; throughput can reach > 300 tokens / second with batch size = 8.

Use Cases

The model shines in any scenario where Portuguese legal text must be transformed into structured data. Typical applications include:

• Contract analysis: Automatic extraction of article numbers, law references, and jurisdiction clauses for contract‑review platforms.
• Judicial opinion mining: Tagging statutes and legal concepts in court decisions to enable semantic search and case‑law clustering.
• Regulatory compliance: Scanning corporate policies for mentions of specific legal provisions (e.g., labor law articles) to flag compliance gaps.
• Legal research assistants: Powering chat‑bots that can surface relevant statutes when a user asks a legal question.
• Public‑sector document management: Indexing legislative archives for fast retrieval of entity‑rich passages.

Training Details

Training followed a two‑stage pipeline:

1. Domain‑specific language‑model pre‑training: Starting from BERTimbau‑base, the model was further trained on a masked‑language‑model objective using the LeNER‑Br language‑modeling corpus. This step helped the transformer internalize legal terminology and syntax.
2. NER fine‑tuning: The specialized language model was then fine‑tuned on the token‑classification task with the LeNER‑Br annotated dataset. Training was performed in a Google Colab notebook (single GPU, ~ 12 GB VRAM) using the Trainer API from Hugging Face Transformers. Early stopping was triggered after the validation loss plateaued, resulting in a final loss of 0.188.

Key training hyper‑parameters (as inferred from the notebook):

• Batch size: 16 (gradient accumulation to emulate larger batches).
• Learning rate: 3e‑5 (AdamW optimizer).
• Number of epochs: 3–4 (over‑fit observed after epoch 3).
• Maximum sequence length: 128 tokens (sufficient for most legal sentences).

The model is fully compatible with the Hugging Face Trainer and can be re‑fine‑tuned on additional Portuguese legal corpora or adapted to a different entity schema by swapping the classification head.

Related Papers

While the README does not list specific DOI or arXiv links, the model’s foundation rests on two well‑known research strands:

• BERT: “BERT: Pre‑training of Deep Bidirectional Transformers for Language Understanding” (Devlin et al., 2018) – the core architecture.
• BERTimbau: “BERTimbau: Pre‑trained BERT for Portuguese” (Souza et al., 2020) – the base model that was later specialized on legal text.

The LeNER‑Br dataset itself is described in the paper “LeNER‑Br: A Corpus for Named Entity Recognition in Brazilian Legal Texts” (Guillou et al., 2021), which provides the annotation schema and benchmark splits used for this model.

Licensing Information

The model card lists the license as unknown. In practice, an “unknown” license means that the author has not explicitly granted any usage rights, and the default legal position is that the work is all‑rights‑reserved. Consequently, you should treat the model as non‑commercial until you obtain explicit permission from the author (pierreguillou).

• Commercial use: Not guaranteed. Contact the author for a commercial‑friendly license if you plan to embed the model in a product.
• Restrictions: Redistribution, modification, or public deployment may be limited without an explicit license grant.
• Attribution: Even with an unknown license, best practice is to cite the model card and the original LeNER‑Br dataset.

If you need a clear legal footing, consider using the underlying BERTimbau‑base model (licensed under the Apache 2.0 license) and fine‑tune it yourself on LeNER‑Br, which provides the same legal‑domain capabilities with a permissive license.