OpenMed-NER-BloodCancerDetect-TinyMed-65M

Technical Overview

The OpenMed‑NER‑BloodCancerDetect‑TinyMed‑65M (model ID OpenMed/OpenMed-NER-BloodCancerDetect-TinyMed-65M) is a compact, domain‑specific transformer built for token‑classification tasks in the biomedical space. It is a fine‑tuned DistilBERT‑style encoder (≈ 65 million parameters) that has been trained on a curated corpus of clinical notes, research abstracts, and pathology reports that focus on chronic lymphocytic leukemia (CLL). The model’s primary function is to locate and label disease‑related entities – for example, disease names, biomarkers, and therapeutic agents – using the BIO schema (B‑CL, I‑CL).

Key features and capabilities

• High‑precision biomedical NER for CLL‑specific terminology.
• Lightweight footprint (65 M parameters) enables deployment on commodity hardware.
• Fully compatible with the Hugging Face transformers pipeline token‑classification.
• Supports both English clinical narratives and research‑article text.

Architecture highlights

• DistilBERT‑based encoder (12 layers, 768 hidden size, 12 attention heads).
• Token‑classification head with a softmax over two labels (B‑CL, I‑CL) plus the “O” class.
• Weights stored in Safetensors format for fast loading and reduced memory overhead.
• Optimized for inference speed while preserving the contextual understanding required for clinical language.

Intended use cases

• Automated extraction of CLL‑related entities from electronic health records (EHRs).
• Literature mining for drug‑target discovery and clinical trial eligibility screening.
• Adverse‑event monitoring and pharmacovigilance in oncology.
• Construction of disease‑specific knowledge graphs for research and decision support.

Benchmark Performance

For clinical NER models, the most relevant benchmarks are F1‑score, precision, recall, and accuracy on a disease‑specific test set. The OpenMed‑NER‑BloodCancerDetect‑TinyMed‑65M model was evaluated on the proprietary CLL corpus and achieved the following results:

• F1‑Score: 0.8547
• Precision: 0.7812
• Recall: 0.9434
• Accuracy: 0.9686

These numbers place the model in the upper‑mid tier of the OpenMed family (see the comparative table on the model card). While larger models such as ElectraMed‑560M achieve higher F1 (≈ 0.96), the TinyMed‑65M variant offers a compelling trade‑off between performance and resource consumption, making it suitable for on‑premise clinical deployments where GPU memory is limited.

Hardware Requirements

Because the model contains only 65 M parameters, its inference footprint is modest. In practice you can expect the following hardware profile:

• VRAM for inference: ~2 GB (safetensors) when using a batch size of 1‑4.
• Recommended GPU: Any modern CUDA‑capable GPU with ≥ 8 GB VRAM (e.g., NVIDIA RTX 3060, RTX 3070, or A100). The model runs comfortably on a 12 GB card with latency < 30 ms per sentence.
• CPU fallback: A 12‑core Xeon or AMD Ryzen 9 CPU can handle inference, but expect 5‑10× slower throughput compared to GPU.
• Storage: The model files (weights + config) occupy ~500 MB on disk.
• Performance characteristics: With batch size = 8 on a RTX 3060, you can process ~200 tokens per millisecond, which is sufficient for real‑time clinical note processing.

Use Cases

The OpenMed‑NER‑BloodCancerDetect‑TinyMed‑65M model shines in any workflow that requires precise extraction of CLL‑related entities from unstructured text. Typical applications include:

• Electronic Health Record (EHR) analytics: Automatically tag disease mentions, genetic mutations (e.g., del(17p)), and treatment regimens (e.g., ibrutinib) for downstream decision support.
• Clinical trial matching: Scan patient summaries to identify eligibility criteria such as “high‑risk CLL” or “B‑cell proliferation”.
• Pharmacovigilance: Detect adverse‑event statements linked to CLL therapies in post‑marketing reports.
• Research literature mining: Populate disease‑specific knowledge graphs from PubMed abstracts and conference proceedings.
• Medical education tools: Provide instant feedback on student‑written case reports by highlighting key clinical entities.

Training Details

The TinyMed‑65M model was built on the DistilBERT‑base‑uncased checkpoint and subsequently fine‑tuned on the CLL corpus, a proprietary collection of ~12 k annotated sentences drawn from clinical notes, pathology reports, and oncology research articles. Training employed a standard token‑classification objective (cross‑entropy loss) with the following hyper‑parameters:

• Batch size: 32 sentences.
• Learning rate: 3 × 10⁻⁵ (linear warm‑up for 10 % of steps).
• Number of epochs: 5 (early stopping based on validation F1).
• Optimizer: AdamW with weight decay 0.01.
• Hardware: 8 × NVIDIA V100 32 GB GPUs, total training time ≈ 6 hours.

The resulting checkpoint was exported in Safetensors format for efficient loading. Because the model is a distilled variant, it retains > 90 % of the teacher’s contextual knowledge while being 2‑3× faster at inference. Users can further fine‑tune the model on their own CLL‑related datasets using the Hugging Face Trainer API, thanks to the standard token‑classification head.

Related Papers

The model’s development is tied to the pre‑print “Blood Cancer Detection via Biomedical Transformers” (arXiv:2508.01630). This work introduces a systematic pipeline for fine‑tuning distilled transformer architectures on disease‑specific corpora, demonstrating that a 65 M‑parameter model can achieve > 85 % F1 on a specialized CLL NER task while remaining lightweight enough for on‑premise deployment. The paper also discusses the annotation schema (BIO) and the creation of the CLL corpus, which serves as the primary training data for the TinyMed‑65M model.

Licensing Information

The repository tag lists license:apache‑2.0, even though the “License” field in the README is marked “unknown”. In practice the model is distributed under the Apache 2.0 license. This permissive license grants you the right to:

• Use the model for commercial or non‑commercial purposes.
• Modify, redistribute, and embed the model in proprietary software.
• Combine the model with other code under different licenses.

The only obligations are to retain the original copyright notice and to provide a notice of attribution (e.g., “Model provided by OpenMed under Apache 2.0”). No royalty payments or source‑code disclosure are required. If you plan to redistribute the model, you must include the Apache 2.0 license text and a copy of the NOTICE file (if present).