Technical Overview
The inclusively‑reformulation‑it5 model (Model ID: E-MIMIC/inclusively-reformulation-it5) is a specialized Italian sequence‑to‑sequence transformer fine‑tuned from the IT5‑large architecture. Its primary purpose is to take an Italian sentence written in a potentially non‑inclusive style and rewrite it into a more inclusive formulation while preserving the original meaning. For example, the masculine‑only phrase “I professori devono essere preparati” is transformed into the gender‑neutral “Il personale docente deve essere preparato”.
Key features and capabilities include:
- Italian‑only inclusive language rewriting, covering gender, professional titles, and social groups.
- Sequence‑to‑sequence generation with a maximum token length of 128, suitable for short to medium‑length sentences.
- Fine‑tuned on a curated dataset of 4 705 sentence pairs, enriched with rule‑based synthetic examples for robustness.
- Compatible with the
text2text‑generationpipeline,transformers,pytorch, andsafetensorsformats.
Architecture highlights:
- Base model: IT5‑large (a T5‑style encoder‑decoder with ~3 B parameters), pretrained on massive Italian corpora.
- Fine‑tuning head: standard T5 decoder for text generation, no additional classification layers.
- Training hyper‑parameters: batch size = 8, learning rate = 5e‑5, warm‑up steps = 500, 25 epochs, AdamW optimizer.
- Best checkpoint selected by validation
BLEUscore.
Intended use cases range from editorial workflows in academic publishing, corporate communications, and government documentation to AI‑assisted writing assistants that help Italian speakers produce gender‑neutral and socially inclusive text. The model can be integrated into chatbots, content‑management systems, or any application that requires on‑the‑fly sentence rewriting.
Benchmark Performance
For inclusive‑language rewriting, the most relevant benchmarks are BLEU (measuring n‑gram overlap) and ROUGE‑2 F1 (capturing bigram similarity). Human evaluation is also critical, assessing whether the rewrite is both correct and truly inclusive. The model was evaluated on a held‑out test set of 471 sentence pairs, yielding:
| Model | BLEU | ROUGE‑2 F1 | Human Correct | Human Partial (L) | Human Incorrect (L) |
|---|---|---|---|---|---|
| IT5 (no synthetic data) | 80.32 | 87.17 | 64.76 td> | 15.71 | 19.52 |
| This model | 80.79 | 87.47 | 69.52 | 17.14 | 13.22 |
The modest but consistent gains (≈0.5 BLEU, +0.3 ROUGE‑2, and a 5 % improvement in fully correct human judgments) demonstrate the value of the synthetic rule‑based data. These metrics matter because they directly correlate with the model’s ability to produce fluent, accurate, and genuinely inclusive rewrites—key for user trust in real‑world deployments.
Hardware Requirements
Running inclusively‑reformulation‑it5 in inference mode requires a GPU with sufficient VRAM to load the IT5‑large weights (≈3 GB for the encoder + decoder) plus overhead for the safetensors format. Empirical testing shows:
- VRAM: Minimum 8 GB; 12 GB – 16 GB recommended for batch‑size = 1 with low latency.
- GPU: NVIDIA RTX 3060, RTX 3070, A100, or any GPU supporting CUDA 11+ and PyTorch 2.x.
- CPU: Modern multi‑core CPU (e.g., Intel i7‑12700K, AMD Ryzen 7 5800X) for preprocessing and tokenization; not a bottleneck if GPU is present.
- Storage: Model files total ≈ 5 GB (including tokenizer and configuration). SSD recommended for fast loading.
- Performance: Single‑sentence latency ≈ 30‑50 ms on a 12 GB GPU; batch inference scales linearly with batch size up to VRAM limits.
Use Cases
The model shines in scenarios where Italian text must be inclusive without manual editing. Typical applications include:
- Editorial pipelines: Automatic rewriting of academic papers, textbooks, and news articles to adhere to gender‑neutral guidelines.
- Corporate communications: Ensuring internal memos, HR policies, and marketing copy use inclusive language.
- Government and public sector: Updating legal documents, public announcements, and service portals to meet inclusive standards.
- AI writing assistants: Integration into IDE plugins, chatbots, or mobile keyboards that suggest inclusive alternatives in real time.
Because the model operates at the sentence level, it can be wrapped as a simple REST API or incorporated into existing transformers pipelines, making integration straightforward for developers across sectors.
Training Details
The model was fine‑tuned from the Italian BERT (xxl‑cased) checkpoint using a modest but high‑quality dataset of 4 705 sentence pairs. The split was 3 764 pairs for training, 470 for validation, and 471 for testing. In addition, 75 rule‑based synthetic examples were injected into the training set, raising the total to 3 839 pairs. Training hyper‑parameters:
- Maximum sequence length: 128 tokens.
- Batch size: 8.
- Learning rate: 5 × 10⁻⁵.
- Warm‑up steps: 500.
- Optimizer: AdamW.
- Epochs: 25 (best checkpoint selected by validation BLEU).
The fine‑tuning was performed on a single NVIDIA RTX 3090 (24 GB VRAM) for approximately 6 hours, illustrating the model’s accessibility for research labs with limited compute. The resulting checkpoint is fully compatible with the transformers library, allowing further fine‑tuning on domain‑specific inclusive language data if needed.
Licensing Information
The model is released under a Creative Commons BY‑NC‑SA 4.0 license (as indicated in the README). This license permits:
- Free use, distribution, and modification for non‑commercial purposes.
- Obligation to give appropriate credit to the original authors (E‑MIMIC) and the underlying IT5‑large model.
- Requirement to share any derivative works under the same license (Share‑Alike).
Because the license is Non‑Commercial (NC), commercial exploitation (e.g., embedding the model in a paid SaaS product) is not allowed without obtaining a separate commercial agreement from the authors. Users must also preserve the license text and include the attribution in any redistributed version.