Technical Overview
Model ID: nie3e/sentiment-polish-gpt2-small
Model name: sentiment-polish-gpt2-small
Author: nie3e
This model is a Polish‑language sentiment classifier built on top of the Polish‑GPT‑2‑Small transformer. It has been fine‑tuned for the Polemo2‑official dataset, which contains manually annotated sentences with four sentiment categories: NEUTRAL, NEGATIVE, POSITIVE and AMBIGUOUS. The model accepts raw Polish text and returns a probability distribution over these four classes.
Key Features & Capabilities
- Polish‑native language understanding: Trained on a large Polish corpus, the model captures idiomatic expressions, diacritics, and syntax specific to Polish.
- Four‑class sentiment detection: Unlike binary classifiers, it distinguishes ambiguous sentiment, which is crucial for nuanced user‑generated content.
- Lightweight architecture: Based on the 124‑M‑parameter GPT‑2‑Small backbone, it runs comfortably on consumer‑grade GPUs.
- Multiple inference options: Works with
AutoModelForSequenceClassification, thepipelineAPI, vLLM, and OpenAI‑compatible serving. - High accuracy: Achieves 96.27 % accuracy on the official test split (see Benchmark Performance).
Architecture Highlights
The underlying architecture is the standard GPT‑2 transformer decoder stack with 12 layers, 12 attention heads, and a hidden size of 768. For classification, a linear head is added on top of the final hidden state of the [CLS] token (or the first token in GPT‑2’s case). The model was fine‑tuned using a binary cross‑entropy loss with logits (i.e., torch.nn.BCEWithLogitsLoss) to accommodate the multi‑label nature of the four sentiment classes.
Intended Use Cases
- Social‑media monitoring for Polish brands.
- Customer‑review analysis on e‑commerce platforms.
- Sentiment‑aware chat‑bots and virtual assistants serving Polish speakers.
- Academic research on Polish sentiment datasets.
Benchmark Performance
For a sentiment‑analysis model, the most relevant benchmark is the classification accuracy on a held‑out test set. The model was evaluated on the Polemo2‑official test split (20 % of the data). The reported metrics are:
- Loss: 0.4659
- Accuracy: 96.27 %
These numbers indicate that the model correctly predicts the sentiment label for roughly 96 out of 100 Polish sentences, a level of performance comparable to state‑of‑the‑art Polish sentiment classifiers that use larger transformer backbones. The high accuracy, combined with a small parameter count, makes it an attractive choice for production environments where latency and memory are constraints.
Hardware Requirements
VRAM & Inference
- Model size (including tokenizer and classification head): ~500 MB.
- Typical GPU memory consumption for a single‑sentence batch: 2–3 GB VRAM.
- Batch inference (e.g., 16 sentences) stays under 6 GB VRAM on a modern GPU.
Recommended GPU
Any GPU with at least 4 GB of VRAM will comfortably run the model in inference mode. For higher throughput, a RTX 3060 (12 GB) or RTX A5000 (24 GB) is recommended.
CPU & Storage
- CPU‑only inference is possible but will be slower (≈ 150 ms per sentence on a 12‑core Intel i7).
- Disk space: ~600 MB for the model files (including safetensors and tokenizer).
- SSD storage is advised to reduce loading latency.
Performance Characteristics
On a RTX 3080 (10 GB VRAM) the model processes ~120 tokens per millisecond, yielding sub‑100 ms latency for typical review sentences (≈ 30 tokens). The vLLM integration further reduces latency by leveraging compiled kernels and asynchronous batching.
Use Cases
- Brand sentiment monitoring: Automatically tag Polish tweets, Facebook posts, and Instagram comments as positive, negative, neutral, or ambiguous.
- Review aggregation: Summarize customer feedback on Polish e‑commerce sites (e.g., Allegro) to surface trending sentiment.
- Chat‑bot emotion awareness: Adjust bot responses based on the detected sentiment of a user’s message.
- Academic corpus analysis: Researchers can quickly label large Polish text corpora for sentiment‑related studies.
- Call‑center analytics: Transcribe Polish calls and run the classifier on the transcript to gauge caller mood.
Training Details
Methodology
The model was trained using the Trainer API from the 🤗 Transformers library. A DataCollatorWithPadding was employed to batch variable‑length sentences, and the loss function was binary cross‑entropy with logits to handle the four mutually exclusive sentiment labels.
Dataset
- Source: Polemo2‑official (Polish sentiment dataset).
- Split: 80 % training, 20 % testing.
- Classes: NEUTRAL, NEGATIVE, POSITIVE, AMBIGUOUS.
Compute Requirements
Training was performed on a single NVIDIA RTX 3090 (24 GB VRAM) for approximately 3 hours, using a batch size of 32 and a learning rate of 2e‑5. The total compute footprint is roughly 0.5 GPU‑hours, reflecting the efficiency of fine‑tuning a small GPT‑2 variant.
Fine‑tuning Capabilities
Because the model is based on a standard 🤗 Transformers checkpoint, you can further fine‑tune it on domain‑specific Polish data (e.g., finance or healthcare reviews) by re‑using the same Trainer pipeline. The classification head can be re‑initialized if you need a different number of classes.
Licensing Information
The model card lists the license as MIT in the model-index section, but the top‑level License field is marked “unknown”. In practice, the MIT license is permissive, allowing:
- Commercial and non‑commercial use.
- Modification, redistribution, and private use without royalties.
- Inclusion in proprietary software, provided the original copyright notice is retained.
Because the top‑level field is “unknown”, you should verify the exact licensing terms on the repository’s files page. If the MIT license is confirmed, you may safely deploy the model in production, embed it in SaaS products, or ship it with hardware devices.
Attribution
When using the model, include a citation to the original Hugging Face repository and, if possible, to the Polemo2‑official dataset. Example attribution:
Model:
nie3e/sentiment-polish-gpt2-small– licensed under MIT. Dataset:clarin-pl/polemo2-official.