Benchmarks for Slovak
Compare models on Slovak data using a consistent methodology.
SLAIH compares AI models on Slovak data. Multilingual leaderboards often omit Slovak or include only a small amount of Slovak data. These benchmarks focus on Slovak and evaluate models on tasks ranging from speech recognition to text understanding. Each benchmark includes results, metric explanations, and links to publications.
For the taskTranscribe speech
Slovak Speech Recognition (ASR) Benchmark
This is the first dedicated Speech-to-Text (STT) benchmark for Slovak: 39 hours of curated audio across 31 models. Existing resources focus on English or multilingual datasets where Slovak is only a few percent of the data. This benchmark fills that gap, comparing models on Slovak data across different domains.
- Metric
- Word error rate (WER)
- Metric direction
- lower is better
- Updated
Full leaderboard31 evaluated models- 1Microsoft MAI-Transcribe-26.54%
- 2Google Chirp 38.14%
- 3ElevenLabs Scribe v29.15%
- 4OpenAI GPT Realtime Whisper10.94%
- 5Microsoft Azure Speech to Text13.29%
For the taskWrite in Slovak
sk-bench – a benchmark for evaluating generative language models (LLMs) in Slovak
sk-bench measures how well large language models handle Slovak. It consists of 30 datasets from ten domains. Most of the data was originally created in Slovak, and translations were used only where no Slovak original existed. All models are evaluated using the same methodology and the same prompts. Results are reported as an overall score and separately for six task groups. The table below ranks multiple models by their overall score. Some of them were also evaluated with thinking mode enabled, so they appear in the table twice.
- Metric
- Overall
- Metric direction
- higher is better
- Updated
Full leaderboard79 evaluated models- 1Gemini 3.1 Pro(API)72.65
- 2GPT-5.5(API)71.02
- 3Claude Opus 4.6(API)68.13
- 4Gemini 3.1 Flash-Lite(API)66.69
- 5Gemma 4 31B(31B)65.97
For the taskUnderstand text
skLEP — Slovak Language Understanding Benchmark
skLEP is the first comprehensive benchmark designed specifically for evaluating Slovak natural language understanding (NLU) models. It covers nine tasks across three groups: token-level classification, sentence-pair tasks, and document-level classification. Created as the Slovak equivalent of the GLUE benchmark, it fills a long-standing gap in the systematic evaluation of language models for Slovak.
- Metric
- Average score across skLEP tasks
- Metric direction
- higher is better
- Updated
Full leaderboard16 evaluated models- 1mDeBERTaV3Base85.17
- 2FERNET-CCBase84.27
- 3mmBERTBase84.17
- 4SlovakBERT83.95
- 4DeBERTaV3Large83.95
For the taskSearch by meaning
SkMTEB — Slovak Text Embedding Benchmark
SkMTEB is the first comprehensive benchmark for evaluating text embedding models on Slovak, covering 31 datasets across 7 task types. It is the Slovak equivalent of the international MTEB benchmark, which has become the standard for comparing embedding models across languages. Text embeddings underpin modern AI applications — semantic search, RAG pipelines, and document classification — and SkMTEB enables their systematic evaluation for Slovak for the first time.
- Metric
- Average over all 31 datasets
- Metric direction
- higher is better
- Updated
Full leaderboard31 evaluated models- 1multilingual-e5-large-instruct560M77.49
- 2gemini-embedding-00177.23
- 3jina-embeddings-v30.6B75.10
- 4text-embedding-3-large75.07
- 5e5-sk-large365M74.70