Skip to content

Benchmarks for Slovak

Compare models on Slovak data using a consistent methodology.

SLAIH compares AI models on Slovak data. Multilingual leaderboards often omit Slovak or include only a small amount of Slovak data. These benchmarks focus on Slovak and evaluate models on tasks ranging from speech recognition to text understanding. Each benchmark includes results, metric explanations, and links to publications.

  • For the taskTranscribe speech

    Slovak Speech Recognition (ASR) Benchmark

    This is the first dedicated Speech-to-Text (STT) benchmark for Slovak: 39 hours of curated audio across 31 models. Existing resources focus on English or multilingual datasets where Slovak is only a few percent of the data. This benchmark fills that gap, comparing models on Slovak data across different domains.

    Metric
    Word error rate (WER)
    Metric direction
    lower is better
    Updated
    Full leaderboard31 evaluated models
    1. 1Microsoft MAI-Transcribe-26.54%
    2. 2Google Chirp 38.14%
    3. 3ElevenLabs Scribe v29.15%
    4. 4OpenAI GPT Realtime Whisper10.94%
    5. 5Microsoft Azure Speech to Text13.29%
  • For the taskWrite in Slovak

    sk-bench – a benchmark for evaluating generative language models (LLMs) in Slovak

    sk-bench measures how well large language models handle Slovak. It consists of 30 datasets from ten domains. Most of the data was originally created in Slovak, and translations were used only where no Slovak original existed. All models are evaluated using the same methodology and the same prompts. Results are reported as an overall score and separately for six task groups. The table below ranks multiple models by their overall score. Some of them were also evaluated with thinking mode enabled, so they appear in the table twice.

    Metric
    Overall
    Metric direction
    higher is better
    Updated
    Full leaderboard79 evaluated models
    1. 1Gemini 3.1 Pro(API)72.65
    2. 2GPT-5.5(API)71.02
    3. 3Claude Opus 4.6(API)68.13
    4. 4Gemini 3.1 Flash-Lite(API)66.69
    5. 5Gemma 4 31B(31B)65.97
  • For the taskUnderstand text

    skLEP — Slovak Language Understanding Benchmark

    skLEP is the first comprehensive benchmark designed specifically for evaluating Slovak natural language understanding (NLU) models. It covers nine tasks across three groups: token-level classification, sentence-pair tasks, and document-level classification. Created as the Slovak equivalent of the GLUE benchmark, it fills a long-standing gap in the systematic evaluation of language models for Slovak.

    Metric
    Average score across skLEP tasks
    Metric direction
    higher is better
    Updated
    Full leaderboard16 evaluated models
    1. 1mDeBERTaV3Base85.17
    2. 2FERNET-CCBase84.27
    3. 3mmBERTBase84.17
    4. 4SlovakBERT83.95
    5. 4DeBERTaV3Large83.95
  • For the taskSearch by meaning

    SkMTEB — Slovak Text Embedding Benchmark

    SkMTEB is the first comprehensive benchmark for evaluating text embedding models on Slovak, covering 31 datasets across 7 task types. It is the Slovak equivalent of the international MTEB benchmark, which has become the standard for comparing embedding models across languages. Text embeddings underpin modern AI applications — semantic search, RAG pipelines, and document classification — and SkMTEB enables their systematic evaluation for Slovak for the first time.

    Metric
    Average over all 31 datasets
    Metric direction
    higher is better
    Updated
    Full leaderboard31 evaluated models
    1. 1multilingual-e5-large-instruct560M77.49
    2. 2gemini-embedding-00177.23
    3. 3jina-embeddings-v30.6B75.10
    4. 4text-embedding-3-large75.07
    5. 5e5-sk-large365M74.70