yuriyvnv/parakeet-tdt-0.6b-EN-Medical

🩺 Parakeet-TDT-0.6B English Medical 🇬🇧

A fine-tune of nvidia/parakeet-tdt-0.6b-v3 on the English subset of MultiMed mixed with Common Voice 17 English (train + validation). The mix is the trick: it pushes the model toward medical vocabulary (TAVI, intervertebral disc herniation, drug names, dosing instructions) while keeping the everyday English it already knew.

Outputs cased English text with punctuation. Drop-in for the base Parakeet: same NeMo API, same long-form support, same timestamps.

🔥 Quick start

pip install nemo_toolkit[asr]

import nemo.collections.asr as nemo_asr

asr = nemo_asr.models.ASRModel.from_pretrained("yuriyvnv/parakeet-tdt-0.6b-EN-Medical")
print(asr.transcribe(["audio.wav"])[0].text)

📊 Results

One model, one training mix (MultiMed-en train + Common Voice 17-en train + validation, concatenated and shuffled per epoch — same .nemo for every row below). The two rows are the same checkpoint evaluated on two different held-out test sets: one in-domain (medical) and one out-of-domain (general English). Neither test set was seen during training.

The zero-shot column is the unmodified nvidia/parakeet-tdt-0.6b-v3, measured on the same test set with the same evaluator. All numbers are normalized (lowercase + strip punctuation), the standard protocol used by the MultiMed paper and the Open ASR Leaderboard, so they are directly comparable to other published results.

Test set (held out)	Samples	Zero-shot WER	Fine-tuned WER¹	Δ WER	Zero-shot CER	Fine-tuned CER¹
🩺 MultiMed English (test)	7,567	19.22	14.31	-4.91 (-25.5%)	14.16	10.37
🗣️ Common Voice 17 EN (test)	16,393	8.58	9.11	+0.53 (+6.2%)	4.01	4.10

For reference, the MultiMed paper's best published result is Whisper-Small multilingual fine-tune at 16.62% WER (arXiv 2409.14074, Table 6). This model beats that by 2.31 absolute / 14% relative, at 0.6B params.

¹ Both fine-tuned numbers are this single model — the one trained on the MultiMed-en + CV17-en mix.

Two things worth noting:

• 🎯 Medical: 25% relative WER reduction on MultiMed-en test (19.22% → 14.31%), beating the published MultiMed paper SOTA at 0.6B params.
• 🛡️ General English is essentially preserved. On CV17-en test the same model lands at 9.11% WER vs the base Parakeet's 8.58% — a +0.53 absolute / +6.2% relative change. That's the total cost of medical specialisation. A previous medical-only version of this model (trained on MultiMed alone, no CV) scored 13.53% normalized WER on CV17-en — a real catastrophic-forgetting hit. Including CV17-en in the training mix is what made the trade-off this cheap.

🛠️ Training

Property	Value
Base model	nvidia/parakeet-tdt-0.6b-v3
Architecture	FastConformer-TDT (~600M params)
Training data	CV17-en (train + validation) + MultiMed-en (train), concatenated, shuffled per-epoch
Total train clips	~1.07M (CV17-en ~97.5% / MultiMed-en ~2.5%)
Validation	MultiMed-en eval (~2,816 clips) — drives early stopping
Optimiser	AdamW (β=0.9/0.98, weight decay 1e-3)
Learning rate	5e-5 (cosine annealing, min 1e-6)
Warmup	10% of total steps
Batch size	32
Precision	bf16-mixed
Gradient clipping	1.0
Max clip duration	30 s
Early stopping	val_wer plateau, patience 10
Best epoch	31 (val WER 22.84%)
Hardware	Single NVIDIA H100

The natural CV17:MultiMed ratio (~97.5% : ~2.5%) means each batch of 32 contains on average ~0.8 medical clips and the rest general English — strong CV gradient keeps the base distribution intact while medical clips slowly steer the model into the clinical domain.

🎯 Intended use

• Transcription of English medical speech: clinical consultations, surgical procedures, patient narratives, medical podcasts, healthcare interviews.
• A drop-in replacement for nvidia/parakeet-tdt-0.6b-v3 in pipelines targeting medical English — picks up clinical vocabulary without breaking general transcription.

📚 Training data

• leduckhai/MultiMed English subset — multilingual medical ASR dataset (~84h English).
• fixie-ai/common_voice_17_0 English train + validation splits — Common Voice 17, crowdsourced English speech.

🙏 Acknowledgements

• NVIDIA NeMo team for releasing Parakeet-TDT-0.6B-v3 and the NeMo toolkit.
• Khai Le-Duc and the MultiMed authors for releasing the MultiMed multilingual medical ASR dataset.
• The Mozilla Common Voice community for the crowd-sourced English speech corpus used as the general-English anchor in the training mix.