Tomeno/whisper-small-tok-synth-1

Whisper Small - Toki Pona - Synthetic Test 1

This experimental model is a fine-tuned version of openai/whisper-small on a mix of custom synthetic data and Common Voice 23.0 - Toki Pona. It achieves the following results on the evaluation set, made up exclusively of Common Voice data:

• Loss: 0.1691
• Wer: 10.0408

Model description

This is an experimental model trained for speech recognition for Toki Pona.

As the original model is multilingual with explicit language specification tokens, we have replaced the Czech (cs) language with Toki Pona, as we have determined it to have the closest phonetics.

The model's performance for other languages seems to have been at least partially preserved, but no testing has been done for other languages.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• training_steps: 1600
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.122	0.2156	100	0.2669	17.8776
0.0538	0.4313	200	0.1818	12.4490
0.0397	0.6469	300	0.1525	10.5714
0.0309	0.8625	400	0.1430	10.5306
0.0167	1.0776	500	0.1430	9.8367
0.0161	1.2933	600	0.1404	9.5918
0.0144	1.5089	700	0.1438	10.4082
0.0133	1.7245	800	0.1405	9.3061
0.012	1.9402	900	0.1440	10.4898
0.0064	2.1553	1000	0.1491	9.7143
0.0061	2.3709	1100	0.1498	10.1633
0.0056	2.5865	1200	0.1589	9.6327
0.0054	2.8022	1300	0.1616	9.8776
0.0051	3.0173	1400	0.1678	9.9184
0.0033	3.2329	1500	0.1673	10.2449
0.003	3.4485	1600	0.1691	10.0408

Framework versions

• Transformers 4.50.3
• Pytorch 2.9.0+cu126
• Datasets 3.6.0
• Tokenizers 0.21.4