sage-bergerson/whisper-large-edacc

Whisper Large EdAcc

This model is a fine-tuned version of openai/whisper-large on the EdAcc dataset. It achieves the following results on the evaluation set:

• Loss: 1.0862

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 32
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.8893	0.6494	200	0.6173
0.4959	1.2987	400	0.5871
0.4654	1.9481	600	0.5799
0.308	2.5974	800	0.6095
0.2504	3.2468	1000	0.6823
0.1877	3.8961	1200	0.6828
0.1028	4.5455	1400	0.7804
0.0896	5.1948	1600	0.8240
0.0516	5.8442	1800	0.8491
0.0291	6.4935	2000	0.9035
0.0276	7.1429	2200	0.9402
0.0141	7.7922	2400	0.9443
0.0098	8.4416	2600	0.9972
0.0073	9.0909	2800	1.0118
0.0056	9.7403	3000	1.0176
0.0027	10.3896	3200	1.0468
0.0021	11.0390	3400	1.0564
0.0016	11.6883	3600	1.0703
0.0009	12.3377	3800	1.0840
0.0011	12.9870	4000	1.0862

Framework versions

• Transformers 4.44.0
• Pytorch 2.3.1
• Datasets 2.21.0
• Tokenizers 0.19.1