The hardware requirement for above model for training - GPU RAM size
#4
by
anilpatelia
- opened
Dear Team,
I am using wav2vec2-large-xlsr-53model for training the Indian Languages.
When I train on 6 GB GPU RAM, it gives me out of memory error.
Please let me know what is the minimum GPU-RAM required for training the model.
Anil
Hello, have you found an answer to this question?
Fine-tuning Wav2Vec2-XLSR-53 on a custom Japanese dataset with the following training arguments required a little less than 80 GB of GPU RAM for my use case, if I am understanding vast.ai's metrics correctly.
training_args = TrainingArguments(
output_dir="facebook-wav2vec2-large-xlsr-53-ja",
group_by_length=False,
per_device_train_batch_size=32,
evaluation_strategy="steps",
max_steps=655000,
fp16=False,
gradient_checkpointing=True,
save_steps=500,
eval_steps=500,
logging_steps=500,
learning_rate=3e-4,
warmup_steps=1000,
save_total_limit=2,
)