"Unknown model size" when submitting model

#945
by alvations - opened

When trying to submit the google/mt5-large model for LLM leaderboard, it throws a "Unknown model size" error.

image.png

This error did not happen when submitting google/mt5-base or google/mt5-xl model.

Any way to resolve this?

It also happened for umt5-small

image.png

Open LLM Leaderboard org

Hi! We extract model size from the safetensors weights (or the name in last resort), so you would need to provide models stored as safetensors - it's also a safer format for model storing than pickles for example.

For these older models that are not in safetensors, can there be (i) exceptions made or (ii) some way to rewrite the models artefacts to add the safetensors to the model files?

Sign up or log in to comment