persian-llama2 / tokenizer.json

Commit History

[Added] - Special Tokens to indicate Farsi and English text has been added
d6e48cb

amirakhlaghiqqq commited on