Regarding weird Unicode.
#3
by
djuna
- opened
Have you try replacing the tokenizer with original llama 3.1? I hash the file, and it's different. Maybe that would help.
I didn't really try. By the time it started failing I didn't have the model files anymore, just a quantified GGUF. When I was merging Brinebreath I noticed some of my pre-merge files were corrupted by resuming download. This might be a cause of the weird output on Cathallama. You should probably still be able to get good output from the model by reverting the commit of llamacpp to the one I used to validate and quantitizing with that at 4_0, that's how I ran evaluation. The commit hash is on the readme.
Thanks. I might try with that
djuna
changed discussion status to
closed