Issue with running some mistral models

[HarvieKrumpet]

Seen this happen on a few of the newer models. It loads ok, but upon tokenization getting a crash in...

llm.Tokenize(llmMessages).Length;
Non-negative parameter is required (count)

Mistral-Nemo-Instruct-12B-Philosophy-Math.i1-Q6_K.gguf

Have not seen the tokenizer ever crash before, I can't see anything I am feeding it that would cause this either...

[dranger003]

Did you use the one from here? https://huggingface.co/mradermacher/Mistral-Nemo-Instruct-12B-Philosophy-Math-i1-GGUF/tree/main

[dranger003]

Looks like this is possibly an issue upstream, take a look at this: ggml-org/llama.cpp#9864

You can apply this patch to fix the issue I think: https://github.com/ggerganov/llama.cpp/pull/9869/files