fix bug in tokenizer `decode` method
#84
by njunlp - opened
the _decode method of ChatGLMTokenizer class should use customized _decode in sp_tokenizer rather than that in parent class, otherwise, the negative id would never be filtered.
the _decode method of ChatGLMTokenizer class should use customized _decode in sp_tokenizer rather than that in parent class, otherwise, the negative id would never be filtered.