yunnian/fast-bytelevel-bpe-go

A blazing-fast Go trainer for HuggingFace-compatible ByteLevel BPE tokenizers, optimized for large JSONL datasets and up to 6x faster in local benchmarks.

语言：Go · GitHub 星标：9 · Fork：0

在 GitHub 上查看原仓库