fai-build-tokenizer

Implement custom tokenizers for domain-specific AI models optimizing vocabulary size and encoding efficiency.

Overview

Usage

Reference this skill in your fai-manifest.json:

{
  "primitives": {
    "skills": ["../../skills/fai-build-tokenizer/SKILL.md"]
  }
}

Source

• GitHub: skills/fai-build-tokenizer/SKILL.md

Auto-generated from the FrootAI primitive catalog .