# tokenizer

> English word · Noun

## Definitions
1. A system that parses an input stream into its component tokens.

## Etymology
From tokenize + -er.

## Source
Compiled from Wiktionary via kaikki.org (CC BY-SA). Data vintage: 2026-05-06.
Canonical page: https://plainspell.com/en/word/tokenizer