lexer_bash.ts

Hand-written Bash/shell lexer.

Emits: shebang, comment, string (double-quoted strings are containers whose $-expansions nest as variable/command_substitution), keyword, builtin, boolean, number, variable, command_substitution (a container), function, file_descriptor, operator, punctuation, plus heredocs (heredoc container + heredoc_delimiter).

Notable behavior:

• command substitution is recognized in both $(…) and legacy backtick form; both are command_substitution containers with an ordinary-bash interior.
• arithmetic expansion $((…)) is recognized as distinct from command substitution $(…); its $((/)) are punctuation and the interior lexes as ordinary bash (numbers, $vars, and general operators fall out naturally — no dedicated arithmetic token types).
• heredocs match any delimiter, honor <<-, and support quoted (no expansion) vs unquoted (expanded) bodies; multiple heredocs redirected on one line are queued and their bodies consumed in order.
• ${…} parameter expansion spans balanced braces — nested expansions (${a:-${b}}) and quoted }s stay inside the one variable token.

Word classification (keyword/builtin/boolean) is a context-free Map lookup.

Nesting ($(…)/$((…))/backtick interiors, double-quoted string bodies, unquoted heredoc bodies) runs on an explicit pooled frame stack, so arbitrarily deep input tokenizes fully without touching the JS call stack.

Scope: the bash family — registered as sh, with bash/shell as aliases. POSIX sh is a syntactic subset of bash for highlighting purposes (everything sh scripts use — $(…), $((…)), backticks, heredocs, ${…} — is shared syntax), and bash-only forms don't occur in sh input. Shells with their own syntax (fish, PowerShell) are out of scope.

Resilience: unterminated single-line constructs stop at the line boundary; unterminated strings, command substitutions, and heredocs extend to the window end. $(…)/$((…)) interiors discover their own closing delimiter during real tokenization (nothing is prescanned), so a ) inside a comment, string, or heredoc body never ends a substitution early. The tradeoff is that damage propagates rather than being contained: a malformed interior that consumes the closing ) (say an unterminated string) extends the substitution past it, editor-style.