spacehub

surshar / claude-code

Claude Code is an agentic coding tool that lives in your terminal, understands your codebase, and helps you code faster by executing routine tasks, explaining complex code, and handling git workflows - all through natural language commands.

Built by Anthropic · github.com/anthropics

claude-code / mods / diff / hooks / word-diff / tokens-of / tokens-of.ts
9 lines · 354 BRaw
1/**
2 * A line as the tokens a word diff compares: runs of word characters, runs
3 * of whitespace (kept, as diffWordsWithSpace does), other characters alone.
4 *
5 * @param text one line without its diff marker
6 * @returns the tokens, which join back to `text`
7 */
8export const tokensOf = (text: string): string[] =>
9 text.match(/\w+|\s+|[^\w\s]/gu) ?? []