How our Unicode text converter works under the hood

Open the Unicode converter and paste this line into the main input:

U+00E9 é é é 0xE9 %C3%A9 \u{1F600}

The Characters panel answers with é é é é é é 😀, and the fourteen other panels update at the same time. That hub-and-spoke shape is the whole architecture of the tool, and this post walks through it the way the code does: decode everything to characters, encode out to every format, and never let two formats convert to each other directly.

Decode everything to characters first

The main input accepts any mix of notations because it runs a fixed cascade of regex passes, one per syntax. This is the actual pipeline from lib/conversion/to-chars.ts:

export function convertAllEscapes(
  str: string,
  numbers: NumbersMode,
  singleLetterEscapes: boolean
): string {
  str = convertUnicode2Char(str)
  str = convert0x2Char(str)
  str = convertuBracket2Char(str)
  str = convertuBrSequence2Char(str)
  str = convertxBracket2Char(str)
  str = convertx002Char(str)
  str = convertHexNCR2Char(str)
  str = convertDecNCR2Char(str)
  str = convertU0000002Char(str)
  str = convertU00002Char(str)
  str = convertCSS2Char(str, false)
  str = convertpEnc2Char(str)
  str = convertEntities2Char(str)
  str = convertGreenNumbers2Char(str, numbers)

  if (singleLetterEscapes) str = convertSlashChar2Char(str)
  return str
}

Each pass consumes exactly one syntax in place, and the file header warns that the call order is sensitive. One concrete example of why: the 0x pass first swaps every 0x prefix for a placeholder, so a run like 0x1F4680x200D cannot be misread as one enormous number. Bare numbers go near the end because they carry no delimiter at all, and the single-letter pass (\n, \t and friends) runs last, gated by its own toggle. Numbers become characters through String.fromCodePoint, capped at U+10FFFF, so the pipeline thinks in code points from the very first step.

Encode once, fan out to fifteen panels

convertToAllFormats takes the decoded character string and produces every panel in a single pass. Typing é😀 yields:

Characters           é😀
XML/HTML entities    é😀
Unicode U+           U+00E9U+1F600
0x notation          0xE90x1F600
Hex NCRs             é😀
Decimal NCRs         é😀
JavaScript / Java    \u{E9}\u{1F600}
Rust                 \u{E9}\u{1F600}
Perl                 \x{E9}\x{1F600}
CSS                  \00E9 \01F600
Percent-encoded      %C3%A9%F0%9F%98%80
UTF-8 code units     C3 A9 F0 9F 98 80
UTF-16 code units    00E9 D83D DE00
Hex code points      00E9 1F600
Decimal code points  233 128512

Two things are worth noticing. First, the U+ and 0x panels run codes together with no separator, which is why they carry a Separate button that inserts spaces on demand. Second, the numbers are code points, not code units. The grinning face is the single code point U+1F600, yet the UTF-16 row needs two 16-bit code units for it: D83D is the lead surrogate and DE00 the trail surrogate (also called high and low). If that distinction is unfamiliar, our introduction to code points covers it.

The UTF-8 row is the only one that deals in bytes, and the encoder is the RFC 3629 bit layout written out by hand:

      if (cc <= 0x7f) {
        outputString += " " + dec2hex2(cc)
      } else if (cc <= 0x7ff) {
        outputString +=
          " " +
          dec2hex2(0xc0 | ((cc >> 6) & 0x1f)) +
          " " +
          dec2hex2(0x80 | (cc & 0x3f))
      } else if (cc <= 0xffff) {
        outputString +=
          " " +
          dec2hex2(0xe0 | ((cc >> 12) & 0x0f)) +
          " " +
          dec2hex2(0x80 | ((cc >> 6) & 0x3f)) +
          " " +
          dec2hex2(0x80 | (cc & 0x3f))
      }

Lead byte first, then continuation bytes in the 10xxxxxx pattern carrying six payload bits each: U+00E9 becomes C3 A9, U+1F600 becomes F0 9F 98 80. No library calls, just masks and shifts. The CSS panel is just as direct: a backslash, one to six hex digits and a trailing space, as CSS Syntax Level 3 defines.

Decode once, encode everywhere. Characters sit in the middle, and every format lives at the edge.

Every panel is also an input

Each panel has its own Convert button, and pressing it reverses the flow: convertPanelInput decodes that panel's text into characters, then convertToAllFormats refills every other panel while leaving the one you typed in untouched. Paste 00E9 D83D DE00 into the UTF-16 code units panel, press Convert, and the Characters panel says é😀 while the other fourteen panels follow. The panels are not read-only outputs. They are fifteen different editors for one underlying string.

Typing in the main input is debounced at 200 ms, so the full cascade runs when you pause instead of on every keystroke, and every explicit path (a Convert button, an option change) cancels the pending timer first, so a stale closure cannot overwrite fresh results.

Ambiguity is a user decision

Some inputs cannot be decoded safely by guessing, so the tool asks. The Treat bare numbers as control decides what a bare token like 00E9 means: plain digits by default, or hex code points, decimal code points, UTF-8 code units or UTF-16 code units. Set it to hex code points and 00E9 becomes é; type 233 with decimal code points selected and you get the same character. One honest caveat: the numbers are replaced where they sit, so the input 63 61 66 E9 decodes to c a f é, spaces included. And in hex mode the word cafe counts as a number, which is exactly why the default is Just numbers.

Extract escapes is the companion for messy prose. It scans the input and keeps only the escapes, so the sentence "The letter U+00E9 also appears as \u00E9 and é in text" collapses to U+00E9 \u00E9 é. It is a scanner rather than a parser: single-letter escapes and named entities like é pass through uncollected, a limitation the in-app help states outright.

Inspect reads a generated slice of the UCD

The Characters panel holds one more feature. Inspect opens a character inspector that names every character. For é it reports:

é   U+00E9   LATIN SMALL LETTER E WITH ACUTE   Latin-1 Supplement · Lowercase Letter · Latin

No server answers that lookup. We generate lib/unicode-data/data.json from the official Unicode Character Database 16.0.0 files (UnicodeData.txt, Blocks.txt, Scripts.txt) with a build script, and commit the 1.7 MB result. The inspector lazy-loads it through a dynamic import on first open, then answers queries with a Map for individually named characters and binary search over sorted range tables for blocks and scripts. Hangul syllable names are synthesized with the algorithm from section 3.12 of The Unicode Standard, and CJK ideograph names are composed from range labels, so the file does not store one row per character for the more than 154,000 assigned code points in Unicode 16.0.

Why we diff-test against the original GPL code

The conversion core is a TypeScript port of Richard Ishida's app-conversion, published under GPL v2, and the original JavaScript sits in our repository at lib/conversion/vendor. That directory exists for one reason: differential testing. Regex-heavy string code invites silent drift during a port, so lib/conversion/differential.test.ts loads the original into a Node vm sandbox and compares it against the port function by function.

The corpus mixes a fixed set of hostile inputs (combining marks, lone surrogates, bidi controls, and text that merely looks like escapes, such as cafe 0x41 U+00E9) with 200 fuzz strings drawn from a seeded PRNG, so failures reproduce exactly. Where we deliberately diverge from the original, the deviation is written down: the XML panel iterates by code point instead of by UTF-16 code unit, which lets it catch supplementary-plane invisibles like U+13430 EGYPTIAN HIEROGLYPH VERTICAL JOINER that the original loop can never match, and an empty-input quirk that returned undefined is normalized to an empty string. Everything else must agree byte for byte, and the suite fails if it does not.

What this means when you use it

• Paste anything into the main input of the converter; the Characters panel is the ground truth for what your escapes actually say.
• Treat every panel as an input: UTF-16 hex from a debugger, percent-encoded text from a URL, CSS escapes from a stylesheet.
• When your input is a bare hex dump, set Treat bare numbers as first, then run Extract escapes if prose is mixed in.
• Open Inspect to put an official name on an invisible character instead of guessing.

All of this runs in your browser. The converters are pure string functions, the UCD subset ships with the site, and nothing you paste is uploaded anywhere. Fifteen panels, one string, viewed fifteen ways.