What is UTF-8? Code points to bytes, step by step

Type the single character é into the Unicode converter. The Unicode panel reports the code point, U+00E9, and the UTF-8 code units panel reports two bytes: C3 A9. Those two lines describe the whole job of UTF-8. Unicode assigns every character a number (see What is Unicode?), and UTF-8 is the standard recipe for turning that number into bytes that a file, a network socket or a database can actually store.

The recipe is defined by the Unicode Standard and published by the IETF as RFC 3629, also known as STD 63. It is the one encoding the web has settled on: the WHATWG Encoding Standard requires new protocols and formats to use UTF-8 exclusively, and browsers treat it as the default for pages that declare nothing at all.

The official range table

UTF-8 encodes each code point in one to four bytes, depending on where the number falls. This is the table from RFC 3629, section 3. The x positions are the payload: they carry the bits of the code point itself.

Code point range    Bytes  Byte pattern
U+0000..U+007F      1      0xxxxxxx
U+0080..U+07FF      2      110xxxxx 10xxxxxx
U+0800..U+FFFF      3      1110xxxx 10xxxxxx 10xxxxxx
U+10000..U+10FFFF   4      11110xxx 10xxxxxx 10xxxxxx 10xxxxxx

Two rules make the scheme readable. In a multi-byte sequence, the lead byte announces the length by counting its leading 1 bits: 110 means two bytes total, 1110 three, 11110 four. Every trailing byte starts with 10 and carries six payload bits.

Walking 中 through the bit layout

Take the character 中, CJK UNIFIED IDEOGRAPH-4E2D, from the CJK Unified Ideographs block. Its code point U+4E2D sits in the third row of the table, so it needs three bytes with sixteen payload bits. Write the code point in binary, then pour the bits into the template from the right:

U+4E2D in binary: 0100 1110 0010 1101      (16 bits)
Template:         1110xxxx 10xxxxxx 10xxxxxx  (16 x slots)

  byte 3 takes the low 6 bits:  10 101101  -> AD
  byte 2 takes the next 6:      10 111000  -> B8
  byte 1 takes the last 4:      1110 0100  -> E4

Result: E4 B8 AD

The UTF-8 code units panel confirms exactly that: 中 becomes E4 B8 AD. Decoding runs the same path in reverse. A decoder reads E4, sees the 1110 prefix and knows to collect two more bytes, then masks off the prefixes and concatenates the payload bits: 0100 111000 101101 is U+4E2D. The same procedure gives é its two bytes: U+00E9 in binary is 11101001, padded into the eleven payload slots of the two-byte row as 110 00011 and 10 101001, which is C3 A9. A supplementary character like 😀 U+1F600 GRINNING FACE fills all four bytes: F0 9F 98 80.

ASCII compatibility and self-synchronization

Row one of the table is the design's masterstroke. Every code point from U+0000 to U+007F encodes as the single byte with that same value, so the letter A is byte 41 in ASCII and byte 41 in UTF-8. A plain ASCII file is already a valid UTF-8 file, and ASCII bytes never appear inside a multi-byte sequence. Software that scans bytes for ASCII markers, HTML tags, JSON punctuation, C format strings, works on UTF-8 text without knowing anything about Unicode.

Any byte tells you its role at a glance. Below 80 it is ASCII, 80 to BF it is a trail byte, C2 to F4 it is a lead byte whose length you can count. C0, C1 and F5 through FF never occur in valid UTF-8 at all.

That last property is called self-synchronization. If a decoder is dropped into the middle of a byte stream, it finds the next character boundary by scanning forward to the first byte that does not start with 10. Corruption in one character cannot shift the framing of everything after it, and byte-oriented search algorithms work without false matches across character boundaries.

Why overlong encodings are illegal

The rows of the table are mutually exclusive: there is exactly one valid byte sequence per code point, the shortest one. Nothing stops you from pouring U+00E9 into the three-byte template as E0 83 A9, and a naive decoder would even read it back as é. That sequence is an overlong encoding and it is invalid UTF-8. The NUL character U+0000 as C0 80 is the classic case.

This is a security rule, not pedantry. RFC 3629, section 10 describes filters that block the byte sequence for ../ yet accept the illegal form C0 AE for the dot, a hole a widespread worm used against web servers in 2001. Any check that compares raw bytes can be bypassed by a spelling the checker did not expect, so decoders MUST reject overlong forms outright. The same section of the standard bans encoding the surrogate code points U+D800 through U+DFFF, which exist only as UTF-16 bookkeeping. The variant that does encode them is CESU-8, a different format that is not UTF-8 and not meant for interchange.

The BOM is optional, and the web discourages it

Some editors start UTF-8 files with the bytes EF BB BF, the UTF-8 form of U+FEFF, officially named ZERO WIDTH NO-BREAK SPACE and known informally as the byte order mark. In UTF-16 the mark settles which of the two byte orders a file uses. UTF-8 has exactly one byte order, so the mark says nothing about order; at best it announces "this file is UTF-8," at worst it is three stray bytes that leak into output or break concatenation. RFC 3629 recommends that protocols which already mandate UTF-8 forbid the signature, and HTML authoring guidance says the same: declare the encoding instead, with a <meta charset="utf-8"> tag or the Content-Type header. For genuine word-joining semantics in text, Unicode 3.2 added U+2060 WORD JOINER so that U+FEFF can be left to its signature role.

Mojibake: how café becomes café

Save café as UTF-8 and the bytes are 63 61 66 C3 A9. Now hand those bytes to a program that decodes them as Windows-1252. Byte C3 is U+00C3 LATIN CAPITAL LETTER A WITH TILDE and byte A9 is U+00A9 COPYRIGHT SIGN, so the screen shows café. That garbling is mojibake: correct bytes, wrong decoding assumption. Note the signature pattern. Text that shows à followed by another accented or symbol character is almost always UTF-8 that was decoded as Latin-1 or Windows-1252.

The repair is to go back to the original bytes and decode them as UTF-8. Paste C3 A9 into the UTF-8 code units panel of the converter and the Characters panel answers é. Resaving the visible mojibake as UTF-8 makes things worse: é encodes to C3 83 C2 A9, a double encoding that needs two rounds of repair to undo.

Putting it to work

When bytes and characters disagree, the numbers settle it. Paste the text into the UTF-8 converter and work from evidence:

• The UTF-8 code units panel shows the exact bytes, and it accepts hex bytes back, so E4 B8 AD pasted there decodes to 中.
• Declare UTF-8 explicitly in every page and API response rather than relying on a BOM, and save files without one.
• Treat overlong sequences, surrogate encodings and the never-used bytes C0, C1 and F5 through FF as hard errors, never as text.
• When you see é-style mojibake, recover the original bytes and re-decode them as UTF-8; do not re-encode the garbled characters.

UTF-8 shares the codespace with UTF-16 and UTF-32, which make different trade-offs on code unit size. How those three encodings divide up the same code points, and why UTF-16 needs surrogate pairs, is covered in the companion articles on UTF-16 and the three-way comparison.