How Computers Store Text: ASCII and UTF-8

There are no letters in computer memory. Only bytes — numbers from 0 to 255, as we saw in the articles on binary notation and two's complement. All the text you are reading now is a sequence of numbers plus an agreement about which number means which letter. Let's walk through that agreement step by step — from ASCII in 1963 to UTF-8, which runs the internet.

Letters are numbers

A computer can only store and transmit bits. To store text, we need an agreed table: which number stands for which letter. Such an agreement is called a character encoding. An encoding is not a device or a program — just a convention, like Morse code: "•−" means A because everyone agreed so.

If two computers share one encoding, text arrives intact. If they use different ones, you see "mojibake": the numbers are the same, the letters are different.

ASCII: a table of 128 characters

In 1963 the USA standardized ASCII — the American Standard Code for Information Interchange. Seven bits give 128 positions, all occupied: Latin letters, digits, punctuation and control codes (line feed, bell). A fragment of the table:

There is neat arithmetic hidden in the table: the code of the digit "0" is 48, so "character minus 48" gives the digit's value. Uppercase and lowercase letters differ by exactly one bit: A = 65, a = 97, a difference of 32 — a single bit in the fifth place. Case conversion in any programming language is an AND/OR operation on one bit, which ties text processing to the circuitry of the masks article.

256 characters are not enough: UNICODE

One byte — 256 values: the Latin alphabet fits, but the Cyrillic, Greek, Hebrew and Arabic alphabets, and certainly Chinese characters, do not. Before the internet era, every "code page" was invented separately, and the same byte 0xD0 meant "Ð" in one encoding and "Р" in another. Exchanging texts became impossible.

The solution is UNICODE: one table for all the world's characters (over 150,000: every alphabet, Chinese characters, math symbols, emoji). Each character received a permanent number — a code point. The letter "Ж" is number 1046, wherever it appears.

UTF-8: clever packing

Storing UNICODE numbers naively is wasteful: the Latin "A" (number 65) fits in one byte, but if every character took four bytes, English text would quadruple in size. UTF-8 solves this with variable length:

— Characters with numbers 0–127 (all of ASCII) — 1 byte, indistinguishable from old ASCII.
— Cyrillic and most alphabets — 2 bytes.
— Chinese characters — 3 bytes, rare symbols and emoji — 4 bytes.

The length is built into the first byte itself: if the high bit is 0, it is one-byte ASCII; if the byte starts with 110, the character is 2 bytes long; 1110 — 3 bytes. Continuation bytes always start with 10. Thanks to this, UTF-8 has no "middle of a character" problem: even a randomly cut string reveals its boundaries. The letter "Ж" in UTF-8 is two bytes: 0xD0 0x96 (number 1046 = 0x416 packed by the scheme 110xxxxx 10xxxxxx).

That is why UTF-8 became the language of the internet: English text stays one byte per character, everything else is honestly encoded, and a single encoding covers the whole planet.