liveBook · Manning

Encoding

As explained in Modern C

Encoding is the transformation of information into a representation defined by a scheme for storage, processing, transmission, or interpretation.

A newline is not necessarily a file byte

In C, fputs("hello\n", stdout) writes a text line. The program uses '\n' for its newline, but text output can convert that character to a platform-dependent external representation. Counting characters in the string therefore does not necessarily tell you how many bytes reach a file or console. section 8.3.3

Encoding determines how characters are represented. A C program can encounter different encodings in its source, during execution, and in external text. The C library handles text-I/O conversions where it can; portable string processing should avoid assuming a particular numeric code for a character. section 8.3.3 section 8.4

Three practical rules follow:

  • Use character constants such as 'a' and '\n', and string literals such as "hello:\tx", rather than substituting presumed numeric character codes. An integer character constant has type int, not char. section 8.4
  • Use <ctype.h> classifiers and case conversions for character classes instead of assuming every alphabetic character belongs to a fixed numeric range. Some classes may include platform-specific characters. section 8.4
  • Treat numeric escapes carefully: a hexadecimal escape consumes subsequent hexadecimal digits. Separating adjacent string literals can mark its end. A numeric escape compiling on multiple platforms does not guarantee that its character code exists on each one. section 8.4

TAKEAWAY: A text character, a numeric character code, and a byte of external text are not interchangeable ideas. The limited basic character set cannot express many languages and scripts. Multibyte text and wide characters provide a route to handling extended characters, but operations on them must account for conversion rather than simply treating each char as one character. section 14.3 section 8.3.3

43   #define VBAR "\u2502"      /**< a vertical bar character   */
44   #define HBAR "\u2500"      /**< a horizontal bar character */
45   #define TOPLEFT "\u250c"   /**< topleft corner character   */
46   #define TOPRIGHT "\u2510"  /**< topright corner character  */


!@%STYLE%@!
{"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":0,\"ch\":0},{\"line\":0,\"ch\":2}],[{\"line\":0,\"ch\":5},{\"line\":0,\"ch\":12}],[{\"line\":1,\"ch\":5},{\"line\":1,\"ch\":12}],[{\"line\":2,\"ch\":5},{\"line\":2,\"ch\":12}],[{\"line\":3,\"ch\":5},{\"line\":3,\"ch\":12}],[{\"line\":1,\"ch\":0},{\"line\":1,\"ch\":2}],[{\"line\":0,\"ch\":5},{\"line\":0,\"ch\":12}],[{\"line\":1,\"ch\":5},{\"line\":1,\"ch\":12}],[{\"line\":2,\"ch\":5},{\"line\":2,\"ch\":12}],[{\"line\":3,\"ch\":5},{\"line\":3,\"ch\":12}],[{\"line\":2,\"ch\":0},{\"line\":2,\"ch\":2}],[{\"line\":0,\"ch\":5},{\"line\":0,\"ch\":12}],[{\"line\":1,\"ch\":5},{\"line\":1,\"ch\":12}],[{\"line\":2,\"ch\":5},{\"line\":2,\"ch\":12}],[{\"line\":3,\"ch\":5},{\"line\":3,\"ch\":12}],[{\"line\":3,\"ch\":0},{\"line\":3,\"ch\":2}],[{\"line\":0,\"ch\":5},{\"line\":0,\"ch\":12}],[{\"line\":1,\"ch\":5},{\"line\":1,\"ch\":12}],[{\"line\":2,\"ch\":5},{\"line\":2,\"ch\":12}],[{\"line\":3,\"ch\":5},{\"line\":3,\"ch\":12}]]"}
!@%STYLE%@!

These string macros use \u escapes to identify box-drawing characters without writing their source-file glyphs directly. section 14.3

Measure characters after conversion

A separator-printing routine passes multibyte strings to fputs but uses mbsrlen to determine their character lengths before filling the remaining space with horizontal bars. Its helper calls mbsrtowcs to obtain a wide-character length. The example separates the bytes used for output from the characters counted for layout. section 14.3

Before printing the example’s multibyte characters, setlocale(LC_ALL, "") selects the environment’s locale. This is a concrete reminder that successful interpretation can depend on the execution environment, not just on the characters visible in a C source file. section 14.3

How Modern C covers it

How deep it goes
Works through character constants, numeric escapes, locale setup, and multibyte-to-wide conversion; stops short of specifying particular external byte encodings.
Examples
Uses '\n', hexadecimal escapes, Unicode box-drawing macros, and a separator routine that counts multibyte text through wide-character conversion.
What it assumes
Assumes familiarity with C strings and basic text I/O; introduces differing encodings and extended character sets in context.
What you can do after
Apply it via C literals, <ctype.h>, setlocale, and multibyte conversion functions while avoiding character-to-byte assumptions.
How it connects
Connects encoding to text streams, character classes, locales, multibyte strings, wide characters, and Unicode escapes.

Also explained in

Compare how 9 books explain encoding →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Modern C. The section links point to the parts of the book it draws on.