UTF-8 is a variable-width Unicode encoding that represents each code point with one to four bytes. In Go, a rune represents a Unicode code point and is an alias of int32, while a string is only a sequence of arbitrary bytes—not necessarily UTF-8. String literals are UTF-8 encoded, but data read from a filesystem may use another encoding. section 5.1
Mistaking byte positions for rune positions
For s := "hêllo", len(s) returns 6 because ê occupies two UTF-8 bytes, although the string contains five runes. Iterating with for i := range s produces rune-starting byte indexes—0, 1, 3, 4, and 5—so s[i] accesses a byte and can print à instead of ê. Use for i, r := range s to obtain each rune, or convert with []rune(s) when rune indexes are required. section 5.2
Choose the operation according to whether the code needs bytes, rune values, or rune counts:
- Use len(s) for the number of bytes in the string. section 5.2
- Use utf8.RuneCountInString(s) for the number of runes when the string is UTF-8. section 5.2
- Use range values for direct UTF-8 rune iteration; use []rune(s) when integer positions should represent rune indexes. section 5.2
s := "hêllo"
for i, r := range s {
fmt.Printf("position %d: %c\n", i, r)
}
Inspect the corrected range loop that prints each rune with its starting byte position. section 5.2
