Julia strings use UTF-8 to represent Unicode text. Unicode assigns a number, called a code point, to a character; UTF-8 encodes that code point using a variable number of bytes. Familiar ASCII letters use one byte, while characters such as Æ and 😏 use more. The one-byte encodings remain compatible with ASCII. section 11.1 section 11.1.1
Four characters, five bytes
In the Julia REPL, length("Æser") returns 4, but sizeof("Æser") returns 5. Æ occupies two bytes, so "Æser"[1] returns 'Æ', "Æser"[2] raises a StringIndexError, and "Æser"[3] returns 's'. A string index is not necessarily a character’s position in the word. section 11.1
Use codepoint('Æ') to inspect its Unicode number and ncodeunits('Æ') to see that its UTF-8 representation takes two code units. By comparison, ncodeunits('A') returns 1 and ncodeunits('😏') returns 4. These are different questions: what number identifies the character, and how much encoded space does it take? section 11.1.1
For ordinary character-by-character work, iterate directly with for ch in s. When you also need valid indices, use eachindex(s), or advance from firstindex(s) with nextind(s, i), rather than adding one to an index. section 11.1.1
Listing 11.1 Iterating over characters in a string using a while loop
julia> s = "Þrúðvangar"
julia> i = firstindex(s)
1
julia> while i <= lastindex(s)
println((i, s[i]))
i = nextind(s, i)
end
(1, 'Þ')
(3, 'r') #1
(4, 'ú')
(6, 'ð') #2
(8, 'v')
(9, 'a')
(10, 'n')
(11, 'g')
(12, 'a')
(13, 'r')
The while-loop listing makes the uneven index progression visible as it visits every character in a UTF-8 string. section 11.1.1
