Encoding converts a Unicode string into a bytes object. Strings contain Unicode characters; bytes contain integer values, so the two types may look similar but cannot be used interchangeably. Use decode to convert bytes back into a string. section 6.9
Keep these distinctions in mind:
- Encoded text may occupy multiple bytes and no longer print like the original character. section 6.9
- A bytes object cannot be concatenated directly with a string; convert one type before combining them. section 6.9
- UTF-8 preserves basic ASCII characters while supporting a much wider range of alphabets and symbols. section 21.2.1 section 21.7
- Choosing ASCII for broader text can produce decoding errors when files contain accented or otherwise non-ASCII characters. section 21.2.1
Encode and decode an accented character
unicode_a_with_acute = '\N{LATIN SMALL LETTER A WITH ACUTE}' xb = unicode_a_with_acute.encode() # xb prints as b'\xc3\xa1' xb.decode() # 'á' xb += 'A' # TypeError: can't concat str to bytes
unicode_a_with_acute = '\N{LATIN SMALL LETTER A WITH ACUTE}'
unicode_a_with_acute
'á'
xb = unicode_a_with_acute.encode() #1
xb
b'\xc3\xa1' #2
xb += 'A' #3
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-93-2c4ecee004ef> in <cell line: 1>()
----> 1 xb += 'A'
TypeError: can't concat str to bytes
xb.decode() #4
'á'
#1 The encode method converts string to bytes.
#2 The encoded bytes object is 2 bytes; it doesn’t print as a character.
#3 bytes string objects are different types.
#4 The object’s decode method will convert it back to a string.
Inspect the complete runnable example showing the original Unicode character, its encoded bytes, the failed mixed-type concatenation, and the decoded result. section 6.9
