Chapter 3. Buffers: Working with bits, bytes, and encodings
This chapter covers
- Introduction to the Buffer data type
- Changing data encodings
- Converting binary files to JSON
- Creating your own binary protocol
JavaScript has historically had subpar binary support. Typically, parsing binary data would involve various tricks with strings to extract the data you want. Not having a good mechanism to work with raw memory in JavaScript was one of the problems Node core developers had to tackle when the project started getting traction. This was mostly for performance reasons. All of the raw memory accumulated in the Buffer data type.
Buffers are raw allocations of the heap, exposed to JavaScript in an array-like manner. They’re exposed globally and therefore don’t need to be required, and can be thought of as just another JavaScript type (like String or Number):
If you haven’t worked much with binary data, don’t worry; this chapter is designed to be friendly to newcomers but also equip those who are more familiar with the concept. We’ll cover simple and more advanced techniques:
- Converting a Buffer to different encodings
- Using the Buffer API to transform a binary file to JSON
- Encoding and decoding your own binary protocol
Let’s look first at changing encodings for buffers.
If no encoding is given, file operations and many network operations will return data as a Buffer. Take this fs.readFile as an example:
