liveBook · Manning

Pickle format

As explained in Python How-To

A pickle file is a persistent serialized representation of Python objects produced by the pickle protocol.

Pickling preserves Python objects in files; unpickling restores them. Since pickle files are binary, open them in read mode with "rb" and pass the file object to pickle.load. Assign its return value to keep the restored object. section 11.3.2

with open("task_tuple_saved.pickle", "rb") as file:
   task_tuple_loaded = pickle.load(file)

with open("task_dict_saved.pickle", "rb") as file:
   task_dict_loaded = pickle.load(file)

Load saved tuple and dictionary objects from binary pickle files. section 11.3.2

Pickle can restore instances of custom classes, but the class must be defined when unpickling. The restored instance can have the same attributes as the original without being the same object. section 11.3.2

class Task:
    def __init__(self, title, urgency):
        self.title = title
        self.urgency = urgency
    
task = Task("Laundry", 3)
 
with open("task_class_saved.pickle", "wb") as file:
    pickle.dump(task, file)
 
with open("task_class_saved.pickle", "rb") as file:
    task_class_loaded = pickle.load(file)
 
assert task.__dict__ == task_class_loaded.__dict__
assert task is not task_class_loaded

Compare a custom instance's attributes after restoration, then check that it is a distinct object. section 11.3.2

Keep these capabilities and cautions in mind:

  • Pickling can preserve functions, but does not support every object: modules, file objects, and database connections are identified as limitations. section 11.3.3
  • Treat pickled data cautiously. A custom class can use __reduce__ to arrange behavior that runs when its instance is unpickled. section 11.3.3
import os
 
class MaliciousTask:
    def __init__(self, title, urgency):
        self.title = title
        self.urgency = urgency
 
    def __reduce__(self):
        print("__reduce__ is called")
        return os.system, ('touch hacking.txt',)    #1

See how a custom __reduce__ method can specify a system command as part of the pickle behavior. section 11.3.3

How Python How-To covers it

How deep it goes
Explains binary-file loading, custom-class lookup, and object identity, but does not specify protocol internals.
Examples
Uses tuple, dictionary, custom-instance, function, and malicious-class examples to demonstrate loading and pickle limitations.
What you can do after
Apply pickle.load with binary file mode to restore objects, and check equality and custom-class availability.
How it connects
Compares pickle with JSON serialization and connects unpickling to custom classes and __reduce__.

Also explained in

Compare how 2 books explain pickle format →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Python How-To. The section links point to the parts of the book it draws on.