Serialization puts graph or other structured data into a format that can be stored, exported, transferred between systems, or sent over the internet. In a data pipeline, it connects raw data with graph-processing and GNN libraries. section 8.3 section 8.3.1
Choose a serialization format by checking four pipeline requirements:
- Data model: decide whether the data uses a simple model, property graph, or another representation. section 8.3.1
- Schema: identify which entities represent nodes, edges, and properties. section 8.3.1
- Data structure: select adjacency matrices, adjacency lists, edge lists, or another storage structure. section 8.3.1
- Receiving systems: verify which encodings and structures libraries or processing systems accept, and whether custom reading code is required. section 8.3.1
Available choices include language- and system-agnostic formats such as JSON, CSV, and XML; language-specific formats such as Python Pickle; and software-specific formats associated with SNAP, NetworkX, and Gephi. Avro and sparse or matrix-market formats target large datasets. section 8.5
Recommendation logs as JSON
A pipeline can begin with queried recommendation logs transformed into JSON: each key identifies a recommending candidate and its value lists recommended candidates. Profile data adds industry and job type, while hashed identifiers protect personally identifiable information. The JSON can then be transformed into an edge list or adjacency matrix for graph processing. section 8.3.1 section 8.5
Figure 8.7 Summary of steps in the data pipeline process in this section
Use the pipeline summary to follow serialization from raw data through transformation and preparation for graph learning. section 8.3
