A graph represents entities as nodes or vertices and their relationships as edges. Social networks illustrate the idea: friendships are usually undirected, while follower relationships are directed. section 6.4.6
Graphs turn similarity into a structure that can be inspected visually. Sentence embeddings provide node representations, while sufficiently similar sentences become connected edges; force-directed layouts then gather related concepts into clusters. section 6.4.6
A practical graph-building workflow is:
- Extract sentences and noun phrases from text with spaCy, then use each sentence vector as its embedding. section 6.4.6
- Normalize the vectors and calculate their pairwise dot products to form a similarity or affinity matrix. section 6.4.6
- Keep strong similarities, add the corresponding nodes and weighted edges to a NetworkX undirected graph, and plot it with a spring layout. section 6.4.6
- Use conversation graphs when transitions need loops: unlike a directed acyclic graph, a dialog graph can revisit earlier states. section 12.3.2
Connecting concepts in a chapter
After computing normalized sentence vectors, retain similarities above 0.95, create NetworkX edges weighted by similarity, and draw the result with nx.spring_layout(G, k=0.15, seed=42). The resulting plot exposes clusters of closely related concepts. section 6.4.6
Listing 6.19 Creating the undirected graph
>>> import re >>> import networkx as nx >>> similarity_matrix = np.triu(similarity_matrix, k=1) #1 >>> iterator = np.nditer(similarity_matrix, ... flags=['multi_index'], order='C') >>> node_labels = dict() >>> G = nx.Graph() >>> pattern = re.compile( ... r'[\w\s]*[\'\"]?[\w\s]+\-?[\w\s]*[\'\"]?[\w\s]*' ... ) #2 >>> for edge in iterator: ... key = 0 ... value = '' ... if edge > 0.95: #3 ... key = iterator.multi_index[0] ... value = str(noun_phrases[iterator.multi_index[0]]) ... if (pattern.fullmatch(value) ... and (value.lower().rstrip() != 'figure')): ... node_labels[key] = value ... G.add_node(iterator.multi_index[0]) ... G.add_edge(iterator.multi_index[0], ... iterator.multi_index[1], weight=edge)
Inspect the executable graph-construction listing, including thresholding similarities, labeling nodes, and adding weighted edges. section 6.4.6
Listing 6.20 Plotting an undirected graph
>>> import matplotlib.pyplot as plt >>> plt.subplot(1, 1, 1) #1 >>> pos = nx.spring_layout(G, k=0.15, seed=42) #2 >>> nx.draw_networkx(G, ... pos=pos, #3 ... with_labels=True, ... labels=node_labels, ... font_weight='bold') >>> plt.show()
Inspect the plotting listing for rendering the graph and displaying labels with a deterministic spring layout. section 6.4.6
