liveBookmanning.com

What you will learn in Graph Neural Networks in Action

by Keita Broadwater and Namid Stillman
Foreword by Matthias Fey

Read it on liveBook Get the book at manning.com

Chapter by chapter

Concepts explained in this book

• graphA graph data model specifies how entities, relationships, attributes, and constraints are represented as graph elements.
• graph neural networkA graph neural network architecture specifies the layers, computations, and connectivity used by a graph neural network.
• embeddingA node embedding is a numerical vector representation of a graph node that captures information about the node and its graph context.
• graph attention networkA graph attention network is a graph neural network that uses learned attention weights to aggregate information from neighboring nodes.
• graph convolutional networkA convolutional graph neural network is a class of graph neural network architectures that use convolution-like neighborhood aggregation.
• data aggregationAn aggregation function combines multiple neighborhood messages or feature values into a single representation.
• decoderA model component that maps a latent representation back into a reconstruction of the input or its target structure.
• GraphSAGEGraphSAGE is a graph neural network method that learns node representations by aggregating sampled neighborhood information.
• variational graph autoencoderA variational graph autoencoder is a probabilistic graph autoencoder whose encoder defines a distribution over latent representations and whose decoder reconstructs graph-related inputs or structure.
• graph autoencoderA graph autoencoder is a graph neural network encoder-decoder model that learns latent graph representations and reconstructs graph-related inputs or structure.
• Kullback–Leibler divergenceA measure of the difference between two probability distributions, quantifying the information cost of approximating one with another.
• encoderA model or method that maps an input data object to a lower-dimensional learned representation.
• message passingMessage passing is a graph neural network technique in which node representations are updated by aggregating information from neighboring nodes.
• signalInformation associated with an individual graph node, including attributes that can contribute to its learned representation.
• PyTorch GeometricPyTorch Geometric is an open-source Python library for developing and training graph neural networks with PyTorch.
• graph samplingNode sampling is a graph-sampling technique that selects a subset of graph nodes for computation.
• recurrent neural networkA recurrent neural network is a neural network architecture that processes sequences using recurrent state updates.
• batchingMini-batch training is a model-training technique that processes subsets of examples or graph nodes in separate batches rather than the entire dataset at once.
• latent spaceThe space of lower-dimensional representations produced by an encoder, with each point representing encoded information about an input.
• cross entropy lossCrossEntropyLoss is a PyTorch loss module that measures classification error from logits and target class labels.
• reconstruction lossA measure of the discrepancy between an input and the output reconstructed from its latent representation.
• schemaA schema is a formal specification of the classes, attributes, relationships, constraints, and other structure permitted in a data model.
• link predictionLink prediction, or edge prediction, is a graph-learning task that estimates whether an edge exists or should exist between graph entities.
• Node2VecNode2Vec is a graph embedding algorithm that learns node vectors using biased random walks and a word-context objective.
• adjacency matrixAn adjacency matrix is a matrix representation of a graph in which entries encode whether or how strongly pairs of nodes are connected.
• neural relational inferenceNeural relational inference is a neural model that infers latent relational structure and dynamics from observed system data.
• ogbn-productsogbn-products is an Open Graph Benchmark dataset of Amazon products connected by co-purchase edges, with product features and category labels.
• graph coarseningA graph-reduction technique that clusters or aggregates nodes and edges to create a smaller graph while attempting to preserve important structural or task-relevant information.
• pose estimationThe task of predicting a body's configuration or future positions of its parts.
• GRUA recurrent neural-network architecture that uses reset and update gates to control information flow through its recurrent state.
• remote backendThe PyG remote backend is an interface for accessing graph structure and features from external storage systems during graph learning.
• attentionAn attention mechanism assigns learned, data-dependent importance weights to inputs when computing a representation.
• generative modelingA deep generative model is a generative model that uses neural networks to learn and generate data distributions.
• memory usageThe amount or proportion of available memory consumed by a computation.
• relationship typeAn edge type is a category identifying the kind of relationship represented by a graph edge.
• autoencoderAn autoencoder is a neural network that encodes inputs into latent representations and decodes those representations to reconstruct the inputs.
• inductive inferenceInductive learning learns a generalizable function or representation from observed data for application to unseen data.
• graph schemaA graph schema is a specification of the permitted node types, edge types, properties, and structural rules of a graph.
• constraintA constraint is a formal rule that restricts the permitted structure, values, or relationships in data or a model.
• edgeAn edge is a graph element representing a relationship or connection between nodes and optionally carrying attributes or properties.
• nodeA node is a discrete element in a graph structure that can represent an entity such as an atom.
• bipartite graphA bipartite graph is a graph whose nodes are partitioned into subsets and whose edges connect nodes in different subsets rather than within the same subset.
• hidden layerAn intermediate layer of a neural network whose activations are not directly exposed as the model's inputs or outputs.
• transductive learningA transductive embedding method directly learns representations for nodes in a fixed graph and does not naturally generalize to unseen nodes or graphs.
• parallelismParallel computing is a technique that divides work among multiple processing units so computations can execute concurrently.
• local neighborhoodA node neighborhood is the set of nodes connected to a particular graph node, typically within a specified number of hops.
• random walkA random walk is a stochastic traversal of a graph that repeatedly moves from a node to one of its connected neighbors according to transition probabilities.
• spatial methodA graph-processing technique that combines data across neighborhoods defined by the graph's connectivity or spatial organization.
• spectral methodA graph-processing technique that analyzes graph signals in a spectral domain using tools from signal processing.
• DistributedDataParallelDistributedDataParallel is a PyTorch API that runs replicated model processes across multiple GPUs or machines while synchronizing gradients.
• maskA mask is a Boolean array or tensor that selects a subset of data elements.
• GATv2 layerA PyTorch Geometric graph-neural-network layer implementing the GATv2 attention operation with dynamic attention across layers.
• database tableInformation describing how entities are connected or related to one another.
• permutation invariancePermutation invariance is the property that a computation produces the same result regardless of the ordering of equivalent graph elements.
• instance modelAn instance model is a concrete representation of the entities, relationships, and properties instantiated from a data model.
• multi-head attentionAn attention technique that computes multiple attention operations in parallel and combines their outputs.
• max poolingmax_pool is a PyTorch Geometric pooling function that coarsens a graph by pooling clustered nodes using maximum-valued features.
• class imbalanceClass imbalance is a property of a classification dataset in which the classes contain substantially different numbers of examples.
• semi-supervised learningSemi-supervised learning is a learning setting that combines labeled and unlabeled examples during model development.
• time per epochAverage time per epoch is the mean elapsed time required to complete one full pass of training data during model training.
• time to convergenceTime to convergence is the elapsed training time required for an optimization process to reach a specified convergence condition.
• sparse representationSparseTensor is the torch_sparse sparse multidimensional tensor structure that stores values and indices rather than a full dense array.
• temporal graph neural networkA temporal graph attention model that represents time-dependent interactions by incorporating timestamps into graph edges.
• vanishing gradient problemAn optimization problem in which gradients become progressively smaller as they are propagated through many computational steps, hindering learning of long-range dependencies.
• dimensionality reductionDimensionality reduction is the transformation of data into a representation with fewer numerical dimensions while retaining selected information or structure.
• GATConvGATConv is a PyTorch Geometric graph-attention layer that computes attention scores using self-attention and can use multiple attention heads.
• over-squashingOver-squashing is the compression of information from many distant graph nodes into limited-dimensional node representations, reducing useful signal.
• graph machine learningGraph-based learning applies machine learning to data represented as a graph and learns from its structural relationships.
• convolutional neural networkA convolutional neural network is a neural network that applies learned convolutional filters to exploit local spatial structure, especially in grid-like data such as images.
• betweenness centralityA graph metric that quantifies how often a node lies on shortest paths between other nodes.
• Data classThe PyTorch Geometric Data class represents a graph and its associated attributes.
• PyG data moduleThe PyTorch Geometric data module provides classes and operations for creating and manipulating graph data objects.
• PyTorch Geometric DatasetA PyTorch Geometric base class for defining datasets and their raw-data processing, filtering, transformation, and retrieval behavior.
• PyG InMemoryDatasetA PyTorch Geometric dataset class that stores its processed data in memory.
• discriminative modelA discriminative model learns to distinguish among data classes or predict target values without modeling the full data-generating distribution.
• benchmarkBenchmarking is the systematic measurement of a system or model against representative data to establish performance references.
• neighborhood aggregationNeighborhood aggregation is a graph neural network technique that gathers and combines feature information from a node’s immediate neighbors.
• node degreeThe degree of a node is the number of graph edges incident to it, subject to conventions such as self-loop counting.
• space complexitySpace complexity describes how the memory requirements of an algorithm grow as the size of its input increases.
• GCNConvA PyTorch Geometric neural-network layer that performs graph convolution and neighborhood message aggregation.
• preprocessingData preprocessing is the transformation and preparation of data into a form suitable for model training or inference.
• SAGEConvA PyTorch Geometric neural-network layer that computes GraphSAGE neighborhood aggregation using a configurable aggregation function.
• permutation equivarianceThe property that permuting equivalent input elements produces a correspondingly permuted output rather than changing the represented result arbitrarily.
• property prediction lossA loss function that quantifies the discrepancy between predicted property values and their target values.
• NeighborLoaderNeighborLoader is a PyTorch Geometric data loader that samples node neighborhoods and produces mini-batches for graph learning.
• trajectoryA time-ordered sequence of positions or states describing the evolution of an entity.
• convolutionConvolution is an operation that combines nearby values through a local filter or weighted aggregation to produce a transformed representation.
• NetworkXNetworkX is a Python library for creating, manipulating, analyzing, and visualizing network and graph structures.
• graph signalA graph signal is data consisting of values or features defined on the nodes of a graph, typically represented as a vector over those nodes.
• connected componentA connected component is a maximal set of graph nodes in which every pair of nodes is connected by a path.
• conceptual schemaA conceptual schema specifies graph elements, rules, and constraints without being tied to a particular database or processing system.
• graph datasetA dataset representing nodes, edges, their relationships, and associated node or edge features.
• entity-relationship diagramAn entity-relationship diagram is a visual model showing entities, relationships, attributes, and constraints in a data or graph design.
• adjacency listAn adjacency list is a graph representation that records the neighboring nodes associated with each node.
• attention scoreAn unnormalized learned score quantifying the importance of one connected node's features to another node.
• subgraphA subgraph is a graph formed from a subset of the nodes and edges of another graph.
• distributed programmingDistributed processing is the execution of computation across multiple cooperating devices or processes.
• GPUA graphics processing unit (GPU) is a massively parallel processor designed to accelerate graphics, numerical, and machine-learning computations.
• TPUA tensor processing unit is a specialized hardware accelerator designed for machine-learning computations involving tensor operations.
• max aggregatorMax aggregation combines neighbor features by selecting the maximum value in each feature dimension.
• samplingSampling is the selection of a subset of data elements for computation or analysis.
• undirected graphAn undirected graph is a graph whose edges have no direction, so a connection from one node to another also connects the reverse pair.
• serializationSerialization is the conversion of in-memory or structured data into a representation suitable for storage or transmission and later reconstruction.
• problem radiusProblem radius is the graph distance over which nodes can have a significant effect on one another for a task.
• feature matrixA matrix whose entries encode feature values for a collection of entities or graph elements.
• gumbel-softmaxGumbel-Softmax is a differentiable approximation that samples from categorical distributions while enabling gradient-based optimization.
• technical debtTechnical debt is the future cost of design or implementation decisions that require later rework, workarounds, or replacement.
• graclusA graph clustering algorithm that partitions graph vertices to support multilevel graph coarsening and related optimization tasks.
• network hopA graph hop is one traversal from a node to an adjacent node along an edge.
• edge listAn edge list is a graph representation consisting of records that enumerate the graph's edges and their incident nodes.
• graph convolutionGraph convolution combines a node's representation with information aggregated from its graph neighborhood.
• dynamic graph neural networkA graph neural network model that represents and predicts changes in graph structure or node and edge information over time.
• area under the curveThe area under a receiver operating characteristic curve, measuring a classifier's ability to rank positive examples above negative examples across thresholds.
• convolution kernelA learnable weighted operation that transforms input data to emphasize particular patterns or features.
• temporal adjacency matrixA temporal adjacency matrix is a matrix representation that combines graph connectivity across multiple time steps into a single larger graph structure.
• floating-point operation countA floating-point operation is a numerical operation on floating-point values, with FLOPs denoting the count required by a computation.
• memory-to-data ratioThe memory-to-data ratio is the quotient of available memory capacity divided by the size of the dataset being processed.
• cycleA cycle is a graph path that begins and ends at the same node without repeating intermediate nodes.
• multilayer perceptronA multilayer perceptron is a feed-forward neural network composed of one or more fully connected hidden layers between its input and output layers.
• sequence predictionA prediction task in which a model uses ordered past observations to predict subsequent observations or events.
• graph densityGraph density is the ratio of a graph's actual edges to the total number of possible edges.
• mean squared errorA loss function equal to the average squared difference between predicted and target values.
• UMAPUMAP is a nonlinear dimensionality-reduction technique that projects high-dimensional data into a lower-dimensional space while attempting to preserve local and global structure.
• distributed trainingDistributed training is a technique in which model computation and data processing are divided across multiple computing processes or devices.
• ETLExtract, transform, load is a data-integration process that retrieves raw data, transforms it into a target structure, and loads the result for downstream use.
• nonlinear mappingA parameterized operation that maps inputs to outputs without preserving linearity, allowing a neural network to model nonlinear relationships.
• dense representationA dense representation explicitly stores entries for all possible relationships or attribute positions, including zeros or absent values.
• PyTorch Geometric DataA PyTorch Geometric Data object is a container for graph connectivity and associated node, edge, or graph-level attributes.
• update ruleThe update step is the message-passing stage in which each node’s features are replaced or revised with its newly computed representation.
• exponential linear unitA nonlinear activation function that outputs a smooth negative-valued response for negative inputs and a linear response for positive inputs.
• one-hot encodingA representation technique that encodes a categorical value as a vector with one active entry and all other entries inactive.
• pose dataData describing the configuration or positions of body parts over time.
• jumping knowledge networkA jumping knowledge network is a graph neural network architecture that combines representations produced at multiple layers to capture information from different neighborhood ranges.
• objective functionA loss function is a function that quantifies the discrepancy between a model's output and the desired target.
• poolingA graph operation that reduces a collection of representations to a smaller representation, often for graph-level prediction.
• temporal edgeAn edge connecting corresponding or related nodes across different time-step layers of a temporal graph.
• poseThe spatial configuration or arrangement of the body parts of an entity.
• inner product decoderAn inner product decoder reconstructs pairwise graph connectivity by scoring pairs of latent node representations with their inner product and converting the scores into edge probabilities.
• residual connectionA skip connection is a neural-network pathway that passes information around one or more intermediate layers to preserve or combine representations.
• end-to-end modelA model that jointly learns intermediate representations and task predictions during a single training process.
• log-softmax functionA function that applies logarithms to softmax-normalized scores to produce log probabilities over classes.
• node classificationNode classification is a graph-learning task that assigns labels or categories to individual nodes.
• FeatureStoreFeatureStore is a PyTorch Geometric interface or object for storing and retrieving features associated with graph nodes or other graph entities.
• GraphStoreGraphStore is a PyTorch Geometric interface or object for storing graph connectivity and retrieving edge indices for graph operations such as sampling.
• node predictionA graph machine-learning task that predicts labels, attributes, or values associated with individual nodes.
• hidden stateA hidden state is the recurrent internal representation that carries sequence information between processing steps.
• intra-timestep edgeAn edge connecting nodes within the same time-step layer of a temporal graph.
• node maskingA technique that hides selected node values so a model must infer them from the remaining graph information.
• Xavier/Glorot initializationA neural-network parameter initialization method that scales initial weights according to the sizes of adjacent layers to help maintain activation and gradient variance.
• cosine similarityCosine similarity is a measure of similarity between vectors based on the cosine of the angle between them.
• similarity matrixA similarity matrix is a matrix whose entries quantify the similarity between pairs of items or representations.
• CSVCSV is a delimited text format that represents tabular data as rows of fields separated by commas or another specified delimiter.
• JSONJSON is a language-independent text format for representing structured data using objects, arrays, values, and name-value pairs.
• jumping knowledgeA graph neural network technique that combines representations from multiple neighborhood depths to adapt the effective receptive field for each node.
• JumpingKnowledgeA PyTorch Geometric layer that combines outputs from multiple neural-network layers using modes such as concatenation, max pooling, or learned attention.
• multi-aggregationMulti-aggregation is a graph neural network technique that combines multiple aggregation functions within one layer or across layers.
• teacher forcingTeacher forcing is a sequential-model training technique that supplies the ground-truth previous output as input when predicting the next step.
• unweighted graphAn unweighted graph is a graph in which all edges have equal importance or weight.
• data modelA data model is a formal structure that specifies the entities, attributes, relationships, and constraints of a dataset or information system.
• activation functionAn activation function is a nonlinear or otherwise specified function applied to a layer's transformed values.
• over-smoothingA graph neural network failure mode in which repeated neighborhood aggregation makes node representations increasingly similar and less distinguishable.
• trainingModel training is the optimization process that fits a machine-learning model to data by adjusting its parameters.
• overfittingA modeling failure mode in which a model fits training data patterns too closely and generalizes poorly to validation or unseen data.
• learning curveA plot showing a model's performance measure, such as loss, across training epochs.
• NeighborSamplerNeighborSampler is a PyTorch Geometric data-loading function or component that samples node neighborhoods and returns subgraph batches.
• acyclic graphAn acyclic graph is a graph containing no cycle that returns to a starting node without backtracking.
• directed acyclic graphA directed acyclic graph is a directed graph containing no cycles.
• heterogeneous graphA heterogeneous graph is a graph containing multiple types of nodes or edges.
• homogeneous graphA homogeneous graph is a graph containing one type of node and one type of edge.
• hypergraphA hypergraph is a graph in which a single hyperedge can connect any number of nodes rather than exactly two.
• knowledge graphA knowledge graph is a heterogeneous graph that represents entities and relationships with explicit semantic context, metadata, and schema information.
• sparse graphGraph sparsity is the property of a graph having relatively few edges compared with the number of possible edges.
• Fourier transformA Fourier transform expresses a signal as a combination of components associated with different frequencies.
• data explorationThe process of examining and summarizing data to discover patterns, structure, relationships, and potential anomalies.
• training loopA training loop is an iterative procedure that processes data through a model and updates its parameters across training iterations.
• quantitative estimate of druglikenessQED, or quantitative estimate of drug-likeness, is a score estimating how closely a molecule's properties resemble those of known drugs.
• SMILESSMILES is a textual notation for encoding the structure of chemical compounds as molecular strings.
• principal component analysisPrincipal component analysis is a statistical dimensionality-reduction technique that transforms correlated variables into a smaller set of ordered orthogonal components.
• dataset splitA partition of a dataset into training and testing subsets for model fitting and evaluation.
• property prediction layerA neural-network layer that maps a learned representation to a predicted property value.
• heterogeneous graph convolutional networkA graph convolutional network that applies relation-specific transformations or parameters to graphs containing multiple edge types.
• molecular graphA molecular graph is a graph representation in which atoms are nodes and chemical bonds are edges.
• COO formatCOO format represents sparse data as collections of coordinates and corresponding values for nonzero entries.
• ToSparseTensorToSparseTensor is a PyTorch Geometric transformation that converts graph adjacency data into a sparse tensor representation.
• negative samplingA training technique that selects a subset of nonexistent relationships as negative examples instead of evaluating all possible negative examples.
• class reshufflingA class-balancing technique that redistributes or resamples training examples to improve class representation without generating synthetic data.
• variational autoencoderA variational autoencoder is a probabilistic autoencoder that learns a parameterized latent-variable distribution and decodes samples from it to reconstruct or generate data.
• pipelineA data pipeline is a sequence of processes that moves data from an input state through transformation and preparation stages to an output suitable for storage or computation.
• torch.distributedtorch.distributed is the PyTorch package for initializing processes and coordinating communication in distributed computation.
• self-loopA graph edge that connects a node to itself and, in this recurrent-model diagram, represents a temporal state update.
• coarsened graphA simplified graph produced by aggregating nodes and edges from an original graph.
• reset gateA gated recurrent unit component that controls how much prior hidden-state information is discarded when computing a new state.
• gradient synchronizationGradient synchronization is the coordination of gradients computed by multiple training processes so that replicated models receive consistent parameter updates.
• depthThe number of computational layers in a neural network, especially the number of graph-processing layers in a graph neural network.
• low-dimensional spaceA low-dimensional vector representation encodes an object as a vector with relatively few numerical dimensions.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.