An embedding is a vector of floating-point numbers that represents an object for a particular purpose. An image, document, audio clip, or table row can each have an embedding. Its values may capture useful properties without having individually intelligible meanings. An embedding of a mug photograph, for example, might reflect color, lighting, orientation, and background without assigning a recognizable label to any one number. section 16.5.1
One way to create an embedding is with an autoencoder. Its encoder reduces an input to values in a smaller middle layer, also called the bottleneck or latent representation; its decoder tries to reconstruct the input from those values. Another approach trains a neural network on a proxy task chosen to make it learn useful features. A network trained to predict an image’s rotation, for instance, can use its second-last layer as an image embedding. The learned representation reflects the training task: being useful for prediction does not automatically make it useful for detecting outliers. section 16.2.1 section 16.5.1
Figure 16.2 Example of an AE. The neural network learns to condense any input to as many elements as appear in the middle layer and then reconstruct the data to its original values.
The narrow middle layer shows where an autoencoder turns each input into a compact vector before reconstruction. section 16.2.1
For outlier detection, embeddings can serve as features for methods such as Isolation Forest or k-nearest neighbors, or feed a combined model such as an autoencoder Gaussian mixture model. But applying a standard detector directly to prediction-oriented embeddings often works poorly. Hundreds or thousands of dimensions bring the curse of dimensionality, while features irrelevant to outlier detection add noise. A detector may treat those features equally rather than learning which matter for its purpose. section 16.3.1 section 16.5.2 section 16.4
The trade-off depends on the data. Compressing a million-pixel image to a shorter vector can retain high-level content while discarding pixel detail. Turning a 50-column table row into a vector of 50 or more values offers no comparable compression and can lose information. Embeddings also transfer more readily between image tasks: features learned for one image problem may help another, whereas tables often lack comparable shared structure. For tabular outlier detection, finding representations that retain the right differences remains difficult. section 16.5.2 section 16.5.3
Learning from parts of a table row
Interventional contrastive learning forms subsets of a row’s features and learns embeddings that are similar for subsets of the same row but different for other rows. This is one attempt to make a tabular representation useful for outlier detection, not a guarantee that every unusual record will be meaningful. DeepOD provides an ICL detector that can be fitted to tabular data and used to produce outlier scores. section 16.5.4 section 16.5.5
Figure 16.10 ICL. For each row in the dataset, we create subsets of features and an embedding for each subset. As the subsets for a given row represent the same object, they should have similar embeddings but they should be different from the embeddings for other rows. The neural network learns to create embeddings for each subset of features such that this is as true as possible.
The diagram shows the intended relationship between embeddings of feature subsets from the same row and from different rows. section 16.5.5
