liveBook · Manning

Sequential machine learning

As explained in Optimization Algorithms: AI techniques for design, planning, and control problems

Sequential machine learning is machine learning on data whose observations have an order that affects interpretation or prediction.

Sequential machine learning handles data whose observation order affects interpretation or prediction, including time series, sentences, and permutations. Tasks range from one-to-one and one-to-many mappings to many-to-one and many-to-many sequence transformations. section 11.3.3

Table 11.2 Sequential ML (view table figure)

Task

Example

One-to-one

Image classification. We provide a single image as input, and the model outputs the classification or category, like “dog” or “cat,” as a single output.

One-to-many

Image captioning. We input a single image into the model, and it generates a sequence of words describing that image.

Many-to-one

Sentiment analysis. We input a sequence of words (like a sentence or a tweet), and the model outputs a single sentiment score (like “positive," “negative,” or “neutral”).

Many-to-many (type 1)

Sequence input and sequence output, like in the case of named entity recognition (NER). We input a sentence (a sequence of words), and the model outputs the recognized entity, such as a person, organization, location, etc.

Many-to-many (type 2), known as a synchronized sequence model

Synced sequence input and output. The model takes a sequence of inputs but doesn’t output anything until the entire sequence has been read. Then it outputs a sequence. An example of this is video classification, where the model takes a sequence of video frames as input and then outputs a sequence of labels for those frames.

Use the task table to distinguish sequence-learning arrangements such as image captioning, sentiment analysis, named entity recognition, and synchronized sequence prediction. section 11.3.3

Pointer networks for combinatorial sequences

Traditional sequence-to-sequence models struggle when output categories depend on variable input size. A pointer network uses attention to select members of the input sequence, supporting tasks such as sorting, travelling salesperson solutions, and convex hull construction. For a convex hull, the output polygon must enclose every point while minimizing its boundary. section 11.3.3

How Optimization Algorithms: AI techniques for design, planning, and control problems covers it

How deep it goes
Classifies input-output sequence structures and explains pointer-network attention, stopping short of implementation details or training procedure.
Examples
Uses time series, sentences, permutations, image captioning, sentiment analysis, named entity recognition, sorting, travelling salesperson, and convex hull examples.
What you can do after
Evaluate sequence-learning task structures and understand why pointer networks handle variable-sized output dictionaries.
How it connects
Connects sequence-to-sequence models with recurrent neural networks, long short-term memory, attention, and combinatorial optimization.
Cover of Optimization Algorithms: AI techniques for design, planning, and control problems
liveBookmanning.com
Read Optimization Algorithms: AI techniques for design, planning, and control problems on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Optimization Algorithms: AI techniques for design, planning, and control problems. The section links point to the parts of the book it draws on.