Training from scratch

Q: How deeply does Starting Data Analytics with Generative AI and Python cover training from scratch?

Explains benefits, costs, and expertise demands but stops short of implementation guidance and training procedures.

As explained in Starting Data Analytics with Generative AI and Python

Training from scratch is the process of learning a model's parameters from initialized parameters using task-relevant training data.

Training from scratch means learning a model’s parameters from initialized parameters with task-relevant data, rather than adapting an existing model. For sentiment analysis, this can tailor the model closely to a particular dataset and domain. section 6.2.4

Consider this approach when a standard or fine-tuned model cannot capture the task well, but weigh the practical costs:

Gathering enough labeled data requires substantial time and effort. section 6.2.4
Designing and training the model requires significant expertise, even with generative-AI assistance. section 6.2.4
Training demands substantial computational resources. section 6.2.4
Fine-tuning an existing model is another advanced option, while detailed from-scratch implementation lies beyond this treatment. section 6.2.4

How Starting Data Analytics with Generative AI and Python covers it

Cover of Starting Data Analytics with Generative AI and Python

liveBook manning.com

Read Starting Data Analytics with Generative AI and Python on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

LLMs in ProductionTraining from scratch is the process of creating a model by learning all of its parameters from randomly initialized weights rather than adapting a pretrained model.
liveBook manning.com

Sutskever's ListTraining from scratch is optimization of a model starting from newly initialized parameters rather than pretrained weights.

Compare how 3 books explain training from scratch →

Related concepts

fine-tuningcontrasts with · Fine-tuning is adapting a pretrained machine-learning model by training it further on targeted data or objectives for a task, domain, or behavior.
foundational modelalternative to · A base or foundation model is a pretrained model that provides learned parameters or capabilities as a starting point for reuse or adaptation to downstream tasks.
labeluses · A label is a value or annotation associated with an item that identifies its target category, outcome, or reference answer for learning or evaluation.
pretrainingcontrasts with · Pretraining is the initial training of a model on broad data to learn general patterns or capabilities before adaptation to specific tasks or domains.
open-source modelalternative to · An open-source model is a machine-learning model whose relevant artifacts, such as code or weights, are made available under terms permitting public access and specified forms of use or modification.

This explainer was generated by Manning's knowledge graph from the text of Starting Data Analytics with Generative AI and Python. The section links point to the parts of the book it draws on.