Hugging Face is a prominent model repository and high-level deep-learning wrapper. It gives practitioners a practical way to find pretrained transformer and diffusion models instead of building every model from scratch. section 1.4.1 section 2.3
Its role becomes clearer through three connected ideas:
- A model zoo collects pretrained models behind more consistent access patterns than historically available. section 2.3
- The transformers package provides access to transformer models, including GPT-2 for text generation. section 2.3
- Transformer workflows connect models with tokenizers, which convert raw text into numerical inputs. section 9.12
Generate text with GPT-2
A pipeline can download a pretrained GPT-2 model and complete a short prompt: `from transformers import pipeline\ngenerator = pipeline('text-generation', model='gpt2')\ngenerator('AI models are so smart they can replace my', max_length=10)`. The generated continuation uses learned language patterns, context, and a sampling strategy. section 2.3
from transformers import pipeline
generator = pipeline('text-generation', model='gpt2') #1
generator("AI models are so smart they can replace my",
max_length=10) #2
# [{'generated_text': 'AI models are so smart they can replace my brain with a totally different one.'}]
Inspect the complete pipeline listing and its generated output. section 2.3
