Source-based dataset building creates fine-tuning examples by extracting paired content from existing materials rather than writing every prompt and completion by hand. It reduces manual effort while allowing datasets to be tailored to an audience, scenario, and output type; specificity is presented as the route to better results. section 7.4.1
Choose source fields that naturally form the behavior to learn. For title generation, an article subheading can become the prompt and its title the completion. Collect many such pairs, place them in a CSV, then convert them into the JSONL message structure required for fine-tuning. section 7.4.1
Building title examples from an RSS feed
Parse a feed for article titles and links, request each article, extract its subheading, and save subheadings as prompts with titles as completions. This turns a set of published articles into a reusable training dataset. section 7.4.1
Listing 7.8 How to extract data from feeds
import feedparser
import pandas as pd
import requests
from bs4 import BeautifulSoup
url = 'https://alod83.medium.com/feed'
feed = feedparser.parse(url)
titles = []
links = []
subheadings = []
for entry in feed.entries:
titles.append(entry.title)
links.append(entry.link)
print(entry.link)
response = requests.get(entry.link)
soup = BeautifulSoup(response.content, 'html.parser')
subheading = soup.find('h2', attrs={'class': 'pw-subtitle-paragraph'}).text
subheadings.append(subheading)
df = pd.DataFrame({'prompt': subheadings,'completion': titles})
df.to_csv('medium-articles.csv', index=False)
The extraction listing makes the pipeline concrete: parse the feed, fetch each linked page, select subheadings, and write prompt–completion pairs to CSV. section 7.4.1
