The producer-consumer pattern separates data generation from data consumption so each component can operate independently. A background worker fetches RSS feeds and stores articles, while an API serves already-fetched data without waiting on network requests. section 11.3.3
A practical Go implementation emphasizes:
- Give the worker one responsibility: periodically fetch feeds and persist new articles. section 11.3.3
- Connect services through shared domain interfaces while keeping storage and adapters reusable. section 11.3.3
- Use an adapter to transform external RSS types into domain types, isolating the application from library changes. section 11.3.3
- Schedule repeated fetches, handle individual failures without stopping the feed list, and cancel through context during graceful shutdown. section 11.3.3
RSS news service
The worker reads configured New York Times and BBC feeds, calls a domain.Fetcher, and stores returned articles. The API handles /articles requests independently from those slow network operations. section 11.3.3
Listing 11.6 Feed list and core fetch-and-store logic
package main
import (
"context"
"fmt"
"log"
"os"
"os/signal"
"syscall"
"time"
"github.com/YOUR_USERNAME/go-news/domain"
"github.com/YOUR_USERNAME/go-news/storage"
"github.com/YOUR_USERNAME/go-news/worker/internal/reader"
)
var feeds = []string{ #1
"https://rss.nytimes.com/services/xml/rss/nyt/World.xml",
"https://feeds.bbci.co.uk/news/rss.xml",
}
func fetchAndStore(ctx context.Context, fetcher domain.Fetcher,
store domain.Storage, urls []string) { #2
for _, url := range urls { #3
feed, err := fetcher.FetchFeed(ctx, url) #4
if err != nil {
log.Printf("Error fetching %s: %v", url, err)
continue #5
}
if err := store.AddArticles(feed.Articles); err != nil { #6
log.Printf("Error storing articles from %s: %v", url, err)
continue
}
fmt.Printf("Fetched and stored %d articles from %s\n",
len(feed.Articles), feed.Title)
}
}
Inspect the feed list and fetchAndStore function to see the producer’s core workflow. section 11.3.3
