A common mistake is to add more goroutines whenever a computation is slow. CPU-bound work is limited chiefly by processor computation, so a larger worker pool can increase context switching without increasing useful parallelism. I/O-bound work is different: its worker count depends more on the external system. section 8.5
func read(r io.Reader) (int, error) {
count := 0
for {
b := make([]byte, 1024)
_, err := r.Read(b) #1
if err != nil {
if err == io.EOF { #2
break
}
return 0, err
}
count += task(b) #3
}
return count, nil
}
The read loop processes each chunk with task(b). If that task spends most of its time computing, it is the CPU-bound part worth considering for parallel work; reading remains an I/O operation. section 8.5
func read(r io.Reader) (int, error) {
var count int64
wg := sync.WaitGroup{}
var n = 10
ch := make(chan []byte, n) #1
wg.Add(n) #2
for i := 0; i < n; i++ { #3
go func() {
defer wg.Done() #4
for b := range ch { #5
v := task(b)
atomic.AddInt64(&count, int64(v))
}
}()
}
for {
b := make([]byte, 1024)
// Read from r to b
ch <- b #6
}
close(ch)
wg.Wait() #7
return int(count), nil
}
A fixed pool receives chunks from a channel and computes task(b) in its workers. The listing sketches the pool; its reading loop is not a complete implementation. Choosing the worker count is the important decision. section 8.5
Choose a starting worker count
For CPU-bound workers, start near n := runtime.GOMAXPROCS(0), rather than automatically using the number of logical CPU cores. GOMAXPROCS can be set lower than the core count. This creates favorable conditions for workers to run separately, but it cannot dictate when or where the OS schedules threads. Benchmark the result. section 8.5
Worker count is not the only CPU-bound optimization. Data layout and access patterns also matter: spatial locality makes better use of cache lines, while predictable access can help the CPU. Use profiling and the execution tracer to investigate performance rather than assuming that more concurrency is the fix. section 12.11
