Oversubscription occurs when more threads are ready to run than the hardware can execute concurrently. The operating system must switch between them, adding task-switching overhead and potentially worsening cache problems. section 8.2.1 section 8.2.5
A high thread count is not automatically a problem: threads waiting on I/O, mutexes, or condition variables leave processors available for other work. Excess runnable threads are more concerning when CPU-bound work is divided among tasks or spawned without limits. section 8.2.5
Ways to limit or address oversubscription include:
- Limit worker threads when dividing data into tasks. section 8.2.5
- Use std::thread::hardware_concurrency() as a scaling guide, but remember that separate callers may not account for one another or for other applications. section 8.2.1
- When the natural division of CPU-bound work exceeds available processors, consider another division only when performance measurements justify the change. section 8.2.5
Listing 2.9. A naïve parallel version of std::accumulate
template<typename Iterator,typename T>
struct accumulate_block
{
void operator()(Iterator first,Iterator last,T& result)
{
result=std::accumulate(first,last,result);
}
};
template<typename Iterator,typename T>
T parallel_accumulate(Iterator first,Iterator last,T init)
{
unsigned long const length=std::distance(first,last);
if(!length)
return init;
unsigned long const min_per_thread=25;
unsigned long const max_threads=
(length+min_per_thread-1)/min_per_thread;
unsigned long const hardware_threads=
std::thread::hardware_concurrency();
unsigned long const num_threads=
std::min(hardware_threads!=0?hardware_threads:2,max_threads);
unsigned long const block_size=length/num_threads;
std::vector<T> results(num_threads);
std::vector<std::thread> threads(num_threads-1);
Iterator block_start=first;
for(unsigned long i=0;i<(num_threads-1);++i)
{
Iterator block_end=block_start;
std::advance(block_end,block_size);
threads[i]=std::thread(
accumulate_block<Iterator,T>(),
block_start,block_end,std::ref(results[i]));
block_start=block_end;
}
accumulate_block<Iterator,T>()(
block_start,last,results[num_threads-1]);
for(auto& entry: threads)
entry.join();
return std::accumulate(results.begin(),results.end(),init);
}
The parallel accumulation example bounds its thread count by both the input size and hardware concurrency, with a fallback when the hardware count is zero. section 2.4
