15 Batch schedulers: bringing order to chaos
This chapter covers
- the role of batch schedulers in high performance computing
- submitting a job to a batch scheduler
- linking job submissions for long runs or more complex workflows
Most high performance computing systems use batch schedulers to schedule the running of applications. We’ll give you a brief idea why in the first section of the chapter. Because they are ubiquitous on high-end systems, you should have at least a basic understanding of these schedulers to be able to run on them. We’ll cover the purpose and usage of the batch schedulers. We won’t go into how to set up and manage them (that’s a whole other beast). Set up and management is a topic for system administrators and we are users.
What if you don’t have access to a system with a batch scheduler? We don’t recommend installing a batch scheduler just to try out these examples. Rather, count your blessings and keep the information in this chapter handy for when the need appears. If your demand for computational resources grows and you begin using a larger multi-user cluster, you can come back to this chapter.
There are many different batch schedulers and each installation has its own unique customizations. We’ll discuss two batch schedulers that are freely available -- the Portable Batch System (PBS) and the Simple Linux Utility for Resource Management (Slurm). There are variants of each of these, including commercially supported versions.