1 Introducing Fortran

This is a book about Fortran, one of the first high-level programming languages in history. It will teach you the language by guiding you step-by-step through the development of a fully-featured, parallel physics simulation app. Notice the emphasis on parallel. I will introduce the concept of parallel programming early on, and start applying it from first principles. Parallel programming allows you to break down your problem into pieces, and let multiple processors each work on only part of the problem, thus reaching the solution in less time. By the end, you will be able to recognize problems that can be parallelized, and you will be able to use modern Fortran techniques to solve them.

Modern Fortran is not a comprehensive reference manual for every Fortran feature. There are significant parts of the language that I have omitted on purpose. Instead, I focus on the most practical Fortran features that you would use to build a real-world application. As we work on our app chapter by chapter, we will apply modern Fortran features and software design techniques to make our app robust, portable, and easy to use and extend. That said, I stand corrected. This is not just a book about Fortran. This is a book about building robust, parallel software using modern Fortran.

1.1  What is Fortran?

Fortran is a general-purpose, parallel programming language that excels in scientific and engineering applications. Originally called FORTRAN (FORmula TRANslation) in 1957, it has evolved over decades to a robust, mature, and high perfomance-oriented programming language. Today, Fortran keeps churning under the hood of many systems that we take for granted:

Here’s a specific example. In my work, I deal mostly with the development of numerical models for weather, ocean surface waves, and deep ocean circulation. Speaking about it over the years, I found that most people didn’t really know where weather forecasts come from. The general idea is that a group of meteorologists would gather and together come up with a chart of what the weather will be like tomorrow, in a week, or a month from now. This is only partially true. In reality, we use sophisticated numerical models that crunch a huge amount of numbers on very large computers. In layman terms, these models simulate the atmosphere to create an educated guess of what the weather will be like some time in the future. The results of these simulations are then used by meteorologists to create a meaningful weather map (Figure 1.1). This map shows just a sliver of all data that is produced by the model. The output size of a weather forecast like this counts in hundreds of gigabytes.

The most powerful Fortran applications run in parallel on hundreds or thousands of CPUs. Development of the Fortran language and its libraries have been largely driven by the need to solve extremely large computational problems in physics, engineering, and biomedicine. To access even more computational power than what the most powerful single computer at the time could offer, in the late 20th century we started connecting many computers with high-bandwidth networks, and let them each work on a piece of the problem. The result is the so-called supercomputer, a massive computer that is typically made of thousands of commodity CPUs (Figure 1.2). Supercomputers are similar to modern server farms hosted by Google or Amazon, except that the network infrastructure in supercomputers is designed to maximize bandwidth and minimize latency between the servers themselves, rather than the outside world. As a result, the CPUs in a supercomputer act like one giant processor with distributed-memory access that is almost as fast as local memory access. To this day, Fortran remains the dominant language used for such massive-scale parallel computations.

1.2  Fortran features

In the context of programming languages, Fortran is:

1.3  Why learn Fortran?

Since the early 1990s, we have seen an explosion of new programming languages and frameworks, mainly driven by the widespread use of the internet, and later, mobile devices. C++ took over computer science departments, Java has been revered in the enterprise, Javascript redefined the modern web, R became the mother tongue of statisticians, and Python rose up as an all-around great programming language for most tasks. Where does Fortran fit in all this? Through steady revisions of the language, Fortran has maintained a solid footing in its niche domain, High Performance Computing (HPC). Its computational efficiency is still unparalleled, with only C and C++ coming close. However, unlike C and C++, Fortran has been designed for array-oriented calculations, and is in my opinion significantly easier to learn and program. Finally, a strong argument for Fortran is in its native support for parallel programming that was introduced in the 2008 revision of the standard.

Despite being a decades-old technology, Fortran has several attractive features that make it indispensable, even compared to more recent languages:

In summary, learn Fortran if you need to implement efficient and parallel numerical operations on large multi-dimensional arrays.

1.4  Advantages and disadvantages

Many Fortran features give it both an advantage and a disadvantage. I list some below:

The comparison of Fortran to Python that follows will help you better understand its advantages and disadvantages in the general-purpose programming context.

1.4.1  Side-by-side comparison with Python

How does modern Fortran compare to a more recent general-purpose programming language? Python has had the most rapidly growing ecosystem in the past few years for data analysis and light number crunching (stackoverflow.blog/2017/09/14/python-growing-quickly). It is used by many Fortran programmers that I know for post-processing of model output and data analysis. In fact, Python is my second favorite programming language (guess which one is my number one). Because of the application domain overlap between Fortran and Python, it is useful to summarize the main differences between these languages. If you are a Python programmer, this summary will give you an idea of what you can and cannot do with Fortran.

Going through Table 1.1, we notice the key differences between Fortran and Python:

In summary, it is difficult to use Fortran to write device drivers, graphical video games, or a web browser. However, if you need to solve a large numerical problem that can be distributed across multiple computers, Fortran is the ideal implementation language.

1.5  The parallel Fortran mental model

Let me take a few minutes to illustrate the kind of problem where Fortran really shines.

Now you must be thinking, what the heck does old Ralph’s farm have to do with parallel Fortran programming? More than meets the eye, I can tell you! Old Ralph and his big pasture are an analogy to a slow computer and a big compute problem. Just like Ralph asked his sons and daughters to help him cut the grass, in a typical parallel problem we will divide the computational domain, or input data, into equal pieces and distribute them between CPUs. Recall that his children cut the grass row-by-row — some of the most efficient and expressive Fortran code are the whole-array operations and arithmetic. Periodically, they met at the edges to sharpen the tools and have a chat. In many real-world apps, you will instruct the parallel processes to exchange data between each other, and this is true for all the parallel examples that I will guide you through in this book. Finally, each parallel process will asynchronously write its data to disk. I illustrate this pattern on Figure 1.3.

Much like farmer Ralph, Fortran is old. This is by no means a bad thing! It is a mature, robust, and dependable language that isn’t going anywhere. While it does carry some quirks of an old programming language, it has been improved decade over decade by generations of computer scientists and programmers, and has been battle-tested in countless applications where performance is critical. The ease of parallel programming with Fortran is key for high-performance apps, which is why I chose to make it the focus of this book.

1.6  What will you learn in this book?

This book will teach you how to write modern, efficient, and parallel Fortran programs. Working through each chapter, we will build from scratch a fully-functional, parallel, fluid dynamics solver with a specific application to tsunami prediction. If you work through the book, you will come out with three distinct skill sets:

In this book, I assume that you have at least some programming experience, and understand basic concepts like variables, loops, and branches. Ideally, you already have coded basic scripts in Python or MATLAB. Since our running example is centered around solving a system of partial differential equations, it is helpful if you have some knowledge of calculus and linear algebra. We will also be working a lot in the terminal, so some experience with Linux or UNIX-like shell is expected. Given the topic of the book, I expect that this book will be ideal for:

If you fit in one of the above categories, you may already know that Fortran’s main selling point is its ease of programming efficient and parallel programs for large supercomputers. This has kept it as the dominant HPC language of physical sciences and engineering. While this book will teach you Fortran from the ground up, I will also take the unconventional approach and teach it in the context of parallel programming from the get go. Rather than gaining just another technical skill as an afterthought, you will learn how to think parallel. You will recognize ways in which the workload and memory can be distributed to arrive at the solution more efficiently. With parallel thinking, you will come out with two critical advantages:

The first is a definite nice-to-have, but the second is a deal-breaker. Some problems simply can’t be solved without parallel programming. Next section will give you a gentle introduction and an example of parallel programming.

1.7  Think parallel!

Parallel programming is only becoming more important with time. The rate of semi-conductor density increase, as described by Moore’s law, while still positive, is limited. Traditionally we went past this limit by placing more processing cores on a single die. Even the processors in most smartphones today are multicore. Beyond the shared-memory computer, we have connected many machines using sophisticated networks, and made them talk to each other to solve huge computational problems. As I mentioned earlier, the weather forecast that you saw this morning on your favorite TV channel or news website was computed on hundreds or thousands of parallel processors. Due to the practical limits of Moore’s law and the current tendency toward many-core architectures, there is a sense of urgency to teach programming parallel-first.

All parallel problems fall into two categories:

Because our application domain deals mainly with non-embarrassingly parallel problems, we will focus on how to implement parallel data exchange between processors in a clean, expressive, and minimal way. This will involve both distributing the input data among processors (downward dashed arrows in Figure 1.4), and exchanging the data between them whenever there is inter-dependency (horizontal arrow in Figure 1.4).

Parallel Fortran programming in the past has been done either using the OpenMP directives for shared-memory computers only, or with the Message Passing Interface (MPI) for both shared and distributed memory computers. Differences between shared-memory (SM) and distributed-memory (DM) systems are illustrated on Figure 1.5. The main advantage of SM systems is very low latency in communication between processes. However, there is a limit to the number of processing cores that can exist in a SM system. Since OpenMP was designed for SM parallel programming exclusively, we will focus on MPI for our specific example below.

1.7.1  Copying an array from one processor to another

In most scientific and engineering parallel applications, there is data dependency between computational processes. Typically, a 2-d array is decomposed into tiles like a chess board, and the workload of each tile is assigned to a processor. Each tile has its own data in memory that is local to its processor. To illustrate the simplest case of parallel programming in a real-world scenario, let’s take the following meteorological situation for example. Suppose that the data consists of two variables, wind and air temperature. Wind is blowing from one tile with lower temperature (cold tile) toward another tile with higher temperature (warm tile). If we were to solve how the temperature evolves in time, the warm tile would need to know what temperature is coming in with the wind from the cold tile. Because this is not known a priori (remember that the data is local to each tile), we need to copy the data from the cold tile into the memory that belongs to the warm tile. On the lowest level, this is done by explicitly copying the data from one processor to another. When the copy is finished, the processors can continue with the remaining computations. Copying an array from one process to another is the most common operation done in parallel programming (Figure 1.6).

Since we’re barely starting, let’s focus on getting just this one operation done. Our goal is to do the following:

I will show you two examples of how to solve this problem. One is the traditional approach using an external library like MPI. Unless you’re a somewhat experienced Fortran programmer, don’t try to grok this example. I merely want to demonstrate how complicated and verbose this approach is. Then, I will show you the solution using the new Fortran Coarray approach. In contrast to MPI, with coarrays you can use an array indexing-like syntax to perform remote data exchange between parallel processes.

MPI: The traditional way of parallel programming

MPI has been often described as the assembly of parallel programming, and indeed, that was developers' original intention! The main vision of MPI was to be implemented by compiler developers to enable native parallel programming languages. However, over the past three decades, application developers were much faster at adopting MPI directly in their programs, and MPI has become, for better or for worse, a de facto standard tool for parallel programming in Fortran, C, and C++. As a result, most HPC applications today still rely on low-level MPI calls.

Below is a Fortran program that sends data from one process to another using MPI:

Running the program on 2 processors outputs the following:

The above output confirms that our program did what we wanted - copy the array values from process 0 to process 1.

Enter Fortran Coarrays

Coarray Fortran (CAF) is the native Fortran model for parallel programming. Originally developed by Robert Numrich and John Reid in the 1990s as an extension for the Cray Fortran compiler, CAF has been introduced into the standard starting with Fortran 2008 revision. Coarrays are very much like arrays, as the name implies, except that their elements are distributed along the axis of parallel processes (think cores or threads). As such, they provide an intuitive way to send and receive data between remote processes.

What follows is the coarray implementation of our array copy example:

The output of the program is the same as in the MPI variant:

These two programs are thus semantically the same. Let’s look at the key differences:

As we saw in this example, both MPI and CAF can be effectively used to exchange data between parallel processes. However, MPI code is low-level and verbose, and would soon becomes tedious and error-prone as the complexity of our app increases. In contrast, CAF offers an intuitive indexing syntax that is analogous to the familiar operations with arrays. Furthermore, with MPI, you tell the compiler what to do; with CAF, you tell the compiler what you want, and let it decide what’s the best way to do it. This approach relieves a big deal of responsibility off your shoulders, and lets you focus on your application. I hope that this convinces you that Fortran coarrays are the way to go for expressive and intuitive implementation of data exchange between parallel processes.

1.8  A Partitioned Global Address Space language

Fortran is also a Partitioned Global Address Space (PGAS) language. In a nutshell, PGAS abstracts the distributed-memory space and allows you to:

For example, with Fortran Coarrays, PGAS allows you to use one image to initiate a data exchange pattern between two remote images:

In this snippet, the if-statement ensures that the assignment executes only on image 1. However, the indices inside the square brackets refer to images 7 and 8! What this means is that the image 1 will asynchronously request an array copy from image 8 to image 7.

The power of PGAS is that, from the programmer’s point of view, the indices inside the square brackets can be treated just like any other array elements that are local in memory. However, in practice, these images could be mapped to different cores on the same shared-memory computer, or across the server room and connected via the local interconnect, or even across the world and connected through the internet!

Other notable PGAS languages are Chapel (chapel-lang.org) and Unified Parallel C (upc-lang.org).

1.9  Running example: A parallel tsunami simulator

I believe that most learning happens by doing rather than reading, especially if immersed in a longer-term project. Lessons in this book are thus framed within the context of developing your own, fully featured, parallel app.

1.9.1  Why tsunami simulator?

A tsunami is a series of long water waves that are triggered by a displacement of a large body of water. This typically occurs due to earthquakes, underwater volcanoes, or landslides. Once generated, a tsunami propagates radially outward and grows in height and steepness as it enters shallow water. I think a tsunami simulator is a good running example for this book because tsunamis are:

To simulate tsunamis, we will write a solver for the shallow water system of equations.

1.9.2  Shallow water equations

Shallow water equations (SWE) are a simple system equation derived from Navier-Stokes equations. They are also known as the Saint Venant equations, after the french engineer and mathematician A. J. C. Barre de Saint-Venant, who derived the 1-d form from first principles in pursuit of his interest in hydraulic engineering and open-channel flows. SWE are powerful because they can reproduce many observed motions in the atmosphere and the ocean:

SWE system consists of only a few terms:

What is the physical interpretation of the above system? The top equation states that where there is slope along the water surface, water will accelerate and move from region of higher level to region of lower level due to pressure gradient. The advection term is non-linear and causes chaotic behavior in fluids, known as turbulence. The bottom equation states that if there is an area where velocity is converging (coming together), there will be an increase in water level because water has to go somewhere - this is why we call it conservation of mass. Similarly, if the velocity is diverging (moving apart), there will be a decrease in water level in response.

Shallow water equations are dear to me because I first learned Fortran programming by modeling these equations in my undergraduate meteorology program at the University of Belgrade. Despite my Fortran code looking (and working) much differently now than back then, I still find this system of equations an ideal case for teaching parallel Fortran programming. I hope you enjoy the process as much as I do!

1.9.3  What we want our app to do

Let’s decide on some requirements for the features of our Tsunami Simulator:

In the above snippets, parallel decomposition and data exchange would be implemented inside the operators .dot., .grad., and .div., which correspond to dot product, gradient, and divergence operators, respectively. This way, the technical implementation is encapsulated inside these functions, and on high-level we would be able to code our equations much like we would write them on a blackboard.

If you haven’t already, I encourage you to go ahead and check out the code for the running example from Github:

Once you got it, take a look around, explore, peek inside the source files. It’s the project that we will build together. We start the next chapter by setting up the development environment so you can compile and run the minimal working version of our app.

1.10  Summary

In this chapter you learned that Fortran is:

Fortran is not a language for everybody. It is definitely not a systems programming language, nor a web development language. Programming a graphical video game or a web browser in Fortran is possible, but extremely difficult. However if you are working on computationally intensive problems in science or engineering, it may be exactly what you need. Modern Fortran will take you on a journey through core Fortran features from a parallel-first perspective. Where applicable, you will also apply object-oriented and functional techniques, as well as adapt existing Fortran libraries into your application. By working through this book chapter by chapter, you will gain the experience of developing a fully-featured parallel app from scratch. If it’s your first software project, I hope it excites your inner software developer and inspires you to go make something on your own.