concept thread in category akka

This is an excerpt from Manning's book Reactive Design Patterns. Login to get full access to this book.

Each of these blocking calls interacts with the network equipment, generating messages and reacting to messages under the hood, but this fact is completely hidden in order to construct a synchronous façade on top of the underlying message-driven system. The thread executing these commands will suspend its execution if not enough space is available in the output buffer (for the first line) or if the response is not immediately available (on the second line). Consequently, this thread cannot do any other work in the meantime: every activity of this type that is ongoing in parallel needs its own thread, even if many of those are doing nothing but waiting for events to occur.

If the number of threads is not much larger than the number of CPU cores in the system, then this does not pose a problem. But given that these threads are mostly idle, you want to run many more of them. Assuming that it takes a few microseconds to prepare the requestMessageBytes and a few more microseconds to process the responseBuffer, whereas the time for traversing the network and processing the request on the other end is measured in milliseconds, it is clear that each thread spends more than 99% of its time in a waiting state.

In order to fully utilize the processing power of the available CPUs, this means running hundreds if not thousands of threads, even on commodity hardware. At this point, you should note that threads are managed by the operating system kernel for efficiency reasons.^[2] Because the kernel can decide to switch out threads on a CPU core at any point in time (for example, when a hardware interrupt happens or the time slice for the current thread is used up), a lot of CPU state must be saved and later restored so that the running application does not notice that something else was using the CPU in the meantime. This is called a context switch and costs thousands of cycles^[3] every time it occurs. The other drawback of using large numbers of threads is that the scheduler—the part of the kernel that decides which thread to run on which CPU core at any given time—will have a hard time finding out which threads are runnable and which are waiting and then selecting one such that each thread gets its fair share of the CPU.

Listing 3.1. Unsafe, mutable message class, which may hide unexpected behavior

import java.util.Date; public class Unsafe {     private Date timestamp; #1     private final StringBuffer message; #2     public Unsafe(Date timestamp, StringBuffer message) {         this.timestamp = timestamp;         this.message = message;     }     public synchronized Date getTimestamp() {         return timestamp;     }     public synchronized void setTimestamp(Date timestamp) { #3         this.timestamp = timestamp;     }     public StringBuffer getMessage() { #4         return message;     } } #1 - Fields should be final so the compiler can enforce immutability. #2 - Mutable because the content of the buffer is not stable #3 - Mutable because the timestamp can be changed via the setter. Synchronizing both the getter and setter adds thread safety but does nothing to relieve contention between threads accessing the object. #4 - Mutable because the content of the buffer is not stable

copy

The problem you are facing is that executing a database query may take an arbitrary amount of time: the database may be overloaded or failing, the query may not be fully optimized due to a lack of indices, or it might be a very complex query over a large dataset to begin with. If you execute a slow query in an Actor that runs on a shared thread pool, that thread will effectively be unavailable to the pool—and thereby to all other actors—until the result set has been communicated back. Other actors on the same thread pool may have scheduled timers that may be processed only with a large delay unless enough other threads are available. The bigger the thread pool, the more such blocking actions it can tolerate at the same time; but threads are a finite resource on the JVM, and that translates into a limited tolerance for such blockers.

The source of the problem is that a shared resource—the thread pool—is being used in a fashion that is violating the cooperative contract of the group of users. Actors are expected to process messages quickly and then give other actors a chance to run: this is the basis for their efficient thread-sharing mechanism. Making a resource unavailable to others by seizing it means the Actor that blocks the thread claims exclusive ownership, at least for the duration of the database query in this example. You have seen for the Resource Loan pattern that the owner can grant exclusive access to another component, but that must always happen explicitly: you must ask the owner for permission.

A thread pool usually does not have a mechanism for signaling that a given thread is being blocked exclusively for the currently running task.^[13] If you cannot ask the owner of a shared thread for permission, the logical conclusion is that you need to own a thread on which you can run the database query. This can be done by creating a private thread pool that is managed by the actor: now you can submit the blocking JDBC calls as tasks to this pool.

This is an excerpt from Manning's book Akka in Action. Login to get full access to this book.

So where are those new concurrency features? Support for concurrency in most programming languages, especially on the JVM, has hardly changed. Although the implementation details of concurrency APIs have definitely improved, you still have to work with low-level constructs like threads and locks, which are notoriously difficult to work with.

Table 16.2. Available built-in dispatchers

Type	Description	Use case
Dispatcher	This is the default dispatcher, which binds its actors to a thread pool. The executor can be configured, but uses the fork-join-executor as default. This means that it has a fixed thread pool size.	Most of the time, you’ll use this dispatcher. In almost all our previous examples, we used this dispatcher.
PinnedDispatcher	This dispatcher binds an actor to a single and unique thread. This means that the thread isn’t shared between actors.	This dispatcher can be used when an actor has a high utilization and has a high priority to process messages, so it always needs a thread and can’t wait to get a new thread. But you’ll see that usually better solutions are available.
BalancingDispatcher	This dispatcher redistributes the messages from busy actors to idle actors.	We used this dispatcher in the router load-balancing example in section 9.1.1.
CallingThreadDispatcher	This dispatcher uses the current thread to process the messages of an actor. This is only used for testing.	Every time you use TestActorRef to create an actor in your unit test, this dispatcher is used.

When we look at our receiver actor with 100 workers, we could use the Pinned-Dispatcher for our receiver. This way it doesn’t share the thread with the workers. And when we do, it solves the problem of the receiver being the bottleneck. Most of the time a PinnedDispatcher isn’t a solid solution. We used thread pools in the first place to reduce the number of threads and use them more efficiently. In our example the thread will be idle 33% of the time if we use a PinnedDispatcher. But the idea of not letting the receiver compete with the workers is a possible solution. To achieve this, we give the workers their own thread pool by using a new instance of the dispatcher. This way we get two dispatchers, each with its own thread pool.

In our example we see that the workers can’t keep up with the arrival of messages, which is caused because we have too few threads, so why don’t we increase the number of threads? We can, but the result greatly depends on how much CPU the workers need to process a message.

Increasing the number of threads will have an adverse effect on the total performance when the processing is heavily dependent on CPU power, because one CPU core can only execute one thread at any given moment. When it has to service multiple threads, it has to switch context between multiple threads. This context switch also takes CPU time, which reduces the time used to service the threads. When the ratio of number of threads to available CPU cores becomes too large, the performance will only decrease. Figure 16.9 shows the relationship between the performance and number of threads for a given number of CPU cores.

Figure 16.9. Performance versus number of threads

The first part of the graph (up to the first dotted vertical line) is almost linear, until the number of threads is equal to the number of available cores. When the number of threads increases even more, the performance still increases, but at a slower rate until it reaches the optimum. After that, the performance decreases when the number of threads increases. This graph assumes that all the available threads need CPU power. So there’s always an optimum number of threads. How can you know if you can increase the number of threads? Usually the utilization of the processors can give you an indication. When this is 80% or higher, increasing the number of threads will probably not help to increase the performance.

But when the CPU utilization is low, you can increase the number of threads. In this case the processing of messages is mainly waiting. The first thing you should do is check whether you can avoid the waiting. In this example it looks like freezing your actor and not using nonblocking calls, for example, using the ask pattern, would help. When you can’t solve the waiting problem, it’s possible to increase the number of threads. In our example we’re not using any CPU power, so let’s see in our next configuration example if increasing the thread number works.

16.5. Changing thread releasing

In previous sections you saw how you can increase the number of threads, and that there’s an optimum number, which is related to the number of CPU cores. When there are too many threads, the context switching will degrade performance. A similar problem can arise when different actors have a lot of messages to process. For each message an actor has to process, it needs a thread. When there are a lot of messages waiting for different actors, the threads have to be switched between the actors. This switching can also have a negative influence on performance.

Akka has a mechanism that influences the switching of threads between actors. The trick is not to switch a thread after each message when there are messages still in the mailbox waiting to be processed. A dispatcher has a configuration parameter, throughput, and this parameter is set to the maximum number of messages an actor may process before it has to release the thread back to the pool:

In these examples we have better performance when we set the parameter high, so why is the default 5 and not 20? This is because increasing throughput also has negative effects. Let’s suppose we have an actor with 20 messages in its mailbox, but the service time is high, for example, 2 seconds. When the throughput parameter is set to 20, the actor will claim the thread for 40 seconds before releasing the thread. Therefore, other actors wait a long time before they can process their messages. In this case the benefit of the throughput parameter is less when the service time is greater, because the time it takes to switch threads is far less than the service time, which can be negligible. The result is that actors with a high service time can take the threads for a long time. For this there’s another parameter, throughput-deadline-time, which defines how long an actor can keep a thread even when there are still messages and the maximum throughput hasn’t been reached yet:

my-dispatcher {
  throughput = 20
  throughput-deadline-time = 200ms
}

copy

By default, this parameter is set to 0ms, meaning that there isn’t a deadline. This parameter can be used when you have a mix of short and long service times. An actor with a short service time processes the maximum number of messages after obtaining a thread, while an actor with a long service time will only process one message or multiple messages until it reaches 200 ms each time it obtains a thread.