Software Design/Construction -- Week 7
Introduction
This week we'll look at multithreading.
Note that the links from this page are to handouts that will be distributed the night of class. Only print out those handouts if you could not attend class.
Main topics this week:
C++ Programming Assignment Due
Multithreading Concepts
Threads in Java
Thread Priority
Thread States
Controlling Threads
Solo Threads
Interacting Threads
Using Wait and Notify
Synchronization to Avoid Race Conditions
Deadlock
Multithreading in C++
Multithreaded Programming Assignment
Midterm Review
Next Week
C++ Programming Assignment Due
Your C++ programming assignment is due this week for full credit.
Java allows multithreading. Multithreading is the use of more than one thread. Okay, fine, but what's a thread?
"thread" is short for "thread of execution". Your normal program executes one instruction at a time, in a sequence. When one instruction is being executed, no other instruction is being executed at the same time. That's a single thread of execution. If it helps, picture the thread of execution being a line that is drawn down through the code, crossing each instruction as it executes.
So, Java allows you to have multiple threads of execution. That means that more than one instruction in your program can be executed at exactly the same time. Each thread only has one instruction executing at a time, but all the threads may be executing at the same time.
Multithreading is can be useful to keep a program responsive to user input. Let's say that the user has asked the program to perform a calculation than can take a few minutes. The calculation could be performed in a separate thread, while the main thread continues to process user input. The user may very well be able to do other things in the program while waiting on the calculation to finish. If you only had a single thread of execution, the program would be unable to respond to user input while the calculation was being performed.
Multithreading can also introduce very difficult to find bugs into your program. The types of bugs introduced because of multithreading are called "race conditions". A race condition is a bug that is timing sensitive. In other words, the bug only happens when several conditions happen at exactly the same time. With multithreading, the possibility of race conditions increases.
Unfortunately, it's almost impossible to know if a multithreading program has bugs or not. Each thread runs at the same time as the other threads. A computer with a single CPU, however, cannot really run multiple instructions at the same time. So it runs instructions from one thread for a while, and then runs some instructions from another thread. You have no way of knowing exactly when a thread may be interrupted and another thread run.
So, you want to write a program that uses a second thread. Java provides an interface and a class to help you out.
The interface is called Runnable, and is implemented by an object that wants to be run as a seprate thread. The Runnable interface has only one method, public void run (). The run method is the thread's equivalent to main. It's called when the thread starts, and when the run method ends, the thread ends.
The class is called Thread. This class does the technical work of getting a new thread started. It takes as a parameter an object that implements the Runnable interface. The start method gets the thread going. Note that when you call the start method, it returns immediately in the current thread. The other thread starts immediately in the Runnable object's run method.
For example:
class Test implements Runnable
{
public void run ()
{
...do something...
}
}class Example
{
public static void main (String [] args)
{
Thread t = new Thread (new Test ());t.start ();
...do something at the same time as Test.run is doing something...
}
}
There are a lot more methods on the Thread class that we'll look at as we go.
Every Thread has a priority, which can be set by calling the setPriority method on the Thread instance. Threads with higher priority are run more often than threads with lower priority. Note that you may try to set the priority higher than is legal, depending on other factors, and Java will lower the priority to the maximum possible legal value.
A Thread object can be in one of several states. These include:
Running -- The thread is currently executing.
Dead -- The thread has completely finished executing, and will never execute again. This typically means that the run () method has finished executing. The object still exists, so you can call methods on it to get information out of the thread, but you cannot start the thread again.
Ready -- The thread can execute, but is currently not executing. The CPU has switched to another thread and will get back to this one in time.
Blocked -- A blocked thread is waiting for some external event to finish. Examples would include a thread that is trying to read from a file. The thread will block at the call to read from the file, and not continue executing until the read is finished. In this way, a thread that starts an external task will allow other threads to run until that external task is finished.
Waiting -- A thread can wait for other threads to finish work it needs to use, by calling the wait () method. A thread that is waiting for other threads to finish some work will not execute until those threads call the notify () method.
Sleeping -- A thread can voluntarily put itself to sleep for a certain period of time. The thread will start executing again only after the given period of time has passed. The thread puts itself to sleep using the Thread.sleep () method. A thread should put itself to sleep if it only wants to execute every so often.
There are various methods for controlling threads. We've seen Thread.start () to begin a thread. Thread.sleep () allows a thread to sleep for a specific period of time.
There's also Thread.yield (), which allows a thread to voluntarily give up control of the CPU to allow other threads to execute. If there are no other threads to execute, the thread will continue immediately.
Java keeps a queue of threads called the "ready queue". These are threads that are in the Ready state. Threads go from the Running state into one of the other states via method calls. For example, a thread that is Running will go into the sleeping State when Thread.sleep () is called. When the time period for the sleep is over, the thread goes into the Ready state and is placed into the ready queue. Whenever Java needs to figure out which thread to execute next, it pulls a thread from the ready queue.
Here's a diagram of how threads move from state to state:
The thread transitions from Ready to Running when Java decides to run the thread some more. There's also a transition from Running directly to Ready when the thread uses the Thread.yield method to voluntarily give up control of the CPU.
There's a method called Thread.interrupt that can be used to interrupt a thread that is waiting, sleeping, or blocked. The thread will begin executing and will receive an InterruptedException.
Most threads must interact with other threads in some way. This interaction can give rise to various sorts of problems. Let's look at an example. The problem is to have one thread that produces data, and one or more threads that consume that data (you'll also hear these refered to as writers and readers). Here are the two classes we'll look at:
class Producer implements Runnable
{
private LinkedList data;public void setList (LinkedList l)
{
data = l;
}public void run ()
{
while (true)
{
data.add ("Hello");try
{
Thread.sleep (1 + (int) (Math.random () * 1000));
}
catch (InterruptedException e)
{
/* If we were interrupted, go ahead and end */
break;
}
}
}
}class Consumer implements Runnable
{
private LinkedList data;public void setList (LinkedList l)
{
data = l;
}public void run ()
{
while (true)
{
if (data.size () > 0)
System.out.println (data.removeFirst ());try
{
Thread.sleep (50);
}
catch (InterruptedException e)
{
/* If we were interrupted, go ahead and end */
break;
}
}
}
}
The producer thread will add a String to the linked list at random intervals, anywhere from 1 to 1000 milliseconds. The consumer thread will check for a new string every 50 milliseconds. If a string is found, it will be printed to the screen.
What is wrong with this way of having threads interact via sleep ()?
Even if we fix that, what might happen if we have multiple readers operating at the same time? There's a race condition that can cause incorrect results, depending on how the threads run.
Synchronization to Avoid Race Conditions
The way to avoid race conditions is to synchronize access to any shared resources. Any data members that are accessed by more than one thread count as resources. When you synchronize access to shared resources, you are saying that only one thread at a time can access those resources.
The key to using synchronization is to identify what are known as "critical sections" of code. A critical section is a section of code that actually uses some sort of shared resource. We synchronize only these critical sections of code.
Note that synchronization is only one way to avoid race conditions. It is the method used in Java, but not the only method used in other languages. In the operating systems course you'll learn about other methods to avoid race conditions.
When you use synchronization, you must ask for a lock on a shared resource. If the lock is already taken by another thread, you will wait until the lock is available again. If the lock is available, your critical section will execute. When the critical section is done, the lock will be released automatically.
In Java, we synchronize a critical section of code using the synchronized keyword. When we synchronize, we need to give the reference of the resource to lock. For example, to lock on the LinkedList in the example above.
class Producer implements Runnable
{
private LinkedList data;public void setList (LinkedList l)
{
data = l;
}public void run ()
{
while (true)
{
synchronized (data)
{
data.add ("Hello");
}try
{
Thread.sleep (1 + (int) (Math.random () * 1000));
}
catch (InterruptedException e)
{
/* If we were interrupted, go ahead and end */
break;
}
}
}
}class Consumer implements Runnable
{
private LinkedList data;public void setList (LinkedList l)
{
data = l;
}public void run ()
{
while (true)
{
synchronized (data)
{
if (data.size () > 0)
System.out.println (data.removeFirst ());
}try
{
Thread.sleep (50);
}
catch (InterruptedException e)
{
/* If we were interrupted, go ahead and end */
break;
}
}
}
}
This prevents two readers from conflicting with one another, since as soon as one reader enters the critical section, no other object can get a lock on the linked list until that reader finishes the critical section.
You can also synchronize and entire method in an object, like this:
class Foo
{
public synchronized void doSomething ()
{
...do something...
}
}
If you synchronize an entire method, the lock you are getting is on the instance of the object in which that method lives. So it's the exact same effect as doing:
class Foo
{
public void doSomething ()
{
synchronized (this)
{
...do something...
}
}
}
You always need to make sure that you're getting locks on the right objects. Two synchronized blocks that are getting locks on different objects will not prevent each other from running.
Any time that you have synchronization, you have the potential for deadlock. Deadlock is when thread A is waiting on a resource currently being used by thread B, and thread B is waiting on a resource currently being used by thread A. Neither thread can continue. (Show the dining philosophers example)
Preventing deadlock usually means imposing an ordering on the resources, such that everyone is always trying to get the same resource first. That way, the one that gets it can continue on and the rest will wait for that resource before getting locks on any other resources.
We said that using sleep for the reader was not very efficient. An alternative to using sleep that is more efficient is using wait and notify. These are methods on the Object class, so any object can call them. Both wait and notify must be performed inside a synchronized block.
Calling wait gives up the CPU and moves the thread to the Waiting state. When another thread calls notify from within a syncrhonized block with a lock on the same object as the thread that called wait, the thread that called wait is moved out of the waiting state and into the Ready state. But, remember, the call to wait was inside a syncrhonized block. So before the thread can actually be run again, it must regain the lock it had. So it will have to wait until it can get the lock again before continuing execution. Here's a diagram of how this flows:
The use of wait and notify is efficient, because a thread will not wake up until there is actually something for it to do. There are some exceptions to this. A thread that is Waiting will wake up if it is interrupted. That is why a call to wait () must catch the InterruptedException. A thread that is Waiting will wake up if the timeout for the wait () call is reached...this means that the thread will only wait a certain amount of time before continuing.
Here are the various wait methods that you can use:
void wait () -- Thread will wait an infinite amount of time. This call will only return if the thread was notified, or interruped. Thread gives up control of the CPU.
void wait (long timeout) -- Thread will wait for the given number of milliseconds. After that number of milliseconds, the thread will resume executing, even if there has been no notify call made. Thread gives up control of the CPU.
void wait (long timeout, int nanoseconds) -- Thread will wait for the given number of milliseconds and nanoseconds. After that amount of time, the thread will resume executing, even if there has been no call to notify made. Thread gives up control of the CPU.
void notify () -- Thread notifies one other thread that is waiting inside a lock on the same object as this thread has locked. In other words, the argument to synchronized must have been the same for both threads. The current thread retains control of the CPU.
void notifyAll () -- Thread notifies all other threads that are waiting inside a lock on the same object as this thread has locked. The current thread retains control of the CPU.
Let's look at our producer/consumer example, using wait and notify instead of sleep. This also moves the linked list into a separate object and synchronizes the methods on that object. This keeps the producer and consumer from needing to synchronize and reduces the errors possible with multithreading.
class DataStore
{
private LinkedList data = new LinkedList ();
public synchronized void insert (String s)
{
data.add (s);
notifyAll ();
}
public synchronized String get ()
{
while (data.size () == 0)
{
try
{
wait ();
}
catch (InterruptedException e)
{
/* Do nothing if we were interrupted. Let the while loop
do another wait if the data store is still empty */
}
}
return (String) data.removeFirst ();
}
}
class Producer implements Runnable
{
private DataStore data;
private int num;public void setDataStore (DataStore ds)
{
data = ds;
}public void setNum (int n)
{
num = n;
}public void run ()
{
while (true)
{
System.out.println ("Producer adding data");
data.insert ("Hello");
try
{
Thread.sleep (1 + (int) (Math.random () * 1000));
}
catch (InterruptedException e)
{
/* If we were interrupted, go ahead and end thread */
break;
}
}
}
}class Consumer implements Runnable
{
private DataStore data;
private int num;
public void setDataStore (DataStore ds)
{
data = ds;
}
public void setNum (int n)
{
num = n;
}
public void run ()
{
while (true)
{
System.out.println ("Consumer " + num + ":" + data.get ());
}
}
}
Note that we still use sleep for the producer...that is a legitimate use of sleep, in a thread that is creating data for others to use. But for the consumer, we use wait instead (inside the DataStore.get method). The producer uses notify (inside the DataStore.insert method) to wake all consumer threads when it adds data. You can run this example using a main that creates several instances of producers and consumers and runs them, such as:
class ThreadTest
{
public static void main(String[] args)
{
DataStore ds = new DataStore ();
Consumer c = new Consumer ();
c.setDataStore (ds);
c.setNum (1);
new Thread (c).start ();
c = new Consumer ();
c.setDataStore (ds);
c.setNum (2);
new Thread (c).start ();
c = new Consumer ();
c.setDataStore (ds);
c.setNum (3);
new Thread (c).start ();
c = new Consumer ();
c.setDataStore (ds);
c.setNum (4);
new Thread (c).start ();
Producer p = new Producer ();
p.setDataStore (ds);
p.setNum (1);
new Thread (p).start ();
p = new Producer ();
p.setDataStore (ds);
p.setNum (2);
new Thread (p).start ();
}
}
Here, we create four consumers and two producers. You can run this to see the order in which producers and consumers process data. There are still things we would need to do to make this a realistic example. Both the producer and consumer threads should have methods that could be called to tell the thread to stop. These methods would set a flag that the run method would check to see if the thread should stop or not. We may also need to set priorities on the threads depending on the problem we're solving. You can set some priorities on the above threads to see how that changes when each thread is run.
Multithreaded programming is extremely difficult to get right for any complex problem. You always want to keep the critical sections as small as possible, so they're only dealing with the resources that are shared. If possible, encapsulate the synchronization code in a class that handles the shared data. This limits the number of places you need to put synchronization code, and so reduces the chances for error. But no matter how good your synchronization code, there are probably still going to be errors that will only show up after a month or two of use.
Multithreading is possible in C++, but is not directly supported by the C++ libraries. Instead, each operating system provides a different set of system calls for multithreading in C++. Multithreading on Windows is different from multithreading on Unix for C++.
While this makes using multithreading in C++ a bit harder, there is also quite a bit more flexibility. Java provides a basic object lock through synchronization, so an object has exclusive access to a critical section. In most operating systems, C++ can do much more sophisticated multithreading. For example, consider the example of having one producer and multiple consumers. If the consumers are not modifying the data, then there is no reason why multiple consumer threads cannot run at the same time. Java will not allow this, but C++ will, using a technique called semaphores.
So multithreading in C++ can be more sophisticated, which makes it more powerful but also harder to code correctly. You'll learn more about multithreading in C++ in the Operating Systems course.
Multithreaded Programming Assignment
The multithreaded programming assignment is assigned today, and is due on week 10. Details on the assignment are in your student manual.
We'll review what to expect from the midterm.
Next week is the midterm exam.