Run-Time Environments (RTE)One of the main objectives of this course is learning to be intelligent users of modern, sophisticated RTEs. 1.

What is a runtime environment (RTE)?

A runtime environment is a virtual machine state which provides software services for processes or programs while a computer is running. It may pertain to the operating system itself, or the software that runs beneath it. The primary purpose is to accomplish the objective of "platform independent" programming. Runtime activities include loading and linking of the classes needed to execute a program, optional machine code generation and dynamic optimization of the program, and actual program execution. For example, a program written in Java would receive services from the Java Runtime Environment by issuing commands from which the expected result is returned by the Java software. By providing these services, the Java software is considered the runtime environment of the program. Both the program and the Java software combined request services from the operating system. The operating system kernel provides services for itself and all processes and software running under its control. The Operating System may be considered as providing a runtime environment for itself. In most cases, the operating system handles loading the program with a piece of code called the loader, doing basic memory setup and linking the program with any dynamically linked libraries it references. In some cases a language or implementation will have these tasks done by the language runtime instead, though this is unusual in mainstream languages on common consumer operating systems. Some program debugging can only be performed (or are more efficient or accurate) when performed at runtime. Logical errors and array bounds checking are examples. For this reason, some programming bugs are not discovered until the program is tested in a "live" environment with real data, despite sophisticated compile-time checking and pre-release testing. In this case, the end user may encounter a runtime error message. Early runtime libraries such as that of Fortran provided such features as mathematical operations. Other languages add more sophisticated memory garbage collection, often in association with support for objects. More recent languages tend to have considerably larger runtimes with considerably more functionality. Many object oriented languages also include a system known as the "dispatcher" and "classloader". The Java Virtual Machine (JVM) is an example of such a runtime: It also interprets or compiles the portable binary Java programs (bytecode) at runtime. The .NET framework is another example of a runtime library.

Examples of RTEs: 1. Unix and Win32 Operating Systems 2. Java Virtual Machine (JVM) 3. On occasion, Servlet containers and distributed RTEs

The operating system (OS)

The operating system is an example of a Run Time Environment (RTE). The entity running/executing inside the operating system is called a process System calls are special low level functions which enable the interaction between the process and the operating system.

Operating System
The operating system (OS) manages the sharing of the resources of a computer. An operating system performs basic tasks such as controlling and allocating memory, controlling input and output devices, controlling CPU time, facilitating networking and managing file systems. Most services offered by the OS are abstracted as Objects (e.g., files and directories). The interface with which a process may request service from the OS is termed system calls.

Block diagram of Operating System modules

Process
The main abstraction used by the operating system for an execution unit is a process. A process is an instance of a program, which is currently executing. While a program itself is just a passive collection of instructions, a process is the actual execution of those instructions. There may exist many incarnations of the same program simultaneously, but each incarnation is a process on its own.

Process / OS interaction
The nature of the interaction between the OS and the Process is about resource allocation. The OS is in charge of managing the computer resources between all of the processes. The main resources the OS offers and manages for all processes are: CPU time Disk space Memory Input and Output devices

The OS may also offer other services to processes, for example: communication. CPU time The CPU is the main resource shared by all processes, which can only execute machine code sequentially, one instruction at a time. To support multiple processes executing concurrently, the OS needs to share the CPU between all these processes. This sharing of CPU time is termed multitasking. Although being a nice property to have, multitasking has its price. It takes time to tell the CPU to stop executing one process and switch to another one. The act of switching between processes is called a context switch. In modern OSs, running on recent CPUs, a context switch usually takes around ~10 microseconds. The entity inside the OS which is in charge of selecting which process will be executed next is called the scheduler Processes may be in several states. A process may either be running, if it is currently executed by the CPU or, otherwise, waiting. Waiting is further divided to: Waiting for CPU time (because the CPU is busy executing instructions for another process) Waiting for Input-Output to complete (this is because an Input/Output operation is much slower than a CPU operation) Waiting for an event (because the process is waiting for another process to reach a certain state before it can continue)

The OS interacts with processes in many different ways. The interaction process/OS can be partitioned into two categories: Services which are initiated by the OS Services which are requested by the process

Examples of services initiated by the OS include CPU and initial memory allocation (to load the code itself). Services which the process might request include request for more

memory, allocation of disk space, Access to external devices like mice and outputting to the screen.

Process Life Cycle

The OS manages the life cycle of a process. Generally there are 3 main steps: Creating the process, managing the process and terminating the process. In further detail we can mark the following steps: The OS is asked by the user to create a process. In general, this is done through a shell. The OS identifies which program (= executable file) is to be executed to create the process. This is a file that contains instructions to be executed on the CPU to activate the process. The format of executable files is specified by the OS. The OS creates a process instance from the executable. This instance is identified by its process ID, which is unique among all running processes. All information about running processes is maintained by the OS in a process table. The OS allocates memory for the new process in the computer's main memory (RAM). The OS loads the instructions from the program executable file into main memory. The OS identifies the starting function in the program (e.g "main" in C and C++) and invokes it (calls the function). The main function receives arguments passed from the OS. These are called the command line arguments. From that point on, the instructions of the main function are executed as specified in the program. When the process invokes a system call (that is, the program executes a line such as printf("Hello");, which initiates a system call to print to the screen), the process is interrupted until the OS executes the service requested. This means that each time a system call is invoked, the process undergoes a context switch. When the process executes the special system call exit, the process is terminated: memory and all other resources allocated to the resource are

freed, the process ID is freed and the process entry is removed from the OS process table.

When we write a program, we must understand the difference between calling a regular function, defined in the program, and invoking a system call. They superficially look the same (that is, calling the function f(1, 2); and (indirectly) invoking a system call via printf("Hello"); seem to be doing the same thing. But a system call invocation is completely different from a function call. The interface between a program and the OS is defined as a set of system calls, each with its own parameters. For modern OSs such as Linux or Win32, there are a few hundreds to a few thousands system calls defined. The following figure summarizes the main elements involved when an OS RTE creates a process.

Elements involved in a process creation

Summary
The operating system is an example of a basic but complex RTE. It is the basic interface between the user code and the hardware of the machine hosting the user code. Using abstractions like "a process" which is a collection of variables: state, current execution point, current resources, the OS can create the feeling of simultaneous execution or more precisely multitasking.

The Java virtual machine (JVM)

The Java virtual machine is an RTE. It executes Java classes The JVM is a computer program (executable in a binary form) belonging to a family of programs called interpreters. Interpretation is one of the two major ways in which a programming language can be implemented, the other being compilation to machine code (like in the case of C and C++), which the CPU can directly execute (as managed by the OS).
The JVM simulates a complete machine, on which Java programs are executed. When we compile a Java source file, we create a file which contains instructions specific for the JVM, called byte code. This is equivalent to the process of compiling a regular C/C++ program into regular machine code.

The JVM byte code is then interpreted. This gives Java some nice properties. The most prominent one is that Java programs are portable; a Java program compiled on any architecture can be interpreted on any other architecture given that an implementation of the JVM for the destination architecture exists. For almost all known architectures there exists a Java interpreter (a JVM) implementation. The JVM is itself a process inside some RTE (in general an OS RTE), and acts as an intermediate level between the interpreted program and the RTE. (The JVM is concretely an executable called java.) In contrast to the mechanism of system calls in which a process must call functions to interact with the OS, a Java program interacts with the JVM using the object oriented paradigm. For example one can have an object File. during the creation of the object the JVM might call the system call openFile. The interface between the JVM and Java programs (aka the abilities given to the program by the JVM) is described in the Java language. For example, consider the object System.out, which you use to print strings to the screen. It is an object that represents the JVM support for printing. Similarly, the JVM supports security models, communications, distributed objects and more.

The OS RTE running a general process and a JVM RTE which runs a Java class

A Java Program Lifecycle

A program written in Java is compiled into a set of files, in a format called "class". For example, given a text file P.java, the compiler generates a java byte code file P.class in. The class file format is a standard specified as part of the JVM specification. The JVM manages the lifecycle of a Java process in the following manner: The JVM is invoked with a parameter which refers to a class file. For example, java P.class. The JVM starts running as a process inside the OS. The JVM then activates, inside this process, the following operations. The JVM loads the class file. If the class refers to other classes, the JVM also loads the other classes. This is indicated in Java by the import mechanism. The import clauses are resolved by loading the requested class files into the main memory of the JVM process.This mechanism is recursive (that is, an imported class may import another class and so on). Once all classes are loaded, the JVM identifies a method with signature
static Main(String args[]) in the executed class file. The JVM

parses the command line parameters that appear after the name of the class path in the main command line, and then invokes the Main method with these parameters. When the Java program requires a service from the JVM, it invokes the corresponding method from the JVM interface. In general, the JVM services are located in the System package. When the program executes the System.exit method, the JVM cleans up all the resources allocated to the program and then the JVM process is terminated. NB//

The Java program interacts with the JVM exclusively - it never invokes a system call. Instead, it calls methods from Object Oriented interfaces which are part of the Java standard environment. When the JVM executes the code corresponding to such a service request, it translates it into the appropriate system call to the containing OS. A single JVM process can execute more than one Java class simultaneously. Naturally, it is also possible to run several instances of the JVM in the same OS.

The following figure summarizes the main elements involved when the JVM creates a process inside an OS RTE:

Elements involved when a Java program executes in the JVM RTE itself executed in the OS RTE

Internet Browser
An Internet browser (such as Firefox, Internet Explorer or Opera) is also an RTE. The browser provides the run time environment for applets. Applets are mini-applications that run in the context of HTML pages within an Internet Browser. Applets are written in Java. They interact with a special Applet Container implemented inside the Internet browser. An applet programmer must write a program adapted to its very special environment: the Applet container. The interaction between the environment and the applet is defined by the Applet interface: An applet is loaded in the container The applet is "initialized" - this happens only once in the life of the applet. The applet is "started" - this happens when the page that contains the applet becomes visible on screen. It can happen many times as the user browses through different pages. The applet is "stopped" - this happens when the page is not visible anymore. The applet is "destroyed" - this happens when the container does not have any more resources to maintain the applet, or when the container is shutdown.

The applet is a visual object - it is used in general to display information in a window allocated by the container. +

The relation between the Applet, the browser container and the OS RTE

Servlets and HTTP Servers

The Web and the Client-Server Model The Web is an example of the general client/server model: the Web browser is the client of a Web server; the client submits requests to the server through the network and receives replies. The rules of the conversation between the client and the server are called a protocol. In the case of the Web, the protocol is called HTTP (Hyper Text Transfer Protocol). A protocol determines: a) How the client locates the server (in the case of the Web, as a URL). b) Which requests the client can submit and how the requests are encoded (in the case of the Web, there are 2 main request types: GET and POST). c) Which types of replies the server can send back to the clients and how these replies are encoded (in the case of the Web, the replies are encoded as HTML).

In the same way that a Web browser can host applets, a Web server can also host programs, to be run on demand. Such programs embedded in a server are called servlets. Servlets open up the possibility of serving dynamic content: when the server receives a request for a certain URL, it executes a servlet, and sends back to the client the output of the invocation. In general, a servlet is an object used to extend the capabilities of servers that host applications accessed via a request-response programming model. A servlet is executed by a Servlet Container. For example, Tomcat is a well-known Java Servlet Container. To run a servlet, you need the following environment: A Tomcat HTTP server running as a process

A servlet, compiled in the form of a Java class file, located in a specific directory accessible by Tomcat. In a Tomcat configuration file, one specifies to which URL the servlet is associated. A Web client to submit requests to the Tomcat server.

The following figure illustrates how the client can send a request that eventually reaches a servlet hosted by an HTTP servlet container:

The relation between the Servlet, the HTTP server container and the Web Client

How a servlet is invoked

The Web client connects the HTTP server by issuing a specific URL. For example, when the client requests this URL: http://localhost:8080/servlet1?action=edit the following steps occur: The client connects the server at address localhost:8080 The HTTP server running at this address gets the request /servlet1?action=edit The HTTP server decides that this request is mapped to the servlet1 servlet The HTTP server activates the servlet, and invokes the method doGet with a parameter that encapsulates the "action=edit" string.

The servlet executes its code and returns a string which is encapsulated in an Object The HTTP server passes back this computed string to the client The client displays the answer on the screen

A URL (Uniform Resource Location) is a standard format to uniquely identify a "resource" that can be accessed through a protocol. For example, the URL http://www.cs.bgu.ac.il/~spl081 identifies the home page of this course. Servlet Lifecycle
The life cycle of a servlet is controlled by the container in which the servlet has been deployed (that is, the servlet class file has been located in a folder managed by the container and associated to a specific URL under the server). When a request is mapped to a servlet (that is, the URL sent by the client corresponds to the one configured to activate the servlet), the container performs the following steps.

Lifecycle of a Servlet
The Servlet lifecycle consists of the following steps: 1. The Servlet class is loaded by the container during start-up. 2. The container calls the init() method. This method initializes the servlet and must be called before the servlet can service any requests. In the entire life of a servlet, the init() method is called only once. 3. After initialization, the servlet can service client-requests. Each request is serviced in its own separate thread. The container calls the service() method of the servlet for every request. The service() method determines the kind of request being made and dispatches it to an appropriate method to handle the request. The developer of the servlet must provide an implementation for these methods. If a request for a method that is not implemented by the servlet is made, the method of the parent class is called, typically resulting in an error being returned to the requester. 4. Finally, the container calls the destroy() method which takes the servlet out of service. The destroy() method like init() is called only once in the lifecycle of a Servlet.

Java Enterprise Edition (JEE) and Enterprise Java Beans (EJB)

J2EE is a special environment for distributed component-based applications: an application is viewed as a collection of components that run on several computers across a network and collaborate with each other. Some components are used to store data structures in a database, some others implement usage policies or enforce access rights.

The run-time environment is called Enterprise Java Beans Server. It is a distributed application (it runs on several machines on a network that communicate through a dedicated protocol). In the case of EJB three level of abstractions exists: The EJB Server is an RTE that runs containers. Containers are RTE themselves and they run components which are the java classes doing the actual work. Recall the EJB Server itself can be an executable running in the OS RTE, however it can be (and usually is) written in Java, hence it is running in the JVM RTE. The EJB server/container (we will not distinguish them at this point) are responsible to implement the following services: o Naming: components are named so that other components can find them based on their name and not on their location. This way, components can move from machine to machine without interfering with each other. The naming server maps abstract component names to actual network addresses. o Messaging: components send messages to each other. The messaging server makes sure messages reach their destination and dispatches messages based on flexible policies. o Activation: the Application Server determines when each component should run. It can also decide to de-activate a component, either permanently (destroy) and temporarily (unload). The lifecycle of components within the Application Server environment is similar to that of applets within a browser. The Application Server manages the state of components (that is, it stores and re-loads information in such a way that components are re-activated in the same state they were when they were de-activated). o Security: the Application Server enforces access rights according to application roles. Each component can specify which users can perform which actions. The Application Server enforces these rules. o Transactions: The programmer can specify that a sequence of actions that start in one component, and continue into other components behaves as a "unit" transaction - that is, a primitive operation that can either succeed as a whole (commit), or fail as a whole without leaving partial results in any component (full rollback). o Resource Pooling: To allow scaling (possibility for a server to work in the presence of many clients), the Application Server can "pool" resources. For example, instead of creating a new file for each client,

it can reuse the same file for many clients. In this case, whenever a component accesses a resource (e.g., a file, a thread, a database connection), the Application Server must make sure the resource is acquired by the component.

EJB server, container and object and the relation between them

Summary
The programs we write interact with the environment that executes them and their behavior is defined by it. The interface between the program and the RTE is bi-directional: the program invokes services provided by the container, and the container can send events or change the state of the process as it is running. The main responsibilities of an RTE include: 1. Process management 2. Storage management (memory, file, dbms) 3. Communication management 4. Hardware/Devices management 5. Security

There is a wide range of different RTEs (OSes such as Linux or Win32, JVM, applet container, servlet container, distributed RTEs) but the structure of the interface between a program and its container remains quite stable.

Object Oriented Design and Runtime Environments

Objectives
This lecture introduces basic concepts of Object Oriented Design (OOD) and explains how they are necessary to a proper understanding of modern Runtime Environments (RTE): What is a high-quality software system How to build high-quality systems Design vs. Implementation

The quality of software is measured by the following criteria: Usability/Usefulness Reliability Flexibility Affordability Availability

Which means our software must be: Useful - Must define Requirements Reliable - Must define and verify Invariants, Pre and Post Conditions Flexible, Affordable - Must use Encapsulation, Modularity, Reusable code Affordable - Must reuse existing components, must use efficient code

Design vs. Implementation

Software Design does not determine how code will be implemented but how it will be organized in components, how components will be interfaced logically (not physically), how components fulfill given requirements. Special languages have been defined to specify software at this level of abstraction (as opposed to programming languages). The most used one is called UML (Unified Modelling Language).

Modularity

The main objective of design is to preserve modularity among components. Several criteria exist to measure modularity: Plugability: possibility to add a component in a design independently of other requirements. Low coupling: low level of dependency among components. High cohesion: each component is only responsible for a single task.

An "Architecture" is a set of related components fulfilling together a given requirement. The RTE provides a set of architectures to manage processes and concurrency, storage and communication.

Processes and Scheduling

Process vs. Program
Process is defined as the basic unit of execution managed by Operating Systems. An Operating System, as any RTE, executes processes. The Operating System creates processes from programs, and manages their resources. Process resource. To keep running, a process requires at least the following resources:

The program (from which the process is created) Two blocks of memory: the stack and the heap Tables for resources of specific types: File handles, socket handles, IO handles, window handles. Control attributes: Execution state, Process relationship Processor state information: contents of registers, Program Counter (PC), Stack Pointer

The program context is defined as the Program Counter, the Registers and all the memory associated with the process. Practically, the program context contains all the information the OS needs to keep track of the state of a process during its execution.

Scheduling
Operating Systems have the capability to run more than one process simultaneously on a single computer. In most cases, an OS runs more processes than there are physical processors in hardware.

The OS is, therefore, responsible to interleave the execution of the active processes that is, the OS runs a process a little while, interrupts it and keeps track of its context in memory, then runs another process for a little while, and it keeps switching this way from process to process. The component of the OS responsible for sharing the CPU between processes is called the scheduler. On a single CPU system (called UP, for uni-processor), only a single process may be executing at any given time. On an SMP system (SMP - symmetric multi processor), where there may be several CPUs (or several cores inside a single CPU), a number of processes may be concurrently executed on different CPUs. To control the scheduling activities, the scheduler maintains a list of all active processes in the system, and at some point decides to switch from one executing process to another. There are two paradigms with which the scheduler may interact with the currently active processes: Non-Preemptive Scheduling Preemptive Scheduling

In non-preemptive systems, the process signals to the operating system that the process is ready to relinquish the CPU. The process does so with a special system call or when the process waits for an external event such as I/O. The scheduler will then select a different process to execute according to a scheduling policy (a method to pick the highest priority process among all processes that would like to run). Some examples for non-preemptive operating systems are Dos, Windows 3.1 and older versions of Mac OS. In preemptive systems, the scheduler may interrupt the executing process, and select a different process to execute. The interrupt is issued with the help of special timer hardware, which sends an interrupt at regular intervals, suspends the currently executing process and starts executing the scheduler. Another scenario where the scheduler may be executed is when the process passes control to the OS, which happens each time the process invokes a system call. Once the scheduler is running, it selects a new process to execute, according to the scheduling policy. All modern operating systems support preemptive scheduling.

Context Switch
A context switch is the operation that occurs when the OS switches between one executing process and the next one the scheduler chose, and may consume several milliseconds of processing time (that is, a context switch can take the time of several tens of thousands of simple CPU operations). The context switch is transparent to all processes, meaning a process is not able to tell it was preempted. One of the most expensive operation when executing a process, is accessing the memory. The reason is that accessing a cell of memory in RAM is much slower than executing a computation step on the CPU. To reduce this cost, the CPU maintains a

cache of frequently used memory cells in a small very-fast memory section, located on the CPU. When a memory cell is copied in the CPU cache, it can be read and written as fast as executing a single computation step on the CPU. When there is a context switch, all the memory cells stored in the CPU cache become invalid - since they refer to values from the old process. Therefore, the whole cache must be ``flushed'' (that is, all the cells must be written back to actual RAM and cleared). Actually, the CPU cache is not flushed, only a part of it which is called the TLB.
A context switch includes at least the following sub-steps: 1. Timer interrupt - suspend the currently executing process (the old process) and start the execution of the scheduler 2. Save the context of the process, so that it may be resumed later 3. Select the next process to execute (use a scheduling policy) 4. Retrieve the context of the next process to execute 5. Restore the state of the new process: restore the registers and the program counter. 6. Flush the CPU cache (as the new process has a new memory map, it can not use the cache which is filled with the data of the old process) 7. Resume the new process (start executing the code of the new process from the instruction that was interrupted)

The most costly operation (amortized over the lifetime of a process) is flushing the CPU cache, as it will degrade the performance of the new process. As a result we would like to lower the number of context switches our application requires. To our rescue comes the notion of the Thread.

Threads
Let us start by trying to design a video player, which should be able to play a compressed video stream.

The video player example

The player should take the following steps in playing the video: 1. Reading the video from disk 2. decompressing the video 3. decode the video and display on the screen

Now, if the video is very small, say several MegaBytes, we have no problem. We can read the entire video into memory, decompress it and then display it on the screen. However,

consider what will happen when the video is larger than the actual memory your computer has? and what if the video is downloaded from the Internet, and you want to start watching it before the download completes The Interleaving solution Consider the following solution: read some of the video data (from disk or the network), decompress it, decode it and display on the screen. Repeat until the video ends. Let us call this the "Interleaving" solution. There is a very fundamental difficulty when trying to implement the Interleaving solution; namely, it is very difficult to program, and is very error prone. The multi-process solution So, why not try a different approach? Since the task of playing the movie is easily decomposed into several independent tasks, why not have one process read the movie, another to decompress it, the next one to decode it and the final one which displays the movie on the screen. This makes our life as programmers much more easier, as we do not need to control the interleaving of the tasks by ourselves, as we had to in the Interleaving solution. However, another difficulty arises; how will all of these processes communicate with each other? The multi-threading solution If we could have only one process, which would be able to do all of these task simultaneously, our life would be much more tolerable. To our rescue comes the concept of Threads. In a single process, several threads my be executing concurrently. They share all the resources allocated to the process, and may communicate with each other using constructs which are built in most modern languages, thus making the programmer happier. In contrast to different processes, threads inside the same process share the same memory space (however, each thread has its own stack), the same opened files and access rights. Moreover, the cost of a context switch between threads in the same process is much lower than between processes or threads of different processes, as the CPU cache need not be flushed. To conclude our pet problem of the video player, we can now design a single process with several threads. One thread will read the video stream and place chunks of it in the chunk queue. Another thread will read the video data chunks from the chunk queue, decompress them and place them in the decompressed queue. The next thread takes decompressed chunks from the decompressed queue, decodes them into frames and queues them in the frame queue. The final thread takes frame after frame from the frame queue and displays them on screen.

Applications of Concurrency
Advantages Concurrency opens up design possibilities that would be impractical otherwise. Threads liberate you from the basic limitation of invoking a method and blocking, doing nothing, while waiting for reply. There are many reasons for using threads;

Reactive Programming: Some programs are required to do more than one thing at a time, performing each as a reactive response to some input. GUI applications. Its impossible to program a GUI without concurrent programming.

Availability: One of the most common design patterns for programs that are service providers is to have one thread as a gateway for incoming service consumers and a different thread for handling the requests of each one. For example, an FTP server will have a gateway thread to handle new clients connecting and a separate thread (per client) to deal with long file transfers.

Controllability: A thread can be suspended, resumed or stopped by another thread. This gives the designer and programmer a freedom that he would not be able to gain otherwise.

Simplified design: Software objects usually model real objects. In real life objects in many cases are acting independently and in parallel. Even if not modeling real objects, designing autonomous behavior is in many cases easier than designing sequential (interleaving between objects) one.

Simplified implementation: With threads it is simple to implement cases where a process needs to wait for external blocking event but perform some other vital processing in the meanwhile.

Parallelization: On multiprocessor machines, the operating system can execute threads truly concurrently with one another by allowing one thread to run on each CPU. Even if only one CPU exists, interleaving the execution paths avoid delays when it comes to part of the program that is busy doing some heavy computation or waiting for events.

Even if you choose not to use threads some services provided by modern RTE operate in a concurrent manner, hence leaving you no choice rather than mastering concurrent programming.

Limitations Benefits of concurrency should be weighed against its cost in resource consumption, efficiency and program complexity: Safety: when multiple threads share resources (this is the most common case) they need to use some synchronizations mechanisms to ensure that they

maintain consistent state . Failing to do so may lead to random locking, hard to debug inconsistencies. Liveness: In concurrent programming a thread may fail to be alive - an activity performed by a thread may stop for many reasons we will explore later. Non determinism: No two executions of a concurrent program need be identical. This makes multi-threaded program harder to understand, predict and debug. Context switching overhead: when a job performed in a thread is small enough the overhead of thread creation and context switching overrides the benefits. Request/Reply programming: Threads are not a good choice when an object actually needs to wait for a reply from another object in order to continue. In such cases synchronizing the activities costs extra time and introduce more complexity. Synchronization overhead: even with good design where multi-threading is necessary , synchronization can not be avoided. Synchronization constructs consume execution time and add complexity. In some (rare) cases where activity is self contained and sufficiently heavy it may be easier to encapsulate it in a different standalone program. A standalone program can still be accessed via system level services.

Abstract Object Models

In order to discuss programming issues related to threads, we introduce a formal model of concurrent objects. This model can be implemented in different manners in different RTEs or programming languages. We first discuss the abstract model, then we study its implementation in Java.

Static properties of the system

The structure of each object is described (normally via a class) in terms of internal attributes (state), connections to other objects, local (internal) methods, and methods for accepting messages from other objects (interface).

Object identity
New objects can be constructed at any time (subject to system resource constraints) by any other object (subject to access control). Once constructed, each object maintains a unique identity that persists over its lifetime.

Encapsulation
Objects have separation between their inside and outside parts. The internal state can be directly modified only by the object itself.

Communication between objects

Objects communicate only via message passing. Objects issue messages that trigger actions in other objects. The forms of these messages may range from simple procedural calls to those transported via arbitrary communication protocols.

Connections w.r.t. communication

One object can send messages to others if it knows their identities. Some models rely on the identities of the communication channel rather than or in addition to object identities. Two objects that share a communication channel may pass messages to each other through that channel without knowing each other's identity. Objects can perform only the following operations: Accept a message Update their internal state Send a message Create a new object

Computation Models
The three different computational models for our abstract object model.

1.

Sequential mappings

An ordinary general-purpose computer can be exploited so that this computer can pretend it is any object. This can be done by loading a description of the corresponding class into a virtual machine (VM). The VM can then construct a passive representation of an instance and then interpret the associated operations. It also extends to programs involving many objects of different classes, each loaded and instantiated as needed, by having the VM at all times record the identity of the object it is currently simulating. In other words, the VM is itself an object, although a very special one that can pretend it is any other object.

We know this model from single threaded applications.

2.

Active objects

In the active objects model, every object is autonomous and possibly as powerful as a sequential VM. Such a model is natural when different objects may reside on different machines, hence all message passing is performed via remote communication. This model also serves as an object-oriented view of most operating-system-level processes. Each process is independent of, and shares as few as possible resources with other processes.

A popular programming paradigm, so called agents, is an example of this model.

3.

Mixed models - Multithreading

This model fall between the two extremes of passive and active models. A concurrent VM may be composed of multiple threads of execution. Each thread acts in about the same way as a single sequential VM, but unlike pure active objects - all threads share access to the same set of passive representations. This model can simulate the first two models, but not vice versa. Thread-based concurrent object oriented models conceptually separate "normal" passive objects from active objects (threads). However, the passive objects typically show threadawareness not seen in sequential programming, e.g., by protecting themselves via locks.

Multithreading A multithreaded program contains two or more parts that can run concurrently. Each part of such a program is called a thread, and each thread defines a separate path of execution. Thus, multithreading is a specialized form of multitasking. There are two distinct types of multitasking: process-based and thread-based.process-based multitasking is the feature that allows your computer to run two or more programs concurrently. Forexample, process-based multitasking enables you to run the Java compiler at the same time that you are using a text editor. In process-based multitasking, a program is the smallest unit of code that can be dispatched by the scheduler.In a thread-based multitasking environment, the thread is the smallest unit of dispatchable code. This means that a single program can perform two or more tasks simultaneously. For instance, a text editor can format text at the same time that it is printing, as long as these two actions are being performed by two separate threads. Thus, process-based multitasking deals with the "big picture," and thread-based multitasking handles the details. Multitasking threads require less overhead than multitasking processes. Processes are heavyweight tasks that require their own separate address spaces. Interprocess communication is expensive and limited. Context switching from one process to another is also costly. Threads, on the other hand, are lightweight. They share the same address space and cooperatively share the same heavyweight process. Interthread communication is inexpensive, and context switching from one thread to the next is low cost. Multithreading enables you to write very efficient programs that make maximum use of the CPU, because idle time can be kept to a minimum. This is especially important for the interactive, networked environment in which Java operates, because idle time is common. For example, the transmission rate of data over a network is much slower than the rate at which the computer can process it. Even local file system resources are read and written at a much slower pace than they can be processed by the CPU. And, of course, user input is much slower than the computer. In a traditional, single-threaded environment, your program has to wait for each of these tasks to finish before it can proceed to the next oneeven though the CPU is sitting idle most of the time.Multithreading lets you gain access to this idle time and put it to good use. Single-threaded systems use an approach called an event loop with polling.In this model, a single thread of control runs in an infinite loop, polling a single event queue to decide what to do next. Once this polling mechanism returns with, say, a signal that a network file is ready to be read, then the event loop dispatches control to the appropriate event handler. Until this event handler returns, nothing else can happen in the system. This wastes CPU time. It can also result in one part of a program dominating the system and preventing any other events from being processed. In general, in a singled-threaded environment, when a thread blocks (that is, suspends execution) because it is waiting for some resource, the entire program stops running.

The benefit of multithreading is that the main loop/polling mechanism is eliminated. One thread can pause without stopping other parts of your program. For example, the idle time reated when a thread reads data from a network or waits for user input can be utilized elsewhere. Multithreading allows animation loops to sleep for a second between each frame without causing the whole system to pause. When a thread blocks in a Java program, only the single thread that is blocked pauses. All other threads continue to run. Threads exist in several states. A thread can be running. It can be ready to run as soon as it gets CPU time. A running thread can be suspended, which temporarily suspends its Activity. A suspended thread can then be resumed, allowing it to pick up where it left off. A thread can be blocked when waiting for a resource. At any time, a thread can be terminated, which halts its execution immediately. Once terminated, a thread cannot be resumed.

Thread Priorities
OS assigns each thread a priority that determines how that thread should be treated with respect to the others. Thread priorities are integers that specify the relative priority of one thread to another. As an absolute value, a priority is meaningless; a higher-priority thread doesn't run any faster than a lower-priority thread if it is the only thread running.Instead, a thread's priority is used to decide when to switch from one running thread to the next. This is called a context switch. The rules that determine when a context switch takes place are simple: A thread can voluntarily relinquish control. This is done by explicitly yielding, sleeping, or blocking on pending I/O. In this scenario, all other threads are examined, and the highest-priority thread that is ready to run is given the CPU. A thread can be preempted by a higher-priority thread. In this case, a lower-priority thread that does not yield the processor is simply preemptedno matter what it is doingby a higher-priority thread. Basically, as soon as a higher-priority thread wants to run, it does. This is called preemptive multitasking. In cases where two threads with the same priority are competing for CPU cycles, the situation is a bit complicated. For operating systems such as Windows 98, threads of equal priority are timesliced automatically in round-robin fashion. For other types of operating systems, such as Solaris 2.x, threads of equal priority must voluntarily yield control to their peers. If they don't, the other threads will not run.

Synchronization
Because multithreading introduces an asynchronous behavior to your programs, there must be a way for you to enforce synchronicity when you need it. For example, if you want two threads to communicate and share a complicated data structure, such as a linked list, you need some way to ensure that they don't conflict with each other. That is,you must prevent one thread from writing data while another thread is in the middle of reading it. You can think of a monitor as a very small box that can hold only one thread. Once a thread enters a monitor, all other threads must wait until that thread exits the monitor. In this way, a monitor can be used to protect a shared asset from being manipulated by more than one thread at a time.

The Thread Class and the Runnable Interface

Java's multithreading system is built upon the Thread class, its methods, and its companion interface, Runnable. Thread encapsulates a thread of execution. To create a new thread, your program will either extend Thread or implement the Runnable interface. The Thread class defines several methods that help manage threads. The ones that will be used in this chapter are shown here: Method getName getPriority isAlive Join Run Sleep Start M eaning O btain a thread's name. Obtain a thread's priority. Determine if a thread is still running. Wait for a thread to terminate. Entry point for the thread. S uspend a thread for a period of time. S tart a thread by calling its run method

The Main Thread

When a Java program starts up, one thread begins running immediately. This is usually called the main thread of your program, because it is the one that is executed when your program begins. The main thread is important for two reasons: It is the thread from which other "child" threads will be spawned. It must be the last thread to finish execution. When the main thread stops, your program terminates. Although the main thread is created automatically when your program is started, it can be controlled through a Thread object. To do so, you must obtain a reference to it by calling the method currentThread( ), which is a public static member of Thread. Its general form is shown here: static Thread currentThread( ) This method returns a reference to the thread in which it is called. Once you have a reference to the main thread, you can control it just like any other thread.

Let's begin by reviewing the following example: // Controlling the main Thread. class CurrentThreadDemo { public static void main(String args[]) { Thread t = Thread.currentThread(); System.out.println("Current thread: " + t); // change the name of the thread t.setName("My Thread"); System.out.println("After name change: " + t); try { for(int n = 5; n > 0; n--) { System.out.println(n); Thread.sleep(1000); } } catch (InterruptedException e) { System.out.println("Main thread interrupted"); } } }

In this program, a reference to the current thread (the main thread, in this case) is obtained by calling currentThread( ), and this reference is stored in the local variable t.Next, the program displays information about the thread. The program then calls setName( ) to change the internal name of the thread. Information about the thread is then redisplayed. Next, a loop counts down from five, pausing one second between each line. The pause is accomplished by the sleep( ) method. The argument to sleep( ) specifies the delay period in milliseconds. Notice the try/catch block around this loop. The sleep( ) method in Thread might throw an InterruptedException. This would happen if some other thread wanted to interrupt this sleeping one. This example just prints a message if it gets interrupted. In a real program, you would need to handle this differently. Here is the output generated by this program: OUTPUT Current thread: Thread[main,5,main] After name change: Thread[My Thread,5,main] 5 4 3 2 1 Notice the output produced when t is used as an argument to println( ). This displays, in order: the name of the thread, its priority, and the name of its group. By default, the name

of the main thread is main. Its priority is 5, which is the default value, and main is also the name of the group of threads to which this thread belongs. A thread group is a data structure that controls the state of a collection of threads as a whole. This process is managed by the particular run-time environment and is not discussed in detail here. After the name of the thread is changed, t is again output. This time, the new name of the thread is displayed. Let's look more closely at the methods defined by Thread that are used in the program. The sleep( ) method causes the thread from which it is called to suspend execution for the specified period of milliseconds. Its general form is shown here: static void sleep(long milliseconds) throws InterruptedException The number of milliseconds to suspend is specified in milliseconds. This method may throw an InterruptedException. The sleep( ) method has a second form, shown next, which allows you to specify the period in terms of milliseconds and nanoseconds: static void sleep(long milliseconds, int nanoseconds) throws InterruptedException This second form is useful only in environments that allow timing periods as short as nanoseconds.

Multithreading Sample 1 public class Mythread implements Runnable { String Sname; int age; public Mythread(String x,int y) { Sname=x; age=y; } public void run() { try{ for(int i=1;i<=age;i++) { System.out.println("Happy birthday "+Sname); Thread.sleep(2000); } }

catch(InterruptedException c) { System.out.print("Happy thread interrupted"); } } } class usethred { public static void main(String x[]) { Mythread M=new Mythread("Jeremy",12); Thread k=new Thread(M); k.start(); Thread t=Thread.currentThread(); try{ for(int i=1;i<=20;i++) { System.out.println("main thread executing"); Thread.sleep(500); } } catch(InterruptedException n) { System.out.println("Interrupted:"); } } } To create multiple threads you can create instances of thread e.g Thread k=new Thread(M); Thread w=new Thread(M); Thread q=new Thread(M); Start each thread at a time.

Thread Priorities
Thread priorities are used by the thread scheduler to decide when each thread should be allowed to run. In theory, higher-priority threads get more CPU time than lower-priority threads. In practice, the amount of CPU time that a thread gets often depends on several factors besides its priority. (For example, how an operating system implements multitasking can affect the relative availability of CPU time.) A higher-priority thread can also preempt a lower-priority one. For instance, when a lower-priority thread is running and a higher-priority thread resumes (from sleeping or waiting on I/O, for example), it will preempt the lower-priority thread.

For safety, threads that share the same priority should yield control once in a while. This ensures that all threads have a chance to run under a nonpreemptive operating system. In practice, even in nonpreemptive environments, most threads still get a chance to run, because most threads inevitably encounter some blocking situation, such as waiting for I/O. When this happens, the blocked thread is suspended and other threads can run. But,if you want smooth multithreaded execution, you are better off not relying on this. lso,some types of tasks are CPU-intensive. Such threads dominate the CPU. For these types of threads, you want to yield control occasionally, so that other threads can run.To set a thread's priority, use the setPriority( ) method, which is a member of Thread. This is its general form: final void setPriority(int level) Here, level specifies the new priority setting for the calling thread. The value of level must be within the range MIN_PRIORITY and MAX_PRIORITY. Currently, these values are 1 and 10, respectively. To return a thread to default priority, specify NORM_PRIORITY,which is currently 5. These priorities are defined as final variables within Thread. You can obtain the current priority setting by calling the getPriority( ) method of Thread, shown here: final int getPriority( )

Synchronization
When two or more threads need access to a shared resource, they need some way to ensure that the resource will be used by only one thread at a time. The process by which this is achieved is called synchronization. As you will see, Java provides unique,language-level support for it. Key to synchronization is the concept of the monitor (also called a semaphore). A monitor is an object that is used as a mutually exclusive lock, or mutex. Only one thread can own a monitor at a given time. When a thread acquires a lock, it is said to have entered the monitor. All other threads attempting to enter the locked monitor will be suspended until the first thread exits the monitor. These other threads are said to be waiting for the monitor. A thread that owns a monitor can reenter the same monitor if it so desires.

Using Synchronized Methods

Synchronization is easy in Java, because all objects have their own implicit monitor associated with them. To enter an object's monitor, just call a method that has been modified with the synchronized keyword. While a thread is inside a synchronized method, all other threads that try to call it (or any other synchronized method) on the same instance have to wait. To exit the

monitor and relinquish control of the object to the next waiting thread, the owner of the monitor simply returns from the synchronized method. Example // This program is not synchronized. class Callme { void call(String msg) { System.out.println(msg); try { Thread.sleep(2000); } catch(InterruptedException e) { System.out.println("Interrupted"); } } } class Caller implements Runnable { String msg; Callme target; Thread t; public Caller(Callme targ, String s) { target = targ; msg = s; t = new Thread(this); t.start(); } public void run() { target.call(msg); } } class Synch { public static void main(String args[]) { Callme ta = new Callme(); Caller ob1 = new Caller(ta, "Hello"); Caller ob2 = new Caller(ta, "Synchronized"); Caller ob3 = new Caller(ta, "World"); } } To fix the preceding program, you must serialize access to call( ). That is, you must restrict its access to only one thread at a time. To do this, you simply need to precede call( )'s definition with the keyword synchronized, class Callme { synchronized void call(String msg) {

Any time that you have a method, or group of methods, that manipulates the internal state of an object in a multithreaded situation, you should use the synchronized keyword to guard the state from race conditions.

Interthread communication
The Java RTE offers the programmer a single, simple, mechanism to facilitate guarded suspension and suspension in multi-threaded context in general, the Wait and Notify constructs.

Wait() and Notify()

Java offers threads the possibility of wait()-ing. That is, threads may wait until some other thread wakes them up. We use this mechanism to implement guarded methods. To be more concrete, Each object in Java has an associated wait set. Threads may enter the queue of an object, o, by invoking the wait() method of o. A thread wishing to wakeup threads from the queue of o may call o.notify() (or o.notifyAll()) to wakeup just one thread (resp. all of the threads) waiting in the wait set of o. To further clarify, the o.wait(), o.notify() and o.notifyAll() methods work as follows :

o.wait(). A wait invocation results in the following actions:

1. 2.
o.

The current thread (which invoked the method) is blocked. The JVM places the thread in the wait set associated with

o.notify(). A notify invocation results in the following actions:

1. An arbitrarily chosen thread T is removed by the JVM from the wait set associated with o (if such a thread exists). 2. Once unblocked, T is resumed from the point of its wait.

o.notifyAll(). A notifyAll works like notify except that the

steps occur (in effect, simultaneously) for all threads in the wait set of
o.

Interrupting Threads
Before discussing the actual use of wait() and notify . Since sometimes threads may need to stop waiting for a notify() to take place, for example when the program needs to exit, we need a mechanism to remove a thread from a waiting queue and let it resume its execution. Java implements this by sending a special exception to threads. When a thread is going to invoke a wait() method, it needs to take into account that an exception might be thrown. Therefore, code using waits usually looks like this:
4. try{ 5. 7. 9. someObject.wait(); //handle the interrupted wait. //do something if needed. 6. } catch(InterruptedExcpetion e) { 8. } finally {

The code below shows interthread communication using wait() and notify() import java.util.Scanner; class Processor { public synchronized void produce() throws InterruptedException { System.out.println("Producer thread running ...."); wait(); System.out.println("Resumed."); } public void consume() throws InterruptedException { Scanner scanner = new Scanner(System.in); Thread.sleep(2000); synchronized (this) { System.out.println("Waiting for return key."); scanner.nextLine(); System.out.println("Return key pressed."); notify(); Thread.sleep(5000); } } } class App { public static void main(String[] args) throws InterruptedException { final Processor pro = new Processor(); Thread t1 = new Thread(new Runnable() { public void run() { try { pro.produce(); } catch (InterruptedException e) { System.out.println("interrupted"); } } }); Thread t2 = new Thread(new Runnable() {

public void run() { try { pro.consume(); } catch (InterruptedException e) { System.out.println("inturrupted"); } } }); t1.start(); t2.start(); t1.join(); t2.join(); } }

Performance
In this lecture, we consider the following aspects of performance: Throughput - How much work can your program complete in a given time unit? For example, when writing an HTTP web server, we may be interested in how many pages per second can the server actually serve. Latency - How quickly can your program respond to events? Efficiency - What is the computational complexity of your program? How efficiently are you using the RTEs resources?

Throughput and Latency are not the same; consider the next example: Spl AirLines, a prominent airline company, operates an air link between Tel-Aviv and New York. Each day, two airplanes make the trip from TA to NY. Each airplane holds 250 passengers. Therefore, the throughput of the TA-NY line is 500 passengers per day. However, consider now Mr. Smith, who is interested in taking a flight from his house in TA to his parents in NY. To Mr. Smith, the latency of his flight is the time interval since he left his house in TA till he reached his parents' house in NY. So, we would like our programs to perform well, and then a bit better. In this course we do not presume to talk about all the issues related to performance, but one issue comes to mind in the context of the previous lectures: Liveness. Liveness issues arise mainly due to safety techniques.

Liveness
Liveness is a general property you would normally like to have in your programs. It is about your program actually doing something useful. Liveness can be summarized thus: "Something useful eventually happens within an activity". When can the progress of our program be stopped? Up to now we have seen several cases: Acquiring locks. Waiting on Objects. Waiting for I/O. Waiting for CPU time. Failures. Resource exhaustion.

Liveness issues can be divided to several categories. We will next consider livelock, deadlock and starvation.

LiveLock
Consider a narrow hallway, in which only one person may pass at a time. One day, Alice started walking down the hallway. Halfway through, Alice suddenly realized that Bob is approaching her from the other side. As Alice is a really nice person, Alice immediately decided to let Bob pass her by moving to her left. However, Bob is a zen-buddhist, and, at the same time, decided to let Alice pass him by moving to his right. But now, since they both moved to the same side of the hallway, they are still in each-other's way! Alice did not want to delay Bob any further, and decided to move to her right. Bob, at the same time, moved to his left. Again, they obstruct each other. Alice and Bob may still be moving to their left and right, but not one of them managed to pass. This is called a LiveLock: both Alice and Bob are doing something, potentially useful, but there is no global progress! In the context of computer programs, livelock issues may be caused by two threads canceling each others' actions, in a similar way to Alice and Bob. Livelock in programming usually arise from bad design, and can be solved by re-designing the system with livelock in mind. Consider the following abstract example:
1. public class LiveLock{ 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. } } System.out.println("done"); Thread t1 = new Thread(new Runnable(){ public void run(){ synchronized(lock){ while (state1 == false){ System.out.println("thread 1 waiting..."); state1 = true; state2 = false; lock.notifyAll(); try { lock.wait(); } catch (Exception ignored) {} public static void main(String [] a){ public static boolean state1 = false; public static boolean state2 = false; public static final Object lock = new Object();

22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. } }

}}); Thread t2 = new Thread(new Runnable(){ public void run(){ synchronized(lock){ while (state2 == false){ System.out.println("thread 2 waiting..."); state1 = false; state2 = true; lock.notifyAll(); try { lock.wait(); } catch (Exception ignored) {} } System.out.println("done"); } }}); t1.start(); t2.start();

DeadLock
We say that a deadlock occurs when several active objects are waiting for each other in a circular way.

This is a situation where a process is holding a resource R1 and waiting for a resource R2 being held by process B1 which in turn is waiting for a resource R1 to complete execution. Both the processes (A and B) cannot continue execution since they are waiting for each others resource; hence a deadlock i.e.
Requesting

A
Requesting

R1&R2 R1&R2

R1
Deadlock.

R2

To understand deadlock fully, it is useful to see it in action. The next example creates two classes, A and B, with methods foo( ) and bar( ), respectively, which pause briefly before trying to call a method in the other class. The main class, named Deadlock, creates an A and a B instance, and then starts a second thread to set up the deadlock condition. The foo( ) and bar( ) methods use sleep( ) as a way to force the deadlock condition to occur.
class A { synchronized void foo(B b) { String name = Thread.currentThread().getName(); System.out.println(name + " entered A.foo"); try { Thread.sleep(1000); } catch(Exception e) { System.out.println("A Interrupted"); } System.out.println(name + " trying to call B.last()"); b.last(); } synchronized void last() { System.out.println("Inside A.last"); } } class B { synchronized void bar(A a) { String name = Thread.currentThread().getName(); System.out.println(name + " entered B.bar"); try { Thread.sleep(1000); } catch(Exception e) { System.out.println("B Interrupted"); } System.out.println(name + " trying to call A.last()"); a.last(); } synchronized void last() { System.out.println("Inside A.last"); } } class Deadlock implements Runnable { A a = new A(); B b = new B(); Deadlock() { Thread.currentThread().setName("MainThread"); Thread t = new Thread(this, "RacingThread"); t.start(); a.foo(b); // get lock on a in this thread.

System.out.println("Back in main thread"); } public void run() { b.bar(a); // get lock on b in other thread. System.out.println("Back in other thread"); } public static void main(String args[]) { new Deadlock(); } }

Take a look at the next example:

1. class BadCell { 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. }}); } } public BadCell(long v) { value_ = v; private long value_;

// Do not use

synchronized long getValue() { return value_; } synchronized void setValue(long v) { value_ = v; } synchronized void swap(BadCell other) { long t = getValue(); long v = other.getValue(); setValue(v); other.setValue(t);

public static void main(String [] args){ final BadCell a = new BadCell(1); final BadCell b = new BadCell(2); Thread t1 = new Thread(new Runnable(){ public void run(){ a.swap(b);

25. 26. 27. 28. 29. 30. 31. 32. } }

Thread t2 = new Thread(new Runnable(){ public void run(){ b.swap(a); }}); t1.start(); t2.start();

Consider the following turn of events: Thread 1

Thread 2 acquire lock for b on entering

b.swapValue(a)

acquire lock for a on entering

a.swapValue(b)

execute t = getValue() successfully (since already held) block waiting for lock of b on entering v =
other.getValue()

execute t = getValue() successfully (since already held) block waiting for lock of a on entering v =
other.getValue()

Both of our threads are now blocked, and we have a circular wait. No thread will ever break out of it. We can deal with the problem by making sure both of the threads try to lock the Cell objects in the same order. This is called resource ordering. If all the locks in our system are grabbed (and released) in the same order, we can avoid deadlocks. Consider the following improved version, in which the Cell objects themselves enforce resource ordering:
1. class Cell { 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. else other.doSwapValue(this); m.identityHashCode(other)) this.doSwapValue(other); public void swapValue(Cell other) { if (other == this) return; else if (System.identityHashCode(this) < Syste // alias check synchronized void setValue(long v) { value_ = v; } synchronized long getValue() { return value_; } private long value_;

15. 16. 17. 18. 19. 20. 21. 22. 23. 24. }

} protected synchronized void doSwapValue(Cell other) { // same as original public version: long t = getValue(); long v = other.getValue(); setValue(v); other.setValue(t); }

The Dining Philosophers Problem

The dining philosophers problem captures almost all aspects of liveness in concurrent environments. Consider the following image:

We have N philosophers, each with a plate of spaghetti in front of him. There are also N forks, such that between any two philosophers there is exactly one fork. Now, as philosophers go, they like to ponder the meaning of life, the universe and everything (if you wonder, the answer is 42). However, once in a while, a philosopher might become really hungry (pondering is hard work, after all). To eat, a philosopher must grab both forks to his left and right. He then eats his full, cleans the forks, returns them to the table and resumes pondering. Assuming our philosophers are active objects (threads) in Java, consider the following implementation of a philosopher:

1. /* The forks philosophers use. */ 2. class Fork { } 3. 4. /* The class Philosopher. This is an abstract class, and must be ex tended */ 5. public abstract class Philosopher implements Runnable{ 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. public void run(){ while (true){ ponder(); if (isHungry()) eat(); protected abstract void eat(); protected abstract boolean isHungry(); protected abstract void ponder(); } public Philosopher(Fork l, Fork r){ left_ = l; right_ = r; /* the forks each philosopher has */ protected final Fork left_; protected final Fork right_;

27. 28. 29. } }

Also consider the following implementation of Confucius. Take in mind that Confucius is a bit old, hence he needs to rest after taking a fork
1. public class Confucius extends Philosopher{ 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. } } } protected void eat() { synchronized(left_) { Thread.currentThread().yield(); synchronized(right_){ System.out.println(id_ + " is eating"); try { Thread.currentThread().sleep(MAX_SLEEP); } catch (Exception ignored){} } protected void ponder() { System.out.println(id_ + " is pondering"); try { Thread.currentThread().sleep(MAX_SLEEP); } catch (Exception ignored){} } public Confucius(Fork l, Fork r, int id){ super(l, r); id_ = id; private static final int MAX_SLEEP = 1000; private int id_;

30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49. 50. 51. 52. 53. 54. 55. 56. 57. 58. 59. 60. 61. 62. }

protected boolean isHungry() { System.out.println(id_ +" is hungry"); return true; }

public static void main(String [] args){ if (args.length != 1){ System.out.println("Usage: java Confucius num_of_instances"); System.exit(1); } int num = Integer.parseInt(args[0]); if (num < 3) { System.out.println("you need at least 3 philosophers"); System.exit(1); } Fork [] forkArr = new Fork[num]; Confucius [] confArr = new Confucius[num]; for (int i=0; i<num; i++) forkArr[i] = new Fork(); for (int i=0; i<num; i++) confArr[i] = new Confucius(forkArr[i % num], forkArr[(i + 1) % num], i); for (int i=0; i<num; i++) new Thread(confArr[i]).start(); }

This is what you will get when running the code with 3 Confuciuses:
0 is pondering

1 2 0 1 2

is is is is is

pondering pondering hungry hungry hungry

And thats it. Nothing more will ever be printed! There is now a deadlock: each grabbed his left fork, and will wait forever for his right fork.

Solutions
Many solutions to deadlock exists (most of them are out of the scope of this course). However we would like to state the following 3 possible solutions to the dining philosophers problem. 1. Break symmetry - make sure one Philosopher grabs the right fork first. 2. Resource ordering - make sure forks are grabbed in some global order (rather than left and right of each philosopher). 3. Request all resources atomically - Each thread asks for all its needed resources atomically. This is done by adding another lock (a semaphore initiated to 1) known to all the philosophers, which they must grab before trying to grab any fork and release once they have grabbed both forks. This solution is not recommended in general as it requires another lock, managed by the programmer (rather than released by the scope) and requires book-keeping, or careful implementation.

Resource ordering 1. protected void eat() { 2. 3. 4. 5. 6. 7. 8. 9. 10. } if (System.identityHashCode(big) < System.identityHashCode(small)) { big = right_; small = left_; Fork big = left_; Fork small = right_;

11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. } } } synchronized(big) { Thread.currentThread().yield(); synchronized(small){ System.out.println(id_ + " is eating"); try { Thread.currentThread().sleep(MAX_SLEEP); } catch (Exception ignored){}

The code above makes sure there are no circular waits (but has other problems we will discuss later). Let us prove that. Assume we have philosophers, . Denote by the left and right forks of philosopher . Note that . Assume, towards contradiction, that there is a circular wait. Since all philosophers must be part of this circular wait, there are only two options; either there is a circular wait orient clockwise or counterclockwise. Assume the wait is oriented clockwise (the proof is the same for the counter clockwise orientation). It follows that is waiting for a fork is holding, is waiting for a fork is holding, , is waiting for a fork is holding. This means that is waiting for , is waiting for , ., is waiting for . However, since each philosopher first grabs the bigger fork, it implies that, as each philosopher holds its right fork, . Using the equality we get that, for instance, , which is a contradiction

Starvation
Consider the previous example of our dining philosophers. Now, assume that both Plato and Confucius sit at our table, next to each other, running the code we saw earlier (grab the bigger fork first). Now, as Plato is much younger than Confucius (about 100 years younger), Plato is much quicker. Plato ponders quickly and eats even more quickly. This, in turn, results in the fact the Confucius rarely (if at all) succeeds in grabbing the fork he is sharing with Plato. Poor Confucius is Starving. So, Starvation is about several threads all waiting for a shared resource, which is repeatedly available. However, at least one thread never (or rarely) gets its hands on the shared resource. Identifying starvation in our code is hard at best, and solving starvation is usually done at design time. One solution to the starvation problem is to use synchronization primitives which support ordering. The standard synchronized construct in Java does not guarantee any ordering on blocked threads; e.g., consider a thread, , blocked on some lock, . Then comes along another thread, , which also blocks on . When becomes available, Java does not guarantee the will be woken first.

In contrast, the Semaphore class does support ordering. This property of the semaphore is called fairness. Consider the following solution for the dining philosophers problem, which solves the starvation problem:
1. import java.util.concurrent.Semaphore; 2. 3. class Fork { 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. protected void eat() { Fork big = left_; Fork small = right_; } protected void ponder() { System.out.println(id_ + " is pondering"); try { Thread.currentThread().sleep(MAX_SLEEP); } catch (Exception ignored){} } public Confucius(Fork l, Fork r, int id){ super(l, r); id_ = id; private static final int MAX_SLEEP = 1000; private int id_; public class Confucius extends Philosopher{ } public void grab() throws InterruptedException { lock_.acquire(); } public void release() { lock_.release(); } /* a fair semaphore with one permit */ private Semaphore lock_ = new Semaphore(1, true);

34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49. 50. 51. 52. 53. 54. 55. 56. 57. 58. 59. 60. 61. 62. 63. 64. 65. 66. 67. 68. 69. } protected boolean isHungry() { System.out.println(id_ +" is hungry"); return true; } /* release forks, in the opposite order!!! */ small.release(); big.release(); System.out.println(id_ + " is eating"); try { Thread.currentThread().sleep(MAX_SLEEP); } catch (Exception ignored){} } /* grab the forks */ try { big.grab(); } catch (InterruptedException ignored) { return; } Thread.currentThread().yield(); try { small.grab(); } catch (InterruptedException ignored) { big.release(); return; } if (System.identityHashCode(big) < System.identityHashCode(small)) { big = right_; small = left_;

70. 71. 72. 73. 74. nstances"); 75. 76. 77. 78. 79. 80. 81. 82. 83. 84. 85. 86. 87. 88. 89. 90. 91. 92. 93. 94. 95. 96. 97. } } for (int i=0; i<num; i++) new Thread(confArr[i]).start(); for (int i=0; i<num; i++) forkArr[i] = new Fork(); for (int i=0; i<num; i++) confArr[i] = new Confucius(forkArr[i % num], forkArr[(i + 1) % num], i); Fork [] forkArr = new Fork[num]; Confucius [] confArr = new Confucius[num]; } if (num < 3) { System.out.println("you need at least 3 philosophers"); System.exit(1); int num = Integer.parseInt(args[0]); } System.exit(1); if (args.length != 1){ System.out.println("Usage: java Confucius num_of_i public static void main(String [] args){

This lecture will explain why (and how) we should cleanly terminate our threads.

Why we should terminate our threads cleanly

The first answer which comes to mind is that there is no other way! Consider the following simple code, which does almost nothing, but, nevertheless, never ends:

1. class NeverEnds{ 2. public static void main(String [] args) 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. } } { /* create a new Runnable Object, which loops * and does nothing */ Runnable r = new Runnable(){ public void run(){ while(true) Thread.currentThread().yield(); }}; Thread t = new Thread(r); t.start();

When executing the code above, (at least) two threads are created: the first thread is the main thread, which starts executing the main function. The second thread is t. Now, the first thread terminates almost immediately. However, t never terminates, but keeps on looping forever. This, in turn, implies that the program will never terminate, as a process only terminates when all of its threads have terminated (a Runnable based thread terminates when the run() method does). The designers of Java, in their infinite wisdom (which we can never hope to fully grasp), devised no interface to shutting down threads from the outside (actually, they first devised such an interface, and then realized their mistake and repented by deprecating it). That is, there is no way to invoke something along the following lines from our main thread:
t.stop()

Why is that? The reason is simple. Assume Java did support such an interface, which allows one thread, to stop another thread, . Further assume that while calls , holds some locks and is in the middle of changing the internal state of some object protected by these locks. If these locks will not be released when is stopped, you can bet there will be a deadlock in the near future. On the flip side, if these locks are immediately released, and is stopped, the object which was working on will remain in an inconsistent state!

Why not introduce a new Exception?

Why not indeed? Let us introduce a new exception, and call it ThreadDeath. We will also make sure that once someone calls the stop() method of a thread, the ThreadDeath will be thrown to the thread, interrupting the thread's current execution. This, in theory, seems nice. But there are two main complications, which render such a solution impractical, and very very dangerous:

The ThreadDeath exception can be thrown anywhere! We will have to closely examine all of our code, to take care of any cleanup which needs to be done prior to terminating the thread uncleanly. The ThreadDeath exception can be thrown even while we are cleaning after a previous ThreadDeath exception! Bummer!

For more information and discussion on why there is no preemptive stop() mechanism in Java and no ThreadDeath exception, read here.

How to terminate threads cleanly

The way to terminate threads is to use a variable in the Thread object that indicates that someone has requested this thread to stop. The thread object, in its run() method must check this variable at regular intervals - when it is safe to do so - and determine whether it accepts to stop (return from run) or not. In other words, thread cancellation is cooperative: one thread asks another one to stop, the receiving thread can either accept or reject the request. Consider the following simple implementation of a worker thread:
1. class Worker implements Runnable { 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. _ && IcanStopNow()) 14. 15. } break; public void run() { while (true){ doSomeWork(); synchronized(this){ if (shouldStop public synchronized void stop() { should Stop_ = true; } public Worker() { } private boolean shouldStop_ = false;

16. 17. ;)"); 18. 19. } }

} System.out.println("stopping

How to interrupt long waits

Threads may sometime be stopped for a long period of time. For example, threads may: Blocked waiting to enter synchronize block
Sleep() wait() join()

Perform IO

Sometimes, when we want to tell a thread that it needs to stop, may not notice our request for some time, since 's execution is currently stopped (since initiated one of the activities above). To solve this problem, Java introduces a simple internal flag to every thread in the system. This flag is called the interrupted status. This flag may be set by any other thread, by calling the interrupt() method of a different thread. This flag may be also cleared, and its condition may be checked with the Thread.isInterrupted() method. The rules governing the interrupted flag are simple: 1. If a thread,

, is in a

Sleep()

or wait() or join() calls T.interrupt(), then an . Moreover, the will be cleared.

and another thread, interrupted status of 2. If

InterruptedException will be thrown to

is currently performing IO, using an interruptible 's interrupt status will

channel, the channel will be closed, thrown to .

be set, and a ClosedByInterruptException will be

3. If

is waiting for a Selector to return (we will learn more 's

on Selectors in the second part of the course), return. 4. If none of the previous conditions holds then status will be set.

interrupt status will be set and the call to the selector will 's interrupt

Usually, the interrupt mechanism is used in conjunction with the methods mentioned above, e.g., we need to use the following code:

1. . 2. . 3. Worker r = new Worker(); 4. Thread t = new Thread(r); 5. . 6. . 7. r.stop(); 8. t.interrupt();

Read Dealing with Interrupted Exception by Brian Goetz (May 2006) for a good explanation of how to deal with the interrupt side.
Page last modified on 14 February 2008, 12:18 by elhadad powered by CourseWiki 2.4 generated in 0.00047002 sec.

Concurrency Summary
Concurrent Runtime Environment
A concurrent RTE manages several active objects and schedules their execution. The objective is to share the CPU among a set of active objects. We introduced an abstract model of computation where each active object is modeled by an object: The object is fully encapsulated (only the object can access its internal state) The object is characterized by a unique Identity The object holds an internal state (set of fields of various types) The object knows of other objects through communication channels (reference to other objects to whom it can send and from whom it can receive messages)

The object runtime behavior is: To receive messages React to known messages (the object interface) by executing code which can: create a new object, update the internal state of the object or send messages to known objects.

This model can be summarized as: (State, Identity, Interface, Connections). In this abstract model, objects fully control their state and there are no shared resources among the objects. When we want to map this model to a concrete runtime environment, we can translate the abstract model to 3 models: Sequential model: only one active object simulates the interleaved execution of the abstract active objects. Fully concurrent model: several active objects exchange data through message exchange in completely separate memory spaces (process model with inter-process communication channels) Hybrid model: several active objects (threads) share a set of passive object

We focused in this chapter on the problems introduced by the hybrid model. The root problem is that shared memory accessed by several active objects can become "corrupted" because of problems of read/write conflicts.

Safety and Liveness

The problems we must face when executing code in a hybrid RTE are of 2 types: Safety: ensure that read/write conflicts on the state of shared passive objects do not lead to loss of correctness (invariant corruption, preconditions do not hold) Liveness: ensure that additional protection code (in the form of locks or preventive actions) do not lead to a loss of progress in the overall computation.

Correctness Criteria
We used the mechanism of invivariant, pre and post condition to capture what it means for an object to perform a correct computation. In addition, we introduced a global constraint: the result of a concurrent computation must be one of the results that can be reached through a sequential computation over the same set of objects. Code is "thread safe" if it can be run in a concurrent (thread-based) RTE and insure correct results.

How to write Safe Code

The mechanisms rely on: Use stateless objects (they are safe by definition) Use immutable objects (they are safe by definition) Use fully-synchronized objects Confinement: make sure an object is only accessible by a single thread at a time (because it is a local variable and is not passed to any other threads).

Safety and Reusability

Making reusable classes is hard. Making reusable thread safe classes is even harder. There are no universal valid rules for making classes that can serve as useful superclass for all possible extensions. Two well-known design rules are: Avoid premature optimization Encapsulate design decisions

Both of these rules can be surprisingly hard to follow. One of the aspects of OO developing is iterative refactoring and cleaning up. This is a good time to apply those rules.

Synchronization

In order to enforce safety, the programmer relies on RTE services (part of the scheduler) that make sure that at times, only one thread at a time can access an object. When another thread attempts to access this object, the thread becomes blocked and remains in this state until the object it attempts to lock becomes free again. This is implemented in Java using the synchronized monitor mechanism, or variants such as semaphore or locks.

Guarded Methods
In a sequential RTE, when an object invokes a method and the precondition does not hold, there is no other behavior that makes sense besides throwing an exception. In concurrent RTE, there is an alternative: the caller can wait until the precondition becomes true. It makes sense to wait because we hope that another thread may change the value of the object and make the precondition true. To make threads collaborate in this way, we rely on the wait/notifyAll mechanism in Java. Rules to remember: A thread must hold the object O synchronized lock before calling O.wait() or O.notifyAll(). A thread must test the precondition in a loop When a thread is blocked by a wait on O.wait(), it releases the synchronized lock on O.

Composition and Safety

When an object is composed of sub-objects which we must all lock before performing a transaction, there are several methods to achieve safe locking: Client-side locking (the client locks all the sub-objects then performs the transaction) [This is bad in general since clients are not reliable] Transaction method: define a synchronized method to perform the transaction on the whole container. [This is bad because it requires the definition of new transaction methods for each case and leads to high contention on the container] Use a generic apply procedure mechanism (such as the Apply Procedure code discussed in class) [Still leads to high contention on the container] Snapshot method: copy the container into a confined variable and perform the transaction without locking [Problematic if the transaction requires side-effects]

Optimistic try and fail (such as the versioned iterator method discussed in class) [Problematic if the transaction has apply-exactly-once semantics]

Liveness
Liveness is the property of the computation system that "progress is made". All methods insuring safety reduce liveness to some extent (because locks cause threads to be blocked instead of making progress). An extreme case of liveness is a deadlock: this is a situation where a set of threads depend on each other while trying to acquire a shared set of resources. A deadlock can occur whenever threads can block while attempting to lock resources and keep the resources they have already locked.

Deadlock Prevention Methods

Resource ordering: make sure all threads lock the shared resources always in the same order Container lock: group all shared resources in a container and lock the container before attempting to lock any of the resources. This is bad in general as the container becomes a source of systematic contention. Optimistic try and fail with exponential backoff: try to lock the resources one by one (in any order), when a resource cannot be locked (use a nonblocking tryAcquire method), release all locked resources, wait some time, and try again. If all threads perform the same method, this can lead to a problem of livelock, but if the threads wait more and more each time before they retry, this reduces contention on the resources and reduces the risk of failure at each try. Reorganize the design so that threads do not wait for the same set of resources in a blocked manner.

Starvation
Starvation can occur when a high-priority thread, or a fast low-priority thread performs a lot of activity and prevents another thread from ever getting access to the resources it requires.

Starvation can be avoided by using fair-scheduler methods. Take into account that insuring fairness increases the cost of the synchronization method. For example, the Java Semaphore class can be used in a fair method.