Processes

Remember what a process is. A process is a running instance of a program (where a program is an executable file stored in the computer's file system). We make a distinction between a "program" and a "process" because a program, like Notepad, can have several instances of itself running at any given time. Each such instance is a different process (but each instance can be said to be the same program). Notice that a process is to a program much like an object is to a class.

Process isolation

In Operating Systems you learn that each process runs in its own virtual memory space. Virtual memory spaces were invented so that each process is purposely walled off from every other process. You do not want a bug in one process (say an instance of Word) to cause the crash of another process (say an instance of Firefox). You also do not want one process (say an instance of Firefox) to be able to look at, and read, the memory of another process (say Word) and report back over the Internet what you are typing in your Word documents. So for stability and security reasons, processes are strictly isolated from each other (each in its own virtual memory space).

if processes are strictly isolated from each other, they cannot communicate, or cooperate, with each other. That is too severe a restriction (think of how useful copy and paste is, among many other ways in which programs communicate with each other). So operating systems provide very tightly controlled mechanisms for processes to communicate (and therefore cooperate) with each other. These mechanisms are referred to as Inter-process Communication.

If you would like to review the idea of a process, here are explanations from two online operating systems textbooks.

Inter-process Communication

Inter-process Communication (IPC) is when one process passes a piece of data (a message) to another process. ALL forms of IPC involve the operating system kernel acting as an intermediary in the passing of the message (in order to guarantee the stability and security of the whole system). In fact, every form of IPC involves the kernel copying a segment of data (a buffer) from the virtual memory space of the sending process into the virtual memory space of the receiving process.

There are seven simple mechanisms for IPC (and many complicated ones). The simple IPC mechanisms are,

  1. command-line arguments
  2. environment variables
  3. program exit code (return value)
  4. files
  5. I/O redirection
  6. pipes
  7. sockets

Command-line arguments can only be used by a parent process to pass (in a one-way direction) a message to a child process (a child process is a process that is started by the parent process). Also, this message must be fairly small and it can only be sent at the time the child process is created. So command-line arguments are a fairly restrictive form of IPC (but they get used a lot).

Environment variables are similar to command-line arguments. They are fairly small messages, they can only be passed in one direction (from parent to descendant processes) and the messages must be sent at the time a child process is created. The most significant difference between environment variables and command-line arguments is that an environment variable is also inherited by every descendant of the original child process. So an environment variable can be used to send a message to every descendant of the parent process. There are times when this feature can be useful.

A program's exit code is a single integer value that can be used by a child process to send a (very small) message back to its parent process. This form of communication can only take place when the child terminates. Exit codes are used mostly as error codes that let the parent know something about the success or failure of a child process.

Files can be used by any two processes to communicate with each other (more specifically, any two processes that have access to a shared file system). In this sense, files are a very general form of IPC (the messages can be as large as any file, the messages can be passed in both directions, and the communicating processes need not even be running on the same computer or at the same time), but it is a very slow form of IPC.

I/O redirection can be used by a parent to communicate with a child while the child is running (not just when the child starts up). When the parent creates the child, the parent can have the operating system redirect one of the parent's output streams to the child's standard input stream, and then redirect the child's standard output stream to one of the parent's input streams. The amount of data that can be communicated this way is unlimited and the data can be communicated in both directions, from parent to child and from child to parent. This is a very common and useful form of IPC.

Pipes are a way for two sibling processes to communicate. Two processes are siblings if they were created by the same parent processes. A pipe must be created by the parent process and it must be created at the time the two child processes are created. But otherwise, pipes are a very general form of communication. The most common example of a parent process creating a pipe between two child processes is when we use a command-line shell to run two programs and we connect the two programs on the command-line with the pipe ('|') symbol. The shell process is the parent process and the two programs on the command-line become the child processes. The shell process has the operating system create a pipe object, then redirect the standard output of the first child process (the child that can send messages) to the input of the pipe, then redirect the standard input of the second child process (the child that will receive messages) to the output from the pipe. We say that pipes are a form of "stream" communication, in the sense that once the pipe is created, the two child processes can pass an unlimited stream of bytes through the (one-directional) pipe. Pipes are very fast (almost as fast as writing and reading from physical memory). The biggest restriction on pipes is that the two processes must be running on the same computer.

Sockets are a way of providing stream communication between any two processes running on the same or different computers (but connected together by a network). Sockets are straightforward to use, they are very versatile, fairly fast, and they are now ubiquitous (these days, every language running on every operating system provides sockets).

The most important advanced form of IPC is shared memory. Shared memory is when the operating system arranges for two processes (running on the same computer) to share a physical page frame in their virtual memory page tables. This means that the two processes can both read and write to the physical page frame. This is a fast, bidirectional, symmetrical, form of IPC but it causes many of the same problems that shared memory causes with threads (race conditions). Shared memory is the basis for many other forms of IPC.

Creating Processes

An important fact to remember is that every process running in an operating system was created by the operating system kernel at the request of some other process. When one process requests that the kernel create another process, the requesting process is called the parent process and the newly created process is called a child process.

Every operating system has a function (a "system call") that a process can call to request that a child process be created. In the Linux operating system, this system call is the fork() function. In the Windows operating system this system call is the CreateProcess() function.

When a parent process creates a child process, the parent has the ability to communicate with that child process by using, as we mentioned above, command-line parameters, environment variables, or I/O redirection. It is very common for the parent process to create the child process as a helper process to do work for the parent process. Sometimes the parent and child work in parallel (to speed up the parent's work) and sometimes the parent delegates work to the child in a manner much like one method calling another method.

The operating system kernel keeps track of the parent/child relationship for all running processes. It is very instructive for you to observe this relationship on your computer. To see the parent/child relationship on Windows, you need to download and run the ProcessExplorer.exe program.

This program does not need to be installed. You download a zip file, unzip it, and then just run the executable ProcessExplorer.exe program that will be in the unzipped folder. To see the parent/child relationships, be sure to select the menu item "View -> Show Process Tree". Since a child process can become a parent to another process, the parent/child relationship is in fact a family tree.

You can also use the program "System Informer", which is an open source alternative to ProcessExplorer. You can download either an installer for the program or a zip file (portable) distribution of the program.

Java ProcessBuilder class

If a programming language wants to let programmers write code to create a process, then the programming language needs to create an interface to the fork() function on Linux and the CreateProcess() function on Windows. For example, the Java language has two interfaces to these functions. Java's java.lang.Runtime class has the exec() method for creating processes on either Linux or Windows. But the Runtime class is not very versatile, so Java created a better way for Java programs to create processes. Java's java.lang.ProcessBuilder class is a very modern, sophisticated way for Java programs to create processes on either Linux or Windows.

In the creating_processes folder there are Java programs that create processes. Compile and run those programs. They show how a Java program can start up other programs. Notice that some of these child processes are given command-line parameters when they are started. For example, on Windows the notepad.exe editor is started and it is told to open a file. Also, the Windows explorer.exe program is told to open the folder C:\Windows\System32. This demonstrates a simple example of a parent process starting up a child processes and communicating to the child processes a bit of (inter-process) information.

To see the parent/child relationships created by these examples on Windows, run the ProcessExplorer.exe program and be sure to select the menu item "View -> Show Process Tree". You really should download ProcessExplorer (or System Informer, or both) and use it on your computer!

Here are some references for using the Java Process API.

Creating Processes in JShell

An effective way to understand the Java code that creates processes is to execute the code using the Java Shell program. The Java Shell (jshell.exe) lets us run one line of Java code at a time. This gives us a way to experiment with individual lines of Java code, to see the effect each line of code has.

Open a command-prompt window and on the command-line type the following command.

   > jshell

You should see a response that looks something like this.

|  Welcome to JShell -- Version 11.0.7
|  For an introduction type: /help intro

jshell>

At the jshell prompt, type the following three lines of Java code.

    jshell> var pb = new ProcessBuilder("C:\\Windows\\system32\\charmap.exe")
    jshell> var p = pb.start()
    jshell> p

You should see an instance of the "Character Map" program running on your desktop.

The first line of code uses a ProcessBuilder constructor to create a ProcessBuilder object. The String parameter to the constructor should be the name of a program on your computer. The second line of code uses the start() method in the ProcesBuilder object to do two things. First, the start() method asks the operating system (either Linux or Windows) to create a running process from the program named in the ProcessBuilder constructor (using either the fork() function on Linux, or the CreateProcess() function on Windows). Second, if the process was started successfully, then the start() method creates a Java Process object that represents the operating system's running process.

The toString() method of the Process object p tells you the "process ID number" (PID) of the running instance the charmap.exe program.

Exit (close) the "Character Map" program and then type the following line of code in your JShell session.

    jshell> p

The toString() method of p tells you that the charmap.exe program is no longer running.

Once we have created a ProcessBuilder object, we can use it several times to create multiple instances (processes) of the same program.

    jshell> var p2 = pb.start()
    jshell> var p3 = pb.start()
    jshell> var p4 = pb.start()

Use the Windows "Task Manager" program to verify that there are three instances of the charmap.exe program.

We can use a Process object to terminate the process that it represents. As you execute the following lines, you should see the instances of charmap.exe terminate (close).

    jshell> p2.destroy()
    jshell> p3.destroy()
    jshell> p4.destroy()

You can use a Process object to wait for the termination of the process that the object represents. Type the following two line of code into your jshell prompt.

    jshell> var p5 = pb.start()
    jshell> pb5.waitFor()

After you enter the second line, jshell should "freeze", because the jshell.exe process is waiting for the charmap.exe process to terminate. Use your mouse to terminate (close) the charmap.exe process. That should free up the jshell prompt.

Type the following two lines of code into the JShell prompt. (Important questions: What is the purpose of \\? Why not use \?)

    jshell> var pb = new ProcessBuilder("C:\\Windows\\system32\\bob.exe")
    jshell> var p = pb.start()

There is no program named bob.exe in the folder C:\Windows\system32, so we get a "file not found" exception. But notice that the exception did not happen when we created the ProcessBuilder object. The exception happened when we tried to start the process. The ProcessBuilder constructor does not check the validity of the filename it is given. The filename is not checked until we try to start the process.

Type the following two lines of code into the JShell prompt.

    jshell> var pb = new ProcessBuilder("C:\\Windows\\explorer.exe", "C:\\Users")
    jshell> var p = pb.start()
    jshell> pb.command()

The String parameters to the ProcessBuilder constructor are essentially a Windows command-line. The pb.command() method returns a String array holding the parts of the command-line, the program name and then its command-line arguments. The above ProcessBuilder object is equivalent to opening a command-prompt window and typing the following command-line (try it).

    > explorer  C:\Users

Open another command-prompt window and on the command-line start another JShell session.

   > jshell

When the jshell prompt appears, copy-and-paste the following block of code into the prompt. JShell allows you to copy several lines of code at a time. It will execute each line of code and then give you another prompt.

var pb = new ProcessBuilder("cmd.exe", "/C", "dir")
pb.directory(new java.io.File("C:\\Users"))
var home = System.getenv("USERPROFILE")
var desktop = home + "\\Desktop\\"
pb.redirectOutput(new java.io.File(desktop + "temp.txt"))
var p = pb.start()

After this code executes, you should have a new file, called temp.txt, on your computer's desktop. The contents of the file is a directory listing of your computer's "C:\Users" folder.

Type the following additional line of code into your jshell prompt.

p

The toString() method of p tells you that the cmd.exe program is no longer running.

To remind ourselves what code we have typed into the jshell prompt, use the JShell list command.

    jshell> /list

If you want to see a list of all the JShell commands, use the following command, where <TAB> means tap the Tab key on the keyboard.

    jshell> /<TAB>

The ProcessBuilder class is an example of the Builder design pattern. This is a software design pattern that is used in a lot in modern versions of Java. Like the Factory pattern, the Builder pattern replaces the use of constructors. If you look at the Javadoc for the Process class, you will see that the class does not have a public constructor. The only way to create a Process object is the call the start() method in a ProcessBuilder object. The ProcessBuilder class controls how Process objects are constructed and configured.

The ProcessBuilder class is also an example of a fluent interface. In a fluent interface, the methods in a class return this so that we can "chain" method calls. This allows us to write compact code like the following example.

Process p = new ProcessBuilder("cmd.exe", "/C", "dir")
                .redirectOutput(new File("output.txt"))
                .redirectError(new File("errors.txt"))
                .directory(new File("C:\\Users"))
                .start();

Each method call returns this which is a reference to the ProcessBuilder object that we are constructing. The final start() method builds and returns the Process object that we want. (In this example, the ProcessBuilder object that we constructed is immediately garbage collected since we did not create a reference to it.)

The old fashion alternative to a Builder pattern with a fluent interface is a class with a lot of (overloaded) constructors that take many parameters along with a lot of set methods that mutate the object. This old fashion, traditional way to write code produces code that is harder to read than modern code using a builder class with a fluent interface. Almost all modern Java classes use builders with a fluent interface.

Simple IPC

As we mentioned above, the simplest forms of Inter-process Communication are:

  1. command-line arguments
  2. environment variables
  3. program exit code (return value)
  4. files
  5. I/O redirection
  6. pipes
  7. sockets

In this section we will briefly go over the first four of these techniques. I/O redirection and pipes will be explained in the document about Streams. Sockets will be explained in the documents about network programming.

Command-line arguments

As we saw above, the creation of a process is based on a command-line string. This is true for the Java ProcessBuilder and Runtime classes and also the Windows CreateProcess() function (but not the Linux fork() function). The command-line string mimics what we would type in a command-prompt window to run a program. A command-line to run a program starts with the name of the program's executable file, followed by command-line arguments. Every process running on a computer was started from a command-line and every process has access to its command-line arguments.

A process's command-line arguments are collected into an array of String and that array is a parameter to the program's main() method. Traditionally, that array argument is named args (but it can be given any name you want).

    public static void main(Strin[] args)

Since the command-line arguments are entries in an array, it is easy to access them. Here is a complete Java program that prints out all of its command-line arguments.

public class CommandLineArguments
{
   public static void main(String[] args)
   {
      System.out.println("There were " + args.length + " command line arguments.");
      System.out.println("They are:");

      for (int i = 0; i < args.length; ++i)
      {
         System.out.println( args[i] );
      }
   }
}

Command-line arguments are a simple form of one way IPC between a parent process and a child process. The parent process "sends" the command-line arguments when it calls the ProcessBuilder constructor. The child process "receives" the command-line arguments when it accesses its args array.

We have said several times that every process is created from a command-line and has command-line arguments (not just programs run from a command-prompt window). Here is a way to verify this on a Windows computer.

Run Window's "Task Manager" program. There are several ways to start this program.

  1. Hold down the Ctrl and Shift keys and then strike the Esc key.
  2. Click on the Windows Start menu, search for the "Run" item, type in the program name taskmgr.exe, and click on OK.
  3. Open a command-prompt window and at the prompt type the program name, taskmgr.exe.
  4. Go to the folder C:\Windows\System32, find the file taskmgr.exe and double click on it.

When you have Task Manager running, click on the "Details" tab. Then right-click on the "Name" column heading and click on "Select columns". Scroll down the list in the pop-up window and look for the item called "Command line" (the items are not in alphabetical order). Click on the box next to this item to put a check-mark in the box. Click on the "OK" button. In the Task Manager window you should now see a list of all the processes you are running and there should be a "Command line" column for each process. This lets you see the actual command-line string used to create each of those processes.

If you start a program by double-clicking on a "shortcut icon" (for example, a shortcut icon on your desktop, or an item in Window's Start Menu), then the command-line (and command-line arguments) for the process is stored in the icon. Right click on a shortcut icon and then click on the "Properties" item from the pop-up menu. You should see a tabbed pop-up window with a tab called "Shortcut". In that tab is a textbox labeled "Target" and in that textbox is the process's command-line along with any command-line arguments. Since this is a textbox, you can modify the command-line and its arguments to change what the shortcut does when you double-click on it.

Environment variables

The operating system provides every process with a data structure called the environment. This is a data structure of key-value pairs, where each key and value is a string. A key in the environment is called an environment variable.

The operating system provides functions that a process can call to query and modify its environment. A process can retrieve or modify the value of any environment variable, a process can create new environment variables, and a process can delete an existing environment variable (the four basic CRUD operations).

When a parent process creates a child process, the child process inherits a copy of the parent's (current) environment. This makes the environment a simple form of IPC. The parent process can send small (string) messages to the child process as the values of environment variables. This IPC is one directional, from parent to child, and it must be done when the child process is created.

The operating system maintains a master copy of an environment data structure which can be used as a default environment for some processes (usually process that are created by the operating system itself). On a Windows computer, you can access and modify this master copy of the environment, but it is usually a good idea to never do that.

Most programming languages provide functions that processes can use to query and modify their environment. Java has the getenv(String) method in the java.lang.System class for reading the value of an environment variable (but Java does not have a method for changing the value of an environment variable).

You can see your computer's default set of environment variables by opening a command-prompt window and at the prompt typing the following simple command-line.

    > set

The set command will print out a list of all the current environment variables and their values. If you want to see the value of just one environment variable, put its name as a command-line argument after the set command. For example the prompt environment variable determines what your command-line prompt looks like.

    > set prompt

You can use the set command to change the value of an environment variable. For example, the following command will change the prompt in the current command-prompt window (but not in other command-prompt windows).

    > set prompt=$T$G

You can create a new environment variable by defining it using the set command.

    > set hello=there
    > set

The second command displays all the environment variables so that you can see that the new hello variable is there.

You can remove an environment variable by setting it value to be "empty".

    > set hello=
    > set

Notice that the hello variable is no longer in the environment.

You can use the ProcessExplorer.exe or SystemInformer.exe programs to observe the environment data structure of any running process. Start up either ProcessExplorer.exe or SystemInformer.exe. Right click on any process and choose the "Properties" item from the pop-up menu. You should see a tabbed pop-up window with one tab labeled "Environment". Click on that tab and you will see a list of all of the environment variables that the process inherited from its parent process. On a Windows computer, most of the processes that you look at will have pretty much the same environment variables. Windows programs do not usually make much use of environment variables (other than the standard ones set by the operating system). On Unix/Linux computers, lots of programs make use of lots of environment variables.

Exit code

Just as a method will have a return value, a process will have an exit code. Methods let you declare the type of their return value, but all processes have the same type for their exit code, an integer.

The purpose of the exit code is much more specific than a return value from a method. For the most part, a process uses its exit code to send a simple "success or failure" message to its parent process. Many programs are designed to return the value 0 to mean a "successful" completion of the process. Any non-zero exit code means that the process "failed" somehow, and the value of the exit code can be considered an "error code" that denotes the failure mode. (Notice how this kind of mimics the C definition that the integer value 0 is false and any non-zero integer is true, but here the exit code 0 means "success" and any non-zero exit code means "failure".)

A Java program can set its exit code using the static method java.lang.System.exit(int).

A Java program that is the parent of a process can use the java.lang.Process.exitValue() method to retrieve the exit code from its child process after the child process has terminated (the child process does not need to be a Java program).

Files

A very simple form if IPC is for a parent process to write some data into a file and then expect its child process to open the file and read the data. The parent process might use some other form of IPC (an environment variable or a command-line argument) to let the child know the name of the file, or the parent and child programs might have the name of the file coded directly into them.

Configuration files

A common example of this is "configuration files". These are files with a specific name that a program is coded to open and retrieve data from. The data in a configuration file is used to set the initial configuration of the program's state. A parent process can open its child's configuration file and update values in the file to configure how the child process initially behaves.

Properties files

Java provides built in support for simple configuration files. Java calls these files properties files and they are implemented by the java.util.Properties class. A properties file is a text file with the extension .properties. The contents of the file are key-value pairs, one pair per line (but a properties file can also be written as an XML file).

Suppose you have a simple properties file called myApp.properties and it contains the following three key-value pairs.

myArgument1 = true
myArgument2 = 5
myArgument3 = 3.14

Here is a brief outline of how you could read this properties file.

    boolean p1 = false; // default value
    int p2 = 0;         // default value
    double p3 = 0.0;    // default value
    final Properties properties = new Properties();
    try (final var fis = new FileInputStream(new File("myApp.properties")))
    {
       properties.load(fis);
       final String op1 = properties.getProperty("myArgument1");
       final String op2 = properties.getProperty("myArgument2");
       final String op2 = properties.getProperty("myArgument3");
       if (op1 != null)     {p1 = Boolean.parseBoolen(op1);}
       if (op2 != null) try {p2 = Integer.parseInt(op2);}   catch (NumberFormatException e){}
       if (op2 != null) try {p2 = Double.parseDouble(op2);} catch (NumberFormatException e){}
    }
    catch (FileNotFoundException e)
    {
       // Ignore the configuration file.
    }
    catch (IOException e)
    {
       e.printStackTrace(System.err);
       System.exit(-1);
    }

Notice how every key has a String value. Just because a key's value is "true" does not mean that the value is a boolean. Each key's value needs to be parsed to a value with the appropriate Java type. In this simple code, if a key's value doesn't parse (the parser throws an exception) then this code just ignores that key. This is usually a good idea, since we do not want to crash a program just because there is a syntax error in its config file. It would be a good idea though to log some kind of simple error message.

Also notice that we ignore the FileNotFoundException. It is usually not considered an error to not have a configuration file. If the config file is absent, then the program should proceed with all its default values. But if there is an error while reading the config file, then that is a problem that needs to be logged and may be a good reason to halt the program.

Layered configuration

When a parent process creates a child process, it is common for the parent to "configure" the child process. The parent process wants to send information to the child process that specifies how the child should behave. The configuration information passed from the parent to the child is an example of IPC.

A parent process needing to configure a child process is such a common design pattern that there is a (semi) standard hierarchy of configuration information.

The child process will have certain variables in its code that determine aspects of the program's behavior. Changing the values of these variables changes how the program acts. Configuring the program is then a matter of giving these variables values when the program starts executing. Most programs set the values of these kinds of variables by going through a hierarchy of changes.

First, the program gives every configuration variable a default value.

Second, the program looks for a configuration file and uses values from the config file to update is configuration variables.

Third, the program looks for environment variables that can update the program's configuration variables.

Fourth, the program looks for command-line arguments that can update the program's configuration variables.

A simple example of this hierarchy is in the folder streams_and_processes/2_simple_ipc/simple_ipc_examples/example 3.