User Interface - part 2

The role of the operating system at the interface

Last time we saw different ways in which commands can be sent to the operating system to start a program running. Now we are going to think about this some more. We will have to restrict our study to the two most common interfaces, the CLI (command line interface) and the GUI (graphical user interface).

Terminals

Our discussion of terminals will mostly consist of keyboard and display (and mouse or similar pointing device). Operating systems can regard terminals as two (or three) devices: the screen for output and the keyboard (and mouse) for input. Originally the only link between the two devices was the echo, whereby what you typed was displayed on the screen. We saw that the advent of the bitmapped display and the GUI has meant a very close relationship has to be maintained between what appears on the screen and the commands the user is giving.

Now we have to see that the terminal is not just another device (or pair of devices) attached to the computer. The main reason is that because the terminal is where input comes from the user, and a lot of this input is control information (what should the system start running next). Also the user is going to want to do things which normal sources of input won't, e.g. stopping the currently running program for some reason (time for lunch, it seems to be stuck in an infinite loop, etc).

Terminal I/O

We may be tempted to think that all we require is a simple connection whereby the operating system takes takes input from the terminal and passes it on to the program the user is interacting with. This program then performs the task indicated by the user and passes update information about the display to the operating system which shows the changes.

Problems with the simple view

User Interface Management System

It becomes clear that we need to have a carefully designed part of the operating system which is going to manage all of this. This is known as the User Interface Management System (UIMS).

Setting conventions

It is the UIMS job to catch all of the input to the system and to direct it to the correct program making any necessary translations. Output must be handled similarly, ensuring that the correct output ends up in the correct window, for example, or the correct file. We come back to how this is done later in the course.

By providing this layer of translation between users and programs the system can ensure consistency (at one level anyway). The same command can be used to resize windows or stop programs regardless of the program running.

Graphical User Interfaces

Several of the problems we saw above have been made worse with the advent of GUIs. Of course the solution of enforcing a uniform interface via a well defined API, Application Program Interface, is essential with GUIs such as the Macintosh and Windows desktops.

Compare two windowing systems, say the Macintosh and an X window manager or Windows 95. In particular how are they similar.

Make a list of the information the UIMS must keep track of in order to perform these tasks.

There are all sorts of interesting questions of implementation of such systems.

Covering and uncovering windows

How should the UIMS handle windows being covered and uncovered by other windows? Why?

One method would be to store off-screen bitmaps corresponding to all windows. When a window is uncovered the resulting chunk of the window is dumped to the window. This has the disadvantage of requiring a lot of memory, especially if the window shows 24-bit per pixel data. A window of 500x600 pixels takes up approximately a megabyte of memory. There will likely be several such windows in a normal display. Of course not all windows need to store bitmaps at this level.

A common solution is to force the application programs to update their own windows. The interface manager (or window manager part of it) sends the program a request to update the window. This request will include information about which part of the window which needs to be redisplayed.

Usually the bitmapped solution is much faster (the Macintosh uses it for menus for example. Menus popping up and down is very fast compared to the redraws of normal windows.) Some systems allow both methods, whereby a maximum amount of memory for storing the bitmaps of windows is given to the system. When this memory is all used up the update method has to be used.

Resizing windows

Another interesting question: which is the best way to do something like resizing a window. Should the program itself do this, should the program catch the request and pass it on to the system to do, should the system do it or should the system catch the request and ask the program to do it?

Let's briefly look at one of the consequences of the system resizing the window.

If the system is tracking the resize operation of the mouse and doesn't send information to the application program until the user has finished the window contents can only be updated after the resizing is completed. So the resize must appear as a dotted rectangle on the screen. Of course this method is also used because redrawing the contents as the window is resized takes a lot of computing power.

Other controls or widgets requiring feedback are similar. If the application wants live action scrolling, either it has to handle the scrolling, or the system has to report back live, where the scroll thumb currently is.

So we see that there is a lot of work for our UIMS to do.

Job Control and Scripting Languages

Ignoring pointing and clicking for a while we will return to typing our commands. What are known as command line interfaces are not only potentially very flexible they are also easy to automate. Files containing sequences of commands can be selected by the user interface management system (or some lower level system as in UNIX) as the input for the system. The inability of GUI systems to be controlled in this way is one of the major criticisms made of them. This criticism is now being rebuffed by adding scripting languages to GUIs. We will look at AppleScript soon.

Choosing the right words

We have to be careful what we call our programs, especially our system utilities which people will want to use to do frequent tasks. We have already commented on names such as "vi" for visual editor, which no one seems to be able to guess. UNIX is by far the best example of what not to do when choosing names. Other good examples are:

Another good one is how to link precompiled object files together to make an executable program. No "ln" does something completely different. "cc" will do the linking for you.

The usual UNIXophile's excuse is that most shells provide an alias system which means that commands can be renamed to anything the user wants. While this is true, it certainly does not help the new user get used to the system.

Another defence is even more extreme, "you can write your own shell". This means if you don't like the user interface that comes with your system software, make one you do like. Once again, an interesting exercise for those who have plenty of time and want to become UNIX programming experts, just like you will do with assignment 3.

More human interface

Some study has gone in to making the choice of words natural, at least for English speaking users. One nice study used a human computer expert as a translator for new users of a mail system. A new user typed a command thinking they were interacting with an intelligent interface. The command was read and translated into the equivalent mail command by the expert user if it appeared that the request could be automatically parsed and the response was passed back to the new user.

Gradually the system was extended to be able to cope with more of the new user's input. Over a series of interative designs the hit rate of commands typed and then understood by the system rose from 7% to 76%. That is pretty impressive.

We may wonder why someone hasn't done the same sort of thing for file or process management. One reason is that people who want to move files around or start and stop processes are supposed to be knowledgable users and we don't have to do anything for them. (Even though other studies show that systems which are easy for new users to learn tend to be good for experienced users as well.)

The other, and possibly more important reason, is that it takes time and money. Some companies are more prepared to spend this money, Apple prides itself on carefully testing and revising its interface, and it still gets things wrong.

Natural language UNIX

Another more recent approach trying to do the same thing was published in 1994. There is a relatively new acronym NLIOS (Natural Language Interface to Operating Systems).

We want users to be able to give commands in natural language. The first step is to work out which aspects of the system should be accessible with this method, then the hard bit is working out what the allowable natural language instructions for the actions should be. This approach has advantages similar to the last method. People new to the system should find it simpler to direct. Another indirect advantage is that it is possible to implement a very similar interface to a variety of operating systems. The same natural language commands can be used on different systems (within limitations).

Scripting

It seems naturally obvious that we should use the same language to control the system from files as we do from the keyboard. The simple reason is that the user doesn't have to learn two different languages. This is going to cause problems with trying to write command files for GUIs.

Making decisions and looping

Of course when we are working at the terminal we can make decisions according to the output we receive. If our commands are to be executed from a stored file, we need to be able to include some form of conditional execution.

if the compile fails then stop

else link

The most important conditional requirement is some way of knowing if commands succeed or fail. It would be nice if we could get more information than this, such as an indication of why a command failed, but at least this much is essential.

To be able to make decisions on results we need some sort of conditional execution in our language - which from now on we will refer to as our JCL.

It is very simple to make JCLs which open command files and start executing the commands in sequential order. All we have to do is to redirect the normal command interpreter to look at the file rather than at the terminal. Now that we have a UIMS this is easy to do.

It is not much harder to include rudimentary conditional execution. A comparison is made on the basis of which the next command is executed or the command interpreter skips some of the command file until a specific keyword is hit.

Especially when working with files (very common when interacting with the operating system, reflect on how many operating systems have the acronym DOS in their name) it is useful to be able to perform a certain action on a large number of objects.

This usually requires the ability to loop, returning to an earlier point in the command file. This is usually different from what can be done with keyboard commands. Some interactive JCLs keep a history of recently used commands. In this way even interactive sessions can implement looping.

Expressions, Variables

In order to have conditional execution we need to be able to get values for operations and if we are going to do that we might as well enable our JCL to work with these values in "natural" expressions.

By "natural" I mean the ability to concatenate strings (e.g. to add file names to pathnames) and do arithmetic on numbers (e.g. to count the number of files in a directory).

It is certainly not true that all JCLs allow us to do arithmetic for example, but some do. More commonly we want to do simple string manipulation and comparisons.

Since we now have comparisons and expressions it is sensible to include some form of variable for our control language. Once again string (or straight text) variables are common.

More recently some of the newer trends in language design have made themselves felt in new JCLs. AppleScript for example is an object oriented scripting language in which new objects and classes can be defined.

Getting info into programs

One of the difficult decisions is how do we get information into our running programs. It isn't too difficult to start a program up from our JCL, maybe we can even pass some parameters to the new program.

There are different methods for passing parameters to our programs, ranging in flexibility.

Of course this is one solution. If our program can accept input from a file, we can tell it what file, via a parameter passed from our command file to the program.

Parameters

One of the most important requirements for our JCL is the ability for command files to accept parameters. We usually want to perform the same actions on different files for example, without having to rewrite the command file.

bash shell

By about now you should be getting at least partially familiar with the bash shell. You will already have discovered that it contains all of the requirements we have just specified. I will must mention a couple of things about UNIX which makes its implementation reasonably easy.

AppleScript example

You would probably find it quite interesting to work out a way in which a scripting language could be added to an existing GUI. What the Apple designers did was to add a new way of communicating with programs known as Apple Events. New programs can catch and respond to Apple Events and send Apple Events to other programs. We will come back to look at these again later in the course.

In AppleScript each program which wants to be scriptable needs to publish commands and object types (this means to make them available to the part of the operating system which deals with scripts).

So what happens with AppleScript is that the script is interpreted by an extension to the operating system. This extension understands some commands and knows what commands the applications understand (they exist in the resource fork of the application). The extension passes commands on to applications via Apple Events. The result goes in the opposite direction.

Rather than trying to control events by means of the mouse and menus, AppleScript treats the whole scriptable world (those applications which are scriptable) as an object oriented environment. In such an environment the important thing is sending messages to objects. The objects themselves know what to do with the messages. Frequently the objects are the application programs, however there are a huge variety of object classes, ranging from documents to windows to drawings.

Another interesting feature of AppleScript is that it is an underlying language. The same programs can be written in a variety of different dialects. The only dialect I have seen is the English dialect, but there are two Japanese dialects, a French dialect and different types of programming language dialects (to make it look like another programming language). This is pretty neat. The running programs are identical. In fact you can read the same program in another dialect if you have the other dialect available on your system.

As I have already suggested, the variables which AppleScript can handle are far more sophisticated.

Back to the lecture index

On to the next lecture