TACO and PCs
TACO: An object oriented system for PC's running Linux, Windows/NT, OS-9, LynxOS or VxWorks
A.Götz, W-D.Klotz, P.Mäkijärvi, J.Meyer, E.Taurel, J.Quick*
ESRF, BP 220, 38043 Grenoble, France
*HartRAO, PO Box 443, 1740 Krugersdorp, South Africa
Abstract
TACO is (yet another) modern control system based on distributed objects. Developed at the ESRF it has been successfully applied to control the accelerators, beamlines and a radio telescope. In TACO control elements are seen as objects in a distributed environment. Actions are implemented in classes. Classes can be in C (using Objects in C) or in C++. TACO has been ported to PC's running Linux, Windows/NT, OS-9, LynxOS and VxWorks. This paper will briefly describe TACO, the ports undertaken recently and how TACO can be used on PC's to implement front-ends, back-ends, servers and consoles.
Introduction
TACO is an object oriented distributed control system developed to control the ESRF accelerators and beamlines. It was originally developed to run on OS-9 (on 680x0 CPUs on VME) and Unix (HP-UX and SunOS). With TACO it is possible to encapsulate any control functions in a class and make it accessible as an object in network space. Network communication is handled by the system libraries of TACO as is name resolution and a database. All network communication is based on SUN's remote procedure call, ONC/RPC's. A few years ago TACO was ported to PC's running Windows 3.1. Although this proved feasible it was eventually abandoned due to the limiting 16 bit memory model and the instability of the Windows 3.1 operating system. In the last year however with the arrival of new stable 32 bit operating systems on PC's e.g. Windows/NT and Linux, it was decided to re investigate the PC world. It was found that the PC world has matured considerably and that it is possible to run TACO in a pure PC environment. The PC world has distinct advantages which make it an interesting development and runtime platform. This paper presents the result of this recent work (carried out at the European Synchrotron Radiation Facility, Grenoble, France, and the Hartebeesthoek Radio Astronomy Institute, Krugersdorp, South Africa). It will present TACO and how it can be used on a PC running one (or more) of the following OS's - Linux, Windows/NT, OS-9 and LynxOS. Finally it will conclude with a chapter on Do-It-Yourself (DIY) TACO on PC's.
TACO
TACO refers to the ESRF control system. TACO is used to control accelerators, beamlines, a radio telescope and laboratory measuring equipment. TACO is written in C and uses the SUN ONC/RPC and XDR data format for doing network communication. Control logic is implemented in classes. Classes are written in C (using a locally developed methodology called Objects in C, OIC) or in C++. Various papers, describe TACO in detail. In what follows a brief description will be given of the principal elements of TACO.
TACO consists of the following basic components :
1. Manager - the manager is the single entry point into TACO, it starting/stopping the manager starts/stops TACO, all servers and clients connect first to TACO to find out where the other principal system servers are e.g. database server.
2. Static Database - is a simple database which is used to store parameters which are required at startup and the dynamic network address of all control objects. It consists of a database and a server. The database can be either ndbm or RTDB (a real-time database originally from HP).
3. Devices - are the encapsulated control objects. Each device belongs to a class. The class implements the device characteristics. Every device has a name. A three field naming convention has been adopted ("domain/family/member"). Devices are usually exported onto the network i.e. their network address is exported to the static database. Secure device access is implemented in the root class. Six levels of security are implemented.
4. Device Servers - create and serve devices so that clients can access them via the device server API (DSAPI). Device servers can serve one or many devices of one or many classes. The exact configuration is defined in the static database. The majority of device servers run on the frontend computers (close to the hardware) but because anything (even applications) can be encapsulated in a device they are found everywhere in the system.
5. Data Collectors - a mixture of pollers, shared memory and servers for storing the results of commands from a configurable list of devices. The data collectors provide TACO with a dynamic database of the state of the system. The data collectors are accessed via a similar object-oriented DSAPI as the real devices.
6. Applications - are the clients of the control system. They can be generic or specific and are often graphical. A typical generic application is signal monitoring while a typical specific application is an orbit correction.
7. History Database - keeps a long term history of the state of the system. Data are stored as signals in a non-object oriented relational database (Oracle). The history database is accessed via SQL, C or a spreadsheet.
Refer to figure 1 for a diagram of TACO in its simplest form. TACO is very scalable and can be used for small (100 control points) to medium (10000 control points) to large (100000) control systems. So far TACO has scaled to all control problems it has been applied to so far without breakdown occurring.
Linux
The Linux operating system (cf. the Linux home page http://sunsite.unc.edu/mdw/linux.html) is a complete rewrite of the Unix kernel. It is licensed under the GNU General Public Licence and is available free of charge via Internet or on CD-ROM via one of the many distributors. Originally developed by Linus Torvalds for Intel 386 platforms it has been ported to a number of other processors (e.g. Alpha, Sparc, 680x0, PowerPC). Because of its low cost and efficient implementation it is rapidly turning into the freeware development platform par excellence. The Linux kernel is actively developed by a large number of volunteers scattered around the globe according to the so-called "open development model". While everyone is free to develop anything a small group of kernel programmers decide what goes into each release. The result is (believe it or not) a stable, efficient kernel which supports a wide-range of hardware devices.
TACO was ported to Linux a year ago when one of the authors (AG) spent a sabbatical year in South Africa at the Hartebeesthoek Radio Astronomy Institute (HartRAO). HartRAO had decided to replace the existing control system to control the 26m radio telescope. Linux had been chosen as operating system for cost reasons (PC's running Linux are cheaper than workstations running proprietary Unix or other operating systems), because it has all the necessary development tools (e.g. GCC) and features (e.g. Posix compliance) and last but not least because Linux was already in house.
The Linux port was done using Linux kernel 1.2.12 based on the Slackware release. It took two weeks to port TACO to Linux. The only problems encountered were minor problems involving differences in XDR function prototypes and in the implementation of the GNU version of the ndbm database which was used for the static database.
Linux proved to be sufficiently well adapted to be used as frontend, central server and operator console. All basic elements of TACO except for the data collector and history database were ported to Linux. With all the basic elements of TACO ported it is possible to run a pure PC-based TACO control system with Linux.
Network performance on Linux is quite impressive. A typical control message sent via rpc takes 2 ms between two 486s equipped with 8 bit Ethernet cards.
Predictable real-time performance is a problem because Linux was not designed for real-time performance. Interrupt routines and drivers are not interruptible and can therefore interfere in an unpredictable manner with scheduling of user tasks. It was found that the network and X11 were the main problems. Under a heavy load network (listing a externally mounted NFS file system for example) a jitter of up to 500+ ms was measured. Fortunately solutions to this problem do exist. It is possible to replace the standard scheduler with a Posix real-time scheduler (available from Dot4). This ensures better response for user processes. An even better solution is to implement the time critical tasks in the interrupt routine of a driver which is triggered by an external signal. Using this solution and playing around with the interrupt levels it was possible to guarantee response times under 4ms for external interrupt even during times of heavy network load. The minimum response time is 15 m s but with no processing involved. At HartRAO a driver which responds to an external interrupt is used to acquire data at 100 Hz and to update the telescope co-ordinates at 10 Hz. Linux can also be configured to run diskless (without an internal disk) - a useful configuration for frontend controllers.
Linux is rich in graphics utilities and offers most of the well-known freeware graphics packages (e.g. Tcl/Tk, Python, XForms) as well as some of the commercial packages (e.g. Motif, IDL).
To conclude Linux has proved to be a very attractive development and runtime platform. It is an attractive choice for an ultra low-budget control system and a serious contender as an alternative to proprietary Unix workstations due to its low cost, hardware support and the number of software packages which have been ported to Linux.
Microsoft Windows
The TACO-API libraries (without the manager and static database) have been ported to Windows/3.1 and Windows/NT. The Windows/3.1 port has been started about 3 years ago. The difficulties caused by the 16bit environment of Windows/3.1 in comparison to the 32bit UNIX environment, and the unstable behaviour of Windows/3.1 itself were the main reasons why this port never reached a stable release. With the first release of Windows/NT the idea was taken up again to port the TACO-API to one of Microsoft’s platforms. We now have a tested and stable release of the API that runs on NT 3.51. The part of the API that has been ported and tested includes both the client- and server- part of the library and allows TACO programmer's to develop control applications (clients) and device servers on Windows/NT thereby allowing Windows/NT workstations and servers to be integrated into our control environment which is still primarily based on UNIX and OS9. This integration is completely transparent. In a heterogeneous TACO environment there is no difference from a device server’s point of view between a client on UNIX, Window/NT, OS9 or whatever and vice versa.
The development tools used for this port were :
Þ NetManage’s(cf. http://www.netmanage.com) TCP/IP communication stack and Sun’s ONC RPC/XDR Software Developer’s Kit (NEWT-SDK) Version 4.5. This SDK is intended to assist in developing programs that interface with NetManage’s TCP/IP DLL, which is included in Chameleon32NFS TCP/IP for Windows/NT/95.
Þ NEWT provides compatibility by offering support for standard industry interfaces. At the application level, NEWT supports Berkeley’s 4.3BSD UNIX and Windows Sockets for Microsoft Win32S/NT/95 standard socket interfaces. At the driver level NEWT uses the industry standard NDIS interface for all communication with network adapters. NEWT is implemented as a DLL. The SDK completes this with import libraries, include files, and ‘rpcgen’, the standard RPC interface definition language compiler.
Þ The C compiler used was Microsoft’s Visual C++ 4.0, which is shipped with a powerful integrated development environment called ‘Developer Studio’.
The TACO/NT-API is implemented as static and/or dynamic link library. The port of the UNIX source code was rather straightforward. It was decided to base the ported libraries on Windows Sockets instead of BSD sockets. Since the Windows Sockets API is slightly different at the lowest level from the BSD sockets API, this decision forced us to emulate some ONC RPC calls. The most important being the svc_run() call that never returns but stays in an endless loop, in which it polls sockets- and file- descriptors through the select() call.
The TACO-API supports both UDP and TCP remote procedure calls. We found, that the NT port was more sensitive to UDP errors than the UNIX version. With UDP and in case of a transmission error client applications crash. We therefore use TCP only for RPC's to NT.
Today we are experimenting with different software packages on Windows/NT to write client applications that use the TACO-API for device access :
Þ Microsoft Visual C++ and MFC class libraries,
Þ Microsoft OLE2 custom controls (also known as .ocx files) as OLE extensions for device objects,
Þ Microsoft Visual Basic 4.0 using ocx-controls,
Þ Labview,
Þ Tcl/Tk scripts,
Þ CORBA objects,
Þ JAVA applications/applets.
The main advantages in using Windows/NT for control tasks from our point of view come from the possibility of integrating standard desktop software packages, like Microsoft Office, and multi media tools, e.g. sound, images and hypertext, into control applications. Windows/NT also makes it very easy to connect to any peripheral device from the desktop industry. In this function the NT workstation is best situated on the console level. It can be used as a front-end to provide access to hardware for which drivers exist under Windows/NT but due to Windows/NT's limited real-time behaviour, we do not think it that this is its optimal place in a control system.
OS-9
TACO has been available for a long time for the OS-9 real-time operating system. OS-9 has been mainly used on VME-systems based on Motorola's 680x0 (68k) processor family. The manufacturer of OS-9, Microware Systems Corporation, has moved its marketing strategy towards 80x86/Pentium (i86) and PowerPC processor families. In reality, OS-9/68k is written in assembler while OS-9/i86 and OS-9/PowerPC are written in C (formerly called OS-9000).
TACO's OS-9/68k version was ported to the OS-9/i86 version simply by recompiling with Microware's Ultra-C compiler. Microware provides a relatively low cost graphical cross-development environment for Windows, called FasTrak, which can produce object code for any of the above target processors.
OS-9/i86 can be easily installed on a PC as an alternative booting operating system with Windows. This is the most low cost development alternative, since the disk-based OS-9 can be used as native development system with Ultra-C compiler and line based source level debugger. X11/Motif is available for OS-9/i86 but as a separate product.
An interesting alternative for OS-9/i86 based PC installation is the PC/104, which is a full, industrial standard PC in compact form. PC/104 does not have a motherboard but modules are stackable through the 104-pin ISA-connector (hence the name) which makes it extremely compact. Using OS-9/i86 on a PC/104 instead of DOS/Windows makes a very powerful, low cost embedded controller.
TACO has been used with a PC/104 and OS-9/i86 for a small scale control system simulation. The communication was done over a serial line using CSLIP-protocol. The resulting system is a very low cost PC-controller for about half of the price of a "real" PC, which includes analogue and digital I/O (the PC/104 is controlled with a terminal emulator).
LynxOS/VxWorks
TACO is also available on two other real-time operating system platforms for PC's - today LynxOS and VxWorks ports are available.
LynxOS is a fully POSIX-conformant real-time UNIX system. In its PC-form, it is usually installed on a hard disk partition as an alternative booting operating system (diskless LynxOS target licenses are available). LynxOS can be used as a development environment in the same way as any other UNIX system with an X11/Motif user's interface.
VxWorks is typically used on embedded target systems, such as on VME single board computers based on Motorola's or Intel's processors. VxWorks is easy to set up for "hard real-time" applications. It requires a host-machine for development and a diskless target machine.
Both LynxOS and VxWorks rely on GNU-C/C++ compilers. In addition VxWorks has an advanced graphical user's interface called Tornado.
Other
Although TACO is presently "only" available for the five PC operating systems described above it easy to port it to other 32-bit operating systems running on PC hardware. All that is required is a C/C++ compiler and the SUN ONC/RPC. This includes especially all Unix-like operating systems e.g. Solaris, FreeBSD, SCO Unix, and also real-time operating systems like QNX.
DIY
TACO is intended to be a Do It Yourself (DIY) control system. You get the source and/or binaries. Install them. Write the device classes you need for your control system. Configure the database with the devices which must be instantiated and their corresponding resources. Start the Manager. Start the Device Servers. Start the Clients. You are ready-to-run !
In this scenario it is interesting to consider how PC's fit in. With regards to the platforms on which TACO runs today only Linux, Windows/NT, OS-9, and LynxOS can be used on PC's "out-of-the-box". VxWorks needs a host CPU and a specially supported CPU board.
Assuming you want to run TACO using PC's here a some ways in which PC's can be used to build a TACO control system :
front-ends - refers to the computers which are directly connected to the sensor and actuators of the control system.
Linux - use a standard "out-of-the-box" to interface PC hardware or other hardware connected via standard interfaces (e.g. serial lines, GPIB). For a large installation it might be advisable to use diskless Linux. Time critical tasks require using a driver or Linux real-time extensions.
OS-9 - best used on a PC/104 as an embedded controlled. This has the advantage that every device becomes a device server in network space and eliminates the need for a field-bus. Can also be used on an "out-of-the-box" to control PC hardware.
Windows/NT - can be used on an "out-of-the-box" to control PC hardware. Not well-suited for time critical tasks due to the lack of real-time support.
LynxOS - use on a standard "out-of-the-box" to control PC hardware. Has the advantage that LynxOS supports Posix real-time extensions.
VxWorks - cannot be used on an "out-of-the-box" PC. Can only be used on a PC CPU embedded in VME or special hardware.
operator console - refers to the computer which provides the operator with a graphical interface to the control system.
Linux - running on a PC is equivalent to a commercial workstation running a proprietary Unix, the only difference being that it is much cheaper. Linux comes with most public domain software pre-installed including all popular graphics packages e.g. X11, Tcl/Tk, OpenGL.
Windows/NT - is best suited to be used as an operator console. It has a powerful development environment and good support for doing graphics. It has the extra advantage of providing an integrated office automation environment thereby allowing controls applications to be incorporated with typical desktop applications. It is unfortunately a proprietary operating system.
OS-9/LynxOS/VxWorks - are intended to be used as front-end computers and therefore not suited to being used as operator consoles. Note however that it is possible to run graphics applications on these systems (all of them support X11 and Motif).
server - machine which offers system services e.g. database, manager
Linux - because it like any Unix a workstation has no problem to be used as a server.
Windows/NT - could theoretically be used as a server to run the TACO manager and database however the relevant software still has to be ported. As database it is proposed to use ACCESS which is sufficient for the basic requirements of the TACO static database
OS-9/LynxOS/VxWorks - are intended to be used as front-end computers and are therefore not suited to being used as system servers.
Conclusion
PC-hardware does not have the sophistication of VME-bus electronics, but not its price tag either. Alternative operating systems for PC-hardware, such as Linux or real-time operating systems can reveal its full power and make even a 386-based system perform as today's standards require. If you cannot afford using "over kill" when designing your system, a PC based control system using on TACO should be seriously considered.
If you have the money to invest in VME hardware but would like to integrate your desktop environment with your control applications then TACO with Windows/NT could be the answer.
In conclusion it can be said that never before has the PC world, with its present CPU-power/price ratio, the availability of stable 32 bit operating systems (both free and proprietary) and modern control systems like TACO, been so attractive from the controls point of view. Anyone building or maintaining a control system today should not be thinking about whether to integrate PC’s but rather how to integrate PC's into their control system (if they haven't done so already !).