This Old Technology

When assessing your organizational business needs, do you totally rebuild your heritage applications or do you incrementally migrate to state-of-the-art technology?

With the growing success of object technology and the continued growth of the M(UMPS) marketplace, at least one company is extending the capabilities of the 100,000+ systems and more than 4 million users of M(UMPS) technology¹. Given the strong foundation that M is built upon and the huge investment that organizations have put into developing applications, it seems like a logical step to extend the standard M technology foundation to objects, thereby providing an incremental migration path for the thousands of applications in use today.

Insuring a Solid Foundation

As we all know, when remodeling a house, it is important to at least start with a solid foundation. Obviously, when attempting to re-engineer a technology, this becomes a fundamental requirement. A few years ago we at ESI Technology Corporation embarked upon a project to migrate the ANSI standard MUMPS (M) database and scripting language to objects. This effort resulted in a product called EsiObjects (pronounced 'easy objects'). What we learned about the object paradigm on M was formulated into a Type C proposal and submitted to the M Development Committee (MDC), the M ANSI standardization body. Additionally, a white paper on the subject was submitted to CORBAmed in response to an RFI for an object to M binding. MUMPS (MGH Utility Multi-Programming System), known as M today, is an ANSI, FIPS and ISO standard scripting language used to access an integrated hierarchical database². It was developed at the Laboratory of Computer Science, Massachusetts General Hospital in 1967 in response to a need for a language and database that would simplify, and accelerate the development of applications. Being a standard language, it is portable, and runs on all major hardware and OS platforms. Late in the 1970's, because numerous dialects were appearing, the MDC was formed to standardize the language. The first ANSI standard was published through the X11.1 subcommittee. Since the 1980 standard, the language has gone through numerous revisions. It eventually became an ISO and FIPS standard as well. Today, M is a highly portable system that runs on all popular operating systems. It is used extensively, but not exclusively, within the medical industry. Most Department of Defense, Veterans Administration and Indian Health Services clinical systems are M based. Additionally, M is used to implement clinical and administrative systems by numerous commercial laboratory, HMO and hospital organizations.

M System Overview

Figure 1 below illustrates the general block flow of a typical M system. Traditionally, M started out as multi-user, multi-tasking systems.

Figure 1: Typical M System Components

Users are associated with a process when they log on. Historically, the user interface has been dumb terminal based and still is in many cases. The User Process contains the M environment that includes a local symbol table as well as routine execution buffers, an interpretation and other structures needed to maintain the state of an M process.

The interpreter executes M code. M supports one data type, a string. The language is heavily oriented towards handling strings. It supports only one, highly recursive data structure - multi-dimensional, sparse (no dimensioning needed) arrays. These array structures are supported locally, within the process space or globally, that is, persistently on disk. Persistent arrays are commonly referred to as Globals.

Global structures (the database) are supported by the Global Module, which, in most cases implements them as balanced self-optimizing multi-way tree structures. This simply insures that the number of disk blocks traversed to get to any given record within a global is the same. It also supports efficient key compression that minimizes the physical structures of the database and maximizes access to data records. The Global Module is central to M database operations. It acts as the avenue to traditional M networking if the global is mapped to another system. Most implementations also support Before Image Journaling (BIJ) and After Image Journaling (AIJ) if it is enabled.

Whenever a global is accessed for read or write by M code, the Global Module handles that request. Generally, it works in concert with large, sophisticated, in-memory Disk Cache structures that provide fast access to frequently accessed data. All blocks read into cache reside there until flushed out naturally by more frequently accessed blocks. Any block in cache that is modified is flagged for writing. Flagged blocks are written asynchronous to the disk by a process called Write Demon. Write Demon is also responsible for handling the Before Image Journaling (BIJ) of any blocks. BIJ insures a recovered state in the event of a system crash during the writing process.

When the user process deletes all or a portion of the array, the Global Module dynamically restructures the hierarchical structure on disk. The blocks that the deleted nodes reside in are attached to a garbage tree. A Garbage Collector process reclaims these blocks for the system. A similar mechanism is employed to maintain a local symbol table.

Renovating the House

When renovating an existing structure, the redesign phase raises questions like what to reuse and what to add (or hide). The same is true for technological reconstruction. Java exemplified this approach as the next evolutionary step beyond C++. Questions arise like; what should be brought forward for reuse? What should be eliminated or hidden? What should be added to attain the reconstruction goals?

Goals

The following lists the reconstruction goals:

• Build an ANSI Standard M language preprocessor with the required object-oriented extensions.

• Integrate features to assist in the migration of heritage M application systems.

• Leverage the underlying M strengths into an object oriented database system.

• Simplify development by providing the developer with libraries of reusable foundation and business framework classes.

• Provide a high performance runtime environment.

• Build a development environment on Windows NT and Windows 95 platforms.

• Expose server side objects to the enterprise via standard, high performance, open client server and Internet technology.

Our goal, simply put, was to implement a well-defined object layer on top of M in a way that would exploit the good features of M and hide or avoid the inadequate features.

Assessment and Inspection Phase

Aside from the physical features outlined above, M inherently contains some features that are fundamental to extending the language and database to objects.

Some important language features are:

• Powerful string handling capabilities

• Integrated database array structures

• Powerful indirection capabilities

• Interpretive nature of the language

Referring to Figure 1 again, features offered by a local symbol table are:

• Non-persistent sparse arrays required for in-memory transient storage of objects.

• Array subscript indirection necessary for code generalization.

• Automatic garbage collection required for the reclamation for memory space.

• Dynamic allocation of storage required for transient object instantiation.

Important database features that enable persistent objects are:

• Persistent sparse arrays required for permanent non-transient storage of objects.

• Array subscript indirection necessary for code generalization.

• Networked database that enables the distribution of objects.

• Automatic garbage collection required for the reclamation for disk space.

Features of dynamic allocation of storage are:

• Environment required for non-transient object instantiation.

• Locking mechanism that insures mutual exclusion.

• System level that supports programming services and handles memory management issues.

• Multi-user, multi-tasking support needed by any enterprise database system.

Runtime execution features crucial to the dynamic nature of objects are:

• Dynamic decision making.

• Late binding of values to names.

• Runtime loading of routines eliminates linking and participates in the dynamic decision making and late binding features.

• Code generation aids in the preprocessing of object oriented code into standard M code.

In general, each feature combines to form a synergy that makes M a powerful enabling technology for objects.

Demolition Phase

As part of the transformation, we wanted to eliminate some undesirable features of M that have caused problems over the years. Some general problem areas that the object paradigm would eliminate are:

• Lack of a consistent data-organizing paradigm. The M array structure offers a generalized, flexible mechanism for the organization and storage of persistent and non-persistent data. Without an organizing paradigm superimposed upon it, database designs can become wildly inconsistent and difficult to maintain and extend.
• Lack of encapsulation. By default, the scope of all variables is across the execution context of any routine that is loaded and executed. It has always been the responsibility of the programmer to protect the symbol's value by stacking them. Adding symbol scoping and declaration eliminated this burdensome, and often error prone shortcoming.
• Specific M networking. Although not a bad feature, it did not fully represent a generic solution. What was needed was a more robust, generalized model.
• Dumb Terminal Interface. The traditional roll and scroll interface had to be replaced with modern, Windows based interfaces.

Structural Enhancement Phase

The renovation goals were accomplished by first extending the foundation into the object domain. We wanted to build a comprehensive, robust foundation that would contain all the object-oriented constructs needed to first wrap legacy M applications and at the same time contain all the constructs needed to evolve to a pure object database.

Extending the foundation involved adding the following features and functions.

• Because other vendors owned the M systems, we had to build a preprocessor for the language. This enabled us to implement the M standard as well OO extensions to the existing language. We could add constructor and destructor commands, event handling commands, messaging, special variables, functions, and variable scoping needed to enforce encapsulation as well as single and multiple inheritance.
• Message syntax had to be designed to facilitate communication between objects.
• In the spirit of implementing open technologies, we based the object model on the Smalltalk model with enhancements. To keep the message interface small, the class model is divided into interfaces that support four services: Methods, Properties, Relationships and Events. Method, Property and Relationships are invoked via a typical message construct. Additionally, we added model side event processing. Not only is it important for objects to communicate with each other directly, it is important that they be able to listen for unsolicited events that they may be interested in.
• To start development, we built a primitive browser based on character cell technology. This enabled developers to build the required layers until the GUI interface was complete.
• Libraries were implemented as a way to organize classes. Actual libraries hold concrete or abstract classes. Virtual libraries can be created to organize classes across actual library boundaries.

Wiring and Plumbing Phase

The M language has had all the elements to implement the database, application and presentation tiers of an information system. In a way, this fostered a closed environment that did not necessitate a well defined, open interface. Everything could be accomplished using one language and within the same environment. The non-M world often viewed M systems as being closed. This all changed with the PC revolution. Pressure to replace the presentation and subsequently the application tiers forced M into a stricter database role. Consequently, it forced a more open architecture.

Our approach was to build on open networking standards. An object messaging API was made available through the standard TCP/IP protocol. On top of this layer we have implemented the following:

• A fundamental CORBA ORB using EsiObjects. This ORB can act as a Server and Client. It does not contain all the services of a commercial ORB.

• ActiveX control that uses the TCP/IP Gateway

• ActiveX control that uses a local COM Gateway

• Java Proxy that integrates all the TCP/IP communications

• COM Proxy that integrates all the TCP/IP communications

Finishing and Furnishing the House

Once the infrastructure was in place, we added those things that made it useable.

• As stated, M is a scripting language with an integrated database. From its conception these systems have always had an integrated development environment. Extending the heritage M foundation to objects necessitated a modern development environment. We developed a Windows development environment for the client. Because many M development organizations are large and use multiple M systems that may be different, we built in multi-session support where the programmer could attach to multiple databases at one time. EsiObjects hide the differences of each M system giving each one look and feel.
• Collection and Immutable classes were the first reusable classes implemented as part of the foundation. Collections implemented were Set, Array, Bag, List, Dictionary, Log, Map and Multi-Map. Each collection has an associated iterator class that manages collection iteration. Immutables implemented were Date, NameValuePair, Time, TimeRange and TimeStamp.

Once the M foundation features were extended into the object domain, we started to benefit from reusability. For example, the OO and event handling features along with the Dictionary collection class were used to build a DataManager class in a matter of hours. An instance of this class can track all instances of a target class automatically via an embedded dictionary that is dedicated to a property of the target class. Anytime the property changes in the target class, the proper indices are automatically updated.

Beyond these reusable layers we built a Business Framework with a set of Business Objects that are common to all information system applications. These libraries of reusable objects give organizations using EsiObjects a jump implementing applications.

Occupying the House

In the same way that M acts as an enabling layer for objects, the object foundation acts as an enabling layer for implementing more sophisticated layers. The foundation was built such that a completely new application could be built upon it or an existing heritage application could be migrated using wrapper classes.

One such project where this approach has been used is with the DOD Healthcare agencies Composite Health Care System (CHCS). It is a very large clinical application implemented in M and it runs at about 108 sites throughout the world. We were given the task of wrapping the database for Patient Registration as well as Radiology, Laboratory and Pharmacy Results Retrieval and exposing that data through the C++ API to a CORBA ORB. The data elements were then mapped to a new business model. Because of the reusability and OO features, we were able to quickly build an object repository for the storage of data definitions (extracted for the heritage DD) and business rules from which we could generate surrogate classes that wrapped the data elements.

The Government Clinical Patient Record (GCPR) Program is another large project where EsiObjects has proven that M Technology does have a migration path into the future. The strategic goal of the GCPR program is gather a longitudinal (integrated) patient record based on patient visits to Indian Health Service, Veterans Affairs and Department of Defense Health Affairs clinics and hospitals around the world and present it to any provider authorized to view it. ESI’s job was to build a Clinical Observation Access Service (COAS) server to access clinical observations on the VISTA system (VA’s clinical information system based on M) and pass requested information back to the system that is responsible for integrating the patients record. To facilitate implementation and communications, we implemented a fundamental CORBA ORB. These components were successfully demonstrated on schedule and within budget largely due to the Object Oriented approach taken.

An Open House Party

Now that the house has been renovated, it’s time to share it with our friends and neighbors. Over the years of this transformation, ESI Technology Corporation has undergone a change as well. Today our business is based on a service model, not a product model. We have been very successful in using EsiObjects in large projects would like to share that success with the world. ESI has decided to put EsiObjects under an Open Source license, making all the sources and runtime freely available to anyone who wants it. Why are we doing this? Here are some reasons:

1. M technology has been good to ESI over the last 16 years and we have profited from it. Now in its hour of need we want to give something back.

2. The users of M Technology seemed to have resigned themselves to the fact that they have no choice. Now they have at least a partial choice. Hopefully, the Open Source M initiative currently underway will bear fruit. An Open EsiObjects and an Open M system would bring back a potent state-of-the-art, competitive system.

3. We believe that the Open Source movement will be the way software is developed and distributed in the future.

4. It will be good for our business! We can benefit from the contributions many talented engineers around the world can make to this technology.

Conclusion

For over 30 years M Technology has filled the needs of organizations that depended on it to satisfy their business needs. Today these organizations are faced with hard decisions. Do they continue with M Technology and take a chance the only vendor of M Technology will take them securely into the future or do they move to more mainstream technology? Now there is another alternative. If you belong to one of those organizations, by supporting the EsiObjects and M Open Source initiatives, you can:

1. Take control of your future by controlling the future of the software that runs your business.

2. Extend the life of your M applications by giving them a clear migration path based on your needs. Retain your investment in your systems.

3. Eliminate the dependency on other organizations to answer support problems. Under Open Source licensing you have control over the source code – fix them yourself!

4. Eliminate the dependency on other organizations to add new features that you dearly need today. Under Open Source licensing you have control over the source code – add them yourself!

5. Help revive M Technology by bringing it mainstream.

The Open Source Initiative currently underway can help This Old Technology to become a vital, energetic New Technology.

1 MDC Representative Report to ISO/IEC JTC1/SC22 for the 1999 SC22 Business Plan.

Back to Top of Page
Back to White Papers