The early telemetry systems employed in the water industry acquired and stored data from the remote-stations for the engineerÕs inspection. The data, which depicts a snap shot of the state of the water system, was used to assist the engineer in deciding operational controls of the water distribution system. The term used in water industry to describe such telemetry/telecontrol systems is Supervisory Control and Data Acquisition(SCADA).
However, the gradual growth of the water system resulted in the accumulation of the increasing volume of data which had to be attended to by the engineer. To help reduce the work load and achieve a more efficient and cost effective operational control of the system, simulation software were introduced. The role of the simulation software was to analyze the system parameters and arrive at a decision which would then be accepted or modified by the engineer who is responsible for the system operations.
The integration of the simulation software with the existing telemetry systems into a realtime decision support system proved to be a challenging task due to the rapid increase of computational requirement of the simulation algorithms with increase of the physical network size. A classical solution to this problem, frequently adopted in the case of linear systems, was to partition the network into a number of subnetworks, solve the subnetworks in isolation and then coordinate the subsystem solutions in order to find the overall system solution. This approach was developed and generalized for nonlinear networks in an earlier research.
Having developed the distributed diakoptics based simulation algorithm, we have focused on its efficient and flexible implementation on the distributed computing hardware. As the simulation software is envisaged to undergo continuous development in line with the development of the physical telemetry systems, it has been important to select the implementation language that is particularly well suited to such a task. Ada is a language that was specifically designed to be used for the development of large scale software systems. It provides strong typing, enforces public/private structure and provides explicit specification of parallelism. However, since Ada was not originally designed for distributed systems, it has been necessary to extend AdaÕs rendezvous mechanism to facilitate network communications. This mechanism is the Ada Virtual Node model described in Atkinson C. et al., 1988.
Implementation of a distributed program in Ada to run on the loosely_coupled processing nodes requires a special mechanism by which the distributed processing tasks can exchange messages in the course of computations. Although Ada provides an intertask communication mechanism in the form of rendezvous, for the single processor environment, this mechanism requires extension because Ada was not originally designed for distributed systems [ANSI/MIL-SAD, 1815A, 1983]. This mechanism is the Ada Virtual Node model described in [Atkinson C. et al., 1988]. Virtual Nodes can be completely defined in terms of Ada concepts. In the most general case they are the transitive closure of a procedure in an Ada library; the units of such closure must be compliant with a set of 'composition rules' which provides the virtual node with the required properties. These properties include: complete encapsulation of internal state, that can only be accessed through a proper interface; the lack of reference to shared objects among virtual nodes, each virtual node must have its own thread of control. Communication between virtual nodes is constrained to happen via "Remote Entry Call"; remote entries are those declared in a task contained in an interface package provided by a virtual node. Such entry calls can be called by other virtual nodes if the interface package is visible to them (by the means of the WITH statement). The communication protocol is then the Ada rendezvous model.
Since virtual nodes are functional abstractions they do not require any immediate mapping onto physical nodes. Therefore the whole application can be designed as a single Ada program made of many communicating virtual nodes, where the synchronisation and communication mechanism used is Ada tasking. A Virtual node allocated to a physical node(a processor in the underlying architecture) is transformed into a separate Ada program: the remote entry calls that happen to cross the boundaries of the physical node are translated into calls to primitives provided by a "standard interface" that implement an interprocess communication protocol(e.g Internet_UDP) semantically coherent to the Ada rendezvous on a given execution environment(e.g. UNIX environment). Strong type checking is enforced at the boundaries of the communicating subsystems, and a single interprocess communication scheme with well defined semantics(i.e. the Ada rendezvous) is used.
In our water network simulation program, it was necessary to identify the program components that can be programmed as Virtual Nodes or Virtual Node Types(worker tasks). Since the decomposed model of the system consists of coordination and subsystem solution processes, they can be coded as separate VNs. Therefore, a coordination process can be defined as a VN and subsystem solutions can be constructed from the instances of VN types. VN types represent the subsystem solution worker tasks which can vary in numbers depending on the networkÕs partitioning scheme.
Once all the components are identified and programmed as VNs or instances of VN types, they can be mapped onto the underlying target distributed system. Ideally each unique VN and instances of VN type should map onto a single processor in the target system, however, there may not be as many processors as there are VNs and instances of VN types, thus Virtual Node grouping is required. VNs are organised into different libraries which are part of the same family of libraries. For example all the public template units(i.e. template-units used by all or some of the virtual nodes (Fig 1)) are grouped together in a single library unit, thus it is important for VNs to have access to this library. Alsys Multi_library mechanism provides this facility by allowing references from one library to units of other libraries in the same family of libraries.
The program is implemented on a network of SUN/SPARC workstations connected by an Ethernet communication link, having a UNIX operating system. The inter_process communication provided by UNIX is based on connectionless User Datagram Protocol(UDP) sockets.
The primary task of every worker program is to send its interface task's address which is referenced by an access value to the coordination program. This would then enable the coordination program to call the target workerÕs interface task. However, sending the access values referencing task objects on another machine is a hazardous task since the use of such parameters in a different address space may have unpredictable effect. Thus it is imperative to restrict the use of such parameters only to the machine where they originated from.
Upon receiving the access value of the worker's interface task, the coordination task incorporates it with the workerÕs other identification parameters(i.e VN number, and its processor ID number) in a record. In fact there is an array of such records whose elements represent unique identification information about the communicating worker tasks.
In contrast, the coordination interface task prepares the data destined for each worker task. This forms a packet of data incorporated in which is the unique identification information of each worker task. In our implementation the largest packet of data sent once from the coordination VN to the worker VNs, is of the order of 18KBytes for a 130 node water network. However, such large packets would only be sent if the network is partitioned into two subnetworks, which does not typify a normal partitioning criteria of a large network(i.e usually such a large network would be further partitioned into more than two subnetworks).
Prior to receiving the data packet, the worker task creates a receiving buffer which should be as large as the data packet or even larger to avoid buffer overflow. As the size of the data packet depends linearly on the size of the subnetwork, the greater number of partitioning is decreasing the size of the packet while increasing their number. These two factors balance out.
The worker task uses the access value of the worker interface unpacked from the data packet,in order to call the interface task of the target worker. This ensures that the correct instance of the worker VN type receive the data. The worker tasks upon completion of their task(i.e solution of subsystems), organise the results in a packet and send their result packets as soon as they become available. These packets are queued on the coordination VN. This queuing is necessary since the same communication entry point is used by the coordination program to receive the result packets sent by the worker task VN types.
The order in which these packets arrive is arbitrary since each packet has associated with it a software node identification number indicating the worker from which it originated from. This number is used to unpack and store the parameters of the result packet in the coordination storage area.