MODULES IN DHIS SERVER RELEASE 5
================================

From release 5 the DHIS server distribution no longer includes DNS specific 
code. Instead of being a DNS-oriented server DHIS is now meant to be flexible 
enough to provide all sorts of services to hosts that are assigned a dynamic 
IP address on the Internet.

This document describes how these services (modules) are organised and 
processed within the DHIS server. To implement a dynamic DNS server please see 
an external module that implements this feature.

The R5 DHIS server is service independent. In other words, DHIS may be used
to update DNS, to update a tunneling service, to update a firewall, etc ... 
based on a dynamically changing IP address from its clients.

The new server has an additional database (by default /etc/dhis/db/services.db)
containing the list of support services (modules).

In order to run dhisd >= 5 you will need to create and configure this
file first.

Main operation mode of the DHIS server:
---------------------------------------

When executed the DHIS server reads its client's database (by default from
/etc/dhis/db/dhis.db) into memory and proceeds to listening mode on a UDP
port (by default 58800). When messages arrive the server responds accordingly,
authenticates the clients (based on the keys and information loaded from the
database) and marks them online. Furthermore the server keeps sending periodic
checks to each and every client in order to verify that these are still 
connected. If a particular client fails to reply the server will consider it 
offline and mark it accordingly.

Each client is identified to a server by a single ID number called the HostID.
The new modular structure implements a means to mark these hosts online and 
offline.
Instead of simply updating DNS, the DHIS server now sends the request to a 
child process which will, in its turn, perform the required updating function 
for the particular client. Likewise if the client is turned offline a similar 
request is submitted.

What are DHIS 5 modules ?
------------------------- 

DHIS server R5 modules are sub-processes (engines) that execute a particular
updating task. One module may update DNS whilst another may open a port in a
firewall for the particular dynamic IP. Clients in DHIS R5 may subscribe 
services. The server 5 supports multiple modules/services. Thus, one particular
client may be subscribed into the DNS service while another subscribed to the 
Firewall service while yet another to both. 

A module structure has the following fields:

service tag - a single word identifying the module / must match service in 
	      dhis.db

number of proceeses - the number of concurrent processes that implement the 
                      module

command - the module engine process command to be executed


Example of /etc/dhis/db/services.db
-----------------------------------

Before executing dhisd make sure that you have created a valid 
/etc/dhis/db/services.db file.

The syntax for the file is as follows:

service_tag number_of_processes_command

For example, let us assume we have two services named dns and fw (as service 
tags). One possible services.db could look like:

;------------------------------------------------------------------------------
; This is a comment line
dns 4 /etc/dhis/bin/dhis-dns-engine
fw  2 /etc/dhis/bin/dhis-fw-engine
;------------------------------------------------------------------------------

Inter-process communication for modules:
----------------------------------------

On startup dhisd (the DHIS server) reads the services.db database and loads its
lines into memory. In the example above two module references are loaded, dns 
and fw.

It then checks how many processes it should spawn for each module (4 for dns 
and 2 for fw). For each module/process instance it creates a pipe, forks 
itself, redirects the child's  stdin to the pipe and executes the command 
given in column 3.

The idea behind the number of processes is one to allow load-sharing. The 
number of processes should be carefully chosen. For example a service with 
many clients may require several concurrent processes while a not so used 
service may have just one. It is up to each and everyone to decide how many 
processes per service should be spawned. The more processes the more concurrent
updates may be performed, especially if these are likely to consume a reasonable
ammount of time since they are processes sequencially by each process. 

Each of the processes is exactly the same and does exactly the same thing for
each service. 

In the example above dhisd (the main process) would give birth to 4 dns 
processes plus 2 fw processes. 

Upon a DNS request the server passes the update task to one of its 4 dns 
childs.

Upon a firewall request the request to one of its 2 firewall childs.

The algorithm to decice which process to pass a request is a round-robin one.

The first DNS update request will be sent to process 1, the second to 2, ...
the 5th to 1 and so on ...

If any one child dies the server re-executes it upon detection.

The only thing to take into account is open file descriptors.
DHIS by default uses 2 file descriptors (for UDP socket and log file)
In addition, for each child it needs two file descriptors one of which 
is closed soon after creation.

If a particular system has a maximum file descriptor number per process of 
say 30 (this should be OS specific and configurable):

dhisd - 2 + 1 
per child - 1

The server may spawn in total 27 child processes (all services * instances)
It would be wise to leave a few descriptors available (for reloading database, 
etc) so therefore a maximum number of child processes could for be set to 25 
or 24.


Service Membership for clients
------------------------------

Defining which clients are subscribed to which services is straight forward.
For each client record in dhis.db (between { and }) add a line starting with
service and followed by the service tag name.

For example:

dhis.db:

1000 {
	hostpass	mypassword
	service		dns
}

1001 {
	hostpass	myotherpassword
	service		fw
}

1002 {
	hostpass	someotherpassword
	service		dns
	service		fw
}

For each client record in dhis.db, if the service keyword is not specified
the server will automatically try to bind it to a service with a tag of 
"default".

Thus naming a service "default" in services.db makes all clients which don't 
have a service keyword subscribe to it automatically.

Note however that by specifying a second service for a particular client the 
default service will no longer be used. In this case "service default" would
also need to be added for that particular client record.

The algorithm is simple:

	for each client:
	is there a list of services ? if so, update all those registered  
	if not, is there a service called "default" ?
	if so, update using default 
	otherwise do nothing for that client

Database reloads
----------------

Whenever a database reload is issued (with SIGHUP to dhisd) the dhis.db file
is re-read. However the services file is not reloaded and the child processes
are kept open and running. Once services.db is updated the server dhisd
needs to be terminated (wish SIGTERM) and restarted for these changes to take
effect.

Communication to the child processes
------------------------------------

Communication to child processes is achieved through pipes. Before forking
for each service/instance engine process the server creates a pipe with 2
file descriptors.

Once forked the server closes its reading descriptor for that pipe. The client
closes its write descriptor. (The server sends messages to childs but not 
the other way around).

The open descriptor is kept in a list of service/instance.

Both stdout and stderr of child processes are redirected to /dev/null

On each update the server simply writes a text message to the corresponding
pipe terminated by a \n character. The child reads that message, executes it,
and proceeds waiting for a new message.

The language used between server and childs is formed by 5 commands, namely
add, update, delete, reload and exit. This language is called the DHIS
module interface and must be understood by all module engines that are to
work with this DHIS server.

server -> exit\n -> child

The server is telling the child engine that it will terminate (possibly has been
issued a SIGTERM) and therefore that the child should also terminate itself.

server -> reload\n -> child

The server is telling the child that it has reloaded its dhis.db file. Since 
the child may need itself to match the ID numbers to additional information, 
if required the child should also reload any external databases. A clear 
example of this is when a record is added to dhis.db and the child keeps a 
in-memory list of existing IDs and, for example, hostnames for DNS. The 
reload instruction tells the child process that (possibly) a new host has 
been added and therefore it should as well reload its own database file.

server -> add service_tag id x.x.x.x\n -> child

The server issues an online task to the child. Service tag is the name of the
service (as in dhid.db and services.db), id is the client's ID number and 
x.x.x.x is its newly acquired dynamic IP address.

The service_tag may be discarded in an engine that only does one thing. 
However one may wish to write an engine that does 2 or 3 different things and 
therefore, a means to differentiate these is also passed.

server -> update service_tag id x.x.x.x y.y.y.y\n -> child

Like in the add command the server tells the client that the IP address of 
client ID has changed. The child should then update its entries. A typical 
structure of update is that of issuing a delete followed by an add.
x.x.x.x is the new IP address, y.y.y.y is the old IP address.

server -> delete service_tag id x.x.x.x\n -> child

The server tells the child that client id has been disconnected or is no
longer reachable at the previous IP address.

These are the 5 commands that need to be implemented in any dhisd module 
engine.

The engine program:
-------------------

The engine (child) process is a program and may be as simple as a shell script
or as complex as wished. In its pure form it just does:

	while(true)
		read a line from stdin
		process a line



Example engine for DNS:
-----------------------

while(true)
	read line from stdin
	if(line == exit) exit
	if(line == reload) reload internal database
	if(line == add) {
		get hostname for passed id
		issue nsupdate for hostname with passed address
	}
	if(line==update) {
		get hostnamed for passed id
		issue nsupdate for hostname with passed address
	}
	if(line==delete) {
	get hostname from passed id
	issue nsupdate for hostname with an offline address
	}

Writting engines
----------------

As previously said, module engines may easily be written and used with
DHIS providing they comply with the above interface.

Writting of engines is encouraged and we (at DHIS) would very much like to 
include any third party modules in our distributions. Modules may be
for a particular task ... or many ... for a particular platform/os ... or many

If you have any modules that you would like to share, please email 
support (at) dhis.org with your contributions and we will make them available 
for download by others.


Example Engine:
---------------

An example shell script engine is provided in
	
	engines/dhis-dummy-log-engine.sh

This is a very simple engine module that only reads from the parent
dhisd process and writes output lines to a log file in tmp. It may be
used for testing purposes before trying to use a more "useful" engine.