mds -- The micro-display server
*******************************

Copyright © 2014 Mattias Andrée

     Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled ``GNU Free Documentation License''.

Overview
********

mds (mds stands for micro-display server) is a display server protocol and an implementation of said protocol. What makes mds stand out is its core design choice: it is desigend just like a microkernel. Rather than one, possibly modular, process --- a monolithic process --- mds is comprised of many small servers, each exchangable and responsible for one thing.

mds goal is neither security, performance nor a perfect graphical experience. mds is all about flexibility and freedom 0 (The freedom to run the program as you wish, for any purpose).

The reason for having a display server architectured as a microkernel is so that components can be added, remove and replaced online. Additionally, the message passing between the servers makes it easy to design a system that lets you make clients that can listen on messages between the servers and perhaps modify them. This enables you to do so much more with your display server. Moreover, if a single part of the system crashes it does not bring down the whole system, and the crashed server can be respawned with minor side effects. mds is architectured in three layers: a microkernel, a master server and a collection of servers. And clients are actually located on the same layer as the servers, because there is no actual difference, the only thing that separates a server from a client is for what purpose you run it. mds's kernel is a minimal program that do initialisation of the display, such as giving it an index and create runtime files and directories for servers and other programs to use. Then the kernel creates a domain socket for the master server and spawns the master server and respawns it if it crashes. Because of this, if the master server crashes it will not lose its socket when it is respawned. The master server than, on its initial spawn, starts the all servers and other programs that the user have choosen and then starts accepting connections to it and coordinates messages between servers and clients. Further, separating all components into separate processes enables us to only give the servers the privileges they actually need, rather than having one program with root privileges that takes care of everything even things that do not do require any privileges.

All mds's servers, that is all running parts of mds except the kernel, are designed so that they can re-exec themself so that they can be updated online without any side effects. Servers serialises their state, saves it to RAM (in a directory created by the kernel), re-exec themself and loads their serialised state. The kernel cannot do this because when it has spawned the master server it has no reason to re-exec, its only mission is to respawn the master server it if would happen to crash. It would technically be possible to enable the kernel to re-exec but it is not worth it as it as no reason to re-exec, and doing so puts the display server at risk of crashing.

Architecture
************

Layers
======

The mds display server in architectured in three layers. The first layer is called the kernel. The kernel is responsible for acquiring a display server index (As with any display server, the system can have multiple instances of mds running at the same time.), set up environment variables to indicate which display server and display server instance is being used, create a domain socket for the display server and start the master server and restart it if it crashes, and then clean up the system when the display server closes. The kernel only responsible for creating the domain socket for communication with the display server, it is not responsible for using it, that mission falls to the master server.

The second layer is the master server. The master server has two responsibilities: coordinating message passing between other servers and clients (In mds their is no functional distinction between servers and clients, the distinction is purely semantic.) and starting other servers.

The third layer is the other servers and clients. protocolwise there is no specification on how they are started. But in the reference implementation of the master server, this is done by starting a shell script with the pathname ${XDG_CONFIG_HOME}/mdsinitrc and the user is responsible for providing the logic in that shell script. (Moonstruck users are allowed to implement this in C or any other language of their choosing.) These servers implements the actual functionality of the display server.

Interprocess Communication
==========================

Intrinsic to mds is a powerful interprocess communication mechanism. Servers and clients connect to the display server by connecting to a domain socket served by the master server. A server or client that has connected to the display server can do three things:

  * Request assignment of a unique ID.

  * Multicast a message.

  * Join or leave a multicast groups.

Upon assignment of an ID the master server will automatically place the client in a multicast group for that specific client. This automatically multicast group assignment is done by the master server simply so you as a debugger do not forget to do so. When a client is disconnected it will and out a message to a specific multicast group that the client, refered to by it's ID, have closed.

A message in the mds protocol is comprised of two parts: headers and a payload. When a client joins a multicast group it is actually say that it is interested and receiving broadcasts containing a specific header or a specific header--value pair, or that it is interesting in all messages (This could be used for logging, possibly spying and networking.). Thus a message is automatically multicasted to groups indicated by its headers.

The multicast groups and receiving of groups is called interceptions. The interesting property of interceptions is that they may be modifying. When a server registers for message interception it can say that it wants to be able to modify messages. If this is done and the server receives a message for which it has said it want to be able to modify it, the master server will wait for that server to respond before it send the message to the next server in the interception list. The server can choose to do three things with a message that it has opted in for modification of: leave the message as-is, modify the message, or consume the message. A message consumption is done by modify the message to make it empty. A consumed message will not be send to any further clients or servers in the interception list.

To make this mechanism sensible, a server or client can set a priority when it registers for interception (does not need to be modifying.) When a message is broadcasted it will be received by all servers in the interception except the original sender, unless it gets consumes. The order in which the master server sends the message to the recipients is determined by priority the servers registed with. The message first sent to the recipients with highest priority and last to the recipients with lowestr priority, and orderd by the priority between those priorities. Of two or more servers have the same priority the order in which they will receive the message, of those recipients, is arbitrary.

An interesting property of this machanism is demonstrated in the mds-vt server. Unlike most servers mds-vt maintains two concurrent connections to the display. Once mds-vt receives a signal from the OS kernel requesting to switch virtual terminal, mds-vt will from one of its connections send out a message and wait for it to be received in its other connection and the let the OS kernel switch virtual terminal. The secondary connection to the display has registered interception with lower priority of the message that the primary connection broadcasts. This message will be received by other servers that will let the message continue to the next server in the interception list once that server is ready for the OS kernel to switch virtual terminal. All of these server has registered modifying interception of the message but none will actually modify or consume the message; it is only used a mechanism for letting mds-vt know when all servers are ready for the switch without having to know how many they are and wait for a reply from all of them.

Protocol
********

Environment Variables
=====================

A crucial of any display server is letting child processes know which display server they should connect to. X.org does by setting the environment variable DISPLAY to <host>:<display index>, where <host> is empty if the display is one the local machine. In this tradition mds does the same thing with the environment variable MDS_DISPLAY.

mds also creates a new process group and export the new process group ID to the environment variable MDS_PGROUP. This process group can be used to send signals to all mds servers collectively.

Signals
=======

mds servers can re-execute into an updated version of their binary. This can be used to update display server online after a new version has been installed. To do this send the signal SIGUSR1 to the server you want update. If a server does not support online updating it will ignore this signal. If the operating system defines a signal named SIGUPDATE, this signal is used instead of SIGUSR1.

If you need servers to free up allocated memory that they do not use, send the signal SIGDANGER, or if not defined SIGRTMAX. Unimportant servers may choose to die on SIGDANGER.

Filesystem
==========

The mds kernel creates two directories for the mds servers to use: one for runtime data and one for temporary data. These directories are named by MDS_RUNTIME_ROOT_DIRECTORY and MDS_STORAGE_ROOT_DIRECTORY, respectively, by the header file <libmdsserver/config.h>. If the systems runtime data directory is /run and transient temporary data directory is /tmp, and the package name of mds is mds, these directories will be /run/mds and /tmp/.{system-directory}.mds, respectively. In /tmp/.{system-directory}.mds the kernel will create a directory for the display server instance named .data prefixed by the display server index. For example if the display server index is zero, temporary data may be stored in /tmp/.{system-directory}.mds/0.data

As defined by SHM_PATH_PATTERN by <libmdsserver/config.h>, when a server re-executes itself it will marshal its state to the POSIX shared memory unit named /.proc-pid-%ji, where %ji (%ji is the pattern in *printf functions for the data type intmax_t.) is replaced with the process ID of the server. This file will be bound to the pathname /dev/shm/.proc-pid-%ji if POSIX shared memory is stored in /dev/shm by the operating system.

In MDS_RUNTIME_ROOT_DIRECTORY the kernel will create two files. .pid and .socket, both prefixed with the display server index (0.pid and 0.socket if the display server index is 0.). The .pid file contains the process ID of the display server and is used by the kernel to figure out whether an display server index is still in use or just not properly cleaned up. Of course it can be used by any program to find the process ID of the kernel process of a display server instance. The .socket is the domain socket used for communication with the display server and its servers and clients.

Message Passing
===============

Message passing over domain sockets is the underlaying technique for communicating with the display server. To communicate with the display server in the local machine a process must connect to the domain socket created by the display server kernel as named in Filesystem.

Clients should request a unique ID when it connects to the display server. (There is seldom a reason for servers to do this.) To do this the client sends

     Command: assign-id\n
     Message ID: 0\n
     \n

where \n is an LF-line break. The value on the Message ID line does not need to be 0, but servers and clients often start with 0 and count upwards. The value is however bound to an unsigned 32-bit integer. All message must contain this Message ID header, otherwise the message is considered corrupt and is ignored.

The empty line signifies the end of the header list, and in this case the end of the message. But a message may contain payload beneath this empty line. To include a payload, add the header Length that says how many bytes the payload is comprised of.

A header must contain a header name and header value without any trailing or leading spaces, and `: ' (colon, one regular blank space) exactly delimits the name and the value.

When the master server receives this Command: assign-id message it will assign the client a unique ID and send it to the client. (The master server is the only server than can address the client uniquely before it has an ID, so this part can only be implement in the master server.) If the client already has an ID, it will send back that ID to the client. This response consists of two headers ID assignment and In response to, containing the client's new (or possibly already assigned) ID and the value that was in the Message ID header, respectively. For example:

     ID assignment: 0:1\n
     In response to: 0\n
     \n

Notice that the master server never includes Message ID in message originating from it.

As seen in this example, the client ID consists of two integers delimited by a colon (`:'). Both of these integers are unsigned 32-bit integers. This is done this way because unsigned 64-bit integers are forbidden because it is not supportable natively be some programming languages.

Before a has gotten a unique client ID assigned to it, it will be `0:0'.

If a client gets disconnected from the master server, the master server will sends out a signal header message. This header will be Client closed and contain ID of the client that closed. For example:

     Client closed: 0:1\n
     \n

Be aware that if a server or client closes and does not have a unique client ID, this message will be:

     Client closed: 0:0\n
     \n

Once a client has an unique client ID assigned to it, it should always include the header Client ID in its messages. The value of Client ID should be the client's ID. If a server wants to address this client, it should include the header To with the value set to the recipient's client ID. Be aware that such message may not be sent to that recipient uniquely, any server or client is free to sign up for receive of such message, any messages or message contain any other header or header--value pair that may also be included in the header.

GNU Free Documentation License
******************************

     [file fdl.texinfo not included]