path: root/doc/info/mds.texinfo

\input texinfo   @c -*-texinfo-*-

@c %**start of header
@setfilename mds.info
@settitle mds
@afourpaper
@documentencoding UTF-8
@documentlanguage en
@finalout
@c %**end of header


@dircategory Graphics environment
@direntry
* mds: (mds).                        The micro-display server
@end direntry


@copying
Copyright @copyright{} 2014 Mattias Andrée

@quotation
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with no
Invariant Sections, with no Front-Cover Texts, and with no Back-Cover
Texts. A copy of the license is included in the section entitled
``GNU Free Documentation License''.
@end quotation
@end copying

@ifnottex
@node Top
@top mds -- The micro-display server
@insertcopying
@end ifnottex

@titlepage
@title mds
@subtitle The micro-display server
@author by Mattias Andrée (maandree)

@page
@c @center `'
@vskip 0pt plus 1filll
@insertcopying
@end titlepage

@contents



@menu
* Overview::                        Brief overview of @command{mds}.
* Architecture::                    Architectural overview of @command{mds}.
* Protocol::                        The @command{mds} procotol.
* libmdsserver::                    Overview of @command{libmdsserver}.
* GNU Free Documentation License::  Copying and sharing this manual.
@end menu



@node Overview
@chapter Overview

@command{mds}@footnote{mds stands for micro-display server}
is a display server protocol and an implementation of said
protocol. What makes @command{mds} stand out is its core
design choice: it is desigend just like a microkernel.
Rather than one, possibly modular, process --- a monolithic
process --- mds is comprised of many small servers, each
exchangable and responsible for one thing.

@command{mds} goal is neither security, performance nor
a perfect graphical experience. @command{mds} is all
about flexibility and freedom 0@footnote{The freedom to run
the program as you wish, for any purpose}.

The reason for having a display server architectured as a
microkernel is so that components can be added, remove
and replaced online. Additionally, the message passing
between the servers makes it easy to design a system that
lets you make clients that can listen on messages between
the servers and perhaps modify them. This enables you to
do so much more with your display server. Moreover, if
a single part of the system crashes it does not bring down
the whole system, and the crashed server can be respawned
with minor side effects. @command{mds} is architectured
in three layers: a microkernel, a master server and a
collection of servers. And clients are actually located
on the same layer as the servers, because there is no
actual difference, the only thing that separates a server
from a client is for what purpose you run it. @command{mds}'s
kernel is a minimal program that do initialisation of the
display, such as giving it an index and create runtime
files and directories for servers and other programs
to use. Then the kernel creates a domain socket for the
master server and spawns the master server and respawns
it if it crashes. Because of this, if the master server
crashes it will not lose its socket when it is respawned.
The master server than, on its initial spawn, starts
the all servers and other programs that the user have
choosen and then starts accepting connections to it and
coordinates messages between servers and clients. Further,
separating all components into separate processes enables
us to only give the servers the privileges they actually
need, rather than having one program with root privileges
that takes care of everything even things that do not do
require any privileges.

All @command{mds}'s servers, that is all running parts of
@command{mds} except the kernel, are designed so that they
can re-exec themself so that they can be updated online
without any side effects. Servers serialises their state,
saves it to RAM (in a directory created by the kernel),
re-exec themself and loads their serialised state. The
kernel cannot do this because when it has spawned the
master server it has no reason to re-exec, its only mission
is to respawn the master server it if would happen to crash.
It would technically be possible to enable the kernel to
re-exec but it is not worth it as it as no reason to
re-exec, and doing so puts the display server at risk
of crashing.



@node Architecture
@chapter Architecture

@menu
* Layers::                          The layers of the display server.
* Interprocess Communication::      How servers and clients communicate.
@end menu

@node Layers
@section Layers

The @command{mds} display server in architectured in
three layers. The first layer is called the kernel.
The kernel is responsible for acquiring a display
server index@footnote{As with any display server,
the system can have multiple instances of
@command{mds} running at the same time.}, set up
environment variables to indicate which display
server and display server instance is being used,
create a domain socket for the display server and
start the master server and restart it if it crashes,
and then clean up the system when the display server
closes. The kernel only responsible for creating
the domain socket for communication with the display
server, it is not responsible for using it, that
mission falls to the master server.

The second layer is the master server. The master
server has two responsibilities: coordinating
message passing between other servers and clients
@footnote{In @command{mds} their is no functional
distinction between servers and clients, the
distinction is purely semantic.} and starting
other servers.

The third layer is the other servers and clients.
protocolwise there is no specification on how
they are started. But in the reference
implementation of the master server, this is
done by starting a shell script with the
pathname @file{$@{XDG_CONFIG_HOME@}/mdsinitrc}
and the user is responsible for providing the
logic in that shell script.@footnote{Moonstruck
users are allowed to implement this in C
or any other language of their choosing.}
@c Which is better: cray-cray users, lunatic users,
@c moonstruck users, insane users, ballers, madmen,
@c loony tunes?
These servers implements the actual functionality
of the display server.



@node Interprocess Communication
@section Interprocess Communication

Intrinsic to @command{mds} is a powerful
interprocess communication mechanism. Servers
and clients connect to the display server by
connecting to a domain socket served by the
master server. A server or client that has
connected to the display server can do three
things:

@itemize
@item
Request assignment of a unique ID.

@item
Multicast a message.

@item
Join or leave a multicast groups.
@end itemize

Upon assignment of an ID the master server
will automatically place the client in a
multicast group for that specific client.
This automatically multicast group assignment
is done by the master server simply so you
as a debugger do not forget to do so. When
a client is disconnected it will and out a
message to a specific multicast group that
the client, refered to by it's ID, have closed.

A message in the @command{mds} protocol is
comprised of two parts: headers and a payload.
When a client joins a multicast group it is
actually say that it is interested and receiving
broadcasts containing a specific header or a
specific header--value pair, or that it is
interesting in all messages@footnote{This
could be used for logging, possibly spying and
networking.}. Thus a message is automatically
multicasted to groups indicated by its headers.

The multicast groups and receiving of groups
is called interceptions. The interesting
property of interceptions is that they may
be modifying. When a server registers for
message interception it can say that it wants
to be able to modify messages. If this is done
and the server receives a message for which it
has said it want to be able to modify it,
the master server will wait for that server
to respond before it send the message to
the next server in the interception list.
The server can choose to do three things
with a message that it has opted in for
modification of: leave the message as-is,
modify the message, or consume the message.
A message consumption is done by modify
the message to make it empty. A consumed
message will not be send to any further
clients or servers in the interception list.

To make this mechanism sensible, a server or
client can set a priority when it registers
for interception (does not need to be
modifying.) When a message is broadcasted it
will be received by all servers in the
interception except the original sender,
unless it gets consumes. The order in which
the master server sends the message to the
recipients is determined by priority the
servers registed with. The message first sent
to the recipients with highest priority and
last to the recipients with lowestr priority,
and orderd by the priority between those
priorities. Of two or more servers have the
same priority the order in which they will
receive the message, of those recipients,
is arbitrary.

An interesting property of this machanism
is demonstrated in the @command{mds-vt}
server. Unlike most servers @command{mds-vt}
maintains two concurrent connections to
the display. Once @command{mds-vt} receives
a signal from the OS kernel requesting to
switch virtual terminal, @command{mds-vt}
will from one of its connections send
out a message and wait for it to be
received in its other connection and the
let the OS kernel switch virtual terminal.
The secondary connection to the display
has registered interception with lower
priority of the message that the primary
connection broadcasts. This message will
be received by other servers that will
let the message continue to the next
server in the interception list once that
server is ready for the OS kernel to switch
virtual terminal. All of these server has
registered modifying interception of the
message but none will actually modify or
consume the message; it is only used a
mechanism for letting @command{mds-vt} know
when all servers are ready for the switch
without having to know how many they are
and wait for a reply from all of them.



@node Protocol
@chapter Protocol

@menu
* Environment Variables::           Identifying the active display server
* Signals::                         Signalling individual servers
* Filesystem::                      The display server's footprint on the filesystem
* Message Passing::                 Sending messages between servers and clients
* Interception::                    Implementing protocols and writing unanticipated clients
@end menu

@node Environment Variables
@section Environment Variables

A crucial of any display server is letting child
processes know which display server they should
connect to. @command{X.org} does by setting the
environment variable @env{DISPLAY} to
@code{<host>:<display index>}, where @code{<host>}
is empty if the display is one the local machine.
In this tradition @command{mds} does the same thing
with the environment variable @env{MDS_DISPLAY}.

@command{mds} also creates a new process group and
export the new process group ID to the environment
variable @command{MDS_PGROUP}. This process group
can be used to send signals to all @command{mds}
servers collectively.



@node Signals
@section Signals

@command{mds} servers can re-execute into an
updated version of their binary. This can be
used to update display server online after
a new version has been installed. To do this
send the signal @command{SIGUSR1} to the server
you want update. If a server does not support
online updating it will ignore this signal.
If the operating system defines a signal named
@command{SIGUPDATE}, this signal is used
instead of @command{SIGUSR1}.

If you need servers to free up allocated
memory that they do not use, send the signal
@command{SIGDANGER}, or if not defined
@command{SIGRTMAX}. Unimportant servers may
choose to die on @command{SIGDANGER}.



@node Filesystem
@section Filesystem

The @command{mds} kernel creates two directories
for the @command{mds} servers to use: one for
runtime data and one for temporary data.
These directories are named by
@code{MDS_RUNTIME_ROOT_DIRECTORY} and
@code{MDS_STORAGE_ROOT_DIRECTORY}, respectively,
by the header file @file{<libmdsserver/config.h>}.
If the systems runtime data directory is @file{/run}
and transient temporary data directory is @file{/tmp},
and the package name of @command{mds} is @command{mds},
these directories will be @file{/run/mds} and
@file{/tmp/.@{system-directory@}.mds}, respectively.
In @file{/tmp/.@{system-directory@}.mds} the kernel
will create a directory for the display server instance
named @file{.data} prefixed by the display server index.
For example if the display server index is zero,
temporary data may be stored in
@file{/tmp/.@{system-directory@}.mds/0.data}

As defined by @code{SHM_PATH_PATTERN} by
@file{<libmdsserver/config.h>}, when a server
re-executes itself it will marshal its state to
the POSIX shared memory unit named
@file{/.proc-pid-%ji}, where @file{%ji}
@footnote{@code{%ji} is the pattern in @code{*printf}
functions for the data type @code{intmax_t}.} is
replaced with the process ID of the server. This
file will be bound to the pathname
@file{/dev/shm/.proc-pid-%ji} if POSIX shared
memory is stored in @file{/dev/shm} by the
operating system.

In @code{MDS_RUNTIME_ROOT_DIRECTORY} the kernel
will create two files. @file{.pid} and @file{.socket},
both prefixed with the display server index
@footnote{@file{0.pid} and @file{0.socket} if
the display server index is 0.}. The @file{.pid}
file contains the process ID of the display server
and is used by the kernel to figure out whether
an display server index is still in use or just
not properly cleaned up. Of course it can be used
by any program to find the process ID of the
kernel process of a display server instance.
The @file{.socket} is the domain socket used
for communication with the display server and
its servers and clients.



@node Message Passing
@section Message Passing

Message passing over domain sockets is the
underlaying technique for communicating with
the display server. To communicate with the
display server in the local machine a process
must connect to the domain socket created by
the display server kernel as named in
@ref{Filesystem}.

Clients should request a unique ID when it
connects to the display server.@footnote{There
is seldom a reason for servers to do this.}
To do this the client sends

@example
Command: assign-id\n
Message ID: 0\n
\n
@end example

where @code{\n} is an LF-line break.
The value on the @code{Message ID} line
does not need to be 0, but servers and
clients often start with 0 and count
upwards. The value is however bound to
an unsigned 32-bit integer. All message
must contain this @code{Message ID} header,
otherwise the message is considered corrupt
and is ignored.

The empty line signifies the end of the
header list, and in this case the end of
the message. But a message may contain
payload beneath this empty line. To
include a payload, add the header
@code{Length} that says how many bytes
the payload is comprised of.

A header must contain a header name and
header value without any trailing or
leading spaces, and `: ' (colon, one
regular blank space) exactly delimits
the name and the value.

When the master server receives this
@code{Command: assign-id} message it
will assign the client a unique ID
and send it to the client.@footnote{The
master server is the only server than
can address the client uniquely before
it has an ID, so this part can only
be implement in the master server.}
If the client already has an ID, it
will send back that ID to the client.
This response consists of two headers
@code{ID assignment} and @code{In
response to}, containing the client's
new (or possibly already assigned) ID
and the value that was in the
@code{Message ID} header, respectively.
For example:

@example
ID assignment: 0:1\n
In response to: 0\n
\n
@end example

Notice that the master server never
includes @code{Message ID} in message
originating from it.

As seen in this example, the client ID
consists of two integers delimited by
a colon (`:'). Both of these integers
are unsigned 32-bit integers. This is
done this way because unsigned 64-bit
integers are forbidden because it is
not supportable natively be some
programming languages.

Before a has gotten a unique client ID
assigned to it, it will be `0:0'.

If a client gets disconnected from the
master server, the master server will
sends out a signal header message.
This header will be @code{Client closed}
and contain ID of the client that closed.
For example:

@example
Client closed: 0:1\n
\n
@end example

Be aware that if a server or client
closes and does not have a unique client
ID, this message will be:

@example
Client closed: 0:0\n
\n
@end example

Once a client has an unique client ID
assigned to it, it should always include
the header @code{Client ID} in its
messages. The value of @code{Client ID}
should be the client's ID. If a server
wants to address this client, it should
include the header @code{To} with the
value set to the recipient's client ID.
Be aware that such message may not be
sent to that recipient uniquely, any
server or client is free to sign up
for receive of such message, any messages
or message contain any other header or
header--value pair that may also be
included in the header.



@node Interception
@section Interception

As discussed in @ref{Interprocess Communication},
interception in the primary feature of
@command{mds}'s message passing system.
Not only does it enable servers to select
which message it wants to receive in order
to provide it's service. It also enables
clients to do anything, things that was
never anticipated. As an exaple of its
power, @command{mds} does not provide any
protocol for taking screenshots or recording
a session. Instead, a screenshot application
signs up for messages pass between the
compositor and presentation servers, and
simply requests that the compositor resends
the screen, a feature intended for the
presentation servers. A screen recoding
application would do the same and just
hang on and record all message passed
between the servers.

If you want your server or client to
receive all messages passed around in
the display server, simply sign up for
all messages:

@example
Command: intercept\n
Message ID: 0\n
\n
@end example

But if you only want messages contain
the header @code{Command}, include
that header in the payload of the message:

@example
Command: intercept\n
Message ID: 0\n
Length: 8\n
\n
Command\n
@end example

It is allowed to include multiple headers.
You can also be more strict, and require
a specific value for a header, for example:

@example
Command: intercept\n
Message ID: 0\n
Length: 16\n
\n
Command: get-vt\n
@end example

You may mix these two types of requirements
freely. Your client will receive any message
that satisfies at least one of the requirements,
these requirements may be split into multiple
message or coalesced into one message; but
you cannot request to include receive a message
if multiple requirements are satisfied.

Alternatively you can choose to stop receiving
message that satisfies requirements. For example:

@example
Command: intercept\n
Stop: yes\n
Message ID: 1\n
Length: 16\n
\n
Command: get-vt\n
@end example

Or stop receiving all messages:

@example
Command: intercept\n
Stop: yes\n
Message ID: 1\n
\n
@end example

Note that this will stop you from receiving
messages contain the @code{To} header addressed
to you until you request to receiving such
messages again.

When you sign up for message you may request
to be able to modify them before that are
send to the next client in the list of client
that should receive them. To do this include
the header--value pair @code{Modifying: yes}:

@example
Command: intercept\n
Modifying: yes\n
Message ID: 0\n
Length: 30\n
\n
Command: keyboard-enumeration\n
@end example

It is up to the client to keep track of
which message that it may modify. When
you receive a message that you can modify
you must respond when you are done with
the message.

For example, if you have signed up
for @code{Command: keyboard-enumeration}
with the ability to modify such messages
and the message

@example
Command: keyboard-enumeration\n
To: 0:1\n
In response to: 2\n
Message ID: 1\n
Length: 7\n
\n
kernel\n
@end example

is send from a server, you may receive
it as

@example
Command: keyboard-enumeration\n
To: 0:1\n
In response to: 2\n
Message ID: 1\n
Length: 7\n
Modify ID: 4\n
\n
kernel\n
@end example

Be aware that the @code{Modify ID} may
be included even if you have not signed
up to be able to modify the message,
it is enough that one client before you
has or it was originally included
@footnote{You may however not include
this header when you send out an
orginal message}.

If you receive the message as such
and want to add the line
@code{on-screen-keyboard-20376} to
the payload should send out:
@footnote{The first line containing
starting with @code{Message ID} is an
example, it should be whatever is
appropriate for your client.}

@example
Modify ID: 4\n
Message ID: 2\n
Modify: yes\n
Length: 127\n
\n
Command: keyboard-enumeration\n
To: 0:1\n
In response to: 2\n
Message ID: 1\n
Length: 32\n
Modify ID: 4\n
\n
kernel\n
on-screen-keyboard-20376\n
@end example

If you however decide not to modify
the message send out

@example
Modify ID: 4\n
Message ID: 2\n
Modify: no\n
\n
@end example

There is also a third option:
to consume to the message. This
stops any further clients from
receiving the message. This is
done by modifying the message
into an empty message:

@example
Modify ID: 4\n
Message ID: 2\n
Modify: yes\n
\n
@end example

You may choose to include the
header--value pair @code{Length: 0},
it is however redundant and
discouraged.

This mechanism of being able to
modify message does not make much
sense unless you can control in
the order the clients receive
messages. This is done with what
is called priority. The higher
priority you have, the earlier
you will receive the message. The
default priority is zero, and the
priority is bound to a signed
64-bit integer. If you want to
be able to list yourself in
@code{Command: keyboard-enumeration}
message, you should sign up
with a positive priority since
the final recipient or requested
the enumeration will receive it
with priority zero. Therefore
you should sign up for such message
with a message like:
@footnote{4611686018427387904 is
halfway to the maximium value.}

@example
Command: intercept\n
Modifying: yes\n
Priority: 4611686018427387904\n
Message ID: 0\n
Length: 30\n
\n
Command: keyboard-enumeration\n
@end example



@node libmdsserver
@chapter libmdsserver

libmdsserver is library written for the reference
implementation of the @command{mds} servers.
libmdsserver does not contain support or any
protocols, rather it contains auxiliary functions,
macros, data structures such as linked lists and
hash tables, and support the basics of the message
passing protocol: receiving message and decode it
into headers and payloads.

The header file @file{<libmdsserver/macros.h>}
contains macros for readability and code reduction,
it also contains macros and definitions for portability;
they may either provide portability by nature, or
provide one place to do modifications to port the
system.

@table @asis
@item @code{xsnprintf} [(@code{char[] buffer, char* format, ...}) @arrow{} @code{int}]
This is a wrapper for @code{snprintf} that allows you
to forget about the buffer size. When you know how long
a string can be, you should use @code{sprintf}. But when
you cannot know for sure you should use @code{xsnprintf}.
@code{xsnprintf} works exactly as @code{sprintf}, but
it will require that the first argument is defined
using @code{[]} rather than @code{*} because it will use
this to find out how large the buffer is so it can call
@code{snprintf} with that size.

@item @code{eprint} [(@code{const char* format}) @arrow{} @code{int}]
A wrapper for @code{fprintf} that prints a string prefixed
with the value value of @code{*argv} to @code{stderr}.
Because @code{eprintf} naïvely wraps @code{fprintf}, all
`%':s in the string must be duplicated.

@item @code{eprintf} [(@code{const char* format, ...}) @arrow{} @code{int}]
@code{eprint} extends @code{eprint} with variadic arguments
that can be used to insert values into the format string
just like you can do in @code{fprintf}.

@item @code{with_mutex} [(@code{pthread_mutex_t mutex, instructions})]
Wraps @code{instructions} with @code{errno = pthread_mutex_lock(mutex);}
and @code{errno = pthread_mutex_unlock(mutex);}, so a set of
instructions can be invoked inside mutex protection.

@item @code{with_mutex_if} [(@code{pthread_mutex_t mutex, condition, instructions})]
An alternative to @code{with_mutex} where @code{instructions}
is wrapped around @code{if (condition)} which in turn is
wrapped inside the mutex protection.

@item @code{max} [(@code{a, b})]
Returns the higher value of @code{a} and @code{b}.

@item @code{min} [(@code{a, b})]
Returns the lower value of @code{a} and @code{b}.

@item @code{buf_cast} [(@code{char* buffer, type, size_t index})]
Casts @code{buffer} to a @code{type} buffer and
subscripts to the @code{index}:th element. You
can either use this function as a getter or a
setter.

@item @code{buf_set} [(@code{char* buffer, type, size_t index, type variable}) @arrow{} @code{type}]
Wrapper for @code{buf_cast} that sets the addressed
element to the value of @code{variable}.

@item @code{buf_get} [(@code{const char* buffer, type, size_t index, type variable}) @arrow{} @code{type}]
Wrapper for @code{buf_cast} that sets the value of
@code{variable} to the value of the addressed element.

@item @code{buf_next} [(@code{char* buffer, type, size_t count}) @arrow{} @code{char*}]
Increases the pointer @code{buffer} by the size of
@code{type} @code{count} types.

@item @code{buf_prev} [(@code{char* buffer, type, size_t count}) @arrow{} @code{char*}]
Decreases the pointer @code{buffer} by the size of
@code{type} @code{count} types.

@item @code{buf_set_next} [(@code{char* buffer, type, type variable}) @arrow{} @code{type}]
@example
buf_set(buffer, type, 0, variable),
buf_next(buffer, type, 1);
@end example

@item @code{buf_get_next} [(@code{char* buffer, type, type variable}) @arrow{} @code{type}]
@example
buf_get(buffer, type, 0, variable),
buf_next(buffer, type, 1);
@end example

@item @code{strequals} [(@code{const char* a, const char* b}) @arrow{} @code{int}]
Evaluates whether the strings @code{a} and @code{b}
are equals, neither may be @code{NULL}.

@item @code{startswith} [(@code{const char* haystack, const char* needle}) @arrow{} @code{int}]
Evaluates whether the string @code{haystack}
starts with the string @code{needle}, neither
may be @code{NULL}.

@item @code{drop_privileges} [() @arrow{} @code{int}]
Sets the effective user to the real user and the
effective group to the real group. This is used
by most servers and ensure that they are not
running with unnecessary privileges. Returns zero
on and only on success.

@item @code{monotone} [(@code{struct timespec* time_slot}) @arrow{} @code{int}]
Stores the time of an unspecified monotonic clock
into @code{time_slot}. Returns zero on and only on
success.

@item @code{close_files} [(@code{condition}) @arrow{} @code{void}]
Closes all file descriptors named by a variable
@code{fd} for which @code{condition} evalutes
to non-zero.

@item @code{xfree} [(@code{void** array, size_t elements}) @arrow{} @code{void}]
Calls @code{free} on the first @code{elements}
elements in @code{array}, and than calls
@code{free} on @code{array}. This macro
requires @code{size_t i} is declared.

@item @code{xmalloc} [(@code{type* var, size_t elements, type}) @arrow{} @code{int}]
@item @code{xcalloc} [(@code{type* var, size_t elements, type}) @arrow{} @code{int}]
@item @code{xrealloc} [(@code{type* var, size_t elements, type}) @arrow{} @code{int}]
@item @code{growalloc} [(@code{type* old, type* var, size_t elements, type}) @arrow{} @code{int}]
@item @code{xperror} [(@code{const char* str}) @arrow{} @code{void}]
@item @code{fail_if} [(@code{condition}) @arrow{} @code{void}]
@item @code{exit_if} [(@code{condition, instructions}) @arrow{} @code{void}]
@end table

@c SIGDANGER SIGUPDATE macro-bits.h



@node GNU Free Documentation License
@appendix GNU Free Documentation License
@include fdl.texinfo

@bye