\input texinfo @c -*-texinfo-*- @c %**start of header @setfilename mds.info @settitle mds @afourpaper @documentencoding UTF-8 @documentlanguage en @finalout @c %**end of header @dircategory Graphics environment @direntry * mds: (mds). The micro-display server @end direntry @copying Copyright @copyright{} 2014 Mattias Andrée @quotation Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled ``GNU Free Documentation License''. @end quotation @end copying @ifnottex @node Top @top mds -- The micro-display server @insertcopying @end ifnottex @titlepage @title mds @subtitle The micro-display server @author by Mattias Andrée (maandree) @page @c @center `' @vskip 0pt plus 1filll @insertcopying @end titlepage @contents @menu * Overview:: Brief overview of @command{mds}. * Architecture:: Architectural overview of @command{mds}. * Protocol:: The @command{mds} procotol. * libmdsserver:: Overview of @command{libmdsserver}. * GNU Free Documentation License:: Copying and sharing this manual. @end menu @node Overview @chapter Overview @command{mds}@footnote{mds stands for micro-display server} is a display server protocol and an implementation of said protocol. What makes @command{mds} stand out is its core design choice: it is desigend just like a microkernel. Rather than one, possibly modular, process --- a monolithic process --- mds is comprised of many small servers, each exchangable and responsible for one thing. @command{mds} goal is neither security, performance nor a perfect graphical experience. @command{mds} is all about flexibility and freedom 0@footnote{The freedom to run the program as you wish, for any purpose}. The reason for having a display server architectured as a microkernel is so that components can be added, remove and replaced online. Additionally, the message passing between the servers makes it easy to design a system that lets you make clients that can listen on messages between the servers and perhaps modify them. This enables you to do so much more with your display server. Moreover, if a single part of the system crashes it does not bring down the whole system, and the crashed server can be respawned with minor side effects. @command{mds} is architectured in three layers: a microkernel, a master server and a collection of servers. And clients are actually located on the same layer as the servers, because there is no actual difference, the only thing that separates a server from a client is for what purpose you run it. @command{mds}'s kernel is a minimal program that do initialisation of the display, such as giving it an index and create runtime files and directories for servers and other programs to use. Then the kernel creates a domain socket for the master server and spawns the master server and respawns it if it crashes. Because of this, if the master server crashes it will not lose its socket when it is respawned. The master server than, on its initial spawn, starts the all servers and other programs that the user have choosen and then starts accepting connections to it and coordinates messages between servers and clients. Further, separating all components into separate processes enables us to only give the servers the privileges they actually need, rather than having one program with root privileges that takes care of everything even things that do not do require any privileges. All @command{mds}'s servers, that is all running parts of @command{mds} except the kernel, are designed so that they can re-exec themself so that they can be updated online without any side effects. Servers serialises their state, saves it to RAM (in a directory created by the kernel), re-exec themself and loads their serialised state. The kernel cannot do this because when it has spawned the master server it has no reason to re-exec, its only mission is to respawn the master server it if would happen to crash. It would technically be possible to enable the kernel to re-exec but it is not worth it as it as no reason to re-exec, and doing so puts the display server at risk of crashing. @node Architecture @chapter Architecture @menu * Layers:: The layers of the display server. * Interprocess Communication:: How servers and clients communicate. @end menu @node Layers @section Layers The @command{mds} display server in architectured in three layers. The first layer is called the kernel. The kernel is responsible for acquiring a display server index@footnote{As with any display server, the system can have multiple instances of @command{mds} running at the same time.}, set up environment variables to indicate which display server and display server instance is being used, create a domain socket for the display server and start the master server and restart it if it crashes, and then clean up the system when the display server closes. The kernel only responsible for creating the domain socket for communication with the display server, it is not responsible for using it, that mission falls to the master server. The second layer is the master server. The master server has two responsibilities: coordinating message passing between other servers and clients @footnote{In @command{mds} their is no functional distinction between servers and clients, the distinction is purely semantic.} and starting other servers. The third layer is the other servers and clients. protocolwise there is no specification on how they are started. But in the reference implementation of the master server, this is done by starting a shell script with the pathname @file{$@{XDG_CONFIG_HOME@}/mdsinitrc} and the user is responsible for providing the logic in that shell script.@footnote{Moonstruck users are allowed to implement this in C or any other language of their choosing.} @c Which is better: cray-cray users, lunatic users, @c moonstruck users, insane users, ballers, madmen, @c loony tunes? These servers implements the actual functionality of the display server. @node Interprocess Communication @section Interprocess Communication Intrinsic to @command{mds} is a powerful interprocess communication mechanism. Servers and clients connect to the display server by connecting to a domain socket served by the master server. A server or client that has connected to the display server can do three things: @itemize @item Request assignment of a unique ID. @item Multicast a message. @item Join or leave a multicast groups. @end itemize Upon assignment of an ID the master server will automatically place the client in a multicast group for that specific client. This automatically multicast group assignment is done by the master server simply so you as a debugger do not forget to do so. When a client is disconnected it will and out a message to a specific multicast group that the client, refered to by it's ID, have closed. A message in the @command{mds} protocol is comprised of two parts: headers and a payload. When a client joins a multicast group it is actually say that it is interested and receiving broadcasts containing a specific header or a specific header--value pair, or that it is interesting in all messages@footnote{This could be used for logging, possibly spying and networking.}. Thus a message is automatically multicasted to groups indicated by its headers. The multicast groups and receiving of groups is called interceptions. The interesting property of interceptions is that they may be modifying. When a server registers for message interception it can say that it wants to be able to modify messages. If this is done and the server receives a message for which it has said it want to be able to modify it, the master server will wait for that server to respond before it send the message to the next server in the interception list. The server can choose to do three things with a message that it has opted in for modification of: leave the message as-is, modify the message, or consume the message. A message consumption is done by modify the message to make it empty. A consumed message will not be send to any further clients or servers in the interception list. To make this mechanism sensible, a server or client can set a priority when it registers for interception (does not need to be modifying.) When a message is broadcasted it will be received by all servers in the interception except the original sender, unless it gets consumes. The order in which the master server sends the message to the recipients is determined by priority the servers registed with. The message first sent to the recipients with highest priority and last to the recipients with lowestr priority, and orderd by the priority between those priorities. Of two or more servers have the same priority the order in which they will receive the message, of those recipients, is arbitrary. An interesting property of this machanism is demonstrated in the @command{mds-vt} server. Unlike most servers @command{mds-vt} maintains two concurrent connections to the display. Once @command{mds-vt} receives a signal from the OS kernel requesting to switch virtual terminal, @command{mds-vt} will from one of its connections send out a message and wait for it to be received in its other connection and the let the OS kernel switch virtual terminal. The secondary connection to the display has registered interception with lower priority of the message that the primary connection broadcasts. This message will be received by other servers that will let the message continue to the next server in the interception list once that server is ready for the OS kernel to switch virtual terminal. All of these server has registered modifying interception of the message but none will actually modify or consume the message; it is only used a mechanism for letting @command{mds-vt} know when all servers are ready for the switch without having to know how many they are and wait for a reply from all of them. @node Protocol @chapter Protocol @menu * Environment Variables:: Identifying the active display server * Signals:: Signalling individual servers * Filesystem:: The display server's footprint on the filesystem * Message Passing:: Sending messages between servers and clients * Interception:: Implementing protocols and writing unanticipated clients @end menu @node Environment Variables @section Environment Variables A crucial of any display server is letting child processes know which display server they should connect to. @command{X.org} does by setting the environment variable @env{DISPLAY} to @code{:}, where @code{} is empty if the display is one the local machine. In this tradition @command{mds} does the same thing with the environment variable @env{MDS_DISPLAY}. @command{mds} also creates a new process group and export the new process group ID to the environment variable @command{MDS_PGROUP}. This process group can be used to send signals to all @command{mds} servers collectively. @node Signals @section Signals @command{mds} servers can re-execute into an updated version of their binary. This can be used to update display server online after a new version has been installed. To do this send the signal @command{SIGUSR1} to the server you want update. If a server does not support online updating it will ignore this signal. If the operating system defines a signal named @command{SIGUPDATE}, this signal is used instead of @command{SIGUSR1}. If you need servers to free up allocated memory that they do not use, send the signal @command{SIGDANGER}, or if not defined @command{SIGRTMAX}. Unimportant servers may choose to die on @command{SIGDANGER}. @node Filesystem @section Filesystem The @command{mds} kernel creates two directories for the @command{mds} servers to use: one for runtime data and one for temporary data. These directories are named by @code{MDS_RUNTIME_ROOT_DIRECTORY} and @code{MDS_STORAGE_ROOT_DIRECTORY}, respectively, by the header file @file{}. If the systems runtime data directory is @file{/run} and transient temporary data directory is @file{/tmp}, and the package name of @command{mds} is @command{mds}, these directories will be @file{/run/mds} and @file{/tmp/.@{system-directory@}.mds}, respectively. In @file{/tmp/.@{system-directory@}.mds} the kernel will create a directory for the display server instance named @file{.data} prefixed by the display server index. For example if the display server index is zero, temporary data may be stored in @file{/tmp/.@{system-directory@}.mds/0.data} As defined by @code{SHM_PATH_PATTERN} by @file{}, when a server re-executes itself it will marshal its state to the POSIX shared memory unit named @file{/.proc-pid-%ji}, where @file{%ji} @footnote{@code{%ji} is the pattern in @code{*printf} functions for the data type @code{intmax_t}.} is replaced with the process ID of the server. This file will be bound to the pathname @file{/dev/shm/.proc-pid-%ji} if POSIX shared memory is stored in @file{/dev/shm} by the operating system. In @code{MDS_RUNTIME_ROOT_DIRECTORY} the kernel will create two files. @file{.pid} and @file{.socket}, both prefixed with the display server index @footnote{@file{0.pid} and @file{0.socket} if the display server index is 0.}. The @file{.pid} file contains the process ID of the display server and is used by the kernel to figure out whether an display server index is still in use or just not properly cleaned up. Of course it can be used by any program to find the process ID of the kernel process of a display server instance. The @file{.socket} is the domain socket used for communication with the display server and its servers and clients. @node Message Passing @section Message Passing Message passing over domain sockets is the underlaying technique for communicating with the display server. To communicate with the display server in the local machine a process must connect to the domain socket created by the display server kernel as named in @ref{Filesystem}. Clients should request a unique ID when it connects to the display server.@footnote{There is seldom a reason for servers to do this.} To do this the client sends @example Command: assign-id\n Message ID: 0\n \n @end example where @code{\n} is an LF-line break. The value on the @code{Message ID} line does not need to be 0, but servers and clients often start with 0 and count upwards. The value is however bound to an unsigned 32-bit integer. All message must contain this @code{Message ID} header, otherwise the message is considered corrupt and is ignored. The empty line signifies the end of the header list, and in this case the end of the message. But a message may contain payload beneath this empty line. To include a payload, add the header @code{Length} that says how many bytes the payload is comprised of. A header must contain a header name and header value without any trailing or leading spaces, and `: ' (colon, one regular blank space) exactly delimits the name and the value. When the master server receives this @code{Command: assign-id} message it will assign the client a unique ID and send it to the client.@footnote{The master server is the only server than can address the client uniquely before it has an ID, so this part can only be implement in the master server.} If the client already has an ID, it will send back that ID to the client. This response consists of two headers @code{ID assignment} and @code{In response to}, containing the client's new (or possibly already assigned) ID and the value that was in the @code{Message ID} header, respectively. For example: @example ID assignment: 0:1\n In response to: 0\n \n @end example Notice that the master server never includes @code{Message ID} in message originating from it. As seen in this example, the client ID consists of two integers delimited by a colon (`:'). Both of these integers are unsigned 32-bit integers. This is done this way because unsigned 64-bit integers are forbidden because it is not supportable natively be some programming languages. Before a has gotten a unique client ID assigned to it, it will be `0:0'. If a client gets disconnected from the master server, the master server will sends out a signal header message. This header will be @code{Client closed} and contain ID of the client that closed. For example: @example Client closed: 0:1\n \n @end example Be aware that if a server or client closes and does not have a unique client ID, this message will be: @example Client closed: 0:0\n \n @end example Once a client has an unique client ID assigned to it, it should always include the header @code{Client ID} in its messages. The value of @code{Client ID} should be the client's ID. If a server wants to address this client, it should include the header @code{To} with the value set to the recipient's client ID. Be aware that such message may not be sent to that recipient uniquely, any server or client is free to sign up for receive of such message, any messages or message contain any other header or header--value pair that may also be included in the header. @node Interception @section Interception As discussed in @ref{Interprocess Communication}, interception in the primary feature of @command{mds}'s message passing system. Not only does it enable servers to select which message it wants to receive in order to provide it's service. It also enables clients to do anything, things that was never anticipated. As an exaple of its power, @command{mds} does not provide any protocol for taking screenshots or recording a session. Instead, a screenshot application signs up for messages pass between the compositor and presentation servers, and simply requests that the compositor resends the screen, a feature intended for the presentation servers. A screen recoding application would do the same and just hang on and record all message passed between the servers. If you want your server or client to receive all messages passed around in the display server, simply sign up for all messages: @example Command: intercept\n Message ID: 0\n \n @end example But if you only want messages contain the header @code{Command}, include that header in the payload of the message: @example Command: intercept\n Message ID: 0\n Length: 8\n \n Command\n @end example It is allowed to include multiple headers. You can also be more strict, and require a specific value for a header, for example: @example Command: intercept\n Message ID: 0\n Length: 16\n \n Command: get-vt\n @end example You may mix these two types of requirements freely. Your client will receive any message that satisfies at least one of the requirements, these requirements may be split into multiple message or coalesced into one message; but you cannot request to include receive a message if multiple requirements are satisfied. Alternatively you can choose to stop receiving message that satisfies requirements. For example: @example Command: intercept\n Stop: yes\n Message ID: 1\n Length: 16\n \n Command: get-vt\n @end example Or stop receiving all messages: @example Command: intercept\n Stop: yes\n Message ID: 1\n \n @end example Note that this will stop you from receiving messages contain the @code{To} header addressed to you until you request to receiving such messages again. When you sign up for message you may request to be able to modify them before that are send to the next client in the list of client that should receive them. To do this include the header--value pair @code{Modifying: yes}: @example Command: intercept\n Modifying: yes\n Message ID: 0\n Length: 30\n \n Command: keyboard-enumeration\n @end example It is up to the client to keep track of which message that it may modify. When you receive a message that you can modify you must respond when you are done with the message. For example, if you have signed up for @code{Command: keyboard-enumeration} with the ability to modify such messages and the message @example Command: keyboard-enumeration\n To: 0:1\n In response to: 2\n Message ID: 1\n Length: 7\n \n kernel\n @end example is send from a server, you may receive it as @example Command: keyboard-enumeration\n To: 0:1\n In response to: 2\n Message ID: 1\n Length: 7\n Modify ID: 4\n \n kernel\n @end example Be aware that the @code{Modify ID} may be included even if you have not signed up to be able to modify the message, it is enough that one client before you has or it was originally included @footnote{You may however not include this header when you send out an orginal message}. If you receive the message as such and want to add the line @code{on-screen-keyboard-20376} to the payload should send out: @footnote{The first line containing starting with @code{Message ID} is an example, it should be whatever is appropriate for your client.} @example Modify ID: 4\n Message ID: 2\n Modify: yes\n Length: 127\n \n Command: keyboard-enumeration\n To: 0:1\n In response to: 2\n Message ID: 1\n Length: 32\n Modify ID: 4\n \n kernel\n on-screen-keyboard-20376\n @end example If you however decide not to modify the message send out @example Modify ID: 4\n Message ID: 2\n Modify: no\n \n @end example There is also a third option: to consume to the message. This stops any further clients from receiving the message. This is done by modifying the message into an empty message: @example Modify ID: 4\n Message ID: 2\n Modify: yes\n \n @end example You may choose to include the header--value pair @code{Length: 0}, it is however redundant and discouraged. This mechanism of being able to modify message does not make much sense unless you can control in the order the clients receive messages. This is done with what is called priority. The higher priority you have, the earlier you will receive the message. The default priority is zero, and the priority is bound to a signed 64-bit integer. If you want to be able to list yourself in @code{Command: keyboard-enumeration} message, you should sign up with a positive priority since the final recipient or requested the enumeration will receive it with priority zero. Therefore you should sign up for such message with a message like: @footnote{4611686018427387904 is halfway to the maximium value.} @example Command: intercept\n Modifying: yes\n Priority: 4611686018427387904\n Message ID: 0\n Length: 30\n \n Command: keyboard-enumeration\n @end example @node libmdsserver @chapter libmdsserver libmdsserver is library written for the reference implementation of the @command{mds} servers. libmdsserver does not contain support or any protocols, rather it contains auxiliary functions, macros, data structures such as linked lists and hash tables, and support the basics of the message passing protocol: receiving message and decode it into headers and payloads. @menu * Macros:: Writing macroscopic systems. * Auxiliary Functions:: Auxiliary functions for servers. * Data Structures:: Data structures available in libmdsserver. @end menu @node Macros @section Macros The header file @file{} contains macros for readability and code reduction, it also contains macros and definitions for portability; they may either provide portability by nature, or provide one place to do modifications to port the system. @table @asis @item @code{xsnprintf} [(@code{char buffer[], char* format, ...}) @arrow{} @code{int}] This is a wrapper for @code{snprintf} that allows you to forget about the buffer size. When you know how long a string can be, you should use @code{sprintf}. But when you cannot know for sure you should use @code{xsnprintf}. @code{xsnprintf} works exactly as @code{sprintf}, but it will require that the first argument is defined using @code{[]} rather than @code{*} because it will use this to find out how large the buffer is so it can call @code{snprintf} with that size. @item @code{eprint} [(@code{const char* format}) @arrow{} @code{int}] A wrapper for @code{fprintf} that prints a string prefixed with the value value of @code{*argv} to @code{stderr}. Because @code{eprintf} naïvely wraps @code{fprintf}, all `%':s in the string must be duplicated. @item @code{eprintf} [(@code{const char* format, ...}) @arrow{} @code{int}] @code{eprint} extends @code{eprint} with variadic arguments that can be used to insert values into the format string just like you can do in @code{fprintf}. @item @code{with_mutex} [(@code{pthread_mutex_t mutex, instructions})] Wraps @code{instructions} with @code{errno = pthread_mutex_lock(mutex);} and @code{errno = pthread_mutex_unlock(mutex);}, so a set of instructions can be invoked inside mutex protection. @item @code{with_mutex_if} [(@code{pthread_mutex_t mutex, condition, instructions})] An alternative to @code{with_mutex} where @code{instructions} is wrapped around @code{if (condition)} which in turn is wrapped inside the mutex protection. @item @code{max} [(@code{a, b})] Returns the higher value of @code{a} and @code{b}. @item @code{min} [(@code{a, b})] Returns the lower value of @code{a} and @code{b}. @item @code{buf_cast} [(@code{char* buffer, type, size_t index})] Casts @code{buffer} to a @code{type} buffer and subscripts to the @code{index}:th element. You can either use this function as a getter or a setter. @item @code{buf_set} [(@code{char* buffer, type, size_t index, type variable}) @arrow{} @code{type}] Wrapper for @code{buf_cast} that sets the addressed element to the value of @code{variable}. @item @code{buf_get} [(@code{const char* buffer, type, size_t index, type variable}) @arrow{} @code{type}] Wrapper for @code{buf_cast} that sets the value of @code{variable} to the value of the addressed element. @item @code{buf_next} [(@code{char* buffer, type, size_t count}) @arrow{} @code{char*}] Increases the pointer @code{buffer} by the size of @code{type} @code{count} types. @item @code{buf_prev} [(@code{char* buffer, type, size_t count}) @arrow{} @code{char*}] Decreases the pointer @code{buffer} by the size of @code{type} @code{count} types. @item @code{buf_set_next} [(@code{char* buffer, type, type variable}) @arrow{} @code{type}] @example buf_set(buffer, type, 0, variable), buf_next(buffer, type, 1); @end example @item @code{buf_get_next} [(@code{char* buffer, type, type variable}) @arrow{} @code{type}] @example buf_get(buffer, type, 0, variable), buf_next(buffer, type, 1); @end example @item @code{strequals} [(@code{const char* a, const char* b}) @arrow{} @code{int}] Evaluates whether the strings @code{a} and @code{b} are equals, neither may be @code{NULL}. @item @code{startswith} [(@code{const char* haystack, const char* needle}) @arrow{} @code{int}] Evaluates whether the string @code{haystack} starts with the string @code{needle}, neither may be @code{NULL}. @item @code{drop_privileges} [(void) @arrow{} @code{int}] Sets the effective user to the real user and the effective group to the real group. This is used by most servers and ensure that they are not running with unnecessary privileges. Returns zero on and only on success. @item @code{monotone} [(@code{struct timespec* time_slot}) @arrow{} @code{int}] Stores the time of an unspecified monotonic clock into @code{time_slot}. Returns zero on and only on success. @item @code{close_files} [(@code{condition}) @arrow{} @code{void}] Closes all file descriptors named by a variable @code{fd} for which @code{condition} evalutes to non-zero. @item @code{xfree} [(@code{void** array, size_t elements}) @arrow{} @code{void}] Calls @code{free} on the first @code{elements} elements in @code{array}, and than calls @code{free} on @code{array}. This macro requires @code{size_t i} is declared. @item @code{xmalloc} [(@code{type* var, size_t elements, type}) @arrow{} @code{int}] Allocates a @code{type*} with @code{elements} elements and store the allocated pointer to @code{var}. Returns zero on and only on success. @item @code{xcalloc} [(@code{type* var, size_t elements, type}) @arrow{} @code{int}] Allocates a zero-initialised @code{type*} with @code{elements} elements and store the allocated pointer to @code{var}. Returns zero on and only on success. @item @code{xrealloc} [(@code{type* var, size_t elements, type}) @arrow{} @code{int}] Reallocates @code{var} and updates the variable @code{var} accordingly. @code{var} will be allocated to have @code{elements} elements of the type @code{type}. If @code{var} is @code{NULL} a new allocation is created. If @code{elements} is zero, @code{var} will be deallocated. Returns zero on and only on success. On failure, @code{var} will be @code{NULL}, so you must store the @code{var} into another variable in case this macro fails. @item @code{growalloc} [(@code{type* old, type* var, size_t elements, type}) @arrow{} @code{int}] When using this macro @code{var} should be a @code{type*} pointer allocated for @code{elements} elements of the type @code{type}. This macro will reallocate @code{var} to contain twice as many elements and update @code{elements} accordingly. On failure nothing changes. You must specify an auxiliary @code{type*} variable and specify it in as the @code{old} parameter. Returns zero on and only on success. @item @code{xperror} [(@code{const char* str}) @arrow{} @code{void}] Invokes @code{perror(str)} if and only if @code{errno} is non-zero and then sets @code{errno} to zero. @code{str} should unless you have a specific reason be @code{*argv}. @item @code{fail_if} [(@code{condition}) @arrow{} @code{void}] If @code{condition} is satisfied, a jump is made to the label @code{pfail}. @code{pfail:} should be used for calling @code{xperror} and return @code{-1}. @item @code{exit_if} [(@code{condition, instructions}) @arrow{} @code{void}] If @code{condition} is satisfied, @code{instructions} is invoked and @code{1} is @code{return}:ed. @end table Additionally, @file{} defines any missing signal name: currenly @code{SIGDANGER} and @code{SIGUPDATE}, and by inclusion of @file{}, variants of @code{atoi} for portability and convenience: @table @code @item atoz Parse a human readable @code{const char*} 10-radix integer to a @code{size_t}. @item atosz Parse a human readable @code{const char*} 10-radix integer to a @code{ssize_t}. @item atoh Parse a human readable @code{const char*} 10-radix integer to a @code{short int}. @item atouh Parse a human readable @code{const char*} 10-radix integer to an @code{unsigned short int}. @item atou Parse a human readable @code{const char*} 10-radix integer to an @code{unsigned int}. @item atoul Parse a human readable @code{const char*} 10-radix integer to an @code{unsigned long int}. @item atoull Parse a human readable @code{const char*} 10-radix integer to an @code{unsigned long long int}. @item ato8 Parse a human readable @code{const char*} 10-radix integer to an @code{int8_t}. @item atou8 Parse a human readable @code{const char*} 10-radix integer to an @code{uint8_t}. @item ato16 Parse a human readable @code{const char*} 10-radix integer to an @code{int16_t}. @item atou16 Parse a human readable @code{const char*} 10-radix integer to an @code{uint16_t}. @item ato32 Parse a human readable @code{const char*} 10-radix integer to an @code{int32_t}. @item atou32 Parse a human readable @code{const char*} 10-radix integer to an @code{uint32_t}. @item ato64 Parse a human readable @code{const char*} 10-radix integer to an @code{int64_t}. @item atou64 Parse a human readable @code{const char*} 10-radix integer to an @code{uint64_t}. @item atoj Parse a human readable @code{const char*} 10-radix integer to an @code{intmax_t}. @item atouj Parse a human readable @code{const char*} 10-radix integer to an @code{uintmax_t}. @end table @node Auxiliary Functions @section Auxiliary Functions In the header file @file{}, libmdsserver defines common functions to help write servers more concisely. @table @asis @item @code{parse_client_id} [(@code{const char* str}) @arrow{} @code{uint64_t}] Convert a client ID string into a client ID integer. @item @code{getenv_nonempty} [(@code{const char* var}) @arrow{} @code{char*}] Read an environment variable, return @code{NULL} if the variable's value is an empty string. @item @code{prepare_reexec} [(@code{void}) @arrow{} @code{int}] Prepare the server so that it can re-execute into a newer version of the executed file. This is required for two reasons: @enumerate 1 @item We cannot use @code{argv[0]} as @env{PATH}-resolution may cause it to reexec into another pathname, and maybe to wrong program. Additionally @code{argv[0]} may not even refer to the program, and @code{chdir} could also hinter its use. @item The kernel appends ` (deleted)' to @file{/proc/self/exe} once it has been removed, so it cannot be replaced. @end enumerate The function will should be called immediately, it will store the content of @file{/proc/self/exe}. Return zero on success and @code{-1} on error. @item @code{reexec_server} [(@code{int argc, char** argv, int reexeced}) @arrow{} @code{void}] Re-exec the server. This function only returns on failure. If `prepare_reexec` failed or has not been called, `argv[0]` will be used as a fallback. param argc The number of elements in `argv` param argv The command line arguments param reexeced Whether the server has previously been re-exec:ed @item @code{xsigaction} [(@code{int signo, void (*function)(int signo)}) @arrow{} @code{int}] @code{sigaction} with the same parameters as @code{signal}. This function should only be used for common @command{mds} signals and signals that does not require any special settings. This function may choose to add additional behaviour depending on the signal, such as blocking other signals. Returns zero on success and @code{-1} on error. @item @code{send_message} [(@code{int socket, const char* message, size_t length}) @arrow{} @code{size_t}] Send the message @code{messsage}, of length @code{length} over the socket that is access with the file descriptor @code{socket}. Returns the number of bytes that have been sent, even on error. @item @code{strict_atoi} [(@code{const char* str, int* value, int min, int max}) @arrow{} @code{int}] A version of @code{atoi} that is strict about the syntax and bounds. Parses the string @code{str} into an @code{int} and stores it in @code{*value}. If the string is not a 10-radix integer or has a value outside [@code{min}, @code{max}], @code{-1} is returned, otherwise zero is returned. @item @code{full_write} [(@code{int fd, const char* buffer, size_t length}) @arrow{} @code{int}] Send the buffer @code{buffer}, with the length @code{length}, into the file whose file descriptor is @code{fd} and ignores interruptions. Returns zero on success and @code{-1} on error. @item @code{full_read} [(@code{int fd, size_t* length}) @arrow{} @code{char*}] Read the file whose file descriptor is @code{fd} completely and ignore interruptions. If @code{length} if not @code{NULL}, the length of the read file is stored in @code{*length}. On success, the read content is retured, on error @code{NULL} is returned. @item @code{startswith_n} [(@code{const char*, const char*, size_t, size_t}) @arrow{} @code{int}] Check whether a string begins with a specific string, where neither of the strings are necessarily NUL-terminated. The parameters are: @table @code @item const char* haystack The string that should start with the other string. @item const char* needle The string the first string should start with. @item size_t haystack_n The length of @code{haystack}. @item size_t needle_n The length of @code{needle}. @end table Returns 1 if @code{haystack} beings with @code{needle}, otherwise zero is returned. @item @code{uninterruptable_waitpid} [(@code{pid_t pid, int* restrict status, int options}) @arrow{} @code{pid_t}] Wrapper around @code{waitpid} that never returns on an interruption unless it is interrupted one hundred times within the same clock second. The parameters and return value are exactly those of @code{waitpid}. @end table @node Data Structures @section Data Structures libmdsserver provides a small set of datastructures that are used by the @command{mds} servers. All of these are written with marshal-functionallity. @table @asis @item @code{client_list_t} @{also known as @code{struct client_list}@} In the header file @file{}, libmdsserver defines a dynamic list for storing client ID:s. @item @code{linked_list_t} @{also known as @code{struct linked_list}@} In the header file @file{}, libmdsserver defines a double linked linear sentinel-array-linked list. @item @code{hash_table_t} @{also known as @code{struct hash_table}@} In the header file @file{}, libmdsserver defines a hash table. @item @code{fd_table_t} @{also known as @code{struct fd_table}@} In the header file @file{}, libmdsserver defines a lookup table for small positive integer keys, intended as an alternative to hash tables for file descriptors as keys. @item @code{mds_message_t} @{also known as @code{struct mds_message}@} In the header file @file{}, libmdsserver defines a data structure for message between the server or client and the master server, with the capability of reading for a socket. @end table These data structures share a common set of associated function. However, they do not use the same functions; they are identical except they are are named with the associated data structure. We will use @code{X_t} as an example. @table @asis @item @code{X_destroy} [(@code{X_t* restrict this}) @arrow{} @code{void}] Releases all resouces in @code{*this}, @code{this} itself is however not @code{free}:d. However, @code{hash_table_destory} and @code{fd_table_destory} have another signature. @item @code{X_clone} [(@code{const X_t* restrict this, X_t* restrict out}) @arrow{} @code{int}] Create a deep duplicate of @code{*this} and store it in @code{*out}. @item @code{X_marshal_size} [(@code{const X_t* restrict this}) @arrow{} @code{size_t}] Calculates the exact allocate size needed for the parameter @code{data} in the function @code{X_marshal} if called with the same @code{this} parameter. @item @code{X_marshal} [(@code{const X_t* restrict this, char* restrict data}) @arrow{} @code{void}] Marshal the state of @code{*this} into @code{data}. The number of bytes that will be stored (contiguously) in @code{data} can be calculated with @code{X_marshal_size}. @item @code{X_unmarshal} [(@code{X_t* restrict this, char* restrict data)}) @arrow{} @code{int}] Unmarshal a @code{X_t} from @code{data} into @code{*this}. Returns zero on success and @code{-1} on error. The number of bytes read from @code{data} should, if required, have been precalculated with @code{X_marshal_size} and stored in an earlier location of @code{data}. However, @code{hash_table_unmarshal} and @code{fd_table_unmarshal} have another signature. @end table @menu * Client List:: The @code{client_list_t} data structure.. @end menu @node Client List @subsection Client List To create a client list, allocate a @code{client_list_t*} or otherwise obtain a @code{client_list_t*}, and call @code{client_list_create} with that pointer as the first argument, and the @code{0} as the second argument, unless you want to tune the initialisation. @code{client_list_create} will return zero on and only on successful initialisation. @code{client_list_create}'s second parameter --- @code{size_t capacity} --- can be used to specify how many element the list should initially fit. It will grow when needed, but it is a good idea to tell it how many elements you are planning to populate it with. @code{client_list_t} has two associated functions for manipulating its content: @table @asis @item @code{client_list_add} [(@code{client_list_t* restrict this, uint64_t client}) @arrow{} @code{int}] This function will add the element @code{client} to the list @code{*this}, and return zero on and only on success. @item @code{client_list_remove} [(@code{client_list_t* restrict this, uint64_t client}) @arrow{} @code{void}] This function will remove exactly one occurrence, provided that there is at least on occurrence, of the element @code{client} for the list @code{*this}. @end table The retrieve the number elements stored in a list, reads its variable @code{size_t size}. The variable @code{uint64_t* clients} is used to retrieve stored elements. @example void print_elements(client_list_t* this) @{ size_t i; for (i = 0; i < this->size; i++) printf("Element #%zu: %" PRIu64 "\n", i, this->elements[i]); @} @end example @node GNU Free Documentation License @appendix GNU Free Documentation License @include fdl.texinfo @bye /** * The size of the arrays */ size_t capacity; /** * The index after the last used index in * `values` and `next` */ size_t end; /** * Head of the stack of reusable positions */ size_t reuse_head; /** * Stack of indices than are no longer in use */ ssize_t* reusable; /** * The value stored in each node */ size_t* values; /** * The next node for each node, `edge` if the current * node is the last node, and `LINKED_LIST_UNUSED` * if there is no node on this position */ ssize_t* next; /** * The previous node for each node, `edge` if * the current node is the first node, and * `LINKED_LIST_UNUSED` if there is no node * on this position */ ssize_t* previous; /** * The sentinel node in the list */ ssize_t edge; /** * Create a linked list * * @param this Memory slot in which to store the new linked list * @param capacity The minimum initial capacity of the linked list, 0 for default * @return Non-zero on error, `errno` will have been set accordingly */ int linked_list_create(linked_list_t* restrict this, size_t capacity); /** * Pack the list so that there are no reusable * positions, and reduce the capacity to the * smallest capacity that can be used. Not that * values (nodes) returned by the list's methods * will become invalid. Additionally (to reduce * the complexity) the list will be defragment * so that the nodes' indices are continuous. * This method has linear time complexity and * linear memory complexity. * * @param this The list * @return Non-zero on error, `errno` will have been set accordingly */ int linked_list_pack(linked_list_t* restrict this); /** * Insert a value in the beginning of the list * * @param this:linked_list_t* The list * @param value:size_t The value to insert * @return :ssize_t The node that has been created and inserted, * `LINKED_LIST_UNUSED` on error, `errno` will be set accordingly */ #define linked_list_insert_beginning(this, value) \ (linked_list_insert_after(this, value, this->edge)) /** * Remove the node at the beginning of the list * * @param this:linked_list_t* The list * @return :ssize_t The node that has been removed */ #define linked_list_remove_beginning(this) \ (linked_list_remove_after(this, this->edge)) /** * Insert a value after a specified, reference, node * * @param this The list * @param value The value to insert * @param predecessor The reference node * @return The node that has been created and inserted, * `LINKED_LIST_UNUSED` on error, `errno` will be set accordingly */ ssize_t linked_list_insert_after(linked_list_t* restrict this, size_t value, ssize_t predecessor); /** * Remove the node after a specified, reference, node * * @param this The list * @param predecessor The reference node * @return The node that has been removed */ ssize_t linked_list_remove_after(linked_list_t* restrict this, ssize_t predecessor); /** * Insert a value before a specified, reference, node * * @param this The list * @param value The value to insert * @param successor The reference node * @return The node that has been created and inserted, * `LINKED_LIST_UNUSED` on error, `errno` will be set accordingly */ ssize_t linked_list_insert_before(linked_list_t* restrict this, size_t value, ssize_t successor); /** * Remove the node before a specified, reference, node * * @param this The list * @param successor The reference node * @return The node that has been removed */ ssize_t linked_list_remove_before(linked_list_t* restrict this, ssize_t successor); /** * Remove the node from the list * * @param this The list * @param node The node to remove */ void linked_list_remove(linked_list_t* restrict this, ssize_t node); /** * Insert a value in the end of the list * * @param this:linked_list_t* The list * @param value:size_t The value to insert * @return :ssize_t The node that has been created and inserted, * `LINKED_LIST_UNUSED` on error, `errno` will be set accordingly */ #define linked_list_insert_end(this, value) \ (linked_list_insert_before((this), (value), (this)->edge)) /** * Remove the node at the end of the list * * @param this:linked_list_t* The list * @return :ssize_t The node that has been removed */ #define linked_list_remove_end(this) \ (linked_list_remove_before((this), (this)->edge)) /** * Wrapper for `for` keyword that iterates over each element in a linked list * * @param list:linked_list_t The linked list * @param node:ssize_t The variable to store the node in at each iteration */ #define foreach_linked_list_node(list, node) \ for (node = (list).edge; node = (list).next[node], node != (list).edge;) /** * Print the content of the list * * @param this The list * @param output Output file */ void linked_list_dump(linked_list_t* restrict this, FILE* restrict output); /** * Hash table entry */ typedef struct hash_entry { /** * A key */ size_t key; /** * The value associated with the key */ size_t value; /** * The truncated hash value of the key */ size_t hash; /** * The next entry in the bucket */ struct hash_entry* next; } hash_entry_t; /** * Value lookup table based on hash value, that do not support */ typedef struct hash_table { /** * The table's capacity, i.e. the number of buckets */ size_t capacity; /** * Entry buckets */ hash_entry_t** buckets; /** * When, in the ratio of entries comparied to the capacity, to grow the table */ float load_factor; /** * When, in the number of entries, to grow the table */ size_t threshold; /** * The number of entries stored in the table */ size_t size; /** * Check whether two values are equal * * If this function pointer is `NULL`, the identity is used * * Be aware, this variable cannot be marshalled */ compare_func* value_comparator; /** * Check whether two keys are equal * * If this function pointer is `NULL`, the identity is used * * Be aware, this variable cannot be marshalled */ compare_func* key_comparator; /** * Calculate the hash of a key * * If this function pointer is `NULL`, the identity hash is used * * Be aware, this variable cannot be marshalled * * @param key The key * @return The hash of the key */ hash_func* hasher; } hash_table_t; /** * Create a hash table * * @param this Memory slot in which to store the new hash table * @param initial_capacity The initial capacity of the table * @param load_factor The load factor of the table, i.e. when to grow the table * @return Non-zero on error, `errno` will have been set accordingly */ int hash_table_create_fine_tuned(hash_table_t* restrict this, size_t initial_capacity, float load_factor); /** * Create a hash table * * @param this:hash_table_t* Memory slot in which to store the new hash table * @param initial_capacity:size_t The initial capacity of the table * @return :int Non-zero on error, `errno` will have been set accordingly */ #define hash_table_create_tuned(this, initial_capacity) \ hash_table_create_fine_tuned(this, initial_capacity, 0.75f) /** * Create a hash table * * @param this:hash_table_t* Memory slot in which to store the new hash table * @return :int Non-zero on error, `errno` will have been set accordingly */ #define hash_table_create(this) \ hash_table_create_tuned(this, 16) /** * Release all resources in a hash table, should * be done even if construction fails * * @param this The hash table * @param keys_freer Function that frees a key, `NULL` if keys should not be freed * @param values_freer Function that frees a value, `NULL` if value should not be freed */ void hash_table_destroy(hash_table_t* restrict this, free_func* key_freer, free_func* value_freer); /** * Check whether a value is stored in the table * * @param this The hash table * @param value The value * @return Whether the value is stored in the table */ int hash_table_contains_value(const hash_table_t* restrict this, size_t value) __attribute__((pure)); /** * Check whether a key is used in the table * * @param this The hash table * @param key The key * @return Whether the key is used */ int hash_table_contains_key(const hash_table_t* restrict this, size_t key) __attribute__((pure)); /** * Look up a value in the table * * @param this The hash table * @param key The key associated with the value * @return The value associated with the key, 0 if the key was not used */ size_t hash_table_get(const hash_table_t* restrict this, size_t key); /** * Look up an entry in the table * * @param this The hash table * @param key The key associated with the value * @return The entry associated with the key, `NULL` if the key was not used */ hash_entry_t* hash_table_get_entry(const hash_table_t* restrict this, size_t key); /** * Add an entry to the table * * @param this The hash table * @param key The key of the entry to add * @param value The value of the entry to add * @return The previous value associated with the key, 0 if the key was not used. * 0 will also be returned on error, check the `errno` variable. */ size_t hash_table_put(hash_table_t* restrict this, size_t key, size_t value); /** * Remove an entry in the table * * @param this The hash table * @param key The key of the entry to remove * @return The previous value associated with the key, 0 if the key was not used */ size_t hash_table_remove(hash_table_t* restrict this, size_t key); /** * Remove all entries in the table * * @param this The hash table */ void hash_table_clear(hash_table_t* restrict this); /** * Wrapper for `for` keyword that iterates over entry element in a hash table * * @param table:hash_table_t The hans table * @param i:size_t The variable to store the buckey index in at each iteration * @param entry:hash_entry_t* The variable to store the entry in at each iteration */ #define foreach_hash_table_entry(table, i, entry) \ for (i = 0; i < (table).capacity; i++) \ for (entry = (table).buckets[i]; entry != NULL; entry = entry->next) /** * Unmarshals a hash table * * @param this Memory slot in which to store the new hash table * @param data In buffer with the marshalled data * @param remapper Function that translates values, `NULL` if not translation takes place * @return Non-zero on error, errno will be set accordingly. * Destroy the table on error. */ int hash_table_unmarshal(hash_table_t* restrict this, char* restrict data, remap_func* remapper); /** * The number of entries stored in the table */ size_t size; /** * Map from keys to values */ size_t* values; /** * Map from keys to whether that are in used, bit-packed */ uint64_t* used; /** * Check whether two values are equal * * If this function pointer is `NULL`, the identity is used * * Be aware, this variable cannot be marshalled */ compare_func* value_comparator; /** * Create a fd table * * @param this Memory slot in which to store the new fd table * @param initial_capacity The initial capacity of the table * @return Non-zero on error, `errno` will have been set accordingly */ int fd_table_create_tuned(fd_table_t* restrict this, size_t initial_capacity); /** * Create a fd table * * @param this:fd_table_t* Memory slot in which to store the new fd table * @return :int Non-zero on error, `errno` will have been set accordingly */ #define fd_table_create(this) \ fd_table_create_tuned(this, 16) /** * Release all resources in a fd table, should * be done even if construction fails * * @param this The fd table * @param keys_freer Function that frees a key, `NULL` if keys should not be freed * @param values_freer Function that frees a value, `NULL` if value should not be freed */ void fd_table_destroy(fd_table_t* restrict this, free_func* key_freer, free_func* value_freer); /** * Check whether a value is stored in the table * * @param this The fd table * @param value The value * @return Whether the value is stored in the table */ int fd_table_contains_value(const fd_table_t* restrict this, size_t value) __attribute__((pure)); /** * Check whether a key is used in the table * * @param this The fd table * @param key The key * @return Whether the key is used */ int fd_table_contains_key(const fd_table_t* restrict this, int key) __attribute__((pure)); /** * Look up a value in the table * * @param this The fd table * @param key The key associated with the value * @return The value associated with the key, 0 if the key was not used */ size_t fd_table_get(const fd_table_t* restrict this, int key) __attribute__((pure)); /** * Add an entry to the table * * @param this The fd table * @param key The key of the entry to add * @param value The value of the entry to add * @return The previous value associated with the key, 0 if the key was not used. * 0 will also be returned on error, check the `errno` variable. */ size_t fd_table_put(fd_table_t* restrict this, int key, size_t value); /** * Remove an entry in the table * * @param this The fd table * @param key The key of the entry to remove * @return The previous value associated with the key, 0 if the key was not used */ size_t fd_table_remove(fd_table_t* restrict this, int key); /** * Remove all entries in the table * * @param this The fd table */ void fd_table_clear(fd_table_t* restrict this); /** * Unmarshals a fd table * * @param this Memory slot in which to store the new fd table * @param data In buffer with the marshalled data * @param remapper Function that translates values, `NULL` if not translation takes place * @return Non-zero on error, errno will be set accordingly. * Destroy the table on error. */ int fd_table_unmarshal(fd_table_t* restrict this, char* restrict data, remap_func* remapper); /** * Message passed between a server and a client or between two of either */ typedef struct mds_message { /** * The headers in the message, each element in this list * as an unparsed header, it consists of both the header * name and its associated value, joined by ": ". A header * cannot be `NULL` (unless its memory allocation failed,) * but `headers` itself is NULL if there are no headers. * The "Length" should be included in this list. */ char** headers; /** * The number of headers in the message */ size_t header_count; /** * The payload of the message, `NULL` if none (of zero-length) */ char* payload; /** * The size of the payload */ size_t payload_size; /** * How much of the payload that has been stored (internal data) */ size_t payload_ptr; /** * Internal buffer for the reading function (internal data) */ char* buffer; /** * The size allocated to `buffer` (internal data) */ size_t buffer_size; /** * The number of bytes used in `buffer` (internal data) */ size_t buffer_ptr; /** * 0 while reading headers, 1 while reading payload, and 2 when done (internal data) */ int stage; } mds_message_t; /** * Initialise a message slot so that it can * be used by `mds_message_read` * * @param this Memory slot in which to store the new message * @return Non-zero on error, errno will be set accordingly. * Destroy the message on error. */ int mds_message_initialise(mds_message_t* restrict this); /** * Zero initialise a message slot * * @param this Memory slot in which to store the new message */ void mds_message_zero_initialise(mds_message_t* restrict this); /** * Extend the header list's allocation * * @param this The message * @param extent The number of additional entries * @return Zero on success, -1 on error */ int mds_message_extend_headers(mds_message_t* restrict this, size_t extent); /** * Read the next message from a file descriptor * * @param this Memory slot in which to store the new message * @param fd The file descriptor * @return Non-zero on error or interruption, errno will be * set accordingly. Destroy the message on error, * be aware that the reading could have been * interrupted by a signal rather than canonical error. * If -2 is returned errno will not have been set, * -2 indicates that the message is malformated, * which is a state that cannot be recovered from. */ int mds_message_read(mds_message_t* restrict this, int fd); /** * Get the required allocation size for `data` of the * function `mds_message_compose` * * @param this The message * @return The size of the message when marshalled */ size_t mds_message_compose_size(const mds_message_t* restrict this) __attribute__((pure)); /** * Marshal a message for communication * * @param this The message * @param data Output buffer for the marshalled data */ void mds_message_compose(const mds_message_t* restrict this, char* restrict data);