libmessage(3) FreeBSD Library Functions Manual libmessage(3) NAME libmessage – TCP Server Library SYNOPSIS #include -I/usr/local/include -L/usr/local/lib -lmessage -lcrypto -lssl DESCRIPTION LibMessage implements a generic, event-driven, TCP server with Transport Layer Security. The library performs network and concurrency tasks. You supply code to service connections. A companion library, libmessageclient(3), provides client functions. An echo server and client are included in the source distribution. A chat server and client (AVALON.README) are included in the source distribution. Libmessage provides no mechanism for calling code to set events on arbitrary descriptors. MESSAGES LibMessage provides its own means of message discrimination. Endpoints send and receive messages without concerning themselves with the details of message discrimination or transmission. Both sides of a connection can send messages at any time. Your code must be able to handle incoming messages while outgoing messages are being delivered to the client. Servers and clients communicate in messages from 0 to 262144 bytes in length. The -s option changes the maximum size. Messages are delivered in one or more frames containing 0 to 32767 bytes of payload. The first 2 bytes of a frame form the frame's header. The high-order bit of the frame header indicates whether or not the frame is the last frame in a message. The succeeding 15 bits specify the length of the frame's payload in bytes. Frames of 0 length are valid. It is useful to send zero-length terminator frames when you do not know the length of messages in advance. The server attempts to read or write up to one frame when servicing a connection. The maximum frame size is therefore the server's multiplexing quantum. 32 KiB was chosen because that is the default size of a socket send buffer on FreeBSD. The client and server libraries assemble incoming frames into complete messages and deliver them to your code. There are 2 Bytes of metadata for every 32767 bytes of payload. A message of 262144 bytes is fragmented into 9 frames. Therefore, with the default maximum message size, the protocol overhead is 9 x 2 = 18 bytes. USAGE The library provides your server's "main" function. You define 6 functions to match the following prototypes. void ms_init_func(); void ms_exit_func(); int ms_accept_callback( void * ); int ms_open_callback( void * ); int ms_read_callback( void *, int, int, unsigned char * ); void ms_write_callback( void * ); void ms_close_callback( void * ); Do not define any other global symbol beginning with the 3 characters ´ms_' because the library reserves that namespace. MS_INIT_FUNC() In ms_init_func() perform the initialization tasks your server needs to do once at server start-up. LibMessage calls ms_init_func(): • before attempting to change the server's user and group to the values specified by the command line options. If the server starts as root ms_init_func(), executes as root. • before the server becomes a daemon and starts listening for connections. The standard streams are connected to the terminal from which the server was started. Error and informative messages should be sent to the terminal. MS_EXIT_FUNC() If you have exit handlers you would like to register to run before the server exits, do not invoke atexit() from ms_init_func(). If you do so, your handlers are run when the server becomes a daemon. The server forks to create a new session. The dying parent process will call your handlers when it exits. Instead, call your cleanup code from ms_exit_func(). MS_SET_NAME() Call ms_set_name() inside ms_init_func() to set the server's name. void ms_set_name( char * ); If not set the server's name defaults to "message". The server's name is used in 2 ways: • When the server is running, stderr is connected to /dev/null. Errors must are reported with syslog(3). LibMessage calls openlog() with the server's name as argument to ensure that log entries are identified by the server's name. • The server's pidfile is written to /var/run/ if the server is started as root. The filename is the server's name with ".pid" appended to it. This file is used by rc.d scripts to stop the server. A sample script is included in the LibMessage distribution. MS_SET_PERIODIC() Install a function to be invoked periodically with: void ms_set_periodic( void (*)(), int ); LibMessage calls the function pointed to by first argument when the number of seconds specified by the second argument has elapsed and then again repeatedly when that number of seconds has elapsed since the last call. MS_ACCEPT_CALLBACK() LibMessage calls ms_accept_callback() when a new TCP connection has been accepted. int ms_accept_callback( void * ); The connection is not yet ready to send and receive data. The purpose of the callback is to allow you to store the opaque pointer argument so that you can timeout the connection if the client does not complete the TLS handshake. Return 0 from ms_accept_callback() to indicate that the connection should be serviced. Return a non-zero value to indicate that the connection should be dropped. If you drop the connection, the library DOES NOT call ms_close_callback(). MS_OPEN_CALLBACK() LibMessage calls ms_open_callback() when a new TLS connection is ready to send and receive data. int ms_open_callback( void * ); The argument is an opaque pointer that uniquely identifies the connection. You pass the pointer to other library functions as necessary. Return 0 from ms_open_callback() to indicate that the connection should be serviced. Return a non-zero value to indicate that the connection should be dropped. If you drop the connection, the library calls ms_close_callback(). If the protocol your server implements requires the server to speak first, send the initial data with ms_queue_message() in ms_open_callback(). MS_SET_DATA() MS_GET_DATA() Use ms_set_data() to associate a void pointer with a connection. void ms_set_data( void *conn, void *data ); The data argument is a pointer of any kind that you want to associate with the connection referenced by the conn argument. Use ms_get_data() to retrieve the stored pointer. void *ms_get_data( void *conn ); MS_GET_QLEN Use ms_get_qlen() to know the number of outgoing frames currently queued for delivery to a client. unsigned int ms_get_qlen( void *conn ); To control memory exhaustion, use this function to detect connections that have gone idle and are not consuming their queues. Close the connections, or stop queueing data for them. MS_CLOSE_CALLBACK() LibMessage calls ms_close_callback() when a connection is closed. void ms_close_callback( void *data ); LibMessage frees any outgoing queued data when ms_close_callback() returns. You must be prepared for connections to be closed unexpectedly due to internal errors. MS_READ_CALLBACK() LibMessage calls ms_read_callback() when a complete message has been read from the client. int ms_read_callback( void *, int len, unsigned char *buffer ); The second argument is the length of the payload data in bytes. The third argument is a character pointer to the payload data. The third argument points into a buffer that is freed on return. LibMessage accepts messages of 0 length. In that case, the buffer argument is NULL. return 1 if you called ms_close_conn() to close the connection. Otherwise, return 0. If you want to discard a message, return 0. MS_WRITE_CALLBACK() LibMessage calls ms_write_callback() when the outgoing frame queue for a client has been exhausted. void ms_write_callback( void * ); MS_QUEUE_MESSAGE() Queue an outgoing message for clients with ms_queue_message(). int ms_queue_message( void **, int conns, unsigned int len, unsigned char *data ); The first argument is a pointer to an array of opaque pointers representing the connections to which the data is to be written. The second argument is the number of elements in the first argument. The third argument is the length of the payload data in bytes. The fourth argument is a character pointer to the payload data. ms_queue_message() frames the data and queues the frames for delivery. Each time a write event for a connection is generated, the library sends no more than one frame to the client. The library calls ms_write_callback() when a connection's queue has been exhausted. ms_queue_message() returns 0 on success and 1 on error. If conns is zero, ms_queue_message() does nothing and returns 0. An error return means that the system is low on memory and the function could not allocate memory to enlarge an outgoing message queue. Some connections may have had the message queued for them. Some will definitely not have had the message queued for them. Note that if you pass a length argument outside the range 1-262144 (or -s), ms_queue_message() also returns 1. MS_QUEUE_FRAME() Queue an individual outgoing frame for clients with ms_queue_frame(). This function is useful when you do not know the length of a message in advance. You can queue fragment frames containing the data followed by a final terminator frame of 0 length. int ms_queue_frame( void **, int conns, int final, unsigned int len, unsigned char *data ); The third argument indicates whether or not the frame header should have its high-order bit set. ms_queue_frame() frames the data in one frame and queues the frame for delivery. If the len argument is outside the range 0-32767, ms_queue_frame() returns 1. Otherwise, The function behaves similarly to ms_queue_message(). MS_CLOSE_CONN() To close a connection, invoke ms_close_conn(). void ms_close_conn( void *conn, int now ); If "now" is non-zero, ms_close_conn() closes the connection immediately and frees any queued data. Otherwise, libmessage closes the connection when the queue of outgoing data has been written to the client. Do not call other library functions for a connection that you have closed. The connection identifier pointer is no longer valid. SIGNALS LibMessage needs to catch SIGTERM, so do not change the disposition of that signal. CONFIGURATION LibMessage writes its pidfile into /var/run/ if is started as root. The library is stopped with SIGTERM. A sample control script is provided in the LibMessage distribution. To use the script, you must replace all occurrences of "message" with the value you pass to ms_set_name(). The script must be renamed as the value you passed to ms_set_name() and installed in /usr/local/etc/rc.d. Two variables must be added to /etc/rc.conf to use the script. Substitute your server's name for "message": message_enable="YES" message_flags="-u www -g www If the "enable" variable is set to "YES", the server is started at system start. Use the following rc commands: service message start service message stop service message restart service message status If you do not want the server started on system start, then set message_enable="NO" and use the following commands: service message onestart service message onestop service message onerestart service message onestatus COMMAND-LINE OPTIONS The following command line options are recognized by LibMessage servers. The -t and -r options must be defined. The rest are optional. -r The -r option specifies the root directory of your server. The server invokes chdir(2) on the value of this option before changing to the user and group specified by the -u and -g options. -p The -p option specifies the port to listen on. This defaults to 10000. To bind to a port lower than 1024, your server must be started as root. -i By default, LibMessage accepts connections on all interfaces it can find capable of IPv4 or IPv6. The -i option limits the server to accepting connections from a specified interface. Pass the IP address of the desired interface as argument. -m By default, LibMessage maintains no more than 16384 simultaneous connections. The -m option changes this value. -u -g The -u and the -g options are used to specify the user and group for the server. If the server starts as an unprivileged user and group, then these options must be set to the user and group. For LibMessage to change to a different user and group, it must be started as root. Both values default to "nobody". -x The -x option prevents LibMessage from becoming a daemon. LibMessage runs in the foreground of its launching terminal. Use this option in conjunction with the compiler options -O0 -g to run your servers under the debugger. If this option is not present, LibMessage forks into two processes. The parent exits, while the child continues. This ensures that the child is not a process group leader. The child then calls setsid() to start a new session. This detaches the server from its controlling terminal so that job control signals cannot be sent to the server (ie., you cannot use the shell commands: fg, ctrl-c, or ctl-z). At this point, the server is a daemon process. After becoming a daemon, the server forks again. The parent waits for the child to terminate. The child runs your server code. If a signal terminates the child, the parent spawns another. If the child calls exit(), the parent exits. When the parent receives a SIGTERM, the parent propagates the signal to the child. The child catches the signal and exit()s. Because the master automatically respawns crashed workers, you must not run your server without -x until you are sure that your server has no catastrophic bugs of the sort that occur during development. A bug that generates a signal that terminates the worker may cause the master to enter an expensive loop that locks your system. -f The -f option takes a filename as argument. LibMessage assigns the filename to the global character pointer ms_config_file. This enables code in ms_init_func() and to access a configuration file. -s The -s option changes the maximum message size from the default of 262144 bytes to a user-specified value. The value cannot be set lower than the frame size of 32767 bytes. -t The -t option specifies the fully qualified path to the TLS configuration file. The file must contain 3 lines of text. The first line is the fully qualified path to the file holding the server key in PEM format. The second line is the password for the key. This line can be blank if there is no password. The third line is the fully qualified path to the file containing the certificate chain in PEM format. AUTHORS James Bailie ⟨jimmy@mammothcheese.ca⟩ http://www.mammothcheese.ca Tue Jan 30, 2024