libmessage(3) FreeBSD Library Functions Manual libmessage(3) NAME libmessage – TCP Server Library SYNOPSIS #include -I/usr/local/include -L/usr/local/lib -lmessage -lcrypto -lssl DESCRIPTION LibMessage implements a generic, event-driven, TCP server with Transport Layer Security. The library performs network and concurrency tasks. You supply code to service connections. If available, LibMessage uses Kernel TLS for maximum performance. A companion library, libmessageclient(3), provides client functions. Two example programs are included in the source distribution: a bare- bones echo server and client (make echo echoclient), and a chat server and client (AVALON.README). Libmessage provides no mechanism for calling code to set events on arbitrary descriptors. MESSAGES TCP clients and servers need to discriminate structure in their streams. LibMessage provides the abstraction of messages. Messages are opaque to the library. You put whatever data you want into messages. Endpoints send and receive messages without concerning themselves with the details of message discrimination or transmission. Both sides of a connection can send messages at any time. Your code must be able to handle incoming messages while outgoing messages are being delivered to the client. If your server implements a call and response protocol, you may choose to discard incoming messages while delivering responses. Servers and clients communicate in messages from 0 to 262128 bytes in length. The -s option changes the maximum message size. The maximum message size controls memory consumption and must be a multiple of the frame payload size of 32766 bytes. Functions in the client and server libraries allow clients and servers to change the maximum message size dynamically. Messages are delivered in one or more frames containing 0 to 32766 bytes of payload with a 2-byte header. The high-order bit of the frame header indicates whether or not the frame is the last frame in a message. The succeeding 15 bits specify the length of the frame's payload in bytes. Frames of 0 length are valid. It is useful to send zero-length terminator frames when you do not know the length of messages in advance. The server attempts to read or write up to one frame when servicing a connection. The maximum frame size is therefore the server's multiplexing quantum. 32 KiB was chosen because that is the default size of a socket send buffer on FreeBSD. The client and server libraries assemble incoming frames into complete messages and deliver them to your code. USAGE The library provides your server's "main" function. You define 6 functions to match the following prototypes. void ms_init_func(); void ms_exit_func(); int ms_accept_callback( void * ); int ms_open_callback( void * ); int ms_read_callback( void *, int, int, unsigned char * ); void ms_write_callback( void * ); void ms_close_callback( void * ); Do not define any other global symbol beginning with the 3 characters ´ms_' because the library reserves that namespace. MS_INIT_FUNC() In ms_init_func() perform the initialization tasks your server needs to do once at server start-up. LibMessage calls ms_init_func(): • before attempting to change the server's user and group to the values specified by the command line options. If the server starts as root ms_init_func(), executes as root. • before the server becomes a daemon and starts listening for connections. The standard streams are connected to the terminal from which the server was started. Error and informative messages should be sent to the terminal. MS_EXIT_FUNC() If you have exit handlers you would like to register to run before the server exits, do not invoke atexit() from ms_init_func(). If you do so, your handlers are run when the server becomes a daemon. The server forks to create a new session. The dying parent process will call your handlers when it exits. Instead, call your cleanup code from ms_exit_func(). MS_SET_NAME() Call ms_set_name() inside ms_init_func() to set the server's name. void ms_set_name( char * ); If not set the server's name defaults to "message". The server's name is used in 2 ways: • When the server is running, stderr is connected to /dev/null. Errors must are reported with syslog(3). LibMessage calls openlog() with the server's name as argument to ensure that log entries are identified by the server's name. • The server's pidfile is written to /var/run/ if the server is started as root. The filename is the server's name with ".pid" appended to it. This file is used by rc.d scripts to stop the server. A sample script is included in the LibMessage distribution. MS_SET_PERIODIC() Install a function to be invoked periodically with: void ms_set_periodic( void (*)(), int ); LibMessage calls the function pointed to by first argument when the number of seconds specified by the second argument has elapsed and then again repeatedly when that number of seconds has elapsed since the last call. MS_ACCEPT_CALLBACK() LibMessage calls ms_accept_callback() when a new TCP connection has been accepted. int ms_accept_callback( void * ); The connection is not yet ready to send and receive data. The purpose of the callback is to allow you to store the opaque pointer argument so that you can timeout the connection if the client does not complete the TLS handshake. Return 0 from ms_accept_callback() to indicate that the connection should be serviced. Return a non-zero value to indicate that the connection should be dropped. If you drop the connection, the library DOES NOT call ms_close_callback(). MS_OPEN_CALLBACK() LibMessage calls ms_open_callback() when a new TLS connection is ready to send and receive data. int ms_open_callback( void * ); The argument is an opaque pointer that uniquely identifies the connection. You pass the pointer to other library functions as necessary. Return 0 from ms_open_callback() to indicate that the connection should be serviced. Return a non-zero value to indicate that the connection should be dropped. If you drop the connection, the library calls ms_close_callback(). If the protocol your server implements requires the server to speak first, send the initial data with ms_queue_message() in ms_open_callback(). MS_SET_DATA() MS_GET_DATA() Use ms_set_data() to associate a void pointer with a connection. void ms_set_data( void *conn, void *data ); The data argument is a pointer of any kind that you want to associate with the connection referenced by the conn argument. Use ms_get_data() to retrieve the stored pointer. void *ms_get_data( void *conn ); MS_GET_QLEN Use ms_get_qlen() to know the number of outgoing frames currently queued for delivery to a client. unsigned int ms_get_qlen( void *conn ); To control memory exhaustion, use this function to detect connections that have gone idle and are not consuming their queues. Close the connections, or stop queueing data for them. MS_CLOSE_CALLBACK() LibMessage calls ms_close_callback() when a connection is closed. void ms_close_callback( void *data ); LibMessage frees any outgoing queued data when ms_close_callback() returns. You must be prepared for connections to be closed unexpectedly due to internal errors. MS_READ_CALLBACK() LibMessage calls ms_read_callback() when a complete message has been read from the client. int ms_read_callback( void *, int len, unsigned char *buffer ); The second argument is the length of the payload data in bytes. The third argument is a character pointer to the payload data. The third argument points into a buffer that is freed on return. LibMessage accepts messages of 0 length. In that case, the buffer argument is NULL. return 1 if you called ms_close_conn() to close the connection. Otherwise, return 0. If you want to discard a message, return 0. MS_WRITE_CALLBACK() LibMessage calls ms_write_callback() when the outgoing frame queue for a client has been exhausted. void ms_write_callback( void * ); MS_QUEUE_MESSAGE() Queue an outgoing message for clients with ms_queue_message(). int ms_queue_message( void **, int conns, unsigned int len, unsigned char *data ); The first argument is a pointer to an array of opaque pointers representing the connections to which the data is to be written. The second argument is the number of elements in the first argument. The third argument is the length of the payload data in bytes. The fourth argument is a character pointer to the payload data. ms_queue_message() frames the data and queues the frames for delivery. Each time a write event for a connection is generated, the library sends no more than one frame to the client. The library calls ms_write_callback() when a connection's queue has been exhausted. ms_queue_message() returns 0 on success and 1 on error. If conns is zero, ms_queue_message() does nothing and returns 0. An error return means that the system is low on memory and the function could not allocate memory to enlarge an outgoing message queue. Some connections may have had the message queued for them. Some will definitely not have had the message queued for them. Note that if you pass a length argument outside the range 1-262128 (or -s), ms_queue_message() also returns 1. MS_QUEUE_FRAME() Queue an individual outgoing frame for clients with ms_queue_frame(). This function is useful when you do not know the length of a message in advance. You can queue fragment frames containing the data followed by a final terminator frame of 0 length. int ms_queue_frame( void **, int conns, int final, unsigned int len, unsigned char *data ); The third argument indicates whether or not the frame header should have its high-order bit set. ms_queue_frame() frames the data in one frame and queues the frame for delivery. If the len argument is outside the range 0-32766, ms_queue_frame() returns 1. Otherwise, The function behaves similarly to ms_queue_message(). MS_QUEUE_SENDFILE_FRAME() If you have Kernel TLS enabled, use ms_queue_sendfile_frame() to efficiently send file content with preliminary data to clients. int ms_queue_sendfile_frame( void **items, int conns, int final, unsigned int dlen, unsigned char *data, int fd, ssize_t offset, ssize_t slen ); This function is an extension of ms_queue_frame(). Three additional arguments provide a file descriptor, an offset into the file, and the amount of file data to send. Instead of placing file data in each queued frame, the file data is read from the file starting at offset when the output queues for connections are processed. The dlen and data arguments specify optional data to insert into the start of the frame before the file data. The dlen argument can be zero if there is no prelimary data. In this case, the data pointer can be NULL. If slen is zero, no file data is sent to clients. In this situation, ms_queue_sendfile_frame() functions identically to ms_queue_frame(). The sum of dlen + slen must be equal to or less than the frame payload size of 32766 bytes. ms_queue_sendfile_frame() returns the same values in the same situations as ms_queue_frame() does. Additionally, if any one of the connections in conns does not support Kernel TLS, ms_queue_sendfile_frame() returns 2 and no frames are queued for any connection. MS_QUEUE_SENDFILE_MESSAGE() If you have Kernel TLS enabled, use ms_queue_sendfile_message() to efficiently send file content to clients. int ms_queue_sendfile_message( void **items, int conns, int fd, ssize_t offset, ssize_t len ); This function is a variant of ms_queue_message(). The three arguments after the first two provide a file descriptor, an offset into the file, and the amount of file data to send to clients. len must be equal to or less than the maximum message size. ms_queue_sendfile_message() works similarly to ms_queue_message() but instead of placing data in each queued frame, each frame's data is read from the file starting at offset when the output queues for connections are processed. ms_queue_sendfile_message() returns the same values in the same situations as ms_queue_message() does. Additionally, if any one of the connections in conns does not support Kernel TLS, ms_queue_sendfile_message() returns 2 and no frames are queued for any connection. MS_CLOSE_CONN() To close a connection, invoke ms_close_conn(). void ms_close_conn( void *conn, int now ); If "now" is non-zero, ms_close_conn() closes the connection immediately and frees any queued data. Otherwise, libmessage closes the connection when the queue of outgoing data has been written to the client. Do not call other library functions for a connection that you have closed. The connection identifier pointer is no longer valid. MS_GET_MAX_CONN Use ms_get_max_conn() to retrieve the maximum number of simultaneous connections (-m) the library accepts, which defaults to 16384. int mc_get_max_conn(); MS_GET_MAX_MESSAGE Use ms_get_max_message() to retrieve the maximum message size that both server and client use. The default maximum message size is 262128 bytes. unsigned int mc_get_max_message(); MS_SET_MAX_MESSAGE Use ms_set_max_message() to set the maximum message size. unsigned int mc_set_max_message( unsigned int max ); The function returns 0 if max is not a multiple of the internal frame payload size, which is 32766. In this case, the maximum message size is unchanged. Otherwise, the function returns the new maximum message size. The value set by ms_set_max_message() overrides the value set by the -s command-line option. SIGNALS LibMessage needs to catch SIGTERM, so do not change the disposition of that signal. CONFIGURATION Place the following lines in /etc/sysctl.conf and reboot. The 2 files values determine the maximum number of open files system-wide and per- process. The 3 subsequent options turn on Kernel TLS for maximum performance. kern.maxfiles=262128 kern.maxfilesperproc=131072 kern.ipc.mb_use_ext_pgs=1 kern.ipc.tls.ifnet.permitted=1 kern.ipc.tls.enable=1 LibMessage writes its pidfile into /var/run/ if is started as root. The library is stopped with SIGTERM. A sample control script is provided in the LibMessage distribution. To use the script, you must replace all occurrences of "message" with the value you pass to ms_set_name(). The script must be renamed as the value you passed to ms_set_name() and installed in /usr/local/etc/rc.d. Two variables must be added to /etc/rc.conf to use the script. Substitute your server's name for "message": message_enable="YES" message_flags="-u www -g www If the "enable" variable is set to "YES", the server is started at system start. Use the following rc commands: service message start service message stop service message restart service message status If you do not want the server started on system start, then set message_enable="NO" and use the following commands: service message onestart service message onestop service message onerestart service message onestatus COMMAND-LINE OPTIONS The following command line options are recognized by LibMessage servers. The -t and -r options must be defined. The rest are optional. -r The -r option specifies the root directory of your server. The server invokes chdir(2) on the value of this option before changing to the user and group specified by the -u and -g options. -p The -p option specifies the port to listen on. This defaults to 10000. To bind to a port lower than 1024, your server must be started as root. -i By default, LibMessage accepts connections on all interfaces it can find capable of IPv4 or IPv6. The -i option limits the server to accepting connections from a specified interface. Pass the IP address of the desired interface as argument. -m By default, LibMessage maintains no more than 16384 simultaneous connections. The -m option changes this value. -u -g The -u and the -g options are used to specify the user and group for the server. If the server starts as an unprivileged user and group, then these options must be set to the user and group. For LibMessage to change to a different user and group, it must be started as root. Both values default to "nobody". -x The -x option prevents LibMessage from becoming a daemon. LibMessage runs in the foreground of its launching terminal. Use this option in conjunction with the compiler options -O0 -g to run your servers under the debugger. If this option is not present, LibMessage forks into two processes. The parent exits, while the child continues. This ensures that the child is not a process group leader. The child then calls setsid() to start a new session. This detaches the server from its controlling terminal so that job control signals cannot be sent to the server (ie., you cannot use the shell commands: fg, ctrl-c, or ctl-z). At this point, the server is a daemon process. After becoming a daemon, the server forks again. The parent waits for the child to terminate. The child runs your server code. If a signal terminates the child, the parent spawns another. If the child calls exit(), the parent exits. When the parent receives a SIGTERM, the parent propagates the signal to the child. The child catches the signal and exit()s. Because the master automatically respawns crashed workers, you must not run your server without -x until you are sure that your server has no catastrophic bugs of the sort that occur during development. A bug that generates a signal that terminates the worker may cause the master to enter an expensive loop that locks your system. -f The -f option takes a filename as argument. LibMessage assigns the filename to the global character pointer ms_config_file. This enables code in ms_init_func() and to access a configuration file. -s The -s option changes the maximum message size from the default of 262128 bytes to a user-specified value. The value must be a multiple of the frame payload size of 32766 bytes. The value provided must fit into an unsigned int on the host system. -t The -t option specifies the fully qualified path to the TLS configuration file. The file must contain 3 lines of text. The first line is the fully qualified path to the file holding the server key in PEM format. The second line is the password for the key. This line can be blank if there is no password. The third line is the fully qualified path to the file containing the certificate chain in PEM format. AUTHORS James Bailie ⟨bailie9@icloud.com⟩ mammothcheese.ca Fri Nov 21, 2025