Blog Post

Mach Messages in macOS

Example of low-level inter-process communication (IPC) in C++ using Mach messages.

Mach Messages in macOS - Example of low-level inter-process communication (IPC) in C++ using Mach messages.

Intro

While working on my launch daemon for macOS I needed to find a way to provide the inter-process communication (or IPC) between it and my launch agent. Me being me, I decided to look into what is the most low-level way to do it? I did some research and found out that Mach messages are used internally by most of the macOS (and iOS) components for communication. (The concept of "Mach messages" comes from the Mach microkernel.)

Unfortunately, the documentation provided by Apple for Mach messages was far from ideal (here's an example) and anything else available was either incomplete, or too old to use for today's development. Moreover, I couldn't find a complete example (in C or C++) that could illustrate how to use Mach messages for sending an receiving data.

At some point I went to the popular and highly glorified programming forum, and tried to find my answers there. Not to be surprised, there weren't many people there that were willing to help (aside from just downvoting my questions.) I eventually found one guy there that pointed me in the right direction, but otherwise he pretty much told me in private that he gets paid to answer questions. So that was it for me. Thus I ended up looking for answers on my own.

Eventually I was able to find what I needed to do to implement a basic client/server IPC using Mach messages. It took me over a week to iron everything out. Thus, I want to share my findings here in case anyone else wants to know how to use Mach messages for the low-level IPC on macOS.

I will try to provide a complete example in C++ for anyone who doesn't like reading blog posts. The code sample will have sufficient comments to get you started.

I am sure this mechanism will also apply to other operating systems developed by Apple, such as iOS, watchOS, etc. But I am also more than sure that they will not allow an app that uses such low-level mechanism, as Mach messages, to their App Store. So you may want to check before you start using it in your code.

IPC for macOS

I want to preface this by saying that if you need any means of the IPC between two or more processes in macOS, there are other higher level mechanisms that you can use. You don't need to go with Mach messages. To name just a few:

  • Sockets and Socket Streams - this is probably the most obvious approach that is also cross-platform and supports sending a message to a remote computer (which Mach messages do not.)
  • XPC - this is an IPC mechanism that is based on Mach messages. It may be easier to implement in your program, written in a higher-level language than C or C++. Apple provides some Objective-C code samples in their tutorials on how to use XPC.
  • CFMachPort - this is Apple's own wrapper for the low-level implementation of Mach messages.

For the purpose of this article, I will eschew all higher-level implementations that I listed above and will stick with the low-level Mach concepts only. After all, when you implement a very low-level IPC you are not bound by the higher-level limitations. For instance, unlike CFMachPort, we are not limited to receiving messages in our run-loop and can spin up our own worker threads, which may improve performance of our messaging pipeline in a highly concurrent system.

Mach Messaging Concepts

It may be prudent to familiarize yourself with the Mach messaging concepts before you review my code below:

  • General Mach IPC Concepts - this is the page from the creators of Mach, Carnegie Mellon University. It contains a list of all necessary terms and APIs.
  • Inter Process Communication - another long manual from the Carnegie Mellon University with the details of the IPC using Mach messages. It also includes the discussion of the implementation of the virtual memory by the Mach kernel, which is used internally during that type of IPC.
  • Mach Concepts - this page contains somewhat dated Mach basics as they were implemented by the NeXT Computer, Inc. If reading this, keep in mind that the information was written in 1995.

There could be other useful resources from individual bloggers, that you can look up by searching online.

Synopsis

If you ask me to summarize the Mach messaging IPC, I would do it as such:

  • There could be only one server and one or multiple clients.
  • A communication via Mach messages always originates from a client and is received by a server.
  • The Mach kernel provides capabilities to marshal virtual memory pointers between processes.
  • To communicate both end-points need to have a port. This port is not the same as a socket port. It is more like a handle.
  • Mach messages can be sent only between processes (or tasks, as they are called in a Mach kernel terminology) running on the same computer. Remote IPC is not supported.
  • There's no access control (or ACL) associated with Mach ports. They can only have send and receive rights assigned to them.
  • A Mach message can be of any arbitrary size for as long as the virtual memory supports it.
  • Unlike network packets, Mach messages cannot be lost, kernel guarantees that.

For the brevity of the code sample, I will also make a few assumptions:

  • Even though Mach messages support sending and receiving at the same time, I will show the one-way communication only: or when a client sends a message and a server receives it. In other words, the server will not be sending a response to the client. (If such is required you can open two ports for communication in both directions, or re-write my code sample with that capability.)
  • The server side of the IPC is written with the assumption that the server will process messages quickly and won't hold the port. If that is a possibility, the server-side implementation has to be re-written with the use of worker threads for processing the incoming messages.
  • I'm assuming that my sample code is compiled with the Apple's Xcode.

MachComm Code Sample

So without further adieu, the following is the code with my implementation of the client/server class for the IPC communication using Mach messages. I bunched it all into one .h file for brevity. But you are welcome to separate it into proper C++ header and definition files.

C++ (MachComm class)[Copy]
//
//  Class that provides a wrapper to send and receive Mach messages
//
//  dennisbabkin.com
//

#ifndef MachComm_h
#define MachComm_h

#include <stdio.h>
#include <string>
#include <new>

#include "locks.h"				//Our implementation of the synchronization locks

#include <mach/mach_port.h>
#include <launch.h>

#include <mach/boolean.h>
#include <servers/bootstrap.h>

#include <mach/mach.h>
#include <mach/mach_error.h>



struct MACH_MSG_BASE
{
	//Base structure for sending a Mach message.
	//It must always start with the header, and
	//have the following members if we're sending 
	//a block of memory to another process.
	//
	mach_msg_header_t hdr;
	
	mach_msg_body_t body;
	mach_msg_ool_descriptor_t data;
	mach_msg_type_number_t count;
	
};


struct MACH_MSG_RCV
{
	//Base structure for receiving a Mach message.
	//
	MACH_MSG_BASE msg;

	//Space for mach_msg_trailer_t, which could be of a variable size.
	//It is provided by the kernel. Let's give it some extra space.
	//
	char trailing_bytes[sizeof(void*) * 8];
};

#define INFINITE 0


///Possible results of receiving a Mach message
///
enum MACH_RCV_RES
{
	MRRES_Error = -1,
	MRRES_Success,
	MRRES_TimedOut,
};



///A helper class for receiving binary data of an arbitrary size
///from a Mach port. You technically do not need it for IPC.
///I'm providing it only for a convenience of memory allocation
///and proper de-allocation using the RAII technique...
///
///IMPORTANT: Please be aware that this class will be a bottleneck 
///           for large data transmissions since it uses one extra 
///           heap allocation and memcpy to store the data!
///
struct MACH_RCV_DATA
{
	MACH_RCV_DATA()
	{
	}
	
	~MACH_RCV_DATA()
	{
		//Free data
		freeData();
	}

	
	///Free allocated data
	void freeData()
	{
		if(_pData)
		{
			delete[] _pData;
			_pData = nullptr;
			
			_szcb = 0;
		}
	}
	
	
	///Retrieve pointer to the byte array stored in this class.
	///IMPORTANT: DO NOT free it outside of this class!
	///'pcbOutSz' = if not 0, will receive the size of returned data in bytes
	///RETURN:
	///     - Pointer to data - do not release, or change it!
	///     - 0 if no data is stored here
	const void* getData(size_t* pcbOutSz = nullptr)
	{
		if(pcbOutSz)
			*pcbOutSz = _szcb;
		
		return _pData;
	}
	
	
	///Set data in this class by copying it from 'pMem'
	///'szcb' = size of 'pMem' in bytes
	///RETURN:
	///     - true if no errors
	bool setData(const void* pMem, size_t szcb)
	{
		//Free old data first
		freeData();
		
		if(pMem)
		{
			//Reserve mem
			_pData = new (std::nothrow) char[szcb];
			if(!_pData)
			{
				//No memory
				assert(false);
				
				return false;
			}
			
			memcpy((void*)_pData, pMem, szcb);
			_szcb = szcb;
		}
		
		return true;
	}
	
	
private:
	
	//Copy constructor and assignments are NOT available!
	MACH_RCV_DATA(const MACH_RCV_DATA& s) = delete;
	MACH_RCV_DATA& operator = (const MACH_RCV_DATA& s) = delete;


private:
	const char* _pData = nullptr;
	size_t _szcb = 0;                        //Size of 'pData' in bytes

};



///Defines the function of the MachComm class
///
enum MACH_ENDPOINT_TYPE
{
	MET_Unknown,
	
	MET_SERVER,         //Class acts as the IPC server
	MET_CLIENT,         //Class acts as one of the IPC clients
};




///This is the main class that provides the IPC via Mach messages
///
struct MachComm
{
	MachComm()
	{
		//Constructor
	}
	
	~MachComm()
	{
		//Destructor
		uninitialize();
	}

	

	///Initialize this class for IPC
	///INFO: Does nothing, if this class was already initialized.
	///'type' = type of the end-point to initialize this class as (it cannot be changed later.)
	///'pPortName' = unique port name (in reverse DNS format) - it must match what is specified 
	///              in the MachServices key for the launchd.plist file that is used to launch 
	///              the IPC server in a deamon:
	///               https://www.manpagez.com/man/5/launchd.plist/
	///RETURN:
	///     - true if no errors
	bool initialize(MACH_ENDPOINT_TYPE type,
					const char* pPortName)
	{
		bool bRes = false;
		
		if(true)
		{
			kern_return_t nKRet;
			
			//Act from within a lock
			WRITER_LOCK wrl(_lock);
			
			//See if we haven't initialized previously
			if(_port == MACH_PORT_NULL)
			{
				//Get the port for communication
				if(type == MET_SERVER)
				{
					//Server - get port number from the launchd.plist
					nKRet = _createServerPort(pPortName,
												&_port);
				}
				else if(type == MET_CLIENT)
				{
					//Client - find previously created port by the server
					nKRet = _findServerPort(pPortName,
											&_port);
				}
				else
				{
					//Wrong endpoint type
					nKRet = KERN_INVALID_ARGUMENT;
				}
				
				if(nKRet == KERN_SUCCESS)
				{
					//Remember the port info
					assert(_port);
					
					_strPortName = pPortName;
					_endpoint = type;
					
					//Done!
					bRes = true;
				}
			}
			else
			{
				//Already initialized
				bRes = true;
			}
		}
		
		return bRes;
	}
	
	
	///Uninitialize this class to free resources
	///INFO: Does nothing if it was already uninitialized.
	///RETURN:
	///     - true if no errors
	bool uninitialize()
	{
		bool bRes = false;
		
		if(true)
		{
			//Act from within a lock
			WRITER_LOCK wrl(_lock);
			
			bRes = _uninit();
		}
		
		return bRes;
	}
	
	
	///Send message to the IPC server
	///'pPtrMsg' = pointer to the message to send (as a byte array)
	///'ncbMsgSz' = size of 'pPtrMsg' in bytes
	///'nmsTimeout' = timeout for sending in ms, or INFINITE (or 0) for no timeout
	///RETURN:
	///     - KERN_SUCCESS if success
	///     - Other if error
	kern_return_t sendMsg(const void* pPtrMsg,
							mach_msg_size_t ncbMsgSz,
							unsigned int nmsTimeout = INFINITE)
	{
		kern_return_t nKRet;
		
		MACH_MSG_BASE msg = {};
		if(!_set_msg_for_sending_mem_ptr(msg, pPtrMsg, ncbMsgSz))
		{
			//initialize() was not called successfully prior to this call
			assert(false);
			
			return KERN_NOT_IN_SET;
		}
		
		nKRet = mach_msg(&msg.hdr,
							MACH_SEND_MSG |
							(nmsTimeout != INFINITE ? MACH_SEND_TIMEOUT : 0),
							sizeof(msg),
							0,
							MACH_PORT_NULL,
							nmsTimeout != INFINITE ? nmsTimeout : MACH_MSG_TIMEOUT_NONE,
							MACH_PORT_NULL);
			
		return nKRet;
	}
	
	
	
	///Receive message from the IPC client
	///'rcvData' = object to receive data in (it will be filled only if return is a success)
	///'puiOutClientPID' = if not 0, receives PID of the client process that sent this message, or 0 if error
	///'nmsTimeout' = how long to wait for the message to arrive in ms, or INFINITE (or 0) to wait for as long as it takes
	///'pOutKernErr' = if not 0, and an error occurs, receives a kernel error code of a fault
	///RETURN:
	///     - Result of the operation
	MACH_RCV_RES receiveMsg(MACH_RCV_DATA& rcvData,
							pid_t* puiOutClientPID = nullptr,
							unsigned int nmsTimeout = INFINITE,
							kern_return_t* pOutKernErr = nullptr)
	{
		MACH_RCV_RES res = MRRES_Error;
		kern_return_t nKRet;
		
		mach_port_t port;		
		pid_t pid_client = 0;

		if(true)
		{
			//Act from within a lock
			READER_LOCK rdl(_lock);
			
			port = _port;
		}

		//Only if we have a port
		if(port != MACH_PORT_NULL)
		{
			MACH_MSG_RCV rcv = {};
			rcv.msg.hdr.msgh_local_port = port;
			rcv.msg.hdr.msgh_size = sizeof(rcv);

			//Here we specify flags for the type of a trailing header that we want to receive,
			//and that we want to receive a special "audit" trailer that should contain
			//the process ID of a client that sent this message. That last part is
			//important for us for security reasons, so that we can vet the sending process!
			//
			nKRet = mach_msg(&rcv.msg.hdr,
								MACH_RCV_MSG |
								MACH_RCV_LARGE |
								(nmsTimeout != INFINITE ? MACH_RCV_TIMEOUT : 0) |
								MACH_RCV_TRAILER_TYPE(MACH_MSG_TRAILER_FORMAT_0) |
								MACH_RCV_TRAILER_ELEMENTS(MACH_RCV_TRAILER_AUDIT),
								0,
								sizeof(rcv),
								port,
								nmsTimeout != INFINITE ? nmsTimeout : MACH_MSG_TIMEOUT_NONE,
								MACH_PORT_NULL);
			
			if(nKRet == KERN_SUCCESS)
			{
				//We got the message, see if we have valid data in it...
				const void* pData = rcv.msg.data.address;
				mach_msg_size_t ncbSzData = rcv.msg.data.size;
				
				//INFO: It is important to check for MACH_MSGH_BITS_COMPLEX,
				//      as it will tell us if 'rcv.msg.data.address' was marshalled
				//      by the kernel for us, and thus we can read from it!
				if(pData &&
					(rcv.msg.hdr.msgh_bits & MACH_MSGH_BITS_COMPLEX))
				{
					//Copy raw data
					//
					//IMPORTANT: I am using my helper class to store the data that I receive
					//           from the Mach port. Keep in mind that doing this will be a
					//           definite bottleneck in speed for larger data transmissions!
					//
					//           In that case, my advice is to keep the 'pData' pointer and
					//           to use it directly when you need to read the received data.
					//           After that, make sure to free that memory with a call to 
					//           vm_deallocate(), as I showed below, to avoid memory leaks.
					//
					if(rcvData.setData(pData, ncbSzData))
					{
						//Do compile-time check of our trailing buffer - and make sure that
						//it is large enough to contain the needed trailing header.
						static_assert(sizeof(rcv.trailing_bytes) >= sizeof(mach_msg_audit_trailer_t), "Size of the trailing buffer is too small!");
						
						mach_msg_audit_trailer_t* pTrlr = (mach_msg_audit_trailer_t*)rcv.trailing_bytes;
						
						if(pTrlr->msgh_trailer_size == sizeof(*pTrlr))
						{					
							//Get PID of the sending process
							pid_client = pTrlr->msgh_audit.val[5];
							
							//Done
							res = MRRES_Success;
						}
						else
						{
							//Wrong size returned
							nKRet = KERN_MEMORY_ERROR;
							assert(false);
						}
					}
					else
					{
						//No memory
						nKRet = KERN_MEMORY_FAILURE;
					}
					
					
					//And free the memory that we received from the kernel
					kern_return_t nKRet2 = vm_deallocate(mach_task_self(),
														(vm_address_t)pData,
														ncbSzData);
					if(nKRet2 != KERN_SUCCESS)
					{
						//Error
						nKRet = nKRet2;
						assert(false);
						res = MRRES_Error;
					}
					
					//Reset the pointer & size since we just freed the memory.
					//We technically don't need to do this, but it is safer
					//this way in case of a possible use-after-free that may follow ...
					//
					pData = nullptr;
					ncbSzData = 0;
				}
				else
				{
					//Bad data received
					nKRet = KERN_INVALID_VALUE;
				}
			}
			else if(nKRet == MACH_RCV_TIMED_OUT)
			{
				//Timed out
				res = MRRES_TimedOut;
			}
		}
		else
		{
			//initialize() was not called successfully prior to this call
			assert(false);
			
			nKRet = KERN_NOT_IN_SET;
		}

		if(pOutKernErr)
			*pOutKernErr = nKRet;
		if(puiOutClientPID)
			*puiOutClientPID = pid_client;

		return res;
	}
	
	
	
private:
	
	
	///IMPORTANT: Must be called from within a lock!
	///RETURN:
	///     - true if no errors
	bool _uninit()
	{
		bool bRes = false;
		
		kern_return_t nKRet;
		
		//See if we haven't uninitialized previously
		if(_port != MACH_PORT_NULL)
		{
			if(_endpoint == MET_SERVER)
			{
				//Server
				nKRet = _removeServerPort(_strPortName.c_str());
				if(nKRet == KERN_SUCCESS)
				{
					//Done
					bRes = true;
				}
				else
				{
					//Failed
					assert(false);
				}
			}
			else if(_endpoint == MET_CLIENT)
			{
				//Client - nothing to do
				bRes = true;
			}
			else
			{
				//Bad end-point type
				assert(false);
			}
			
			//Reset internal variables
			_endpoint = MET_Unknown;
			_port = MACH_PORT_NULL;
			_strPortName.clear();
		}
		else
		{
			//Already un-initialized
			bRes = true;
		}

		return bRes;
	}
	
	
	///Prepare 'msg' for sending
	///'pPtr' = memory pointer for the local buffer to send
	///'szcb' = size of 'pPtr' in bytes
	///RETURN:
	///     - true if port is available
	bool _set_msg_for_sending_mem_ptr(MACH_MSG_BASE& msg,
										const void* pPtr,
										mach_msg_size_t szcb)
	{
		mach_port_t port;
		
		if(true)
		{
			//Act from within a lock
			READER_LOCK rdl(_lock);
			
			port = _port;
		}

		if(port == MACH_PORT_NULL)
		{
			//No port
			return false;
		}
		
		msg = {};
	
		//NOTE that the MACH_MSGH_BITS_COMPLEX flag instructs the kernel
		//     to marshal the memory buffer pointed by 'msg.data.address' into the
		//     target process where the Mach message will be received.
		//
		msg.hdr.msgh_remote_port = port;
		msg.hdr.msgh_local_port = MACH_PORT_NULL;
		msg.hdr.msgh_bits = MACH_MSGH_BITS(MACH_MSG_TYPE_COPY_SEND, MACH_MSG_TYPE_MAKE_SEND) |
		                    MACH_MSGH_BITS_COMPLEX;
		msg.hdr.msgh_size = sizeof(msg);
		
		msg.body.msgh_descriptor_count = 1;
		msg.data.address = (void*)pPtr;
		msg.data.size = szcb;
		msg.data.copy = MACH_MSG_VIRTUAL_COPY;
		msg.data.type = MACH_MSG_OOL_DESCRIPTOR;
		msg.count = msg.data.size;

		return true;
	}

	
	
	///Associate a unique name with a port
	///INFO: This name can be used by the IPC client to find the port
	///'pName' = unique port name (in reverse DNS format) - it must match what is specified in the 
	///          MachServices key for the launchd.plist file that is used to launch the deamon:
	///           https://www.manpagez.com/man/5/launchd.plist/
	///'pOutPort' = receives associated port, if success -- call _removeServerPort() to de-associate it
	///RETURN:
	///     - KERN_SUCCESS if success, otherwise
	///     - Kernel error code
	kern_return_t _createServerPort(const char* pName,
									mach_port_t* pOutPort)
	{
		kern_return_t nKRet = KERN_INVALID_ARGUMENT;
		assert(pOutPort);
		
		mach_port_t port = MACH_PORT_NULL;

		if(pName &&
			pName[0])
		{
			//Retrieve the receive rights for our named port from the launchd
			nKRet = bootstrap_check_in(bootstrap_port,
										pName,
										&port);
			
			if(nKRet != KERN_SUCCESS)
			{
				//Error
				//BOOTSTRAP_UNKNOWN_SERVICE = 1102 = 0x44E - if launchd.plist for this daemon does not contain 'pName'
				assert(false);
			}
		}
		
		*pOutPort = port;
		
		return nKRet;
	}
	
	
	
	///Find previously created port by 'pName'
	///'pName' = unique port name (in reverse DNS format) - it must match what is specified 
	//           in the MachServices key for the launchd.plist file that was used to launch the deamon:
	///           https://www.manpagez.com/man/5/launchd.plist/
	///'pOutPort' = receives associated port, if success (there's no need to release it)
	///RETURN:
	///     - KERN_SUCCESS if success, otherwise
	///     - Kernel error code
	kern_return_t _findServerPort(const char* pName,
									mach_port_t* pOutPort)
	{
		kern_return_t nKRet = KERN_INVALID_ARGUMENT;
		assert(pOutPort);
		
		mach_port_t port = MACH_PORT_NULL;

		if(pName &&
			pName[0])
		{
			nKRet = bootstrap_look_up(bootstrap_port, pName, &port);
			if(nKRet != KERN_SUCCESS)
			{
				//BOOTSTRAP_UNKNOWN_SERVICE = 1102 - if there's no such port name
				assert(false);
			}
		}

		*pOutPort = port;

		return nKRet;
	}
	
	
	
	///De-associate 'pName' port - that was previously associated with the _createServerPort() function
	///RETURN:
	///     - KERN_SUCCESS if success, otherwise
	///     - Kernel error code
	kern_return_t _removeServerPort(const char* pName)
	{
		kern_return_t nKRet = KERN_INVALID_ARGUMENT;
		
		if(pName &&
			pName[0])
		{
			//INFO: The following is needed only if port was created here.
			//      In our case the port was created by launchd, that owns it...
			
			//mach_port_t* p_null_port = nullptr;
			nKRet = KERN_SUCCESS;   //bootstrap_check_in(bootstrap_port, pName, p_null_port);
			if(nKRet != KERN_SUCCESS)
			{
				//BOOTSTRAP_SERVICE_ACTIVE == 1103 - if port is still active
				assert(false);
			}
		}
		
		return nKRet;
	}
	
	
	
private:
	//Copy constructor and assignments are NOT available!
	MachComm(const MachComm& s) = delete;
	MachComm& operator = (const MachComm& s) = delete;
	
private:
	
	RDR_WRTR _lock;                                 //Lock for accessing this class' private members
	
	MACH_ENDPOINT_TYPE _endpoint = MET_Unknown;     //Type of the end-point for this class
	mach_port_t _port = MACH_PORT_NULL;             //Port used for IPC
	std::string _strPortName;                       //Name of the port
	
};




#endif /* MachComm_h */

The code above uses a helper class for providing synchronization locks:

C++ (READER_LOCK, WRITER_LOCK classes)[Copy]
//
//  Implementation of synchronization reader/writer locks
//
//  dennisbabkin.com
//

#ifndef locks_h
#define locks_h

#include <pthread.h>
#include <sched.h>
#include <exception>


///Class that implements a reader/writer lock's base
///
struct RDR_WRTR
{
	RDR_WRTR()
	{
	}
	
	///RETURN:
	///     - 0 if success
	///     - Otherwise it's an error, for instance: EDEADLK
	int EnterReaderLock()
	{
		return pthread_rwlock_rdlock(&_lock);
	}
	
	void LeaveReaderLock()
	{
		int nErr = pthread_rwlock_unlock(&_lock);
		if(nErr != 0)
		{
			//Logical problem - need to crash
			assert(false);
			abort();
		}
	}
	
	///RETURN:
	///     - true if lock was not available for reading (NOTE that it may be now!)
	///     - false if lock was available for reading (NOTE that it may not be anymore!)
	bool WasReaderLocked()
	{
		//Try to acquire it for reading, but do not block
		int nErr = pthread_rwlock_tryrdlock(&_lock);
		if(nErr == 0)
		{
			//Acquired it, need to release it
			nErr = pthread_rwlock_unlock(&_lock);
			if(nErr == 0)
			{
				return false;
			}
			else
			{
				//Logical problem - need to crash
				assert(false);
				abort();
			}
		}
		else if(nErr == EBUSY)
		{
			//Not available
		}
		else
		{
			//Logical problem - need to crash
			assert(false);
			abort();
		}

		return true;
	}
	
	
	///RETURN:
	///     - 0 if success
	///     - Otherwise it's an error, for instance: EDEADLK
	int EnterWriterLock()
	{
		return pthread_rwlock_wrlock(&_lock);
	}
	
	void LeaveWriterLock()
	{
		int nErr = pthread_rwlock_unlock(&_lock);
		if(nErr != 0)
		{
			//Logical problem - need to crash
			assert(false);
			abort();
		}
	}

	
	///RETURN:
	///     - true if lock was not available for writing (NOTE that it may be now!)
	///     - false if lock was available for writing (NOTE that it may not be anymore!)
	bool WasWriterLocked()
	{
		//Try to acquire it for reading, but do not block
		int nErr = pthread_rwlock_trywrlock(&_lock);
		if(nErr == 0)
		{
			//Acquired it, need to release it
			nErr = pthread_rwlock_unlock(&_lock);
			if(nErr == 0)
			{
				return false;
			}
			else
			{
				//Logical problem - need to crash
				assert(false);
				abort();
			}
		}
		else if(nErr == EBUSY)
		{
			//Not available
		}
		else
		{
			//Logical problem - need to crash
			assert(false);
			abort();
		}

		return true;
	}
	
	

	
private:
	//Copy constructor and assignments are NOT available!
	RDR_WRTR(const RDR_WRTR& s) = delete;
	RDR_WRTR& operator = (const RDR_WRTR& s) = delete;
	
	
private:
	pthread_rwlock_t _lock = PTHREAD_RWLOCK_INITIALIZER;
};




///Class that implements a reader-lock
//INFO: This lock will block only if a writer-lock is currently holding it.
//
struct READER_LOCK
{
	READER_LOCK(RDR_WRTR& rwl)
		: _rwl(rwl)
	{
		int nErr = rwl.EnterReaderLock();
		if(nErr != 0)
		{
			//Logical problem - need to crash
			assert(false);
			abort();
		}
	}

	~READER_LOCK()
	{
		_rwl.LeaveReaderLock();
	}

	
private:
	//Copy constructor and assignments are NOT available!
	READER_LOCK(const READER_LOCK& s) = delete;
	READER_LOCK& operator = (const READER_LOCK& s) = delete;
	
private:
	RDR_WRTR& _rwl;
};





///Class that implements a writer-lock
//INFO: This lock will block if a reader- or a writer-lock is currently holding it.
//
struct WRITER_LOCK
{
	WRITER_LOCK(RDR_WRTR& rwl)
		: _rwl(rwl)
	{
		int nErr = rwl.EnterWriterLock();
		if(nErr != 0)
		{
			//Logical problem - need to crash
			assert(false);
			abort();
		}
	}

	~WRITER_LOCK()
	{
		_rwl.LeaveWriterLock();
	}

private:
	//Copy constructor and assignments are NOT available!
	WRITER_LOCK(const WRITER_LOCK& s) = delete;
	WRITER_LOCK& operator = (const WRITER_LOCK& s) = delete;
	
private:
	RDR_WRTR& _rwl;
};

Now that we have our wrapper class we can start using it.

IPC Server

To use my MachComm class as the IPC server you would place it in a worker thread so that it can safely wait for incoming messages. Then use its timeout feature to check periodically for a stop-now variable (or g_bStopNow) that can alert it when the daemon is exiting:

C++ (IPC Server)[Copy]
bool g_bStopNow = false;		//Global variable that will be set to true when our daemon is exiting

///Thread that implements the IPC server using Mach messages
///
void* thread_mach_server(void* p)
{
    MachComm mc;
    
    //Initialize our MachComm class as a server end-point
    if(mc.initialize(MET_SERVER, MACH_COMM_PORT_NAME))
    {
        MACH_RCV_DATA mrd;
        pid_t pid;
        kern_return_t nKRet;
        
        //Go into a receiving loop
        for(; !g_bStopNow;)
        {
            //Wait for a Mach message to arrive
            MACH_RCV_RES rez = mc.receiveMsg(mrd,
                                             &pid,
                                             100,       //100 ms timeout
                                             &nKRet);
            
            if(rez == MRRES_Success)
            {
                //Received a message, but first we need to vet the sender
				//NOTE that this is very important to do since Mach message can
				//     be sent to us by any process!
				//
                bool resChk = checkIpcClient(pid);
                if(resChk)
                {
                    //Can go ahead and process this message
					
					//IMPORTANT:
					//			Ideally you would use a thread-pool (or worker threads) for processing the messages
					//			and won't do it here. The main idea is not to block this receiving loop for too long.
                    
                    //In our simple test, the client will be sending us a C string.
					//Thus, let's output it to the console.
					//
					size_t szLen;
                    const char* pMsg = (const char*)mrd.getData(&szLen);
                    assert(pMsg);
                    std::string strMsg;
                    strMsg.assign(pMsg, szLen);

					printf("Received: %s\n", strMsg.c_str());

                }
                else
                {
                    //Bad client request - ignore it
                    assert(false);
                }
            }
            else if(rez != MRRES_TimedOut)
            {
                //Error receiving a Mach message
                assert(false);
                
                break;
            }
        }
    }
    else
    {
        //Failed to initialize the IPC server
		assert(false);
    }
    
    return nullptr;
}

In the code sample above, you would set the MACH_COMM_PORT_NAME variable on a global scope to a unqiue reverse-DNS name for the communication port. For instance: "com.example.MachComm.port.name". The reason you would declare it on a global scope is so that it can be also used from the IPC clients.

But before you can use the Mach port name, it has to be included in the launchd.plist file that is used by the Launchd to run your daemon. It uses the MachServices node in that .plist, that should be set to the type Boolean with the value true, and contain the exact same name of your Mach port, as you used in MACH_COMM_PORT_NAME:

com.example.MachComm.plist
com.example.MachComm.plist[Copy]
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>MachServices</key>
	<dict>
		<key>com.example.MachComm.port.name</key>
		<true/>
	</dict>
	<key>Disabled</key>
	<false/>
	<key>Label</key>
	<string>com.example.MachComm</string>
	<key>Program</key>
	<string>/Library/PrivilegedHelperTools/com.example/MachComm</string>
	<key>ProgramArguments</key>
	<array>
		<string>-d</string>
	</array>
	<key>RunAtLoad</key>
	<true/>
	<key>KeepAlive</key>
	<true/>
</dict>
</plist>	
You can specify more than one Mach port in the MachServices node to be used by your daemon. But in our case we need only one port.

Additional point of interest is how the IPC server and clients get a Mach port (handle) from the Mach port name. If you noticed, the Mach port is represented by just an integer (or a handle.) So the conversion on the server-side is done by the bootstrap_check_in API inside my MachComm::_createServerPort function, that is a part of the servers/bootstrap.h library. And on the client side, I'm using the bootstrap_look_up API inside my MachComm::_findServerPort function, that looks up an already created Mach port. By the way, the latter limitation stipulates that the IPC server must be initialized first, which will be the case with the launch daemon.

Security Considerations For IPC Server

Another point of interest is the checkIpcClient function. Its main purpose is to vet the incoming request. You would want to do so because a Mach message could be sent to our daemon (running as root) by any process that knows the port name. So we need to make sure that the IPC client process, that is sending the message, is the one that we can trust.

There are several ways to establish an IPC client process trust. The one suggested for macOS is for you to code-sign your IPC client process, and to employ what is known as the "Hardened Runtime" (here's how Apple suggests you can configure it.)

In my case though, mostly for simplicity of this example, I will rely on a different technique. Since we have the PID of the IPC client process (which we can trust, as we received it from the kernel), we can query the kernel for the location of the client process on disk and make sure that it's located in the same directory as our daemon. Then, assuming that we placed our deamon in a directory that is write-protected for non-root processes, and as long as our IPC client process is in the same directory, we will assume that it is safe for our purposes.

One obvious bypass of my assumption above is a code-injection into our IPC client process. So keep that in mind when using my checkIpcClient function. A much safer approach would be to rely on the "Hardened Runtime".

So let's put this into code:

C++[Copy]
#define SIZEOF(f) (sizeof(f) / sizeof(f[0]))

///Determines if we can receive Mach messages from the process with 'pid' 
///RETURN:
///     - true if yes, we can
///     - false if no, we cannot
bool checkIpcClient(pid_t pid)
{
	bool res = false;
	
	char buffPath[PROC_PIDPATHINFO_MAXSIZE] = {};
	
	//Retrieve process path
	//
	int nln = proc_pidpath(pid, buffPath, SIZEOF(buffPath));
	if(nln > 0 &&
		nln < SIZEOF(buffPath))
	{
		//The 'pid' process and our process must run in the same directory
		
		//Look for last slash
		//
		const char* pClientFile = findLastSlash(buffPath, nln);
		if(pClientFile)
		{
			intptr_t nchLnClientDir = pClientFile - buffPath;
			if(nchLnClientDir > 0 &&
				nchLnClientDir < SIZEOF(buffPath))
			{
				std::string strClientDir(buffPath, nchLnClientDir);
				
				const char* pThisFile = findLastSlash(g_ProcInfo.strProcFilePath);
				if(pThisFile)
				{
					intptr_t nchLnThisDir = pThisFile - g_ProcInfo.strProcFilePath.c_str();
					if(nchLnThisDir > 0 &&
						nchLnThisDir < g_ProcInfo.strProcFilePath.size())
					{
						std::string strThisDir(g_ProcInfo.strProcFilePath.c_str(), nchLnThisDir);
						
						//Directories must be the same
						//
						if(strcmp(strClientDir.c_str(), strThisDir.c_str()) == 0)
						{
							//Yes, they are
							res = true;
						}
					}
				}
			}
		}
	}
	
	return res;
}



///Helper function that finds the last slash in 'pPath'
///'szcbLn' = size of 'pPath' in chars, or negative to calculate it automatically
///RETURN:
///     - pointer to the last slash in 'pPath', or
///     - 0 if not found
const char* findLastSlash(const char* pPath, intptr_t ncbLn)
{
    const char* pFndPtr = nullptr;
    
    if(pPath)
    {
        //Do we have the length?
        if(ncbLn < 0)
        {
            ncbLn = strlen(pPath);
        }
     
        for(intptr_t i = ncbLn - 1; i >= 0; i--)
        {
            if(pPath[i] == '/')
            {
                pFndPtr = pPath + i;
                break;
            }
        }
    }
    
    return pFndPtr;
}

Another important security consideration that I need to mention is in my MachComm::receiveMsg function for the IPC server. If you look it up in the code above, see that I'm checking for the presence of the MACH_MSGH_BITS_COMPLEX bit in the msgh_bits in the received Mach message, and proceed with using the rcv.msg.data.address pointer only if that flag is present. It is very important to do, as if that flag is not set, this would mean that the Mach kernel did not send us that pointer, and if we start reading from it we will most certainly crash our daemon. And thus subject it to the Denial-Of-Service attack.

On the other hand, if the MACH_MSGH_BITS_COMPLEX bit is set in msgh_bits, this will guarantee that the kernel had marshalled the pointer that we received in rcv.msg.data.address and we can safely read rcv.msg.data.size number of bytes from it.

IPC Client(s)

To use the MachComm class as the IPC client to send messages to the server, do the following:

  1. You need to declare a global variable of the type MachComm and initialize it as the IPC client. The best place to do this is when your IPC client process starts up. You will need to do it only once:
    C++ (IPC Client initialization)[Copy]
    MachComm g_mc;		//Declared on a global scope
    
    ///Process initialization routine that is called once during the process startup
    ///
    void InitializeProcess()
    {
    	//...
    
    	//Set up IPC client
    	if(!g_mc.initialize(MET_CLIENT, MACH_COMM_PORT_NAME))
    	{
    		//Failed
    		assert(false);
    	}
    
    	//...
    }

    Note that we are using the same (shared) MACH_COMM_PORT_NAME variable as we did for the IPC server.

  2. Then, whenever you want to send a Mach message to the IPC server, do the following:
    C++ (IPC Client - send message)[Copy]
    const char* pMsg = "Hello!";
    if(g_mc.sendMsg((const char*)pMsg, (int)strlen(pMsg)) != KERN_SUCCESS)
    {
    	//Failed to send Mach message
    	assert(false);
    }

    The binary format of the message that is accepted by the MachComm::sendMsg function is totally up to you, as well as it's size. In the example above I'm just sending a simple Hello string, and the server outputs it to the console to demonstrate the concept. In a real example you would probably declare a shared struct and use it to send the data between the clients and the server.

    You can repeat sending messages as many times as you need to. In case of an error in transmission of the message the MachComm::sendMsg call will fail with an error code other than KERN_SUCCESS, so you will know that the server did not receive your message.

Conclusion

The example above is just a simple implementation of the IPC using Mach messages, doing just the basic send/receive function. I am ignoring more advanced features of Mach messages, such as atomic send-receive operation, port notifications, as well as dealing with port permissions such as transferring them during a transmission. If there's an interest in it, I may explore those more advanced topics in one of my future blog posts.

The mach_msg function (and its cousin mach_msg_overwrite) are quite versatile and support additional options. I will leave it up to the reader to explore those. This blog post is just a primer to get you started, although if you expand my MachComm class with added functionality, feel free to share a link in the comments below.

Related Articles