XMPP, the Internet Standard eXtensible Messaging and Presence Protocol is being widely adopted for Instant Messaging (IM), Group Chat and Presence services in military networks.
In this article Steve Kille looks at the military tactical requirements for IM, Group Chat and Presence, discusses briefly why XMPP is ideal for these services and as a building block for situational awareness systems in support of voice and video communication. Tactical networks often need to make use of Radio and Satellite networks with constrained bandwidth, high latency and difficult operational characteristics. The article concludes by looking at the problems of deploying XMPP over such networks, and reveals how XMPP can be effectively deployed in such environments.
IM, Presence and Group Chat for Tactical Networks
Modern tactical communication has a complex mix of requirements that can include deployed units with a variety of communication links, participants from multiple countries working closely together and involvement of remote personnel (for example to provide specialist advice, or legal involvement with decisions to engage). The Instant Messaging family of services is a useful and important component of tactical communications.
One-To-One Chat
There are situations in which using 1:1 IM to send or exchange short messages is more effective than formal messaging or voice communication:
- When communication links have capacity to send data but not voice.
- In very noisy situations where voice cannot be heard.
- In situations where absolute silence must be maintained.
- To provide information from a location where typing is easy (e.g., field HQ) to field locations in order to provide information that can complement voice.
In some operational deployments (including many military scenarios) group communication is used more than 1:1 IM communication. If data is being provided, it makes sense to share it so that all interested parties can see the information. For example, it will enable external strategists or lawyers to observe communication in real time, and provide input as appropriate. It often makes sense to share information in the field, for example a group of ships jointly working out who will target what and how. Group chat is an important operational capability.
Presence
Information regarding online presence can be useful data in support of other communication. Extended presences (additional information associated with presence) can also enable useful sharing. In particular geo-location can be supported as extended presence, enabling presence as a means of location tracking.
Radio & Satellite Constraints on Tactical Networks
Tactical communication needs to use data communication links of widely varying speed and quality. It is important to be able to gain the benefits of fast networking when it is available to support a range of modern applications. However it is also important to be able to use slower links, when they are the only option available. As well as speed, latency and reliability are important characteristics that impact applications using data communications links. Key network technologies are:
- Satellite. Modern satellite systems provide bandwidth of 1 Mbps+, although many deployed systems are much slower (e.g., 4800 bps). Geostationary satellites have a latency of about 0.5 secs— it’s common to chain multiple satellite links, giving greater end-to-end latency.
- Line Of Sight Radio (VHF). VHF Radio is widely used in tactical communications. Data links usually operate at 9600 bps (single VHF channel). Multiple channels can be combined to give full duplex communication and higher data rates. Although the physical latency is low, for a standard half duplex link, the low data rate will lead to turnaround times of half a second or more.
- Line Of Sight Radio (UHF and faster). Higher frequency radio will provide higher bandwidth than VHF. Different bands give different operational characteristics, ranges and opportunities for deployment. All are restricted to line of sight communications.
- Beyond Line Of Sight Radio (HF). HF Radio provides data rates from 75-9600 bps. Data rates can be highly variable. Turnaround time is typically 5-30 seconds. In order to optimize link utilization, data link protocols will hold the link open, leading to operational latency of two minutes or so. HF links can often be unreliable.
Why XMPP for Tactical Networks?
XMPP is the protocol family of choice for military networks for one simple reason: Standardization. It enables interconnection of heterogeneous components, and integration of partner networks from other countries. In particular:
- The standard client/server protocol enables integration of users on a wide variety of systems, from specialized deployed units to office systems at HQ.
- The standard server/server protocol enables easy peer system integration.
XMPP appears to be an ideal base for a standardized situational awareness protocol family. This gives interoperability benefits over proprietary systems, where all participants must use the same product.
XMPP Configuration Options
The diagram on the next page shows two options for providing XMPP service over a slow link to a single client using standard XMPP protocols. In XMPP a client connects to a single server, and then there are direct server to server connections to support communication with clients on other servers.
In the first option, the client connects to its server over a slow link. In the second option, the client is local to its server (fast) and the server communicates with other XMPP servers over a shared slow link. The relative performance of the two configurations can be considered for basic traffic:
1—For message exchange, a message will traverse exactly one link in each case, so traffic load is similar.
2—Per connection overhead (setup and keepalive) is higher in option 2, as there are more connections over the slow link.
3—When a peer changes its presence status, this will be transmitted exactly once to the client, so overhead is the same in both scenarios.
4—When the client changes its presence status, this will be sent once over the client/server link and then over each of the server/server links which has a client that is monitoring presence. This gives a higher overhead for option 2.
This clearly shows that, for a single client, operating the client/server protocol over the slow link (option 1) is going to be most efficient at the network level. Analysis in the next section is focused on client/server.
XMPP Protocol Performance
This section looks at some XMPP protocol examples, to give a sense of the protocol overhead associated with XMPP. It is not intended as a formal analysis. XMPP protocol uses an XML text encoding.
<message from=’juliet@example.com’
to=’romeo@example.net’
xml:lang=’en’>
<body>Art thou not Romeo, and a Montague?</body>
</message>
This is an example message taken from the core XMPP standard (RFC 3920). A minimal message such as this example will have an overhead of around 100 bytes. Typical XMPP clients will use more features leading to a typical operational overhead of 2-300 bytes per message. The overhead for messages to group chat is similar. The main difference is that a client will send the message to a room, and then the same message will come back again (from the room) so the line is used twice.
Presence updates (Chat State Notifications) are a similar size to messages (2-300 bytes). One of these will be received whenever a roster member changes status. When the client changes status, one will be sent and then returned back from the server.
Another common message type is IQ, which is used by the client to check server status from time to time. This has a typical overhead of 70 bytes each way. Startup has the highest overhead. The following measurements use two popular XMPP Clients (Pidgin; PSI) for a user with about 20 entries on the roster Basic data transfer as follows:
- Pidgin: 32 Kbyte (4.6 Kbyte sent to the server; 27 Kbyte back)
- PSI: 48 Kbyte (10.6 Kbyte sent to the server; 37.4 Kbyte back)
The startup retrieves a fairly large JPEG photo as a part of the user profile. If this is not done, the modified data is:
- Pidgin: 13.9 Kbyte (4 Kbyte sent to the server; 9.9 Kbyte back)
- PSI: 31.3 Kbyte (9.3 Kbyte sent to the server; 21.9 Kbyte back)
XMPP Compression
XMPP provides compression using the DEFLATE algorithm. This can be applied in one of two ways:
- Directly with the XMPP protocol.
- Within TLS (Transport Layer Security)
1—XMPP is a text encoded protocol, and DEFLATE will give an immediate benefit for typical traffic.It is worth considering how much compression is provided. The DEFLATE specification in RFC 1951 notes that “English text usually compresses by a factor of 2.5 to 3” (i.e., to 33-40 percent of the original size). Given that IM and MUC traffic is the primary user data carried by XMPP, this is useful compression. Protocol also compresses effectively.
2—XMPP has a regular structure, and common elements are often repeated. DEFLATE optimizes for this by reference to data transmitted, and will give substantial compression as use increases. For example, if a peer user is changing presence status between a small number of values, the same packets will be used to report this change, and DEFLATE will give very high compression.
Ad hoc measurements of a short lived connection suggest that typical presence updates will compress from 100 to 50 bytes, and typical message overhead will compress from 300 bytes to 120 bytes. Other measurements suggest that higher compression can be achieved (factor 4.5-5.8, so reducing data to range 17-23 percent of original size).
Startup measurements using PSI (Pidgin does not support compression) and the earlier setup gives:
- Without Compress: 31.3 Kbyte (9.3 Kbyte sent to the server; 21.9 Kbyte back)
- With Compression: 8.2 Kbyte (3 Kbyte sent to the server; 5.2 Kbyte back).
These compression characteristics and the high startup overhead mean that network performance is strongly optimized for long lived connections.
XMPP Design and Scaling
When looking at the data numbers in the context of very slow networks, it might appear that XMPP has poor optimization. It is worth considering the broad characteristics and design goals of XMPP:
- XMPP is designed to provide an extensible communications and information publishing infrastructure. XML is a natural choice to achieve this, and provides an extensible approach that can be easily used in many environments. Although XML is not very compact, the data sizes are small on modern networks, particularly in comparison with voice, video and other data in wide use.
- On a modern network, XMPP’s network usage is very light.
- XMPP clients are generally developed to provide “best service” to the user. There is no need to focus on optimizing network traffic.
- The hard problem for a distributed or federated IM system is support of presence. Message switching load scales in a natural manner, with load proportional to usage. With presence, there is a need to update many clients over the network for each status change. Care needs to be taken to ensure that this scales well, and the XMPP design has taken considerable care on this point.
With this basic understanding in place of XMPP performance, we can consider performance of XMPP Client/Server interaction over a medium speed network of 28 Kbyte’s per second (3.5 Kbyte per second).
Startup of a typical client/server connection will take a few seconds and saturate the network during this time. After this two things will come into play:
- Compression (which should be used) will work increasingly effectively to optimize data transfer volumes.
- The traffic caused by typical short message and chat use (e.g., participating in a number of simultaneous 1:1 chats and group chat sessions) and presence update from a moderate sized roster will be feasible. A link of this speed would support a peak load of around 20 short messages per second. This would appear sufficient.
Some optimization could be achieved by using XMPP in a “more efficient” manner and to reduce the number of messages sent. For example, many clients send information of the form “User XXX is Typing”. It could be argued that this is not really needed and is just wasting network capacity. On the other hand, in many situations there is ample network capacity to do this, and this additional information provides value to the recipient (they may have an urgent requirement on the response, and it is useful to know that it is being prepared). There is a danger that attempts to optimize traffic will reduce the value of the service.
This will be particularly difficult to handle in a network that has a mixture of links of varying speed.
It is suggested that in all but the very slowest of networks, that straightforward deployment of XMPP will be viable and sensible.
HF Radio & STANAG 5066
HF Radio is the most difficult communications medium for which there is a general support requirement. It has low bandwidth, very high latency, and poor reliability. In order to use HF efficiently for data traffic, STANAG 5066 is the approach of choice. To use the HF Network efficiently, it is important to operate the application directly over STANAG 5066, and not to use IP. This is discussed in the Isode whitepaper “HF Radio & Network Centric Warfare”, which can be found on the Isode website (www.isode.com). In order to use XMPP over HF Radio, it is important to include STANAG 5066 as a part of the solution architecture.
Point to Point Deployment Over Slow Networks
Consider operating standard XMPP over a network running at 2.4 Kbyte/sec (300 bytes per second), which is a typical (but not minimum) HF radio speed. Startup at this speed is going to be very slow (over a minute for the example connection described earlier). This will be compounded by two elements:
1—Long Lived connections may be impractical for several reasons:Even in steady state, 300 bytes per second is going to be tight for IM traffic. Consider (without protocol overhead) a user monitoring several group chats. It is easy to picture that there would be 300 bytes per second of user data, without even considering XMPP protocol and presence overhead.
2—High latency will make things much worse. XMPP startup involves around nine handshakes, which will have a significant impact if network latency is high.
- Slow networks are often unreliable, which would mitigate against long lived connections.
- For a slow network, the overhead of maintaining an open connection may be unacceptable.
- HF radio does not do data and voice simultaneously, so data links have to be closed for voice traffic.
It is very clear that simple deployment of XMPP over a slow network is not going to work. We now consider what needs to be done to address this.
The diagram at the top of the next page shows the architecture Isode recommends for deployment of XMPP over slow links. On both sides of the slow link is “standard XMPP” and a special protocol is used between the servers over the slow link. There are a number of advantages to this architecture:
- The impact of the slow link is hidden from users not affected by it.
- Standard XMPP clients can be used.
- Multiple clients can be supported on an end system sharing access over a slow link, without significant overhead. Note that the slow link has only one server at each end, so that distribution of common data to multiple servers does not happen over the slow link.
1—There should be no connection establishment. This will be achieved in two ways:Multicast and EMCON
2—Data encoding should be optimized for each packet, as algorithms such as DEFLATE are not very useful for connectionless operation.
- It should offer a connectionless mapping onto IP using UDP, with reliability provided by the application.
- There should be a mapping onto RCOP (STANAG 5066 Reliable Connection Oriented Protocol) to provide efficient operation for HF. This will typically be used for short data transfers.
3—There should be a filtering option, to remove traffic that is not considered necessary over the slow link.
4—Retransmission should be XMPP aware. For example, when sending presence, the current value should always be used.
Many slow networks use underlying broadcast transmission, and it is desirable that the application can make use of this. A related problem is that it is desirable to support end point in radio silence (EMCON or Emission Control). This means operation without acknowledgements. The architecture for this is at the top of the next page.
The optimized protocol discussed in the previous section may be extended to support this.
- Use of multicast will require a completely connectionless mapping: IP networks are supported with UDP and IP Multicast.
- HF is supported with STANAG 5066 UDOP (Unreliable Datagram Oriented Protocol).
- Presence status will always be broadcast, so that all interested parties can read it.
- Group Chat messages will always be broadcast and selected by interested parties.
- Retransmission’s can be selective or automatic (to support EMCON).
XMPP is important technology for supporting military tactical communication. It is useful directly, and as a basis for interoperable situational awareness systems. The protocol has good functionality, extensibility and scaling characteristics and can be deployed directly over fast and medium speed networks.
For very slow networks (e.g., 2.4 Kbyte’s per second) and over HF Radio at all speeds, the protocol overheads of XMPP, in particular startup, are too high, and a modified approach is needed to take advantage of XMPP.
Isode recommends a server-to-server architecture, which isolates the performance impact of the slow link. Variants are proposed to deal with:
- Operation over IP (using UDP) and over HF Radio using STANAG 5066.
- Support of point to point links, and multicast configuration to optimize use of Satellite and Radio networks.
About the author
Steve Kille is the CEO of Isode, which he founded in 1992 and re-launched in 2002. Steve has been closely involved with many key Internet technologies since 1980 and has brought several of them to market with Isode. Steve has 20 years of experience with messaging, directory, and security, and has been responsible for a range of widely deployed products and standards. He has written 40+ RFCs (Internet Standards), and is one of the authors of LDAP (Lightweight Directory Access Protocol). From 1981 to 1992, he was a Senior Research Fellow at University College London and led U.S. and European funded research projects on messaging, directory, networking and distributed systems. He has published a book and numerous papers and articles.
Steve has BA and MA honours degrees in Physics from Oxford University, and Masters degrees in Electrical Engineering from University of Manchester Institute of Science and Technology and Stanford University, where he was a Fulbright scholar. He was born in London, where he now lives.