Protocol definition

From XAP Automation
Jump to: navigation, search

Contents

xAP Specification

xAP eXtensible Automation Protocol Specification Specification Release v.1.2 - 29th November, 2002 Document Release v.1.2-9 (Final) - 1st December, 2002

Introduction

This document is an RFC which seeks to establish an open, extensible communications protocol to allow appliances and software applications to interact within the domestic environment. This document is currently owned and maintained by Patrick Lidstone. (mailto:Patrick@xAPautomation.org).

Acknowledgements

Many members of the xAP group mailing list have contributed directly and indirectly to this specification. Special thanks to: Ian Lowe, James Brett, Jon Whiten and, as well as the xAP specification sub-committee: Mark Harrison, Kevin Hawkins, John Tankard and Stuart Poulton.

Copyright

This document is Copyright (c) 2002, Patrick Lidstone, Mark Harrison, Kevin Hawkins, and John Tankard, all rights reserved.

Change History

Spec v.1.1: Introduces the keyword "class" into the header; sequence of all header tags is fixed; messages containing multiple sections are explicitly identified through the use of "style= multi" tag in the header; ASCII hex case specified.

Spec v.1.2:

  • Compact form of source.instance
  • Introduction of targeted addressing mode (through target tag)
  • Introduction of uid unique identifier tag
  • Deprecation of "style" tag
  • Wild card addressing
  • Variable depth instances
  • Header tag order revised

Synopsis

xAP is an open protocol intended to support the integration of telemetry and control devices primarily within the home - it is the glue that supports the interconnectivity of Home Automation devices.

The primary design objectives are:

  • Minimalist, elegant and simple, easy to implement/retrofit
  • Suitable for use with a wide range of processing capabilities, from embedded controllers to fully fledged PC's
  • Operating system agnostic
  • Programming language agnostic
  • Network agnostic

These are lofty design goals, and initial implementation is currently focused on IP based networks, PC's running Linux, Win/32, Rabbit and PIC embedded microcontrollers.

In addition, the chosen implementation mechanism for xAP has three further key benefits:

  • it reuses existing infrastructure where possible (for example it is able to co-exist on a wireless LAN)
  • requires no central controller,
  • and as such it provides a highly distributed and fault tolerant architecture which allows continued operation of systems wherever possible even in the event of individual component failure.

Status

The xAP group website can be found at http://www.xapautomation.org/.

Reference code implementation of xAP for Win/32 PCs, Linux (including the TiVO PVR), Mac OS-X, Pocket PC (Windows CE 2002), as well as for the PIC and Rabbit microcontrollers is available via the xAP group website under the xAP Apps section. Resources

Discussion mailing list for all xAP related: http://groups.yahoo.com/group/xapautomation

The official xAP web site: http://www.xapautomation.org/

Protocol Basics

A xAP enabled application (a xAPp) may be either a "sender", "receiver", or both. The xAP enabled device is connected to a communications network. Common communications networks include, but are not limited to: Ethernet ("computer" networks); RS232, RS485 serial networks; wireless.

As a sender, a xAPp broadcasts a message. This message contains sufficient information to uniquely identify the sender and the semantics of the message (the significance of the message contents - what the message actually means). Note that the message does not necessarily contain information which explicitly identifies an intended recipient, it only identifies the source itself.

As a receiver, a xAPp listens to broadcast messages. The receiver introspects the message (header) to identify whether the message contains information that is of specific interest to the receiver. Accordingly, the receiver then either discards the message or processes it.

It is possible for many receivers to intercept the same broadcast message concurrently. It is also possible for receivers to receive and process messages from more than one source. This allows for ad-hoc many-to-many relationships, on an implicit basis, without the need to configure explicit connections between various devices.

Multiple Network Support

xAPps that are connected to different physical networks (RS232, Ethernet, radio) may intercommunicate. In order to pass messages between the different physical networks, a protocol converter - a bridge - is required. In order to limit the traffic passing between networks, the bridge may be optionally configured to only forward messages which originate from specifically nominated sources.

Depending on the physical network the xAP message is transported over, a xAP message may be wrapped in an additional transport layer wrapper.

Basic Message Structure

Irrespective of transport, a xAP message is structured as a number of lines. Groups of lines make up sections.

  • A header section
  • A message comprising
    • One or more message blocks
      • Each with it's own message section

Anatomy.jpg

The message header is mandatory and the schema (structure) is defined by the xAP protocol. The schema of each message block is defined by an individual xAPplication. Recommended schemas for commonly encountered Home Automation devices (e.g. thermostats, caller id units) are maintained by the xAP group, and may be re-used as required.

Since xAP messages are often targeted at low power, limited capability devices, such as PIC's, the use of multiple message blocks is discouraged - although under certain circumstances it may be unavoidable.

Each section is named, and is enclosed between curly braces { }. Each line is terminated by a <LF> (linefeed, ASCII character 10 decimal).

Each message block contains one or more name-value pairs. An example of a name-value pair is "temp=25", which might convey the fact that the internal room temperature is 25'C.

A typical xAP message conveying this temperature information might look like this. The full structure is explained in more detail later in this document.

xap-header
{
v=12
hop=1
uid=FF123400
class=xap-temp.notification
source=ACME.thermostat.lounge
}
temp.current
{
temp=25
units=C
}

Message Schema

xAP promotes the use of common data definitions wherever possible. Commonly encountered elements - caller id information, X-10 control messages and so on - can be represented using standard schema. A device intercepts and acts on messages. A message comprises one or more message blocks. The semantics of a specific message block are determined by the schema.

A xAP class is a pre-defined collection of message blocks (or schema) to be expected in the message body. It effectively defines the structure of the entire message in terms of message blocks. Schema represent entities that are commonly encountered in the home control environment - such as temperature, phone numbers, device status and device control. These entities can be used in many different contexts. A class defines a context within which one or more schema are applied.

xAP identifies the schema that applies to a given message through the "class" keyword in the message header. A class may comprise one or more subtypes, each of which is identified by the "type" field. This is discussed in more detail below.

Message Grammar

Message Block Keyword Definition


Each message block is defined by a keyword line.

This keyword uses alpha-numeric printable characters a..z (a through z) 0-9 (zero through nine), _ (underscore), - (minus) and, additionally, embedded _ (space).

Leading and trailing white space (_ - character space, ASCII character 32 decimal) is not permitted. Message block names may include embedded spaces.

The maximum keyword length is 32 characters. This value is arbitrary, but chosen to suit the limited storage capabilities of low-end devices.

Keywords are not case sensitive. Thus "my thing", "My Thing" and "MY THING" are all treated as literally equivalent.

The message block name does not have to be unique within the message - that is, the same message sections may be repeated if required (albeit, usually with different content). Message Block Definition

The start of a message body is defined by { (open brace, ASCII character 123 decimal), and terminated by } (close brace, ASCII character 125 decimal).

Braces always appear on their own line.

Message blocks may be nested if required.

A message body is always explicitly named.

Message bodies contain successive lines defining name, value pairs.

The order of name, value pairs in the message block is not significant. Name-Value Pair Definition

Each name-value pair within the message block is defined by a keyword, a delimiter, a value and a terminator.

  • The keyword follows the same construct as that described above for a message section keyword definition.
  • The delimiter determines the method of value encoding.
    • A = (equals, ASCII character 61 decimal) indicates that a value is encoded as an ASCII string. This is the preferred encoding format.
    • A ! (pling, ASCII character 33 decimal) determines that a value is encoded as an ASCII hex representation. This allows the broadcast of raw binary data. Whilst unavoidable under certain circumstances, it's use is discouraged because of the issues with portability (numeric representations differ across processor types).
  • The value is encoded either as printable ASCII characters (where the delimiter is = ) or upper case ASCII hex (where the delimiter is ! ).
  • ASCII character strings (denoted by the delimiter = ) may represent any arbitrary value, including integer and floating point numbers. Embedded spaces are permitted.
  • ASCII hex strings (denoted by the delimiter !) are case sensitive and should use upper case. (ie. 0A is acceptable and represents decimal value 10, 0a is illegal and has no meaning). Embedded spaces are not permitted.
  • Each name-value pair is terminated with a <LF> (linefeed) control character (ASCII value 10 decimal).
  • The default behaviour is for leading and trailing white space to be treated as significant in the value field. All white space between the = and terminating <LF> character is considered significant. This allows for the inclusion of leading white space and trailing white space in string literal values.
  • The value assigned to a keyword is case sensitive and is interpreted literally. Thus fred=hello and fred=HELLO assign literally different values (hello, HELLO) to the element fred.
  • The use of hex encoding with the ! character is strongly discouraged since the meaning of messages becomes opaque, and device interoperability promoted by the use of xAP is compromised. Where hex encoding is used, it is mandatory, and a requirement for xAP compliance, to publicly document the packet format.
  • The order of name-value pairs within a message section has no significance. The same name may be used more than once in the same section if required, although by convention, multiple entities are usually differentiated through the use of an index element (e.g. my.thing.0 ; my.thing.1).

Maximum Message Size

xAP does not currently support fragmentation or re-concatenation of messages. The maximum message size is determined by the capabilities of the xAP device and the network infrastructure. For example, the maximum message size supported by xAP over Ethernet is typically 1,500 bytes (the conventional maximum packet size for a UDP datagram). Example Messages

The following illustrates the use of the message grammar to construct a conventional ASCII string value based message. For clarity, the linefeed <LF> terminators have been omitted and the message has been padded with white space which would not normally be present in the message when transmitted on the wire. The following message might be generated by a caller-id unit to notify a new incoming call.

xap-header
{
v=12
hop=1
uid=FF456700
class=cid.notification
source=acme.CID.home.line1
}
Call.Incoming
{
number=0207 2828 2929
time=12:08
date=13/08
}

The corresponding on the wire form of this message, removing the white space and including the <LF> characters would be:

xap-header<LF>
{<LF>
v=12<LF>
hop=1<LF>
uid=FF456700<LF>
class=cid.notification<LF>
source=acme.CID.home.line1<LF>
}<LF>
Call.Incoming<LF>
{<LF>
number=0207 2828 2929<LF>
time=12:08<LF>
date=13/08<LF>
}<LF>

The following illustrates the use of the message grammar to construct an ASCII hex string value based message. Again, for clarity, the linefeed <LF> terminators have been omitted and white space is present which would not normally be found in the message transmitted on the wire. The message sends a byte-encoded equivalent of "hello" (ASCII characters 68, 65, 6C, 6C, 6F respectively).

xap-header
{
v=12
hop=1
uid=FF345600
class=AClass.AClassType
source=AVendor.ADevice.AnInstance
}
ASchema.ASchemaType
{
mybinary!68656C6C6F
}

The corresponding on the wire form of this message, removing the white space and including the <LF> characters would be:

xap-header<LF>
{<LF>
v=12<LF>
hop=1<LF>
uid=FF345600<LF>
class=AClass.AClassType<LF>
source=AVendor.ADevice.AnInstance<LF>
}<LF>
ASchema.ASchemaType<LF>
{<LF>
mybinary!68656C6C6F<LF>
}<LF>

Message Addressing Schemes

Identifying a Device

xAP is a broadcast protocol, and as such all messages are received by all devices. The design of xAP enforces this in all circumstances. Each physical device on a xAP network, and any logical enpoints included within the device, is identified by a unique identifier (uid ).

Unique Identifier

The unique identifier element of the address is assigned at configuration time, prior to the device being connected to the xAP network for the first time. The UID, a hexadecimal number, identifies a particular hardware device for life, even if remaining elements of the address are reconfigured. Whilst most receivers will generally act on messages irrespective of the UID, choosing to filter instead on the devices logical name (defined by vendor, device and instance ), low power devices may prefer to filter messages based on UID, allowing for memory efficient configuration and address management.

The UID is generated as an upper case ASCII hex number, comprising nn dd dd ss where nn is the xAP network and domain identifier (FF by default), dd dd is the unique device identifier and ss is a subaddress node within the device. For all elements of the UID, the values FF and 00 are reserved for future use. Sub addressing is explained in more detail below.

xAP Addresses

In addition to the uid, every xAP device supports a human readable address construct. Messages always include the xAP source address of the originator and may optionally include a target address which indicates that a message is of specific interest to a particular device or devices. Even when targeted in this way, the message will still be received by all other connected xAP devices, although they will usually ignore it. A xAP source or target address comprises two parts: a logical name and an optional sub-address.

Subaddress.jpg

Logical Name

The logical name for a device is determined by the elements <vendor>, <device>, and one or more <instance> fields. The combined fields should uniquely identify a physical hardware device or software application on the xAP network. The <vendor> field is assigned to the device developer by the xAP group. The device field identifies the hardware or software application. Typically the instance reflects the physical location of the device or some other application-specific identifier. For example, one copy of an Outlook mail connector which notifies new mail to the xAP network might have a logical name: acme.outlook.mark for Mark's mail box, whilst another instance connected to Mary's mailbox (on the same xAP network) might have the logical name: acme.outlook.mary. Similarly, a lamp controller in the lounge might have the logical name: acme.lamp.lounge. If required, one or more instance fields may be concatenated. This allows structured naming of devices - for example, two physically separate lamp controllers might be identified as acme.lamp.lounge.table and acme.lamp.lounge.ceiling. This feature is particularly powerful when combined with wildcarding, since it permits a primitive form of group addressing.

As a consequence of the fact that the logical name uniquely identifies a device (or software application), there are constraints imposed on the allocation of logical names. If a device exists on the xAP network with a logical name a.b.c, a second device may not use the logical name a.b.c.d.

Sub-Addressing

Sub-addressing is typically used when a number of inputs or outputs are served by a single item of xAP hardware. For example, a lightswitch that provided four switches on a single faceplate would use sub-addressing to identify these individually on the xAP network. Sub-addressing is also relevant to more complex multi-function devices, such as the Homevision controller, which includes both analogue and digital ports. Whilst the logical name identifies the physical device which will receive a message, the sub-address allows a specific sub-section of the device to be uniquely addressed. For example, in the case of Homevision, sub-addresses: parallel.input , parallel.output and analogue.input and analogue.output might be appropriate.

In this way, all endpoints within a device are visible to the network and may be uniquely addressed. A sub-address is separated from the logical name by a : (colon) separator.

Limits on Address Size

The <instance> and <sub address> elements are variable depth. No limit is imposed on the maximum depth, although a combined maximum depth of 4 should be adequate for most applications.

The recommended maximum length for the entire xAP address is 128 characters. This provides a compromise between the use of meaningful naming schemes, storage requirements, and the time taken to transmit data over a slow serial connection.

Much consideration has been given to balancing the requirements of a flexible addressing scheme against the limited storage capability of small devices. Note that in many instances it is possible for devices to identify the originator of a message solely from the uid, minimising the requirements for string storage space.

Addressing Modes

On receipt of a message, all devices inspect the message header. On the basis of the name, value pairs contained in the header, the device decides whether a message warrants further processing, or should be discarded. Two alternative addressing schemes are supported to allow devices to determine whether or not they should act on a given message.

Source Based Addressing

A recipient device determines whether the message is of interest by inspecting the source of the message. This mode of addressing works well for one-to-many relationships. There are many possible receivers of data broadcast by a temperature sensor located in a particular room, for example, and they may all intercept the same message concurrently provided they know its source. Source based addressing is available to any receiver since it is mandatory for senders to include their source address, and the address is guaranteed to uniquely identify the sender. If required, low powered devices may choose to filter on incoming messages based on the uid rather than the full xAP source address. Whilst very limited storage is required to identify the source of incoming messages, the use of uid filtering obviously precludes the use of wildcards. Source addr.jpg

Target Based Addressing

A sending device may wish to target a message at a specific device. It does this by including the address of the target device in the header using the target parameter. This is particularly true of simple devices which may not be capable of filtering messages that originate from multiple sources. For example, a command may be sent to a simple curtain controller, to close a specific set of curtains. The curtain controller need only recognise messages aimed at itself, rather than retaining a list of all possible originators. Note that the target message is still broadcast, however, and may also be intercepted by other nodes on the network - such as, for example, a cache which tracks the current status of all devices. Whether a device, as a receiver, chooses to support target addressing is determined by the implementer and is dependent on the functionality offered by the device.

Target addr.jpg

Wildcarding of Addresses via Header

Target addresses may be wildcarded in certain circumstances. A matching wildcard address indicates that the entire remaining message (whether a single block or multi-block message) is intended for the recipient.

If a device supports target addressing, it must also support wildcarding of target addresses. Wildcards are specified on a per field basis, using the reserved characters * and >.

The * character indicates that any value may be substituted for the current field.

The > character indicates that any further input is matched.

Thus a device which is identified by an address of a.b.c.d would be expected to respond to messages targeted at a.*.c.d but would not consider a message identified by an address a.*.c as a match, because no wild card has been explicitly specified for the last element.

A device identified by an address of a.b.c.d would be expected to respond to messages targeted at a.b.c.>

Unless explicitly specified, the : (colon) sub-address separator is treated as being synonomous with a . (dot) separator. Exact matching against a : (colon) is only required in situations where wild card filtering against just the sub-address element is required.

Target addresses may be wildcarded by either the sender, the receiver, or both. Source addresses may only be wildcarded by a receiver. Wildcarding is simple to implement, but surprisingly powerful, and can lead to complex control scenarios.

Basic Wildcard Modes

Target based wildcarding From fixed source, wildcard target to fixed target filtering receiver Sender targets multiple, selected, recipients using just one message. Useful for control of a group of devices. E.g. all curtain controllers in a single room, with individual addresses of acme.K400.lounge.curtain.0, acme.K400.lounge.curtain.1 and so on, might be collectively targeted by acme.K400.lounge.curtain.* .

Source based wildcarding From fixed source, no target to wildcard source filtering receiver The receiver is able to intercept messages from multiple sources through the application of a single filter criteria. Useful for low power devices. For example, a receiver device may match messages from a source acme.digitstat.> in order to display the temperatures broadcast by each thermostat in the house, avoiding the need to explicitly list and configure the addresses of each individual thermostat at the receiver. New thermostats can also be added at any time without the need to modify the existing configuration.

Advanced Wildcard Modes

Advanced wildcard modes are included here for completeness. Any device which is capable of supporting the basic wildcarding methodology is inherently capable of supporting these modes, although the relationships between devices rapidly increases in complexity. For this reason, these modes are unlikely to find widespread application.

Target based wildcarding From fixed source, fixed target to wildcard target filtering receiver. Sender targets a recipient. Recipient filters target address on wildcard basis. Allows for a receiver to easily respond to multiple target addresses which are specified using a single criteria. E.g. a receiving device which supports control of multiple output ports might use a filter acme.iodevice.port.* to intercept messages targeted at acme.iodevice.port.1, acme.iodevice.port.2 etc. Such messages might originate from many different sources on the network.

From fixed source, wildcard target to wildcard target filtering receiver. Senders target multiple, selected, recipients using just one message. Multiple senders may be matched since the receiver also matches the target against a wildcard. This mode may be used to create specific sets of many-to-many relationships.

Restrictions on Wildcarding

Note that it is neither legal, nor meaningful, for a sending device to wildcard its own source. The source address should always provide a means of unambiguously identifying the originating device.

A receiving device must implement target based wildcarding if it supports target based addressing. This ensures that any sender can successfully control a receiving device using a wildcard target address.

A receiving device may implement source based filtering on a wild card basis at the developers discretion.

The following diagrams illustrate the use of wildcarded addressing:

Wildsource.jpg

Wildtarget.jpg

Wildcarding of Addresses via Body (Multi-Block messages only)

Any multi-block message with a wildcarded header address may optionally include additional target filters within each message body. This allows selected parts of a message to be targeted at individual receivers. The receiver first matches the header target tag. On finding a match, it may then choose to inspect the target tag in each message block, processing the message block if it is a good match. Inclusion of the target tag within a message block is optional. Message blocks which do not include a target tag are assumed to be aimed at all recipients.

For example, the following message illustrates how multiple message blocks in a single message can be used to achieve selective control over a number of instances of the same device:

xap-header
{
v=12
hop=1
uid=FF445500
class=a.big.message
source=acme.fooDevice.anInstance:aSubInstance
target=*.barDevice.*
} my.block.1
{
target=*.*.lounge
... message block aimed at lounge instance of barDevice ...
}
my.block.2
{
target=*.*.bedroom
... message block aimed at bedroom instance of barDevice ...
}
my.block.3
{
... message block aimed at all instances of barDevice ...
}

Restrictions on Body Wildcarding

Receiver support for Body Wildcarding is not mandatory. However, if a given message schema is supported by a receiver, and that message schema uses body wildcarding, then the receiver must correctly apply filtering to that message.

Message Header Policy

Every xAP compliant message must include a xAP-header message section. It is mandatory that this header is the first section of a xAP message. A receiver should discard any mal-formed message. The order of items in the header is fixed to make parsing easier for simple devices, however not all items in the header are mandatory.

Message Header Structure

The xAP header name-value pairs are specified as follows. The order in which the name,value pairs are included in the message header is significant, and must follow the order specified here.

The following header items are mandatory:

Version v=<version>. Header version. Reflects the xAP specification against which the message complies. Has value 12 for messages complying with this document. E.g. version= 12

Hop Count hop=<hop-count>. Always has value of "1" set by the originator. This value will be incremented each time a message traverses a bridge or gateway (but not when it traverses a hub). E.g. hop=1.

Unique Identifier uid=<uid> Uniquely identifies a particular hardware device on a xAP network. The UID is represented as ASCII hex number, with all characters in upper case, of the form nn dd dd ss. The first two digits, nn, of the UID are reserved for future use and must be FF by default. The digits dd dd are assigned at device configuration time to ensure that the hardware device is uniquely identified within the local xAP domain. The last two digits identify the device sub address, if supported, or are otherwise zero. E.g. FF123400. A UID may be no longer than 8 digits in total. The UID is usually configured at device installtime.

Message Class class=<class>.<type> Provides a cue which indicates how the remainder of the message is composed, in terms of which schema it uses. The xAP group maintain standard message classes (and associated schema) for common source types, and is happy to accept contributions from developers. Standard classes can be identified through the use of the reserved prefix xap-.... <class> Identifies the collection of schema applicable to this message. <type> Identifies the sub-schema applicable to this message.

E.g. class=xap-x10.command identifies that the message content that follows on from the header represents a xAP group standard x10 command, and should follow a prescribed (but implicit) format for x10 commands.

E.g. class=xap-x10.status identifies that the message that follows on from the header is a xAP group standard x10 status request, and should follow the prescribed (but implicit) format for x10 status requests.

Message Source source=<vendor>.<device>.<instance1>[<instance2..n>][:<sub1..n>] Provides a means of uniquely identifying the application or device originating this message. The <uid> element is assigned when the device is connected to a xAP network for the first time, and is guaranteed to be unique within that network. The remaining elements of the address identify a specific end-point within a specific device. As such they are also required to be unique for a given xAP network. <Vendor> Identifies the vendor of the device (or application) originating the xAP message. Vendor names are managed and allocated upon request by the xAP group to ensure that they are unique. This element is fixed, and determined by the programmer at design time. The vendor id may be no longer than 8 characters. <Device> Identifies the physical device or application originating the xAP message. Examples include: CM12U (an X-10 controller), Meteor (a caller id unit), Outlook (the microsoft e-mail application). This element is fixed, and determined by the programmer at design time. The device id may be no longer than 8 characters. <Instance1> [.<instance2..n>] Identifies this instance of the device or application. Typically reflects the geographic location of a device or system (e.g. "sitting-room"). First element is mandatory, additional elements may added if required. The instance field(s), in conjunction with the vendor and device field, uniquely identifies the originator of the message at the level of "that instance of the application [on a known host] on my network", and so provides a cue which indicates how the message content should be handled. The value of the instance field is usually determined by the end-user at the time when the device or application is installed for the first time. Each element is delimited by a period. <:sub1..n> (Optional) Used for subaddressing of multi-function device or application. The value of the field(s) may be determined either by the programmer at design time or by the end-user at run time when the device or application is installed for the first time, depending on the application. Each element is delimited by a period.

It is recommended overall length of a source address is limited to 128 characters inclusive of delimiters.

E.g. source=acme.cm12.node0 identifies a CM12U sited at node 0 which is connected to the xAP network using software written by the mythical vendor Acme.

E.g source=xapcorp.cid.home.line1 identifies a caller id unit manufactured by the mythical xAPCorp. The device is connected to the home phone system's line 1.


The following header items are optional:

Target target= <vendor>.<device>.<instance1>[.<instance2..n>][:<sub1..n>] Provides a means of targeting a specific application or device family interested in this message or event. Devices are not required to support target based addressing, but when they do, they must also include support for wildcarding of the target address (see later). Target addressing

<Vendor> Identifies the vendor of the device (or application) originating the xAP message. Vendor names are managed and allocated by the xAP group to ensure that they are unique. This element is fixed, and determined by the programmer at design time. A vendor id may be no longer than 8 characters. <Device> Identifies the physical device or application originating the xAP message. Examples include: CM12U (an X-10 controller), Meteor (a caller id unit), Outlook (the microsoft e-mail application). This element is fixed, and determined by the programmer at design time. A device name may be no longer than 8 characters. <Instance1> [<instance2..n>] Identifies this instance of the device or application. Typically reflects the geographic location of a device or system (e.g. "sitting-room"). First element is mandatory, additional elements may added if required. This, in conjunction with the vendor and device field, uniquely identifies the originator of the message at the level of "that instance of the application [on a known host] on my network",and so provides a cue which indicates how the message content should be handled. The value of the instance field is usually determined by the end-user at the time when the device or application is installed for the first time. Each element is delimited by a period. <:sub1..n> (Optional) Used for subaddressing of multi-function device or application. The value of the field(s) may be determined either by the programmer at design time or by the end-user at run time when the device or application is installed for the first time, depending on the application. Each element is delimited by a period.

It is recommened the overall length of a targetaddress is limited to 128 characters inclusive of delimiters.

A device should ignore all unknown name, value pairs included in the header.

E.g. target=acme.cm12.node0 identifies a message which is explicitly intended to be actioned by a CM12U sited at node 0 which is connected to the xAP network using software written by the mythical vendor Acme.

Message Body Policy

A message generally comprises one or more message blocks. Messages which do not include a body are legal - the heartbeat message is one such example - but they are limited to status reporting, and their general use is discouraged. Where possible, the use of single message-block messages is encouraged because they are readily accessible to low power devices.

Each message block is labelled with a section name. Multiple message blocks may share the same name if, and only if, they use the same schema. In array type situations, it is recommended that message blocks are labelled with an index (e.g. my.block.0, my.block.1) to differentiate them. Within a message, the message block name is used as a cue to indicate the schema used to encode that message block, and thus which name, value pairs a receiver might expect to find within the message block and their semantic significance.

Device Monitoring - Heartbeats

In order to allow health monitoring of all xAP enabled components within the home, it is strongly recommended that a xAP enabled device generates a periodic heartbeat. It is, however, recognised that some low-end (embedded) devices may not be able to support this functionality.

The heartbeat message has the following minimum structure:

xap-hbeat
{
v=12
hop=1
uid=FFAABB00
class=xap-hbeat.alive
source=acme.meteor.home.line1
interval=60
}

where:

Version v=<version>. Header version. Reflects the xAP specification against which the message complies. Has value 12 for messages complying with this document. E.g. v= 12

Hop Count hop=<hop-count>. Always has value of "1" set by the originator. This value will be incremented each time a message traverses a bridge or gateway (but not when it traverses a hub). E.g. hop=1.

Unique Identifier uid=<uid> Uniquely identifies a particular hardware device on a xAP network. The UID is represented as ASCII hex number, with all characters in upper case, of the form nn dd dd ss. The first two digits, nn, of the UID are reserved for future use and must be FF by default. The digits dd dd are assigned at device configuration time to ensure that the hardware device is uniquely identified within the local xAP domain. The last two digits identify the device sub address, if supported, or are otherwise zero. E.g. FF123400. A UID may be no longer than 8 digits in total. The UID is usually configured at device installtime.

Message Class class=<class>.<type> Indicates that the message is a xAP group standard heartbeat, indicating that the device is alive.

Message Source source=<vendor>.<device>.<instance> Provides a means of uniquely identifying the application or device family that generated this heartbeat.

<Vendor> Identifies the vendor of the device (or application) originating the xAP message. Vendor names are managed and allocated by the xAP group to ensure that they are unique. This element is fixed, and determined by the programmer at design time. <Device> Identifies the physical device or application originating the xAP message. Examples include: CM12U (an X-10 controller), Meteor (a caller id unit), Outlook (the microsoft e-mail application). This element is fixed, and determined by the programmer at design time. <Instance> [<instance2..n>] Identifies this instance of the device or application. Typically reflects the geographic location of a device or system (e.g. "sitting-room"). First element is mandatory, additional elements may added if required. This, in conjunction with the vendor and device field, uniquely identifies the originator of the message at the level of "that instance of the application [on a known host] on my network",and so provides a cue which indicates how the message content should be handled. The value of the instance field is usually determined by the end-user at the time when the device or application is installed for the first time.

E.g. source=acme.cm12.node0 identifies a CM12U sited at node 0 which is connected to the xAP network using software written by the mythical vendor Acme.

E.g source=xapcorp.cid.home.line1 identifies a caller id unit manufactured by the mythical xAPCorp. The device is connected to the home phone system's line 1.

Heartbeat Interval interval=<seconds> Specifies the period between heartbeats in seconds. A minimum heartbeat interval of 5s is recommended, in most applications once a minute is sufficient.

E.g. interval = 5 specifies a 5s heartbeat interval.

Additionally, an optional field indicating the port and process id may be added:

Port port=<port number> Optional. Defines the UDP port on which the application listens for xAP messages. Applicable to ethernet based xAP devices only. E.g. port=3640 indicates that a device is listening for inbound xAP messages on port 3640.

Process ID pid =<ip_address>:<process number> Optional. Defines the process id for the sending application. Receivers (especially hubs, but also any other end-point), may use this to identify that an application has re-started. xAP aware management applications may use this to facilitate inter-process messaging at an operating system level. Note that inter-process messaging should only be used to control the visual appearance of an application, and should not be used to subsume communications which would otherwise be performed by the xAP protocol itself.

The order of the fields in the heartbeat message is fixed, and must follow the sequence prescribed above.

Additional network specific message sections may be added to the heartbeat to convey xAP routing configuration information (see later). The remaining fields are defined as per xap-header.

Since xAP heartbeat messages are broadcast, it is potentially possible to monitor the health of the entire xAP infrastructure from any point on the network. Automatic monitoring software is available to detect, for example, the failure of a specific process and raise an appropriate alert automatically.

Since every device on the network receives heartbeat messages, it is strongly recommended that devices do not transmit heartbeats more frequently than absolutely necessary. An application will typically send a heartbeat immediately on start-up, to ensure immediate recognition by any local hub, and then subsequently at intervals of a minute or more.

Standard Message Schema

The structure of a message is defined by a class. The name of a class provides an implicit indication to the receiver as to which the message blocks within the message may be anticipated in the message body. Each message block is itself defined by a schema. The schema simply defines the name, value pairs contained in the message block, and the semantic significance. Schema are often re-used from class-to-class. This means that developers do not have to waste time re-inventing a message which represents a phone number, or a temperature, for example, but can adopt a common representation for that entity. In xAP terminology, a class is therefore simply a collection of schema.

Recognising that interoperability is key to the widespread adoption of xAP, the xAP group maintain a list of standard device schema. It is strongly recommended that wherever possible developers use the standard device schema in their own work. The group welcomes the submission of new schema and requests for extensions to existing schema.

Standard device schema can be identified from the use of the prefix "xap-" in the xap-header "class" and message "type" fields.

Schema.jpg

Bridging

xAP is, so far is as practicable, network agnostic. Where two networks are to be joined, so that xAP messages can be passed between then, a software bridge is used. The bridge understands the implementation details of the xAP protocol on each physical network, and performs the necessary protocol conversions to pass the message from one network to another.

In some cases, the bridge may provide a configuration interface to allow traffic in one or both directions to be restricted.

A proposal has been made to permit messages to be flagged "non-routable" in the xAP-Header, and for bridges to never pass these messages. Currently this item is still open.

Messages passed over the bridge always have the xAP-header hop count increased. A bridge may be configured to discard messages with hop-count > n to prevent looping/broadcast storms.

Gatewaying

A gateway provides access to a xAP network from a remote point over the same physical network medium. For example, an Ethernet gateway might allow access to the home xAP network over the internet. Typically a gateway acts a bridge, but uses point-to-point or connection oriented protocols to connect the two bridges. Gateways may elect to filter messages, depending on the capabilities of the bridged network, in order to restrict either traffic volumes or message size.

Transport Wrapper

Depending on the chosen network infrastructure, it may be necessary to encapsulate the basic xAP message in an additional transport wrapper. Examples of networks which might require this are those based on RS232 serial datalinks, where an indication of the start and end of a message, and a checksum, are useful because corruption over such a link is not uncommon, or those with specialized addressing requirements, such as SNAP.

Basic Ethernet Wrapper

The use of xAP over Ethernet does not require any additional wrapper. Messages are usually sent as UDP broadcast datagrams, although in special cases they may also be sent using connection oriented protocols if required.

Basic Serial Transport Wrapper

The xAP transport wrapper for a basic serial connection is defined as follows:

  • The xAP message is prefixed with the ASCII control character <STX> (ASCII character decimal 2)
  • The core xAP message is transmitted. Any instances of the <STX> character and <ETX> character (ASCII characters decimal 2 and decimal 3 respectively) are escaped by prefixing the character with <ESC> (ASCII character 27). i.e. an embedded <STX> becomes <ESC> <STX> and an embedded <ETX> becomes <ESC> <ETX> . This mechanism is defined here for completeness; in practice it would be very unusual to transmit non-printable characters as part of a xAP message.
  • Instances of the <ESC> character (ASCII character 27) are also escaped. i.e. <ESC> becomes <ESC> <ESC>. Again, it would be rare to embed <ESC> characters within a xAP message in practice.
  • At the end of the xAP message a 16-bit CRC checksum is appended as four ASCII-hex digits (ie. human readable, not binary). The checksum is applied to all data within the message envelope: the <STX>, checksum itself, and <ETX> character are not included in the CRC calculation. Hex digits A-F are represented in upper case. Source code examples illustrating the calculation of 16-bit CRC checksums can be found at www.planetsourcecode.com.
  • If the checksum is not calculated, four dashes (ASCII code decimal 45) are substituted in its place.
  • The checksum is immediately followed by <ETX> (ASCII character decimal 3)

xAP for Ethernet - Practical Implementation Details

xAP over Ethernet uses UDP Broadcast. UDP broadcasts are only visible within the subnet within which they originate, and are limited to 1,500 bytes in size. A broadcast may be received by any device within the subnet.

xAP has been allocated a dedicated UDP port, 3639, on which to communicate. This port is used in the examples that follow.

Sending a xAP message from, say, Visual Basic, is very straightforward. The following code fragment (courtesy Mark Harrison) illustrates how to construct a message with the schema. The <uid> marker would be replaced with a true unique identifier for this instance of this application in real life. This would have the form FFABCD00 or similar.

xAP-header
{
v=12
hop=1
uid=<uid>
class=mail.notification
source=Acme.Outlook.<username>
}
inbox.unread
{
unread=<number of unread waiting>
}

xAPMessageBody = xAPMessageBody & "xAP-header" & Chr$(10) ' mandatory header

xAPMessageBody = xAPMessageBody & "{" & Chr$(10) ' start of header section

xAPMessageBody = xAPMessageBody & "v = 12"& Chr$(10) ' mandatory version number

xAPMessageBody = xAPMessageBody & "hop = 1"& Chr$(10) ' mandatory hop count ' must always be set to 1 by an input plugin (changed by bridges)

xAPMessageBody = xAPMessageBody & "uid = <uid>"& Chr$(10) ' mandatory UID, unique to this application in this xAP domain

xAPMessageBody = xAPMessageBody & "class=mail.notification" & Chr$(10) ' mandatory class = class.type ' identifies message schema

xAPMessageBody = xAPMessageBody & "source = <uid>.Acme.Outlook" &oUserName & Chr$(10) ' mandatory source = vendor. application. instance ' must reflect the name of the application providing the information ' and is guaranteed to be unique within this xAP network ' NB note use of instance to identify user, allows several copies ' of Outlook "his n hers!" to run the plugin

xAPMessageBody = xAPMessageBody & "}" & Chr$(10) ' mandatory end of header section

xAPMessageBody = xAPMessageBody & "inbox.unread" & Chr$(10) ' message body defines <schema>.<schemaType> by convention

xAPMessageBody = xAPMessageBody & "{" & Chr$(10) ' Start of message body section

xAPMessageBody = xAPMessageBody & "unread=" & Trim$(Str$(UnreadMessagesInInbox)) & Chr$(10) ' unread is a plugin-specific keyname, and must be letters and numbers only ' = is a reserved character to separate the key from the value < BR > ' the value follows (in this case, a variable cast to a string type

xAPMessageBody = xAPMessageBody & "}" & Chr$(10) ' mandatory close message body for the source

This message can then be broadcast by calling the SendData method of the TCP/IP control. The TCP/IP control should be configured as follows:

LocalPort =3639 Host = <Broadcast IP> Port = 3639 Protocol = 1 (sckUDPProtocol)

(Those unfamiliar with TCP/IP networking may find this link http://www.telusplanet.net/public/sparkman/netcalc.htm useful in assisting with the calculation of the broadcast address.)

The previous illustration shows how to send a xAP message in it's simplest form. In most cases, though, it will be a requirement to deploy more than one xAPp on a single host. Basic limitations in the way tcp/ip sockets are implemented mean that whilst it is possible to have several senders on the same host, there can only be one receiver active at a time. For this reason, xAP uses a software hub in situations where there is more than one listener on a single host. Embedded systems rarely, if ever, need a hub.

xAP for Ethernet - Hub Protocol

On startup, a xAPp that is a hub client will look for the first free UDP port available on the loopback interface (127.0.0.1), starting from 49152. Once a free port has been identified, the xAPp sends a heartbeat message to the broadcast address on the xAP base port number, 3639, which includes the loopback port number.

The message would have the following schema for device configured with the logical name acme.cid.home.line1 where the first available port was 49152. (ie. there are no other devices using this hub).

xap-hbeat
{
v=12
hop=1
uid=<uid>
class=xap-hbeat.alive
source=acme.cid.home.line1
interval=60
port=49152
}

Subsequently, the xAPp will send all outgoing messages to the host-ip broadcast address on the xAP base port number. The xAPp will receive incoming messages via the hub, on the chosen loopback port number.

The software hub, which is located on the same host as the client xAPp, behaves as follows:

On startup, it opens a listener on the primary network interface, xAP base port number 3639, and opens an outbound socket on the loopback interface.

Any incoming heartbeat on the primary interface is decoded, and the originating ip address is validated. If the heartbeat originates from the same host as the hub, the port value is added to a list. The heartbeat is forwarded to all applications connected to the hub. Any incoming message received on the primary network interface which is not a locally generated heartbeat is forwarded to all hub-ports on the list via the loopback interface. If no heartbeat is received from a given xAPp for two heartbeat intervals, then the hub-port is removed from the list.

In choosing the heartbeat interval for a xAPp, bear in mind that the recovery time for the xAPp, should the hub restart, will be, on average, the same as the heartbeat interval. Recovery following restart of a hub is automatic, and a xAPp need take no specific corrective action. If required a xAPp may detect loss of connection with the local hub by listening for echoes of it's own heartbeat (on the loopback port). Failure to receive the heartbeat indicates a defective or non-existent hub.

Reference software hubs are available for Win/32, PocketPC and Linux. See references at the beginning of this document.

xAP for Ethernet - Example Application

xAP Home Display LCD's are displays which can be located at any convenient point in the home, and are able to display information from a variety of live sources. The display has a small on-board controller and is connected to an Ethernet network. The device may be configured through a web browser, for example, such that each line of the display will display a prescribed name-value pair from any arbitrary xAP message.

In the simplest implementation, the display might be used as a straightforward read-out for a thermostat. In this instance, the thermostat might broadcast a message:

xap-header
{
v=12
hop=1
uid=<uid>
class=Thermostat.status
source=Acme.DigiStat.SittingRoom
}
Current.Status
{
temp=22
setpoint=25
output=on
}

The display might be configured to display:

"Current Temp "+ Acme.DigiStat.SittingRoom:Thermostat.Status:Current.Status:temp + "'C" on line 1, "Target Temp "+ Acme.DigiStat.SittingRoom:Thermostat.Status:Current.Status:Setpoint + "'C" on line 2, and "Heating system "+ Acme.DigiStat.SittingRoom:Thermostat.Status:Current.Status:Output on line 3.

With the end result - updating in real time:

Current Temp 22'C Target Temp 25'C Heating System On

Whilst useful, this is rather unimaginative. Things get more interesting when a bit of intelligence is added to the chain. The display could be more intelligent. Perhaps it would be useful to use the bottom line to display either the number of unread e-mail messages or, if the phone rings, the name of the caller.

In this case, the display configuration is extended to include not only:

Current Temp "+ Acme.DigiStat:SittingRoom:Thermostat.Status:Current.Status:temp + "'C" on line 1, "Target Temp "+ Acme.DigiStat:SittingRoom:Thermostat.Status:Current.Status:Setpoint + "'C" on line 2, and "Heating system "+ Acme.DigiStat:SittingRoom:Thermostat.Status:Current.Status:Output on line 3.

but also: FooCorp.Outlook.<username>:mail.notification:inbox.unread:Unread + " unread mails" --- or --- BarCorp.AddressBook.Personal:DbLookup:Incoming.call:Name + "calling" on line 4

The first item, Outlook.<username>.Unread might be generated by an Outlook plug-in running on the end-users laptop. The Outlook plug-in generates a message with the schema:

xAP-header
{
v=12
hop=1
uid=<uid>
class=mail.notification
source=FooCorp.Outlook.<username>
}
inbox.unread
{
unread=<number of unread waiting>
}

The second item is more interesting. It might be generated by xAPp enabled database service. The server-based application listens to caller ID messages with the logical name Acme.CID.Home.Line1 and schema:

xap-header
{
v=12
hop=1
uid=<uid>
class=CID.notification
source=Acme.CID.Home.Line1
}
Incoming.call
{
number=0207 2828 2929
time=12:08
date=13/08
}

and performs the name lookup against the number. The application then re-broadcasts this information in a new message:

xap-header
{
v=12
hop=1
uid=<uid>
class=DBLookup
source=BarCorp.AddressBook.Personal
}
Incoming.call
{
name=Fred Bloggs
number=0207 2828 2929
time=12:08
date=13/08
}

The result is a display which now shows:

Current Temp 22'C Target Temp 25'C Heating System On 28 unread mails

And changes instantly to:

Current Temp 22'C Target Temp 25'C Heating System On Fred Bloggs calling

When there is an incoming phone call from Fred Bloggs.

The above illustration supposes that the display has sufficient on-board intelligence to allow the definition of relatively complex xAP message filters. This won't always be the case - the simplest display device will only be able to extract values from a single field and display that as text, or perhaps even just light a solitary LED. In these cases, the formatting of display information has been performed by an intermediate process, perhaps a server based xAPplication.

Another useful twist with xAP is that there can be many recipients of the same message. For example, a text to speech engine could also intercept BarCorp.AddressBook.personal:Incoming.call:Name messages and announce the name of the caller over a loudspeaker simultaneously.

Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox