Testing Tool Prototype



Testing Tool Prototype - Features

ttproto features is largely inspired from the TTCN-3 standard (abstract model, templates, snapshots, behaviour trees, communication ports, logging) and from Scapy (syntax, flexibility, customisability).

It is not a subset or a superset of the TTCN-3 language. Actually the language itself is not supported. We preferred to focus on the concepts and features provided by this language. Then we started to experiment some new features, and this is precisely the purpose of ttproto: this tool is implemented in a high level language, which makes it very suitable for prototyping new features.

If you are not familiar with TTCN-3, you can have a look at tutorials

Below is a non-exhaustive list of features we have been working on.

Input and presentation formats

Any user-defined type may accept multiple input formats through customisation.

For example, the type IPv4Address (which is inherited from the built-in type 'bytes' from python) can accept two input formats for initialising values: either a raw byte string or a string representing the address in the human-readable dotted notation. The two following instantiations give the same result:

>>> IPv4Address("")
>>> IPv4Address(b'\xc0\xa8\x00\x02')

By default IPv4Address will be presented in the human-readable format, however it is internally stored as a binary string. Actually, this type inherits from the built-in type bytes and can be converted at any time.

>>> a = IPv4Address("")
>>> isinstance (a, bytes)
>>> bytes (a)

Object-Oriented model

Messages types are represented using an object-oriented model (which is based on python3 object model).

User-defined type (like TTCN-3's subtypes and structured types) are just inherited from a base type provided by ttproto.

However, implemented the inherited class may be error-prone. Thus we secure the process by using metaclasses. For example, an IPv6 packet can be implemented as:

class IPv6 (
  	metaclass = PacketClass, 
  	fields    = [
		# Field Name		Alias		Type

  		("Version",		"ver",		UInt4),
  		("TrafficClass",	"tc",		UInt8),
  		("FlowLabel",		"fl",		UInt20),
  		("PayloadLength",	"len",		UInt16),
  		("NextHeader",		"nh",		UInt8),
  		("HopLimit",		"hl", 		UInt8),
  		("SourceAddress",	"src", 		IPv6Address),
  		("DestinationAddress",	"dst", 		IPv6Address),
  		("Payload",		"pl", 		Value)

Note: the second parameter of each field is an alias for this field. This eases the manipulation of data, while keeping a very verbose field name.

The resulting class inherits from PacketValue which itself inherits for the Value (the base of all message types).

>>> issubclass (IPv6, PacketValue)
>>> issubclass (IPv6, Value)

Note: the type of the Payload field (the last one) is Value, which means that we can encapsulate any kind of message in our IPv6 datagram, it is no longer necessary to list any possible combination. This is an advantage of the object-oriented model.

Default values and template matching

Those who are experienced with TTCN-3 are also experimented with its templates. Templates are a power feature of this language. They allow filling messages with wildcards instead of an actual value (and there is a wide range of matching mechanisms).

For example, to match an IPv6 message whose hop limit is 255, on could write in TTCN-3:

template IPv6 IPv6HopLimit_is_255 :=
  	Version			:= ?,
	TrafficClass		:= ?,
	FlowLabel		:= ?,
	PayloadLength		:= ?,
	NextHeader		:= ?,
	HopLimit		:= 255,
	SourceAddress		:= ?,
	DestinationAddress	:= ?,
	Payload			:= ?,

The interrogation marks '?' indicate that we can match any value. This is useful because in many cases (perhaps most), you will only care only about a part of the message.

Wildcard can not be used in templates for generating messages to be sent. In that case we must define a second template with all fields set to a value, ex:

template IPv6 send_IPv6HopLimit_is_255 :=
  	Version			:= 6,
	TrafficClass		:= 0,
	FlowLabel		:= 0,
	PayloadLength		:= 0,
	NextHeader		:= 0,
	HopLimit		:= 255,
	SourceAddress		:= '00000000000000000000000000000000'O,
	DestinationAddress	:= '00000000000000000000000000000000'O,
	Payload			:= ''O,

There is some burden associated to this model, at least for three reasons:

  • in practice you need to define two sets of templates for your test suites. One is used for sending (with all values set) and one for receiving (possibly with matching mechanisms)
  • the overall number of templates tends to increase rapidly. This example was about HopLimit, let's say you will have another one with the NextHeader field, then with all possible combination of fields… (Note: this can be mitigated in the latest versions of TTCN-3 by using default template parameters and the assignment notation, but you will still need to define some templates)
  • template definitions are very verbose (because we must include every field, even those we don't care about). This is an issue because it decreases the overall readability of the code (we don't know which data are specific to this template and which are not) and because it increase the size of the code (thus lot of copy & paste, and many more opportunity of introducing bugs).

In order to avoid these issues, we introduced the ability of defining default values when defining a structured message type. For the Packet classes, we can add a fourth column in the field definitions to set a default value.

class IPv6 (
  	metaclass = PacketClass, 
  	fields    = [
		# Field Name		Alias		Type		Default value

  		("Version",		"ver",		UInt4,		6),
  		("TrafficClass",	"tc",		UInt8,		0),
  		("FlowLabel",		"fl",		UInt20,		0),
  		("PayloadLength",	"len",		UInt16, 	0),
  		("NextHeader",		"nh",		UInt8,		0),
  		("HopLimit",		"hl", 		UInt8,		64),
  		("SourceAddress",	"src", 		IPv6Address,	"::"),
  		("DestinationAddress",	"dst", 		IPv6Address,	"::"),
  		("Payload",		"pl", 		Value,		b"")

When instantiating a template, we may leave some values undefined. The meaning will depend on how the template is used:

  • when matching an template → an undefined value means any value (like the '?' in TTCN-3)
  • when generating a message → an undefined value will be replaced with the default value defined for this field (or trigger a runtime error if this field does not have a default value)

In addition to this feature, template instantiation is performed with a very cheap syntax (similar to a function call).

Thus the two previous templates we wrote in TTCN-3 can be instantiated in ttproto as a single template:

IPv6 (hl = 255)

If we force the generation of the message, then the other fields will be filled with their default value:

>>> Message (IPv6 (hl=255)).display()
###[ IPv6 ]###
  Version=                  6
  TrafficClass=             0x00
  FlowLabel=                0x00000
  PayloadLength=            0
  NextHeader=               0
  HopLimit=                 255
  SourceAddress=            ::
  DestinationAddress=       ::
###[ BytesValue ]###
    Value=                  b''
Encoded as:
    60 00 00 00 00 00 ff 40  00 00 00 00 00 00 00 00
    00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
    00 00 00 00 00 00 00 00  

Convenient, isn't it ?

In practice most of templates can be instantiated directly as inline templates in the body of the testcase function. It is no longer necessary to pre-define dozens of templates, but only few for the complex and very specific cases.

This is very useful, because:

  • it decreases drastically the size of the code
  • it decreases its complexity
  • it increases the readability of the test case → we have a direct view of which data are relevant to the actual test case

Encoding/Decoding framework

Encoding and decoding messages (i.e. converting between their abstract representation in the test case and the and the actual binary message sent to the system under test) is not addressed in the TTCN-3 language. The standard defines instead an interface to implement this feature in an external module (there are bindings in Java, C, C++ and C#). On the one hand the developer is free to implement his codecs and in the other hand the standards provides no framework at all.

In the T3DevKit toolkit, we implemented an automatic CoDec generator to facilitate this tasks, but with some difficulties since it is like adding another layer above TTCN-3, we had to reimplement many features of the language (parser, message types & variables) and the resulting toolkit is mainly specialised to messages with Left-to-Right parsing.

In ttproto we decided to integrate encoding and decoding mechanisms within the tool, but with some latitude to allow implementing families of codecs. In fact encoding rules are often very similar within a family of protocols. For example in the Internet area, message-based protocols are mostly encoded from left-to-right, using big endian integers, 32-bit padding and ubiquitous patterns like TLV (Type-Length-Value).

Thus we defined a InetPacketClass metaclass, which is derived from InetPacket and implements the specificities of internet protocols.

class IPv6 (
  	metaclass = InetPacketClass, 
  	fields    = [
		# Field Name		Alias		Type		Default value

  		("Version",		"ver",		UInt4,		6),
  		("TrafficClass",	"tc",		UInt8,		0),
  		("FlowLabel",		"fl",		UInt20,		0),
  		("PayloadLength",	"len",		UInt16, 	InetLength ("Payload")),
  		("NextHeader",		"nh",		UInt8,		InetType (nh_bidict, "Payload")),
  		("HopLimit",		"hl", 		UInt8,		64),
  		("SourceAddress",	"src", 		IPv6Address,	"::"),
  		("DestinationAddress",	"dst", 		IPv6Address,	"::"),
  		("Payload",		"pl", 		Value,		b"")

Encoding/decoding is tightly coupled with default value computation (seen in the previous section). For example, the payload length field cannot be computed without knowing how the payload is encoded.

In our IPv6 example, InetLength and InetType are tags that are used to configure the automatic computation when generating the message and as decoding rules when decoding a message. These two tags address the TLV pattern commonly found in internet protocols. InetLength indicates that the PayloadLength field contains the length of the Payload field once encoded. InetType indicates that the NextHeader field indicates the type of the payload. Its parameter nh_bidict is a bidirectional hash table that maps a value to a type of message (according to the official number assignments).

nh_bidict = Bidict ({
  	41:	IPv6,
	58:	ICMPv6,
	4:	IPv4,
	17:	UDP,

Field descriptions

Ttproto allows attaching descriptions for values in a field. This has no use in the test itself, but it helps when analysing test logs (for example when dealing with unsupported messages types, or for protocols).

This is especially useful for protocols that assign a meaning to each value, like the Code field in the CoAP protocol.

 1   -> GET
 2   -> POST
 3   -> PUT
 4   -> DELETE
 :      :
 69  -> 2.05 Content
 :      :
 131 -> 4.03 Forbidden
 132 -> 4.04 Not Found

The following example shows how to attach a description table to the IPv6 NextHeader field (which indicates the type of the payload).

ip_next_header_descriptions = {
	0:	"IPv6 Hop-by-Hop Option",
	1:	"Internet Control Message",
	2:	"Internet Group Management",
	3:	"Gateway-to-Gateway",
	4:	"IPv4 encapsulation",
	5:	"Stream",
	6:	"Transmission Control",
	:          :
	139:	"Host Identity Protocol",
	140:	"Shim6 Protocol",
	141:	"Wrapped Encapsulating Security Payload",
	142:	"Robust Header Compression",

class IPv6 (
  	metaclass = InetPacketClass, 
	fields    = [...],
	descriptions = { "NextHeader": ip_next_header_descriptions },

Packet Variants

It is common to have several different PDUs in the same protocol in order to represent different kind of messages.

For example ICMPv6 defines many types of messages:

  • Echo Request
  • Echo Reply
  • Destination Unreachable
  • Packet Too Big
  • Neighbour Solicitation
  • Neighbour Advertisement

All these messages share the same header (Type/Code/Checksum), but have a different body. The figures below show the format of the Echo Request and Neighbour Solicitation messages.

Echo Request Message Format (RFC 4443)

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    |   Type (128)  |     Code      |          Checksum             |
    |           Identifier          |        Sequence Number        |
    |     Data ...

Neighbour Solicitation Message Format (RFC 4861)

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   |   Type (135)  |     Code      |          Checksum             |
   |                           Reserved                            |
   |                                                               |
   +                                                               +
   |                                                               |
   +                       Target Address                          +
   |                                                               |
   +                                                               +
   |                                                               |
   |   Options ...

In standard TTCN-3 we can represent them by using a combination of record and union structured types. Two options are possible:

  1. a union of records
  2. a union in a record
Option 1  (union of records)				Option 2  (union in a record)
type union ICMPv6					type record ICMPv6
{							{
	EchoRequest		ereq,				UInt8		Type,
	NeighborSolicitation	nsol,				UInt8   	Code,
	...							UInt8   	Checksum,
}								ICMPv6Body	Body
type record EchoRequest					
{							type union ICMPv6Body
	UInt8		Type,				{
	UInt8   	Code,					EchoRequest		ereq,
	UInt8   	Checksum,				NeighborSolicitation	nsol,
	UInt16		Identifier,				...
	UInt16		SequenceNumber,			}
	octetstring	Data				
}							type record EchoRequest
type record NeighborSolicitation				UInt16		Identifier,
{								UInt16		SequenceNumber,
	UInt8		Type,					octetstring	Data
	UInt8   	Code,				}
	UInt8   	Checksum,			
	UInt32		Reserved,			type record NeighborSolicitation
	IPv6Address	TargetAddress,			{
	NDOptions	Options					UInt32		Reserved,
}								IPv6Address	TargetAddress,
								NDOptions	Options

They both have advantages and drawbacks, option 1 is easier to handle (the definition is simple) but option 2 integrates more tightly the structure of the messages (especially it allows manipulating the header fields independently of the message type)

Ttproto provides smarter way to deal with this case: packet variants (this is inspired from scapy). The base ICMPv6 message is defined as follows. It is a generic definition that provides the common header (type, code, checksum) and stores the body in an additional field that accepts any value (Value is the root class of all message types).

class ICMPv6 (
	metaclass = InetPacketClass, 
	fields    = [
		("Type", 		"type",		UInt8,		InetVariant()),
		("Code",		"code",		UInt8,		0),
		("Checksum",		"chk",		Hex (UInt16), 	InetIPv6Checksum()),
		("Payload", 		"pl", 		Value)

You can notice that the Type field uses the InetVariant tag, we will get back to it below. In the mean time you can have this clue: it is very similar to InetType.

This ICMPv6 message type definition is convenient because it can handle any ICMPv6 message.

However the body will by default be decoded as bytes. To overcome this, ttproto allows defining new variants by truncating the end of the fields definition and replacing it with other fields.

Thus, Neighbour Solicitation is defined as:

class ICMPv6NeighborSolicitation (
	metaclass  = InetPacketClass,
	variant_of = ICMPv6,
	prune      = -1,
	id         = 135,
	fields     = [
		("Reserved",		"rsv",	Hex (UInt32),	0),
		("TargetAddress",	"tgt",	IPv6Address,	"::"),
		("Options",		"opt",	ICMPv6OptionList, []),

This InetPacketClass definition introduces three new parameters:

  • variant_of which tells that this packet is a variant of another packet definition defined earlier
  • prune which tells how many field to keep in the original definition. It may be negative, in that case the index starts from the last field (just like a negative index in python lists). If prune is not present, then no fields are truncated. In the present example, Prune=-1 tells to remove the last field (Payload) and replace it with a sequence of three fields (Reserved, TargetAddress and Options).
  • id which associates this message variant to one value. Here, 135 is the IANA-assigned value for the Neighbour Solicitation message type. It is used by the InetVariant tag to fill the Type field when building a message and to choose the correct message variant when decoding a message (thus if we read Type=135 when decoding an ICMPv6 message, then the decoder will switch to the ICMPv6NeighborSolicitation type and continue the decoding)

Following the same method, we can define the Echo Request message. Since it is very similar to the Echo Reply message, we will define a common intermediate type: ICMPv6Echo (the only difference between the two messages is the IANA-assigned type: 128 for Echo Request and 129 for Echo Reply).

class ICMPv6Echo (
	metaclass  = InetPacketClass,
	variant_of = ICMPv6,
	prune      = -1,
	fields     = [
		("Identifier",		"id",	Hex (UInt16),	0),
		("SequenceNumber",	"seq",	UInt16, 	0),
		("Payload", 		"pl",	Value),

class ICMPv6EchoRequest (
	metaclass  = InetPacketClass,
	variant_of = ICMPv6Echo,
	id         = 128):

class ICMPv6EchoReply (
	metaclass  = InetPacketClass,
	variant_of = ICMPv6Echo,
	id         = 129):

Message variants are fully integrated within the object oriented model of python. Thus the class representing a variant message derives from the class that implements the original packet type. Also variants can be defined in cascade.

>>> issubclass (ICMPv6EchoRequest, ICMPv6)


  • Events management
  • Behavioural blocks
  • Subtypes
  • Template inheritance & combination
testing_tool_prototype/features.txt · Last modified: 13/03/2016 11:16 by viho
© 2008 INRIA Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki