While I await components for a toy robot, a somewhat toy 52 Watt quadcopter and a very serious 3D speaker array system, I'll explain some previous research which is not widely known. I'll start with an introduction, move to implementation details, limitations and a possible solution.
Cells
Local Area Networks typically use variable length packets. Our dependence upon wired Ethernet and wireless Ethernet is so widespread that people have difficulty imagining any other techniques. However, while Ethernet dominates short spans of network, long-distance connections invariably use fixed length cells. This includes the majority of digital satellite communication, cell-phones and broadband systems. The division between LAN and WAN [Local Area Networking and Wide Area Networking] remains very real due to pragmatic reasons.
Variable length packets maximize bandwidth but fixed length cells maximize reliability. For long-distance communication, reliability is more important than bandwidth. Indeed, without reliability over a long span, bandwidth is zero.
Framing
Finding the start of a packet is difficult and cumbersome. Typically, there is a start sequence. This applies even in the trivial case of RS-232 serial communication in which start bits, stop bits, payload bits and parity bits are all configurable. For Ethernet, the start sequence is a known pattern of eight bytes. For Zigbee, it is a shorter pattern of four bytes. This inefficient but acceptably so within the span of a broadcast network. However, pre-amble is highly insecure because the patterns used in the start sequence are also valid patterns within a payload. This makes Ethernet, Zigbee and many other protocols vulnerable to packet-in-packet attack. I've described the danger of packet-in-packet attack applied to AFDX but this is often met with dis-belief.
Cell structures don't have this problem because data is inserted between a regular spacing of boundary markers. Admittedly, this arrangement is superior for continuous point-to-point links; especially when a continuous stream of empty padding cells is sent to maintain a link. This isn't an option for burst protocols between many nodes but it is very useful between long-term partners.
Cell boundary markers work in a similar fashion to NTSC horizontal sync markers and allow transmitter and receiver to stay on track over long durations. In the very worst case, the marker pattern should never be off by more than one bit from its expected position. (Any difference occurs because the transmitter oscillator and the receiver oscillator run at very similar frequencies but not identical frequencies.) Furthermore, when this case occurs, the contents of a cell is known to be suspect.
Adaption
It is very useful to transfer packets over cells. This is achieved by fragmenting packets across multiple cells. In this case, a proportion of a cell is required to indicate the first fragment, the last fragment and/or a fragment number. For the remainder of the cell, each packet begins at a cell payload boundary. The last cell of a set may be padded with zeros. This ensures that the next packet begins at the next cell payload boundary.
A common case is also a worked example. If a single byte payload is sent over TCP/IPv4 over PPPoA over AAL5 [ATM Adaption Layer 5] then packet length is typically 41 bytes (40 byte IPv4 header, 40 byte TCP header, one byte payload) and cell size is 48 bytes. This would appear to fit into one cell. However, AAL5 encapulation overhead is 6-8 bytes per cell plus one bit(!) within ATM's five byte header. Anyhow, in this scenario, a packet may be split at the end of its TCP header. In typical cases, TCP/IPv4 packets fragment over PPPoA over AAL5. In all cases, TCP/IPv4 packets fragment over PPPoE over AAL5. However, there is a large amount of inefficiency with this arrangement. It almost shouts "This is badly implemented! Fix me!"
(Technical details are taken from an expired patent application and are therefore public domain.)
A consideration of AAL5's large headers and a consideration of cell re-fragmentation across multiple cell networks led to the insight that cells within a set should be sent backwards with a cell number. This arrangement permits several tricks with buffers and state machines. In particular, it permits simple low-bandwidth implementations and optimized high-bandwidth implementations. In all cases, receipt of fragment zero indicates that the final cell within a set has been received. A counter may determine if fragments are missing. FEC may be performed and/or the next protocol layer may be informed of holes.
Where cells are re-fragmented, a decrementing cell offset allows arbitrary re-fragmentation without knowing the exact length of encapsulated payload. (This incurs up to one cell of additional padding per re-fragmentation.) In some cases, length of bridged headers may not be known either. This occurs at near-line-speed without receipt of a full packet.
Further consideration of payload size and resilience leads to an encapulation header with two or three BER fields. The first field is the fragment number. This can be partially decoded without consideration of packet type. The second field is packet type. For trivial decoders, this acts as a rip-stop for a corrupted fragment number. In the case of frag=0, a third field is present. This is the exact payload length. Many protocols which could be encapsulated (Ethernet, IPv4, IPv6, TCP, UDP) already have one or more payload lengths. This field covers trivial protocols which don't specify their own length. It is typically no more than two bytes. (For BER, this allows values up to 2^14-1 - which is 16383 bytes.) It can be de-normalized with leading zeros but this introduces significant inefficiency in boundary cases. Specifically, a de-normalized byte may incur one extra cell. However, during particular cases of re-fragmentation, there may be boundary cases where the payload length field is itself an ambiguous length. In this case, it must be assumed to be maximum length. Foreseeably, this leads to inefficiency when handling particular packet lengths.
This arrangement allows arbitrary packets to be tunnelled over protocols ranging from CANBus (8 bytes) to TeTRa (10-16 bytes) to ATM (48 byte) to USB (512-1024 bytes).
A variant of this arrangement permits Ethernet over a two byte payload. This is an extreme example which remains impractical even if you convince someone that it is possible. A 16 bit payload can be divided into a variable length fragment number field and a variable length payload field. If the first bit is zero then 10 bits represent fragment number and five bits represent payload. If the first bit is one then 11 bits represent fragment number and four bits represent payload. It is implicit that all 10 bit fragment numbers precede all 11 bit fragment numbers. Regardless, this is sufficient to send Ethernet over two byte payloads in a manner which is optimized for shorter Ethernet packets. Jumbo Ethernet over CANBus is suggested as an exercise.
Cells Versus Packets, Part 1
(This is the 20th of many promised articles which explain an idea in isolation. It is hoped that ideas may be adapted, linked together and implemented.)
While I await components for a toy robot, a somewhat toy 52 Watt quadcopter and a very serious 3D speaker array system, I'll explain some previous research which is not widely known. I'll start with an introduction, move to implementation details, limitations and a possible solution.
Cells
Local Area Networks typically use variable length packets. Our dependence upon wired Ethernet and wireless Ethernet is so widespread that people have difficulty imagining any other techniques. However, while Ethernet dominates short spans of network, long-distance connections invariably use fixed length cells. This includes the majority of digital satellite communication, cell-phones and broadband systems. The division between LAN and WAN [Local Area Networking and Wide Area Networking] remains very real due to pragmatic reasons.
Variable length packets maximize bandwidth but fixed length cells maximize reliability. For long-distance communication, reliability is more important than bandwidth. Indeed, without reliability over a long span, bandwidth is zero.
Framing
Finding the start of a packet is difficult and cumbersome. Typically, there is a start sequence. This applies even in the trivial case of RS-232 serial communication in which start bits, stop bits, payload bits and parity bits are all configurable. For Ethernet, the start sequence is a known pattern of eight bytes. For Zigbee, it is a shorter pattern of four bytes. This inefficient but acceptably so within the span of a broadcast network. However, pre-amble is highly insecure because the patterns used in the start sequence are also valid patterns within a payload. This makes Ethernet, Zigbee and many other protocols vulnerable to packet-in-packet attack. I've described the danger of packet-in-packet attack applied to AFDX but this is often met with dis-belief.
Cell structures don't have this problem because data is inserted between a regular spacing of boundary markers. Admittedly, this arrangement is superior for continuous point-to-point links; especially when a continuous stream of empty padding cells is sent to maintain a link. This isn't an option for burst protocols between many nodes but it is very useful between long-term partners.
Cell boundary markers work in a similar fashion to NTSC horizontal sync markers and allow transmitter and receiver to stay on track over long durations. In the very worst case, the marker pattern should never be off by more than one bit from its expected position. (Any difference occurs because the transmitter oscillator and the receiver oscillator run at very similar frequencies but not identical frequencies.) Furthermore, when this case occurs, the contents of a cell is known to be suspect.
Adaption
It is very useful to transfer packets over cells. This is achieved by fragmenting packets across multiple cells. In this case, a proportion of a cell is required to indicate the first fragment, the last fragment and/or a fragment number. For the remainder of the cell, each packet begins at a cell payload boundary. The last cell of a set may be padded with zeros. This ensures that the next packet begins at the next cell payload boundary.
A common case is also a worked example. If a single byte payload is sent over TCP/IPv4 over PPPoA over AAL5 [ATM Adaption Layer 5] then packet length is typically 41 bytes (40 byte IPv4 header, 40 byte TCP header, one byte payload) and cell size is 48 bytes. This would appear to fit into one cell. However, AAL5 encapulation overhead is 6-8 bytes per cell plus one bit(!) within ATM's five byte header. Anyhow, in this scenario, a packet may be split at the end of its TCP header. In typical cases, TCP/IPv4 packets fragment over PPPoA over AAL5. In all cases, TCP/IPv4 packets fragment over PPPoE over AAL5. However, there is a large amount of inefficiency with this arrangement. It almost shouts "This is badly implemented! Fix me!"
(Technical details are taken from an expired patent application and are therefore public domain.)
A consideration of AAL5's large headers and a consideration of cell re-fragmentation across multiple cell networks led to the insight that cells within a set should be sent backwards with a cell number. This arrangement permits several tricks with buffers and state machines. In particular, it permits simple low-bandwidth implementations and optimized high-bandwidth implementations. In all cases, receipt of fragment zero indicates that the final cell within a set has been received. A counter may determine if fragments are missing. FEC may be performed and/or the next protocol layer may be informed of holes.
Where cells are re-fragmented, a decrementing cell offset allows arbitrary re-fragmentation without knowing the exact length of encapsulated payload. (This incurs up to one cell of additional padding per re-fragmentation.) In some cases, length of bridged headers may not be known either. This occurs at near-line-speed without receipt of a full packet.
Further consideration of payload size and resilience leads to an encapulation header with two or three BER fields. The first field is the fragment number. This can be partially decoded without consideration of packet type. The second field is packet type. For trivial decoders, this acts as a rip-stop for a corrupted fragment number. In the case of frag=0, a third field is present. This is the exact payload length. Many protocols which could be encapsulated (Ethernet, IPv4, IPv6, TCP, UDP) already have one or more payload lengths. This field covers trivial protocols which don't specify their own length. It is typically no more than two bytes. (For BER, this allows values up to 2^14-1 - which is 16383 bytes.) It can be de-normalized with leading zeros but this introduces significant inefficiency in boundary cases. Specifically, a de-normalized byte may incur one extra cell. However, during particular cases of re-fragmentation, there may be boundary cases where the payload length field is itself an ambiguous length. In this case, it must be assumed to be maximum length. Foreseeably, this leads to inefficiency when handling particular packet lengths.
This arrangement allows arbitrary packets to be tunnelled over protocols ranging from CANBus (8 bytes) to TeTRa (10-16 bytes) to ATM (48 byte) to USB (512-1024 bytes).
A variant of this arrangement permits Ethernet over a two byte payload. This is an extreme example which remains impractical even if you convince someone that it is possible. A 16 bit payload can be divided into a variable length fragment number field and a variable length payload field. If the first bit is zero then 10 bits represent fragment number and five bits represent payload. If the first bit is one then 11 bits represent fragment number and four bits represent payload. It is implicit that all 10 bit fragment numbers precede all 11 bit fragment numbers. Regardless, this is sufficient to send Ethernet over two byte payloads in a manner which is optimized for shorter Ethernet packets. Jumbo Ethernet over CANBus is suggested as an exercise.