Implementing Network Protocols in the - schd.wsschd.ws/hosted_files/openiotelcna2017/8d/Implementing...

Leandro A. F. [email protected]

Implementing Network Protocols in the Zephyr* Project

OpenIoT Summit 2017 — Portland, OR

Implementing network protocols in Zephyr* / OpenIoT 2017

About me


Zephyr* in one slide● Microcontroller operating system● Very small footprint

○ Will run in 8KiB of RAM

● Open Source under Apache* 2.0 license● Hosted by Linux* Foundation● Supports multiple architectures

Check the booths for demos and more info!


Table of Contents● New IP stack in Zephyr*● Currently implemented network protocols● Patterns


Zephyr* has a new IP stackOld stack

● uIP from Contiki*○ No simultaneous IPv6 and IPv4○ Only one network interface

● Ported to Zephyr○ Didn't fit well with the rest of the system○ Parts of Contiki had to be ported over○ Changes made it unstable, "unfixable"

New stack

● Multiple interfaces● Simultaneous IPv6 and IPv4● Better fits the rest of the system

○ Buffer management● Available since 1.6

○ 1.7 will enable TCP


Multiple Interfaces● Configurable at compile time● Ethernet, Bluetooth®, 802.15.4, SLIP (for testing)

○ Soon: Wi-Fi*

● Can mix and match interface types


Simultaneous IPv4 and IPv6

IPv4 IPv6 Code size* (.text)

✘ ✓ ~11KiB

✓ ✘ ~9KiB

✓ ✓ ~12KiB

Non-scientific measurement, in KiB, of the echoserver sample application in multiple configurations

* Measurements may vary depending on target architecture and featureset


Buffer Management● Zephyr* is "heapless" inside the kernel

○ Memory pools used instead○ Separate pools for transmitting, receiving, and data○ Timeouts for obtaining things from pools

● Fragment chains○ Fragments of fixed size (128 bytes each)○ Chain can be compacted or copied to a linear buffer


Currently ImplementedNetwork Protocols


DNS Domain Name System● Both IPv4 and IPv6 queries are supported

○ A and AAAA queries only

● Support for concurrent queries○ Pool of 512 byte chunks (maximum UDP response size as per RFC1035)○ Configurable number of concurrent queries

● Fragment chain from the network copied to a linear buffer● Implemented from scratch● Available as part of the platform


HTTP Hypertext Transport Protocol● Both client and server● HTTP/1.0 and HTTP/1.1● Uses parser from Nginx● Available as sample applications (client/server)


NATS publisher/subscriber protocol● Asynchronous API● Line-based protocol● Implemented from scratch● Available as a sample application


MQTT Message Queue Telemetry Transport● Asynchronous API● Implemented from scratch● Available as part of the platform


CoAP Constrained Application Protocol● Asynchronous API

○ GET, POST, PUT, and DELETE methods

● Resource observation & notifications● Implementation based on lessons learned from the Soletta™ project● Available as part of the platform


IRC Internet Relay Chat● Asynchronous API● Line-based protocol● Implemented from scratch● Available as a sample application


Patterns


Asynchronous APIs● Used with request-less protocols● Information can be received at any time

○ No clients blocked while reading from the network○ Background work


Buffer linearization● Used to parse structs larger than the fragment size

○ Fragments usually 128 bytes

● Not optimal○ Unnecessary memory copies○ Memory pools for linearized buffers○ May go away in the future


Reusing vs. Re-implementing● Implementations may be unsuitable for

embedded systems○ Dynamic memory allocations○ Assumes a POSIX* operating system○ Might depend on heavyweight libraries

(libstdc++, glib, etc)○ Etc.

● Existing implementations may be mature

● Reimplementing is tricky○ Not leveraging possible years of

development● A lot of control● Tighter integration with the rest of the

system


Sample application vs. Platform APISample application

● Testbed for APIs● No guarantee the APIs will be stable● Often only suitable for demonstration

purposes● Not necessarily part of the "IoT Suite"

Available as part of the platform

● Stable, complete APIs● Suitable for production use● Protocols from the "IoT Suite"


Compromises: example● NATS protocol requires a JSON parser● Most JSON parsers require heap memory● Wrote our own:

○ Zero-copy○ Decodes directly to a struct○ "Type safe"○ No temporary structs

● Only necessary things implemented:○ No nested objects/arrays○ UTF-8 aware but not strict○ No floating point numbers○ Up to 32 fields


Code Samples


Asynchronous APIs

static const char *const led_default_path[] = { "led", NULL };static struct zoap_resource resources[] = { ZOAP_WELL_KNOWN_CORE_RESOURCE , { .get = led_get, .post = led_post, .put = led_put, .path = led_default_path, .user_data = /* … */, }, { .get = dummy_get, .path = dummy_path, .user_data = /* … */, }, { },};

The CoAP implementation will call the handler functions for each method


Asynchronous APIs

static const char *const led_default_path[] = { "led", NULL };static struct zoap_resource resources[] = { ZOAP_WELL_KNOWN_CORE_RESOURCE , { .get = led_get, .post = led_post, .put = led_put, .path = led_default_path, .user_data = /* … */, }, { .get = dummy_get, .path = dummy_path, .user_data = /* … */, }, { },};

Users of the API don't need to worry about matching resource paths


JSON parsing

struct my_struct { int number; char *string;};

#define FIELD(struct_, member_, type_) /* … */

static const struct json_obj_descr my_struct_descr[] = { FIELD(struct my_struct, number, JSON_TOK_NUMBER), FIELD(struct my_struct, string, JSON_TOK_STRING),};

struct my_struct s;int32_t result = json_obj_parse(json_data, json_data_len,

my_struct_descr, ARRAY_SIZE(my_struct_descr), &s);


JSON parsing



static const struct json_obj_descr my_struct_descr[] = { FIELD(struct my_struct, number, JSON_TOK_NUMBER), FIELD(struct my_struct, string, JSON_TOK_STRING),};



The struct is described, and, while parsing, types are checked against the provided description.


JSON parsing



static const struct json_obj_descr my_struct_descr[] = { FIELD(struct my_struct, field, JSON_TOK_NUMBER), FIELD(struct my_struct, field, JSON_TOK_STRING),};



If successful, result is 0x3: both number (first field, 1<<0) and string (1<<1) fields have been decoded.


Wrapping up● Some of the differences between old and new IP stacks● Protocols currently implemented● Patterns used by the implementations● Some code samples

What would you do differently?

What protocols are you going to implement in Zephyr*?

[email protected]


DisclaimerIntel, the Intel logo, and Soletta are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.*Other names and brands may be claimed as the property of others.© Intel Corporation

Implementing Network Protocols in the - schd.wsschd.ws/hosted_files/openiotelcna2017/8d/Implementing...

Documents

Transcript of Implementing Network Protocols in the - schd.wsschd.ws/hosted_files/openiotelcna2017/8d/Implementing...