Hello, my name is Darshan. This is a lightning talk about a requests wrapper I developed for talking to an embedded device. The class is called RESTInterface, the title of this talk. So welcome.

1

Again, my name is Darshan. I am a software engineer for Enphase Energy, a microinverter manufacturer. I use Python to automate the manufacturing test processes for our products. You can see part of our manufacturing line in the picture there.

2

In this talk, I will first explain the problem with talking to our embedded device. Then I will explain the solution, the RESTInterface class, and show you how to use it. Next, I will reveal how some of the features were implemented. And finally, I will share with you some links to other Python REST client modules.

3

4

This is the Envoy, our communication gateway. It allows us to talk to our microinverters for the purpose of, for example, retrieving power production data. It communicates to our inverters via Power Line Comms and connects to other software via a REST interface. This is where the problem lies. The REST resources are all slightly different. Some resources are encoded in JSON, some in XML, some not at all. Some resources are just to invoke a side effect, such as rebooting. This requires a longer timeout. And some resources, such as the ones that retrieve data from our microinverter sometimes fails and thus require retrying. Basically the problem is that our REST resources behave differently and they some times fail.

5

6

To talk to the Envoy REST interface, I wrote a wrapper around requests. This wrapper, the RESTInterface class has four public methods: get, put, post, delete. It supports decoding different data types. It handles retrying. And it supports specifying different timeouts, auths, etc. All the stuff that requests supports. It basically just forms the call to requests, solving the problems I just mentioned.

7

8

Here is an example of how to use the RESTInterface class. EnvoyRESTInterface inherits from RESTInterface. In this example, the get_info method, which calls the ‘info’ resource, is encoded in XML, so I pass in an XML parser, a callable, for the decoder. The get_flakey_thing method illustrates how to specify retrying. This method will call the ‘thing’ resource up to 5 times, delaying a quarter second between attempts. This particular interface is encoding in JSON, so the JSON decoder is provided. In the reboot method, I have specified a longer time out because this resource means to reboot the device which requires up to 2 minutes. In this example, you can see I have specified resources with different ports, decoders, auths, timeouts and retries for each resource.

9

Lets get in to some implementation details. I will show you how the flexible encoding and decoding works.

10

This is the wrapper around the requests call. It calls requests hence the name. It takes a requests function, like requests.get or requests.put, the requests keyword arguments and some other stuff. This stuff includes the encoder and the decoder, which are both callables. The encoder parameter, if provided, is called with the body_data, to encode, for example a dict to JSON. If the decoder callable is provided, the request’s response.text is passed to it. Providing the encoder and decoder as parameters allows for arbitrary encoding and decoding.

11

Okay, let me show you how retrying was implemented.

12

This is the function which handles retrying. There is some code here that is not shown (the … indicate this). The retry_if function takes as parameters a function, its arguments and key word arguments. It also takes parameters which specify how to retry, like max_attempts, retry_sleep, etc. As you can see here, it is simply a while loop with a try/except/finally clause. It tries to call the provided function with its arguments and keyword arguments. If an exception occurs, it determines if that exception should be raised or ignored. If ignored, the function is retried. In the finally clause, you can see that the attempts and elapsed_time members are set on the function itself. It is a way for the caller to determine how many attempts actually occurred. The previous call_requests method is actually passed to this function, such that the requests call can be retried if an HTTPError occurs.

13

To conclude, I want to share with you some links.

14

I do plan to post the RESTInterface class and retries code on PyPI and GitHub. There are already some Python REST clients available. These were the ones I found. If you are not familiar with the HTTP modules available in Python, check out the comparison article. It shows the API differences among the various modules. urrlib2, httplib2, pycurl, and requests. The obvious conclusion is that requests has the best, simplest API. These links are embedded in the docs, so go to the meetup to download the slide.

15

Thank you!

16