Protocol libraries the right way

29
Building Protocol Libraries The Right Way

Transcript of Protocol libraries the right way

Page 1: Protocol libraries the right way

Building Protocol Libraries

The Right Way

Page 2: Protocol libraries the right way

Hi!

Page 3: Protocol libraries the right way
Page 4: Protocol libraries the right way

Python HTTP• Hyper

• h2, hpack, priority, etc.

• Requests + urllib3

• PyCA (PyOpenSSL)

Page 5: Protocol libraries the right way

Why All Your Libraries Are Garbage, And All But One Of Mine Are

Too

Page 6: Protocol libraries the right way

Python HTTP Rocks

Page 7: Protocol libraries the right way

• Requests/urllib3/HTTPie

• gunicorn

• aiohttp

• Twisted

• Tornado

Page 8: Protocol libraries the right way

~0% Shared Code

Page 9: Protocol libraries the right way

Code Re-Use 👍

Page 10: Protocol libraries the right way

I/O

Page 11: Protocol libraries the right way

• httplib: state machine and parser mixed with I/O

• aiohttp/Tornado/Twisted: state machine and parser use concurrency primitives

Page 12: Protocol libraries the right way

Choice of I/O limits choice of libraries

Page 13: Protocol libraries the right way

So What?

Page 14: Protocol libraries the right way

Wasted Effort

Page 15: Protocol libraries the right way

Reduces Experimentation

Page 16: Protocol libraries the right way

Duplicate Bugs

Page 17: Protocol libraries the right way

Limits Optimisation

Page 18: Protocol libraries the right way

Avoiding This Problem

Page 19: Protocol libraries the right way

Don’t Do I/O

Page 20: Protocol libraries the right way

Alternative APIevents=handle_data(in_bytes)

out_bytes=perform_action()

Page 21: Protocol libraries the right way

Not All-Purpose

Page 22: Protocol libraries the right way

Build Wrappers

Page 23: Protocol libraries the right way

Why Bother?

Page 24: Protocol libraries the right way

=============================testsessionstarts==============================platformdarwin--Python2.7.11,pytest-2.8.5,py-1.4.31,pluggy-0.3.1rootdir:/Users/cory/Documents/Python/hyper-h2,inifile:plugins:hypothesis-1.19.0,cov-2.2.0,xdist-1.13.1gw0[513]/gw1[513]/gw2[513]/gw3[513]schedulingtestsviaLoadScheduling.................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

---------------coverage:platformdarwin,python2.7.11-final-0---------------NameStmtsMissBranchBrPartCoverMissing----------------------------------------------------------------h2/__init__.py2000100%h2/connection.py4250960100%h2/errors.py17000100%h2/events.py95040100%h2/exceptions.py41000100%h2/frame_buffer.py550180100%h2/settings.py740200100%h2/stream.py2860460100%h2/utilities.py450240100%----------------------------------------------------------------TOTAL104002080100%==========================513passedin6.69seconds==========================

Page 25: Protocol libraries the right way

I/O Is Application-

Specific

Page 26: Protocol libraries the right way

Build A Toolbox

Page 27: Protocol libraries the right way

Example• hyper-h2, a pure Python

HTTP/2 protocol stack.

• http://python-hyper.org/h2

Page 28: Protocol libraries the right way

Help Us

Page 29: Protocol libraries the right way

Thanks! Questions?