RubyConf 2012: Custom Reverse Proxies
-
Upload
nickblah -
Category
Technology
-
view
1.990 -
download
1
description
Transcript of RubyConf 2012: Custom Reverse Proxies
![Page 1: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/1.jpg)
Abstracting Features Into Custom Reverse Proxies
Or: Making Better Lemonade From Chaos
Photo by Lori Greighttp://www.flickr.com/photos/lori_greig/4906180111
Nick Muerdter • RubyConf 2012 • November 1, 2012
![Page 2: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/2.jpg)
Photo by Brian Lane Winfield Moorehttp://www.flickr.com/photos/doctabu/342220423
![Page 3: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/3.jpg)
![Page 4: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/4.jpg)
![Page 5: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/5.jpg)
![Page 6: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/6.jpg)
OKAY, BUT…
![Page 7: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/7.jpg)
Photo by Brian Lane Winfield Moorehttp://www.flickr.com/photos/doctabu/342220423
![Page 8: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/8.jpg)
WHY & HOW
![Page 9: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/9.jpg)
A Story…
Photo by Elgin County Archiveshttp://www.flickr.com/photos/elgincountyarchives/
6789204118
![Page 10: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/10.jpg)
Photo by rocketlasshttp://www.flickr.com/photos/rocketlass/525244911
![Page 11: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/11.jpg)
Photo by alandberninghttp://www.flickr.com/photos/14617207@N00/4872111479
![Page 12: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/12.jpg)
![Page 13: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/13.jpg)
![Page 14: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/14.jpg)
Users Don’t Care
Photo from National Media Museumhttp://www.flickr.com/photos/nationalmediamuseum/3589381656
![Page 15: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/15.jpg)
• 1 entry point to all our APIs• 1 user account for all our APIs
• API key access to all our APIs• Rate limiting for all our APIs• Analytics for all our APIs
What We Wanted
![Page 16: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/16.jpg)
• Changes required to each API
What We Didn’t Want
![Page 17: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/17.jpg)
CUSTOMREVERSE
PROXY!!!
![Page 18: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/18.jpg)
![Page 19: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/19.jpg)
Better Lemonade?
Photo by Lara604http://www.flickr.com/photos/lara604/4563803829
![Page 20: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/20.jpg)
• 1 API key for all our services• Shielded from our internal
complexities
For API Users
![Page 21: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/21.jpg)
• Old APIs: Do absolutely nothing• New APIs: Do absolutely nothing
• Get for free:–Authentication–Rate Limiting–Analytics
For Our Developers
![Page 22: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/22.jpg)
• Reduced implementation code• Standardization is enforced• New features in the reverse proxy
benefit everyone• Reverse proxies can also be used
for scaling
![Page 23: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/23.jpg)
Building these things…
Photo from The Library of Congresshttp://www.flickr.com/photos/library_of_congress/
2179849680
![Page 24: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/24.jpg)
• Ruby & EventMachine• Blazing fast• Flexible• Low-level
em-proxy
![Page 25: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/25.jpg)
Proxy.start(:host => "0.0.0.0", :port => 80) do |conn| conn.server :srv, :host => "127.0.0.1", :port => 81
conn.on_data do |data| # Do something with the incoming data... data end
conn.on_response do |backend, resp| # Do something with the response... resp end
conn.on_finish do |backend, name| # Do something when finished... endend
![Page 26: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/26.jpg)
conn.on_data do |data| # Modify the User-Agent on the incoming # request data.gsub(/User-Agent: .*?\r\n/, "User-Agent: em-proxy/0.1\r\n")end
![Page 27: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/27.jpg)
redis = Redis.new(:host => "127.0.0.1")
conn.on_data do |data| # Fun things with Ruby! ip = peer[0] redis.incr(ip)
dataend
![Page 28: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/28.jpg)
parser = Http::Parser.newparser.on_headers_complete = proc do |h| # Hello, friendlier HTTP headers... puts h["User-Agent"]end
conn.on_data do |data| parser << data dataend
![Page 29: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/29.jpg)
Photo by Madison Guyhttp://www.flickr.com/photos/madison_guy/
3386919046
![Page 30: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/30.jpg)
Transparency
Photo by Brett Jordanhttp://www.flickr.com/photos/x1brett/6126873518
![Page 31: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/31.jpg)
Speed & Efficiency
Photo by jamesjustinhttp://www.flickr.com/photos/jamesjustin/3629097108
![Page 32: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/32.jpg)
Direct em-proxy rack-reverse-proxy0
0.5
1
1.5
2
2.5
3
3.5
4
Tim
e (m
illis
econ
ds)
(Terribly unscientific benchmarks)
![Page 33: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/33.jpg)
Direct em-proxy rack-reverse-proxy1400
1500
1600
1700
1800
1900
2000
2100
2200
2300
2400
Tim
e (m
illis
econ
ds)
(Terribly unscientific benchmarks)
![Page 34: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/34.jpg)
Flexibility
Photo from The Library of Congresshttp://www.flickr.com/photos/library_of_congress/
2179047512
![Page 35: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/35.jpg)
What Else Can You Do?
Photo by paul-simpson.orghttp://www.flickr.com/photos/paulsimpson1976/4039170901
![Page 36: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/36.jpg)
• Error handling?• Web page manipulation?
– Insert standard analytics JavaScript snippet?– Add a standard header and footer?
• Add JSONP callbacks for all JSON APIs?• Security checks?• More than HTTP…
– Intercept & manipulate e-mail?– Intercept & manipulate database calls?
![Page 37: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/37.jpg)
Photo by Keoki Seuhttp://www.flickr.com/photos/keokiseu/
4973314636
![Page 38: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/38.jpg)
Buffering
Photo from The Library of Congresshttp://www.flickr.com/photos/library_of_congress/3159321339
![Page 39: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/39.jpg)
Content-Length
Photo by Sterlichttp://www.flickr.com/photos/sterlic/4299633060
![Page 40: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/40.jpg)
gzip
Photo by Kaptain Koboldhttp://www.flickr.com/photos/kaptainkobold/6930870617
![Page 41: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/41.jpg)
Want Bigger?
Photo by elviskennedyhttp://www.flickr.com/photos/elviskennedy/
5465419950
![Page 42: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/42.jpg)
![Page 43: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/43.jpg)
WEBSERVICESBONANZA
![Page 44: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/44.jpg)
• Make it easier for users to find and consume federal APIs
• Make it easier for federal agencies to develop & deploy more APIs
Main Objectives
![Page 45: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/45.jpg)
Photo by alandberninghttp://www.flickr.com/photos/14617207@N00/4872111479
Same Problem
![Page 46: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/46.jpg)
Same Solution?
![Page 47: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/47.jpg)
Stay Tuned…
Photo by Lord Jeromehttp://www.flickr.com/photos/lordjerome/127381557
![Page 48: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/48.jpg)
SO………
![Page 49: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/49.jpg)
Photo by judepicshttp://www.flickr.com/photos/judepics/159365806
![Page 50: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/50.jpg)
• Reverse Proxies: Fun for the whole family!
• Custom Reverse Proxies: You might be able to implement more functionality at this layer than you realize.
• Think Different: They can provide a different way to architect some features of your apps.
![Page 51: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/51.jpg)
Photo by Musée McCord Museumhttp://www.flickr.com/photos/museemccordmuseum/5348751435
Resources & Support
![Page 52: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/52.jpg)
• Our full API management solution– Includes custom Event Machine based proxy
• Open source
API Umbrella
https://github.com/NREL/api-umbrella
(Just recently open sourced, so pardon the current state of things)
![Page 53: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/53.jpg)
• em-proxy– https://github.com/igrigorik/em-proxy– Simple and very capable
• ProxyMachine– https://github.com/mojombo/proxymachine– Simpler, but can only act on requests, not responses
• Goliath– https://github.com/postrank-labs/goliath– More of a framework, uses em-synchrony (Fibers)
Ruby & Event Machine
![Page 54: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/54.jpg)
• HAProxy– http://haproxy.1wt.eu/– General proxy and load balancing awesomeness
• Varnish Cache– https://www.varnish-cache.org/– Proxy caching layer coolness
• nginx– http://nginx.org/– Web server powerhouse and nice proxy
Other Reverse Proxies
![Page 55: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/55.jpg)
• http://developer.nrel.gov/
Renewable Energy APIs
(Lots more APIs coming soon)
![Page 57: RubyConf 2012: Custom Reverse Proxies](https://reader031.fdocuments.in/reader031/viewer/2022012405/554bc9f3b4c90594278b5811/html5/thumbnails/57.jpg)
http://bit.ly/rubystache
Enjoyed this presentation? Enjoyed this ‘stache?
Enjoy charity?
[email protected] @nickblah