Customising Your Own WebFramework in Go20 January 2015

Jonathan GomezEngineer, Zumata

This Talk

Overview - Intro to serving requests with http/net - Customising Handlers - Writing Middleware - Ecosystem

Key takeaways - Compared with Ruby/Node.js, mainly using the standard library is considered normal- Interfaces and first-class functions make it easy to extend functionality - Ecosystem of libraries that work alongside http/net is growing

Intro to Serving Requests with http/net

Serving Requests via Standard Lib (1/4)

package main

import "net/http"

func handlerFn(w http.ResponseWriter, r *http.Request) { w.Write([]byte(̀Hello world!̀))}

func main() { http.HandleFunc("/", handlerFn) http.ListenAndServe("localhost:4000", nil)}

ListenAndServe - creates server that will listen for requests

Each request spawns a go routine: go c.serve()

Serving Requests via Standard Lib (2/4)

ServeMux matches incoming request against a list of patterns (method/host/url)

ServeMux is a special kind of Handler which calls another Handler

Handler interface

type Handler interface { ServeHTTP(ResponseWriter, *Request)}

Serving Requests via Standard Lib (3/4)

Request handling logic in ordinary function func(ResponseWriter, *Request)

func final(w http.ResponseWriter, r *http.Request) { w.Write([]byte("OK"))}

Register the function as a Handler on DefaultServeMux

http.Handle("/", http.HandlerFunc(final))

Also can:

http.HandleFunc("/", final)

Serving Requests via Standard Lib (4/4)

func(ResponseWriter, *Request)

ResponseWriter interface

type ResponseWriter interface { Header() Header Write([]byte) (int, error) WriteHeader(int)}

Request struct

type Request struct { Method string URL *url.URL Header Header Body io.ReadCloser ContentLength int64 Host string RemoteAddr string ...}

Customising Handlers

Demo: Customising Handlers - DRY Response Handling (1/3)

type appHandler struct { h func(http.ResponseWriter, *http.Request) (error)}

func (ah appHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) { err := ah.h(w, r) if err != nil { switch err := err.(type) { case ErrorDetails: ErrorJSON(w, err) default: ErrorJSON(w, ErrorDetails{"Internal Server Error", "", 500}) } }}

In app code we might extend this further:

Add error types and respond differently.

e.g. warn vs error-level log, send alerts, increment error metrics

Demo: Customising Handlers - DRY Response Handling (2/3)

type ErrorDetails struct { Message string ̀json:"error"̀ Details string ̀json:"details,omitempty"̀ Status int ̀json:"-"̀}

func (e ErrorDetails) Error() string { return fmt.Sprintf("Error: %s, Details: %s", e.Message, e.Details)}

func ErrorJSON(w http.ResponseWriter, details ErrorDetails) {

jsonB, err := json.Marshal(details) if err != nil { http.Error(w, err.Error(), 500) return }

w.Header().Set("Content-Type", "application/json") w.WriteHeader(details.Status) w.Write(jsonB)}

Demo: Customising Handlers - DRY Response Handling (3/3)

Use of special struct and special handler function to satisfy Handler interface

http.Handle("/", appHandler{unstableEndpoint})

Reduce repetition, extend functionality.

func unstableEndpoint(w http.ResponseWriter, r *http.Request) (error) {

if rand.Intn(100) > 60 { return ErrorDetails{"Strange request", "Please try again.", 422} }

if rand.Intn(100) > 80 { return ErrorDetails{"Serious failure", "We are investigating.", 500} }

w.Write([]byte(̀{"ok":true}̀)) return nil} Run

Demo: Customising Handlers - Avoiding Globals

Allows injecting dependencies rather than relying on global variables.

type Api struct { importantThing string // db *gorp.DbMap // redis *redis.Pool // logger ...}

type appHandler struct { *Api h func(*Api, http.ResponseWriter, *http.Request)}

func (ah appHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) { ah.h(ah.Api, w, r)}

func myHandler(a *Api, w http.ResponseWriter, r *http.Request) { w.Write([]byte("2015: Year of the " + a.importantThing))} Run

Writing Middleware

Middleware: Why?

Abstract common functionality across a set of handlers

Bare minimum in Go:

func(next http.Handler) http.Handler

Typical uses of middleware across languages/frameworks: - logging - authentication - handling panic / exceptions - gzipping - request parsing

Demo: Middleware Example (Panic Recovery)

func recoveryMiddleware(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { defer func() { if err := recover(); err != nil { log.Println("Recover from error:", err) http.Error(w, http.StatusText(500), 500) } }() log.Println("Executing recoveryMiddleware") next.ServeHTTP(w, r) })

}

func final(w http.ResponseWriter, r *http.Request) { log.Println("Executing finalHandler") panic("walau!") w.Write([]byte("OK"))} Run

Demo: Chaining Middleware

func middlewareOne(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { log.Println("-> Executing middlewareOne") next.ServeHTTP(w, r) log.Println("-> Executing middlewareOne again") })}

Calling chain of middleware

http.Handle("/", middlewareOne(middlewareTwo(http.HandlerFunc(final))))

func middlewareTwo(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { log.Println("---> Executing middlewareTwo") next.ServeHTTP(w, r) log.Println("---> Executing middlewareTwo again") })} Run

Chaining Middleware - Alternate Syntax

3rd Party Library: Alice

Manages middleware with the standard function signature

Nice syntax for setting up chains used in different endpoints

chain := alice.New(middlewareOne, middlewareTwo)http.Handle("/", chain.Then(finalHandler))

Our example

noAuthChain := alice.New(contextMiddleware, loggerMiddleware)authChain := alice.New(contextMiddleware, loggerMiddleware, apiKeyAuthMiddleware)adminChain := alice.New(contextMiddleware, loggerMiddleware, adminAuthMiddleware)

Demo: Creating Configurable Middleware

e.g. Pass the dependency on *AppLogger

var logger *AppLogger = NewLogger()loggerMiddleware := simpleLoggerMiddlewareWrapper(logger)http.Handle("/", loggerMiddleware(http.HandlerFunc(final)))

func simpleLoggerMiddlewareWrapper(logger *AppLogger) func(http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

startTime := time.Now()

next.ServeHTTP(w, r)

endTime := time.Since(startTime) logger.Info(r.Method + " " + r.URL.String() + " " + endTime.String()) }) }} Run

Demo: Customising ResponseWriter (1/3)

type ResponseWriter interface { Header() http.Header Write([]byte) (int, error) WriteHeader(int)}

ResponseWriter as an interface allows us to extend functionality easily

Example:

Step 1: Create a struct that wraps ResponseWriter

type responseWriterLogger struct { w http.ResponseWriter data struct { status int size int }}

Record data that would be otherwise be untracked.

Demo: Customising ResponseWriter (2/3)

Step 2: Define methods required for implicit satisfaction

func (l *responseWriterLogger) Header() http.Header { return l.w.Header()}

func (l *responseWriterLogger) Write(b []byte) (int, error) {

// scenario where WriteHeader has not been called if l.data.status == 0 { l.data.status = http.StatusOK } size, err := l.w.Write(b) l.data.size += size return size, err}

func (l *responseWriterLogger) WriteHeader(code int) { l.w.WriteHeader(code) l.data.status = code}

Demo: Customising ResponseWriter (3/3)

func specialLoggerMiddlewareWrapper(logger *AppLogger) func(http.Handler) http.Handler { return func(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

startTime := time.Now()

w2 := &responseWriterLogger{w: w} next.ServeHTTP(w2, r)

logger.Info(r.Method + " " + r.URL.String() + " " + time.Since(startTime).String() + " status: " + strconv.Itoa(w2.data.status) + " size: " + strconv.Itoa(w2.data.size))

}) }} Run

Growing Middleware Ecosystem

Excerpt from Negroni Github page graceful: (https://github.com/stretchr/graceful) graceful HTTP Shutdown

oauth2: (https://github.com/goincremental/negroni-oauth2) oAuth2 middleware

binding: (https://github.com/mholt/binding) data binding from HTTP requests into structs

xrequestid: (https://github.com/pilu/xrequestid) Assign a random X-Request-Id: header to each request

gorelic: (https://github.com/jingweno/negroni-gorelic) New Relic agent for Go runtime

Mailgun's Oxy stream: (http://godoc.org/github.com/mailgun/oxy/stream) retries and buffers requests and responses

connlimit: (http://godoc.org/github.com/mailgun/oxy/connlimit) Simultaneous connections limiter

ratelimit: (http://godoc.org/github.com/mailgun/oxy/ratelimit) Rate limiter

Other Web Framework Components

Routing & Extracting URL Params - standard library can be inflexible - regex for extracting url params can feel too low level - plenty of third party routers, e.g. Gorilla mux

func ShowWidget(w http.ResponseWriter, r *http.Request) { vars := mux.Vars(r) teamIdStr := vars["team_id"] widgetIdStr := vars["widget_id"] ...}

Request-specific context - sharing data between items in middleware chain and final handler - solutions involve either global map, or per-request map/structs using customhandlers/middleware

Web frameworks vs Build on top of standard library?

Time/expertise to build what you need? Too much re-inventing? Your optimisation vs framework optimisation? Performance? Does performance order of magnitude matter? How much magic do you want? Compatibility with net/http / ecosystem? Framework interchangeability?

Martini -- 6.1k (https://github.com/go-martini/martini)

Revel -- 4.7k (https://github.com/revel/revel)

beego -- 3.7k (https://github.com/astaxie/beego)

goji -- 1.9k (https://github.com/zenazn/goji)

gin -- 1.9k (https://github.com/gin-gonic/gin)

negroni -- 1.8k (https://github.com/codegangsta/negroni)

go-json-rest -- 1.1k (https://github.com/ant0ine/go-json-rest)

Gorilla/mux -- 1.1k (https://github.com/gorilla/mux)

Tiger Tonic -- 0.8k (https://github.com/rcrowley/go-tigertonic)

Gocraft/web -- 0.6k (https://github.com/gocraft/web)

Thank you

Jonathan GomezEngineer, Zumatajonathanbgomez@gmail.com (mailto:jonathanbgomez@gmail.com)

@jonog (http://twitter.com/jonog)