Post on 15-Apr-2017
Copyright 2016 Tendril, Inc. All rights reserved.
FINAGLE YOUR OWN CODEC
Extending Finagle with Protocol Buffers
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
AgendaINTRODUCTION
2
Finagle Protocol Concepts
Finagle ProtobufLessons and Recommendations
Copyright 2016 Tendril, Inc. All rights reserved.
FINAGLE PROTOCOL CONCEPTS
3
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
WHAT IS FINAGLE ANYWAY?
Finagle is an extensible RPC system for the JVM, used to construct high-concurrency servers.
…Most of Finagle’s code is protocol agnostic, simplifying the
implementation of new protocols.
http://twitter.github.io/finagle/(emphasis mine)
4
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Why RPC and Binary Formats?
• RPC vs REST• Operation-oriented• No mapping to fixed verbs
• Binary formats vs JSON• Smaller message size• Serialization performance• Versioning and other semantics
• Interface Definition Language (IDL)• Available operations• Message schema
CONCEPTS
5
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
What is a Protocol?
• Merriam-Webstera set of conventions governing the treatment and especially the formatting of data in an electronic communications system
• A protocol includes• Codec • Dispatchers • Client and server configuration and initialization • Error handling • Integration with code generators or compilers
CONCEPTS
6
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
What is a Codec
• Codec trait• Defines encoding/decoding
• Interface to Netty channel pipeline• Modify the service filter stack• Server and Client sides
• Symmetric – same objects both sides?• Request and Response – same encoding both ways?
• CodecFactory relates client and server
CONCEPTS
7
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
End to End FlowCONCEPTS
8
Client ClientDispatch Encoder
Finagle/Netty
Decoder Service Dispatch
Service Impl
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Interfaces
• What sort of interface to expose?• Finagle interfaces – e.g. Service[Req,Resp]• Your protocol
• Thrift - special compiler – Scrooge • Protobuf - code generated by protoc
CONCEPTS
9
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Connection Handling
• Connection pooling• Connections created for requests• One connection == one active request• Reuse only after a request is done• Configuration of pool size limits, Etc
• Mux (connection multiplexing)• One connection used for many simultaneous requests• Match responses up with open requests• Simpler configuration model
CONCEPTS
10
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Service Stack
• Finagle services are created from a stack of components• Timeouts• Stats
• Stack API • Flexibility but complexity
• ClientBuilder / ServerBuilder• Simpler but limited• Provides basic stack
• ClientBuilder and ServerBuilder will be deprecated eventually
CONCEPTS
11
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Error Handling – Two Types of Errors
• Application• Something gone wrong in the service • Exception• Non-exceptional fail response
• Framework• Timeouts• Rejections• Uncaught exceptions
CONCEPTS
12
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Error Handling – Ways to Handle
• Protocol • E.g. Thrift has an exception type
• Transport • Error message is a certain type on the wire
• Application • User messages must include some room for errors
• How should exceptions be caught and mapped on the server?
• How should they map back on the client? • Same mechanism or different mechanisms for client and
server?
CONCEPTS
13
Copyright 2016 Tendril, Inc. All rights reserved.
FINAGLE-PROTOBUF
14
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Protobuf Message IDL
message Echo { optional string phrase = 1; optional uint32 offset = 2; optional com.tendril.platform.common.Context context = 3;}
val echoMessage = Echo.newBuilder .setPhrase("myPhrase") .setOffset(0) .build
echoMessage.hasContext should be(false)
FINAGLE-PROTOBUF
15
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Protobuf Service IDLservice EchoService { rpc Echo (EchoRequest) returns (EchoResponse);}
public static abstract class EchoService implements com.google.protobuf.Service {… public interface Interface { public abstract void echo( com.google.protobuf.RpcController controller, Echo.EchoRequest request, com.google.protobuf.RpcCallback<Echo.EchoResponse> done); } …}
FINAGLE-PROTOBUF
16
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Request and Response
message EchoRequest { optional string phrase = 1;}
message EchoResponse { optional com.tendril.platform.common.Error error = 1; optional Echo echo = 2;}
FINAGLE-PROTOBUF
17
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Client Creation and Invocation
def buildEchoClient(port: Int, filters:SimpleFilter[ProtobufRequest,ProtobufResponse]*) (implicit factory: RpcFactory, executorService: ExecutorService, tracer: Tracer) = { val clientBuilder = ClientBuilder() …
val serviceStub = EchoService.newStub(null) .asInstanceOf[ {def newStub(channel: RpcChannel): EchoService}]
factory.createStub( … )}
val distributedEchoClient = buildEchoClient(distributedServerPort)val controller = factory.createController()val request = EchoRequest.newBuilder().setPhrase("Hello world”).build()val callback = new RpcCallback[EchoResponse] { … }distributedEchoClient.echo(controller, request, callback)
FINAGLE-PROTOBUF
18
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Client Creation and Dispatch
• RpcFactory – build stub• Stub
• Generated by protoc, implements RPC service interface• Delegates method calls to a channel
• RpcChannelImpl• Uses ClientBuilder to build the Finagle
Service[Request,Response]• callMethod() delegates to the service• When service future completes, calls or fails the callback
• Encoder• Method code from hashed method name
(SimpleMethodService)
FINAGLE-PROTOBUF
19
Copyright 2016 Tendril, Inc. All rights reserved.
Service Creation
def buildEchoServer(port: Int, impl: EchoService.Interface, filters:SimpleFilter[ProtobufRequest,ProtobufResponse]*) (implicit factory: RpcFactory, executorService: ExecutorService, tracer: Tracer) = { val serverBuilder = ServerBuilder() ... factory.createServer( … )}
class ReverseEchoService extends EchoService.Interface { def echo(controller: RpcController, request: EchoRequest, callback: RpcCallback[EchoResponse]) { val response = EchoResponse.newBuilder.setPhrase(request.getPhrase.reverse).build callback.run(response) }}
FINAGLE-PROTOBUF
20Scalae By The Bay | November 11, 2016
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Service Creation and Dispatch
• RpcFactory – build service• Service implementation – implements protoc generated
interface• ServiceDispatcher
• Determine which method to call• Call method and fulfill Promise (Future returned to Finagle)
• RpcControl – failure and cancellation• RpcCallback[T] – run method
• One instance for [Response], one for [Throwable]• In practice we wrap this behind Guava ListenableFuture
interface
FINAGLE-PROTOBUF
21
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Wire Formats
• Wire Format Version 0
• Wire Format Version 1
FINAGLE-PROTOBUF
22
Method code Message length
Protobuf message
Trace span Protobuf messageTracingMarker Span Parent span Method Code Message
lengthVersion
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Version Detection
• Service is configured V0 or V1 on startup• V1 service can handle V0 or V1 frames• Detects version based on version and marker fields
• Client configured as V0 or V1 • All responses V0
• No update to clients for response version detection• Backwards compatibility maintained
FINAGLE-PROTOBUF
23
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Error Handling and Mapping
message Error { optional uint32 code = 1; optional string message = 2; }
• ServiceExceptionHandler• Service-side (ServiceDispatcher)• Maps exception to message
• ExceptionResponseHandler• Client-side (RpcChannelImpl)• Maps message to exception
FINAGLE-PROTOBUF
24
Copyright 2016 Tendril, Inc. All rights reserved.
LESSONS AND RECOMMENDATIONS
25
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Why build your own protocol
• One-side support (e.g. HTTP)• Legacy or other interop• Wrapping another protocol (e.g. MySQL, Redis)
• Use existing protocols (e.g. Thrift) otherwise
LESSONS
26
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Error Handling
• Tricky to get right, tricky to test• Prefer transport or in-protocol handling• Avoid forcing a message format
LESSONS
27
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Support
• Community is solidly Thrift• Twitter support is mostly Mux • If you go another way you’re doing a lot of work on your
own• If you build on Mux you get many Twitter enhancements
“for free”
LESSONS
28
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Adapt or Adopt
• Generated interfaces • Generated interface or Finagle interface• Existing generator or roll your own
• Hide or expose Twitter Futures, Finagle? • Façade approaches are very tricky
• Mapping or leaky abstractions• Hand-rolled or generated code ??• More layers and more magic == fewer experts on your team
LESSONS
29
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Interfaces and Implementations
• Your protocol defines interfaces
• Implementations supplied in your libraries?• Default implementations? • Example implementations?• Good docs?
LESSONS
30
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Make things easy for your developers
• Examples • Testbeds • Seed projects
LESSONS
31
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Learning about Finagle
• Configuration settings• Diagnosing failures
• What behavior is from Finagle? • What behavior is from your protocol?
LESSONS
32
Copyright 2016 Tendril, Inc. All rights reserved.Scalae By The Bay | November 11, 2016
Things to Check Out
• ScalaPB• http://trueaccord.github.io/ScalaPB/• “Spark and Protocol Buffers – An Awesome Combination”
• Nadav Samet, Saturday 9:50• Finagle Serial
• https://github.com/finagle/finagle-serial• Finagle Protobuf
• https://github.com/finagle/finagle-protobuf
LESSONS
33