Grid Toolkits

Building a Cloud API Server using Play(SCALA) & Riak RESTful API for Megam Cloud

We'll Cover

Architecture of our API

Authentication using HMAC

How to handle JSON Requests/Response

How to handle Errors

How do you interface with Riak.

Request(json)

Cloud API ServerAuth OK ?

HMAC

Response(json)

Native API WrappersRubyFunnel RequestFunnel ResponseRouterHerk RiakSnowflakeID

Play 2.2.0 setup => Link

SBT 0.13.0 Migration => Link

play2-auth Authentication

Scala 2.10.3

Play 2.2.0

SBT 0.13.0

Riak 1.4.2We'll Use

We'll also use scalaz

"org.scalaz" %% "scalaz-core" % "7.0.3"

Code is weaved with Functional Programming using scalaz

Code :

https://github.com/indykish/megam_play.git

Beta Launch of Megam Cloud (Polygot PaaS)

Our PaaS design => Link

Register http://www.megam.co for an invite

Twitter : @indykish

Screencast illustrating the Cloud API Servers working live

Play2-Auth : Setup

play2-auth : offers Authentication and Authorization features to play framework applications.

Add a dependency declaration into your Build.scala file:

val appDependencies = Seq( "jp.t2v" %% "play2.auth" % "0.11.0-SNAPSHOT", "jp.t2v" %% "play2.auth.test" % "0.11.0-SNAPSHOT" % "test" )

Authentication

play2-auth uses Stackable Controller.

This is handy for Authentication.

All you need to do is use StackAction in your Controller.

Your Controller will first call StackAction operation and then compose it with other Actions.

Scenario

/nodes

HTTP Request to Nodes shall be authenticated using HMAC

Customer onboarded and has a email/api_key (or) private cert.

Let us create a Nodes controller

object Nodes extends Controller with APIAuthElement

Full code

def post = StackAction(parse.tolerantText) { implicit request => (Validation.fromTryCatch[SimpleResult] { reqFunneled match { case Success(succ) => { val freq = succ.getOrElse(throw new Error("Request wasn't funneled. Verify the header.")) val email = freq.maybeEmail.getOrElse(throw new Error("Email not found (or) invalid.")) val clientAPIBody = freq.clientAPIBody.getOrElse(throw new Error("Body not found (or) invalid.")) models.Nodes.create(email, clientAPIBody) match { case Success(succ) => val tuple_succ = succ.getOrElse(("Nah", "Bah", "Gah")) } case Failure(err) => { val rn: FunnelResponse = new HttpReturningError(err) Status(rn.code)(rn.toJson(true)) } } } case Failure(err) => { val rn: FunnelResponse = new HttpReturningError(err) Status(rn.code)(rn.toJson(true)) } } }).fold(succ = { a: SimpleResult => a }, fail = { t: Throwable => Status(BAD_REQUEST)(t.getMessage) }) }

Controller - Nodes

What is happening ?

A HTTPRequest that comes to Cloud API Server gets funneled implicitly.

FunneledRequest as seen in next page.

FunneledRequest(FR)

A Case class

FunneledRequestBuilder creates FR

Full code

case class FunneledRequest(maybeEmail: Option[String], clientAPIHmac: Option[String], clientAPIDate: Option[String], clientAPIPath: Option[String], clientAPIBody: Option[String]) {

val wowEmail = { val EmailRegex = """^[a-z0-9_\+-]+(\.[a-z0-9_\+-]+)*@[a-z0-9-]+(\.[a-z0-9-]+)*\.([a-z]{2,4})$""".r maybeEmail.flatMap(x => EmailRegex.findFirstIn(x)) } match { case Some(succ) => Validation.success[Throwable, Option[String]](succ.some) case None => Validation.failure[Throwable, Option[String]](new MalformedHeaderError(maybeEmail.get, """Email is blank or invalid. Kindly provide us an email in the standard format.\n" eg: goodemail@megam.com""")) } val mkSign = { val ab = ((clientAPIDate ++ clientAPIPath ++ calculateMD5(clientAPIBody)) map { a: String => a }).mkString("\n") play.api.Logger.debug(("%-20s -->[%s]").format("FunnelRequest:mkSign", ab)) ab } }

FR Builder

//Look for the X_Megam_HMAC field. If not the FunneledRequest will be None. private lazy val frOpt: Option[FunneledRequest] = (for { hmac Future[SimpleResult]): Future[SimpleResult] = { SecurityActions.Authenticated(req) match { case Success(rawRes) => super.proceed(req.set(APIAccessedKey, rawRes))(f) case Failure(err) => { val g = Action { implicit request => val rn: FunnelResponse = new HttpReturningError(err) //implicitly loaded. SimpleResult(header = ResponseHeader(rn.code, Map(CONTENT_TYPE -> "text/plain")), body = Enumerator(rn.toJson(true).getBytes(UTF8Charset) )) } val origReq = req.asInstanceOf[Request[AnyContent]] g(origReq) }

} }

implicit def reqFunneled[A](implicit req: RequestWithAttributes[A]): ValidationNel[Throwable, Option[FunneledRequest]] = req2FunnelBuilder(req).funneled

implicit def apiAccessed[A](implicit req: RequestWithAttributes[A]): Option[String] = req.get(APIAccessedKey).get

}

Full code

Securing API

Uses SecurityActions to authenticate a FunneledRequest

Get FR from controller

Extract information from the FR calculate a HMAC and compares the computed HMAC from Riak.

Authentication Error if match fails.

Respond back As JSON

We respond back as JSON usingFunnelResponse Code (HTTP Status Code : 404, 503.. )

Message (A String message)

More (Detail info like support links)

JSON_CLAZ (A String understood by an unmarshaller or receiver)

FunnelResponse

case class FunnelResponse(code: Int, msg: String, more: String, json_claz: String, msg_type: String = "error", links: String = tailMsg) {

def toJValue: JValue = { import net.liftweb.json.scalaz.JsonScalaz.toJSON import controllers.funnel.FunnelResponseSerialization val funser = new FunnelResponseSerialization() toJSON(this)(funser.writer) }

def toJson(prettyPrint: Boolean = false): String = if (prettyPrint) { pretty(render(toJValue)) } else { compactRender(toJValue) }

}

Funnel Errors

CannotAuthenticateResourceItemNotFoundJSONParsingErrorServiceUnAvailableMalformedHeaderMalformedBodyHTTPReturningError

Funnel Errors Object

Case class *Errors in FunnelErrors

object FunnelErrors {

val tailMsg = """Forum :https://groups.google.com/forum/?fromgroups=#!forum/megamlive. |API :https://api.megam.co |Docs :http://docs.megam.co |Support :http://support.megam.co""".stripMargin

case class CannotAuthenticateError(input: String, msg: String, httpCode: Int = BAD_REQUEST) extends java.lang.Error(msg)

.}

case class HttpReturningError(errNel: NonEmptyList[Throwable]) extends Exception {

def mkMsg(err: Throwable): String = { err.fold( a => """Authentication failure using the email/apikey combination. %n'%s' |Verify the email and api key combination. """.format(a.input).stripMargin,

}

HTTPReturningError folds the App defined error

RichThrowable, implicit error to json

implicit class RichThrowable(thrownExp: Throwable) { def fold[T]( cannotAuthError: CannotAuthenticateError => T, malformedBodyError: MalformedBodyError => T, malformedHeaderError: MalformedHeaderError => T, serviceUnavailableError: ServiceUnavailableError => T, resourceNotFound: ResourceItemNotFound => T, anyError: Throwable => T): T = thrownExp match { case a @ CannotAuthenticateError(_, _, _) => cannotAuthError(a) case m @ MalformedBodyError(_, _, _) => malformedBodyError(m) case h @ MalformedHeaderError(_, _, _) => malformedHeaderError(h) case c @ ServiceUnavailableError(_, _, _) => serviceUnavailableError(c) case r @ ResourceItemNotFound(_, _, _) => resourceNotFound(r) case t @ _ => anyError(t) } }

implicit def err2FunnelResponse(hpret: HttpReturningError) = new FunnelResponse(hpret.code.getOrElse(BAD_REQUEST), hpret.msg, hpret.more.getOrElse(new String("none")), "Megam::Error", hpret.severity)

implicit def err2FunnelResponses(hpret: HttpReturningError) = hpret.errNel.map { err: Throwable => err.fold(a => new FunnelResponse(hpret.mkCode(a).getOrElse(BAD_REQUEST), hpret.mkMsg(a), hpret.mkMore(a), "Megam::Error", hpret.severity), m => new FunnelResponse(hpret.mkCode(m).getOrElse(BAD_REQUEST), hpret.mkMsg(m), hpret.mkMore(m), "Megam::Error", hpret.severity), h => new FunnelResponse(hpret.mkCode(h).getOrElse(BAD_REQUEST), hpret.mkMsg(h), hpret.mkMore(h), "Megam::Error", hpret.severity), c => new FunnelResponse(hpret.mkCode(c).getOrElse(BAD_REQUEST), hpret.mkMsg(c), hpret.mkMore(c), "Megam::Error", hpret.severity), r => new FunnelResponse(hpret.mkCode(r).getOrElse(BAD_REQUEST), hpret.mkMsg(r), hpret.mkMore(r), "Megam::Error", hpret.severity), t => new FunnelResponse(hpret.mkCode(t).getOrElse(BAD_REQUEST), hpret.mkMsg(t), hpret.mkMore(t), "Megam::Error", hpret.severity)) }.some

Interface to RiaK

Scaliak library to Interface with Riak "com.stackmob" % "scaliak_2.10" % "0.8.0"

GSRiak - A Wrapper on top of Scaliak "com.github.indykish" % "megam_common_2.10" % "0.1.0-SNAPSHOT",

Code for megam_common :

https://github.com/indykish/megam_common.git

Interface to Riak

The model class which wishes to store stuff in Riak has :

GSRiak("http://localhost:6999/riak", "firstbucket")

Interface to RiaK

Every model provides its "bucketName".

The RIAK Base URL will be pulled from the play configuration.

Find All List of Nodes By Name

GET : /nodes

def findByNodeName(nodeNameList: Option[List[String]]): ValidationNel[Throwable, NodeResults] = { play.api.Logger.debug(("%-20s -->[%s]").format("models.Node", "findByNodeName:Entry")) play.api.Logger.debug(("%-20s -->[%s]").format("nodeNameList", nodeNameList)) (nodeNameList map { _.map { nodeName => play.api.Logger.debug(("%-20s -->[%s]").format("nodeName", nodeName)) (riak.fetch(nodeName) leftMap { t: NonEmptyList[Throwable] => new ServiceUnavailableError(nodeName, (t.list.map(m => m.getMessage)).mkString("\n")) }).toValidationNel.flatMap { xso: Option[GunnySack] => xso match { case Some(xs) => { //JsonScalaz.Error doesn't descend from java.lang.Error or Throwable. Screwy. (NodeResult.fromJson(xs.value) leftMap { t: NonEmptyList[net.liftweb.json.scalaz.JsonScalaz.Error] => JSONParsingError(t) }).toValidationNel.flatMap { j: NodeResult => play.api.Logger.debug(("%-20s -->[%s]").format("noderesult", j)) Validation.success[Throwable, NodeResults](nels(j.some)).toValidationNel //screwy kishore, every element in a list ? } } case None => { Validation.failure[Throwable, NodeResults](new ResourceItemNotFound(nodeName, "")).toValidationNel } } } } // -> VNel -> fold by using an accumulator or successNel of empty. +++ => VNel1 + VNel2 } map { _.foldRight((NodeResults.empty).successNel[Throwable])(_ +++ _) }).head //return the folded element in the head. }

Notice the below code

riak.fetch(nodeName) leftMap { t: NonEmptyList[Throwable] => new ServiceUnavailableError(nodeName, (t.list.map(m => m.getMessage)).mkString("\n")) }).toValidationNel.flatMap { xso: Option[GunnySack] =>

Which Fetches data from Riak.

private def riak: GSRiak = GSRiak(MConfig.riakurl, "nodes")

Where riak

What does GSRiak Do ?

Connect to the riak system using the scaliak client.

private lazy val client: ScaliakClient = Scaliak.httpClient(uri)

And

Fetch value(V) from Riak for a key(K)

Create the bucket using following syntax. client.bucket(bucketName)

fetch(key) function fetches value by riak.

private def fetchIO(key: String): IO[Validation[Throwable, Option[GunnySack]]] = { logger.debug("\\_/-->fetchIO:" + key)

bucketIO flatMap { mgBucket => //mgBucket is ValidationNel[Throwable, ScaliakBucket] mgBucket match { case Success(realMeat) => (realMeat.fetch(key) flatMap { x => x match { case Success(res) => Validation.success[Throwable, Option[GunnySack]](res).pure[IO] case Failure(err) => Validation.failure[Throwable, Option[GunnySack]](RiakError(err)).pure[IO] } }) case Failure(nahNoBucket) => Validation.failure[Throwable, Option[GunnySack]](RiakError(nels(BucketFetchError(uri, bucketName, key)))).pure[IO] } } }

//old code val fetchResult: ValidationNel[Throwable, Option[GunnySack]] = bucket.fetch(key).unsafePerformIO() def fetch(key: String) = fetchIO(key).unsafePerformIO.toValidationNel

FetchIO

fetchIO method which when interpreted will result in a fetch operation of a bucket using a key. The "key : String, value: Option[GunnySack] are the input and output.

Merely calling this method doesn't fetch results in a fetch operation. It just results in scalaz's IO[x].

Beta Launch of Megam Cloud (Polygot PaaS)

Our PaaS design => Link

Register http://www.megam.co for an invite

Twitter : @indykish

Screencast illustrating the Cloud API Servers working

Thank you

for watching