API Development and Scala @ SoundCloud
-
Upload
bora-tunca -
Category
Software
-
view
781 -
download
3
description
Transcript of API Development and Scala @ SoundCloud
![Page 1: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/1.jpg)
Product Engineering
Bora
![Page 2: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/2.jpg)
SoundCloud Public API
● for external developers● no built-in assumptions
![Page 3: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/3.jpg)
![Page 4: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/4.jpg)
![Page 5: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/5.jpg)
![Page 6: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/6.jpg)
Growing
● api team became a bottleneck● undocumented endpoints
![Page 7: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/7.jpg)
![Page 8: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/8.jpg)
![Page 9: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/9.jpg)
Client Specific APIs
![Page 10: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/10.jpg)
Reduce Chattiness
![Page 11: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/11.jpg)
![Page 12: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/12.jpg)
soundcloud.com/explore/explore/categories
API
/explore/{category}
/tracks?ids={ids}
/visuals/{ids}
![Page 13: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/13.jpg)
soundcloud.com/explore/explore
API
![Page 14: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/14.jpg)
![Page 15: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/15.jpg)
Remove Dependencies
![Page 16: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/16.jpg)
SC Microservices
Public API
![Page 17: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/17.jpg)
SC Microservices
API-MobileAPI-V2 API-Embedded API-Partners API-*
![Page 18: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/18.jpg)
![Page 19: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/19.jpg)
![Page 20: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/20.jpg)
● Dealing with the Monolith● Breaking the Monolith● Microservices in Scala and Finagle
Building Products at SoundCloud
![Page 21: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/21.jpg)
![Page 22: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/22.jpg)
Java Interoperability
![Page 23: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/23.jpg)
SC Microservices
API-MobileAPI-V2 API-Embedded API-Partners API-*
BFF
JVM-KIT
![Page 24: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/24.jpg)
Futures
![Page 25: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/25.jpg)
SC Microservices
API-MobileAPI-V2 API-Embedded API-Partners API-*
![Page 26: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/26.jpg)
def userTracks(userUrn: Urn): Future[List[Track]] = {
for {
trackUrns <- userTrackRepo.trackUrns(userUrn)
tracks <- trackRepo.tracks(trackUrns)
} yield tracks
}
![Page 27: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/27.jpg)
val resources: List[Future[List[Resource]]] = List(
trackRepo.tracks(trackUrns),
userRepo.users(userUrns),
playlistRepo.playlists(playlistUrns),
groupRepo.groups(groupUrns))
Future.collect(resources).map {
case List(tracks, users, playlists, groups) => ...
}
![Page 28: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/28.jpg)
Traits
![Page 29: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/29.jpg)
abstract class JsonMapping(val trackJson: JsValue)
trait MiniTrack extends JsonMapping {
val urn = Urn((trackJson \ "self" \ "urn").as[String])
val title = (trackJson \ "title").as[String]
}
trait Stats extends JsonMapping {
val play_count = (trackJson \ "play_count").as[Int]
val like_count = (trackJson \ "like_count").as[Int]
}
trait Artwork extends JsonMapping {
val artwork_url = (trackJson \ "artwork_url").as[String]
}
![Page 30: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/30.jpg)
new JsonMapping(trackJson)
with MiniTrack
with Stats
with Artwork
...
new JsonMapping(trackJson)
with MiniTrack
with Stats
...
new JsonMapping(trackJson)
with MiniTrack
with Scheduling
![Page 31: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/31.jpg)
Higher-order Functions
![Page 32: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/32.jpg)
trait TracksController extends BaseController {
get("/tracks/:urn") {
request =>
val trackUrn = Urn(request.routeParams("urn"))
trackRepo.track(trackUrn).map(render)
}
}
...
def get(path: String)(callback: RequestHandler) =
register(path, HttpMethod.GET, callback)
type RequestHandler = Request => Future[Response]
![Page 33: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/33.jpg)
Collections
![Page 34: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/34.jpg)
// Playlist has a tracks : List[Tracks]
val playlists: List[Playlist]
val emptyPlaylists =
playlists.filter(_.tracks.isEmpty)
val allTracks =
playlists.map(_.tracks).flatten
val tracksByCreator: Map[User, List[Track]] =
allTracks.groupBy(_.creator)
val trackCount =
playlists.map(_.tracks.size).sum
![Page 35: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/35.jpg)
Case Classes & Pattern Matching
![Page 36: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/36.jpg)
case class JsonResponse(
status: StatusCode,
body: JsValue,
headers: HttpHeaders = HttpHeaders.EMPTY_HEADERS,
pagination: Pagination = Pagination.empty
)
![Page 37: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/37.jpg)
def tracks(urns: List[Urn]) = {
fetch(Path() / "tracks", urns).map {
case JsonResponse(OkStatus, body, _, _) => body
case JsonResponse(NotFoundStatus, _, _, _) => ...
case JsonResponse(UnauthorizedStatus, _, _, _)
=> ...
case _ => throw ...
}
}
![Page 38: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/38.jpg)
class Urn implements Serializable, Comparable<Urn>//JAVA
Interop
object Urn {
...
def unapply(obj: Urn): Option[(String, String, String)]
=
Some(obj.getNamespace, obj.getCollection,
obj.getIdentifier)
}
urn match {
case Urn(_, "tracks", _) => ...
case Urn(_, "playlists", _) => …
case Urn(_, "comments", _) => ...
}
![Page 39: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/39.jpg)
![Page 40: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/40.jpg)
Implicits
![Page 41: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/41.jpg)
def tracks(urns: List[Urn]) =
fetch(Path() / "tracks", urns)
...
def fetch(path: Path, params: Params)
// defined in the package object
implicit def urnsToParams(urns: Iterable[Urn]): Params =
Params("urns" -> urns)
…
// we can still use make this
fetch(Path() / "tracks", Params("ids" -> ids))
![Page 42: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/42.jpg)
![Page 43: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/43.jpg)
XML!!!
![Page 44: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/44.jpg)
trait AssignmentsController extends V2Controller with ExceptionHandler {class TracksXmlVisitor(val wrapped: Node)
extends TracksVisitor {
type TrackType = XmlTrack
def apply(visit: VisitTrack): Option[Node] =
apply(wrapped, visit)
private def apply(node: Node, visit: VisitTrack): Option[Node] = {
node match {
case node: Node if (isTrack(node)) =>
visitTrack(node, visit)
case node: Elem =>
val att = node.attributes
val children = node.child.map {
case text: Text =>
Some(text)
case other =>
apply(other, visit)
}.flatten
Some(new Elem(node.prefix, node.label, node.attributes, node.scope, true, children: _*))
case other =>
Some(other)
}
}
private def visitTrack(node: Node, visit: VisitTrack) = {
val id = (node \ "id").text.toInt
val urn = Urn(s"soundcloud:tracks:$id")
visit(urn, XmlTrack(node))
}
private def isTrack(node: Node) =
(node \ "kind").text == "track"
}
![Page 45: API Development and Scala @ SoundCloud](https://reader035.fdocuments.in/reader035/viewer/2022081413/54626952af7959aa3d8b68ce/html5/thumbnails/45.jpg)