Beyond REST:Coursera’s API Evolution

Bryan Kane ! @bryanskane " bryan-coursera

Problems

• Number of courses grew rapidly

• Product needs changed: • premium course certificates • specializations • multi-university collaborations.

• Growing size of engineering team

Scale matters! In many dimensions

• Queries per second • Data set size • Engineers • Types in ontology

Transition to Microservices

Performance Requirements

• Minimize data transfer across the network • Minimize round trips

Nice-to-have’s • Caching • ETag & Conditional Request Support • Binary Serialization

Experience-based APIs

Queryable APIs

Relational Algebra MeetsREST/JSON APIs

Projection: filter fields in response

Selection: filter items in a collection

Joins: related resources

https://api.coursera.org/api/courses.v1?q=slug&slug=machine-learning

response:{ "elements": [ { "courseType": "v2.ondemand", "id": "Gtv4Xb1-EeS-ViIACwYKVQ", "name": "Machine Learning", "slug": "machine-learning" } ], "linked": {}, "paging": { "total": 1 }}

request endpoint:/api/courses.v1

request query parameters:{ "q": "slug", "slug": "machine-learning"}

https://api.coursera.org/api/courses.v1?q=slug&slug=machine-learning &fields=primaryLanguages

response:{ "elements": [ { "courseType": "v2.ondemand", "id": "Gtv4Xb1-EeS-ViIACwYKVQ", "name": "Machine Learning", "slug": "machine-learning", "primaryLanguages": ["en"] } ], "linked": {}, "paging": { "total": 1 }}

request endpoint:/api/courses.v1

request query parameters:{ "q": "slug", "slug": "machine-learning", "fields": "primaryLanguages"}

https://api.coursera.org/api/courses.v1?q=slug&slug=machine-learning &fields=instructorIds&includes=instructorIds

response:{ "elements": [ { "courseType": "v2.ondemand", "id": "Gtv4Xb1-EeS-ViIACwYKVQ", "name": "Machine Learning", "slug": "machine-learning", ”instructorIds": [1244] } ], "linked": { "instructors.v1": [ { "id": 1244, "name": "Andrew Ng" } ] }, "paging": { "total": 1 }}

request endpoint:/api/courses.v1

request query parameters:{ "q": "slug", "slug": "machine-learning", "fields": "instructorIds", "includes": "instructorIds"}

https://api.coursera.org/api/courses.v1?q=slug&slug=machine-learning &fields=instructorIds,instructors.v1(title)&includes=instructorIds

response:{ "elements": [ { "courseType": "v2.ondemand", "id": "Gtv4Xb1-EeS-ViIACwYKVQ", "name": "Machine Learning", "slug": "machine-learning", "instructorIds": [1244] } ], "linked": { "instructors.v1": [ { "id": 1244, "name": "Andrew Ng", "title": "Associate Professor, Stanford University; Chief Scientist, Baidu; Chairman and Co-founder, Coursera" } ] }, "paging": { "total": 1 }}

request endpoint:/api/courses.v1

request query parameters:{ "q": "slug", "slug": "machine-learning", "fields": “instructorIds, instructors.v1(title)", "includes": "instructorIds"}

0

200

400

600

800Controllers vs Naptime Resources

Controllers

Resources

Naptime Key Learnings

• Leverage type safety, carefully

• Collections of Key-Value pairs

• Opinionated framework improves communication

Principle:Optimize for developer productivity.

context.executeAction(setCourseIdentifiers, {courseId, courseSlug});

context.executeAction(loadMembership);

context.executeAction(getCurrentSession, {courseSlug});

context.executeAction(loadCourseSchedule, {courseId});

context.executeAction(initProfileStore, {authenticated, courseId, externalId, userId});

context.executeAction(loadCourseViewGrade, {userId, courseId});

context.executeAction(loadDomains);

context.executeAction(loadUserGroupsForCourse, {courseId, userId});

context.executeAction(getReferencesList, {courseId});

context.executeAction(loadHonorsUserPreferences, {authenticated});

context.executeAction(loadCertificateData, {courseId, userId});

context.executeAction(loadVerificationDisplay, {authenticated, userId, course, s12n});

context.executeAction(getHomeProgress, {authenticated, courseId});

context.executeAction(getProgress, {authenticated, courseId, userId});

Imperative Data Fetching Declarative Data Fetching

api.loadCourses().then(courses => { const courseIds = courses.map(c => c.id); api.loadCourseMemberships(courseIds) .then(courseMemberships => { this.setState({ courses, courseMemberships }); }); });

render() { if (this.state.courses && this.state.courseMemberships) { return <div>{courses}</div>; } }

const Component = ({courses, memberships}) => { return <div>{courses}</div> }

Naptime.withData(Component, ({courseId}) => { courses: CoursesV1.getAll(), memberships: MembershipV1.byCourse(courseId) });

NaptimeStore: { "courses.v1": { "69Bku0KoEeWZtA4u62x6lQ": {…}, "0HiU7Oe4EeWTAQ4yevf_oQ": {…}, "5zjIsJq-EeW_wArffOXkOw": {…} }, "instructors.v1": { "v9CQdBkhEeWjrA6seF25aw": {…}, "QgmoVdT2EeSlhSIACx2EBw": {…} } }

Global ID:"courses.v1/69Bku0KoEeWZtA4u62x6lQ"

Problems remained

• There are always going to be edge cases • Feature prioritization requires careful balance • Don’t underestimate the power of Googleability

GraphQL Query:query { CoursesV1Resource { slug(slug: "machine-learning") { id } } }

request endpoint:/api/courses.v1

request query parameters:{ "q": "slug", "slug": "machine-learning"}

request endpoint:/api/courses.v1

request query parameters:{ "q": "slug", "slug": "machine-learning", "fields": "primaryLanguages"}

GraphQL Query:query { CoursesV1Resource { slug(slug: "machine-learning") { id primaryLanguages } } }

request endpoint:/api/courses.v1

request query parameters:{ "q": "slug", "slug": "machine-learning", "fields": "instructorIds", "includes": "instructorIds"}

GraphQL Query:query { CoursesV1Resource { slug(slug: "machine-learning") { id instructors: instructorIds { id } } } }

request endpoint:/api/courses.v1

request query parameters:{ "q": "slug", "slug": "machine-learning", "fields": "instructorIds, instructors.v1(title)", "includes": "instructorIds"}

GraphQL Query:query { CoursesV1Resource { slug(slug: "machine-learning") { id instructors: instructorIds { id title } } } }

Edge Tier

GraphQLAssembler

ProfileService

EnrollmentsService

CatalogService

Edge Tier

GraphQLAssembler

ProfileService

EnrollmentsService

CatalogService

Key Lessons LearnedCarefully understand your problem

Look to theory: stand on the shoulders of giants

Plug into the ecosystem

Naptime is developed in the openhttps://github.com/coursera/naptime

Thank you!

Thank you!

Bryan Kane ! @bryanskane " bryan-coursera