Feed Normalization with Ember Data 1.0

53
Normalizing with Ember Data 1.0b Jeremy Gillick

description

So you're working with a web service that doesn't play nice with Ember Data, that's okay! Using Ember Data 1.0.0-beta we will normalize ugly JSON feeds into something that Ember understands and loves.

Transcript of Feed Normalization with Ember Data 1.0

Page 1: Feed Normalization with Ember Data 1.0

Normalizing with Ember Data 1.0b

Jeremy Gillick

Page 2: Feed Normalization with Ember Data 1.0

or

Page 3: Feed Normalization with Ember Data 1.0

True Facts of Using Data in Ember

Page 4: Feed Normalization with Ember Data 1.0

I’m Jeremy

http://mozmonkey.com

https://github.com/jgillick/

https://linkedin.com/in/jgillick

Page 5: Feed Normalization with Ember Data 1.0

I work at Nest

Page 6: Feed Normalization with Ember Data 1.0

We love Emberdepending on the day

Page 7: Feed Normalization with Ember Data 1.0

Ember Data is GreatExcept when data feeds don’t conform

Page 8: Feed Normalization with Ember Data 1.0

Serializers connect Raw Data to Ember Data

{ … }

JSONSerializer

Ember Data

Page 9: Feed Normalization with Ember Data 1.0

Let’s talk about data

Page 10: Feed Normalization with Ember Data 1.0

Ember prefers side loading to nested JSON

But why?

Page 11: Feed Normalization with Ember Data 1.0

For example{! "posts": [! {! "id": 5,! "title":You won't believe what was hiding in this kid's locker",! "body": "...",! "author": {! "name": "Jeremy Gillick",! "role": "Author",! "email": "[email protected]"! }! }! ]!}

Page 12: Feed Normalization with Ember Data 1.0

{! "posts": [! {! "id": 6,! "title": "New Study: Apricots May Help Cure Glaucoma",! "body": "...",! "author": {! "name": "Jeremy Gillick",! "role": "Author",! "email": "[email protected]"! }! },! {! "id": 5,! "title": "You won't believe what was hiding in this kid's locker",! "body": "...",! "author": {! "name": "Jeremy Gillick",! "role": "Author",! "email": "[email protected]"! }! }! ]!}

For example

Redundant, adds feed bloat and which one is the source of truth?

Page 13: Feed Normalization with Ember Data 1.0

This is better{! "posts": [! {! "id": 4,! "title": "New Study: Apricots May Help Cure Glaucoma",! "body": "...",! "author": 42! },! {! "id": 5,! "title": "You won't believe what was hiding in this kid's locker",! "body": "...",! "author": 42! }! ],! "users": [! {! "id": 42,! "name": "Jeremy Gillick",! "role": "Author",! "email": "[email protected]"! }! ]!}

Page 14: Feed Normalization with Ember Data 1.0

Ember Data Expects{! "modelOneRecord": {! ...! }! "modelTwoRecords": [! { ... },! { ... }! ],! "modelThreeRecords": [! { ... },! { ... }! ]!}

No further nesting is allowed

Page 15: Feed Normalization with Ember Data 1.0

Ember Data Expects

{! "posts": [! ...! ],!! "users": [! …! ]!}

App.Post records

App.User records

Page 16: Feed Normalization with Ember Data 1.0

Not all JSON APIs will be flat

Page 17: Feed Normalization with Ember Data 1.0

A nested world{! "products": [! {! "name": "Robot",! "description": "A robot may not injure a human being or...",! "price": {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", "black", "#E1563F"]! }! ]! }! ]!}

Page 18: Feed Normalization with Ember Data 1.0

Ember Data can’t process that

Page 19: Feed Normalization with Ember Data 1.0

{! "products": [! {! "name": "Robot",! "description": "...",! "price": {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ]!}

{! "products": [! {! "id": "product-1",! "name": "Robot",! "description": “...”,! "price": "price-1",! "size": "dimension-1",! "options": [! “options-1”! ]! }! ],! "prices": [! {! "id": "price-1",! "value": 59.99,! "currency": "USD"! } ! ]! "dimensions": [ … ],! "options": [ … ]!}!!

Flatten that feed

Page 20: Feed Normalization with Ember Data 1.0

How do we do this?With a custom Ember Data Serializer!

Page 21: Feed Normalization with Ember Data 1.0

Two common ways• Create ProductSerializer that manually converts the

JSON

• A lot of very specific code that you’ll have to repeat for all nested JSON payloads.

• Build a generic serializer that automatically flattens nested JSON objects

• Good, generic, DRY

Page 22: Feed Normalization with Ember Data 1.0

Defining the model{! "products": [! {! "name": "Robot",! "description": "...",! "price": {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ]!}

App.Product = DS.Model.extend({! name: DS.attr('string'),! description: DS.attr('string'),! price: DS.belongsTo('Price'),! size: DS.belongsTo('Dimension'),! options: DS.hasMany('Option')!});!!App.Price = DS.Model.extend({! value: DS.attr('number'),! currency: DS.attr('string')!});!!App.Dimension = DS.Model.extend({! height: DS.attr('number'),! width: DS.attr('number'),! depth: DS.attr('number')!});!!App.Option = DS.Model.extend({! name: DS.attr('string'),! values: DS.attr()!});

Page 23: Feed Normalization with Ember Data 1.0

Steps

• Loop through all root JSON properties

• Determine which model they represent

• Get all the relationships for that model

• Side load any of those relationships

Page 24: Feed Normalization with Ember Data 1.0

{! "products": [! {! "name": "Robot",! "description": "...",! "price": {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ]!}

App.Product

Relationships• price • size • option

Side load

$$$ Profit $$$

Page 25: Feed Normalization with Ember Data 1.0

JS Methodsextract: function(store, type, payload, id, requestType) { ... }

processRelationships: function(store, type, payload, hash) { ... }

sideloadRecord: function(store, type, payload, hash) { ... }

Page 26: Feed Normalization with Ember Data 1.0

Create a Serializer/**! Deserialize a nested JSON payload into a flat object! with sideloaded relationships that Ember Data can import.!*/!App.NestedSerializer = DS.RESTSerializer.extend({!! /**! (overloaded method)! Deserialize a JSON payload from the server.!! @method normalizePayload! @param {Object} payload! @return {Object} the normalized payload! */! extract: function(store, type, payload, id, requestType) {! return this._super(store, type, payload, id, requestType);! }!!});

Page 27: Feed Normalization with Ember Data 1.0

{! "products": [! {! ...! }! ]!}

extract: function(store, type, payload, id, requestType) {! return this._super(store, type, payload, id, requestType);!}

Page 28: Feed Normalization with Ember Data 1.0

{! "products": [! {! ...! }! ]!}

extract: function(store, type, payload, id, requestType) {! var rootKeys = Ember.keys(payload);!! // Loop through root properties and process their relationships! rootKeys.forEach(function(key){! ! }, this);!! return this._super(store, type, payload, id, requestType);!}

Page 29: Feed Normalization with Ember Data 1.0

extract: function(store, type, payload, id, requestType) {! var rootKeys = Ember.keys(payload);!! // Loop through root properties and process their relationships! rootKeys.forEach(function(key){! var type = store.container!! ! ! ! .lookupFactory('model:' + key.singularize());!! }, this);!! return this._super(store, type, payload, id, requestType);!}

{! "products": [! {! ...! }! ]!}

Page 30: Feed Normalization with Ember Data 1.0

{! "products": [! {! ...! }! ]!}

product

Singularize

container.lookup(‘model:product’)

App.Product

"products"

Page 31: Feed Normalization with Ember Data 1.0

extract: function(store, type, payload, id, requestType) {! var rootKeys = Ember.keys(payload);!! // Loop through root properties and process their relationships! rootKeys.forEach(function(key){! var type = store.container!! ! ! ! .lookupFactory('model:' + key.singularize());!! }, this);!! return this._super(store, type, payload, id, requestType);!}

{! "products": [! {! ...! }! ]!}

Page 32: Feed Normalization with Ember Data 1.0

{! "products": ! [! {! ...! }! ]!}

extract: function(store, type, payload, id, requestType) {! var rootKeys = Ember.keys(payload);!! // Loop through root properties and process their relationships! rootKeys.forEach(function(key){! var type = store.container! .lookupFactory('model:' + key.singularize()),! hash = payload[key];!! }, this);!! return this._super(store, type, payload, id, requestType);!}

Page 33: Feed Normalization with Ember Data 1.0

extract: function(store, type, payload, id, requestType) {! var rootKeys = Ember.keys(payload);!! // Loop through root properties and process their relationships! rootKeys.forEach(function(key){! var type = store.container! .lookupFactory('model:' + key.singularize()),! hash = payload[key];!! // Sideload embedded relationships of this model hash! if (type) {! this.processRelationships(store, type, payload, hash);! }! }, this);!! return this._super(store, type, payload, id, requestType);!}

{! "products": ! [! {! ...! }! ]!}

Page 34: Feed Normalization with Ember Data 1.0

/**! Process nested relationships on a single hash record!! @method extractRelationships! @param {DS.Store} store! @param {DS.Model} type! @param {Object} payload The entire payload! @param {Object} hash The hash for the record being processed! @return {Object} The updated hash object!*/!processRelationships: function(store, type, payload, hash) {!!},

Page 35: Feed Normalization with Ember Data 1.0

{! "products": [! {! ...! }! ]!}

processRelationships: function(store, type, payload, hash) {!! // If hash is an array, process each item in the array! if (hash instanceof Array) {! hash.forEach(function(item, i){! hash[i] = this.processRelationships(store, type, payload, item);! }, this);! }!! return hash;!},

Page 36: Feed Normalization with Ember Data 1.0

{! "products": [! {! ...! }! ]!}

processRelationships: function(store, type, payload, hash) {!! // If hash is an array, process each item in the array! if (hash instanceof Array) {! hash.forEach(function(item, i){! hash[i] = this.processRelationships(store, type, payload, item);! }, this);! }!! else {!! }!! return hash;!},

Page 37: Feed Normalization with Ember Data 1.0

{! "products": [! {! ...! }! ]!}

processRelationships: function(store, type, payload, hash) {!! // If hash is an array, process each item in the array! if (hash instanceof Array) {! hash.forEach(function(item, i){! hash[i] = this.processRelationships(store, type, payload, item);! }, this);! }!! else {!! // Find all relationships in this model! type.eachRelationship(function(key, relationship) {! ! }, this);! }!! return hash;!},

Page 38: Feed Normalization with Ember Data 1.0

!App.Product.eachRelationship(function(key, relationship) {! !!}, this);!

App.Product = DS.Model.extend({! name: DS.attr('string'),! description: DS.attr('string'),! price: DS.belongsTo('Price'),! size: DS.belongsTo('Dimension'),! options: DS.hasMany('Option')!});

key = 'price'! relationship = {! "type": App.Price,! "kind": "belongsTo",! ...! }

key = 'size'! relationship = {! "type": App.Dimension,! "kind": "belongsTo",! ...! }

key = 'options'! relationship = {! "type": App.Option,! "kind": "hasMany",! ...! }

Page 39: Feed Normalization with Ember Data 1.0

{! "products": [! {! "name": "Robot",! "description": "...",! "price": ! {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ]!}

processRelationships: function(store, type, payload, hash) {!! // If hash is an array, process each item in the array! if (hash instanceof Array) {! hash.forEach(function(item, i){! hash[i] = this.processRelationships(store, type, payload, item);! }, this);! }!! else {!! // Find all relationships in this model! type.eachRelationship(function(key, relationship) {! var related = hash[key]; // The hash for this relationship! ! }, this);! }!! return hash;!},

Page 40: Feed Normalization with Ember Data 1.0

processRelationships: function(store, type, payload, hash) {!! // If hash is an array, process each item in the array! if (hash instanceof Array) {! hash.forEach(function(item, i){! hash[i] = this.processRelationships(store, type, payload, item);! }, this);! }!! else {!! // Find all relationships in this model! type.eachRelationship(function(key, relationship) {! var related = hash[key], // The hash for this relationship! relType = relationship.type; // The model for this relationship!! }, this);! }!! return hash;!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": ! {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ]!}

App.Price

Page 41: Feed Normalization with Ember Data 1.0

processRelationships: function(store, type, payload, hash) {!! // If hash is an array, process each item in the array! if (hash instanceof Array) {! hash.forEach(function(item, i){! hash[i] = this.processRelationships(store, type, payload, item);! }, this);! }!! else {!! // Find all relationships in this model! type.eachRelationship(function(key, relationship) {! var related = hash[key], ! relType = relationship.type;!! hash[key] = this.sideloadRecord(store, relType, payload, related);! ! }, this);! }!! return hash;!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": ! {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ]!}

Page 42: Feed Normalization with Ember Data 1.0

/**! Sideload a record hash to the payload!! @method sideloadRecord! @param {DS.Store} store! @param {DS.Model} type! @param {Object} payload The entire payload! @param {Object} hash The record hash object! @return {Object} The ID of the record(s) sideloaded!*/!sideloadRecord: function(store, type, payload, hash) {! !},

Page 43: Feed Normalization with Ember Data 1.0

sideloadRecord: function(store, type, payload, hash) {! var id, sideLoadkey, sideloadArr, serializer;!! // If hash is an array, sideload each item in the array! if (hash instanceof Array) {! id = [];! hash.forEach(function(item, i){! id[i] = this.sideloadRecord(store, type, payload, item);! }, this);! }! ! return id;!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": ! {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ]!}

Page 44: Feed Normalization with Ember Data 1.0

sideloadRecord: function(store, type, payload, hash) {! var id, sideLoadkey, sideloadArr, serializer;!! // If hash is an array, sideload each item in the array! if (hash instanceof Array) {! id = [];! hash.forEach(function(item, i){! id[i] = this.sideloadRecord(store, type, payload, item);! }, this);! }! // Sideload record! else if (typeof hash === 'object') {! ! }! return id;!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": ! {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ]!}

Page 45: Feed Normalization with Ember Data 1.0

sideloadRecord: function(store, type, payload, hash) {! var id, sideLoadkey, sideloadArr, serializer;!! // If hash is an array, sideload each item in the array! if (hash instanceof Array) {! id = [];! hash.forEach(function(item, i){! id[i] = this.sideloadRecord(store, type, payload, item);! }, this);! }! // Sideload record! else if (typeof hash === 'object') {! sideLoadkey = type.typeKey.pluralize(); ! sideloadArr = payload[sideLoadkey] || [];! }!! return id;!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": ! {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ],! "prices": [! ]!}

Page 46: Feed Normalization with Ember Data 1.0

sideloadRecord: function(store, type, payload, hash) {! var id, sideLoadkey, sideloadArr, serializer;!! // If hash is an array, sideload each item in the array! if (hash instanceof Array) {! id = [];! hash.forEach(function(item, i){! id[i] = this.sideloadRecord(store, type, payload, item);! }, this);! }! // Sideload record! else if (typeof hash === 'object') {! sideLoadkey = type.typeKey.pluralize(); ! sideloadArr = payload[sideLoadkey] || [];! id = this.generateID(store, type, hash);! }!! return id;!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": ! {! "id": “generated-1",! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ],! "prices": [! ]!}

Every record needs an ID

Page 47: Feed Normalization with Ember Data 1.0

sideloadRecord: function(store, type, payload, hash) {! var id, sideLoadkey, sideloadArr, serializer;!! // If hash is an array, sideload each item in the array! if (hash instanceof Array) {! id = [];! hash.forEach(function(item, i){! id[i] = this.sideloadRecord(store, type, payload, item);! }, this);! }! // Sideload record! else if (typeof hash === 'object') {! sideLoadkey = type.typeKey.pluralize(); ! sideloadArr = payload[sideLoadkey] || [];! ! // Sideload, if it's not already sideloaded! if (sideloadArr.findBy('id', id) === undefined){! sideloadArr.push(hash);! payload[sideLoadkey] = sideloadArr;! }! }!! return id;!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": ! {! "id": “generated-1",! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ],! "prices": [! {! "id": “generated-1",! "value": 59.99,! "currency": "USD"! }! ]!}

Page 48: Feed Normalization with Ember Data 1.0

sideloadRecord: function(store, type, payload, hash) {! var id, sideLoadkey, sideloadArr, serializer;!! // If hash is an array, sideload each item in the array! if (hash instanceof Array) {! id = [];! hash.forEach(function(item, i){! id[i] = this.sideloadRecord(store, type, payload, item);! }, this);! }! // Sideload record! else if (typeof hash === 'object') {! sideLoadkey = type.typeKey.pluralize(); ! sideloadArr = payload[sideLoadkey] || [];! ! // Sideload, if it's not already sideloaded! if (sideloadArr.findBy('id', id) === undefined){! sideloadArr.push(hash);! payload[sideLoadkey] = sideloadArr;! }! }!! return id;!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": "generated-1",! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ],! "prices": [! {! "id": "generated-1",! "value": 59.99,! "currency": "USD"! }! ]!}

processRelationships: function(store, type, payload, hash) {! ...! hash[key] = this.sideloadRecord(store, relType, payload, related);! ...!},

Page 49: Feed Normalization with Ember Data 1.0

{! "products": [! {! "name": "Robot",! "description": "...",! "price": "generated-1",! "size": "generated-2",! "options": [! “generated-3”! ]! }! ],! "prices": [! {! "id": "generated-1",! "value": 59.99,! "currency": "USD"! }! ],! "dimensions": [{! "id": "generated-2",! "height": 24,! "width": 12,! "depth": 14! }],! "options": [ ! {! "id": "generated-3",! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]!}

{! "products": [! {! "name": "Robot",! "description": "...",! "price": "generated-1",! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ],! "prices": [! {! "id": "generated-1",! "value": 59.99,! "currency": "USD"! }! ]!}

Page 50: Feed Normalization with Ember Data 1.0

Apply the Serializer

App.ApplicationSerializer = App.NestedSerializer;

App.ProductSerializer = App.NestedSerializer.extend({});

- OR -

Page 51: Feed Normalization with Ember Data 1.0

Now for a demo

Page 52: Feed Normalization with Ember Data 1.0

http://emberjs.jsbin.com/neriyi/edit

Page 53: Feed Normalization with Ember Data 1.0

http://emberjs.jsbin.com/neriyi/edit

Questions?

http://www.slideshare.net/JeremyGillick/normalizing-data