FUTURESTACK13: Software analytics with Project Rubicon from Alex Kroman Engineering Manager at New...

Software Analytics with Project Rubicon

ALEX KROMANOCTOBER 25TH, 2013

Wednesday, November 6, 13

1. Learn how to write NRQL queries

2. Aggregating billions of events

3. Building features on top of Rubicon

Today’s agenda


New Relic captures performance data from a bunch of places


But performance data is a small part of application data

Clicks Log messages API callsTransactions


Rubicon captures the raw events from your applications

Python AppiOS appRuby

App

Rubicon


How many customers logged in today?

What is my average revenue per sale?

What is my most used feature?

How many people are live on the app now?

By capturing raw events you can ask your application complex questions


An introduction to NRQL


1. How many people used the site today?

2. How many people are using a new feature?

3. How many people are using a new feature over time?

4. What kind of customers are using this feature?

Four questions you can ask Rubicon


1. How many people used the site today?

SELECT uniqueCount(user) FROM PageView SINCE 1 day ago


Time windows in NRQL

SINCE 8 hours ago

SINCE 2 months ago

SINCE 1 week ago UNTIL 2 hours ago


2. How many people are using a new feature?

SELECT uniqueCount(user) FROM PageView WHERE name LIKE '%x_rays%' SINCE 1 month ago


3. How many people are using the feature over time?SELECT uniqueCount(user) FROM PageView WHERE name LIKE ‘%x_rays%’ SINCE 7 days ago TIMESERIES day


4. What kind of customers are using this feature?

SELECT uniqueCount(user) FROM PageView WHERE name LIKE '%x_rays%' SINCE 7 days ago FACET apm_product


•Plenty of aggregation functions

•Time windows

•Time series

•Facets

NRQL is designed to make it easy to aggregate data over time


How do we aggregate a billion events?


Build a wide cluster

Memory is cheap so use it

Build for failure

Send the cluster real traffic

Four things we’ve learned handling large amounts of data


A brief tour of the architecture...

Router

Queries

Load balancer

RouterRouter

Worker A Worker B Worker C

Metadata

Backup Storage


Building features on top of Rubicon


Making a histogram chart

Rubicon.nrql(“SELECT histogram(duration) FROM PageView SINCE 7 days ago”)


JSON response{ "results": [ { "histogram": [ 1618, 4748, 3736, 4526,etc...

], "bucketSize": 0.25 }


Making a percentile chart

Rubicon.nrql(“SELECT percentile(duration, 50, 95, 99) FROM PageView SINCE 7 days ago TIMESERIES hour”)


JSON response{ "timeSeries": [ { "results": [ { "percentiles": { "50": 1.66, "95": 5.445, "99": 11.942 } } ], "beginTimeSeconds": 1382050194, "endTimeSeconds": 1382053794 },


Adding a custom event

data = {‘salesTeam’: ‘Enterprise’, ‘amount’: 100000}

Rubicon.insert(‘RevenueChange’, data)


Querying a custom event

SELECT count(*) FROM RevenueChange FACET salesTeam SINCE 7 days ago


Learned a bit of NRQL

Overview of the architecture

Explored the Rubicon API’s

Recap


Come visit us in the hacker lounge!


Find out more!

newrelic.com/analytics


FUTURESTACK13: Software analytics with Project Rubicon from Alex Kroman Engineering Manager at New...

Technology

Transcript of FUTURESTACK13: Software analytics with Project Rubicon from Alex Kroman Engineering Manager at New...