(SDD414) Amazon Redshift Deep Dive and What's Next | AWS re:Invent 2014

Post on 24-Jun-2015

1.642 views 1 download

Tags:

description

Get a look under the covers of Amazon Redshift, a fast, fully-managed, petabyte-scale data warehouse service for less than $1,000 per TB per year. Learn how Amazon Redshift uses columnar technology, optimized hardware, and massively parallel processing to deliver fast query performance on data sets ranging in size from hundreds of gigabytes to a petabyte or more. We'll also walk through techniques for optimizing performance. Finally, we'll announce new features that we've been working on over the past few months.

Transcript of (SDD414) Amazon Redshift Deep Dive and What's Next | AWS re:Invent 2014

November 12, 2014 | Las Vegas, NV

Anurag Gupta, General Manager

Rahul Pathak, Principal Product Manager

10 GigE

(HPC)

Ingestion

Backup

Restore

JDBC/ODBC

Leader Node

10 GigE

(HPC)

Ingestion

Backup

Restore

Customer VPC

Internal

VPC

JDBC/ODBC

CREATE FUNCTION f_hostname (VARCHAR url)

RETURNS varchar

IMMUTABLE AS $$

import urlparse

return urlparse.urlparse(url).hostname

$$ LANGUAGE plpythonu;

CREATE FUNCTION f_init_list_agg ()

RETURNS varchar

IMMUTABLE AS $$

return ''

$$ LANGUAGE plpythonu;

CREATE FUNCTION f_list_agg (state varchar, a varchar)

RETURNS varchar

IMMUTABLE AS $$

if state:

return state + ',' + a

return a

$$ LANGUAGE plpythonu;

CREATE FUNCTION f_fin_list_agg (a varchar)

RETURNS varchar

IMMUTABLE AS $$

if a:

return a

return ''

$$ LANGUAGE plpythonu;

CREATE AGGREGATE f_list_agg (varchar)

(

initfunc = f_init_list_agg,

aggfunc = f_list_agg,

finalizefunc = f_fin_list_agg

);

create table x ( a varchar(30), b int );

insert into x values ( 'a',1);

insert into x values ( 'b',1);

insert into x values ( 'c',1);

insert into x values ( 'd',2);

insert into x values ( 'e',2);

insert into x values ( 'f',2);

insert into x values ( 'g',3);

select b, f_list_agg(a) from x group by 1;

b | f_list_agg

---+------------

2 | d,f,e

1 | b,a,c

3 | g

Orders

Product

Time

Customer

Site

10 | 13 | 14 | 26 |…

… | 100 | 245 | 324

375 | 393 | 417…

… 512 | 549 | 623

637 | 712 | 809 …

… | 834 | 921 | 959

00 01 10 11

00

01

10

11

Customers

Products

00 01 10 11

00

01

10

11

Customers

Products

00 01 10 11

00

01

10

11

Customers

Products

[ SORTKEY [ COMPOUND | INTERLEAVED ] ( column_name [, ...] ) ]

http://bit.ly/awsevals