JAWS-UG Osaka Special re:Invent 2013
-
Upload
toshiyuki-konparu -
Category
Technology
-
view
928 -
download
0
description
Transcript of JAWS-UG Osaka Special re:Invent 2013
![Page 1: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/1.jpg)
re:Invent報告2013.11.29
金春 利幸(R3 institute)
![Page 2: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/2.jpg)
Introduction
金春 利幸(こんぱる としゆき) アールスリーインスティテュート マネージャー&ソリューションアーキテクト FB: t.konparu twitter: t_konparu
2
![Page 3: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/3.jpg)
re:Invent 初参加してきました
![Page 4: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/4.jpg)
今回はそのご報告です
![Page 5: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/5.jpg)
AAの機内食はエサ感満載
![Page 6: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/6.jpg)
AAの機内食はエサ感満載
THE 炭水化物
![Page 7: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/7.jpg)
![Page 8: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/8.jpg)
![Page 9: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/9.jpg)
![Page 10: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/10.jpg)
以上、報告でした
![Page 11: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/11.jpg)
システム開発会社の 人間としての感想は
![Page 12: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/12.jpg)
きた!
![Page 13: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/13.jpg)
AWSはもはや 「インフラ」ではない
![Page 14: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/14.jpg)
いい色の絵の具をみたら絵を描きたくなるように 新しい音色を聞いたら曲のイメージが湧くように 新しいスニーカーを買ったら走りたくなるように
今回の新サービスで 想像力をかきたてましょう
![Page 15: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/15.jpg)
Amazon Kinesis
![Page 16: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/16.jpg)
記事とか読んだけど ピンとこない人?
![Page 17: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/17.jpg)
Amazon Kinesisは リアルタイム分析 プラットフォーム
です
![Page 18: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/18.jpg)
は?
![Page 19: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/19.jpg)
想像してください
![Page 20: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/20.jpg)
あなたはTwitter関連サービスの開発者です
![Page 21: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/21.jpg)
ある日上司に言われました !
「ハッシュタグの登場回数カウントして!」
![Page 22: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/22.jpg)
「あっ、バッチじゃダメだよリアルタイムね」
![Page 23: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/23.jpg)
は?(゜_゜)
![Page 24: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/24.jpg)
膨大なデータくるよな・・・ ちゃんと受け取るには・・・
ハッシュタグで分類するには・・・ カウントの負荷読めないよな・・・
![Page 25: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/25.jpg)
そんなあなたに Amazon Kinesis
![Page 26: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/26.jpg)
リアルタイムに 不規則に
発生するデータを 漏らすことなく
スマートに処理するための仕組み
![Page 27: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/27.jpg)
Amazon Kinesis
データ 発生元
Shard
Shard
Shard
Shard
App
App
App
App
S3 RDS
DynamoDB RedShift Kinesis
![Page 28: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/28.jpg)
![Page 29: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/29.jpg)
#jawsug っていいよね
#jawsug すごいよね
#jawsug 大阪熱いよね
#awsサンタ C3インスタンスにEBS Optimized
#awsサンタ Workspacesにもっとスペックを
#iegg 2/15はクロスエッグだよね
![Page 30: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/30.jpg)
![Page 31: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/31.jpg)
Amazon Kinesis
Shard
Shard
![Page 32: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/32.jpg)
Amazon Kinesis
Shard
Shard
#jawsug っていいよね
#jawsug すごいよね
#jawsug 大阪熱いよね
#awsサンタ C3インスタンスにEBS Optimized
#awsサンタ Workspacesにもっとスペックを
#iegg 2/15はクロスエッグだよね
![Page 33: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/33.jpg)
Amazon Kinesis
Shard
Shard
#jawsug っていいよね
#jawsug すごいよね
#jawsug 大阪熱いよね
#awsサンタ C3インスタンスにEBS Optimized
#awsサンタ Workspacesにもっとスペックを
#iegg 2/15はクロスエッグだよね
![Page 34: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/34.jpg)
GetNextRecord
Kinesis App
Kinesis App
Amazon Kinesis
Shard
Shard
#jawsug っていいよね
#jawsug すごいよね
#jawsug 大阪熱いよね
#awsサンタ C3インスタンスにEBS Optimized
#awsサンタ Workspacesにもっとスペックを
#iegg 2/15はクロスエッグだよね
Partion Key
12
3
4
56
GetNextRecord
![Page 35: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/35.jpg)
GetNextRecord
Kinesis App
Kinesis App
Amazon Kinesis
Shard
Shard
#jawsug っていいよね#jawsug すごいよね
#jawsug 大阪熱いよね
#awsサンタ C3インスタンスにEBS Optimized
#awsサンタ Workspacesにもっとスペックを
#iegg 2/15はクロスエッグだよね
Partion Key
12
3
4
56
GetNextRecord
![Page 36: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/36.jpg)
GetNextRecord
Kinesis App
Kinesis App
Amazon Kinesis
Shard
Shard
#jawsug っていいよね#jawsug すごいよね
#jawsug 大阪熱いよね
#awsサンタ C3インスタンスにEBS Optimized#awsサンタ Workspacesにもっとスペックを
#iegg 2/15はクロスエッグだよね
Partion Key
12
3
4
56
GetNextRecord
![Page 37: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/37.jpg)
DynamoDB
Kinesis App
Kinesis App
#jawsug: 3
#awsサンタ: 2
#iegg: 1
![Page 38: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/38.jpg)
できた!
![Page 39: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/39.jpg)
Twitterのストリームなんか処理することないし・・・
![Page 40: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/40.jpg)
![Page 41: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/41.jpg)
#jawsug っていいよね
#jawsug すごいよね
#jawsug 大阪熱いよね
#awsサンタ C3インスタンスにEBS Optimized
#awsサンタ Workspacesにもっとスペックを
#iegg 2/15はクロスエッグだよね
![Page 42: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/42.jpg)
![Page 43: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/43.jpg)
![Page 44: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/44.jpg)
レタス
はくさい
はくさい
キャベツ
キャベツ
レタス
![Page 45: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/45.jpg)
![Page 46: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/46.jpg)
POSから送られるデータからリアルタイムで売れてる
商品を探せ
![Page 47: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/47.jpg)
あっ! ってなった人?
![Page 48: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/48.jpg)
もうあなたは Kinesis使えます
![Page 49: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/49.jpg)
Kinesis Client Library(KCL)Kinesis Appを作りやすくする便利ライブラリ
APIを直接使わなくても簡単に使える データを処理するWorkerの生成 各Shardに対してWorkerがちゃんとつくように制御 Shard増減時のWorkerの増減 処理中にWorkerが死んだときの再処理制御 AutoScaling時にWorkerとShardの割当を自動的に調整
![Page 50: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/50.jpg)
Limited Previewの申込はこちら
http://aws.amazon.com/jp/kinesis/
![Page 51: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/51.jpg)
RDS for PostgreSQL
![Page 52: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/52.jpg)
やっとか・・・
![Page 53: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/53.jpg)
Keynoteでの発表時に会場から 「Yeah!!!!!!!!!!!!!」 の声が上がりました
![Page 54: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/54.jpg)
海外ではMySQLがOracleに買収されてからOracleがオープンソースに素っ気ないので、PostgreSQLに乗
り換える動きが出ています
![Page 55: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/55.jpg)
日本は、元々なぜかPostgreSQLの人気が高く利用者も多いです
![Page 56: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/56.jpg)
日本のAWS界隈でも RDSが出た当初から 「ぽすぐれを・・・」 と声が上がっていました
![Page 57: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/57.jpg)
やっとか・・・
![Page 58: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/58.jpg)
基本、全部入りですDB Snapshot Multi-AZ
Automated Backup Point-in-time Recovery Provisioned IOPS
![Page 59: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/59.jpg)
PostgreSQL独自機能もサポート
PostGIS PL/PGSQL など
![Page 60: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/60.jpg)
1つだけ・・・
リードレプリカ がありません
![Page 61: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/61.jpg)
そのうちできるようになると思います
![Page 62: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/62.jpg)
PostgreSQLのバージョンは9.3.1
バージョン8などを使っている人をちゃんと検証を!
PostgreSQLはバージョン上げると 「SQLが通らなくなったり」 「通るんだけど結果が違う」
とかあります (実話です)
注意
![Page 63: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/63.jpg)
RDS Cross-Region Read Replica
![Page 64: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/64.jpg)
リードレプリカを リージョンを またいで作る
![Page 65: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/65.jpg)
出典:http://aws.typepad.com/aws_japan/2013/11/cross-region-read-replicas-for-amazon-rds-for-mysql.html
書き込みは Masterへ
読み込みは Replicaで分散
レプリケーションの速度は CloudWatchで監視
![Page 66: JAWS-UG Osaka Special re:Invent 2013](https://reader031.fdocuments.in/reader031/viewer/2022020207/5592023a1a28ab5e128b459a/html5/thumbnails/66.jpg)
Thank You! Join Us
http://www.r3it.com/