Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

76
Tips of CakePHP & MongoDB 2011/9/4 CakeFest2011 Yasushi Ichikawa

description

 

Transcript of Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Page 1: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Tips of CakePHP & MongoDB

2011/9/4CakeFest2011

Yasushi Ichikawa

Page 2: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

I amYasushi Ichikawa

Ichi

@ichikawayhttp://cake.eizoku.com/blog

Page 3: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Topic● What's MongoDB?● Using MongoDB with CakePHP

● Setup ● Usage

● Security● Future

@ichikaway http://cake.eizoku.com/blog/

Page 4: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

NoSQLPerformanceScalability

MongoDB

@ichikaway http://cake.eizoku.com/blog/

Page 5: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Good for

● Social-Apps● Calculation on distributed servers

● log analysis● Questionnaire form

@ichikaway http://cake.eizoku.com/blog/

Page 6: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Terms

@ichikaway http://cake.eizoku.com/blog/

RDB MongoDB

Table Collection

Row Document

Column Field

Page 7: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Schema free

@ichikaway http://cake.eizoku.com/blog/

Posts collection

id, title, body

id, name, tel, fax

id, name, nickname, email

Posts Collection

Page 8: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Schema free

@ichikaway http://cake.eizoku.com/blog/

ScreenBlog collection

Title : xxxxText : yyyyTag: [tag1,tag2,tag3]Comment: [ comment1, comment2, comment3 ]

data

Blog

Title xxxxText yyyy

tag1,tag2,tag3

Comment1Comment2Comment3

Page 9: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

MongoDB operators

@ichikaway http://cake.eizoku.com/blog/

Find operators$gt, $gte$lt, $lte

$ne$in

$nin$or

db.posts.find( { age : { $gt: 5 }} )

http://www.mongodb.org/display/DOCS/Advanced+Queries

Page 10: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

MongoDB operators

@ichikaway http://cake.eizoku.com/blog/

Update operators$inc$set

$push$pull$pop

$unset

db.posts.update( { name: “Ichi” }, { $inc: { cnt: 1 }})

http://www.mongodb.org/display/DOCS/Updating

Page 11: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Functions● Geospatial index (location info)● Map/Reduce● Binary file saving (GridFS)● Sharding● etc

@ichikaway http://cake.eizoku.com/blog/

Page 12: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

WebSite

Page 13: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

http://kanael.net

Page 14: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

http://kanael.net

Page 15: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

kanael.net

●Server● VPS(2.4GHz-2core, 1.5GMem) x 1

●Application● 40% write, 60% read● 300,000 ducuments

@ichikaway http://cake.eizoku.com/blog/

Page 16: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

kanael.net

● Peak traffic● 100,000+ requests/day ● CPU 75% (MongoDB 10%)

@ichikaway http://cake.eizoku.com/blog/

Page 17: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Topic● What's MongoDB?● Using MongoDB with CakePHP

● Setup ● Usage

● Security● Future

@ichikaway http://cake.eizoku.com/blog/

Page 18: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

CakePHP MongoDB

@ichikaway http://cake.eizoku.com/blog/

Repositorygithub.com/ichikaway/cakephp-mongodb/

Page 19: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

CakePHP MongoDB

@ichikaway http://cake.eizoku.com/blog/

Repository●Test files●API documents●Sample Applications

Page 20: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

CakePHP MongoDB

@ichikaway http://cake.eizoku.com/blog/

PHP5+ CakePHP1.2, 1.3, 2.0-beta Pecl Mongo driver Documents

● https://github.com/ichikaway/cakephp-mongodb/wiki

Page 21: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Structure

@ichikaway http://cake.eizoku.com/blog/

CakePHP-MongoDB Datasource

MongoDB

MongoCollection

MongoCursor

Model

Page 22: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

Setup

Page 23: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Setup pecl mongo

@ichikaway http://cake.eizoku.com/blog/

pecl install mongo

vi php.iniextension=mongo.so

Page 24: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

CakePHP1.3

Page 25: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Setup Cake Mongo(1.3)

@ichikaway http://cake.eizoku.com/blog/

cd app/pluginsgit clone git://github.com/ichikaway/cakephp-mongodb.git mongodb

vi app/config/database.php

Page 26: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

database.php Cake1.3

@ichikaway http://cake.eizoku.com/blog/

class DATABASE_CONFIG { public $default = array( 'driver' => 'mongodb.mongodbSource', 'database' => 'blog', 'host' => 'localhost', 'port' => 27017, );

Page 27: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

CakePHP2.0

Page 28: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Setup Cake Mongo(2.0)

@ichikaway http://cake.eizoku.com/blog/

cd app/Plugingit clone git://github.com/ichikaway/cakephp-mongodb.git Mongodbgit checkout -b cake2.0 origin/cake2.0vi app/Config/database.php

Page 29: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

database.php Cake2.0

@ichikaway http://cake.eizoku.com/blog/

// app/Config/database.phpclass DATABASE_CONFIG { public $default = array( 'datasource' => 'Mongodb.MongodbSource', 'host' => 'localhost', 'database' => 'blog', 'port' => 27017, );

Page 30: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Load plugin Cake2.0

@ichikaway http://cake.eizoku.com/blog/

//app/Config/bootstrap.phpCakePlugin::load('Mongodb')

Page 31: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Sample Post Model

@ichikaway http://cake.eizoku.com/blog/

class Post extends AppModel{ public $primaryKey = '_id'; }

Page 32: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

Useage

Page 33: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

find data

@ichikaway http://cake.eizoku.com/blog/

class PostsController extends AppController{ public function index() { $this->Post->find('all', $options); }} fields, conditions,

order, limit

Page 34: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Insert data

@ichikaway http://cake.eizoku.com/blog/

$data = array('name' => 'Ichi' 'age' => 32 );

$this->Post->save($data);

Posts collection

_id:xxx1, name: 'Ichi', 'age':32

Page 35: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Update data

@ichikaway http://cake.eizoku.com/blog/

$data = array( '_id' => 'xxx1', 'name' => 'Yasu' );$this->Post->save($data);

// in Cake-Mongo DataSource$MongoCollection->update( array('_id' => 'xxx001'), array('$set' => array('name' => 'Yasu')),);

Page 36: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

$set operator

@ichikaway http://cake.eizoku.com/blog/

Posts collection

id:xxx1, name: 'Yasu', 'age':32

Posts collection

id:xxx1, name: 'Yasu' Without $set

With $set

Page 37: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

Use other update

operators

Page 38: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Update operator ($inc)

@ichikaway http://cake.eizoku.com/blog/

$data = array( '_id' => 'xxx1', '$inc' => array('age' => 1) );$this->Post->save($data);

// in Cake-Mongo DataSource$MongoCollection->update( array('_id' => 'xxx001'), array('$inc' => array('age' => 1)),);

Page 39: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Update operator(result)

@ichikaway http://cake.eizoku.com/blog/

Posts collection

_id:xxx1, name: 'Ichi', 'age':32

Posts collection

_id:xxx1, name: 'Ichi', 'age':33,

Page 40: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Update operator(complex)

@ichikaway http://cake.eizoku.com/blog/

$data = array( '_id' => 'xxx1', '$inc' => array('age' => 1), '$push' => array('tags' => array('php', 'mongo')));$this->Post->save($data);

Page 41: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Update operator(result)

@ichikaway http://cake.eizoku.com/blog/

Posts collection

_id:xxx1, name: 'Ichi', 'age':32

Posts collection

_id:xxx1, name: 'Ichi', 'age':33,tags: ['php', 'mongo']

Page 42: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Update operator

@ichikaway http://cake.eizoku.com/blog/

●see Wiki● https://github.com/ichikaway/cakephp-mongodb/wiki/How-to-use-MongoDB-update-operators

● see test code● testUpdate()● testUpdateWithoutMongoSchemaProperty()

Page 43: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

Get Cake MongoDataSource

Object

Page 44: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Source methods

@ichikaway http://cake.eizoku.com/blog/

● ensureIndex()● mapreduce()● group()See wikihttps://github.com/ichikaway/cakephp-mongodb/wiki/_pages

Page 45: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

ex. make index

@ichikaway http://cake.eizoku.com/blog/

$ds = $this->Post->getDataSource();

$ds->ensureIndex( $this->Post, array('title' => 1));

Page 46: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

Get MongoDB Object

Page 47: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

MongoDB Object● CakeMongo DataSource

● not support all functions of MongoDB

– gridFs

– DbRef

Page 48: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

get MongoDB Object

@ichikaway http://cake.eizoku.com/blog/

$mongo = $this->Post->getMongoDb();

Page 49: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

get MongoDB Object

@ichikaway http://cake.eizoku.com/blog/

$mongo->getGridFs();

$mongo->setSlaveOkay();

$mongo->createDbRef();

See php manualhttp://php.net/manual/en/class.mongodb.php

Page 50: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

Get MongoCollection

Object

Page 51: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

get Mongo Collection

@ichikaway http://cake.eizoku.com/blog/

$mongo = $this->Model->getMongoDb();

$collection = $mongo-> selectCollection('posts');

Page 52: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

get Mongo Collection

@ichikaway http://cake.eizoku.com/blog/

$collection->find();$collection->update();$collection->insert();$collection->createDbRef();

See php manualhttp://php.net/manual/en/class.mongocollection.php

Page 53: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

Replica Sets

Page 54: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

● master/slave replication● automatic failover● automatic recovery

Replica sets

Page 55: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

Replica setsServer1Primary

Server2Secondary

Server3Secondary

ApplicationServer

(CakePHP)

Replication

Replication

Page 56: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

Replica setsServer1Primary

Server2Secondary

Server3Secondary

ApplicationServer

(CakePHP)

Replication

Replication

Page 57: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

Replica setsServer1Primary

Server2Primary

Server3Secondary

ApplicationServer

(CakePHP)

Replication

Page 58: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

database.php Cake1.3

@ichikaway http://cake.eizoku.com/blog/

class DATABASE_CONFIG { public $default = array( 'driver' => 'mongodb.mongodbSource', 'database' => 'blog',

'replicaset' => array( 'host' =>'mongodb://loginid:password@ Server1:27021,Server2:27022/blog', 'options' => array('replicaSet' => 'myRepl') ), );

https://github.com/ichikaway/cakephp-mongodb/wiki/How-to-connect-to-replicaset-servers

Page 59: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Topic● What's MongoDB?● Using MongoDB with CakePHP

● Setup ● Usage

● Security● Future

@ichikaway http://cake.eizoku.com/blog/

Page 60: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

InjectionAttack

Page 61: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

ONLYPHP

( ; ´Д ` )

Page 62: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

WHY??

Page 63: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

● PHP makes array data from GET/POST request● ex. login.php?username=admin&passwd[$ne]=1

Injection Attack$user = $collection->find(array( "username" => $_GET['username'], "passwd" => $_GET['passwd']));

Page 64: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

● PHP makes array data from GET/POST request● ex. login.php?username=admin&passwd[$ne]=1

Injection Attack$user = $collection->find(array( "username" => $_GET['username'], "passwd" => $_GET['passwd']));

$user = $collection->find(array( "username" => 'admin', "passwd" => array("$ne" => 1)));

Page 65: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

● Don't trust user input data● GET/POST/Cookie

● Solution●Cast to string●Check all keys of array

Solution

Page 66: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

Cast to string

Solution

Page 67: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

Solution(cast to string)

$cursor = $collection->find(array( "username" => (string)$_GET['username'], "passwd" => (string)$_GET['passwd']));

Page 68: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

Solution(cast to string)

$cursor = $collection->find(array( "username" => 'admin', "passwd" => 'Array'));

Page 69: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

Check keysof

input data

Solution

Page 70: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

Solution(check keys)

SecurePHPLibrary

https://github.com/ichikaway/SecurePHP

Page 71: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

SecurePHP● Check Post/Get/Cookie●Check all array keys

● allow: a-z0-9:-_./

● Check null byte

Page 72: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

SecurePHPvi webroot/index.php

require_once( 'SecurePHP/config/bootstrap.php');$Dispatcher = new Dispatcher();$Dispatcher->dispatch();

Page 73: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Topic● What's MongoDB?● Using MongoDB with CakePHP

● Setup ● Usage

● Security● Future

@ichikaway http://cake.eizoku.com/blog/

Page 74: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

@ichikaway http://cake.eizoku.com/blog/

In the future

Relational data fetchcoming soon

(hasOne, hasMany, belongsTo)relation branch

Page 75: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

Summary● What's MongoDB?● Using MongoDB with CakePHP

● Setup ● Usage(find, save, MongoObject)

● Security●Injection attack

● Future● Relational data fetch

@ichikaway http://cake.eizoku.com/blog/

Page 76: Tips of CakePHP and MongoDB - Cakefest2011 ichikaway

THANK YOU

@ichikaway http://cake.eizoku.com/blog/