Evgeniy Karelin. Mongo DB integration example solving performance and high load problems. DrupalCamp...
33
MongoDB Integration Example Solving Performance And High Load Problems
-
Upload
vlad-savitsky -
Category
Technology
-
view
1.337 -
download
1
description
This presentation is about real life example of using MongoDB on our not specific project for Drupal which supports more than 25m pageviews per day, more than 500k registered users with page load time less than 1sec. It will give an understanding how MongoDB can be easily used to increase performance of web-site. My presentation will be as easy as possible with simple examples and schemas but it will require at least intermediate level of developers. - project and tasks overview. http://freerice.com/ - quiz game site. There are a lot of dynamic info, users (registered and anonymous), groups and their different game statistis, user statuses etc. - problems while using MySQL - server optimization attempts. Memcache+Varnish. MySQL replication. Using game as separate script and AJAX blocks with "light" bootstrap. - MongoDB overview and it's benefits on current project. - PHP and MongoDB - project's MongoDB architecture overview - nodes and MongoDB - users/groups, their statistics and MongoDB - switching MySQL to MongoDB in Views. - indexing problems and statistic calculations. - multilingual support - scalability and using MongoDB replica set. - totals Этот доклад о применении MongoDB в одном из наших реальных проектов, который на данный момент обслуживает более 25млн показов страниц в день, более 500тыс зарегистрированных пользователей с скоростью загрузки страниц менее 1сек. Он позволит понять каким образом можно использовать MongoDB для увеличения производительности сайта. Мой доклад будет на столько простым на сколько это возможно с несложными схемами и примерами, но он требует как минимум среднего уровня разработчиков для полного понимания. - краткое описание проекта и поставленных задач. http://freerice.com/ - игра-викторина. много динамических данных, группы, игроки (зарегистрированные и анонимусы) и их статистики по разным параметрам, статусы игроков и т.д. - возникшие проблемы с работой MySQL. - попытки серверной оптимизации. Memcache+Varnish. Репликация MySQL. Перенос игры в отдельный скрипт и AJAX блоки с использованием "легкого" бутстрапа. - краткое описание MongoDB и приимущества его применения в текущем проекте. - PHP и MongoDB - общее описание архитектуры MongoDB на проекте. - работа с нодами в MongoDB. - работа с юзерами/группами и их статистиками. - переход с MySQL на MongoDB в Views. - проблемы с индексами, пересчеты статистик. - поддержка многоязычности. - масштабируемость MongoDB. Использование реплики. - итоги
Transcript of Evgeniy Karelin. Mongo DB integration example solving performance and high load problems. DrupalCamp...
MongoDB Integration Example Solving Performance And High
Load Problems
Gold Sponsor ofDrupalCamp Kyiv 2011
Silver Sponsors ofDrupalCamp Kyiv 2011
Freerice v2
Easy questions and categories management Users registration Facebook/Twitter integration Friendship system Groups Detailed game statistics per user/group/total,
rating system, user statuses
Round 1 Questions = Nodes. Data imported.
Categories = Nodes + Taxonomy
User profiles = Content profile module
Groups = Organic groups module
Friendship system = Flag module
Game logic, statistics = custom
Blog, comments… = Core modules
Beta test – OK
Live test - Fail
MySQL is slow
Views
node_load()
user_load()
Organic group
Game data processing
…
= SLOW QUERIES
What can help? Pressflow
Memcache
Varnish + AJAX Block
Game as separate script
MySQL replication
Code optimization
AJAX Blocks
AJAX blocks request -> index.php
New module for managing AJAX blocks
Set of blocks on page with needed bootstrap level
Game as separate script
MySQL is slow
What is MongoDB?
Fast INSERTs
Fast SELECTs
Pretty fast UPDATEs
No JOINs
PHP and MongoDB
PECL extension for MongoDB
Very easy to use
Redundancy of documentation and examples on both MongoDB and PHP.net sites
Code examples
Code example
What is 1 right answer?
10 rice
1 update into user tables
1 update into group tables
1 update into totals tables
1 update in user status tables
…
1.5-4.5 millions right answers per day
statistics.grains collection
Every right answer = 1 insert
Statistics recalculation every night
"uid": 1, "cid": 17555, "gid": 0, "amt": 10, "dat": 1305496800, "lvl": 1
Nodes collections
node.question, node.caterogy collection
hook_nodeapi() for insert, update, delete
Full Question and Category nodes into MongoDB
Questions and Categories SELECTs millions time often than INSERTs/UPDATEs
Nothing changed for site administration
statistics.user_totals collection
Other collections
statistic.group_totals
statistic.totals
statistic.useractivity
Views
Bunch of views used on site
Views produce a lot of SELECTs into MySQL
It is not possible to cache them
It is not possible to replace them
sview module
Sview disadvantages
Hard to support
No page/block view “from a box”, custom code needed
Multilingual support
MySQL data is translated as usual
Nothing to translate from users, groups collections
Questions and Categories nodes are fully saved in MongoDB -> easy to translate
MongoDB replication
Why ReplicaSet
Data redundancy
Automated failover
Distribute read load
Simplify maintenance
Recommended for sharding
How to use ReplicaSet
Freerice today
560k registered users
17k groups
25m page views per day
9.3b rice gained = 1m meals
3 DB servers and 2 Front end servers
If you have any questions
http://www.mongodb.org/ http:/php.net/manual/en/book.mongo.php mail: [email protected]
skype: smart_here
