1

MoSQLMoSQL

MoskyMosky

2

More than SQL, but Less than ORMMore than SQL, but Less than ORM

MoSQLMoSQL

3

OutlineOutline● Why not SQL?Why not SQL?● Why ORM?Why ORM?● MoSQLMoSQL

– SQL BuildersSQL Builders

– Model of Result SetModel of Result Set

● ConclusionConclusion

4

Why not SQL?Why not SQL?

5

SQL SyntaxSQL Syntax● SELECT * FROM article;SELECT * FROM article;

● SELECT * FROM article LIMIT 1;SELECT * FROM article LIMIT 1;

● add “ ORDER BY created ”?add “ ORDER BY created ”?

● add “ OFFSET 10 ”?add “ OFFSET 10 ”?

● add “ GROUP BY author ”?add “ GROUP BY author ”?

● Is “ UPDATE article WHERE title='SQL' SET Is “ UPDATE article WHERE title='SQL' SET title='ORM' ” correct?title='ORM' ” correct?

6

!@#$%!@#$%

7

SQL InjectionSQL Injection● ') or '1'='1') or '1'='1● ' or true; -- ' or true; -- ● ' or 1=1; -- ' or 1=1; -- ● ' or 2=2; --' or 2=2; --● ' or 'str'='str'; --' or 'str'='str'; --● ……

8

It may be hacker friendly.It may be hacker friendly.

9

SQL seems ancient, but ...SQL seems ancient, but ...

10

using SQL is the using SQL is the FASTESTFASTEST way. way.

11

Why ORM?Why ORM?

12

ORM SyntaxORM Syntaxclass User(Base):class User(Base):

__tablename__ = 'users'__tablename__ = 'users'

name = Column(String)name = Column(String)

fullname = Column(String)fullname = Column(String)

password = Column(String)password = Column(String)

13

ORM Syntax (cont.)ORM Syntax (cont.)>>> fake_user = User('fakeuser', 'Invalid', >>> fake_user = User('fakeuser', 'Invalid', '12345')'12345')

>>> session.add(fake_user)>>> session.add(fake_user)

>>> for row in session.query(User, >>> for row in session.query(User, User.name).all(): User.name).all():

... print row.User, row.name ... print row.User, row.name

14

hmmm …hmmm …

15

SQL InjectionSQL Injection● ' or true; -- ' or true; -- ● ' or 1=1; -- ' or 1=1; -- ● ' or 1=1; #' or 1=1; #● ' or 1=1; /*' or 1=1; /*● ') or '1'='1') or '1'='1● ……● SaferSafer

16

It's good!It's good!

17

ORM seems modern, but ...ORM seems modern, but ...

18

the most of ORMs are SLOW.the most of ORMs are SLOW.

19

SQL < ______ < ORMSQL < ______ < ORM

20

SQL < MoSQL < ORMSQL < MoSQL < ORM

21

SQL BuildersSQL Builders

22

SQL Builders (cont.)SQL Builders (cont.)>>> from mosql.build import *>>> from mosql.build import *

>>> >>> select('pycon')select('pycon')

SELECT * FROM "pycon"SELECT * FROM "pycon"

>>> select('pycon', >>> select('pycon', {'id': 'mosky'}{'id': 'mosky'}))

SELECT * FROM "pycon" WHERE "id" = 'mosky'SELECT * FROM "pycon" WHERE "id" = 'mosky'

23

SQL Builders (cont.)SQL Builders (cont.)>>> insert('pycon', >>> insert('pycon', {'yr': 2013, 'id': 'masky'}{'yr': 2013, 'id': 'masky'}))

INSERT INTO "pycon" ("id", "yr") VALUES ('masky', 2013)INSERT INTO "pycon" ("id", "yr") VALUES ('masky', 2013)

>>> update('pycon',>>> update('pycon',

... ... where={'id': 'masky'}where={'id': 'masky'},,

... ... set ={'id': 'mosky'}set ={'id': 'mosky'}

... )... )

UPDATE "pycon" SET "id"='mosky' WHERE "id" = 'masky'UPDATE "pycon" SET "id"='mosky' WHERE "id" = 'masky'

24

SQL Builders (cont.)SQL Builders (cont.)● insert(table, insert(table, setset, …), …)

● select(table, select(table, wherewhere, …), …)

● update(table, update(table, wherewhere, , setset, …), …)

● delete(table, delete(table, wherewhere, …), …)

● ......

25

If you like it,If you like it,

26

sudo pip install mosqlsudo pip install mosql

27

Model of Result SetModel of Result Set

28

Model: Configure ConnectionModel: Configure Connectionimport psycopg2.poolimport psycopg2.pool

from mosql.result import Modelfrom mosql.result import Model

pool = psycopg2.pool.SimpleConnectionPool(1, 5, pool = psycopg2.pool.SimpleConnectionPool(1, 5, database='mosky')database='mosky')

class PostgreSQL(Model):class PostgreSQL(Model):

getconn = pool.getconngetconn = pool.getconn

putconn = pool.putconnputconn = pool.putconn

29

Model: Set the Name of TableModel: Set the Name of Tableclass Person(PostgreSQL):class Person(PostgreSQL):

table = 'person'table = 'person'

>>> Person.select(>>> Person.select({'person_id': 'mosky'}{'person_id': 'mosky'}))

{'name': ['Mosky Liu'], 'person_id': ['mosky']}{'name': ['Mosky Liu'], 'person_id': ['mosky']}

>>> Person.where(person_id=>>> Person.where(person_id=('andy', 'mosky')('andy', 'mosky')))

{'name': ['Andy Warhol', 'Mosky Liu'], 'person_id': {'name': ['Andy Warhol', 'Mosky Liu'], 'person_id': ['andy', 'mosky']}['andy', 'mosky']}

30

Model: Make QueriesModel: Make QueriesPerson.Person.selectselect({'person_id': 'mosky'})({'person_id': 'mosky'})

Person.Person.insertinsert({'person_id': 'tina'})({'person_id': 'tina'})

Person.Person.updateupdate((

where={'person_id': 'mosky'},where={'person_id': 'mosky'},

set ={'name' : 'Yiyu Liu'}set ={'name' : 'Yiyu Liu'}

))

Person.Person.deletedelete({'person_id': 'tina'})({'person_id': 'tina'})

31

Model: Squash ColumnsModel: Squash Columnsclass Person(PostgreSQL):class Person(PostgreSQL):

table = 'person'table = 'person'

squashed = set(['person_id', 'name'])squashed = set(['person_id', 'name'])

>>> Person.select({'person_id': 'mosky'})>>> Person.select({'person_id': 'mosky'})

{'name': {'name': 'Mosky Liu''Mosky Liu', 'person_id': , 'person_id': 'mosky''mosky'}}

>>> Person.where(person_id=('andy', 'mosky'))>>> Person.where(person_id=('andy', 'mosky'))

{'name': {'name': 'Andy Warhol''Andy Warhol', 'person_id': , 'person_id': 'andy''andy'}}

32

Model: ArrangeModel: Arrangeclass Person(PostgreSQL):class Person(PostgreSQL):

......

arrange_by = ('person_id', )arrange_by = ('person_id', )

>>> for person in Person.arrange(>>> for person in Person.arrange({'person_id': {'person_id': ('andy', 'mosky')}('andy', 'mosky')}):):

... print person... print person

{'name': 'Andy Warhol', 'person_id': 'andy'}{'name': 'Andy Warhol', 'person_id': 'andy'}

{'name': 'Mosky Liu', 'person_id': 'mosky'}{'name': 'Mosky Liu', 'person_id': 'mosky'}

33

Model: Arrange (cont.)Model: Arrange (cont.)>>> for detail in >>> for detail in DetailDetail.arrange({'person_id': .arrange({'person_id': ('mosky', 'andy')}):('mosky', 'andy')}):

... print detail... print detail

......

{'detail_id': [5],{'detail_id': [5],

'key': 'email','key': 'email',

'person_id': 'andy','person_id': 'andy',

'val': ['andy@gmail.com']}'val': ['andy@gmail.com']}

......

34

Model: FindModel: Findclass Person(PostgreSQL):class Person(PostgreSQL):

......

arrange_by = ('person_id', )arrange_by = ('person_id', )

>>> for person in Person.>>> for person in Person.findfind((person_id=('andy', person_id=('andy', 'mosky')'mosky')):):

... print person... print person

{'name': 'Andy Warhol', 'person_id': 'andy'}{'name': 'Andy Warhol', 'person_id': 'andy'}

{'name': 'Mosky Liu', 'person_id': 'mosky'}{'name': 'Mosky Liu', 'person_id': 'mosky'}

35

Model: Identify a RowModel: Identify a Rowclass Person(PostgreSQL):class Person(PostgreSQL):

......

ident_by = ('person_id', )ident_by = ('person_id', )

36

Model: ModificationModel: Modification>>> p = Person.where(person_id='mosky')>>> p = Person.where(person_id='mosky')

>>> >>> p['name'] = 'Yiyu Liu'p['name'] = 'Yiyu Liu'

>>> >>> p.name = 'Yiyu Liu'p.name = 'Yiyu Liu'

>>> p.save()>>> p.save()

>>> d = >>> d = DetailDetail.where(.where(person_id='mosky', key='email'person_id='mosky', key='email'))

>>> >>> p['val'][0] = '<modified email>'p['val'][0] = '<modified email>'

>>> >>> p.val[0] = '<modified email>'p.val[0] = '<modified email>'

>>> p.save()>>> p.save()

37

Model: Pop and AppendModel: Pop and Append>>> d = Detail.where(>>> d = Detail.where(person_id='mosky', key='email'person_id='mosky', key='email'))

>>> >>> p.pop(-1)p.pop(-1)

>>> >>> p.append({'val': '<new mail>'})p.append({'val': '<new mail>'})

>>> p.save()>>> p.save()

38

Model: Default ClausesModel: Default Clausesclass Person(PostgreSQL):class Person(PostgreSQL):

......

clauses = dict(clauses = dict(

order_by=('person_id', )order_by=('person_id', )

))

39

PerformancePerformance● AboutAbout 4x 4x faster than SQLAlchemy. faster than SQLAlchemy.● Just a little bit slower than pure SQL.Just a little bit slower than pure SQL.

40

SecuritySecurity● Security by default.Security by default.● Use escaping technique.Use escaping technique.● Prevent SQL injection from both valuePrevent SQL injection from both value

and identifier. and identifier.● Passed the tests from Passed the tests from sqlmapsqlmap at level=5 at level=5

and risk=3. and risk=3.

41

ConclusionConclusion● Easy-to-LearnEasy-to-Learn● ConvenientConvenient● FasterFaster● SecureSecure● sudo pip install mosqlsudo pip install mosql

● http://mosql.mosky.tw/http://mosql.mosky.tw/● Welcome to fork!Welcome to fork!