Avoiding common database pitfalls
-
Upload
derek-binkley -
Category
Data & Analytics
-
view
244 -
download
2
Transcript of Avoiding common database pitfalls
Avoiding Common Database Pitfalls
Derek Binkley@DerekB_WI
About Me★Lead Developer at National Conference of
Bar Examiners★PHP and Java Developer★MySQL DBA★Father of Three★First ever talk
About NCBEExamining Bars?
About NCBENo, developing the bar exam for future lawyers.
Plus, supporting state admission authorities.
What is this talk about anyway?
Relational databases:Why to use them
Avoiding common mistakesGetting the most of a database
Good data design
Common ways to store data★Relational database (MySQL, SQL Server,
Oracle, Postgres)★NoSQL/document database/Key-value store
(CouchDB, MongoDB, Redis)★File system★Custom Built Solutions
Database Normalization1st Normal Form• Primary Key enforcing uniqueness• Consistent Field Types• Consistent Column Count
2nd Normal Form• Parent/Child Relations• Foreign Keys
3rd Normal Form• No transitive dependencies
Pitfall #1
Not understanding how to structure data
No clearly defined columnsColumns were left unlabeled so that customers could define their own data.
Very difficult for future developers
SQL with no defined columns
How maintainable is this?Do you know what this does?How would you map this to an object in PHP?
SQL well defined columns
This statement is quite clear what you are selecting and what you will get.
A more flexible solution?If Developer Needs• Flexible data storage• Customer ability to define meaning of data
NoSQL – Great use case.• Allows data design to change on the fly• Document defines meaning of underlying data.
Not using foreign key
Not taking advantage of database’s ability to store and index a number as a foreign key.
Violates 2nd Normal form.Cannot ensure data consistency
Keys with meaning
In this example any ID with a value < 100 gets administrative rights.
Problems?It is not clear the meaning of the data without the PHP code above.Administrative access could accidentally be given.Artificially limited to 99 administrators over life of system.
Natural Key v. Surrogate Key• A surrogate key has no meaning, e.g. GUID, incremental key• A natural key is a unique data attribute already in table, e.g.
two digit state code.
Reduce Joins with Natural KeySurrogate keys require join to get address of customer
Natural key allows understanding of state field without a join.
Natural Key May ChangeA primary key must be stable over time.In the United States, two digit state codes are historically stable.Examples of data thought to be stable may not be.• Country code – Yugoslavia? Soviet Union?• SSN – Privacy concerns require SSN to be encrypted?• Naturalized surrogate key – Once a surrogate key is shown to
user it starts to have meaning.
Pitfall #2
Not using the database for what it is good at
Not defining keys/indexes1. Ensure Data Integrity2. Boost PerformanceDatabase System optimizes queries for you.Explain plan can help provide useful information to see the
efficiency of keys.
Custom Transaction Handling• Databases are built to handle transaction.• App created its own locking table.• Developers spent time recreating a feature
they had already paid for.
• Deadlock condition required contacting tech support.
Not using processing power of DBDatabase is not just a storage engine.Powerful platform for sorting, filtering, grouping
and summarizing data.
Where do you put your domain logic?
Domain Logic In MemoryLogic is entirely in your PHP code, database is merely
used for storage.Easy to refactorEasy to unit testPart of your version control systemEasy to deployInefficient use of SQLHigh overheard for database connections and queries
Domain Logic In DatabaseSome logic is in SQL (procedures, views or direct SQL)More efficient: less memory in PHP, less connection
timeSQL statement will be much smaller and possibly easier
to understand.
Object Relational Mapping - ORM • Simplifies data access by mapping tables to
objects.• Can lead to more complex in memory
manipulation of data.• Can tie in to query mechanism to offload work
and logic to database.
Mishandling of nulls• Null should not be meaningful.• Null = “I don’t know”• Null column not equal to another null column• Check for “is null”
ResourcesA Simple Guide to Five Normal Forms in Relational Database Theory - http://www.bkent.net/Doc/simple5.htmNormalization: Friend or Foe? - http://www.treelinedesign.com/slides/normalization/confoo14.pdfDomain Logic and SQL - http://martinfowler.com/articles/dblogic.htmlChoosing a Primary Key: Natural or Surrogate? - http://www.agiledata.org/essays/keys.htmlSurrogate Key vs. Natural Key - http://sqlmag.com/business-intelligence/surrogate-key-vs-natural-key
Slides are available at ????
Please give feedback at https://joind.in/16419