SQL Basics+SQL Basics+

Brandon CheckettsBrandon Checketts

Why SQL?Why SQL?

Structured Query LanguageStructured Query Language Frees programmers from dealing Frees programmers from dealing

with specifics of data persistencewith specifics of data persistence Cross-platform, language Cross-platform, language

independentindependent Indexing and data optimizationIndexing and data optimization Data integrityData integrity

Some PitfallsSome Pitfalls

Vendor-Specific featuresVendor-Specific features Standardization is not greatStandardization is not great

Complexity? Additional Overhead?Complexity? Additional Overhead?

SQL EnginesSQL Engines

MySQLMySQL PostgreSQLPostgreSQL InformixInformix OracleOracle MSSQLMSSQL Many othersMany others

Database OrganizationDatabase Organization

A database server may have multiple A database server may have multiple databasesdatabases

Each database is made up of one or Each database is made up of one or more tablesmore tables

Queries can select from multiple Queries can select from multiple databases and tables.databases and tables.

Accessing your DatabaseAccessing your Database

Command LineCommand Line Web / GUI InterfacesWeb / GUI Interfaces ProgrammaticallyProgrammatically Spreadsheets (Excel)Spreadsheets (Excel) Reporting Applications (Crystal Reporting Applications (Crystal

Reports)Reports)

INSERT and SELECTINSERT and SELECT

INSERT INTO kidsINSERT INTO kids

SET name = ‘Noah’,SET name = ‘Noah’,

status = ‘nice’;status = ‘nice’;

SELECT * FROM kidsSELECT * FROM kids

WHERE name = ‘Noah’WHERE name = ‘Noah’

Table ManipulationTable Manipulation

CREATECREATE CREATE TABLE `christmas`.`kids` (CREATE TABLE `christmas`.`kids` (

`name` VARCHAR( 40 ) NOT NULL ,`name` VARCHAR( 40 ) NOT NULL , `status` VARCHAR( 7 ) NOT NULL `status` VARCHAR( 7 ) NOT NULL

) ;) ;

ALTERALTER ALTER TABLE `kids`ALTER TABLE `kids`

CHANGE `name` `first_name` VARCHAR( 40 ), CHANGE `name` `first_name` VARCHAR( 40 ),

ADD `last_name` VARCHAR( 40 ) NOT NULL AFTER ADD `last_name` VARCHAR( 40 ) NOT NULL AFTER `first_name` ; `first_name` ;

DROPDROP

Column TypesColumn Types

Char, varchar, text, longtextChar, varchar, text, longtext Int, tinyint, smallint, mediumint, Int, tinyint, smallint, mediumint,

bigintbigint Float, double, decimal,Float, double, decimal, Blob (binary large objects)Blob (binary large objects) Date, datetime, timestamp, year, Date, datetime, timestamp, year, Enum, boolEnum, bool

Santa’s DatabaseSanta’s Database

Santa would like to move into the 21Santa would like to move into the 21stst century and start keeping all of his century and start keeping all of his required information in a database.required information in a database.

Lets try developing it ourselvesLets try developing it ourselves Demonstrate creating a ‘christmas’ Demonstrate creating a ‘christmas’

database using phpMyAdmin (including database using phpMyAdmin (including user/pass)user/pass)

Create kids tableCreate kids table What columns might we need? What What columns might we need? What

types?types?

Santa’s Christmas AppSanta’s Christmas App

Santa decided that developing this Santa decided that developing this entire application by himself is too entire application by himself is too complicated.complicated.

He found an open-source application He found an open-source application that he wants to use to track his that he wants to use to track his lists. We’ve installed it at: lists. We’ve installed it at: http://roundsphere.com/christmas/http://roundsphere.com/christmas/

Kids TableKids Tablemysql> describe kids;mysql> describe kids;+------------+-------------+------+-----+-------------------+----------------++------------+-------------+------+-----+-------------------+----------------+| Field | Type | Null | Key | Default | Extra || Field | Type | Null | Key | Default | Extra |+------------+-------------+------+-----+-------------------+----------------++------------+-------------+------+-----+-------------------+----------------+| id | int(11) | NO | PRI | NULL | auto_increment || id | int(11) | NO | PRI | NULL | auto_increment || first_name | varchar(40) | NO | | | || first_name | varchar(40) | NO | | | || last_name | varchar(40) | NO | | | || last_name | varchar(40) | NO | | | || status | varchar(7) | NO | | | || status | varchar(7) | NO | | | || zip | varchar(5) | NO | | | || zip | varchar(5) | NO | | | || modified | timestamp | NO | | CURRENT_TIMESTAMP | || modified | timestamp | NO | | CURRENT_TIMESTAMP | |+------------+-------------+------+-----+-------------------+----------------++------------+-------------+------+-----+-------------------+----------------+

Mysql> show create table kids;Mysql> show create table kids;………… CREATE TABLE `kids` (CREATE TABLE `kids` ( `id` int(11) NOT NULL auto_increment,`id` int(11) NOT NULL auto_increment, `first_name` varchar(40) NOT NULL,`first_name` varchar(40) NOT NULL, `last_name` varchar(40) NOT NULL,`last_name` varchar(40) NOT NULL, `status` varchar(7) NOT NULL,`status` varchar(7) NOT NULL, `zip` varchar(5) NOT NULL,`zip` varchar(5) NOT NULL, `modified` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,`modified` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP, PRIMARY KEY (`id`)PRIMARY KEY (`id`) ) ENGINE=MyISAM;) ENGINE=MyISAM;

Adding reportsAdding reports

Santa is very happy with his new Santa is very happy with his new application. Now he’d like to add some application. Now he’d like to add some additional featuresadditional features

What reports might we want to add?What reports might we want to add? What have kids wished for?What have kids wished for? Kids who have been naughtyKids who have been naughty Kids who have been niceKids who have been nice Kids who are avoiding being checked up onKids who are avoiding being checked up on

http://roundsphere.com/christmas/repohttp://roundsphere.com/christmas/reports.phprts.php

Gift Lists (Importing Gift Lists (Importing from CSV)from CSV)

Santa Elves have compiled gift lists and have Santa Elves have compiled gift lists and have them available in a CSV formatthem available in a CSV format

We can create a table for them and load them We can create a table for them and load them directly from CSVdirectly from CSV

mysql>CREATE TABLE `christmas`.`gifts` (mysql>CREATE TABLE `christmas`.`gifts` ( `kid_id` INT NOT NULL ,`kid_id` INT NOT NULL , `gift` VARCHAR( 255 ) NOT NULL `gift` VARCHAR( 255 ) NOT NULL ) ENGINE = MYISAM ;) ENGINE = MYISAM ;

mysql> LOAD DATA local infile ‘gifts.csv' mysql> LOAD DATA local infile ‘gifts.csv' INTO TABLE giftsINTO TABLE gifts FIELDS TERMINATED BY ','FIELDS TERMINATED BY ','

ENCLOSED BY '"'ENCLOSED BY '"'LINES TERMINATED BY '\n' LINES TERMINATED BY '\n'

Manufacturing ReportManufacturing Report

Santa is an optimist and hopes that all Santa is an optimist and hopes that all kids will be good and get what they kids will be good and get what they asked for. He needs a report to pass on asked for. He needs a report to pass on to his elves so that they know what to to his elves so that they know what to manufacturemanufacture

SELECT gift, COUNT(gift) AS countSELECT gift, COUNT(gift) AS countFROM giftsFROM giftsGROUP BY giftGROUP BY giftORDER BY count DESC;ORDER BY count DESC;

Date/Time FunctionsDate/Time Functions

SELECT * FROM sometableSELECT * FROM sometable WHERE timestamp > NOW()WHERE timestamp > NOW()

WHERE timestamp > WHERE timestamp > DATE_SUB( NOW(), INTERVAL 7 DATE_SUB( NOW(), INTERVAL 7 DAY)DAY)

http://roundsphere.com/christmas/report_by_datehttp://roundsphere.com/christmas/report_by_date.php.php

Sleigh Loading ReportSleigh Loading Report We only want to load gifts for kids that We only want to load gifts for kids that

have been nicehave been nice We’ll introduce a JOIN on the kids tableWe’ll introduce a JOIN on the kids table

SELECT gift, COUNT(gift) AS countSELECT gift, COUNT(gift) AS count FROM giftsFROM gifts JOIN kids ON kids.id = gifts.kid_idJOIN kids ON kids.id = gifts.kid_id WHERE kids.status = 'nice'WHERE kids.status = 'nice' GROUP BY gifts.giftGROUP BY gifts.gift ORDER BY count DESCORDER BY count DESC

What is Santa’s sleight What is Santa’s sleight doesn’t have enough room doesn’t have enough room

for all toys?for all toys? He might have to reload his sleigh He might have to reload his sleigh

based on geographybased on geography We could query kids within a radius We could query kids within a radius

of a given location, that would be of a given location, that would be helpfulhelpful

We have the kids zip codes. Maybe We have the kids zip codes. Maybe we could group those together?we could group those together?

Exporting and ImportingExporting and Importing

Mysqldump to exportMysqldump to exportmysqldump db zipcode |gzip -c > mysqldump db zipcode |gzip -c >

zipcode.sql.gzzipcode.sql.gz Import with:Import with:

zcat zipcode.sql.gz| mysql zcat zipcode.sql.gz| mysql christmaschristmas

SQL ArithmeticSQL Arithmetic SQL Can do semi-complicated arithmetic:SQL Can do semi-complicated arithmetic:

Find all zip codes with in a distance of a lat/lon:Find all zip codes with in a distance of a lat/lon:

SELECT zc_zip,SELECT zc_zip,6371*acos(sin('$lat')*sin(zc_lat*pi()/6371*acos(sin('$lat')*sin(zc_lat*pi()/

180)+cos('$lat')*cos(zc_lat*pi()/180)*cos('$lon'-180)+cos('$lat')*cos(zc_lat*pi()/180)*cos('$lon'-zc_lon*pi()/180))/1.6093 AS distancezc_lon*pi()/180))/1.6093 AS distance

FROM zipcodeFROM zipcodeWHERE WHERE

6371*acos(sin('$lat')*sin(zc_lat*pi()/180)+cos('$lat')*6371*acos(sin('$lat')*sin(zc_lat*pi()/180)+cos('$lat')*cos(zc_lat*pi()/180)*cos('$lon'-zc_lon*pi()/cos(zc_lat*pi()/180)*cos('$lon'-zc_lon*pi()/180))<$radius *1.6093180))<$radius *1.6093

Complicated QueriesComplicated Queries

Now that we have a zip code Now that we have a zip code database, we can figure out what database, we can figure out what toys to load for all kids who have toys to load for all kids who have been good and live within a given been good and live within a given radius of some zip coderadius of some zip code

http://roundsphere.com/christmas/report_geo.phhttp://roundsphere.com/christmas/report_geo.phpp

SQL Injection AttacksSQL Injection AttacksThe Grinch wants to stop Christmas from coming, and is The Grinch wants to stop Christmas from coming, and is

attempting to delete Santa’s list.attempting to delete Santa’s list.

We have an SQL injection vulnerability in index.phpWe have an SQL injection vulnerability in index.php

This will select more ids than we intend to:This will select more ids than we intend to:http://roundsphere.com/christmas/index.php?status=bad%27+OR+1http://roundsphere.com/christmas/index.php?status=bad%27+OR+1%3D1+--%3D1+--

++

I’ve tried to construct something that will drop a table, but I’ve tried to construct something that will drop a table, but have been unsuccessful so far….have been unsuccessful so far….

A good page about SQL injection that I found is at:A good page about SQL injection that I found is at:http://unixwiz.net/techtips/sql-injection.htmlhttp://unixwiz.net/techtips/sql-injection.html

Other Useful FeaturesOther Useful Features

EncryptionEncryption Full-Text searchFull-Text search ConditionalsConditionals String functions String functions Spacial functions (GIS)Spacial functions (GIS) Precision MathPrecision Math

Alternatives to SQLAlternatives to SQL

MemCacheMemCache RRDRRD