MySQL overview
-
Upload
marco-tusa -
Category
Technology
-
view
112 -
download
1
description
Transcript of MySQL overview
MySQL Overview a quick look at the general aspects of the most popular open source database
Marco Tusa
MySQL Team Leader
16 September 2011
2 © 2010/2011 Pythian
Why Companies Trust Pythian • Recognized Leader: • Global industry-leader in remote database administration services and consulting for Oracle,
Oracle Applications, MySQL and SQL Server
• Work with over 150 multinational companies such as Forbes.com, Fox Interactive media, and MDS Inc. to help manage their complex IT deployments
• Expertise:
• One of the world’s largest concentrations of dedicated, full-time DBA expertise.
• Global Reach & Scalability:
• 24/7/365 global remote support for DBA and consulting, systems administration, special projects or emergency response
2
3 © 2010/2011 Pythian
Why Companies Trust Pythian
• Recognized Leader: • Global industry-leader in remote database administration services and consulting for Oracle, Oracle Applications, MySQL and SQL Server
• Work with over 150 multinational companies such as Toyota, Fox Interactive media, and MDS Inc. to help manage their complex IT deployments
• Expertise: • One of the world’s largest concentrations of dedicated, full-time DBA expertise.
• Global Reach & Scalability: • 24/7/365 global remote support for DBA and consulting, systems administration, special projects or emergency response
2
4 © 2010/2011 Pythian
Who I am • Currently: Team Lead at The Pythian Group for MySQL database
support
• Manager Professional Service South EMEA at MySQL/SUN/Oracle
• In MySQL before the SUN gets on us
• Leading the team responsible for Oracle & MySQL DBs service in support to technical systems, at Food and Agriculture Organization of United Nations (FAO of UN)
• Leading developer & system administrator teams in FAO managing the Intranet/Internet infrastructure.
• Working (a lot) in developing countries like (Ethiopia, Senegal, Ghana, Egypt …)
• My Profile http://it.linkedin.com/in/marcotusa
3
5 © 2010/2011 Pythian
What is MySQL?
• MySQL is a database management system
• MySQL is a relational database management system
• MySQL software is Open Source.
• MySQL Server works in client/server or embedded systems.
• The official way to pronounce “MySQL” is “My Ess Que Ell” (not “my sequel”)
20
6 © 2010/2011 Pythian
Who is using MySQL
List of customer from MySQL official presentation
20
7 © 2010/2011 Pythian
Main insight on MySQL 1/2
• Written in C and C++.
• Works on many different platforms:
• FreeBSD 5.x.
• HP-UX 11.x.
• Linux.
• Mac OS X.
• Solaris 2.8 on SPARC and x86
• Windows XP, Windows Vista, Windows Server 2003, and Windows Server 2008.
20
8 © 2010/2011 Pythian
Main insight on MySQL 2/2
• Uses multi-layered server design with independent modules.
• Designed to be fully multi-threaded using kernel threads, to easily use multiple CPUs if they are available.
• Provides transactional and non-transactional storage engines. • Designed to make it relatively easy to add other storage engines. This is useful if you want to provide an SQL interface for an in-house database.
• Uses a very fast thread-based memory allocation system.
• Full Localization support
20
9 © 2010/2011 Pythian
What Standards MySQL Follows?
• Supports the full ANSI/ISO SQL standard, but with some flexibility (SQL-92=1992 Release; SQL:1999=1999 Release; SQL:2003=2003 Release; SQL:2008= 2008 Release). http://www.iso.org/iso/home.htm
• ODBC levels 0 to 3.51.
• MySQL can adapt its behavior using SQL Modes like:
• ANSI (SET GLOBAL sql_mode='ANSI';)
• Oracle
• Custom …
• MySQL DATA is structured on the file system by directory and subdirectory (symbolic links widely used)
20
10
© 2010/2011 Pythian
Installing MySQL in 4 easy steps
Installing MySQL is a simple operation:
1. Download software from site like (http://dev.mysql.com/get/Downloads/MySQL-5.5/mysql-5.5.15-linux2.6-x86_64.tar.gz/from/http://mirror.csclub.uwaterloo.ca/mysql/)
2. Expand it in /usr/local/mysql (and set all grants for mysql user)
3. Copy one of the pre-configured my.cnf (from support-files directory) to /etc/my.cnf; copy from same source mysql.server file to /etc/init.d/mysql
4. Start mysql /etc/init.d/mysql start
You can have MySQL running in 30 minutes.
20
11
© 2010/2011 Pythian
MySQL Architecture 1/2 MySQL is composed of two main layers
20
12
© 2010/2011 Pythian
MySQL Architecture 2/2
• MySQL Layer (Parser; Optimizer; Query cache; threads handling )
• Storage engines:
• MyISAM ( no ACID)
• Innodb (ACID)
• Cluster (ACID)
• Archive (only archive and append)
• Memory (Main engine for in memory operations)
• Blackhole (it is exactly what the name indicate)
• .. many others
20
13
© 2010/2011 Pythian
MySQL Storage engine more details 1/2 They have different advantages / trade-offs - know when to use them.
1. MyISAM • No transaction handling; no isolation level; no data buffered (only index); full Table
lock; no foreign key;
• Fully file base; minimal overhead in write; very fast in reading; Btree index; Full text search; Geospatial support; 256TB Storage limit;
2. InnoDB • No full text (5.6 will); no Geospatial support; no cluster support; 64TB Storage limit;
• Full ACID; possible to modify the Isolation level; data is buffered; Write lock by ROW; Foreign-key support; automatic recovery from redo log in case of crash; Btree index; Clustered table; different file format (internal); data compression.
3. Memory • No transaction; no version control; no Geospatial support; no foreign-key; full Table
lock;
• Data cached in memory; Btree index; Hash index; Storage limit is the available RAM.
20
14
© 2010/2011 Pythian
MySQL Storage engine more details 2/2 1. MySQL Cluster
• No geospatial support; no foreign-key; no “Temporary” tables; partition only by [LINEAR] KEY; only the READ COMMITTED isolation level is supported; no partial roll-back (full or nothing); a Select on Blob or Text fields will create a READ LOCK on the table to guarantee consistency.
• Fully ACID; share nothing architecture; single point of failure; write scale by node-group; thread requests parallelization; and a lot more …
20
15
© 2010/2011 Pythian
MySQL Logs 1/2 What they are and how MySQL uses them. • General Log
• each SQL statement received from clients; connection/disconnection as well
• Write intensive to be used ONLY for checking issues
• Binary log
• Contains the statements “events” that modify data structure and datasets
• Point in time recovery; Replication; Backup; can be (statement/mixed/row)
• Relay log
• Is the local copy of the Slave; contains the same information of the Binlogs
• Slow log
• The Error Log
• Self explanatory in the latest MySQL can write to syslog
20
16
© 2010/2011 Pythian
MySQL Logs 2/2
What they are and how MySQL uses them.
• Slow log
• Information related to SLOW QUERY
• Could be write intensive; long time and no index usage configurable
• Crucial for SQL review
• The Error Log
• If not specify --log-error errors are written to stderr
• Warnings are written as well (log_warnings) to disable
• From MySQL 5.1 can also write to syslog
20
17
© 2010/2011 Pythian
MySQL Replication
Replication topology, and when to use what.
• Master -> Slave
• Master <> Master
• Master -> Multi slave
• MySQL replication pro and cons
• Single thread
• Parallel replication
• Asynchronous 20
18
© 2010/2011 Pythian
Scaling with MySQL Replication.
• Scaling for reads
• Add slave(s) but … keep in mind PT = (ND + QT) * NL PT = Propagation Time ND = Network time delay QT = Query Execution Time NL = Number of Level
• Scaling for writes
• Sharding or MySQL Cluster
• Scaling out vs. scaling up
• Partition
• Spread the load in smaller chunk
• Using the right approach
• Database dimension
• Activity (r/w) 20
19
© 2010/2011 Pythian
Partitioning 1/2 Use Partition to organize data
Max number of partition per table !1024!
Types of partitioning
• RANGE
• Numeric range like (store Id; Year) less then “value”
• LIST
• Declared list of value (matching one in the list)
• COLUMNS
• RANGE: List of a single column value or more (partition definition value list) evaluation left->right
• LIST: Declared list of value coming from one or more columns
• No DECIMAL, FLOAT,TIMESTAMP,TEXT,BLOB
20
20
© 2010/2011 Pythian
Partitioning 2/2 Types of partitioning (cont.)
• HASH
• Calculate an hash value on the given value (also expression) then equally distribute the data
• KEY • Calculate an hash value on Primary/Unique
• Subpartitioning
• Works for LIST and RANGE Partitioned tables
• Subpartition only by HASH or KEY
Partition cannot be done on table with: Full Text index ; Spatial columns
20
21
© 2010/2011 Pythian
MySQL programming 2/2 A few point about performance • Not as good you would expect
• Optimize IF
• Recursive VS not recursive calculating Fibonacci numbers
20
22
© 2010/2011 Pythian
MySQL programming Using stored procedure (code for performance on prime numbers)
20
PROCEDURE N_PRIMES ( p_num NUMBER) IS i INT; j INT; nprimes INT; isprime INT; BEGIN i:=2; nprimes:=0; <<main_loop>> WHILE (i<p_num) LOOP isprime:=1; j:=2; <<divisor_loop>> WHILE (j<i) LOOP IF (MOD(i,j)=0) THEN isprime:=0; EXIT divisor_loop; END IF; j:=j+1; END LOOP ; IF (isprime=1) THEN nprimes:=nprimes+1; END IF; i:=i+1; END LOOP; dbms_output.put_line(nprimes||' prime numbers
less than '||p_num); END;
CREATE PROCEDURE sp_nprimes(p_num int) BEGIN DECLARE i INT; DECLARE j INT; DECLARE nprimes INT; DECLARE isprime INT;
SET i=2; SET nprimes=0;
main_loop: WHILE (i<p_num) do SET isprime=1; SET j=2; divisor_loop: WHILE (j<i) DO IF (MOD(i,j)=0) THEN SET isprime=0; LEAVE divisor_loop; END IF; SET j=j+1; END WHILE ; IF (isprime=1) THEN SET nprimes=nprimes+1; END IF; SET i=i+1; END WHILE; SELECT CONCAT(nprimes,' prime numbers less
than ',p_num); END;
23
© 2010/2011 Pythian
MySQL programming Using Store procedure (code for Fibonacci test)
20
Recursive
CREATE PROCEDURE rec_fib(n INT,OUT out_fib INT)
BEGIN DECLARE n_1 INT; DECLARE n_2 INT;
IF (n=0) THEN SET out_fib=0; ELSEIF (n=1) then SET out_fib=1; ELSE CALL rec_fib(n-1,n_1); CALL rec_fib(n-2,n_2); SET out_fib=(n_1 + n_2); END IF; END
Not Recursive CREATE PROCEDURE nonrec_fib(n INT,OUT out_fib INT) BEGIN DECLARE m INT default 0; DECLARE k INT DEFAULT 1; DECLARE i INT; DECLARE tmp INT;
SET m=0; SET k=1; SET i=1;
WHILE (i<=n) DO SET tmp=m+k; SET m=k; SET k=tmp; SET i=i+1; END WHILE; SET out_fib=m; END
24
© 2010/2011 Pythian
MySQL programming 1/2 • Using Store procedure
• Programming is real at a basic level; no packages etc…
• Triggers and function could leads to unexpected results by Storage engine
• Real status about store procedure
• Triggers; Functions; Store procedure
• Using SELECT INTO is fine for single-row queries; CURSORS; Unbound result set (return result set to calling application); Non SELECT actions (update, insert, delete); Internal CALL to other Store Procedure; few info about performance
• Error handling available from 5.5 onwards with SIGNAL and RESIGNAL
20
25
© 2010/2011 Pythian
MySQL HA solution How many nines do you need?
9 0. 0 0 0 % (36 days) MySQL Replication
9 9. 0 0 0 % (8 hours) Linux Heartbeat with DRBD
9 9. 0 0 0 % (8 hours) Linux Heartbeat with Shared Storage (Active/Passive)
9 9. 0 0 0 % (8 hours) Linux Heartbeat with Shared Storage (Active/Active)
9 9. 9 0 0 % (52 minutes) Linux Heartbeat with DRBD and Replication
9 9. 9 0 0 % (52 minutes) Linux Heartbeat with Shared Storage and Replication
9 9. 9 9 9 % (5 minutes) MySQL Cluster
9 9. 9 9 9 % (5 minutes) MySQL Cluster & Replication
9 9. 9 9 9 % (5 minutes) MySQL Cluster Carrier Grade Edition
[Time is possible outage per year]
20
26
© 2010/2011 Pythian
MySQL Backup • Logical MySQL Backup
• mysqldump
• export
• Physical MySQL backup
• LVM Snapshot
• ZFS snapshot
• File copy
• Hot Backup (xtrabackup)
• Incremental backup
– Xtrabackup
– binlogs
20
27
© 2010/2011 Pythian
Managing MySQL MySQL tools
• Client application
• Non GUI
• mysql — The MySQL Command-Line Tool
• mysqladmin — Client for Administering a MySQL Server
• mysqlcheck — A Table Maintenance Program
• mysqldump — A Database Backup Program
• mysqlimport — A Data Import Program
• mysqlshow — Display Database, Table, and Column Information
• GUI
• Workbench; Navicat; SQLyog … Google
20
28
© 2010/2011 Pythian
Managing MySQL Administration tools
• Non GUI • innochecksum — Offline InnoDB File Checksum Utility
• myisam_ftdump — Display Full-Text Index information
• myisamchk — MyISAM Table-Maintenance Utility
• myisampack — Generate Compressed, Read-Only MyISAM Tables
• mysqlaccess — Client for Checking Access Privileges
• mysqlbinlog — Utility for Processing Binary Log Files
• mysqldumpslow — Summarize Slow Query Log Files
• mysqlhotcopy — A Database Backup Program (Archive MyISAM)
• perror — Explain Error Codes
20
29
© 2010/2011 Pythian
Monitoring MySQL Status • MySQL enterprise monitor
• Historical monitor
• Cacti
• MONyog
• Live monitoring
• What we do at Pythian
Avail software for monitoring of paging/alerts and daily pro-active checks
• Scripts
• Nagios plugin
• Issue is: define the correct parameters to monitor
20
30
© 2010/2011 Pythian
Use the right tool for the right use What MySQL is good for • OTLP
• Small/mid size datasets (per instance) say less then 10Tb
• Applications using many simultaneous connections
What is better to avoid
• Data warehouse
• Applications requiring heavy usage of Stored Procedure
• Applications requiring Hierarchical query (connect by prior etc.)
• Datasets that could not scale by SHARD
20
31
© 2010/2011 Pythian
Q & A
Thank you! [email protected] http://www.pythian.com/ http://www.pythian.com/news/
Interested in joining our elite team of trusted database expert? Pythian is hiring.
More details at: http://www.pythian.com/about/careers/job-listings/
20