Selected Presentations, MySQL Users Conference 2005€¦ · Selected Presentations MySQL Users...

1 From conferences.oreillynet.com/pub/w/35/presentations.html 23 July 2005

April 18-21, 2005 at the Santa Clara Convention Center Santa Clara, California, USA

Selected Presentations MySQL Users Conference 2005

Keynotes

State of MySQL AB, Mårten Mickos MySQL CEO Mickos gives an update on how MySQL AB is doing: a view behind the scenes of the business, the vision, the phenomenal growth, and some of the plans for the future.

Closing Session, Tim O'Reilly

Sessions

Flagship Features in MySQL 5.0, Trudy Pelzer Views, stored procedures, and triggers have long been considered a basic requirement of an enterprise-ready DBMS. Now, MySQL 5.0 introduces support for these flagship features, as well as for a standard SQL-compliant INFORMATION_SCHEMA data dictionary, bringing the popular open-source DBMS several steps closer to matching all the capabilities of the competition. This presentation provides a brief overview of MySQL's implementation of views, stored procedures, triggers, and INFORMATION_SCHEMA, with special emphasis on the extensions to the standard SQL requirements which have been added. Topics include: View Concepts:

• Why are views important? • View syntax (creating, dropping, using views) • Privileges: who can create a view, who can use a view?

Stored Procedures Concepts:

• Stored procedure syntax (creating, dropping, using stored procedures) • Using cursors in a stored procedure


Triggers Concepts:

• Trigger syntax (creating, dropping, using triggers) INFORMATION_SCHEMA Concepts:

• What does INFORMATION_SCHEMA do for me? • How to access your metadata

Other Changes:

• Precision math: now 1/.1 is always 10 • Strict error checking: how to make MySQL reject invalid data

MaxDB - That's Cool! Learn Why, Jörg Hoffmeister and Ashwath Kakhandiki MaxDB by MySQL is an enterprise level database with a long tradition. It's capable of running SAP R/3, it offers a unique object technology (liveCache) for the use with SAP's supply chain management, and is successfully used in Data Warehouses. MaxDB has been constantly improved over the years to meet today's business requirements: High Performance, High Availability, and low TCO in terms of administrative costs, hardware requirements, and license costs. Many years of development have lead to exciting technological solutions.

MaxDB is free of any online reorganizations. No special administrative tasks need to be performed that lower the performance of the running database. MaxDB ensures ongoing optimal performance and efficient space usage without extra work for the database administrator even for years of uninterrupted operation.

A unique Converter technology is the key to Snapshots and full featured hot backup with a low performance impact on the running system. Standby- and Hot-standby capabilities are available to ensure High Availability. Existings tools like the Loader have been further improved in the new version 7.6 and the SynchManager, a tool to synchronize and replicate datasources, is on the horizon.

Join this session and learn what's so special and "cool" on the MaxDB Enterprise database.

MySQL in Mission Critical Enterprise Applications - Case: SAP, Patrik Backman MySQL is rapidly maturing towards fulfilling the requirements of mission critical enterprise applications. MySQL is in the process of porting the database for SAP's main application stack, i.e. for systems that are key to the business operations of many major global enterprises.

The SAP porting process has driven the development of several key features coming in the MySQL 5.0 release. In the SAP project, these features and the server overall, has been put to heavy tests already since early stages.

This session looks at the MySQL server development driven by the requirements of enterprise applications, and presents an overview of what other development work has


been done and needs to be done, to get MySQL to support the SAP application stack. The various testing procedures that are carried out in order to get MySQL quality assured and performant for SAP are discussed, and we take a first look into results from the work. The SAP project is one demonstration on how MySQL is closing in on major database vendors, and this session will show how MySQL stands ready to run business critical applications.

Wikimedia, MySQL, and Free Software, Jimmy Wales Since 2001, the Wikipedia encyclopedia project has jumped from a scratchpad side project to one of the top 300 sites on the web (Alexa stats), bringing community and media attention to both wikis and freely-licensed content.

Phenomenal growth in the editing community, non-editing visitors, and the number of topics covered has put continual social and technical pressure on the scalability of the system. The wiki engine MediaWiki has grown up along with the project, following the sometimes-conflicting paths of being both easy to install and use, and performing reasonably well in a multimillion hit per day environment.

Built in the scripting language PHP, MediaWiki attempts to boost performance with "alternate hard and soft layers" of code: most hits to Wikipedia are actually handled by a Squid reverse proxy cache which is faster than any PHP script could be. Pre-parsed page chunks and dynamically editable user interface data are optionally cached using Livejournal's distributed memory object cache memcached or the compatible disk-backed tugelacache. Ongoing work is being put into accelerated native-code diff, parsing, and Unicode normalization modules, while still retaining compatibility with "pure" PHP code for use in more restricted environments.

This talk focusses on how free software has helped Wikipedia achieve its goals, and how MySQL in particular has played an intergral part in its success.

Birds Of a Feather

XML/XPath Support in MySQL, Alexander Barkov Alexander Barkov will introduce XML/XPath support, which is will be incorporated into MySQL sources soon, but currently is not a part of of the official source distribution. Attendees will learn how to create an XML database, use XPath language to fetch and update XML parts, optimize XPath queries.

SCALE OUT!

Mårten MickosCEO, MySQL AB

SCALE OUT!

OR

How to build for today and tomorrow

SCALE OUT!

OR

How to build for today and tomorrow

+ 2 lessons in Swedish

Agenda

Industry observationsOpen source observationsState of MySQL ABScaleoutConcluding remarks & prizes

Software Industry Growing Up

19751980 1990 2000

200519951985

MF C/S WEB 1.0 WEB 2.0MINI

20102015

Database Need

LEGACY DBMS

COMMODITY DBMS

20051995

WEB 1.0 WEB 2.0

Application Evolution

TranxApps

FunctionalApps

CompositeApps

Busi

ness

Evo

lutio

n

Technology Evolution

SOAC/SMF

Operational Efficiency

Organizational Agility & Alignment

TransactionAutomation

Standardisation

Proprietary =>

Database

cost

volume

Standardising => Standard

ERP Web server

Browser

Economic Realities

New Projects

Operations & Maintenance

Shrinking IT Budgets

$

t

Lesson in Swedish #1

Allemansrätten= the right of public access= the possibility for each and everyone to

visit somebody else's land, to take a bath in and to travel by boat on somebody else's waters, and to pick wild flowers, mushrooms, and berries

1000

10

Proportions

• 1 paying customer

• 10 contributing users

• 1000 nonpaying users

1

Moore’s Law?

Every 18 months, the number of transitions to open source doubles.

Community & Customer

1. First community:Those who spend time to save money

3. Then customers:Those who spend money to save time

Differentiation – drives the compelling reason to buy (2), – but may cause a ”compelling reason to abandon” (1)

Commercialisation

Profit is a beautiful word.

IBM ECLIPSE

Red Hat Fedora

JBoss JBoss

Freedom is a beautiful word.

Freedom Profit

“Selling support for free software is a legitimate ethical activity.

As regards these details of how you sell the support, as far as I'm concerned, you should choose them so as to maximize your income.”

Richard M Stallman, February 2005

Four Cultures

Cultivation Culture Competition Culture

Control CultureCollaboration Culture

Google Microsoft

Open Source IBM

Source: Geoffrey Moore, OSBC 2005

What We Do For You

Free software– opportunity to use and modify great software

Community– opportunity to learn, share and collaborate

Ecosystem– opportunity to earn money

Enterprise service– MySQL Network

What You Can Do For Us

Bugs– find them, identify them, report them

Contribute– APIs, tools, utilities, extensions, addons, patches,

storage engines, apps, etc.Marketing

– introduce us to your corporations– spread the message

Business– buy MySQL Network

2. RELIABILITY

3. EASE OF USE

1. PERFORMANCE

3 Software Priorities

MySQL, Sun and BEA WebLogic Set New World Records for Speed & Price/Performance in SPEC Benchmarks

Study: Few bugs in MySQL database.

By Robert Lemos CNET News.com Feb 4, 2005

15 Minute Rule

Download, install and have MySQL up and running in just a quarter-hour.

Building Database Excellence

MaxDB Cluster

MySQL v3

ENTERPRISE WEB CARRIERGRADE

v5

v4

New Offering

MySQL Community Edition release early & often bleeding edge do it yourself

MySQL Network

certified binaries maintenance updates update advisor technical alert advisor knowledge base production support consultative support warranties & indemnification

Airline Analogy

Crew Economy Business 1st Flight Deck

Microsoft

ORACLE

IBMEveryone in the aircraft

will reach the same destination at the same time.

MySQL

Open Source Evolution

LinuxMySQL ApacheSugarCRM

Scale Out

MySQLLEGACY DB

MySQL

MySQL

MySQL

MySQL

MySQL

MySQL

MySQL

MySQL

MySQL

MySQL

MySQL

MySQL

commodity hardware

commodity software stack

scale as you go

Lesson in Swedish #2

Lagom= not too little, not too much= in appropriate proportions= welltempered= both yes and no= whatever makes sense in the situation

Scale Up = Pay Up

too much pain

load

too much pay

too much pay

too much pain

scal

e up

scal

e up

Scaleout Examples

Cox Communications– 10m users with cable modems– 14 regional centers with MySQL Cluster– 35 MySQL databases on MyISAM– 9,000 inserts per second

Sabre Holdings– 45 servers, growing to 200+– 300,000 transactions per hour– maintaining existing OLTP system

MAKING STACKS SCALE OUT

.NET J2EE LAMP

When To Scale Out

Consider a scaleout architecture when:– you need to respond to rapid changes in demand– you are moving towards a serviceoriented architecture– cost is an issue– performance and low latency are critically important

10 Year Celebration

Announcements• MySQL 5.0 beta2 Mon 18 Apr 2005 • Migration suite & methodology• Business Objects partnership• Red Hat scaleout solutions• Ingram distribution• Exhibitor announcements

Partners of the Year

Embarcadero Technologies

GoldenGate Software

Quest Software

Applications of the Year

CNET Networks

Friendster

Wikipedia

Thank you!

[email protected]

1

The FutureBelongs to Data

Tim O’Reilly

O’Reilly Media, Inc.www.oreilly.com

MySQL ConferenceApril 21, 2005

How many of you have O’Reilly books?

2

So you think you know what we do…

What We Really Do At O'Reilly

•Find interesting technologies and peopleinnovating from the edge

•Amplify their effectiveness by spreadingthe information needed for others tofollow them.

"The future is here. It's just notevenly distributed yet."

--William Gibson

News from the Future

3

O’Reilly Radar

radar.oreilly.com

Radar O’Reilly

Pattern Recognition

4

(Control by API)

Desktop ApplicationStack

Proprietary Software

Hardware Lock InBy a Single-Source Supplier

System Assembled fromStandardized

Commodity Components

Free and Open Source Software

Cheap Commodity PCs

Intel Inside

ProprietarySoftware As a Service

Subsystem-Level Lock In

Integration of CommodityComponents

Internet ApplicationStack

Apache

The New "Killer Apps"

5

What Makes Them Interesting To Me• The Internet, not the PC, is their platform• Built on top of open source, but notthemselves open source

• Services, not packaged applications• Exploring how to become platform playersvia web services APIs

• Data aggregators, not just software• Network effects from user contributions keyto market dominance

• The most successful are “semantic learningsystems”, leveraging implicit metadata

Infoware, not software

Web 2.0

Design Patterns for Internet Applications

6

Format of Design Patterns

• Brief statement of situation or problem, andthe context in which it arises.

• Examples and discussion that demonstratethe generality of the situation.

• A prescription that expresses the solution.

Design for ParticipationThe most successful open source projectsconsist of “small pieces loosely joined.”

Discussion: Linux, Apache, Perl and CPAN,vs. OpenOffice, Sendmail, Mozilla

Therefore: Architect your software in such away that it can be used easily as acomponent of a larger system. Use alicense that does not hinder such acombination. Keep your software modular,and make certain that you document all ofthe interfaces.

User-Centered Development

The benefits of open source come fromsharing your development efforts andprocesses with your users

Therefore: Release early and release often.Set up mechanisms for users to submit bugsand patches. Promote your most activeusers into roles of greater responsibility.

The Perpetual Beta

When devices and programs are connectedto the internet, applications are no longersoftware artifacts, they are ongoing services.

7

The Perpetual Beta

When devices and programs are connectedto the internet, applications are no longersoftware artifacts, they are ongoing services.

Therefore: Do not package up new featuresinto monolithic releases, but instead addthem on a regular basis as part of thenormal user experience. Engage your usersas real-time testers, and instrument theservice so that you know how people usethe new features.

Users Add Value

The key to competitive advantage in internetapplications is the extent to which users addtheir own data to that which you provide.

Amazon - JavaScript BN - JavaScript

8

Flickr tags Users Add Value

The key to competitive advantage in internetapplications is the extent to which users addtheir own data to that which you provide.

Therefore: Don’t restrict your “architecture ofparticipation” to software development.Involve your users both implicitly andexplicitly in adding value to your application.

MapQuest Network Effects by Default

Only a small percentage of users will go tothe trouble of adding value to yourapplication.

Therefore: Set inclusive defaults foraggregating user data as a side-effect oftheir use of the application.

9

Flickr defaults Software Above the Level of a SingleDevice

The PC is no longer the only access devicefor internet applications, and applicationsthat are limited to a single device are lessvaluable than those that are connected.

iPod - software above the level

iTunes

• Software above thelevel of a single device• Database back end• Web services-enabled (CDDB)• Rich client front ends• Mobile device support• Rendezvous-enabled

• Not yet collaborative• No "architecture ofparticipation"

Flickr email/blog (or cameraphone)

10

Software Above the Level of a SingleDevice

The PC is no longer the only access devicefor internet applications, and applicationsthat are limited to a single device are lessvaluable than those that are connected.

Therefore: Design your application from theget-go to integrate services across handhelddevices, PCs, and internet servers.

Data is the Next “Intel Inside”

Applications are increasingly data-driven.

Therefore: Owning a unique, hard-to-recreate source of data may lead to an Intel-style single-source competitive advantage.

Google Maps/Navteq Google Maps again

11

Own the Namespace

Some data is a commodity and impossible forany one party to own, but access to the datacan be controlled through legal means.

Therefore: If you can’t own the data, own the namespace or registry for the data.

Syndication and Web Services

On today’s web, there is no need to developall the elements of your own application.

Therefore: Offer web services interfaces andcontent syndication, and re-use the dataservices of others.

Google Maps + Craig’s List Safari

12

Safari as Content Database CodeZoo

www.codezoo.com

Buzz game

13

Buzz taxonomy Flickr Buzz Volume Before Acquisition

http://blog.commerce.net/archives/2005/04/flickr_move_for_1.html

Visualization

• O’Reilly marketresearch

• Flickr ColorWheel

• Baby NameVoyager

For more information

http://www.oreilly.comhttp://tim.oreilly.comhttp://conferences.oreilly.comhttp://www.oreillynet.comhttp://radar.oreilly.comhttp://www.web2con.com

Flagship Features In MySQL 5.0

Presented by:

Trudy Pelzer, Senior Software Architect

MySQL [email protected]

MySQL Users Conference, 2005

April 18-21, 2005

2005-04-19 5.0 Features | © MySQLAB 2005 | www.mysql.com3

Interruption Policy

Any questions?

Interrupt or wait till the end.


The Flagship Features

• Views• Stored Procedures• Triggers• INFORMATION_SCHEMA


The Flagship Features

• Views• Stored Procedures• Triggers• INFORMATION_SCHEMA

Also:• Precision math• Strict data type checking


Views

A view is a virtual table.• It provides another way of looking at

existing data.

Advantage:• A view uses up no disk space.


Views: New DDL Statements

• CREATE VIEW• ALTER VIEW• DROP VIEW


CREATE VIEW Syntax

CREATE [OR REPLACE][ALGORITHM =

{MERGE | TEMPTABLE | UNDEFINED}]VIEW view_name[(column_list)]AS select_statement[WITH [CASCADED | LOCAL] CHECK OPTION]

CREATE VIEW view1 ...


CREATE VIEW Syntax



CREATE OR REPLACE view1 ...


CREATE VIEW Syntax




Two Algorithms for Views (I)

Merge algorithm:

SELECT * FROM v;

becomes

SELECT * FROM t;


Two Algorithms for Views (II)

Temporary table (temptable) algorithm:

SELECT * FROM v;

becomes

CREATE TEMPORARY TABLE temp_tableAS SELECT * FROM t;

SELECT * FROM temp_table;


Letting MySQL Choose theRight Algorithm (I)

CREATE TABLE t (col1 INT);

CREATE VIEW v AS SELECT col1 FROM t;-- MERGE algorithm will be used


Letting MySQL Choose theRight Algorithm (II)

CREATE TABLE t (col1 INT);

CREATE VIEW v AS SELECT AVG(col1) FROM t;-- TEMPTABLE algorithm will be used


Letting MySQL Choose theRight Algorithm (III)

TEMPTABLE algorithm for view definitions with:• DISTINCT• Aggregate function• UNION• GROUP BY• HAVING• Only literal values


Overriding MySQL byChoosing the Algorithm

CREATE ALGORITHM = TEMPTABLE VIEW vAS SELECT * FROM t;


CREATE VIEW Syntax




Omitted [ (column list) ]

By default:

CREATE VIEW v AS SELECT col1, col2 FROM t;

mysql> SELECT * FROM v;+------+------+| col1 | col2 |+------+------+| 10 | 20 |+------+------+


Included [ (column list) ]

Different names specified:

CREATE VIEW v (view_col1, view_col2) ASSELECT col1, col2 FROM t;

mysql> SELECT * FROM v;+-----------+-----------+| view_col1 | view_col2 |+-----------+-----------+| 10 | 20 |+-----------+-----------+


CREATE VIEW Syntax




View Queries can Includeany SELECT Clause

CREATE VIEW v ASSELECT col1 FROM t WHERE col1 > 4;

SELECT * FROM v WHERE col1 < 6;

same effect as:

SELECT * FROM tWHERE col1 > 4 AND col1 < 6;


CREATE VIEW Syntax




WITH CHECK OPTIONCREATE VIEW v1 ASSELECT col1 FROM t WHERE col1 <> 'A'WITH CHECK OPTION;

mysql> INSERT INTO v1 VALUES ('A');ERROR 1369 (HY000): CHECK OPTION failed 'db.v1'

mysql> UPDATE v1 SET col1 = 'A';ERROR 1369 (HY000): CHECK OPTION failed 'db.v1'


WITH LOCAL CHECK OPTION

CREATE VIEW v2 ASSELECT * FROM v1 WHERE col1 <> 'B'WITH LOCAL CHECK OPTION;

mysql> INSERT INTO v2 VALUES ('B');ERROR 1369 (HY000): CHECK OPTION failed 'db.v2'

mysql> INSERT INTO v2 VALUES ('A');Query OK, 1 row affected (0.01 sec)


WITH CASCADEDCHECK OPTION

CREATE VIEW v3 ASSELECT * FROM v2 WHERE col1 <> 'C'WITH CASCADED CHECK OPTION;

mysql> INSERT INTO v3 VALUES ('C');ERROR 1369 (HY000): CHECK OPTION failed 'db.v3'-- because v3’s condition doesn’t allow 'C'




mysql> INSERT INTO v3 VALUES ('B');ERROR 1369 (HY000): CHECK OPTION failed 'db.v3'-- because v2's condition doesn't allow 'B' andv3 depends on v2




mysql> INSERT INTO v3 VALUES ('A');ERROR 1369 (HY000): CHECK OPTION failed 'db.v3'-- because v1's condition doesn’t allow 'A' andv3 depends on v1 through v2


GRANT/REVOKE:New Privileges

GRANT CREATE VIEW ON *.* TO peter;

GRANT SHOW VIEW ON *.* TO peter;

REVOKE CREATE VIEW ON *.* FROM peter;

REVOKE SHOW VIEW ON *.* FROM peter;


Privileges Needed to Use a View

General rule:• Same as for tables.

Special rule:• If you have privilege on a view, you

don't need a privilege on its underlyingtables.


Updatable Views

What is an updatable view?• Can accept UPDATE, DELETE• Can sometimes accept INSERT

Effect:• As if you changed the data in the

underlying base table


Using Views

What are views for?


Use of Views (I)

1. Views can be used in place ofconstraint checks.

CREATE TABLE t (col1 INT, col2 INT);

CREATE VIEW v AS SELECT col1, col2FROM t WHERE(col1 is NOT NULL OR col2 IS NOT NULL)AND col1 BETWEEN 1 AND 5

WITH CHECK OPTION;


Use of Views (I)




WITH CHECK OPTION;


Use of Views (II)

2. Table Restructuring:Views can be used to hide or move columns.

CREATE TABLE t (col1 INT, col2 CHAR(5), col3 DATE);

CREATE VIEW v AS SELECT col3, col1 FROM t;

GRANT SELECT ON v TO some_user;


Use of Views (III)

3. Data Security:Views can be used to hide rows.

CREATE TABLE t (col1 INT, col2 CHAR(5), COL3 DATE);

CREATE VIEW v AS SELECT * FROM tWHERE col1 BETWEEN 10 AND 100;

GRANT SELECT ON v TO some_user;


Why Views (I)

A view is a virtual table.• So it doesn't take up extra disk space.

A view provides another way of looking atexisting data.• So you can effectively "reorganize" your

data.


Why Views (II)

MySQL's view syntax is nearly 100%standard SQL.• So porting your applications is easy.

The CREATE VIEW ALGORITHM clause providesthe option to specify how MySQL willprocess a view.• So you can tailor your views for the

most efficient processing.


Why Views (III)

A view's SELECT definition looks like anySELECT.• So if you know how to query your

database, you also know how to define aview.

A SELECT on a view is the same as aSELECT on a regular table.• So if you know how to query your

database, you also know how to query aview.


Why Views (IV)

The CREATE VIEW WITH CHECK OPTION clausedefines the data that belongs in a view.• So you have an extra layer of data

validity checks.

Users cannot create a view without theCREATE VIEW privilege.• So you can ensure that only users who

should be able to change the databasedefinition can do so.


Why Views (V)

Users cannot see information on underlyingtables without appropriate privileges.• So you can ensure sensitive information

is concealed.

Use of a view requires the same privilegesthat are needed to use a table.• So views let you tailor the actions

users may perform.


Why Views (VI)

Many views will allow INSERT, UPDATE, andDELETE.• So views are easy to use and -- because

they act just like regular tables --their nature can be hidden from users.


Stored Procedures

A stored procedure is a routine that isstored in the database.

Two kinds:• Procedures -- which are called with CALL• Functions -- whose return values are

used in SQL statements


Stored Procedures:New DDL Statements

• CREATE PROCEDURE/FUNCTION• ALTER PROCEDURE/FUNCTION• DROP PROCEDURE/FUNCTION


CREATE Syntax

CREATE PROCEDURE procedure_name([parameter [...]])[characteristic ...]routine_body

CREATE FUNCTION function_name([parameter [,...]])RETURNS data_type[characteristic ...]routine_body


Privilege Checking (I)

CREATE PROCEDURE p1 (param1 INT)LANGUAGE SQLDETERMINISTICSQL SECURITY DEFINER ...

CREATE FUNCTION f1 (param1 INT)RETURNS INTLANGUAGE SQLNOT DETERMINISTICSQL SECURITY INVOKER ...


Privilege Checking (II)

CREATE PROCEDURE p1 (param1 INT)LANGUAGE SQLDETERMINISTICSQL SECURITY DEFINER ...

CREATE FUNCTION f1 (param1 INT)RETURNS INTLANGUAGE SQLNOT DETERMINISTICSQL SECURITY INVOKER ...


Routine Body

Routine body:

• Specifies what the stored procedure will do

• Can contain one or more SQL statements


Compound Statements

Syntax:[begin_label:]BEGIN [statement_list] END[end_label]

CREATE PROCEDURE p1 ()BEGINSELECT * FROM t;

END; //


BEGIN … END Statement List

Can include:• INSERT• UPDATE• DELETE• SELECT• REPLACE• SET• COMMIT / ROLLBACK...


BEGIN … END:Also Allows DECLARE

[begin_label:] BEGIN[DECLARE variable_list][DECLARE conditions][DECLARE cursors][DECLARE handlers][statement_list]

END [end_label]


DECLARE Variables

Syntax:DECLARE variable_name [,...]data_type [DEFAULT value];

CREATE PROCEDURE p1 ()BEGINDECLARE v1 INT DEFAULT 10;SET v1 = 5;INSERT INTO t VALUES (v1);

END; //


DECLARE Conditions

Syntax:DECLARE condition_name CONDITION FOR{SQLSTATE [VALUE] 'sqlstate' |mysql_error_code};

CREATE PROCEDURE p1 () BEGINDECLARE v1 INT DEFAULT 10;DECLARE syntax_error CONDITION FORSQLSTATE '42000';

SELECT col1 INTO v1 FROM t LIMIT 1;END; //


DECLARE Handlers

Syntax:DECLARE {CONTINUE | EXIT} HANDLER FOR

condition_value [,...] action;

action:• Is an SQL statement.• The statement is executed only if the

condition_value is returned by theserver.


DECLARE Cursors

Syntax:DECLARE cursor_name CURSOR FOR select_query;

CREATE PROCEDURE p1 () BEGINDECLARE v1 VARCHAR(7) DEFAULT 'OK';DECLARE syntax_error CONDITION FORSQLSTATE '42000';

DECLARE c1 CURSOR FOR SELECT * FROM t;DECLARE CONTINUE HANDLER FOR SQLWARNINGSET v1 = 'Warning';... ; //


Flow Control

• IF -- if <condition> do <action>• CASE -- switch depending on condition• LOOP -- do <action> repeatedly• LEAVE -- break out of a loop or block• ITERATE -- restart loop• REPEAT -- repeat <action> until

<condition> is true• WHILE -- repeat <action> as long as

<condition> is true


Why Stored Procedures

• Stored procedures are proven technology• Stored procedures are fast• Stored procedures are components• Stored procedures are portable• Stored procedures are stored• Stored procedures are migratory


Use of Stored Procedures:Summary

Use stored procedures to:• improve performance• save time, money, resources


Triggers

A trigger is a named chain reaction thatstarts when a specific event occurs fora specific table.


Triggers: New DDL Statements

• CREATE TRIGGER• DROP TRIGGER


CREATE TRIGGER Syntax

CREATE TRIGGER trigger_name{BEFORE | AFTER}{INSERT | UPDATE | DELETE} ON table_nameFOR EACH ROW trigger_action

CREATE TRIGGER tg1BEFORE DELETE ON t1FOR EACH ROW SET @x = 1;


CREATE TRIGGER Syntax

CREATE TRIGGER trigger_name{BEFORE | AFTER}{INSERT | UPDATE | DELETE} ON table_nameFOR EACH ROW trigger_action

CREATE TRIGGER tg1AFTER INSERT ON t1FOR EACH ROW <compound SQL statement>;


Use of Triggers (I)

1. Log/audit/monitor data-change activity.

CREATE TRIGGER tg1 AFTER UPDATE ON t1FOR EACH ROW<record who did it, when it was done,what row was changed, what value waschanged>;


Use of Triggers (II)

2. Data validation.Change or negate INSERT, UPDATE, DELETEon a table.

CREATE TRIGGER tg1 BEFORE UPDATE ON t1FOR EACH ROW<don't allow the update under specificconditions>;


Use of Triggers (III)

3. Consistency and clean-up.Perform related actions on related tables.

CREATE TRIGGER tg1 AFTER DELETE ON t1FOR EACH ROW<delete corresponding rows from relatedtables>;


INFORMATION_SCHEMA

The INFORMATION_SCHEMA is the standard SQLway to access database metadata.

• A series of tables that describedatabase objects.

• Query them with SELECT.


Why INFORMATION_SCHEMA

Disadvantages of SHOW:

• SHOW commands are non-standard; they arespecific to MySQL.

• SHOW commands require that you learn anentire set of commands to be able toaccess the metadata you need.


Why INFORMATION_SCHEMA

Advantages of INFORMATION_SCHEMA:

• INFORMATION_SCHEMA is standard SQL, soother DBMSs have it too.

• INFORMATION_SCHEMA can be queried via aSELECT statement, so there is no need tolearn a new set of commands to be ableto access the metadata you need.


SELECT ... FROM ...mysql> SELECT table_name, table_type, engine

-> FROM INFORMATION_SCHEMA.tables-> WHERE table_schema = 'tp'-> ORDER BY table_type, table_name DESC;

+------------+------------+--------+| table_name | table_type | engine |+------------+------------+--------+| t2 | BASE TABLE | MyISAM || t1 | BASE TABLE | InnoDB || v1 | VIEW | NULL |+------------+------------+--------+


Using INFORMATION_SCHEMA

• INFORMATION_SCHEMA is a new, virtualdatabase.

• There is no need to create anINFORMATION_SCHEMA file.

• MySQL creates and populates the tablesautomatically.

• Only SELECT is allowed.


Privileges

Easy and automatic:• SELECT on every INFORMATION_SCHEMA table

is automatically granted to every user.

Secure:• You can see only the metadata that

applies to you.


What CharacterSets/Collations Can I Use?

• CHARACTER_SETS

• COLLATIONS

• COLLATION_CHARACTER_SET_APPLICABILITY


What Objects Can I Use?

• SCHEMATA

• TABLES

• VIEWS

• ROUTINES


What Objects Can I Use?

• COLUMNS

SELECT * FROM INFORMATION_SCHEMA.COLUMNS ...**************** 1. row ****************

TABLE_CATALOG: NULLTABLE_SCHEMA: tpTABLE_NAME: t1COLUMN_NAME: col1

...


What Indexes Can I Use?

• STATISTICS

• TABLE_CONSTRAINTS

• KEY_COLUMN_USAGE


What Privileges Are Granted?

• USER_PRIVILEGES

• SCHEMA_PRIVILEGES

• TABLE_PRIVILEGES

• COLUMN_PRIVILEGES


Also New: Precision Math

New math library means:

• DECIMAL/INTEGER arithmetic is done withexact numeric, not floating-point,calculations.

• Now 1/.1 is always 10.


Also New: Strict Mode (I)

mysql> CREATE TABLE t1 (-> col1 TINYINT, col2 CHAR(3));

Query OK, 0 rows affected (0.06 sec)

mysql> SET SQL_MODE='TRADITIONAL';Query OK, 0 rows affected (0.00 sec)


Also New: Strict Mode (II)

mysql> INSERT INTO t1 (col1) VALUES (300);ERROR 1264 (22003): Out of range valueadjusted for column 'col1' at row 1

• TINYINT range is -128 to +127.


Also New: Strict Mode (III)

mysql> INSERT INTO t1 (col2) VALUES ('abcd');ERROR 1406 (22001): Data too long for column'col2' at row 1

• Column's maximum size is 3 characters.

mysql> select * from t1;Empty set (0.00 sec)


Thank You!

More information:forums.mysql.com

Joerg Hoffmeister MaxDB – That’s Cool 1


MaxDB – That’s Cool !

Joerg Hoffmeister, SAP AGGermany

MySQL Users Conference, 2005April 18-21, 2005


What‘s Cool ?

Jazz is coolSome people are cool

Databases are boring – but necessary

Software is boring – and necessary too (in some cases ;-))

But single features can really be cool!


Some of MaxDB‘s Cool Features

… HA scenarios

… a complete set of backup functionality

… database snapshot functionality

… a synchronization manager

MaxDB has

… no needs for reorganization


No Need for Reorganisation


No Reorg – Basics

PREREQUISITESSpace no longer needed must immediately be returned to the

space management (database)The occupation ratio of the data blocks should be held at the

highest possible levelData must be stored dense without gaps that would have to be

managed.

ACCESSOIRESShadowing pages with our ConverterAutomatic space- and I/O management


No Reorg – Basics

Handling SpaceUpdate in placeSort by insertionDelete in place

Storing DataB*Trees for almost all database items:

TablesIndexes (secondary keys)BLOBs


B*- Trees

...

...

...

...

K I L ...

...

...

...

K: Key ColumnI: Indexed ColumnL: BLOB Column

Primary Data Index Data

BLOB data

...

...

...

...


I/O-Concept

Concurrent asynchronous I/0Shadow page concept

ConverterManaging free blocksSavepointsBackup Integration

Datawriter and server tasksSegmentation of the data cache


Shadowing

Storage universe

READ

MODIFY

DATA ITEM

Cached universe

ORIGIN

COPY

Backup and Snapshot are pointersthat will stay pointing to the originif a copy is established

Backup

SNAPSHOT


SAVEPOINT

Storage universe

Ex-ORIGIN

COPY

ORIGIN

Backup

SNAPSHOT

ControlledData flush

Supported through write aheadtasks andconcurrentserver tasks


Converter, Free Block Mgmt, Savepoints

Converter maps current location(s) of data blocksKnows where block images are located in storage

Free block management provides locations to converterKnows about available blocks,when blocks can be released,how to combine blocks for optimized I/O

Savepoints synchronize I/O managementScheduled every 10 minutes (if enough volume appears),supported by 'write ahead' tasks (data writer)


Redundancy

Blocks are short term kept twice if modifiedCopy becomes origin at savepoint. Ex-origin is releasedexcept for backup or snapshot reasons.

Blocks are short term kept twice if modified and backed upBlocks are released when backup has passed them

Blocks are long term kept twice if modified and ‚snapshotted‘Blocks are released when snapshot is dropped


Backup/Recovery


Backup

Every MaxDB data backup is consistent and could berestarted even without recovering log (e.g. unsharp copies)

A savepoint is issued before the backup startsThe data belonging to this savepoint additionally containsall undo information for remaining open transactions. Thuseach data backup is consistent in terms of transactions.

The savepoint checks the converter, detects those blocksrelevant for the backup and marks the backup flag for thoseblocks in the free block manager (FBM)

A backup is executed by concurrent server tasks alonginformations of the free block managment.


Backup Wizard


Recovery Wizard


tape_serv

8x8k-Blocks

ConcurrentBackup

tape_serv

dev_serv

DATA DATA

dev_serv

DATA

dev_serv

Cyclic Buffer


LOG1

PAG2PAG1

DAT2 DAT3

LOG2 LOG8LOG4 LOG5 LOG6 LOG7

PAG3

LOG3 LOG9

DAT1

Backup TypesDAT# CompletePAG# IncrementalLOG# Log backup LOG9LOG9

LOG8LOG8LOG7LOG7LOG9

LOG6LOG6LOG8LOG9

LOG5LOG5LOG7LOG8LOG9

LOG4LOG4LOG6LOG7LOG8LOG9LOG3LOG3LOG5LOG6LOG7LOG8LOG9

LOG2LOG2LOG4LOG5LOG6LOG7LOG8

LOG1PAG1LOG3PAG2LOG5PAG3PAG4

DAT1DAT1DAT2DAT2DAT3DAT3DAT3

PAG4

time

Recovery pathvariety alongmultiple backupgenerations

Backup & Recovery Variety


Backup History


DatabaseManager

Guided Recovery


Snapshot


Snapshot

Freezing a database image

Create Snapshot (ADMIN)Revert to Snapshot (ADMIN)Drop Snapshot (ADMIN)

Ideas of use:Very fast point in time resetting (e.g. during upgrades)Restoring training-systems to a defined status


Master - Slave Support w/ Snapshots

Data 01.01.2005

Master

Data

Slave

Complete

Data 07.01.2005

::::

Create Snapshot

Incremental

Complete

Incremental

Restore Snapshot

Data 07.01.2005

::::

Data 14.01.2005 IncrementalIncremental

Restore Snapshot

Data 14.01.2005

:

:::


Snap Demo: Database after setup


Let's take a snapshot


Now some data has been loaded


Proof: Data is there


Return to admin: prepare reverting


Action: Revert to snapshot


After reverting to initial state


Standby database


Standby Database (w/ Log Shipping)

LogData

Master

Log DataLog

Backup

StagingDirectory

Standby

Initial data backup

Log Backup

Log Restore


Standby Database: Demo Example

LogData

MDMaster

Log Data

MDSlave

Initial data backup

C:\shadowmedia\MDMASTER_COM

Log Backup

Log Restore

MediaDefinitons

C:\S

hado

wM

edia


Standby Database: Demo ExampleMASTER

SLAVE

FILESYSTEM


Standby Database: Slave Steps

Standby ModeInitialize once with complete backup from MasterRedo logs as they appear

Start Slave to online mode in case of emergencyIf possible, back up last piece of log from masterRedo all 'open' log backups (there should be none)Redo final log pieceRestart slave to be the new master


Standby Database: Slave Steps live


Standby Database: Slave Steps live (2)


Standby Database: Slave Steps live (3)

These steps are required ifslave should be started


Standby Database online !


… but looks smaller ... ?


Indexes

Indexes dueto the time impact arenot createdimmediatelyduring log recovery. It isleft to theadministratorwhen this fitsto theschedule.


Now: Same Size !


Hot Standby – standby within seconds

Storage System

ArchiveLogData

Application

Data

DataAfterImages

Master Standby

continousRESTART

IP SWITCH

RECONNECT

Cluster


Synchronization Manager


MaxDB - Sychnronization Architecture

Client1DB

MasterDB

MessageDB

MessageServer

Client2DB

Synchronization Mgr GUI

SynchonizationService




MaxDB - Sync Manager Features

Replication of tables from a master database to client databases

Synchronization Manager decouples master and client DBMS's

Initial replication of the complete DBMS state

Delta replication of transactions

Point-to-point replications (queues)

Broadcast replications (publish/subscribe)

Support for bi-directional replications with conflict resolution

Admin tool to define replication scenarios

Supports MaxDB and MinDB


Example: Master table for Synchronization


SyncMan GUI – Replication Tables

Column Group 'District'

Column Group 'Country'


Vertical Projection: Bob's Client Tab

ColumnSelection


Horizontal Selection: Mona's Client Tab

All Columns Selection Constraint


SyncMan: Monitor

These tables are created on unit activation


Bob adds a record – but only to his columns


The view of the master shows NULL values


Mona updates the row's missing columns


Now this is the final master view


THE END

Thank you!Have a nice further conference

[email protected]


• No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.

• Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.

• Microsoft, Windows, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation. • IBM, DB2, DB2 Universal Database, OS/2, Parallel Sysplex, MVS/ESA, AIX, S/390, AS/400, OS/390, OS/400, iSeries,

pSeries, xSeries, zSeries, z/OS, AFP, Intelligent Miner, WebSphere, Netfinity, Tivoli, and Informix are trademarks or registered trademarks of IBM Corporation in the United States and/or other countries.

• Oracle is a registered trademark of Oracle Corporation.• UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.• Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered

trademarks of Citrix Systems, Inc.• HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium,

Massachusetts Institute of Technology. • Java is a registered trademark of Sun Microsystems, Inc.• JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and

implemented by Netscape. • MaxDB is a trademark of MySQL AB, Sweden.• SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver and other SAP products and services mentioned herein

as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.

• These materials are subject to change without notice. These materials are provided by SAP AG and its affiliated companies ("SAP Group") for informational purposes only, without representation or warranty of any kind, and SAP Group shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP Group products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.

MySQL in Mission Critical EnterpriseApplications – Case SAP

Patrik BackmanDirector of SAP Alliance

MySQL Users Conference, 2005April 18-21, 2005

Goals of the Session

• Look at requirements of Mission Critical EnterpriseApplications

• See what this means for the database

• Clarify how MySQL fit to these requirements, from anSAP point of view

• Present an overview of how MySQL is approaching torun SAP’s business solutions

What does a Mission CriticalEnterprise Application need?

• The solution must fulfil the technical requirements!

• The solution needs to be scalable!

• The processing needs to be fast!

• The platform must be secure and safe!

• The complete solution should be easy to handle!

• All parts should be as inexpensive as possible!

What does this mean for thedatabase?

The database needs:=> 1. Required features=> 2. Scalability=> 3. Performance=> 4. Reliability=> 5. Ease of use=> 6. Low TCO

• In the next slides we will look at each area, and seea) how MySQL fits the requirement, andb) how this is shown in the SAP project

The solution needs to:• fulfil technical requirements• scale• be fast• be secure and safe• be easy to handle• be inexpensive

1. Required features?

• Main SAP requirements: 4.1 5.0 5.1– Transactions x– Unicode handling x– Subqueries x– Server Side Cursors x– Views x– Stored Procedures x– Triggers x– Data Dictionary x– Precision Math x– Std Error Handling x– Data Warehousing x– Unicode feature extensions x

SAP assessment 1

• SAP has extensive feature verification tests

• Millions of statements and transactions testing their

functionality requirements

• With current 5.0 beta, MySQL passed all the tests!

• But this is only the beginning...

2. Scalability

• Concurrency– Lock behaviour– Architecture

• SAP has high requirements here:– Large 3-tier environments– Hundreds – thousands of concurrent users with heavy load

SAP assessment 2

• SAP has various tests to check scalability

• Simulating various kinds of end users, in large numbers

• With the current 5.0 beta, MySQL has proven to be able to handle

hundreds of concurrent simulated SAP users

• This indicates three important things:

1. The MySQL locking mechanism can handle high concurrency

2. The MySQL architecture can handle high concurrency

3. The MySQL architecture can appropriately use multiple processors

• However, these are preliminary tests and not real SAP benchmarks

3. Performance

• SAP requirements:– The database needs to have fast:

• Transactional processing for standard use

• Loading capability for installations

• Modification of table structures, indexes, etc for upgradeprocedures

SAP assessment 3

• In our SAP project the performance is measuredconstantly– Transactional processing seems to be very good

– Loading capability fulfils well the requirements

– On modification of table structures and indexes we are stilla bit slow• We are working on big improvements in ALTER TABLE(including CREATE INDEX)

– However: Large optimisation tests are still ahead!• We estimate a number of optimisations still to be required inMySQL!

4. Reliability

• How to prove MySQL Reliability?

– MySQL Code Quality!

• Reasoning Inc: ”6 times better than proprietary DBs”

– Real life cases of Mission Critical Installations!

• 50 case studies currently available from a large variety of

business areas, available from www.mysql.com

– Community popularity and credibility!

• 6 Million users!

5. Ease of use

• MySQL has always kept ease of use in focus

– 15 min rule!

• MySQL popularity builds on ease of use

– Web developers have traditionally picked the easiest choice

– MySQL technology has proven to be simple to embed in

various solutions

– MySQL is very popular in education worldwide

6. Low TCO

• Low TCO means:– License cost– Support cost– Low Downtime– Easy administration– Easy to learn

Promise:- ”We offer 90% of the features to 10% of the cost!”

Also true for MySQL Network!

Mission statement:- Make superior database technology available and affordable to all!

SAP Project overviewHow to get MySQL into a Mission Critical Enterprise Application?

SAP Project overviewHow to get MySQL into a Mission Critical Enterprise Application?

Database

SAP

NW

D

B

S

L

ComputingInfrastructure

Databaseabstractionlayer

SAPApps

”SQL””OpenSQL”(ABAP)

Project Goal 1:

Develop MySQL to fulfil alldatabase requirements

Project Goal 2:

”Port MySQL” – Build and adapt allintegration programs for MySQL

What is really the DbSl?

MySQL

InnoDB

DB Data Dictionary

DynamicSQL

DbSl

R3LOADR3LOAD LD Import

R3LOAD LD Export

R3LOAD CTL

R3SZCHK

TransportProgram

SynchronizationC Module

SAPLICENSEWeb

ASUpgrade R3UP

R3TRANS

SAPINST

SE11: ABAP DataDictionary

SDB1FMYS

SDB2FMYS

SBD2SMYS

CCMS/TCC-Monitors forAdministration

DB02 DB13

DB12 DB20

RZ20 ST04

ST05

Programs forconversions, migrations

and upgrades

SDB6PMYS SDB6SMYS

SDB6FMYS IMIGJDBI (Java layer)

Java Data Dictionary SAP BWissues

Blue: C-ProgramsRed: ABAP-ProgramsGreen: Java-Programs

SAP

R/3

SAP

NW

Project timeline

• 3 year project, divided in 12 milestones (onefor each quarter)

• We started in June 2003

• Todaywe have done 8/12 Milestones

MySQLdevelopment

Integrationwork

1 2 ... Milestones ... 8 ... ... 12

Summary• Mission Critical Enterprise Applications put a lot of needs on thedatabase:=> Required features => Scalability=> Perfomance => Reliability=> Ease of use => Low TCO

• For SAP we are rapidly fulfilling these requirements

• SAP Apps run the business critical solutions of the leadingcompanies throughout the world

=> As a result, we see less and less resistance in our customerbase, for using MySQL as database for mission critical enterprisesolutions

Wikipedia, MySQL and Free Software

Jimmy Wales

President, Wikimedia Foundation

Wikipedia Founder

“The ideal encyclopedia should be radical. It should stop being safe.”--1962, Charles van Doren, later a senior editor at Britannica

Wikipedia’s Radical Idea:

Imagine a world in which every single person is given free access to the sum of all human knowledge.That’s what we’re doing.

What is the Wikimedia Foundation?

• Non-profit foundation• Aims to distribute a free encyclopedia to

every single person on the planet in their own language

• Wikipedia and its sister projects• Funded by public donations• Applying for grants

wikimediafoundation.org

What is Wikipedia?

• Wikipedia is a freely licensed encyclopedia written by thousands of volunteers in many languages

• Free license allows others to freely copy, redistribute, and modify our work commercially or non-commercially

• Founded January 15, 2001wikipedia.org

Four Freedoms

The famous 4 freedoms of free software apply cleanly to reference works, fundamental information should be free

Wikipedia is the largest effort underway to make this dream a reality

Free Software

• MediaWiki is GPL

• We use all free software on the website

• GNU/Linux

• Apache

• MySQL

• Php

Neutral Point of View policy

• NPOV - Neutral Point of View

• Diverse political, religious, cultural backgrounds

• Kept together by our “NPOV” policy

• NPOV is a social concept of co-operation, avoids some philosophical issues.

How big is Wikipedia?

• English Wikipedia is largest and has over 130 million words

• English Wikipedia larger than Britannica and Microsoft Encarta combined

• In 15 months the publicly distributed compressed database dumps may reach 1 terabyte total size

How big is Wikipedia Globally?

• English – 533,000 articles

• German – 220,000 article

• Japanese – 110,000 articles

• French – 100,000 articles

• Swedish – 71,000 articles

• Nearly 1.5 million across 200 languages

• 20+ with >10,000. 50+ with >1000

How popular is Wikipedia?

• According to Alexa.com, Wikipedia is more popular than the websites of:

• Expedia• Paypal• Excite• Geocities• New York Times• ~500 Million pageviews monthly

Slashdotting

We used to worry about it, but now we are big enough to

barely notice…

Instead we worry about…

Popedotting

Wikimedia Projects

• Wikipedia

• Wiktionary

• Wikibooks

• Wikisource

• Wikiquote

• Wikispecies

• Wikimedia Commons

• Wikinews

We Need Your Help

• Buying servers is easy

• Growing web traffic is easy

• Getting new authors is easy

• Developing software is hard

• Managing server cluster is hard

Wikimedia’s Hardware

• 50+ servers

• Squid caching servers in front to serve cached objects quickly

• Apache/PHP webservers in the middle

• Database backend (MySql)

• 2 facilities now, soon to be 5-6 including donation of facility in Asia by Yahoo!

MediaWiki

• MySQL Application of the Year 2005• MediaWiki is one of many wiki engines• Collaborative software that allows users to add

or edit content• Primarily developed for Wikipedia from 2002

onwards• Scalable and multilingual• Free license

MediaWiki features

• Quality control features (versioning)

• Editing features (simple markup)

• Community features (talk pages, user pages, access levels)

Our use of MySQL

• We serve around a half billion pageviews per month

• 200 million queries per day• 1. 2 million changes per day• At peak times we handle nearly 6000 queries

per second• Using MySQL replication, Master + 4 Slaves + 1

for backup

Problems we have

• Our database schema is suboptimal but will improve in MediaWiki 1.5

• A few slow queries can sometimes slow the site, as performance on a box goes from 2500/s to 1000/s

• Replication is fragile - and if anything goes wrong we have to go read only and resync everything

Development Challenges

• Wiki text is freeform, but many types of data are better handled in a structured way

• Routine server administration by volunteers works o.k. now, but as our traffic continues to double we need help

• Unlike editing and reading, there is a learning curve

Development Challenges

• Unlike editing and reading, there is a learning curve

• We need people to start getting involved now before the need is critical

Conclusion

• Wikipedia is one of the most exciting developments in human history

• Wikipedia is a major force for free software• Wikipedia is competing directly with large

commercial companies• The biggest challenges for us are technical,

not social• Free software developers should take an

interest in Wikipedia

1Copyright 2005 MySQL AB The World’s Most Popular Open Source Database

XML/XPath Support In MySQL5.x

Alexander BarkovFull time developer

April, 2005

MySQL AB


Plan for this session

• Creating XML database• Querying XML data using XPath• Updating XML data• Optimizing XPath queries• Current status• Future development directions


Creating XML columns

An XML value can be stored in any textual data type:• TEXT

• VARCHAR

• CHAR

No special XML type at this moment.

mysql> CREATE TABLE t1 (xml TEXT);



Inserting into an XML column

mysql> insert into t1 select load_file('/home/bar/example.xml');

Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO t1 VALUES ('<a>a1<b>b1</b>b2</a>');


Using INSERT

Using INSERT with LOAD_FILE:


Fetching entire XML Values

mysql> select xml from t1;+----------------------+| xml |+----------------------+| <a>a1<b>b1</b>a2</a> |+----------------------+1 row in set (0.00 sec)


Fetching XML parts using XPath

mysql> select ExtractValue('<a>a1<b>b1</b>a2</a>', '/a/b');

+----------------------------------------------+

| ExtractValue('<a>a1<b>b1</b>a2</a>', '/a/b') |

+----------------------------------------------+

| b1 |

+----------------------------------------------+

Usage example:

New function:ExtractValue(xml, xpath)

Returns a part of XML value xml addressed by an XPath query xpath.


XPath implementation in MySQL

• According to W3C Recommendations• Available from http://www.w3.org/TR/xpath• Subset of XML Path Language Version 1.0• About 90% currently implemented


A more complex XML Example<section> <title>Choosing an Install Package</title> <para> There are three different packages available that you can use to install the MySQL server: </para> <itemizedlist id=”noinstall”> <listitem> <para><literal>mysqlversionwinnoinstall.zip</literal>: This is a Zip archive. It must be manuallyextracted. </para> </listitem> <listitem id=”full”> <para><literal>mysqlversionwin.zip</literal>: This package contains the InstallWizard with all components of MySQL and is intended for a full installation with all optional packages. </para> </listitem> <listitem id=”essential”> <para><literal>mysqlversionessentialwin.msi</literal>: This package contains the InstallWizard with the minimum components required to install a working MySQL server. Omitted packages can be added later. </para> </listitem> </itemizedlist> <para> For most users, the <literal>mysqlversionessentialwin.msi</literal> package is recommended because of its smaller size and decreased download time. </para></section>


Example#1: Simple query<section> <title>Choosing an Install Package</title> <para>...three different packages...</para> <itemizedlist> <listitem id=”noinstall”> <para><literal>....noinstall.zip</literal>: ...Zip archive....</para> </listitem> <listitem id=”full”> <para><literal>...win.zip</literal>: ...full installation with all optional packages...</para> </listitem> <listitem id=”essential”> <para><literal>...win.msi</literal>: ...minimum components...</para> </listitem> </itemizedlist> <para>...win.msi...package is recommended...because of its smaller size.</para></section>mysql> select ExtractValue(xml,'/section/title') from t1;

+-------------------------------------+

| ExtractValue(xml, '/section/title') |

+-------------------------------------+

| Choosing an Install Package |

+-------------------------------------+


Example#2: by index<section> <title>Choosing an Install Package</title> <para>...three different packages...</para> <itemizedlist> <listitem id=”noinstall”> <para><literal>....noinstall.zip</literal>: ...Zip archive....</para> </listitem> <listitem id=”full”> <para><literal>...win.zip</literal>: ...full installation with all optional packages...</para> </listitem> <listitem id=”essential”> <para><literal>...win.msi</literal>: ...minimum components...</para> </listitem> </itemizedlist> <para>...win.msi...package is recommended...because of its smaller size.</para></section>mysql> select ExtractValue(xml, '/section/para[1]') from t1;

+---------------------------------------+

| ExtractValue(xml, '/section/para[1]') |

+---------------------------------------+

| ...three different packages... |

+---------------------------------------+


Example#3: attribute<section> <title>Choosing an Install Package</title> <para>...three different packages...</para> <itemizedlist> <listitem id=”noinstall”> <para><literal>....noinstall.zip</literal>: ...Zip archive....</para> </listitem> <listitem id=”full”> <para><literal>...win.zip</literal>: ...full installation with all optional packages...</para> </listitem> <listitem id=”essential”> <para><literal>...win.msi</literal>: ...minimum components...</para> </listitem> </itemizedlist> <para>...win.msi...package is recommended...because of its smaller size.</para></section>mysql> select ExtractValue(xml,

'/section/itemizedlist/listitem[1]/@id')from t1;

+------------------------------------------------------------+

| ExtractValue(xml, '/section/itemizedlist/listitem[1]/@id') |

+------------------------------------------------------------+

| noinstall |

+------------------------------------------------------------+


Example#4: by value<section> <title>Choosing an Install Package</title> <para>...three different packages...</para> <itemizedlist> <listitem id=”noinstall”> <para><literal>....noinstall.zip</literal>: ...Zip archive....</para> </listitem> <listitem id=”full”> <para><literal>...win.zip</literal>: ...full installation with all optional packages...</para> </listitem> <listitem id=”essential”> <para><literal>...win.msi</literal>: ...minimum components...</para> </listitem> </itemizedlist> <para>...win.msi...package is recommended...because of its smaller size.</para></section>mysql> select ExtractValue(xml,

'/section/itemizedlist/listitem/@id[contains(../para/literal,"noinstall.zip")]')

as value from t1;

+-----------+

| value |

+-----------+

| noinstall |

+-----------+


Example#5: boolean AND<section> <title>Choosing an Install Package</title> <para>...three different packages...</para> <itemizedlist> <listitem id=”noinstall”> <para><literal>....noinstall.zip</literal>: ...Zip archive....</para> </listitem> <listitem id=”full”> <para><literal>...win.zip</literal>: ...full installation with all optional packages...</para> </listitem> <listitem id=”essential”> <para><literal>...win.msi</literal>: ...minimum components...</para> </listitem> </itemizedlist> <para>...win.msi...package is recommended...because of its smaller size.</para></section>mysql> select ExtractValue(xml,

'/section/itemizedlist/listitem/@id[contains(../para/literal,"zip") and contains(../para,"Zip")]') as value from t1;

+-----------+

| value |

+-----------+

| noinstall |

+-----------+


Example#6: boolean OR<section> <title>Choosing an Install Package</title> <para>...three different packages...</para> <itemizedlist> <listitem id=”noinstall”> <para><literal>mysqlversionwinnoinstall.zip</literal>: ...Zip archive....</para> </listitem> <listitem id=”full”> <para><literal>mysqlversionwin.zip</literal>: ...full installation...</para> </listitem> <listitem id=”essential”> <para><literal>mysqlversionessentialwin.msi</literal>: ...minimum components...</para> </listitem> </itemizedlist> <para>...win.msi...package is recommended...because of its smaller size.</para></section>mysql> select ExtractValue(xml,

'/section/itemizedlist/listitem/para[contains(./literal,"zip") or contains(./literal,"win")]/literal')

as value from t1;

+---------------------------------------------------------------------------------------+

| value |

+---------------------------------------------------------------------------------------+

| mysql-version-win-noinstall.zip mysql-version-win.zip mysql-version-essential-win.msi |

+---------------------------------------------------------------------------------------+


Example#7: position()<section> <title>Choosing an Install Package</title> <para>...three different packages...</para> <itemizedlist> <listitem id=”noinstall”> <para><literal>....noinstall.zip</literal>: ...Zip archive....</para> </listitem> <listitem id=”full”> <para><literal>mysqlversionwin.zip</literal>: ...full installation...</para> </listitem> <listitem id=”essential”> <para><literal>...win.msi</literal>: ...minimum components...</para> </listitem> </itemizedlist> <para>...win.msi...package is recommended...because of its smaller size.</para></section>mysql> select ExtractValue(xml,

'/section/itemizedlist/listitem[position()=2]/para/literal')

as value from t1;

+-----------------------+

| value |

+-----------------------+

| mysql-version-win.zip |

+-----------------------+


Updating entire XML values

mysql> update t2 set xml='<a>a1<b>b1<c>c1</c>b2</b></a>';

Query OK, 1 row affected (0.02 sec)

Rows matched: 1 Changed: 1 Warnings: 0


Updating XML parts using XPath

mysql> select updateXML(

'<a>a1<b>b</b></a>','/a/b','<b>b1<c>c1</c>b2</b>'

) as xml;

+-------------------------------+

| xml |

+-------------------------------+

| <a>a1<b>b1<c>c1</c>b2</b></a> |

+-------------------------------+

Usage example:

New function:UpdateXML(xml, xpath, content)

Returns a changed copy of XML value xml by replacing a node addressed by an XPath query xpath with a new content content.


Optimizing XPath queries: table structure

mysql> show create table t1;+-------+----------------------------------------+| Table | Create Table |+-------+----------------------------------------+| t1 | CREATE TABLE `t1` ( || | `id` int(11) NOT NULL auto_increment, || | `xml` text NOT NULL, || | PRIMARY KEY (`id`), || | FULLTEXT KEY `xml` (`xml`) || |) ENGINE=MyISAM |+-------+----------------------------------------+mysql> select count(*) from t1;+----------+| count(*) |+----------+| 65536 |+----------+


Optimizing XPath queries using FULLTEXTmysql> selectExtractValue(xml,'/section/itemizedlist/listitem[1]/para/literal') from t1 where ExtractValue(xml,'/section/title')='NewTitle';+--------------------------------------------------------------------+| ExtractValue(xml,'/section/itemizedlist/listitem[1]/para/literal') |+--------------------------------------------------------------------+| mysql-version-win-noinstall.zip |+--------------------------------------------------------------------+1 row in set (4.84 sec)

mysql> select ExtractValue(xml,'/section/itemizedlist/listitem[1]/para/literal') from t1 where ExtractValue(xml,'/section/title')='NewTitle' and match (xml) against ('NewTitle');+--------------------------------------------------------------------+| ExtractValue(xml,'/section/itemizedlist/listitem[1]/para/literal') |+--------------------------------------------------------------------+| mysql-version-win-noinstall.zip |+--------------------------------------------------------------------+1 row in set (0.02 sec)

Slow query:

Fast query:


Summary

Data types:• Any character string

Handlers supporting XPath optimization:• MyISAM

New SQL functions:• ExtractValue(xml, xpath)

• UpdateXML(xml, xpath, value)


Summary: XPath features

Supported XPath functionality:Absolute/Relative location path with full and abbreviated step syntaxAxes: ancestor, self, parent, descendant, attribute, childBoolean (i.e. predicates) and numeric (i.e. by index) filtersBooleans: AND, OR, NOT, =, !=, <=, >=, >, <, false(), true(), last()Numeric operators: +, , *String functions: contains(), substring(), concat()Numeric functions: mod(), div(), ceiling(), floor(), round(), sum(), count(),

position()Type conversion functions: boolean(), number()


Current status

• Available as a separate patch for MySQL5.0• From: http://d.udm.net/~bar/myxml/mysqlxml.tar.gz• No documentation yet• Questions are welcome at <[email protected]>• This presentation:

http://d.udm.net/~bar/myxml/XMLXpathSupportInMySQL.sxi


Near Future TODO

• Push into the official source tree• Implement 100% XPath Version 1.0

• Special XML data type, for validity checking of inserted values, and perhaps for more optimized storage (compression?)

• ExistsNode() an optimized boolean function to check whether a node exists, without fetching its value

• A preprocessor for FULLTEXT to build indexes optimized for XML purposes

• Automatic invocation of FULLTEXT searches from inside ExtractValue() and ExistsNode() without having to use explicit MATCH operator


Good Bye!

Selected Presentations, MySQL Users Conference 2005€¦ · Selected Presentations MySQL Users...

Documents

Transcript of Selected Presentations, MySQL Users Conference 2005€¦ · Selected Presentations MySQL Users...