Tungsten University: Unleashing the Power of Tungsten Connectors
-
Upload
continuent -
Category
Documents
-
view
935 -
download
0
description
Transcript of Tungsten University: Unleashing the Power of Tungsten Connectors
©Continuent 2013
Tungsten University:Unleashing the Power of
Tungsten ConnectorsGilles Rayrat, Director of Operations/France
©Continuent 2013
What is Tungsten Connector?
2
Connector
App
NativeprotocolCOM_QUERY
COM_INIT_DBCOM_DROP_DB
©Continuent 2013
A Key Tungsten Component
3
• Provide access to the master
• Handle database failures
• Can access slaves for reads
©Continuent 2013
Deployments
4
©Continuent 2013
Deployment 1: app servers
5
App Server+ Connector
App Server+ Connector
App Server+ Connector
App Server+ Connector
App Server+ Connector
©Continuent 2013
Deployment 2: database servers
6
Software Load Balancer
...
App Server App Server App Server App Server App Server App Server
©Continuent 2013
Deployment 3: dedicated hosts
7
...
Hardware Load Balancer
App Server App Server App Server App Server App Server App Server
...
©Continuent 2013
Deployment best practice
8
Install with port 3306!
tools/tpm con!gure <data service name>
<options>
--connector-listen-port=3306
©Continuent 2013
Closer view
• Performance
• Failure handling
9
©Continuent 2013
Packet Streaming
• Fast operations, Low overhead
• Full transparency
10
Connector
App
Protocol packetCOM_QUERY
select * from t
Protocol packetOK
ResultSet Rows:1
©Continuent 2013
Closer view
• Performance
• Failure handling
11
©Continuent 2013 12
Connector-Manager link
Master Slave
App Logic
Tungsten Connector
Replicator Replicator Replicator
App Logic
Tungsten Connector
Manager Manager Manager
Monitoring and control
Monitoring and control
Slave
©Continuent 2013
Node Failure
“Pause” incoming connection requests+
+
Transparent retry (autocommit tx only)
13
Planned switch:wait for disconnect
Unplanned Failover:break connections
©Continuent 2013
Demo?
14
asia
worldeurope
u1
u2
u3
u4
©Continuent 2013 15
Installation
• tools/tpm con!gure <data service name> <options>--dataservice-connectors=node1,node2,...
• --connector-listen-port --connector-user --connector-password --connector-smartscale=true
• tools/tpm help con!gure
©Continuent 2013 16
Specifying Users
• --connector-user=user1
• additional users in
tungsten-connector/conf/user.map
©Continuent 2013 17
user map
<user> <password> <data service> [<preferred site>]
tungsten_testing private world europemyuser p4$$ asiaalice - othersvc
No need to restart
©Continuent 2013 18
Connection Protocol
App
version #, ..., salt
Database, user,Salted password
©Continuent 2013 19
A Man in the Middle
App
Connector
?
©Continuent 2013 20
• Workaround:
@script /opt/continuent/share/gen_user_map.sh
• output identical to regular user.map
• Can be any script: SQL requests, encrypting, etc.
• Accepts parameters
Clear text passwords
©Continuent 2013
Application con!guration
21
©Continuent 2013
Application con!guration - PHP
22
<?php
$host = “u1”;
$port = 9999;
$username = “tungsten_testing”;
$password = “secret_private”;
$dbname = “myapp";
$connection = mysqli_connect($host, $username, $password, $dbname, $port);
......... More application logic here .........
?>
©Continuent 2013
Application con!guration - Java
23
• Hibernate
hibernate.connection.url=jdbc:mysql://u1:9999/mydb
• JBoss
<connection-url>jdbc:mysql://u1:9999/database</connection-url>
• JDBC
jdbc:mysql://u1:9999/dbname
©Continuent 2013
Application con!guration - Ruby on Rails
24
prod:
adapter: mysql2
encoding: utf8
database: my_db_name
username: tungsten_testing
password: private
host: u1
port: 9999
©Continuent 2013
Read/write splitting
• Automatic
• Application Controlled
25
©Continuent 2013
Automatic R/W splitting
• No application changes
• Only outside transaction boundaries
• 2 variants:
• Smart Scale
• @direct connections
26
©Continuent 2013
Automatic R/W splitting - smartScale
• AKA ‘session consistency’ QoS
• Ability to see session’s writes
27
©Continuent 2013
SmartScale
28
App
Connector
MasterDBMS
TransactionHistory Log
SlaveDBMS
Slavereplay
position
write
Session xSession y...
©Continuent 2013 29
SmartScale
App
Connector
MasterDBMS
TransactionHistory Log
SlaveDBMS
Slavereplay
position
read
Session x
©Continuent 2013 30
SmartScale
App
Connector
MasterDBMS
TransactionHistory Log
SlaveDBMS
Slavereplay
position
read
Session x
©Continuent 2013 31
• Enabling:
tpm configure <data service name> <other options> --connector-smartscale=true --connector-smartscale-sessionid= DATABASE | USER | CONNECTION | PROVIDED_IN_DBNAME
SmartScale
©Continuent 2013 32
• Choose you sessionId:
• PROVIDED_IN_DBNAME (++)=> set dbname to ”mydb?sessionId=12345”
• CONNECTION (+)
• USER (-)
• DATABASE (--)
SmartScale
©Continuent 2013
Demo?
33
©Continuent 2013 34
Limitations:
• single connector instance
• select function_that_writes();
• ephemeral SQL objects
SmartScale
©Continuent 2013 35
Trade-o":
• Can read outdated data from other sessions
• Poor e!ciency in write intensive environments
SmartScale
©Continuent 2013
Automatic R/W splitting - directReads
• Blind R/W splitting
• Fast
• Similar limitations:
• ephemeral SQL objects
• select function_that_writes();
• Reads potentially outdated data
36
©Continuent 2013
DirectReads
37
• Enablingtpm configure <ds> <opts> --connector-rwsplitting=true
• Other users in user.map@direct user2@direct user3
• Can be backed by a c3p0 pool
©Continuent 2013
Read/write splitting
• Automatic
• Application Controlled
38
©Continuent 2013 39
Application-driven R/W splitting
• More e#cient...
• ...but potentially dangerous
• 3 possibilities:
• SQL comments
• QoS in database name
• @hostoption
©Continuent 2013
Selective R/W Splitting
• Must enable directReads
tpm update ...--connector-rwsplitting=true \--property=selective.rwsplitting=true
• Embedded SQL comment:
40
!!! mysql -u... -c
/* TUNGSTEN_URL_PROPERTY=qos=RO_RELAXED */select @@hostname;
©Continuent 2013
Demo?
41
©Continuent 2013
QoS at connection time
• No connector con!guration change
• connect to “mydb@qos=RO_RELAXED”
42
©Continuent 2013
Demo?
43
©Continuent 2013
Host-based routing
• /etc/hosts127.0.0.1 master.localhost127.0.0.2 slave.localhost
• tpm configure <data service name> <other options> --connector-rw-addresses=master.localhost --connector-ro-addresses=slave.localhost
44
©Continuent 2013
Demo?
45
©Continuent 2013 46
SOR (multi site)
• Getting a local slave
<user> <password> <data service> [<preferred site>]
tungsten_testing private world europe
©Continuent 2013
Demo?
47
©Continuent 2013 48
Performance
• Pool for DirectReads
tpm configure <ds> <opts> --connector-rwsplitting=true --property=directReadsUsePool=true
• JDBC driver
https://docs.continuent.com/wiki/display/TEDOC/Improving+Connection+Speed
--property=jdbc.driver.options=
cacheServerCon!guration=true\&characterSetResults=UTF8
©Continuent 2013
Getting connector information
• “tungsten connection status” for last request
• “tungsten show processlist” for real client addresses
49
mysql> tungsten connection status;+---------------------------------------------------------------+| message |+---------------------------------------------------------------+| internal(OPEN) DIRECT TO u1@default(master:ONLINE) STATUS(OK) |+---------------------------------------------------------------+1 row in set (0.01 sec)
mysql> tungsten show processlist;+------------+------+----------+-----------+----------+---------+------+-------+------+| DataSource | Id | User | Host | db | Command | Time | State | Info |+------------+------+----------+-----------+----------+---------+------+-------+------+| u1 | 1342 | tungsten | mac:64477 | tungsten | Sleep | 20 | | NULL |+------------+------+----------+-----------+----------+---------+------+-------+------+1 row in set (0.13 sec)
©Continuent 2013
Troubleshooting
• tungsten-connector/log/connector.log
• Lots of connections in TIME_WAIT state
• Too many open !les
• OutOfMemory error
• New connections hanging
50
©Continuent 2013
Tungsten University Sessions
51
• Manage multi-master or fan-in replication topologies - March 7th & 12th
• Zero-downtime maintenance and schema operations - March 21st & 26th
• Geographically distributed MySQL clusters - April 11th & 16th
• Replicate between MySQL and Oracle - May 2nd & 7th
©Continuent 2013
Continuent Web Page:http://www.continuent.com
Tungsten Replicator 2.0:http://code.google.com/p/tungsten-replicator
Our Blogs:http://scale-out-blog.blogspot.comhttp://datacharmer.org/bloghttp://www.continuent.com/news/blogs
560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009e-mail: [email protected]