TiDB: Overview, New Features, Multi-Cloud

Shen Li, Kevin Xu PingCAP

!2

What is Ti(tanium) DB?

• NewSQL (RDBMS at Scale)

• MySQL Compatible (5.7)

• Hybrid OLTP + OLAP Workloads

• Multi-Cloud

!3

Product: the TiDB Platform

• TiDB Platform:• TiDB (stateless SQL layer)

• TiKV (persistent key-value store w/ RocksDB)

• TiSpark (Apache Spark plug-in to TiKV)

• Open Sourced from Day 1• GA 2.0: April 2018

Community

!4

• Stars:

• TiDB: 15,600+

• TiKV: 3,800+

• Contributors:

• TiDB: 210+

• TiKV: 105+

Recent News

!5

TiDB Overview

Platform Architecture

!7Monitoring & Observability: Prometheus + Grafana

TiKV: the Storage Foundation

!8

TiDB: the (My)SQL Layer

!9

300+ Companies in Production

!10

MySQL Compatible Tools and Migration

Tools

!12

• TiDB-DM (Data Migration)

• TiDB-Lightning

• TiDB-Binlog

TiDB-DM

!13

• Why? To support migrating data from MySQL/MariaDB to TiDB

• Full data migration + binlog replication

• Features:

• Data migration job management

• Monitoring system

• Sharded table merge migration

• Data transformation

DM Architecture

!14

Sharded Table Merge

!15

Data Transformation

!16

TiDB-Lightning

!17

• Why? Enables fast, full import of SQL dump -> TiDB cluster• Fast:

• Can import > 100 GB / hour on production grade hardware

• Full Import:

• Import the entire database once from scratch; not incremental import

• Suitable for migrating an existing database to a new TiDB cluster

• TiDB Cluster:

• Lightning interacts with TiDB’s internal structures

• Only works with TiDB cluster

TiDB-Lightning Architecture

!18

TiDB-Binlog

!19

• Why? Provides real-time data synchronization and backup

• Synchronizing downstream data platform in real-time to:

• Hot backup

• MySQL slave

• OLAP data warehouses

• Writing incremental backup files

• Event subscription & trigger

TiDB-Binlog Architecture

!20

Migration: in and out of TiDB

!21

Sneak Peek of TiDB 2.1

TiDB 2.1 - SQL LayerSmarter SQL optimizer • More efficient plan • Dynamic stats update based on

query feedback Faster aggregation operator • New aggregation framework • Multi-thread hash aggregation

operator 50% faster OLTP point query

!23

Faster online DDL • Run DDL concurrently • Faster “Add Index” operation Improved debugging & diagnosis • More readable EXPLAIN result • “Explain Analyze” statement • Expose cluster information through

HTTP API

TiDB 2.1 - Storage Engine (aka TiKV)

Raft learner • Improve cross-DC data availability Hotspot scheduling • Automatically distribute hotspots to more idle instances Batch region-splits • Avoid large region formation under heavy write workloads 2X read performance improvement

!24

1.0 vs 2.0 on TPC-H Scale 50

!25

2.0 vs 2.1 on TPC-H Scale 50

!26

2.1 vs 2.0 on Sysbench Point Select

!27

TiDB Cloud

TiDB As A Service

!29

• Fully Managed by PingCAP

• Upgrades, Patching, Online DDL with No Downtime

• Backup and Restore

• Multi-Cloud Deployment

https://www.pingcap.com/tidb-cloud/

TiDB Operator

!30

GCP Marketplace

!31

See it to believe it

!32

• PingCAP Booth #6

• Demo by Greg Weber TODAY at 1:20PM

Oh…one more thing, TiDB Academy

!33https://www.pingcap.com/tidb-academy/

Thank you!

@PingCAP, @shenli3514, @kevinsxu