EsgynDB Manager User Guide€¦ · • Monitor runtime status of EsgynDB Nodes • Active Sessions...
Transcript of EsgynDB Manager User Guide€¦ · • Monitor runtime status of EsgynDB Nodes • Active Sessions...
EsgynDB Manager User GuideVersion 2.5.0
Table of Contents
1. Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2. Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3. Install and Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1. Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.2. Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.2.1. Configuration Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.2.2. Sample configuration file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.3. Start EsgynDB Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.3.1. When installed using Cloudera Parcels or Ambari . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.3.2. When installed using Python installer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.3.3. Manual Start of DB Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.4. Stop EsgynDB Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.4.1. When installed using Cloudera Parcels or Ambari . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.4.2. When installed using Python installer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.4.3. Manual Stop of DB Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.5. HA Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.5.1. DB Manager HA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
When installed using Cloudera Parcels or Ambari . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
When installed using Python installer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.5.2. Bosun HA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.5.3. Persistence for OpenTSDB TSD and TCollector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.5.4. Persistence using NMON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.6. Log settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.7. Check Status of EsgynDB Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.8. Display EsgynDB Manager Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4. Using EsgynDB Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.1. Connecting to EsgynDB Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.2. Login . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.3. Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.4. Notifications Panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.5. EsgynDB Manager User Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.6. About . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.7. Log Out . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.8. EsgynDB Time-Series Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
5. Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.1. Service Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.2. Node Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5.3. Active Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5.4. Transaction Counts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5.5. Canary Response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
5.6. CPU Busy % . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
5.7. IO Waits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
5.8. Free Memory % . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
5.9. Disk Space Used % . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
5.10. Network IO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
5.11. Regionserver Memory Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
5.12. GC Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
5.13. Memstore Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
5.14. Drill-down details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
6. Workloads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
6.1. Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
6.1.1. Active Workloads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
6.1.2. Active Query Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
6.1.3. Historical Workloads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
6.1.4. Historical Query Detail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
6.1.5. Query Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
6.1.6. Cancel Query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6.2. Workload Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6.2.1. Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6.2.2. View Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
6.2.3. Add Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
6.2.4. Alter Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
6.2.5. Delete Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
6.2.6. SLAs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
6.2.7. View SLAs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
6.2.8. Add SLA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
6.2.9. Alter SLA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
6.2.10. Delete SLA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
6.2.11. Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
6.2.12. View Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
6.2.13. Add Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.2.14. Alter Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6.2.15. Delete Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6.2.16. Profile/SLA/Mapping at Runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6.2.17. Debugging mapping issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
7. Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
7.1. Log Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
7.2. Log Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
8. Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
8.1. Catalogs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
8.2. Schemas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
8.3. Schema Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
8.3.1. Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
8.3.2. DDL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
8.3.3. Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
8.4. Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
8.4.1. EsgynDB Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
8.4.2. Hive Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
8.5. Table Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
8.5.1. Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
8.5.2. Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
8.5.3. Regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
8.5.4. Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
8.5.5. DDL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
8.5.6. Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
8.5.7. Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
8.5.8. Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
8.6. Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
8.7. View Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
8.7.1. Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
8.7.2. Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
8.7.3. DDL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
8.7.4. Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
8.7.5. Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
8.8. Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
8.9. Index Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
8.9.1. Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
8.9.2. Regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
8.9.3. DDL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
8.10. Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
8.10.1. Create Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
8.11. Library Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
8.11.1. Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
8.11.2. DDL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
8.11.3. Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
8.11.4. Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
8.12. Alter Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
8.13. Drop Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
8.14. Download Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
8.15. Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
8.16. Procedure Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
8.16.1. Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
8.16.2. DDL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
8.16.3. Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
8.16.4. Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
8.17. Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
8.18. Function Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
8.18.1. Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
8.18.2. DDL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
8.18.3. Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
8.18.4. Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
9. Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
9.1. Service Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
9.2. Master Executor summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
9.3. Master Executor Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
9.4. Child processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
9.5. Process Stack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
10. Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
10.1. Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
10.1.1. Roles Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
10.1.2. Role Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Grantees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
SQL Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Component Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
10.1.3. Delete Role . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
10.2. Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
10.2.1. Users Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
10.2.2. Add User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
10.2.3. User Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Edit User Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
SQL Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Component Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
10.2.4. Delete User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
10.3. User Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
10.3.1. Add User Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
10.3.2. User Group Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
Edit User Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
Grant/Revoke Role to User Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
10.3.3. Delete User Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
10.4. Roles based Authorization for DB Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
11. Tenants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
11.1. Resource Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
11.1.1. Add Resource Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
11.1.2. Edit Resource Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
11.1.3. Delete Resource Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
11.2. Tenants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
11.2.1. Key Terminologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
11.2.2. Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
11.2.3. Node Allocation Graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
11.2.4. Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
11.2.5. Add Tenant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
11.2.6. Alter Tenant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
11.2.7. Delete Tenant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
11.2.8. Granting Privileges to Tenant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
11.2.9. Connecting as Tenant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
JDBC Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
ODBC Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
11.2.10. Monitoring Tenant Workloads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
12. Query Workbench . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
12.1. Execute Query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
12.1.1. Max Rows in result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
12.2. Explain Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
12.3. Control Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
12.4. Export Workbench Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
12.5. Import Workbench Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
12.6. Clear Workbench . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
13. Alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
13.1. Alerts Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
13.2. Alert Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
14. Backup/Restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
14.1. Backup/Restore Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
14.2. New Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
14.2.1. View Scheduled Backup jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
14.2.2. Modify Schedule of Backup jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
14.2.3. Delete Scheduled Backup jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
14.3. Restore Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
14.4. Export Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
14.5. Import Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
14.6. Delete Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
14.7. Backup/Restore Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
14.8. Additional Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
15. Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
15.1. Create Library Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
15.2. SQL Converter Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
15.2.1. Converting SQL statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
15.3. Execute SQL Script Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
15.3.1. Executing SQL script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
16. Troubleshooting and Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
16.1. Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
16.1.1. Cannot login to DB Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
16.1.2. Page is blank or partially loaded . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
16.1.3. The Datetime values do not look right . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
16.1.4. Scheduling backup jobs fails with error ImportError: No module named crontab . 101
16.2. Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
© Copyright 2015-2018 Esgyn Corporation
The information contained herein is subject to change without notice. This documentation
is distributed on an "AS IS" basis, without warranties or conditions of any kind, either
express or implied. Nothing herein should be construed as constituting an additional
warranty. Esgyn Corporation shall not be liable for technical or editorial errors or omissions
contained herein.
Revision History2.5.0 18Dec2018
EsgynDB Manager User Guide
Preface | 1
Chapter 1. Getting Started
1.1. Introduction
EsgynDB Manager is a web based enterprise management tool for monitoring and managing
EsgynDB platform and workloads.
1.2. Features
• Dashboard
• Monitor runtime status of EsgynDB services
• Monitor runtime status of EsgynDB Nodes
• Active Sessions to EsgynDB
• Time-series charts for monitoring key platform and EsgynDB metrics
• Workloads
• Monitoring
• Monitor queries that are currently active on the system
• Drill-down to view runtime statistics for each query
• View historical queries that completed in the past
• View Top 5 queries based on CPU Time, Memory Usage, Disk IOs and Elapsed Time
• Drill-down into detailed query metrics
• View visual and textual explain plans for queries
• Cancel active queries that are impacting the system
• Configuration
• Setup workload profiles to allow queries to be run on specific nodes with required runtime
SQL profiles
• Define Service Level Agreement (SLA) thresholds
• Map end user queries to specific SLAs and workload profiles
EsgynDB Manager User Guide
2 | Chapter 1. Getting Started
• Database
• EsgynDB
• Browse Schemas, Tables, Views, Indexes, Libraries, Procedures and UDFs
• View Object attributes, columns, privileges, DDL text, object dependencies
• View Table histogram statistics and region statistics
• Create/Alter/Drop Library tool to easily deploy library code files and create/alter/drop library
• Hive
• Browse Hive Schemas and Tables
• View Hive table attributes and columns
• Logs
• View EsgynDB logs via logs TMUDF
• Connectivity
• View status of EsgynDB connectivity services and master executors
• View currently active application sessions
• Security
• Add/Alter/Delete database roles
• Add/Alter/Delete database users
• Grant roles to database users
• Add/Alter/Delete user groups
• Grant roles to user groups
• Grant component privileges to users and roles
• Multi-Tenancy *2
• Add/Alter/Delete tenants
• View resource usage per tenant
• Add/Alter/Delete resource groups *1
EsgynDB Manager User Guide
Chapter 1. Getting Started | 3
• Heterogenous cluster support via resource groups *1
• Query Workbench
• Execute ad-hoc queries
• Generate visual and textual explain plans
• Apply CQDs to change execute plans
• Cancel query execute or explain
• Export query results, explain plans to JSON text file
• Import query results and/or explain plans from JSON file
• Alerts *3
• View Alert notifications
• Acknowledge and update alerts
• Backup/Restore EsgynDB schemas and tables *3
• Tools
• SQL Converter tool to convert SQL DDL and DML statements from Teradata syntax to
EsgynDB
• Script Execution tool to execute mutli-statement SQL Script
*1 New feature in EsgynDB Manager 2.5.0
*2 Requires a license with multi-tenancy add-on
*3 Only in EsgynDB Advanced Edition
EsgynDB Manager User Guide
4 | Chapter 1. Getting Started
Chapter 2. Architecture
EsgynDB Manager is a web management console to monitor and manage an EsgynDB instance.
• DB Manager runs as an embedded jetty webserver.
• The web client layer uses HTML5/JavaScript/CSS.
• The web client to DB Manager webserver communication uses HTTPS encryption.
• The web client session is authenticated using JDBC.
• Communication with EsgynDB core is through JDBC and HTTP requests.
• The system health and performance metrics as well the health and performance of EsgynDB
services are periodically collected as time-series data and published into OpenTSDB.
• TCollector provides several scripts to measure system health and performance and also an
extensible infrastructure to launch scripts at pre-defined intervals. The metrics are published into
OpenTSDB.
• DB Manager provides real-time alerting using the Bosun alerting and notification engine.
• EsgynDB Manager has been tested with Chrome, IE11, Firefox and Microsoft Edge browsers.
• DB Manager also executes scripts locally on the webserver for some of the management functions
EsgynDB Manager User Guide
Chapter 2. Architecture | 5
Figure 1. EsgynDB Manager Architecture
EsgynDB Manager User Guide
6 | Chapter 2. Architecture
Chapter 3. Install and Configuration
3.1. Installation
When you install EsgynDB, DB Manager is also installed and configured along with the core EsgynDB
components. No additional install steps are required.
EsgynDB Manager is installed in $TRAF_HOME/dbmgr-2.5.0 folder.
3.2. Configuration
DB Manager reads its run-time configuration from a configuration file. This file is generated at install
time based on the options selected during install.
When you install EsgynDB using Cloudera Parcels or Ambari, DB Manager is installed as a service
role. You can configure a subset of these DB Manager properties using Cloudera Manager or Ambari
GUI.
When using Cloudera Manager or Ambari, the DB Manager configuration file is
regenerated every time the DB Manager role is started.
3.2.1. Configuration Properties
jdbcUrl
The JDBC url for the EsgynDB Connection Master (DCS Master).
<entry key="jdbcUrl">jdbc:t4jdbc://my.esgyndb.server:23400/:</entry>
If there are multiple DCS Masters configured then you will see a comma separated DCS
quorum. The JDBC driver would try to connect to the first server and if it fails, it would try the
next server in the quorum.
<entry key="jdbcUrl">jdbc:t4jdbc://my.esgyndb.server1:23400,my.esgyndb.server2:23400/:</entry>
adminUserID
EsgynDB Manager User Guide
Chapter 3. Install and Configuration | 7
The database user ID used for the jdbc connections to run the internal system queries.
<entry key="adminUserID">DB__ADMIN</entry>
adminPassword
Password of the database user ID used for the jdbc connections to run the internal system
queries. The password is obfuscated by the installer.
<entry key="adminPassword">OBF:1sov1sov</entry>
connectionTimeout
The Connection Timeout in seconds. Used for the JDBC connections from EsgynDB Manager
to EsgynDB. Default value is 60 seconds (1 minute).
<entry key="connectionTimeout">60</entry>
minPoolSize
DB Manager uses connection pooling for its internal queries. This parameter sets the initial and
minimum pool size. Default value is 2.
<entry key="minPoolSize">2</entry>
maxPoolSize
DB Manager uses connection pooling for its internal queries. This parameter sets the maximum
pool size. Default value is 8.
<entry key="maxPoolSize">8</entry>
sessionTimeoutMinutes
The Session Timeout in minutes. The EsgynDB Manager browser client session to the server
will be timed out after this interval and the client will be prompted to login again. Default value is
120 minutes (2 hours).
<entry key="sessionTimeoutMinutes">120</entry>
EsgynDB Manager User Guide
8 | Chapter 3. Install and Configuration
timeZoneName
The TimeZone name of the EsgynDB server. Specified in canonical time zone format like
America/New_York
<entry key="timeZoneName">America/New_York</entry>
trafodionRestServerUri
The http url for the EsgynDB REST server.
<entry key="trafodionRestServerUri">http://my.esgyndb.server:4200</entry>
openTSDBUri
The HTTP url for fetching the timeseries metrics from TSD (OpenTSDB) process.
<entry key="openTSDBUri">http://my.esgyndb.server:5242</entry>
alertsUri
The HTTP url for viewing and managing Bosun alerts.
<entry key="alertsUri">http://my.esgyndb.server:8070</entry>
Bosun is configured on all nodes where DCS Master is configured to run. If there are multiple
DCS Masters configured then you will see a comma separated Bosun quorum. DB Manager
would try to connect to the first bosun server and if it fails, it would try the next server in the
quorum.
<entry key="alertsUri">http://my.esgyndb.server1:8070,my.esgyndb.server2:8070</entry>
httpReadTimeOutSeconds
The HTTP read timeout in seconds for any HTTP requests initiated by DB Manager. For
example, the HTTP calls to EsgynDB REST server or the OpenTSDB TSD server.
<entry key="httpReadTimeOutSeconds">120</entry>
EsgynDB Manager User Guide
Chapter 3. Install and Configuration | 9
qwbMaxRows
The Maximum number of rows allowed to be fetched in Query Workbench.
<entry key="qwbMaxRows">100000</entry>
httpPort
The HTTP Port for the DB Manager embedded jetty server. Default value is 4205.
<entry key="httpPort">4205</entry>
httpsPort
The HTTPS Port for the DB Manager embedded jetty server. Default value is 4206
<entry key="httpsPort">4206</entry>
keyStoreFile
The SSL Keystore file for HTTPS connections
<entry key="keyStoreFile">/opt/trafodion/sqcert/server.keystore</entry>
securePassword
The EsgynDB installer generates a self-signed certificate and stores it in a SSL keystore with
this obfuscated password.
<entry key="securePassword">OBF:1iup1igf1x8a1tvj1x8k1idr1irx</entry>
requestHeaderSize
The HTTP request header size in bytes for the embedded jetty server.
<entry key="requestHeaderSize">98304</entry>
3.2.2. Sample configuration file
A sample $TRAF_HOME/dbmgr-2.5.0/conf/config.xml is as shown below.
EsgynDB Manager User Guide
10 | Chapter 3. Install and Configuration
<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
<!-- The JDBC url for the Trafodion/EsgynDB instance that you areconnecting to --><entry key="jdbcUrl">jdbc:t4jdbc://my.esgyndb.server:23400/:</entry>
<!-- The JDBC driver class name for the Trafodion/EsgynDB JDBC driver --><entry key="jdbcDriverClass">org.trafodion.jdbc.t4.T4Driver</entry>
<!-- Minimum number of connections for jdbc connection pool --><entry key="minPoolSize">2</entry>
<!-- Max number of connections for jdbc connection pool --><entry key="maxPoolSize">8</entry>
<!-- Connection timeout in seconds for jdbc connections --><entry key="connectionTimeout">60</entry>
<!-- maxIdleTime in seconds for jdbc connection pool --><entry key="maxIdleTime">1200</entry>
<!-- Max number of statements for jdbc statement caching --><entry key="maxStatementsCache">100</entry>
<!-- The Admin User ID used by DBManager for its internal queries--><entry key="adminUserID">DB__ADMIN</entry>
<!-- The Admin User's password--><entry key="adminPassword">OBF:1sov1sov</entry>
<!-- Session Timeout in minutes. Your DB Manager client browser sessionto the DB Manager server will be timed out after this interval andyou will be asked to login again --><entry key="sessionTimeoutMinutes">120</entry>
<!-- The TimeZone name of the EsgynDB server. Enter in canonical time zoneformat like Etc/UTC or America/New_York --><entry key="timeZoneName">America/New_York</entry>
<!-- The Trafodion REST Server URI --><entry key="trafodionRestServerUri">http://my.esgyndb.server:4200</entry>
<!-- The openTSDB HTTP URI--><entry key="openTSDBUri">http://my.esgyndb.server:5242</entry>
<!-- The Bosun Alerts HTTP URI--><entry key="alertsUri">http://my.esgyndb.server:8070</entry>
<!--HTTP read timeout in seconds for external REST calls issued byDB Manager Server --><entry key="httpReadTimeOutSeconds">120</entry>
EsgynDB Manager User Guide
Chapter 3. Install and Configuration | 11
<!--Maximum number of rows allowed to be fetched in Query Workbench--><entry key="qwbMaxRows">100000</entry>
<!-- The following properties are only required and used by the embeddedjetty server -->
<!-- The HTTP Port for the DB Manager embedded jetty server --><entry key="httpPort">4205</entry>
<!-- The HTTPS Port for the DB Manager embedded jetty server --><entry key="httpsPort">4206</entry>
<!-- The SSL keystore file for the EsgynDB Manager embedded jetty server --><entry key="keyStoreFile">/opt/trafodion/sqcert/server.keystore</entry>
<!-- The SSL keystore password for the DB Manager embedded jetty server --><entry key="securePassword">OBF:1iup1igf1x8a1tvj1x8k1idr1irx</entry>
<!-- The HTTP request header size for DB Manager embedded jetty server --><entry key="requestHeaderSize">98304</entry>
</properties>
3.3. Start EsgynDB Manager
3.3.1. When installed using Cloudera Parcels or Ambari
DB Manager is configured as separate role for the EsgynDB service.
When you start EsgynDB service, DB Manager would also be started. You can also select the
instances/roles of the service and start/restart DB Manager role alone.
3.3.2. When installed using Python installer
When you start EsgynDB using sqstart or trafstart script, DB Manager will be started automatically.
3.3.3. Manual Start of DB Manager
You should use this option only when EsgynDB is installed using Python installer. When installed
using Cloudera Parcels or Ambari, you should always use the Cloudera Manager or Ambari to
start/stop/restart EsgynDB Manager.
Use the dbmgr.sh script to do manual start.
EsgynDB Manager User Guide
12 | Chapter 3. Install and Configuration
$ cd dbmgr-2.5.0/bin./dbmgr.sh start
If the EsgynDB Manager starts successfully, you should see a prompt like below:
2017-11-06_17:17:58: EsgynDB Manager is up and running with pid (17122)
If the start fails, you see a message like this. Check the dbmgr.log for errors.
2017-11-06_17:23:36: Failed to start EsgynDB Manager. Please check the dbmgr log.
3.4. Stop EsgynDB Manager
3.4.1. When installed using Cloudera Parcels or Ambari
DB Manager is configured as separate role for the EsgynDB service.
Using Cloudera Manager or Ambari GUI, navigate to the instances/roles for EsgynDB Service, select
DB Manager role and stop.
3.4.2. When installed using Python installer
When you stop EsgynDB using sqstop or trafstop scripts, DB Manager will also be stopped.
You can also use the dbmgr.sh script to stop just the DB Manager process.
3.4.3. Manual Stop of DB Manager
$ cd dbmgr-2.5.0/bin$ ./dbmgr.sh stop2017-11-06_17:23:01: Stopping EsgynDB Manager pid (17122)2017-11-06_17:23:05: Stopped EsgynDB Manager
3.5. HA Configuration
DB Manager and the manageability tools (OpenTSDB, TCollector and Bosun) are persistent and
fault-tolerant.
EsgynDB Manager User Guide
Chapter 3. Install and Configuration | 13
Persistence
If the process were to exit for some reason it would be started right back up.
Fault-tolerant
If a node fails, the processes are available on another node.
3.5.1. DB Manager HA
When installed using Cloudera Parcels or Ambari
Configure 2 or more DB Manager roles. DB Manager roles have to be co-located with Connection
DCS Master roles (same nodes) and we leverage the DCS HA.
When DCS Master is configured to use keepalived or floating IP, you can use the same virtual IP
address to access DB Manager. The requests would go to the DB Manager instance running on the
primary DCS Master node. When that node goes down, the IP will fail over to the secondary DCS
Master node and requests will go to the DB Manager on that node.
The DB Manager roles within the EsgynDB Service can also be marked to be auto-restarted by
enabling the Automatically Restart Process option through Cloudera Manager. If DB Manager
process were to exit abnormally for some reason, the Cloudera supervisord process would
automatically restart DB Manager.
When installed using Python installer
If EsgynDB is installed via python installer, you need to chose multiple DCS Master nodes for HA,
then DB Manager will be started on all those DCS Master nodes and we will leverage the DCS virtual
IP (keepalived or floating IP) to access DB Manager. If the primary node fails, the IP will fail over to
the secondary DCS Master node and requests will go DB Manager on that node.
The node monitor persistent process $NMON will keep watch on DB Manager and if the DB Manager
process fails for any reason, it would be auto-restarted. See Persistence using NMON section for
more details.
3.5.2. Bosun HA
Bosun also relies on the DCS HA infrastructure, in that you need to have two or more DCS Master
nodes. Bosun ONLY runs on the primary DCS Master node in the cluster. If the node fails, Bosun is
EsgynDB Manager User Guide
14 | Chapter 3. Install and Configuration
started on a secondary DCS Master node to which the keepalived or floating IP fails over.
The node monitor persistent process $NMON will keep watch on Bosun and if the Bosun process fails
for any reason, it would be auto-restarted. See Persistence using NMON section for more details.
3.5.3. Persistence for OpenTSDB TSD and TCollector
• OpenTSDB TSD process runs on every node.
• TCollector runs on every node and collects metrics for that node, and the collected metrics are
sent to the TSD running on that local node.
• The fault-tolerance and persistence for TSD and TCollector is provided by NMON (Node Monitor).
See Persistence using NMON section for more details.
3.5.4. Persistence using NMON
The fault-tolerance and persistence for DB Manager (when installed using python installer only),
Bosun, OpenTSDB and TCollector is provided by NMON (Node Monitor).
NMON runs on every EsgynDB node. It is started and managed by the EsgynDB monitor process, so
NMON is inherently persistent.
NMON reads a list of commands from $TRAF_HOME/sql/scripts/node_monitor.cmd file and
executes them every 1 minute.
• The node_monitor.cmd file has a command to check and start DB Manager. This check is only run
in python installer mode.
$DBMGR_INSTALL_DIR/bin/dbmgr.sh watch
With the watch option, the dbmgr.sh script first checks if DB Manager is running. If DB Manager is
running then the script exits normally, otherwise it starts DB Manager.
• Similarly, the node_monitor.cmd file has a command to check and start Bosun.
$MGBLTY_INSTALL_DIR/bosun/bin/runbosun.sh watch
With the watch option, the runbosun.sh script first checks if Bosun is running. If Bosun is running
then the script exits, otherwise it starts Bosun. So by calling this watch command every 1 minute,
EsgynDB Manager User Guide
Chapter 3. Install and Configuration | 15
we are able to keep Bosun persistent.
• The node_monitor.cmd file has a command to check and start TSD (OpenTSDB).
$MGBLTY_INSTALL_DIR/opentsdb/bin/tsd.sh watch
With the watch option, the tsd.sh script first checks if TSD is running. If TSD is running then the
script exits, otherwise it starts TSD.
• Similarly, the node_monitor.cmd file has a command to check and start TCollector.
$MGBLTY_INSTALL_DIR/tcollector/startstop watch
With the watch option, the startstop script first checks if TCollector is running. If TCollector is
running then the script exits, otherwise it starts TCollector.
3.6. Log settings
DB Manager uses logback for its logging.
You can edit the dbmgr-2.5.0/conf/logback.xml to change the log file rollover settings or the default
LOG level.
The EsgynDB Manager runtime logs are written to the dbmgr-2.5.0/logs directory.
3.7. Check Status of EsgynDB Manager
To check the state of EsgynDB Manager, run the following command from the bin directory.
$ ./dbmgr.sh status2016-11-06_17:26:26: EsgynDB Manager process is running with pid (21913)
3.8. Display EsgynDB Manager Version
To display the version of EsgynDB Manager, run the following command from the bin directory. (You
can also view the version using the DB Manager GUI)
EsgynDB Manager User Guide
16 | Chapter 3. Install and Configuration
$ ./dbmgr.sh versionEsgynDB Manager Release 2.5.0 (Branch 0ea1970, Date 12DEC2017)
EsgynDB Manager User Guide
Chapter 3. Install and Configuration | 17
Chapter 4. Using EsgynDB Manager
This chapter describes how to connect to EsgynDB Manager and use its features.
4.1. Connecting to EsgynDB Manager
Open a web browser and point to the host name and HTTP/HTTPS port on which EsgynDB Manager
is listening.
http://my.esgyndb.server:4205
If HTTPS is enabled (the default), then the HTTP request is automatically redirected to the HTTPS
port.
https://my.esgyndb.server:4206
4.2. Login
In the Login page, enter a valid EsgynDB database user name and password and click on the Login
button. EsgynDB Manager relies on database authentication and makes a JDBC connection to
EsgynDB engine to authenticate the user.
If multi-tenancy is enabled, you’ll also have an option to enter the tenant name.
4.3. Navigation
To Navigate the UI and access the different features, use the navigation menus to the left of the
screen.
Use the menu toggle button (the three vertical lines next to the Product Title) to collapse or expand
the navigation menus:
EsgynDB Manager User Guide
18 | Chapter 4. Using EsgynDB Manager
Figure 2. Navigation Menus
4.4. Notifications Panel
In the top-right corner of the page, you will see a bell icon and whenever there is an asynchronous DB
Manager notification, you’ll see a yellow exclamation mark on it indicating there is a new notification.
When you click on the bell icon, you’ll be shown a list of available notifications with details.
The notifications panel is used to notify that an asynchronous DB Manager operation is complete and
the results are available. The results are added to the notifications panel, only if you have navigated
away from the page where the operation was initiated.
For example: In the Query Workbench page, you run a SQL query and if you stay in that page until
the operation completes, then the success or failure of the execution is displayed in the page. But if
you navigate away from the Query workbench page before the execution is complete, then the
success or failure message is added to the notification panel.
You can click on the notifications panel anytime to view the results. Clicking on the x mark on the
notification message will remove the message from the notification panel.
These asynchronous notifications are available for the following functions:
• Execute, Explain and Cancel operations in Query Workbench
EsgynDB Manager User Guide
Chapter 4. Using EsgynDB Manager | 19
• Script execution in the Execute SQL Script page
• Cancel Query from the workload pages
4.5. EsgynDB Manager User Guide
You can access an online version of this user guide using the Help -> User Guide navigation menu.
4.6. About
Use the Help -> About navigation menu to view the version of DB Manager and your EsgynDB
server. The page displays the version and license details of the EsgynDB instance you are currently
connected to. When you contact Esgyn for DB Manager technical support, please provide the
information in the About page.
4.7. Log Out
You can click on the user icon at the top right corner of the application and select the Logout option.
4.8. EsgynDB Time-Series Metrics
EsgynDB leverages the Tcollector infrastructure and publishes the following metrics as time series
data points into OpenTSDB.
These metrics can then be queried and graphed as time series charts or used in alert rule templates
for notifications.
Metric Name Description
esgyn.hbase.regionserver Number of region servers in running state.
esgyn.hbase.running Indicates if HBase is up. A value of 1 means upand 0 means down.
esgyn.nodes.up Number of nodes that are up
esgyn.nodes.down Number of nodes that are down
esgyndb.canary.sqlconnect.time Time in milliseconds to open a JDBC connection
esgyndb.canary.sqlread.time Time in milliseconds to execute a select on thecanary table
esgyndb.dtm.configure Number of DTM processes that are down
esgyndb.dtm.running Number of DTM processes that are up
EsgynDB Manager User Guide
20 | Chapter 4. Using EsgynDB Manager
Metric Name Description
esgyndb.dtm.down Number of DTM processes that are down
esgyndb.dtm.txnaborts Number of transaction aborts. This is a cumulativecounter since the last sqstart. The delta can becomputed in the TSD query.
esgyndb.dtm.txnbegins Number of transaction begins. This is acumulative counter since the last sqstart. Thedelta can be computed in the TSD query.
esgyndb.dtm.txncommits Number of transaction committed. This is acumulative counter since the last sqstart. Thedelta can be computed in the TSD query.
esgyndb.mxosrvr.configure Number of configured master executor processes
esgyndb.mxosrvr.running Number of master executor processes that are up
esgyndb.mxosrvr.down Number of master executor processes that aredown
esgyndb.dcsmaster.configure Number of configured DCS master processes
esgyndb.dcsmaster.running Number of DCS master processes that are up
esgyndb.dcsmaster.down Number of DCS master processes that are down
esgyndb.mxosrvr.aggrstat.total_selects Total number of completed select statements
esgyndb.dcsserver.configure Number of configured DCS server processes
esgyndb.dcsserver.running Number of DCS server processes that are up
esgyndb.dcsserver.down Number of DCS server processes that are down
esgyndb.rms.configure Number of configured RMS processes
esgyndb.rms.running Number of RMS processes that are up
esgyndb.rms.down Number of RMS processes that are down
esgyndb.mxosrvr.aggrstat.total_completed_stmts Total number of completed statements
esgyndb.mxosrvr.aggrstat.total_rows_retrieved Total number of rows retrieved
esgyndb.mxosrvr.aggrstat.total_num_rows_iud Total number of rows inserted/updated/deleted
esgyndb.mxosrvr.aggrstat.total_inserts Total number of completed insert statements
sgyndb.mxosrvr.aggrstat.total_updates Total number of completed update statements
esgyndb.mxosrvr.aggrstat.total_deletes Total number of completed delete statements
esgyndb.mxosrvr.aggrstat.total_ddl_stmts Total number of completed DDL statements
esgyndb.mxosrvr.aggrstat.total_util_stmts Total number of completed SQL util statements
esgyndb.mxosrvr.aggrstat.total_catalog_stmts Total number of completed catalog statements
EsgynDB Manager User Guide
Chapter 4. Using EsgynDB Manager | 21
Chapter 5. Dashboard
The dashboard page displays the status of the EsgynDB services, nodes, active sessions and system
performance as time-series charts for key system and HBase metrics.
The dashboard page by default auto-refreshes every 30 seconds and displays the more current
status. You can change the time window or turn OFF auto-refresh using the Auto Refresh drop-down
control. You can also use the Refresh button to force a refresh anytime.
If you select a different time range using the Custom option, then you see the system metrics as of
that end time.
Figure 3. Dashboard
5.1. Service Status
The service status displays the status of the core EsgynDB services.
For each service, you can see the number of configured processes, the number of actual running
processes, the number of down processes if any.
EsgynDB Manager User Guide
22 | Chapter 5. Dashboard
• The status icon is green if all configured servers are up.
• The status icon is Yellow (Warning) if the number of actual servers is not equal to the configured
servers.
• The status icon is RED (Error) if actual server count is 0 or if the actual servers are < 30% of the
configured servers.
A tooltip on the status icon displays down process information if there are any. You can also export
the data to a spreadsheet or clipboard. The service status information is provided by the EsgynDB
REST Server which invokes the trafcheck script.
Click on the zoom-in icon to open a drill-down page and view the service status in a larger window.
5.2. Node Status
The node status reports a summary of the number of EsgynDB nodes that are UP and/or DOWN.
This information is provided by the EsgynDB REST Server which invokes the trafnodestatus script.
Click on the zoom-in icon to open a drill-down page and view the status of individual nodes. You can
use the search filter or column level filter to quickly search for specific nodes.
5.3. Active Sessions
The active sessions widget reports the number of active connections/sessions to EsgynDB.
If you click on the zoom-in icon or on the active session count hyperlink, you’ll be taken to the
Connectivity page that displays the details of the active sessions. You can see which application/user
is currently connected.
5.4. Transaction Counts
A time series chart to show the number of EsgynDB transaction begins, commits and aborts across
all nodes.
The chart uses the OpenTSDB metrics esgyndb.dtm.txnaborts, esgyndb.dtm.txnbegins,
esgyndb.dtm.txncommits. These metrics are collected and published every 5 minutes.
Click on the zoom-in icon to open a drill-down page and view the time-series data for each of the
transaction metrics.
EsgynDB Manager User Guide
Chapter 5. Dashboard | 23
5.5. Canary Response
The response times from an EsgynDB database canary check. The metrics displayed are the time
taken to open a JDBC connection, and time taken to read (select) from the canary table.
The chart uses the OpenTSDB metrics esgyndb.canary.sqlconnect.time and
esgyndb.canary.sqlread.time. These metrics are collected and published every 5 minutes.
Click on the zoom-in icon to open a drill-down page and view the time-series data for the individual
canary metric.
5.6. CPU Busy %
The average/min/max CPU Busy % across all nodes. It is easy to spot skews if there is big variations
between avg/min/max.
The chart uses OpenTSDB metric proc.stat.cpu_percent.
Click on the zoom-in icon to open a drill-down page and view the % CPU Busy on each node. Each
line in the drill-down graph represents a node.
5.7. IO Waits
The average/min/max IO Wait time across all nodes.
The chart uses OpenTSDB metric proc.stat.cpu{type=iowait}.
Click on the zoom-in icon to open a drill-down page and view the IO Wait time for each node. Each
line in the drill-down graph represents a node.
5.8. Free Memory %
The average/min/max free memory % across all nodes.
The chart uses OpenTSDB metric proc.meminfo.free_percent.
Click on the zoom-in icon to open a drill-down page and view the % free memory on each node.
Each line in the drill-down graph represents a node.
EsgynDB Manager User Guide
24 | Chapter 5. Dashboard
5.9. Disk Space Used %
The average/min/max disk space used percentage across all nodes.
The chart uses OpenTSDB metric df.bytes.percentused.
Click on the zoom-in icon to open a drill-down page and view the disk usage % for each node. Each
line in the drill-down graph represents a node.
5.10. Network IO
The average/min/max network IO (In/Out) measured in MB.
The chart uses OpenTSDB metric proc.net.bytes.
Click on the zoom-in icon to open a drill-down page and view the average network IO on each node.
Each line in the drill-down graph represents a node.
5.11. Regionserver Memory Use
The average/min/max memory (MB) used by the regionservers in the cluster.
The chart uses OpenTSDB metric hbase.regionserver.jvmmetrics.MemHeapUsedM.
Click on the zoom-in icon to open a drill-down page and view the average memory used by each
regionserver/node. Each line in the drill-down graph represents a node.
5.12. GC Time
The average/min/max JVM garbage collection time in milliseconds.
The chart uses OpenTSDB metric hbase.regionserver.jvmmetrics.GcTimeMillis.
Click on the zoom-in icon to open a drill-down page and view the average GC time for each node.
Each line in the drill-down graph represents a node.
5.13. Memstore Size
The total memstore size (MB) of the different regions in the cluster.
EsgynDB Manager User Guide
Chapter 5. Dashboard | 25
The chart uses OpenTSDB metric hbase.regionserver.regions.memStoreSize.
Click on the zoom-in icon to open a drill-down page and view the total memstore file size in each
node. Each line in the drill-down graph represents a node.
5.14. Drill-down details
All the panels in the dashboard page support drill-downs. The drill-downs help you view the dataset
better and for the system metric graphs, you can view the time series data on a per node basis.
In each of the dashboard panels, you should see this double arrow icon as highlighted here.
Figure 4. Dashboard Drilldown Menu
• In the drill-down page, you can view the time-series data on a per node basis.
• You can include or exclude individual nodes from the display by using the checkboxes on the node
names.
• If the graph is a multi-metric graph, like shown in the example below, you can use the Metric Name
selector to switch between metrics.
• As you move the mouse over the graphs, you’ll see a red spline line follow the graph and metric
values for the corresponding data point are displayed next to the node names.
EsgynDB Manager User Guide
26 | Chapter 5. Dashboard
Figure 5. Dashboard Drilldown Graph
EsgynDB Manager User Guide
Chapter 5. Dashboard | 27
Chapter 6. Workloads
This chapter describes how to monitor workloads (queries) that are currently active and also how to
analyze queries that completed in the past. These workloads include all EsgynDB queries including
both user and system queries.
6.1. Monitoring
6.1.1. Active Workloads
The active workloads page displays the list of EsgynDB queries that are currently active. This
includes queries that are currently executing and queries that completed within the last 30 seconds.
The information is obtained from the EsgynDB Runtime Manageability Service.
You can access the Active Workloads page using the Workloads >Active navigation menu.
The list of active queries is displayed with the following information:
Last Activity Seconds
A positive number shows how long the query has been active and is still executing. A negative
number shows how long since the query completed execution. Queries remain in RMS memory
for 30 seconds after completion.
Query ID
The unique identifier for the query. The Query ID is displayed as a hyper-link that will let you
drill-down to view the detailed runtime statistics for the query.
Execution State
The current state of the query. Valid values are OPEN, EXECUTE, FETCH, DEALLOCATED or
CLOSE.
Query Text (preview)
The first 255 characters of the SQL statement of the query.
Full Text Length
The length of the full query text.
EsgynDB Manager User Guide
28 | Chapter 6. Workloads
Node ID
The Node where the master executor processing the query is running.
Process ID
The Process ID of the master executor processing the query.
User
The User who initiated the query.
Tenant
The user has logged as this tenant to run the query. This column is only displayed when multi-
tenancy feature is enabled.
Profile
The workload profile assigned to the session in which the query is executing. See Profiles for
details.
SLA
The workload SLA mapped to the session in which the query is executing. See SLAs for details.
You can click on the Query ID hyper-link to drill-down into the details of that specific query.
The page by default refreshes every 30 seconds and displays the current active queries. Use the
Refresh button to manually refresh the list of active queries.
If you want to cancel one or more active queries, select the queries in the grid and click the Cancel
button to cancel the selected queries. Use CTRL + Click to select more than 1 query in the grid.
6.1.2. Active Query Details
The active query details page displays a summary of the runtime metrics as well the operator level
statistics for the selected query. The metrics are obtained from the EsgynDB Runtime Manageability
Service.
The metrics that have changed since the last refresh for this query, are highlighted in blue color, so
you can observe how the query is progressing. For example, if the CPU time is increasing, you know
the query is progressing. If the CPU time does not increase, it indicates a query expression operator
is waiting for some other resource.
EsgynDB Manager User Guide
Chapter 6. Workloads | 29
The page is set to auto-refresh every 30 seconds by default. You can also use the Refresh button to
refresh the page and reload the latest metrics for the query.
Click on the Explain Plan button to view the explain plan for this query.
Use the Cancel button to cancel the query.
6.1.3. Historical Workloads
You can use the historical workloads page to view and analyze queries that were executed in the
past.
This includes queries that completed in the past and currently active queries that have run longer than
a pre-configured threshold set in the Connectivity services configuration.
The threshold is defined by the dcs.server.user.program.statistics.limit.time property in dcs-
site.xml. If this property is not explicitly set, then the default value of 60 seconds applies.
Queries that run longer (elapesed time) than this run threshold are recorded in the EsgynDB
manageability repository tables in the "_REPOS_" schema. The historical workloads page fetches the
query summary from the "_REPOS_".METRIC_QUERY_TABLE.
You can access the Historical Workloads page using the Workloads >Historical navigation menu.
• By default, the page displays the queries that either started or completed in the last 1 hour or is
still executing.
• You can use the Filters button to change your search parameters. In the filter dialog, you can
specify different time ranges or use additional filters like specific application names or user names
or queries that match a particular query text. All the filter predicates are applied using AND
operator, meaning that only queries that match all the filter predicates are returned.
• Additionally, you can use the search bar above the grid to filter rows displayed in the grid. The
search string will match against all columns in the grid. For example, the query signature column
is a numeric hash of the query text after filtering out the literal values. So the query signature
should compute to the same value every time the query runs. Use the query signature id filter, you
can easily see all past runs of the query and compare the performance of each run.
• You can also use the column level filter text box at the bottom of the grid to display rows that
match that column filter.
EsgynDB Manager User Guide
30 | Chapter 6. Workloads
• You will also see graphs for Top 5 queries that have the longest run times, most memory usage,
most CPU usage and most disk IOs.
• If you click on the data points on the Top 5 Graphs, the corresponding query is selected in the grid
and you can see the query details.
• Also when you click on the Top 5 graphs, the grid is sorted by the corresponding column in the
grid. For example if you click on the Top 5 Memory graph, the list of workloads are sorted by the
Memory Used column in descending order such the queries using the most memory are displayed
first.
Use the Refresh button to refresh the page.
You can click on the Query ID hyper-link to drill-down into the details of a specific query.
6.1.4. Historical Query Detail
The historical query details page displays the details for the selected query.
The details are grouped together as summary, connection metrics, compile time metrics and run time
metrics.
The compile metrics includes estimates from the compiler. The runtime metrics are the actuals
reported by the Runtime Manageability Service. You can compare the compile time metrics to the run
time metrics and analyze the query behavior.
Click on the Explain button to view the explain plan for this query.
Use the Cancel button to cancel the query, if the query is a long running query and is still executing.
6.1.5. Query Plan
The Query Plan page displays the Query ID, the query text, a visual explain plan and a textual explain
plan.
The tables used by the query are displayed as hyper-links. This links will navigate you to the
Database page where you can view the table details like column information or the histogram stats on
the table.
Use the Refresh button to refresh the page and reload the visual and textual plans.
EsgynDB Manager User Guide
Chapter 6. Workloads | 31
If you double-click on a specific node in the explain tree, you will see the metrics for that specific
operator in a pop-up dialog.
You can use the Cancel button to cancel the query if the query is still executing.
6.1.6. Cancel Query
You can invoke the cancel action from any of the workload pages.
If the query is in EXECUTING state, a cancel request will be submitted to EsgynDB engine. If the
user has the necessary SQL privileges to cancel the query, the engine would schedule the cancel of
the query.
The cancel is done asynchronously. If you are in the same page, you’ll see a success or failure
message once the action completes. If you moved away from that page and navigated to a different
page, then the cancellation results are shown via the notifications panel. See Notifications Panel for
details.
6.2. Workload Configuration
Workload Configuration lets you define different runtime profiles and thresholds for queries.
This also helps with multi-tenancy by allowing the administrators to define different profiles for
different tenants or applications or users.
You can access the configuration page using the Workloads > Configuration navigation menu.
6.2.1. Profiles
A Workload profile defines the connection or session attributes and all the queries that execute within
that session inherit the same runtime profile.
The Workload profile lets you :
• Specify control statements that can provide hints to the compiler and executor to influence the
query plan and execution. For example, you can control the degree of parallelism.
• Specify some SET statements that can influence the session runtime behavior. For example, you
can set the default catalog or schema.
EsgynDB Manager User Guide
32 | Chapter 6. Workloads
• If you are running EsgynDB without multi-tenancy feature, then these next options are available.
• Specify the list of hosts to which the connection has to be made.
• You can also specify how the host selection should be enforced.
• If you choose the Preferred option, then the DCS Master will first try to find an available
MXOSRVR from the specified list of hosts. If the all the mxosrvrs are occupied, then DCS
Master will continue to look for an available mxosrvr on the remaining hosts.
• If you choose the Restrict option, the DCS Master will ONLY allow a mxosrvr from the
specified list of hosts. If there are no more mxosrvrs available in the specified list of nodes,
then the connection request is rejected with an error that no more servers are available to
connect.
• If you running EsgynDB with multi-tenancy feature, the host selection properties cannot be set
and are not displayed. The host selection is done implicitly based on the compute units allocated
for the tenant.
6.2.2. View Profiles
To view the list of defined profiles, use the Workloads > Configuration navigation menu and select
the Profiles tab.
The list of profiles is displayed with the following information:
Name
The profile name. The profile name is a displayed as an hyper-link. Clicking on the hyper-link
allows you to edit the profile attributes.
CQDs
The control query defaults defined if any.
SETs
Any defined SET statements.
Hosts
The list of hosts where session connections should be made. This is NOT displayed if multi-
tenancy is enabled.
EsgynDB Manager User Guide
Chapter 6. Workloads | 33
Selection Mode
Preferred or Restrict mode of mxosrvr selection from the list of hosts. This is NOT displayed if
multi-tenancy is enabled.
Last Update Time
When the profile configuration was last modified.
Use the Add Profile button to add a new workload profile definition.
Use the Refresh button to refresh the list of profiles.
6.2.3. Add Profile
When you click on the Add Profile button, you will see a pop-up dialog that lets you create a new
workload profile.
EsgynDB Manager User Guide
34 | Chapter 6. Workloads
Figure 6. Profile Details
• Enter the profile name
• As needed, specify a list of control query statements, each separated by a semi-colon.
EsgynDB Manager User Guide
Chapter 6. Workloads | 35
• As needed, specify a list of SET statements, each separate by a semi-colon.
• If multi-tenancy is not enabled:
• You can optionally specify a list of hosts to which the session connection should be made.
• Select how the mxosrvr host selection from the list of hosts should be enforced.
• If multi-tenancy is enabled, the host selection controls are not available and host selection is
implicitly done based on the compute unit allocations for the tenant.
• Click on Apply button to create the workload profile.
6.2.4. Alter Profile
Click on the profile name hyper-link in the list of profiles to alter the profile attributes. Except for the
profile name, you can modify the remaining attributes and click on Apply button to persist the
changes.
You cannot alter the system created profile defaultProfile.
Any changes to profile settings are picked up the next time a new connection is
mapped to the profile. Existing connections that are mapped to that profile continue
to use the old attributes of the profile.
6.2.5. Delete Profile
In the grid that lists all profiles, the last column displays a delete button. To delete a profile, just click
on its corresponding delete button. Click Yes in the confirmation dialog to delete the profile.
You cannot delete the system created profile defaultProfile.
You cannot delete a profile that is currently used by a SLA.
6.2.6. SLAs
Service Level Agreements (SLAs) allow you to define expected thresholds from an application or
tenant perspective. For example you can define the level of concurrency. SLAs also serve the
purpose of mapping user sessions to specific workload profiles to enforce the runtime attributes
defined in the profile.
EsgynDB Manager User Guide
36 | Chapter 6. Workloads
6.2.7. View SLAs
To view the list of defined SLAs, use the Workloads > Configuration navigation menu and select the
SLAs tab.
The list of SLAs is displayed with the following information:
Name
The SLA name. The SLA name is a displayed as an hyper-link. Clicking on the hyper-link allows
you to edit the SLA attributes.
Is Active
Indicates if the SLA is currently active or disabled. Inactive SLAs will be ignored when trying to
map connection requests to workload profiles.
OnConnect Profile
When the assigned mxosrvr goes from available to connecting state, the CQDs and SETs in the
OnConnect profile are applied.
OnDisconnect Profile
When the mxosrvr session disconnects, the CQDs and SETs in the OnDisconnect profile are
applied.
Max. Concurrent Sessions
The maximum concurrent active sessions allowed for any connections mapped to this SLA.
Last Update Time
When the SLA was last modified.
Use the Add SLA button to add a new workload SLA definition.
Use the Refresh button to refresh the list of SLAs.
6.2.8. Add SLA
When you click on the Add SLA button, you will see a pop-up dialog that lets you create a new
workload SLA.
EsgynDB Manager User Guide
Chapter 6. Workloads | 37
Figure 7. SLA
• Enter the SLA name
• Set the Is Active to Yes or No based on whether you want the SLA to be active immediately.
• Select a OnConnect profile from the drop-down list. When the assigned mxosrvr goes from
available to connecting state, the CQDs and SETs in this profile are applied in the session.
• Select a OnDisconnect profile drop-down list. When the mxosrvr session disconnects, the CQDs
and SETs in the OnDisconnect profile are applied.
• Optionally specify a maximum session count. If this value is not specified, then there is no session
limit. If you set it to 0, no sessions will be allowed for this SLA. Any positive number sets the max
session count.
• Click on Apply button to create the SLA.
EsgynDB Manager User Guide
38 | Chapter 6. Workloads
6.2.9. Alter SLA
Click on the SLA name hyper-link in the list of SLAs to alter the SLA attributes.
Except for the SLA name, you can modify the remaining attributes and click on Apply button to
persist the changes.
You cannot alter the system created SLA defaultSLA.
Any changes to SLA settings are picked up the next time a new connection is
mapped to the SLA. Existing connections that are mapped to that SLA continue to
use the old attributes of the SLA.
6.2.10. Delete SLA
In the grid that lists all SLAs, the last column displays a delete button.
To delete a SLA, just click on the corresponding delete button. Click Yes on the confirmation dialog to
delete the SLA.
You cannot delete the system created profile defaultSLA.
You cannot delete a SLA that is currently used in a mapping.
6.2.11. Mappings
Maps database connections to specific SLAs and profiles through mapping rules based on the
connection attributes.
You can use the connection attributes like application name, user name, tenant name, session name,
client IP address and host name and define different mappings for different applications and/or users.
You can also specify an unique mapping order number between 1 and 99. If multiple mappings match
the connection request, the DCS Master starts it search from order number 1 and returns the first
match. If the connection request does not match any mapping, then the defaultMapping is used.
If mapping contains multiple attributes then all attributes have to match the
connection request.
EsgynDB Manager User Guide
Chapter 6. Workloads | 39
Mappings can be also deactivated and re-activated. DCS Master will only consider the active
mappings.
6.2.12. View Mappings
To view the list of defined mappings, use the Workloads > Configuration navigation menu and
select the Mappings tab.
The list of mappings is displayed with the following information:
Name
The Mapping name. The Mapping name is a displayed as an hyper-link. Clicking on the hyper-
link allows you to edit the mapping attributes.
Tenant Name
The tenant name to match in the DCS connection request. This is only displayed if multi-
tenancy feature is enabled in EsgynDB.
User Name
The user name to match in the DCS connection request.
Application Name
The application name to match in the DCS connection request.
Session Name
The session name to match in the DCS connection request.
Client IP Address
The client IP address as detected by the DCS Master in the DCS connection request.
Client Host Name
The client workstation name to match in the DCS connection request.
SLA
The SLA name to which this mapping is mapped to.
Is Active
Indicates if the mapping is currently active or disabled.
EsgynDB Manager User Guide
40 | Chapter 6. Workloads
Order Number
The order number in which DCS Master tries to match the mappings. Starting from 1 to 99, the
first mapping that matches the connection request attributes is selected by DCS Master.
Last Update Time
When the mapping was last modified.
Use the Add Mapping button to add a new workload Mapping definition.
Use the Refresh button to refresh the list of mappings.
6.2.13. Add Mapping
When you click on the Add Mapping button, you will see a pop-up dialog that lets you create a new
workload Mapping.
EsgynDB Manager User Guide
Chapter 6. Workloads | 41
Figure 8. Mapping
• Enter the Mapping name
• Select a SLA from the drop-down list. If DCS Master selects this mapping, the OnConnectProfile
of this SLA is applied to the session.
• Select if the Mapping should be created in an active state or disabled state.
• Enter a unique order number as the search order.
• Enter one or more mapping criteria to match against the DCS connection request. The Tenant
Name property is available only when multi-tenancy is enabled and it will display a list of tenants
configured in the system.
• Click on Apply button to create the mapping.
Note that all specified mapping criteria have to match against the connection
request for this mapping to get selected.
EsgynDB Manager User Guide
42 | Chapter 6. Workloads
6.2.14. Alter Mapping
Click on the mapping name hyper-link in the list of mappings to alter the mapping attributes.
Except for the Mapping name, you can modify the remaining attributes and click on Apply button to
persist the changes.
Any changes to the mapping settings are picked up the next time a new connection
is received.
You cannot alter the system created mapping defaultMapping.
6.2.15. Delete Mapping
In the grid that lists all mappings, the last column displays a delete button. To delete a mapping, just
click on the corresponding delete button. Click Yes on the confirmation dialog to delete the mapping.
You cannot delete the system created mapping defaultMapping.
6.2.16. Profile/SLA/Mapping at Runtime
This section explains how the workload profile, SLA and mappings all come together at runtime.
• When a client connection request comes to DCS Master, it will use the attributes in the connection
request and try to find a matching mapping definition that is currently active, starting with the
mapping with order number 1 all the way to 99.
• Once a mapping is matched, then DCS Master looks at the SLA defined in the mapping. If the
SLA is marked as inactive, it is ignored and DCS will try to look for the next available mapping that
matches that has an active SLA.
• DCS Master maintains a HashMap of previously used mxosrvrs for specific user and sla. If the
mapped SLA has previously used mxosrvrs from the nodes defined in the mapped profile, that
are now in available state, a mxosrvr is selected randomly from that list to serve the connection
request.
• If the mapped SLA has no previously used mxosrvrs or if none of the previously used mxosrvrs
are in an available state, DCS Master then picks an available mxosrvr from the list of hosts
specified in the mapped profile.
EsgynDB Manager User Guide
Chapter 6. Workloads | 43
• If multi-tenancy is enabled, then DCS will also maintain a list of recently used mxosrvrs for the
tenant and try to reuse them if the mxosrvr is available.
• Once a mxosrvr is assigned to the connection, CQDs and SETs in the mapped profile are
executed on the mxosrvr session
• If no mapping is matched, the connections will use the defaultSLA and defaultProfile and any
available mxosrvr is assigned to serve the connection request.
6.2.17. Debugging mapping issues
If you are seeing that the mapping you defined is not being triggered and connections are being
mapped to incorrect sla and profile, you can set the DCS logging level to DEBUG.
If the dcs-2.5.0/conf/log4j.properties file, uncomment the 2 lines below and enable DEBUG logging
for the 2 class files.
# Uncomment this line to enable tracing of DcsMasterlog4j.logger.org.trafodion.dcs.master.DcsMaster=DEBUGlog4j.logger.org.trafodion.dcs.master.mapping=DEBUG
Save the file and restart DCS. Now when you make a connection, the DCS Master log will contain
debug messages that show the incoming connection request attributes and how DCS Master
attempted to match a mapping. These log entries will give a clue why the mapping is not getting
triggered.
EsgynDB Manager User Guide
44 | Chapter 6. Workloads
Chapter 7. Logs
The Logs page displays the log and event messages from the various EsgynDB processes. The logs
are fetched using the logs Table Mapping Function (TMUDF) which scans and filters the log entries
from all the EsgynDB component log files on all nodes and provides a single consolidated time
sequenced view.
Figure 9. Logs
7.1. Log Summary
The Log details include the following:
Time
The time when the log event occurred
Severity
The event severity (FATAL/ERROR/WARN/INFO/DEBUG)
Node
The Node on which the event was logged
EsgynDB Manager User Guide
Chapter 7. Logs | 45
Component
The EsgynDB component name reporting the event
Process
The component process name reporting the event
Error Code
Error code if applicable. SQL Errors have error codes.
Message
The event message text
QueryID
If the error is a SQL error, you’ll see an associated query id.
The logs page by default auto-refreshes every 5 minutes. You can change this refresh time window or
turn OFF auto-refresh using the Auto Refresh drop-down menu.
You can use the Logs Filters button to search for log entries during specific times or from specific
processes. You can also use the column level filter in the grid to filter entries based on search filters
on multiple columns.
7.2. Log Filter
By default, the logs page displays the log events with severity levels Fatal, Error, Warning in the last
1 hour.
You can use the Filters button to change this and specify custom filters for your search. The available
filters are:
Time Range
Select one of the pre-define time ranges or use the custom time range and provide a start and
end time. Only the log entries that are within this time range are fetched and displayed.
Severity
By default only Fatal, Error, Warning severity log messages are displayed. You can select the
severity levels by selecting the required checkboxes.
EsgynDB Manager User Guide
46 | Chapter 7. Logs
Component Names(s)
If are you only interested in specific components, check those component name checkboxes.
The table below describes the component names:
Component Name Description
MXOSRVR Master Executor
DCS Connectivity Servers
SQL SQL
SQL.COMP Compiler
SQL.ESP Executor Server Process
SQL.EXE Master Executor SQL
SQL.LOB SQL Large Objects
SQL.SSCP Runtime Manageability Service
SQL.SSMP Runtime Manageability Service
SQL.UDR User Defined Routines
MON Monitor
MON-SNMP Monitor SNMP
TM Transaction Monitor
WDG Watchdog
WDG-SNMP Watchdog SNMP
PSD Process Startup Daemon
BACKUP_RESTORE Backup Restore
REST REST Server
DBSECURITY Database Security
KERBEROS Kerberos
Process Name(s)
You can also filter on specific process names by entering the process names as comma
separated list.
Node Name
You can also filter on a specific node by selecting the node name from the drop-down list.
Error Code(s)
You can also filter on specific error codes by entering the error code as comma separated list.
You can only enter non-zero integer values.
EsgynDB Manager User Guide
Chapter 7. Logs | 47
Message
You can also provide a partial message text and all event messages that contain this text will be
selected.
Max Fetch Rows
By default, only the first 5000 rows that match the filter criteria is returned. You can increase this
row count by using the drop-down values.
EsgynDB Manager User Guide
48 | Chapter 7. Logs
Chapter 8. Database
The Database page allows you browse the EsgynDB schemas, Hive schemas and the schema
objects and their attributes.
You can use the hyperlinks on the object names or the bread crumbs at the top of the page to
navigate between the different sub-pages displaying database objects and their attributes.
8.1. Catalogs
The catalogs list displays all the EsgynDB TRAFODION catalog and HIVE catalog.
Use the TRAFODION catalog to browse through EsgynDB schemas and objects. Use the HIVE
catalog to browse hive schemas and tables.
8.2. Schemas
The schemas list displays all the schemas in the selected catalog.
Name
The name of the schema. The schema name is displayed as a hyperlink and allows you
drilldown into the schema details.
Owner
The database user or role that owns the schema. This attribute is only displayed for EsgynDB
schemas in TRAFODION catalog.
CreateTime
The time the schema was created. This attribute is only displayed for EsgynDB schemas in
TRAFODION catalog.
ModifiedTime
The time the schema was last modified. This attribute is only displayed for EsgynDB schemas in
TRAFODION catalog.
EsgynDB Manager User Guide
Chapter 8. Database | 49
8.3. Schema Details
From the Schemas page, you can click on the hyperlink on a specific schema to view the details.
8.3.1. Attributes
The attributes tab displays the schema attributes like the name, the owner, create time and modified
time. The owner, create time and modified time is only displayed for schemas in TRAFODION
catalog.
8.3.2. DDL
The DDL tab displays the CREATE SCHEMA statement used to create this schema. This output is
obtained using the SHOWDDL statement.
8.3.3. Privileges
The Privileges tab displays all the SQL privilege information for an EsgynDB schema in TRAFODION
catalog. It shows the grantee and grantor names and the list of privileges for the grantee.
8.4. Tables
In the schema details page, you can click on the Tables link to view all the tables in that schema. The
information displayed is different for EsgynDB tables and Hive tables.
8.4.1. EsgynDB Tables
For EsgynDB tables in TRAFODION catalog, you can get a short summary list or a detailed list that
includes the partitioning and HBase region attributes for each table.
Use the Summary or Detail radio button to toggle the display.
Table Name
The name of the table. The table name is displayed as a hyperlink and allows you drilldown into
the table details.
Owner
EsgynDB Manager User Guide
50 | Chapter 8. Database
The database user that owns the table.
CreateTime
The time the table was created.
ModifiedTime
The time the table was last modified.
KeyLength
The length of the key columns.
RowTotalLength
The total length of the table row. Indicates how wide the table is.
Number Salt Partitions
For a partitioned table, it shows the number of salt partitions. For non-partitioned tables, this
value is 0.
Region Count
The total number of HBase regions for the table. This value can be higher than the number of
salt partitions because of region splits.
Store File Size
The size of the store files in MB.
MemStore Size
The size of the memstore in MB.
Read Requests Count
The cumulative count of read requests or scans since the last region flush or HBase restart.
Write Requests Count
The cumulative count of write requests or puts since the last region flush or HBase restart.
8.4.2. Hive Tables
The Hive table list displays the following information for all tables in a hive schema.
EsgynDB Manager User Guide
Chapter 8. Database | 51
Table Name
The name of the table. The table name is displayed as a hyperlink and allows you drilldown into
the table details.
Owner
The user that owns the table.
CreateTime
The time the table was created.
Table Type
The hive table type. EXTERNAL_TABLE or MAPPED_TABLE.
File Format
The hive file format (Text / ORC / Parquet).
Number of Columns
The number of columns in the tables
Number of Partition Columns
The number of partition columns for partitioned tables
Field Delimiter
The field delimiter for hive table row
8.5. Table Details
From the Tables page, you can click on the hyperlink on a specific table to view the details.
8.5.1. Attributes
The attributes tab displays the table attributes.
8.5.2. Columns
The Columns tab displays the details about the table columns including column name, data type, size
and default values.
EsgynDB Manager User Guide
52 | Chapter 8. Database
8.5.3. Regions
This feature is only available for EsgynDB tables.
The Regions tab displays the HBase region details for all the HBase regions that make up this table.
8.5.4. Statistics
This feature is only available for EsgynDB tables.
The Statistics tab displays the histogram statistics for the table, if update statistics have been run on
the table.
You can use the last update stats timestamp column to see how outdated the table statistics are and
choose to update the statistics for the table.
8.5.5. DDL
The DDL tab displays the CREATE TABLE statement used to create this table. This output is obtained
using the SHOWDDL statement.
8.5.6. Privileges
This feature is only available for EsgynDB tables.
The Privileges tab displays all the SQL privilege information for the table. It shows the grantee and
grantor names and the list of privileges for the grantee.
8.5.7. Indexes
This feature is only available for EsgynDB tables.
The Indexes tab displays all the indexes defined on the table.
8.5.8. Usage
This feature is only available for EsgynDB tables.
The Usage tab displays all the SQL objects that use this table. For example, you can see all the SQL
EsgynDB Manager User Guide
Chapter 8. Database | 53
Views that use this table.
8.6. Views
This feature is only available for EsgynDB schemas.
In the schema details page, you can click on the Views link to view all the views in that schema.
View Name
The name of the EsgynDB schema. The view name is displayed as a hyperlink and allows you
drilldown into the view details.
Owner
The database user that owns the view.
CreateTime
The time the view was created.
ModifiedTime
The time the view was last modified.
CheckOption
Specifies that no row can be inserted or updated in the database through the view unless the
row satisfies the view definition—that is, the search condition in the WHERE clause of the query
expression must evaluate to true for any row that is inserted or updated. This option is only
allowed for updatable views. Refer to the EsgynDB SQL Reference manual for more details.
Updateable
Indicates if the view is updateable. Single table views can be updatable. Multi-table views
cannot be updatable. Refer to the EsgynDB SQL Reference manual for more details.
Insertable
Indicates if the view is insertable. Single table views can be insertable. Multi-table views cannot
be insertable. Refer to the EsgynDB SQL Reference manual for more details.
EsgynDB Manager User Guide
54 | Chapter 8. Database
8.7. View Details
From the Views page, you can click on the hyperlink on a specific view to see its details.
8.7.1. Attributes
The attributes tab displays the view attributes.
8.7.2. Columns
The Columns tab displays the details about the view columns.
8.7.3. DDL
The DDL tab displays the CREATE VIEW statement used to create this view. This output is obtained
using the SHOWDDL statement.
8.7.4. Privileges
The Privileges tab displays all the SQL privilege information for the view. It shows the grantee and
grantor names and the list of privileges for the grantee.
8.7.5. Usage
The Usage tab displays all the SQL objects that use this view and also all the SQL objects used by
this view. For example, this view can have a child view. This page will display the parent base table as
well as the child view. The Usage_Type column shows either 'Uses' or 'Used By' to indicate the
relation of the view to the displayed object.
8.8. Indexes
This feature is only available for EsgynDB tables.
In the schema details, page you can click on the Indexes link to view all the indexes in that schema.
Index Name
The name of the index. The index name is displayed as a hyperlink and allows you drilldown into
EsgynDB Manager User Guide
Chapter 8. Database | 55
the index details.
Owner
The database user that owns the index.
CreateTime
The time the index was created.
ModifiedTime
The time the index was last modified.
Table Name
The name of the base table on which the index is defined. The table name is displayed as a
hyperlink and allows you drilldown into the table details.
Is Unique
Indicates if the index is a Unique or Non-Unique index.
Key Col. Count
The number of columns that make up the key for the index. For non-unique index, this includes
the columns defined in the index as well as the base table key columns.
Non Key Col. Count
For a unique index, this count shows the number of base table key columns that are not part of
the index key.
8.9. Index Details
From the Indexes page, you can click on the hyperlink on a specific index to view the details.
8.9.1. Attributes
The attributes tab displays the index attributes.
8.9.2. Regions
The Regions tab displays the HBase region details for all the HBase regions that make up this index.
EsgynDB Manager User Guide
56 | Chapter 8. Database
8.9.3. DDL
The DDL tab displays the CREATE INDEX statement used to create this index. This output is
obtained using the SHOWDDL statement.
8.10. Libraries
This feature is only available for EsgynDB schemas.
In the schema details, page you can click on the Libraries link to view all the EsgynDB libraries in
that schema.
Library Name
The name of the library. The library name is displayed as a hyperlink and allows you drilldown
into the library details.
Owner
The database user that owns the library.
CreateTime
The time the library was created.
ModifiedTime
The time the library was last modified.
Code File Name
The name of the jar or dll that this library encapsulates. The library code file is stored under
$TRAF_HOME/udr/lib/DB__ROOT.
8.10.1. Create Library
Creating a library is a two-step process. You first deploy the code file to the EsgynDB platform and
then create a library.
You can use the Create Library tool to perform these two steps with a single click. You can access
this tool in two ways.
EsgynDB Manager User Guide
Chapter 8. Database | 57
• In the Libraries page, use the Create Library button.
• Use the Tools -> Create Library navigation menu.
To create a library, do the following:
• Enter a valid schema name in the Schema Name field. If the schema name contains special
characters or you need to preserve the case, you need to delimit the name within double quotes.
The schema name is pre-filled when you navigate to the Create Library tool from the Libraries
page.
• Enter a valid library name in the Library Name field. If the library name contains special
characters or you need to preserve the case, you need to delimit the name within double quotes.
• Use the browse button and select a .jar file or a .dll file on your workstation
• If you are re-using an existing code file or if the create library fails with an error that the code file
already exists, you can check the Overwrite existing code file option and the code file on the
server will be replaced by the selected code file.
• Click on the Create button to create the library.
• The results of the create will be displayed in a success or failure message box.
8.11. Library Details
From the Libraries page, you can click on the hyperlink on a specific library to see its details.
8.11.1. Attributes
The attributes tab displays the library attributes including the full path of the code file on the server.
8.11.2. DDL
The DDL tab displays the CREATE LIBRARY statement used to create this library. This output is
obtained using the SHOWDDL statement.
8.11.3. Privileges
The Privileges tab displays all the SQL privilege information for the library. It shows the grantee and
grantor names and the list of privileges for the grantee.
EsgynDB Manager User Guide
58 | Chapter 8. Database
8.11.4. Usage
The Usage tab displays all the user defined routines (stored procedures and functions) that use this
library.
8.12. Alter Library
You can alter a library to update the code file. For example, if you have modified the stored procedure
java code and want to update the code file on the server, you can use the Alter library feature to
update the code file.
• Navigate to the Attributes tab of a specific library in the Library Details page.
• Click on the Alter button. It will take to you the Alter Library window.
• The schema name is pre-filled from the Library details page and cannot be changed.
• The library name is pre-filled from the Library details page and cannot be changed.
• Use the browse button and select a .jar file or a .dll file on your workstation.
• If you are re-using an existing code file or if the create library fails with an error that the code file
already exists, you can check the Overwrite existing code file option and the code file on the
server will be replaced by the selected code file.
• Click on the Alter button.
• The results of the alter will be displayed in a success or failure message box.
8.13. Drop Library
You can drop a library and remove it from the EsgynDB metadata.
• Navigate to the Attributes tab of a specific library in the Library Details page.
• Click on the Drop button to drop the library.
• A success or failure message is displayed.
8.14. Download Library
You can download the code file of library to your local workstation. This is helpful if you want to upload
it another EsgynDB instance.
EsgynDB Manager User Guide
Chapter 8. Database | 59
• Navigate to the Attributes tab of a specific library in the Library Details page.
• Click on the Download button to download the code file to the default downloads folder on your
workstation.
• A success or failure message is displayed.
8.15. Procedures
This feature is only available for EsgynDB schemas.
In the schema details, page you can click on the Procedures link to view all the java stored
procedures in that schema.
Name
The name of the procedure. The procedure name is displayed as a hyperlink and allows you
drilldown into the procedure details.
Owner
The database user that owns the procedure.
CreateTime
The time the procedure was created.
ModifiedTime
The time the procedure was last modified.
Library Name
The name of the library object used by the procedure. The library name is displayed as a
hyperlink and allows you drilldown into the library details.
8.16. Procedure Details
From the Procedures page, you can click on the hyperlink on a specific procedure to see its details.
8.16.1. Attributes
The attributes tab displays the procedure attributes.
EsgynDB Manager User Guide
60 | Chapter 8. Database
8.16.2. DDL
The DDL tab displays the CREATE PROCEDURE statement used to create this procedure. This
output is obtained using the SHOWDDL statement. The DDL output shows the number of IN and
OUT parameters and their data types.
8.16.3. Privileges
The Privileges tab displays all the SQL privilege information for the procedure. It shows the grantee
and grantor names and the list of privileges for the grantee.
8.16.4. Usage
The Usage tab displays the library used by this stored procedure.
8.17. Functions
This feature is only available for EsgynDB schemas.
In the schema details, page you can click on the Functions link to view all the user defined functions
in that schema. This lists both the Scalar Functions and Table Mapping Functions. T
Name
The name of the function. The procedure name is displayed as a hyperlink and allows you
drilldown into the function details.
Owner
The database user that owns the function.
CreateTime
The time the function was created.
ModifiedTime
The time the function was last modified.
Function Type
Identifies the type of the function. Scalar Function or Table Mapping Function.
EsgynDB Manager User Guide
Chapter 8. Database | 61
Language Type
Identifies the language in which the function is written.
Library Name
The name of the library object used by the function. The library name is displayed as a hyperlink
and allows you drilldown into the library details.
8.18. Function Details
From the Functions page, you can click on the hyperlink on a specific function to see its details.
8.18.1. Attributes
The attributes tab displays the function attributes.
8.18.2. DDL
The DDL tab displays the CREATE FUNCTION statement used to create this user defined function.
This output is obtained using the SHOWDDL statement. The DDL output shows the number of IN and
OUT parameters and their data types.
8.18.3. Privileges
The Privileges tab displays all the SQL privilege information for the function. It shows the grantee and
grantor names and the list of privileges for the grantee.
8.18.4. Usage
The Usage tab displays the library used by this function.
EsgynDB Manager User Guide
62 | Chapter 8. Database
Chapter 9. Connectivity
The Connectivity page displays information about the Data Connectivity service (DCS) summary and
the configured Master Executor processes and the current active sessions.
Figure 10. Connectivity
9.1. Service Summary
The summary information displays information about the DCS Master. DCS Master is the master
process that routes and assigns client connection requests to master executor processes
(MXOSRVR).
You can see how many DCS masters are configured and on what nodes the current active (primary)
DCS Master is running. You can also see the port on which the DCS Master is listening. This is the
port that clients need to use in their JDBC/ODBC connections.
9.2. Master Executor summary
The Master executor summary provides a summary of the total configured master executors
(MXOSRVR), the actual master executors that are up and the number of down processes if any. For
the Actual number of Master executors you get a further count of how many are in
Available/Connected/Connecting states.
EsgynDB Manager User Guide
Chapter 9. Connectivity | 63
9.3. Master Executor Processes
The master executor processes execute the client queries.
• The list displays all the configured master executor processes that are running across all the
nodes. This includes the master executor processes that are currently connected and the master
executors that are available for new connections.
• If the master executor process is in a connected state, you will see the session information like the
connected user name, the application name, client workstation, the workload profile and sla
mapped to that connection.
• If multi-tenancy is enabled, for the connected master executor processes, you also see the tenant
name
If a client connection is rejected with error message that no more servers are
available, you can use this page to confirm that all master executor processes are
up and are indeed used up and then increase the number of master executors as
needed.
To increase the number of master executor processes, increase the number of configured servers in
the $DCS_INSTALL_DIR/conf/servers file, propagate this file to all nodes and restart the
connectivity service (DCS).
For installs done using Cloudera Parcels or Ambari, use the Cloudera/Ambari GUI to increase the
number of master executors.
9.4. Child processes
You can look at all the child processes launched by an mxosrvr. These processes can be an compiler
process or executor server process (ESPs) that prepare and execute the query in parallel.
Select a mxosrvr from the list and then click on the Process List button. The Process List dialog will
display the mxosrvr process and all the child processes across all the nodes.
9.5. Process Stack
You can look at the current execution process stack for a master executor process, by selecting the
master executor process in the grid and then using the Process Stack button.
EsgynDB Manager User Guide
64 | Chapter 9. Connectivity
A new pop-up window displays the mxosrvr process execution stack. This is helpful in analyzing if a
query is stuck in processing and if the master executor is waiting for resources.
EsgynDB Manager User Guide
Chapter 9. Connectivity | 65
Chapter 10. Security
EsgynDB supports authentication and user/role based authorization. EsgynDB SQL supports
standard ANSI SQL privileges on the EsgynDB database objects.
In addition, EsgynDB supports functional privileges called component privileges. The functional
privileges allow an authorized user to perform specific operations or use certain features.
Privileges can be granted to database users or roles.
You can also register user groups and grant roles to user groups. This is typically done for LDAP
users where users are grouped within LDAP groups.
While Roles are used to group privileges, user groups are used to group users. You grant required
privileges to a role and grant the role to the user group. Then all users in the user group inherit the
privileges.
Access the Roles/Users/User Groups pages using Security -> Accounts menu.
10.1. Roles
A role offers the flexibility of implicitly assigning a set of privileges to users, instead of assigning
privileges individually. A role can be granted to one or more users or user groups.
A role can be granted by or revoked by a database user administrator, a role owner, or a member of
the role.
10.1.1. Roles Summary
The Roles summary page displays all EsgynDB roles.
Use the Add Role button to add a new database role.
10.1.2. Role Details
In the Role summary page when you click on a specific role, you’ll be taken to the role details page.
EsgynDB Manager User Guide
66 | Chapter 10. Security
Attributes
The attributes tab displays the role attributes like the role name, the Role UID, the creator and the
create/modified time.
Grantees
The Grantees tab displays all the database users and user groups granted this role. The Grantee
Type column indicates if the grantee is a user or a user group.
Click on the Grant/Revoke button to grant the role to additional users/groups or revoke the role from
some users/groups.
SQL Privileges
The SQL Privileges tab displays all the EsgynDB SQL privileges granted explicitly to the role.
Component Privileges
The Component privileges tab displays all the component level privileges granted to the role. If you
are an administrative user, you can click on the Grant/Revoke button to grant or revoke component
privileges to the role.
For more details on Component Privileges, please refer to the EsgynDB SQL Reference manual.
10.1.3. Delete Role
In the roles summary page, last column in the tabular grid displays a delete button. To delete a role,
just click on the role’s corresponding delete button.
You can also use the delete button in the role details page to delete that role.
10.2. Users
When authentication is enabled, all access to EsgynDB has to be made with user credentials
registered in the database. If authorization is enabled, then the user needs to have the right
authorizations to access the SQL objects and use the EsgynDB features.
If LDAP or Active Directory is configured, then the login id (LDAP user) has to be registered and
EsgynDB Manager User Guide
Chapter 10. Security | 67
mapped to a database user name in EsgynDB.
If the TRAF_AUTO_REGISTER_USER CQD is set to ON, then the ldap users are
auto-registered as database users when they first connect, if they have no existing
entry in EsgynDB Metadata.
The JDBC/ODBC connections are made using the LDAP user id, but once the external LDAP id is
authenticated, then SQL uses the internal database for checking and enforcing authorizations.
10.2.1. Users Summary
The Users summary page displays all registered EsgynDB database users.
Use the Add User button to add a new database user.
10.2.2. Add User
In the Add User dialog, specify the external LDAP or active directory user name.
The database user name is an optional field. If the external user name does not contain special
characters, you can leave the database user name blank. The external user name will then be used
as the database user name. But you can optionally choose to specify a shorter descriptive name for
the database user name.
If you have configured EsgynDB with multiple LDAP/AD servers, select the LDAP configuration name
that maps to the LDAP Server in which the user is defined.
10.2.3. User Details
In the User summary page when you click on a specific user, you’ll be taken to the user details page.
Attributes
The attributes tab displays the user attributes like the User name, the User UID, the creator and the
create/modified time.
Edit User Attributes
You can use the Edit button to change the LDAP configuration section name or mark the user as
EsgynDB Manager User Guide
68 | Chapter 10. Security
inactive to prevent further logins from that user.
Roles
The Role tab displays all the roles granted to this user. Click on the Grant/Revoke button to grant or
revoke roles from the user.
SQL Privileges
The SQL Privileges tab displays all the EsgynDB SQL privileges granted explicitly to the user.
Component Privileges
The Component privileges tab displays all the component level privileges granted to the user. If you
are an administrative user, you can click on the Grant/Revoke button to grant or revoke component
privileges to the user.
For more details on Component Privileges, please refer to the EsgynDB SQL Reference manual.
10.2.4. Delete User
In the users summary page, last column in the tabular grid displays a delete button. To delete a user,
just click on the users’s corresponding delete button.
You can also use the delete button in the user details page to delete that user.
10.3. User Groups
Multiple users can be combined into a single group called a user group. When privileges or roles are
granted to a user group, it effectively grants them all the users in the group. When privileges or roles
are revoked from a user group, it effectively revokes them from all users in the group. Groups are
defined in AD/LDAP and registered in EsgynDB.
The Groups tab displays all the user groups registered in EsgynDB.
Click on the Add User Group button to register a new user group.
EsgynDB Manager User Guide
Chapter 10. Security | 69
10.3.1. Add User Group
In the Add User Group dialog, specify the external LDAP or active directory group name.
If you have configured EsgynDB with multiple LDAP/AD servers, select the LDAP configuration name
that maps to the LDAP Server in which the group is defined.
10.3.2. User Group Details
In the User Groups summary page when you click on a specific user group, you’ll be taken to the user
group details page. Here you can view/alter the group details and roles granted to the group.
Edit User Group
You can change the LDAP configuration name for the user group using the Edit button.
Grant/Revoke Role to User Group
Click on the Roles tab to view the roles granted to the user group.
Click on the Grant/Revoke button to grant or revoke roles to the user group. All database users
mapped to this user group inherited the role privileges.
10.3.3. Delete User Group
In the user groups summary page, last column in the tabular grid displays a delete button. To delete a
user group, just click on the user group’s corresponding delete button.
You can also use the delete button in the user group details page to delete that user group.
10.4. Roles based Authorization for DB Manager
DB Manager uses the Component Privileges infrastructure to enable and disable some of it features
to authorized users only.
The table below shows the required privileges to access the different DB Manager features.
A user with DB__ROOTROLE can grant these component privileges to roles and users.
EsgynDB Manager User Guide
70 | Chapter 10. Security
DB Manager Page Feature Required Privilege Comments
Dashboard View System andService Status
NONE
Dashboard View System Metrics NONE
Active Workloads View RMS Query Statsand Query Plan
DBMGR.SHOW_ACTIVE_QUERIES
Internally DB Managerwill connect asDB__ADMIN user tofetch this information butthe UI and DB ManagerServer side call willenforce that the loginuser has theDBMGR.SHOW_ACTIVE_QUERIES componentprivilege
Historical Workloads View Query stats inRepository and queryplan
DBMGR.SHOW_REPO_QUERIES
Internally DB Managerwill connect asDB__ADMIN user tofetch this information butthe UI and DB ManagerServer side call willenforce that the loginuser has theDBMGR.SHOW_REPO_QUERIES componentprivilege
Workloads Manage WMSprofiles/slas/mappings
WMS.MANAGE_WMS
Workloads Cancel Query SQL_OPERATIONS.QUERY_CANCEL
Database View EsgynDBSchemas and SchemaObjects
SQL_OPERATIONS.SHOW
Internally DB Managerwill connect asDB__ADMIN user tofetch this information butthe UI and DB ManagerServer side call willenforce that the loginuser has theSQL_OPERATIONS.SHOW component privilege
EsgynDB Manager User Guide
Chapter 10. Security | 71
DB Manager Page Feature Required Privilege Comments
Event Logs View Event Logs DBMGR.SHOW_EVENT_LOGS
Internally DB Managerwill connect asDB__ADMIN to fetchthis information but theUI and DB ManagerServer side call willenforce that the loginuser has theDBMGR.SHOW_EVENT_LOGS componentprivilege
Connectivity View sessions and dcspstack
DBMGR.MANAGE_SESSIONS
Database Create/Alter/Drop/Download Library
SQL_OPERATIONS.MANAGE orSQL_OPERATIONS.MANAGE_LIBRARY
Security Role management SQL_OPERATIONS.MANAGE orSQL_OPERATIONS.MANAGE_ROLES
Security User management SQL_OPERATIONS.MANAGE orSQL_OPERATIONS.MANAGE_USERS
Security User Groupmanagement
SQL_OPERATIONS.MANAGE orSQL_OPERATIONS.CREATE
Security Grant/Revoke SQLPrivileges
SQL_OPERATIONS.SHOW
Security Grant/Revoke Roles toUsers
SQL_OPERATIONS.MANAGE orSQL_OPERATIONS.MANAGE_ROLES
Security Grant/Revokecomponent Privileges
SQL_OPERATIONS.MANAGE orSQL_OPERATIONS.MANAGE_COMPONENTS
Tenants Tenant management SQL_OPERATIONS.MANAGE orSQL_OPERATIONS.MANAGE_TENANTS
EsgynDB Manager User Guide
72 | Chapter 10. Security
DB Manager Page Feature Required Privilege Comments
Tenants Resource Groupmanagement
SQL_OPERATIONS.MANAGE orSQL_OPERATIONS.MANAGE_TENANTS orSQL_OPERATIONS.MANAGE_RESOURCE_GROUPS
Whiteboard Run Adhoc SQL queries DBMGR.USE_QUERY_WORKBENCH
Tools Create Library SQL_OPERATIONS.MANAGE orSQL_OPERATIONS.MANAGE_LIBRARY
Tools SQL Converter NONE Convert SQL queries toEsgynDB
Tools SQL Script Execute DBMGR.EXECUTE_SQL_SCRIPT
Execute mulit-line SQLscripts
Alerts View and update Alerts DBMGR.MANAGE_ALERTS
EsgynDB Manager User Guide
Chapter 10. Security | 73
Chapter 11. Tenants
Multi-tenancy allows multiple tenants (or clients) on a single cluster to run their applications and
manage their compute resources independently of other tenants.
This feature is only available if you are running EsgynDB with multi-tenancy enabled license.
Manage system resources
Provides a mechanism that controls CPU and memory usage among tenants. Each tenant gets
part of the available CPU and memory allocated to EsgynDB on a cluster. Managing resources
is enforced by utilizing Linux cgroups.
Isolate data
Each tenant is able to access data for their application but not able to view or access data
managed by other tenants. Isolation is enforced by granting privileges on objects associated to
users and user groups associated with tenants.
11.1. Resource Groups
Resource Groups is a logical grouping of hosts/nodes. You typically group hosts based on their
hardware configuration and compute capacity. But you can also group the nodes based on your
business application needs to allocate some nodes to application A and some nodes to application B
and so on.
When you create tenants you can choose specific resource groups and the tenant is allocated
compute resources on the hosts mapped to the resource groups.
To view a list of configured resource groups select the Tenants -> Resource Groups.
11.1.1. Add Resource Group
In the Resource Groups summary page, Click on the Add Resource Group button.
In the Add Resource Group dialog, provide a name for the resource group. Select the hosts/nodes
you want to be part of the resource group and click Apply button.
EsgynDB Manager User Guide
74 | Chapter 11. Tenants
11.1.2. Edit Resource Group
In the Resource Groups summary page click on a specific Resource Group.
In the Attributes page, click on the Edit button. You can change the node assignments for the
resource group.
If you edit a resource group that has already tenants mapped, it would internally trigger a rebalance of
the compute unit allocation for the associated tenants.
You cannot edit the system created resource group DB__RGROUP_DEFAULT
11.1.3. Delete Resource Group
In the Resource Groups summary page click the delete button for a specific resource group.
You can also delete a resource group from the Attributes page, using the Delete button.
You cannot delete the system created resource group DB__RGROUP_DEFAULT
11.2. Tenants
• Tenant is a logic grouping of a specific business or application or a group of users.
• EsgynDB service as a whole is allocated compute resources (CPU and Memory). We leverage
Linux CGroups and a CGroup for EsgynDB is created with cpu and memory allocations.
• When a tenant is created, it is allocated a subset of EsgynDB’s compute resources (CPU and
Memory). We create a tenant cgroup as a child of the EsgynDB cgroup.
• Resource limits is enforced by Linux CGroups
• A tenant is also assigned database resources such as a tenant administrator role and assigned
one or more schemas.
• Core EsgynDB infrastructure like monitor, RMS, Transaction Manager, DB Manager, DCS Master,
DCS Server is shared by all tenants. These processes run in the EsgynDB cgroup.
• Application connections for specific tenants run under the Tenant cgroup. This includes the master
executor and ESPs.
EsgynDB Manager User Guide
Chapter 11. Tenants | 75
11.2.1. Key Terminologies
Tenant Name
A descriptive name for the tenant. The tenant will be registered in EsgynDB metadata with this
name. When applications connect to EsgynDB, they have to provide this tenant name in the
connection url.
CPU Shares
contains an integer value that specifies a relative share of CPU time available to the tasks in a
cgroup
cpuSharesPerCore
We assume a total shares of 1024 per node and so this value determines the number of cpu
shares per core. So on a 32 core node, the number of cpu shares per core is 1024/32 = 32.
computeUnit
This is an Esgyn term indicating how many compute resource units to allocate to a tenant. We
set 1 computeUnit as 4 cores and 32 GB Memory. Esgyn recommends at least 4
computeUnits per tenant.
coresPerUnit
Number of cores to allocate for a compute unit. Esgyn uses 4 cores per compute unit.
memoryPerUnit
The amount of memory to allocated for a compute unit. Esgyn uses 32GB per compute unit.
11.2.2. Configuration
You can access this page using the Tenants -> Configuration menu.
In the tenants configuration tab, you can see all registered tenants. By default, there will be a
EsgynDB system tenant. This is configured at install time and maps to the EsgynDB parent cgroup.
You can see the tenant summary includes the following information.
Tenant Name
The tenant Name
EsgynDB Manager User Guide
76 | Chapter 11. Tenants
Tenant ID
An internal unique ID for the tenant. If you specified a default schema when registering the
tenant, the schemas is created in a special HBase namespace of the form TRAF_<tenant id>.
Thus any tables created in the tenant schemas are isolated within their own namespace.
Admin Role
The admin role that can manage this tenant
User Groups
The LDAP user groups mapped to this tenant
Resource Groups
The resource groups mapped to this tenant
Compute Units
The number of compute units allocated to the tenants
Max Sessions
The session concurrency limit to be enforced for the tenants
Default Schema
The default EsgynDB schema for the tenant
Created By
The database user who created the tenant
Create Time
The timestamp when the tenant was created
Modified Time
The timestamp when the tenant was modified
EsgynDB Manager User Guide
Chapter 11. Tenants | 77
Figure 11. Tenants
11.2.3. Node Allocation Graph
The node allocation graph shows how the compute units for each tenant has been allocated across
the different nodes. The compute units are color coded by tenant name as shown in the graph legend.
If there are units available yet to be allocated, they are shown in a different color and shown as
Available.
If you hover your mouse over a cube in the graph, you’ll see a tooltip that shows the node name and
the tenant allocated that unit.
11.2.4. Usage
The Usage tab displays the CGroup thresholds for each tenant and the current utilization.
The grid displays the total CPU and memory thresholds configured across all nodes where the
tenant’s compute units have been placed and the current actual usage of memory and cpu time.
When you click on a specific tenant name, you get the Tenant Resource details. It shows the
configured thresholds and current actuals on a per node basis.
EsgynDB Manager User Guide
78 | Chapter 11. Tenants
11.2.5. Add Tenant
In the Tenants Configuration page, click on the Add Tenant button to create a new tenant.
Figure 12. Add Tenant
In the Add dialog:
• Specify a tenant name
• Specify an administrative role name to manage the tenant.
• Select the number of compute units to allocate to the tenant.
The minimum number of compute units you can specify is 4.
If you do not select any resource groups, then the number of compute units has to be a factor of
the number of nodes where EsgynDB Service is configured.
If you specify one or more resource groups then the compute unit does not need to be a factor of
the number of EsgynDB nodes.
• Optionally, select the user groups to be mapped to this tenant.
If you select user groups and the TRAF_TENANT_GROUP_CHECK CQD is ON, then users
belonging to the selected user groups ONLY will be allowed to connect as this tenant.
EsgynDB Manager User Guide
Chapter 11. Tenants | 79
• Optionally, select the resource groups to be mapped to this tenant.
If you select resource groups, the compute units for the tenant will be allocated ONLY on the hosts
belonging to these resource groups.
• If you want to control the session concurrency, you can set the maximum number of concurrent
sessions allowed for this tenant.
• And optionally you can specify a default EsgynDB schema name for the tenant. The schema will
be created in the TRAFODION catalog, if it does not already exist.
• Click Apply
The tenant is registered in Esgyn metadata and a tenant cgroup is created on the Esgyn nodes. The
tenant cgroup is created as child cgroup under a ESGYNDB parent cgroup. The parent cgroup is
indicated by the ESGYN_CG_CPU, ESGYN_CG_CPUACCT and ESGYN_CG_MEM environment
variables.
The code computes the cgroup thresholds as a % value of the EsgynDB threshold on the node and
uses the % value to create the cgroup.
The example below shows how the compute units translates into cgroup values and how it reflects
against the overall EsgynDB cgroup limits.
Example
• A cluster has 4 nodes
• Each node in cluster has 32 cores/256 GB memory
cpuSharesPerCore = 1024/32 = 32
coresPerUnit = 4
memoryPerUnit = 32GB
• EsgynDB is allocated 50% of the node resources
esgynCPUShares = 512 shares
esgynMemory = 128GB
• Computing Tenant CGroup limits
EsgynDB Manager User Guide
80 | Chapter 11. Tenants
• First compute the tenant cgroup limits as a % value of the EsgynDB limits. This
computation is done on one node.
tenantCPUPercent = (number of slices on node * coresPerUnit * cpuSharePerCore *
100) / esgynCPUShares
tenantMemoryPercent = (number of slices on node * memoryPerUnit * 100) /
esgynMemory
• Next create the tenant cgroup on each node as % of the EsgynDB cgroup values
tenantCPUShares = (tenantCPUPercent/100) * esgynCPUShares
tenantMemory = (tenantMemoryPercent/100) * esgynMemory
• Tenant 1 needs 4 compute units
tenantCPUPercent = (1 * 4 * 32 * 100) / 512 = 25%
tenantMemoryPercent = (1 * 32GB * 100) / 128 = 25%
tenantCPUShares = (25/100) * 512 = 128 shares
tenantMemory = (25/100) * 128GB = 32 GB
The Tenant1 cgroup is created as 128 shares and 32GB memory on each node. The Swap
is always set as twice that of the memory.
• Tenant 2 needs 8 compute units
tenantCPUPercent = (2 * 4 * 32 * 100) / 512 = 50%
tenantMemoryPercent = (2 * 32GB * 100) / 128 = 50%
tenantCPUShares = (50/100) * 512 = 256 shares
tenantMemory = (50/100) * 128GB = 64 GB
The Tenant2 cgroup is created as 256 shares and 64GB memory on each node. The Swap
is always set as twice that of the memory.
EsgynDB Manager User Guide
Chapter 11. Tenants | 81
11.2.6. Alter Tenant
Click on a tenant in the tenant configuration page and you will see the tenant details.
Click on the edit button to change the tenant configuration. You can change the tenant compute units,
user groups, resource groups, default schema or the max sessions for the tenant.
11.2.7. Delete Tenant
In the tenants configuration page, you can click the tenant rows delete button to delete the specific
tenant. You can also delete a tenant from the tenant details page.
When you delete a tenant, you have the option of just deleting the tenant metadata but retaining the
schemas created for the tenant, the admin role and the privileges granted to the role. In the Delete
Tenant dialog, leave the checkbox unselected to retain the schema and admin roles. But if you want to
remove all information about the tenant, select the checkbox.
The tenant cgroup is deleted from the Esgyn nodes when the tenant is deleted.
11.2.8. Granting Privileges to Tenant
You can create tenant specific roles and grant the required privileges to these roles. Then grant these
roles to the user groups mapped to the tenant. Any user connecting as the tenant will inherit the
privileges.
Optionally you can directly grant privileges or the roles to the individual users.
11.2.9. Connecting as Tenant
You need to specify the tenant name in the connection attributes.
JDBC Applications
• You can specify the tenantName property in the connection string
jdbc:t4jdbc://my.esgyndb.server:23400/:tenantName=TENANT1
• If you are using connection properties file, set the tenantName property in the properties file.
EsgynDB Manager User Guide
82 | Chapter 11. Tenants
ODBC Applications
• If you are using Trafodion Windows ODBC Driver and using a DSN, set the Tenant name in the
DSN Configuration
• If you are using EsgynDB Linux odbc driver, set the TenantName property in the DSN
configuration in odbc.ini
• If you are not using DSN, you can set the TenantName property in the odbc connection string.
11.2.10. Monitoring Tenant Workloads
• In the Connectivity page, where you see the list of master executors, you can see the active
sessions that are in CONNECTED state. You can see the user name and the tenant name for
those sessions.
• If you want to view the current active queries being run by the tenant, use the Active Workloads
page. It displays the query id and the tenant name for each active query.
• You can view the historical queries executed by the tenants in the past using the Historical
Workloads page. Use the tenant name column level filter to search specific tenant queries.
EsgynDB Manager User Guide
Chapter 11. Tenants | 83
Chapter 12. Query Workbench
The query workbench page allows you to execute ad-hoc queries and view the results in a tabular
form or to generate a visual and textual explain plan.
Type the SQL query text in the Query text area.
The query text should be valid EsgynDB query syntax.
You can ONLY execute a single EsgynDB SQL statement. It can be a DDL or DML
statement. Compound SQL statements are not supported.
The SQL statement should conform to EsgynDB SQL syntax and any errors
reported by the engine are displayed with full error text.
Figure 13. Query Workbench
EsgynDB Manager User Guide
84 | Chapter 12. Query Workbench
Figure 14. Query Workbench Buttons
12.1. Execute Query
Use the Execute button to execute the query. Once the query starts executing, the Execute button
turns into Cancel button.
The Elapsed Time counter starts ticking as soon the query is submitted for execution and helps to
know how long the query is executing or how long the query took to complete.
If for some reason, you need to cancel the query once the execution has started, click on the Cancel
button.
If you move away from the Query Workbench page and navigate to a different page, before the
execute or cancel operation is complete then the success/fail status of the execute/cancel is shown
via the notifications panel. See Notifications Panel for details. Once you click on the result for the
query workbench notification, you’ll be automatically navigated back to the Query Workbench page
and the results will be displayed.
For a query that returns a result set, the results are displayed in a tabular format. For non-select
queries or queries that returns a scalar result, the results are displayed as normal text.
12.1.1. Max Rows in result
By default the workbench limits the result output rows to 1000.
But you can use the Max Rows counter to specify a different value. You can specify a positive
number that is less than or equal to the value set for the qwbMaxRows property in the DB Manager
server config.xml.
Refer to the Configuration Properties section of this user guide for more details.
EsgynDB Manager User Guide
Chapter 12. Query Workbench | 85
12.2. Explain Plan
You can use the Explain button to generate an explain plan for the query. You get a visual explain
rendered as a tree graph and also a textual plan.
The Elapsed Time counter starts ticking as soon the query is submitted for explain and helps to know
how long it takes to generate the explain.
If you move away from the Query Workbench page and navigated to a different page, before the
explain operation is complete then the success/fail status of the explain is shown via the notifications
panel. See Notifications Panel for details. Once you click on the result for the query workbench
notification, you’ll be automatically navigated back to the Query Workbench page and the results will
be displayed.
You can right mouse click on the graph and move the graph around or use your mouse scroll key to
zoom-in and zoom-out the graph.
If you double-click on a specific node in the tree, you can see the details and cost for that specific
operator in a pop-up window.
12.3. Control Options
Use the Options button to set control options like SET statements or Control Query statements that
can influence the query plan and execution.
You can use these options to tune your query performance. Refer to the SQL Reference manual for
more details on the SET and Control query commands.
For example you can set the default schema if the query is not using fully qualified table names.
The options once set are applied before every query execution or explain. You can use the clear
button to clear the option settings.
12.4. Export Workbench Data
You can use the Save button to save the query text, the execution output, the explain output and any
control options in a JSON format to an external file on your workstation. The file is saved with a .wbj
(Workbench JSON) extension.
EsgynDB Manager User Guide
86 | Chapter 12. Query Workbench
This feature is very handy for both developers who are collaborating with other developers and even
in production support where application users can use this export feature to quickly capture the query
profile for queries that either have bad performance or other defects, and email the .wbj file to Esgyn
Support.
12.5. Import Workbench Data
You can use the Open button to import a previously exported .wbj (Workbench JSON) file or any
other script file that has SQL statements.
By default the File Open dialog only displays the .wbj files. You can use the All Files option to display
all other file types to open .txt or .sql or .ddl files that only contain SQL statements.
Based on what was saved in the imported file, the query text, the query execution results, explain
graph and control options are loaded and rendered into Workbench.
12.6. Clear Workbench
The Clear button lets you clear and completely reset the Workbench. The query text, execution
results, explain results and control options are all cleared.
EsgynDB Manager User Guide
Chapter 12. Query Workbench | 87
Chapter 13. Alerts
DB Manager leverages Bosun open source framework to monitor key system or workload metrics and
generate email or http alerts based on pre-defined rule templates. Alerts feature is only available in
the EsgynDB Advanced Edition.
You can configure rule templates and notification templates in Bosun. When the rules evaluate to true,
the alerts are triggerd. You configure your Bosun alert definitions in the
$MGBLTY_INSTALL_DIR/bosun/conf/bosun.conf file.
Refer to the Bosun documentation at https://bosun.org/documentation.html for instructions on how to
define the alert rule and notification templates.
Additionally, refer to the OpenTSDB documentation and the TCollector documentation to understand
the type of metrics that are available and how to write the TSDB query expression that will be used in
your Bosun alert template.
http://opentsdb.net/docs/build/html/index.html
http://opentsdb.net/docs/build/html/user_guide/utilities/tcollector.html
13.1. Alerts Summary
Using the Alerts page in DB Manager, you can view all open alerts.
The Alerts summary displays all the open alerts. You can narrow down the filter predicates using the
Filter button and find alerts with specific alert status and/or severity and/or alert text.
The Alert summary includes:
Time
The time when the alert notification was generated.
Incident ID
An unique incident ID for the alert
Alert Key
The Alert Key
EsgynDB Manager User Guide
88 | Chapter 13. Alerts
Alert
The subject text of the alert notification.
Status
Status of the alert. Acknowledged or Un-Acknowledged.
Severity
Severity State of the alert. Critical/Warning/Normal/Unknown.
Active
Indicates if the alert is still active, meaning if the alert trigger condition still exists. Only inactive
alerts can be closed.
Silenced
Indicates if the alert has been silenced.
Click on the Alert Key hyperlink to drill-down into the details of a specific alert.
13.2. Alert Details
The alert detail page displays all the details of the alert. This includes the alert summary information
as well as the detail information that is sent out in the notification.
If the alert has been acknowledged already by a user, it shows who acknowledged or closed the alert,
when the alert was updated and the notes entered during the update.
Use the Update button to update the alert. You can acknowledge new alerts and enter a comment.
For alerts that are no longer active, meaning the alert condition is no longer true, you can close the
alerts.
Refer to the Bosun documentation at https://bosun.org/documentation.html for more details.
EsgynDB Manager User Guide
Chapter 13. Alerts | 89
Chapter 14. Backup/Restore
EsgynDB supports online backup and restore of EsgynDB schemas and tables. This feature is only
enabled when using an EsgynDB Advanced Edition license.
You can access the Backup/Restore feature using the Backup/Restore menu.
14.1. Backup/Restore Summary
The Backup/Restore page displays all existing backups, if any.
Figure 15. Backups
The backup summary includes the following information:
Tag Name
The backup tag name. It is made up of the user entered backup tag name followed by a julian
timestamp that is appended at the time of the actual backup operation.
For example if you schedule a monthly backup for a tag named "sales_monthly", each monthly
backup instance will be named sales_monthly_<timestamp>.
Backup Time
The timestamp when the backup operation started
EsgynDB Manager User Guide
90 | Chapter 14. Backup/Restore
Status
The status of the backup.
Backup Type
Indicates if the backup is a regular or incremental backup.
Use the Refresh button to refresh the backup list.
14.2. New Backup
In the backup summary page, use the New Backup button to create or schedule a new backup.
Figure 16. Create Backup
• Enter a backup tag name or select a tag from the drop-down list. This list shows the current list of
user specified tag names.
• Select the backup type, regular or incremental.
• If you want to override an existing backup with the same tag name, check the Override option.
• If override option is checked, then a backup is created with the specified tag name and it
overrides existing backup if any, with the same tag name.
• If override option is unchecked, then a new backup is created with a composite tag name. The
composite tag name consists of the specified backup tag name as prefix and appended with a
unique timestamp.
EsgynDB Manager User Guide
Chapter 14. Backup/Restore | 91
• Browse the object tree and select one or more schemas and/or tables.
• If you select the root tree node, then all user schemas and tables will be backed up.
• You can also select individual schemas and tables by traversing the object tree and selecting
them.
• If all tables in a schema are selected, then the entire schema is backed up.
• The selected objects are displayed in the right pane, so you know which exact objects will get
backed up.
• Specify the time when the backup should be executed.
• Select the Run once now option to execute the backup operation immediately when the user
clicks Submit.
• Select the Run once at selected time option to schedule the backup operation at a later date
and time.
When you select this option, you’ll see a date time selection control that will allow you to
specify the exact date and time when the backup should be executed.
• Select the Recurring option if you want to perform recurring backups of the selected objects.
For example you want to take daily or weekly backups of a specific user schema.
To use the schedule feature, a python module python-crontab has to be
installed on the nodes where DB Manager is running. This is a pre-requisite.
sudo pip install python-crontab
• When you select this option, you’ll see a calendar control. Click on the calendar control to
set the desired recurring schedule.
• You can select a daily or weekly or monthly or yearly schedule and also specify at what
times the backup operation should be executed.
In the example below, you’ll see how to schedule a weekly backup at a defined time.
• The backup schedules are added to the EsgynDB service user’s (trafodion) crontab file and
cron would execute the operations at the scheduled time.
EsgynDB Manager User Guide
92 | Chapter 14. Backup/Restore
Figure 17. Schedule Recurring Backup
• Click on Submit button to submit the backup operation as scheduled.
• If you selected Run once now the backup dialog will remain open until the backup completes. You
can choose to close the dialog and not wait for the backup to complete.
14.2.1. View Scheduled Backup jobs
In the backup summary page, click on the Scheduled Jobs tab.
Figure 18. Backup Schedules
The scheduled job details include the following:
Job ID
An unique identifier to identify the specific backup schedule. This is internally generated when
the backup job is scheduled.
EsgynDB Manager User Guide
Chapter 14. Backup/Restore | 93
Schedule
A human readable description of the recurring job schedule
Command
The job command that will be executed at the scheduled time. The backup operations are
initiated through the edb_br_action.py script.
Actions
Click on the Modify button to modify the schedule of the selected job
14.2.2. Modify Schedule of Backup jobs
Click on the Modify button for a selected job to change its execution schedule. This will display the
backup scheduler dialog where you can set your new recurring schedule.
14.2.3. Delete Scheduled Backup jobs
You can delete one or more scheduled jobs by selecting them in the grid and clicking on the Delete
button. The selected jobs will be removed from the cron schedule. Note this will only remove it from
the cron schedule and will not impact any backup operations that may already be in progress.
14.3. Restore Backup
To restore a backup, select the backup tag in the backup summary page and click on the Restore
button. This will open up a pop-up dialog box.
• All the objects in the selected backup will be fetched and listed in a tree structure. You can pick
and choose which objects you want restored.
• You can select the entire backup by selecting the root tree node or traverse the object tree and
select individual schemas or tables.
• The selected objects are displayed in the right pane, so you know which exact objects will get
restored.
• Click on Reset button to reset your selection if needed.
• Click on Submit to restore the selected objects. The restore is executed immediately after the
submit.
EsgynDB Manager User Guide
94 | Chapter 14. Backup/Restore
The restore dialog will remain open until the restore operation completes. You can choose to close
the dialog and not wait for the restore to complete.
14.4. Export Backup
You can make a copy of your backup to an external HDFS directory.
To export a backup to an external HDFS directory, select the backup tag in the backup summary page
and click on the Export button.
In the pop-up dialog, specify the fully qualified external HDFS destination directory in url form and
click on Submit.
Example: hdfs://node1.abc.com:8020/user/trafodion/backups
14.5. Import Backup
To import a backup from an external HDFS directory, click on the Import button.
• In the pop-up dialog, specify the backup tag name that should be imported.
• Specify the fully qualified external HDFS source directory from where the backup should be
imported.
Example: hdfs://node1.abc.com:8020/user/trafodion/backups
• Click Submit to initiate the import.
14.6. Delete Backup
From the backup summary page, You can delete one or more backups that are no longer needed.
• Sort the backup list by sorting on the Backup Time in descending order.
• Select one or more backup tag names in the grid and then click on the Delete button.
• Select Yes in the confirmation dialog.
• The deletes will be scheduled to run.
EsgynDB Manager User Guide
Chapter 14. Backup/Restore | 95
14.7. Backup/Restore Logs
If the backup/restore operations report a failure, you can analyze the Esgyn logs to diagnose the
failures.
Go to Logs page and use the filter to select BACKUP_RESTORE component logs.
14.8. Additional Reference
For additional details on backup/restore feature, refer to the EsgynDB Backup Restore Guide.
EsgynDB Manager User Guide
96 | Chapter 14. Backup/Restore
Chapter 15. Tools
The Tools navigation menu provides a list of generic tools to manage or make changes to EsgynDB.
15.1. Create Library Tool
Creating a SQL library is a two-step process. You first deploy the code file to the EsgynDB platform
and then create a library. The Create Library tool allows you to perform these two steps with a single
click.
The create library tool is made easily accessible through the Tools navigation menu for convenience,
without having to navigate through the database library pages.
See Create Library for details.
15.2. SQL Converter Tool
The SQL Converter tool allows to convert DDL or DML statements from another database dialect to
EsgynDB.
In this dbmgr-2.5.0 release, we support converting from Teradata dialect to EsgynDB.
If you want to migrate an entire Teradata schema including tables and views, you can extract the DDL
to a text file and use the SQL Converter tool to convert the entire DDL to EsgynDB syntax with just a
few clicks.
EsgynDB Manager User Guide
Chapter 15. Tools | 97
Figure 19. SQL Converter
15.2.1. Converting SQL statements
• Select the source database dialect. In this release, the only supported source dialect is Teradata.
• Use the browse button to search and select a input SQL file on your workstation. The input SQL
file contains all the Teradata statements that need converting.
• Click on the Convert button.
• You should see the SQL Statements converted to EsgynDB syntax and displayed in the right
panel.
• If needed, you can now hand edit the converted SQL and add additional EsgynDB constructs like
adding salt partitioning or GRANT privileges statements etc.
• Use the Save button if you want to save a copy of the converted SQL text to a local file on your
workstation.
• Use the Execute button to immediately execute the SQL statements on the EsgynDB server.
15.3. Execute SQL Script Tool
The Execute SQL Script tool allows you to execute a multiple SQL statements in a single connection
to the EsgynDB engine.
EsgynDB Manager User Guide
98 | Chapter 15. Tools
For example, you can create schema, create tables, load data into the tables, run several reporting
queries on the tables all within a single script and run this through the tool.
When you execute the converted SQL text in the SQL Converter tool, the execution is done through
this SQL Script tool.
Figure 20. SQL Script Executer
15.3.1. Executing SQL script
• Use the browse button to search and select a SQL script file on your workstation.
• Click on the Execute button to execute the SQL statements contained in the script, on the
EsgynDB server.
• If you need to abort the script execution for any reason, you can click the Cancel button.
• Once the script execution completes, the results are displayed in the results text area.
• Use the Save button to save the results to a file on your local workstation.
• The Elapsed Time counter starts ticking as soon the script is submitted for execution and helps to
know how long the script has been executing.
EsgynDB Manager User Guide
Chapter 15. Tools | 99
Chapter 16. Troubleshooting and Support
16.1. Troubleshooting
Use these common tips to troubleshoot any display issues with EsgynDB Manager.
16.1.1. Cannot login to DB Manager
• Check the browser URL for accuracy. Make sure you have the right host name and port number.
• Verify that the DB Manager ports (4205/4206) are not blocked by any firewall.
• Check if DB Manager is running. Use the mgblty_check or dbmgr.sh status to confirm EsgynDB
Manager is running.
• If DB Manager is up, verify that EsgynDB, DCS and REST is up and running.
• trafcheck
• dcscheck
• restcheck
16.1.2. Page is blank or partially loaded
• It could be a transient error like browser load timeouts. Log out and log back in.
• Sometimes refreshing the browser cache also helps.
• Check the browser console for errors.
16.1.3. The Datetime values do not look right
• DB Manager displays all datetime values in the server’s local time zone
• Make sure the timeZoneName property in the DB Manager config.xml matches the EsgynDB
server time zone
• Make sure the timezoneName is specified the correct canonical form like America/Los_Angeles or
Asia/Shanghai or Etc/UTC etc.
Note: You will need to restart DB Manager anytime you change property values in config.xml
EsgynDB Manager User Guide
100 | Chapter 16. Troubleshooting and Support
16.1.4. Scheduling backup jobs fails with error ImportError: No module namedcrontab
• Scheduling feature requires a pre-requisite python module python-crontab. This module needs to
be installed on the nodes where DB Manager is configured to run.
sudo pip install python-crontab
16.2. Support
When contacting Esgyn Support, the following information will be handy and requested:
• Latest logs in $TRAF_HOME/dbmgr-2.5.0/logs/ directory
• The configuration file $TRAF_HOME/dbmgr-2.5.0/conf/config.xml
• EsgynDB Manager version information from the About page.
For UI issues:
• A screen shot of the error message in the screen or in the browser console.
• If error is reproducible, a description of the sequence of actions the user performed to reproduce
the error.
EsgynDB Manager User Guide
Chapter 16. Troubleshooting and Support | 101