Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and...

40
Highly Efficient and Performant Database Version 1.0 June 2016 Prepared by Microsoft Services SQL 2016 Highly Efficient and Performant Database

Transcript of Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and...

Page 1: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

Version 1.0June 2016

Prepared by

Microsoft Services

SQL 2016 Highly Efficient and Performant Database

Page 2: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

SQL Server 2016 Highly Efficient and Performant Database

MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS DOCUMENT. Complying with all applicable copyright laws is the responsibility of the user.  Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation. Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document.  Except as expressly provided in any written license agreement from Microsoft, our provision of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. The descriptions of other companies’ products in this document, if any, are provided only as a convenience to you.  Any such references should not be considered an endorsement or support by Microsoft.  Microsoft cannot guarantee their accuracy, and the products may change over time. Also, the descriptions are intended as brief highlights to aid understanding, rather than as thorough coverage. For authoritative descriptions of these products, please consult their respective manufacturers.© 2016 Microsoft Corporation. All rights reserved. Any use or distribution of these materials without express authorization of Microsoft Corp. is strictly prohibited.Microsoft and Windows are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries.The names of actual companies and products mentioned herein may be the trademarks of their respective owners.

iiHighly Efficient and Performant DatabaseVersion 1.0 last modified on 14 Jun. 16

Page 3: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

Table of Contents

1 Overview....................................................................................................41.1 DBA Considerations...................................................................................................41.2 Highly Efficient and Performant Database Features..................................................5

1.2.1 Always On Availability Groups.......................................................................51.2.2 In Memory Column Store Indexes..................................................................5

2 Example Scenario......................................................................................62.1 Developing the Fabrikam Investments Deployment..................................................62.2 Implementation Steps...............................................................................................7

2.2.1 Fabrikam Database Table Relations..............................................................72.2.2 Adding the Fabrikam Investment Database to the Availability Group......82.2.3 Configure the Read Only Routing for the Availability Group....................212.2.5 Creating the Sample Investment Tables.....................................................232.2.7 Populating the Investment Tables with sample data................................262.2.8 Example Insert Statements from embedded SQL script...........................26

2.3 Testing the new Fabrikam Investments Database...................................................282.3.1 Execute the Read Only Queries against the Listener (load balanced)....28

4 Execute the Reporting Queries................................................................304.1 Execute the Process Open Orders Script against the Primary SQL Server...............33

5 References...............................................................................................375.1 Enhanced Always On...............................................................................................375.2 In Memory Column Store Indexes............................................................................375.3 Query Store.............................................................................................................37

73

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 4: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

1 OverviewTypical SQL Server relational database implementations that need to support mission critical workloads need to implement high availability (HA) and disaster recovery (DR) strategies with demanding high performance and low latency requirements. Additionally, there is an increasing need for real-time insights into the transactional data to support quick business decisions. However, the reporting and dashboarding workloads end up causing performance problems on the OLTP systems. To support both types of workloads, the database implementation extracts data out of the OLTP system and loads it into separate databases that supported operational reporting in real-time on the incoming transactional data. This increases complexity, costs and results in delays in the availability of data in the reporting and dashboards.SQL Server 2016 supports a number of capabilities that enable real-time reporting and dashboarding on a high performance, low latency, HADR OLTP database. With these capabilities hosting service providers can offer a high performance database to their customers. This implementation leverages the following capabilities:

An In-Memory OLTP with AlwaysOn Availability Groups o Synchronous Replica in the primary datacenter for High Availability

and Asynchronous Replicas in the secondary datacenter for disaster recovery

o Read only load balancing to the read only secondary asynchronous replica databases

o Rebalancing of load during failure conditions Query Datastore for performance tuning and troubleshooting ColumnStore Index on the asynchronous replicas to support high

performance querying for reports and dashboards

1.1 DBA ConsiderationsFor a Highly Efficient and Performant Database scenario, it is important to have an in depth understanding of:

Customer’s HADR strategy Performance tuning and optimization of queries of the relational structure of

the database tables and how the data is being accessed for both the OLTP and the real time report and dashboard workloads

1.2 Highly Efficient and Performant Database FeaturesPage 4

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 5: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

Once the access requirements have been finalized, the next step is to work with a Database Administrator to design and implement the final database deployment. The scenario detailed in this document leverages three key features of SQL Server 2016 which are listed below.

1.2.1Always On Availability GroupsAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize availability for one or more user databases. AlwaysOn Availability Groups requires that the SQL Server instances reside on Windows Server Failover Clustering (WSFC) nodes.For more information, see AlwaysOn Availability Groups (SQL Server).

1.2.2In Memory Column Store IndexesThe columnstore index is the standard for storing and querying large data warehousing fact tables. It uses column-based data storage and query processing to achieve up to 10x query performance gains in your data warehouse over traditional row-oriented storage, and up to 10x data compression over the uncompressed data size. Beginning with SQL Server 2016, columnstore indexes enable operational analytics, the ability to run performant real-time analytics on a transactional workload.For more information, see Columnstore Indexes Guide.

75

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 6: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

2 Example ScenarioFabrikam Investments provide security portfolio management services for millions of customers in the US.

High performance - low-latency trading application and OLTP is hosted by HSP.

They have a need to develop real-time dashboards and reports harvesting data during the trading day showing:

o Most Active Stocks.o Top and Bottom Movers

They have already implemented a HADR solution with both synchronous and asynchronous replicas in their datacenters.

HSP analyzes the workload and implements the reporting and dashboarding needs using the highly efficient and performant database offer.

2.1 Developing the Fabrikam Investments DeploymentThe HSP analyzed the workload using the Query Store and determined that the database and tables will need to be reconstructed and the existing data migrated from the existing database.

The new database will be created to include a memory optimized file group and container.

The new database tables will be constructed with non-clustered in memory column store indexes.

For more information see: Monitoring Performance By Using the Query Store

2.2

76

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 7: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

2.2 Implementation Steps

2.2.1Fabrikam Database Table RelationsThe following diagram provides the foreign key relations in the Fabrikam Investments database tables.

1 Creating the Customer DatabaseUSE [master]GO

CREATE DATABASE [FabrikamInvestmentsDb] CONTAINMENT = NONE ON PRIMARY ( NAME = N'FabrikamInvestmentsDb', FILENAME = N'C:\Program Files\Microsoft SQL Server\MSSQL13.MSSQLSERVER\MSSQL\DATA\FabrikamInvestmentsDb.mdf' , SIZE = 8192KB , MAXSIZE = UNLIMITED, FILEGROWTH = 65536KB ) LOG ON ( NAME = N'FabrikamInvestmentsDb_log', FILENAME = N'C:\Program Files\Microsoft SQL Server\MSSQL13.MSSQLSERVER\MSSQL\DATA\FabrikamInvestmentsDb_log.ldf' , SIZE = 8192KB , MAXSIZE = 2048GB , FILEGROWTH = 65536KB )GO

-- Create the memory optimized Filegroup and ContainerALTER DATABASE FabrikamInvestmentsDb ADD FILEGROUP FabrikamInvestmentsDb_mod CONTAINS MEMORY_OPTIMIZED_DATA

77

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 8: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

GO

ALTER DATABASE FabrikamInvestmentsDb ADD FILE (name='FabrikamInvestments_mod1', filename='C:\Program Files\Microsoft SQL Server\MSSQL13.MSSQLSERVER\MSSQL\DATA\FabrikamInvestmentsDb_mod1') TO FILEGROUP FabrikamInvestmentsDb_modGO

ALTER DATABASE FabrikamInvestmentsDb SET MEMORY_OPTIMIZED_ELEVATE_TO_SNAPSHOT=ONGO

2.2.2Adding the Fabrikam Investment Database to the Availability Group

1 Make a full backup of the FabrikamInvestmentsDb database

78

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 9: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

2 Click the OK button

3 Click the OK button

79

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 10: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

4 Add the database to the appropriate availability group

710

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 11: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

5 Click the next button

711

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 12: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

6 Select the FabrikamInvestmentsDb Database

712

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 13: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

7 Select Full Synchronization

713

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 14: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

8 Connect to all the SQL Servers

714

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 15: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

9 Connect using Windows Authentication

715

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 16: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

10 Click the Next button

716

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 17: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

11 Click the Next button

717

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 18: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

12 Click the Finish button

718

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 19: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

13 Wait for the Availability Group updates to complete

719

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 20: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

14 Verify all processing was successful

720

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 21: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

2.2.3Configure the Read Only Routing for the Availability GroupThe following statements will configure the load balancing as follows:

Primary replica will be DATA1-SQL1 Round robin the read only workload across DATA2-SQL1 and DATA2-SQL2 If Site2-DC is down then send the read only workload to DATA1-SQL2 If DATA1-SQL2 becomes the new primary replica, then send the read only

workload to DATA1-SQL1 If a failover occurs to Site2-DC and DATA2-SQL1 becomes the new primary

replica then the read only workload should be sent to DATA1-SQL1 and DATA1-SQL2

If Site1-DC is down then send the read only workload to DATA2-SQL2 If DATA2-SQL2 becomes the new primary replica then send the read only

workload to DATA2-SQL1

1 Create the Failover Load Balanced Routing TablesUSE masterGO

-- Set DATA1-SQL1 to allow read only connections ALTER AVAILABILITY GROUP [TestAG]MODIFY REPLICA ON N'DATA1-SQL1'WITH (SECONDARY_ROLE (ALLOW_CONNECTIONS = READ_ONLY));

-- Set DATA1-SQL1 readonly routing urlALTER AVAILABILITY GROUP [TestAG]MODIFY REPLICA ON N'DATA1-SQL1' WITH (SECONDARY_ROLE (READ_ONLY_ROUTING_URL = N'TCP://Data1.SQL1.Site1.net:5022'));

-- Set DATA1-SQL2 to allow read only connectionsALTER AVAILABILITY GROUP [TestAG]MODIFY REPLICA ON N'DATA1-SQL2'WITH (SECONDARY_ROLE (ALLOW_CONNECTIONS = READ_ONLY));

-- Set DATA1-SQL2 readonly routing urlALTER AVAILABILITY GROUP [TestAG]MODIFY REPLICA ON N'DATA1-SQL2' WITH (SECONDARY_ROLE (READ_ONLY_ROUTING_URL = N'TCP://Data1.SQL2.Site1.net:5022'));

-- Set DATA2-SQL1 to allow read only connectionsALTER AVAILABILITY GROUP [TestAG]MODIFY REPLICA ON N'DATA2-SQL1'WITH (SECONDARY_ROLE (ALLOW_CONNECTIONS = READ_ONLY));

-- Set DATA2-SQL1 readonly routing urlALTER AVAILABILITY GROUP [TestAG]MODIFY REPLICA ON N'DATA2-SQL1'WITH (SECONDARY_ROLE (READ_ONLY_ROUTING_URL = N'TCP://Data2.SQL1.Site1.net:5022'));

721

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 22: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

-- Set DATA2-SQL2 to allow read only connectionsALTER AVAILABILITY GROUP [TestAG]MODIFY REPLICA ON N'DATA2-SQL2' WITH (SECONDARY_ROLE (ALLOW_CONNECTIONS = READ_ONLY));

-- Set DATA2-SQL2 readonly routing urlALTER AVAILABILITY GROUP [TestAG]MODIFY REPLICA ON N'DATA2-SQL2' WITH (SECONDARY_ROLE (READ_ONLY_ROUTING_URL = N'TCP://Data2.SQL2.Site1.net:5022'));

-- Setup the routing --

-- If DATA1-SQL1 is the primary replicaALTER AVAILABILITY GROUP [TestAG]MODIFY REPLICA ON N'DATA1-SQL1' WITH (PRIMARY_ROLE (READ_ONLY_ROUTING_LIST=(('DATA2-SQL1','DATA2-SQL2'),'DATA1-SQL2')));

-- If DATA1-SQL2 is the primary replicaALTER AVAILABILITY GROUP [TestAG]MODIFY REPLICA ON N'DATA1-SQL2'WITH (PRIMARY_ROLE (READ_ONLY_ROUTING_LIST=(('DATA2-SQL1','DATA2-SQL2'),'DATA1-SQL1')));

-- If DATA2-SQL1 is the primary replicaALTER AVAILABILITY GROUP [TestAG]MODIFY REPLICA ON N'DATA2-SQL1'WITH (PRIMARY_ROLE (READ_ONLY_ROUTING_LIST=(('DATA1-SQL1','DATA1-SQL2'),'DATA2-SQL2')));

-- If DATA2-SQL2 is the primary replicaALTER AVAILABILITY GROUP [TestAG]MODIFY REPLICA ON N'DATA2-SQL2' WITH (PRIMARY_ROLE (READ_ONLY_ROUTING_LIST=(('DATA1-SQL1','DATA1-SQL2'),'DATA2-SQL1')));GO

2.2.4

722

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 23: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

2.2.5Creating the Sample Investment TablesUSE [FabrikamInvestmentsDb]GO

------------------------------------------------------------------------------------------- The Order Type table contains the types of orders as an integer (Buy and Sell).-----------------------------------------------------------------------------------------CREATE TABLE [dbo].[OrderType]( [ID] [int] IDENTITY(1,1) NOT NULL PRIMARY KEY NONCLUSTERED HASH WITH (BUCKET_COUNT=1000000), [Name] [varchar](10)) WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);

------------------------------------------------------------------------------------------- The Risk Type table contains risk type of an account (LOW, MEDIUM or HIGH).-----------------------------------------------------------------------------------------CREATE TABLE [dbo].[RiskType]( [ID] [int] IDENTITY(1,1) NOT NULL PRIMARY KEY NONCLUSTERED HASH WITH (BUCKET_COUNT=1000000), [Name] [varchar](10)) WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);

------------------------------------------------------------------------------------------- The Account Manager table contains information relating to an account manager.-----------------------------------------------------------------------------------------CREATE TABLE [dbo].[AccountManager]( [ID] [bigint] IDENTITY(1,1) NOT NULL PRIMARY KEY NONCLUSTERED HASH WITH (BUCKET_COUNT=1000000), [FirstName] [varchar](20) NOT NULL, [LastName] [varchar](20) NOT NULL, [RiskSpecialty] [int] NOT NULL CONSTRAINT FK_AccountManager_RiskType FOREIGN KEY REFERENCES RiskType (ID)) WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);GO

------------------------------------------------------------------------------------------- The Customer Account table contains information relating to a customer.-----------------------------------------------------------------------------------------CREATE TABLE [dbo].[CustomerAccount]( [ID] [bigint] IDENTITY(1,1) NOT NULL PRIMARY KEY NONCLUSTERED HASH WITH (BUCKET_COUNT=1000000), [FirstName] [varchar](20) NOT NULL, [LastName] [varchar](20) NOT NULL, [Email] [varchar](50) NOT NULL, [Address] [varchar](50) NOT NULL,

723

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 24: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

[City] [varchar](20) NOT NULL, [State] [varchar](2) NOT NULL, [Zip] [varchar](10) NOT NULL, [Phone] [varchar](20) NOT NULL) WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);GO

------------------------------------------------------------------------------------------- The Managed Account contains information related to a managed account.-----------------------------------------------------------------------------------------CREATE TABLE [dbo].[ManagedAccount]( [ID] [bigint] IDENTITY(1,1) NOT NULL PRIMARY KEY NONCLUSTERED HASH WITH (BUCKET_COUNT=1000000), [Name] [varchar](10), [RiskLevel] [int] NOT NULL CONSTRAINT FK_ManagedAccount_RiskType FOREIGN KEY REFERENCES RiskType (ID), [CashAvailable] [money] DEFAULT('0.00')) WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);GO

------------------------------------------------------------------------------------------- The Managed Account Open Orders table contains all open orders and the status -- (Completed).-----------------------------------------------------------------------------------------CREATE TABLE [dbo].[ManagedAccountOpenOrders]( [ID] [bigint] IDENTITY(1,1) NOT NULL PRIMARY KEY NONCLUSTERED HASH WITH (BUCKET_COUNT=1000000), [ManagedAccountID] [bigint] NOT NULL CONSTRAINT FK_ManagedAccountOpenOrders_ManagedAccount FOREIGN KEY REFERENCES ManagedAccount (ID), [OrderType] [int] NOT NULL, [CompanySymbol] [varchar](10) NOT NULL, [Shares] [int] NOT NULL, [Price] [money] NOT NULL, [Completed] [bit] DEFAULT(0) NOT NULL, [CreatedDate] [datetime] NOT NULL DEFAULT(getdate())) WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);

------------------------------------------------------------------------------------------- The Managed Account Completed Transactions contain information from the external -- exchange relating to all orders that have been completed on the exchange.-----------------------------------------------------------------------------------------CREATE TABLE [dbo].[ManagedAccountCompletedTransactions]( [ID] [bigint] IDENTITY(1,1) NOT NULL PRIMARY KEY NONCLUSTERED HASH WITH (BUCKET_COUNT=1000000), [OpenOrderID] [bigint] NOT NULL CONSTRAINT FK_ManagedAccountCompletedTransactions_ManagedAccountOpenOrders FOREIGN KEY REFERENCES ManagedAccountOpenOrders (ID), [CompanySymbol] [varchar](10) NOT NULL, [Shares] [int] NOT NULL, [Price] [money] NOT NULL, [CreatedDate] [datetime] NOT NULL DEFAULT(getdate())) WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);

724

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 25: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

------------------------------------------------------------------------------------------- The Customer Account Managed Account contains a relation between the customer account-- and the managed accounts. A customer may have multiple managed accounts.-----------------------------------------------------------------------------------------CREATE TABLE [dbo].[CustomerAccountManagedAccount]( [ID] [bigint] IDENTITY(1,1) NOT NULL PRIMARY KEY NONCLUSTERED HASH WITH (BUCKET_COUNT=1000000), [CustomerAccountID] [bigint] NOT NULL CONSTRAINT FK_CustomerAccountManagedAccount_CustomerAccount FOREIGN KEY REFERENCES CustomerAccount (ID), [ManagedAccountID] [bigint] NOT NULL CONSTRAINT FK_CustomerAccountManagedAccount_ManagedAccount FOREIGN KEY REFERENCES ManagedAccount (ID), [AccountManagerID] [bigint] NOT NULL CONSTRAINT FK_CustomerAccountManagedAccount_AccountManager FOREIGN KEY REFERENCES AccountManager (ID),) WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);GO

------------------------------------------------------------------------------------------- The Managed Account Positions contains all stock positions held in a managed account.-----------------------------------------------------------------------------------------CREATE TABLE [dbo].[ManagedAccountPositions]( [ID] [bigint] IDENTITY(1,1) NOT NULL PRIMARY KEY NONCLUSTERED HASH WITH (BUCKET_COUNT=1000000), [ManagedAccountID] [bigint] NOT NULL CONSTRAINT FK_ManagedAccountPositions_ManagedAccount FOREIGN KEY REFERENCES ManagedAccount (ID), [CompanySymbol] [varchar](10) NOT NULL, [Shares] [int] NOT NULL, [Date] [datetime] NOT NULL) WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);GO

2.2.6

725

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 26: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

2.2.7Populating the Investment Tables with sample dataThe embedded file below contains all table data insert statements used for this scenario while debugging the new database deployment.

2.2.8Example Insert Statements from embedded SQL script1 Populate the Order Type TableINSERT INTO [dbo].[OrderType] ([Name]) VALUES ('BUY'), ('SELL')

GO

2 Populate the Risk Type TableINSERT INTO [dbo].[RiskType] ([Name]) VALUES ('LOW'), ('MEDIUM'), ('HIGH')

GO

3 Populate the Account Manager TableINSERT INTO [dbo].[AccountManager] ([FirstName],[LastName],[RiskSpecialty]) VALUES ('John', 'Doe', 1), ('Steve', 'Smith', 2), ('Jane', 'Doe', 3)

GO

4 Populate the Customer Account TableINSERT INTO [dbo].[CustomerAccount] ([FirstName],[LastName],[Email],[Address],[City],[State],[Zip],[Phone]) VALUES ('Peter', 'James', '[email protected]', '111 Newport Way', 'Nowhere', 'NV', '99999', '555-555-5555'), ('David', 'Smith', '[email protected]', '101 Bellevue Way', 'Bellevue', 'WA', '99999', '444-444-4444'), ('Daniel', 'Peterson', '[email protected]', '2751 North 1250 East', 'Denver', 'CO', '99999', '333-333-3333')

726

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 27: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

GO

5 Populate the Managed Account TableINSERT INTO [dbo].[ManagedAccount] ([Name],[RiskLevel],[CashAvailable]) VALUES ('401K', 3, '1050000.00'), ('Personal', 2, '1100000.00'), ('IRA', 1, '2030000.00')GO

6 Populate the Cuatomer Account Managed Account Relation TableINSERT INTO [dbo].[CustomerAccountManagedAccount] ([CustomerAccountID],[ManagedAccountID],[AccountManagerID]) VALUES (1,1,3), (2,2,2), (3,3,1)GO

7 Populate the Managed Account Positions Table (example)INSERT INTO [dbo].[ManagedAccountPositions] ([ManagedAccountID],[CompanySymbol],[Shares],[Date]) VALUES (1,'AAA',1000,getdate()), (1,'AAB',1000,getdate()), (1,'AAC',1000,getdate())

8 Populate the Managed Account Open Orders Table (example)INSERT INTO [dbo].[ManagedAccountOpenOrders] ([ManagedAccountID],[OrderType],[CompanySymbol],[Shares],[Price]) VALUES (3,2,'BBF',100,1005.3), (3,2,'AAH',100,230.4), (3,2,'AAK',100,195.9)

9 Populate the ManagedAccountCompletedTransactions Table (example)

INSERT INTO [dbo].[ManagedAccountCompletedTransactions] ([OpenOrderID],[CompanySymbol],[Shares],[Price],[CreatedDate]) VALUES (742,'AAH',100,'309.4',getdate()), (743,'AAH',100,'301.7',getdate()), (744,'BBJ',100,'339.4',getdate())

2.3 Testing the new Fabrikam Investments Database727

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 28: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

2.3.1Execute the Read Only Queries against the Listener (load balanced)

1 Setup a connection to the listener

2 Add MultiSubnetFailover=true;ApplicationIntent=readonly; to the connection string and connect

728

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 29: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

2.3.1.2.1 Additional Connection Parameters (Copy/Paste)MultiSubnetFailover=true;ApplicationIntent=readonly;

3

729

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 30: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

4 Execute the Reporting Queries1 Query for all Customer Account InformationUSE [FabrikamInvestmentsDb]GO

-- Select all information relating to a Customer AccountSELECT DISTINCT 'Customer Account Information', ma.Name, ca.*, rt1.Name AS RiskLevel, am.FirstName, am.LastName, rt2.Name AS RiskSpecialty, map.CompanySymbol, map.SharesFROM CustomerAccount caJOIN CustomerAccountManagedAccount cama ON cama.CustomerAccountID = ca.IDJOIN ManagedAccount ma ON ma.ID = cama.ManagedAccountIDJOIN RiskType rt1 ON rt1.ID = ma.RiskLevelJOIN AccountManager am ON am.ID = cama.AccountManagerIDJOIN RiskType rt2 ON rt2.ID = am.RiskSpecialtyJOIN ManagedAccountPositions map ON map.ManagedAccountID = ma.IDJOIN ManagedAccountOpenOrders cmaoo ON cmaoo.ManagedAccountID = ma.IDORDER BY ca.IDGO

2 Query for Open Orders by Company-- Query Open Orders by Company SymbolSELECT DISTINCT 'Open Orders by Company', CompanySymbol, OrderType, Count(*) AS CountFROM ManagedAccountOpenOrdersWHERE Completed = 0GROUP BY CompanySymbol, OrderTypeORDER BY CompanySymbol ASCGO

3 Query for Customer’s Stock Positions-- Query Customer PositionsSELECT DISTINCT 'Customer Positions', ma.Name, ca.*, am.FirstName, am.LastName, map.CompanySymbol, map.SharesFROM CustomerAccount caJOIN CustomerAccountManagedAccount cama ON cama.CustomerAccountID = ca.IDJOIN ManagedAccount ma ON ma.ID = cama.ManagedAccountIDJOIN RiskType rt1 ON rt1.ID = ma.RiskLevelJOIN AccountManager am ON am.ID = cama.AccountManagerIDJOIN RiskType rt2 ON rt2.ID = am.RiskSpecialtyJOIN ManagedAccountPositions map ON map.ManagedAccountID = ma.IDORDER BY ca.IDGO

4 Query for Customer’s Open Orders-- Query Customer Open OrdersSELECT DISTINCT 'Customer Open Orders', ma.Name, ca.*, am.FirstName, am.LastName, ot.Name, maoo.CompanySymbol, maoo.SharesFROM CustomerAccount ca

730

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 31: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

JOIN CustomerAccountManagedAccount cama ON cama.CustomerAccountID = ca.IDJOIN ManagedAccount ma ON ma.ID = cama.ManagedAccountIDJOIN RiskType rt1 ON rt1.ID = ma.RiskLevelJOIN AccountManager am ON am.ID = cama.AccountManagerIDJOIN RiskType rt2 ON rt2.ID = am.RiskSpecialtyJOIN ManagedAccountOpenOrders maoo ON maoo.ManagedAccountID = ma.ID AND maoo.Completed = 0JOIN OrderType ot ON ot.ID = maoo.OrderTypeORDER BY ca.IDGO

5 Query for Customer’s Completed Orders-- Query Customer Completed OrdersSELECT DISTINCT 'Customer Completed Orders', ma.Name, ca.*, am.FirstName, am.LastName, ot.Name, maoo.CompanySymbol, maoo.SharesFROM CustomerAccount caJOIN CustomerAccountManagedAccount cama ON cama.CustomerAccountID = ca.IDJOIN ManagedAccount ma ON ma.ID = cama.ManagedAccountIDJOIN RiskType rt1 ON rt1.ID = ma.RiskLevelJOIN AccountManager am ON am.ID = cama.AccountManagerIDJOIN RiskType rt2 ON rt2.ID = am.RiskSpecialtyJOIN ManagedAccountOpenOrders maoo ON maoo.ManagedAccountID = ma.ID AND maoo.Completed = 1JOIN OrderType ot ON ot.ID = maoo.OrderTypeORDER BY ca.IDGO

6 Query for the Most Active Stocks (Fabrikam Transactions)-- Query for the Most Active stocksSELECT TOP 3 'Most Active Stocks', [CompanySymbol], SUM(Shares) AS 'Shares Traded'FROM [ManagedAccountOpenOrders]WHERE [Completed] = 1GROUP BY [CompanySymbol]ORDER BY SUM(Shares) DESCGO

7 Query for the Top and Bottom Movers-- Top/Bottom Movers-- Create a temp table with the first and last transactions for each companySELECT * INTO #TempTable FROM(SELECT maoo1.ID, maoo1.[CompanySymbol], maoo1.[Price]FROM [ManagedAccountOpenOrders] maoo1 JOIN ( SELECT [CompanySymbol], MAX(ID) AS ID FROM [ManagedAccountOpenOrders] WHERE [Completed] = 1 AND OrderType = 2 GROUP BY [CompanySymbol] ) maoo2 ON ( maoo1.ID = maoo2.ID )UNION

731

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 32: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

SELECT maoo1.ID, maoo1.[CompanySymbol], maoo1.[Price]FROM [ManagedAccountOpenOrders] maoo1 JOIN ( SELECT [CompanySymbol], MIN(ID) AS ID FROM [ManagedAccountOpenOrders] WHERE [Completed] = 1 AND OrderType = 2 GROUP BY [CompanySymbol] ) maoo2 ON ( maoo1.ID = maoo2.ID )) AS Temp;

-- Query the temp table for the Biggest GainersSELECT TOP 5 t1.CompanySymbol, t2.Price - t1.Price AS 'Gain'FROM #TempTable t1JOIN #TempTable t2 ON t2.Price > t1.Price AND t1.CompanySymbol = t2.CompanySymbol AND t2.ID > t1.IDORDER BY t2.Price - t1.Price DESC

-- Query the temp table for the Biggest LosersSELECT TOP 5 t1.CompanySymbol, t2.Price - t1.Price AS 'Loss'FROM #TempTable t1JOIN #TempTable t2 ON t2.Price < t1.Price AND t1.CompanySymbol = t2.CompanySymbol AND t2.ID > t1.IDORDER BY t2.Price - t1.Price ASC

DROP TABLE #TempTableGO

4.1.2

732

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 33: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

4.1 Execute the Process Open Orders Script against the Primary SQL Server

1 Connect to the Primary SQL Server

733

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 34: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

2 Clear the Additional Connection Parameters and click Connect

3 Execute the Process Open Orders SQL scriptUSE [FabrikamInvestmentsDb]GO

--------------------------------------------------------------------------------------------------- NOTE: This script assumes all orders transact 100 shares of stock.-------------------------------------------------------------------------------------------------DECLARE @ID [bigint]DECLARE @OpenOrderID [bigint]DECLARE @Shares [int]DECLARE @Price [money]DECLARE @ManagedAccountID [bigint]DECLARE @OrderType [int]DECLARE @CashAvailable [money]

-- Display the current number of pending transactionsSELECT 'Before Processing Completed Transactions', count(*) FROM [ManagedAccountCompletedTransactions]

DECLARE CompletedOrderCursor CURSOR FOR SELECT [ID],[OpenOrderID],[Shares],[Price] FROM [ManagedAccountCompletedTransactions]

734

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 35: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

OPEN CompletedOrderCursor

WHILE (1=1)BEGIN FETCH NEXT FROM CompletedOrderCursor INTO @ID, @OpenOrderID, @Shares, @Price

IF @@FETCH_STATUS <> 0 BREAK

-- Start the Settlement Transaction BEGIN TRAN StockSettlement -- Update the open orders to completed UPDATE [ManagedAccountOpenOrders] WITH (Snapshot) SET [Completed] = 1 WHERE [ID] = @OpenOrderID

-- Get the Open Order Transaction Information SELECT @ManagedAccountID = maoo.[ID], @OrderType = maoo.[OrderType], @CashAvailable = ma.[CashAvailable] FROM [ManagedAccountOpenOrders] maoo JOIN [ManagedAccount] ma ON ma.ID = maoo.[ManagedAccountID] WHERE maoo.[ID] = @OpenOrderID

-- Buy Order IF @OrderType = 1 BEGIN UPDATE [ManagedAccount] SET [CashAvailable] = [CashAvailable] - (@Price * CAST(@Shares AS DECIMAL)) WHERE [ID] = @ManagedAccountID END ELSE -- Sell Order BEGIN UPDATE [ManagedAccount] SET [CashAvailable] = [CashAvailable] + (@Price * CAST(@Shares AS DECIMAL)) WHERE [ID] = @ManagedAccountID END

-- Complete the Open Order Transaction UPDATE [ManagedAccountOpenOrders] SET [Completed] = 1 WHERE [ID] = @OpenOrderID

-- Delete the transaction from the ManagedAccountCompletedTransactions table DELETE FROM [ManagedAccountCompletedTransactions] WHERE ID = @ID -- Commit the Settlement Transaction COMMIT TRAN StockSettlementEND

-- Clean upCLOSE CompletedOrderCursorDEALLOCATE CompletedOrderCursor

-- Make sure all the transactions were processed (should be 0)SELECT 'After Processing Completed Transactions', count(*) FROM [ManagedAccountCompletedTransactions]GO

735

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 36: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

736

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16

Page 37: Overview€¦ · Web viewAlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize

Highly Efficient and Performant Database

5 References5.1 Enhanced Always On

Enhanced Always On Availability Groups in SQL Server 2016SQL Server 2016 AlwaysOn (Video)

5.2 In Memory Column Store IndexesCREATE COLUMNSTORE INDEX (Transact-SQL) Columnstore Indexes GuideSQL Server 2016 ColumnStore Technology (Video)

5.3 Query StoreQuery Store in SQL Server 2016Monitoring Performance By Using the Query Store

737

Highly Efficient and Performant DatabaseVersion Error! Unknown document property name. last modified on 14 Jun. 16