Query Tuning – Getting it Right the First...

11
First Published in the ISUG-TECH Journal, July 2014 By Janis Griffin Query Tuning – Getting it Right the First Time This article provides the best methods for query tuning by utilizing Response Time Analysis (RTA) and SQL Diagramming techniques. DATA MANAGEMENT :: SAP ASE Janis Griffin has more than twenty- five years of DBA experience, in- cluding design, development, and implementation of many critical database applications. Before com- ing to Solarwinds, Janis held DBA positions, primarily in the telecom Industry (fifteen years), working with real-time routing databases, OLTP business to business applica- tions, and many financial/sales and marketing databases. As a principal architect and senior manager, Janis mentored many other DBAs on best practices in database design and per- formance tuning. She has proven to be an effective collaborator providing cutting-edge database solutions. W hether you are a developer or DBA, this article will outline a method for deter- mining the best approach for tuning a query every time by utilizing Response Time Analysis (RTA) and SQL Diagramming techniques. Regardless of the complexity of the statement, this quick and systematic approach will lead you down the correct tuning path with no guess- ing. If you are a beginner or expert, this approach will save you countless hours tuning a query. Challenges of Tuning Before we start down the path of how to tune, let’s look at the many challenges of tun- ing that often leave the DBA and/or developer frustrated with their results. Let face it, SQL tuning is difficult. To do it correctly, you or the people on your team need to be very familiar with many differ- ent aspects of query. From a technical standpoint, you need to understand the execu- tion plan, all the steps SAP ASE will take to execute the SQL, and how the data will be accessed. You also need to be familiar with SQL design concepts because sometimes tuning the SQL means rewrit- ing it. When you are working with the end users, you need to understand how the appli- cation is used and why they are running certain queries. For example, why do they fill in that field, or what do they do with the information on that screen, etc… Under- standing the purpose of the SQL/app will help you make better decisions down the road on how to tune it and will go a long way in putting attention to your tuning project. After all, many people are impa- tient, but SQL tuning takes time. That’s why working with other people is also about putting a face on your tuning project. Instead of the users saying something like “I’ve been complaining about this problem for weeks and the DBA team says they’re doing something, but no one knows what.”, they might say “I’ve been working with Jan from

Transcript of Query Tuning – Getting it Right the First...

  • First Published in the ISUG-TECH Journal, July 2014

    By Janis Griffin

    Query Tuning – Getting it Right the First TimeThis article provides the best methods for query tuning by utilizing Response Time Analysis (RTA) and SQL Diagramming techniques.

    DATA MANAGEMENT :: SAP ASE

    Janis Griffin has more than twenty-five years of DBA experience, in-cluding design, development, and implementation of many critical database applications. Before com-ing to Solarwinds, Janis held DBA positions, primarily in the telecom Industry (fifteen years), working with real-time routing databases, OLTP business to business applica-tions, and many financial/sales and marketing databases. As a principal architect and senior manager, Janis mentored many other DBAs on best practices in database design and per-formance tuning. She has proven to be an effective collaborator providing cutting-edge database solutions.

    Whether you are a developer or DBA, this article will outline a method for deter-mining the best approach for tuning a query every time by utilizing Response Time Analysis (RTA) and SQL Diagramming techniques. Regardless of the complexity of the statement, this quick and systematic approach will lead you down the correct tuning path with no guess-ing. If you are a beginner or expert, this approach will save you countless hours tuning a query.

    Challenges of TuningBefore we start down the path of how to tune, let’s look at the many challenges of tun-ing that often leave the DBA and/or developer frustrated with their results. Let face it, SQL tuning is difficult. To do it correctly, you or the people on your team need to be very familiar with many differ-ent aspects of query. From a technical standpoint, you need to understand the execu-tion plan, all the steps SAP

    ASE will take to execute the SQL, and how the data will be accessed. You also need to be familiar with SQL design concepts because sometimes tuning the SQL means rewrit-ing it. When you are working with the end users, you need to understand how the appli-cation is used and why they are running certain queries. For example, why do they fill in that field, or what do they do with the information on that screen, etc… Under-standing the purpose of the SQL/app will help you make better decisions down the road on how to tune it and will go a long way in putting attention to your tuning project. After all, many people are impa-tient, but SQL tuning takes time. That’s why working with other people is also about putting a face on your tuning project. Instead of the users saying something like “I’ve been complaining about this problem for weeks and the DBA team says they’re doing something, but no one knows what.”, they might say “I’ve been working with Jan from

    http://my.isug.com/journal/e/home

  • First Published in the ISUG-TECH Journal, July 2014

    SAP ASE :: DATA MANAGEMENT

    “From a technical standpoint, you need to un-

    derstand the execution plan, all the steps SAP

    ASE will take to execute the SQL, and how the

    data will be accessed.”

    the DBA group and she definitely asks a lot of questions and seems to care about my experi-ence.”

    Another huge challenge is that there are a large number of SQL statements in a database, so how do you know you’re working on the right one? I’ll talk about this in more detail, but this is also where the end users come in and can help you focus on the correct things. Often, they can show you the SQLs that af-fect them the most, or they know of a specific screen in an application that is slow. Instead of worrying about 100s or 1000s of SQLs running in the database, find the ones that affect the end user’s performance.

    Tuning is also difficult because all SQL statements are different. Just because you solved the last problem in 30 minutes by tun-ing a certain way, it doesn’t mean the next pro-ject is that easy or can be tuned the same way.

    Plus, some companies in general don’t care about performance. As long as the SQL gives the correct results, they don’t care how long it took. I believe some people just get used to “status quo” – “I always press this button the first thing in the morning and then go get cof-fee, because I know it takes an hour”. Bad per-formance becomes a way of life and sometimes people get mad when the app works efficiently as they have to do more.

    Finally, SQL tuning can be never ending. After tuning one SQL, another one will pop up. Also, as workloads and data change over time, there will always be a next performance prob-lem. However, once you tune something and people notice, they’ll start coming out of the woodwork to get their app tuned as well which could be a good thing for your career.

    Why Focus on QueriesSo why should you focus on tuning SQLs if there are so many challenges? Especially in this day and age where computers just keep getting faster and memory/disk are easy to purchase and add… Well, the reason it’s important to focus on queries is because most business applications connect and use data

    from databases. So a big majority of their processing time is spent reading data from or writing data to the database. Many of these applications deal with small amounts of data. They usually perform simple manipulation on that data or may just display the data. For example, take a web application for an order entry system– the tasks may be to lookup or update customer info, take an order, review order status, or lookup product information. Although, these tasks seem simple, there may be thousands and thousands of these types of requests hitting the database.

    From a Database perspective, most data-bases contain large amounts of data. However, they return very little information back to the application in a single request. So the database might have to read through millions of rows in order to return a few rows. You can see that in-efficient queries can quickly cause bottlenecks for the end user over time. In fact, they can affect the amount of time it takes to complete a task and ultimately affects productivity from a business standpoint.

    So again, why do SQL Tuning? It can give you the most dramatic improvements for an application and the “biggest bang for your buck”. Also, it’s a lot safer to tune a SQL state-ment then to rewrite the entire application or migrate to new hardware. Finally, it’s been my experience that most performance related is-sues at the database level are SQL related.

    Which SQL to Tune?Ok, we are convinced we need to tune our SQL statements, but which ones do we focus on? How do we know what SQLs are causing de-lays for our end users? How do we know we are working on the right problem? I’ve listed a few

    http://my.isug.com/journal/e/home

  • First Published in the ISUG-TECH Journal, July 2014

    QUERY TUNING – GETTING IT RIGHT THE FIRST TIME

    Figure 1: Response Steps

    methods here to help you choose which SQLs to work on.

    • User / Batch Job Complaints • Queries Performing Most I/O (LIO, PIO) • Queries Consuming CPU • Highest Response Times (DPA)

    For example, you may have a user com-plaint or a batch job that just isn’t completing like it should, so you set about tracing and tuning all of the statements for user /job to see if you can improve it. Or you may decide to use some arbitrary method such as sorting all of your SQLs by highest IO or looking at SQLs that are consuming the most CPU and set about tuning those. I believe the best method to use is Response Time Analysis (RTA) to determine which queries are causing your end users to wait. If you think about it, if your end users are waiting on a response from the data-base, they are probably not being very produc-tive. In the case of the order entry web ap-plication, this could directly be affecting your company’s bottom line. At Confio, our DPA product (formally known as Ignite) uses RTA to quickly find the ‘right problems’ to work on. In the next section, I’ll go into more detail about what this method is and how to use it.

    Response Time Analysis (RTA)Response Time Analysis is really about looking at a SQL request as it goes through the data-base and timing it until it returns a response to the end user process. As you can see in Figure 1, as a SQL request is sent to the data-base, there are many steps that it must com-plete in order to get a response back to the end user. Each one of those steps (yellow arrows) represents a unique step being performed. As each step is performed, SAP ASE records the time it took and the resource being used up or waited on by recording the wait event as the step occurs.

    By looking at the total response time and the wait events that SAP ASE transmits, you can quickly identify which SQLs are affecting

    your end users and rank which ones to work on first. This is what DPA does but you can collect this information as well.

    Also, in 15.7, there are 584 wait events and 11 classifications so it’s getting to be quite specific as to what resource these SQL requests are waiting on.

    MDA TablesYou can get the amount of time a SQL takes in the database as well as what resource it’s either waiting on or using by looking the MDA tables. The acronym, MDA, stands for Monitor-ing & Diagnostics for SAP ASE but I’ve seen other definitions for this acronym as well. The important thing is, the key word, ‘Monitoring’. SAP ASE gives you these monitoring tables so you can get all kinds of good information about the system, all objects and processes work-ing in each instance of SAP ASE. SAP ASE came out with the MDA tables in 12.5.0.3 and they’ve been expanding on them ever since. In version 15.7, I counted 83 MDA tables and

    “I believe the best method to use is Response

    Time Analysis (RTA) to determine which que-

    ries are causing your end users to wait. If you

    think about it, if your end users are waiting on

    a response from the database, they are prob-

    ably not being very productive.”

    http://my.isug.com/journal/e/home

  • QUERY TUNING – GETTING IT RIGHT THE FIRST TIME

    DA

    TA

    MA

    NA

    GE

    ME

    NT

    First Published in the ISUG-TECH Journal, July 2014

    Figure 2: Response and Wait Time Tables

    SELECT mP.Login,mP.DBName,mPL.ClientHost, RTRIM(mPL.Application),mPP.ObjectName, mP.WaitEventID,mPS.SQLText,mPS.BatchID, mPS.LineNumber, mP.LineNumber CurrentLineNumberFROM monProcessSQLText mPS, monProcessLookup mPL, (monProcess mP LEFT OUTER JOIN monProcessProcedures mPP ON mP.SPID=mPP.SPID AND mP.KPID=mPP.KPID)WHERE mP.SPID=mPL.SPID AND mP.KPID=mPL.KPIDAND mP.SPID=mPS.SPID AND mP.KPID=mPS.KPIDAND coalesce(mP.FamilyID,0)=0 AND mP.BatchID > 0

    the URL listed below will direct you to a really nice diagram of all the MDA tables and their relationships: http://froebe.net/blog/wp-con-tent/uploads/2013/09/ASE-15.7-Monitoring-Tables-Diagram-ASE-Inc_.pdf

    If you are not familiar with these tables, I’d suggest you take a look at the diagram because it will help you get acquainted quickly. There is a lot of information that can be seen in these tables. For example, you can spot unused in-dexes or heavily used tables.

    However, for this article, we will focus on the five tables that I’ve listed Figure 2. These tables deal with response or wait time and shows the wait events each process encoun-ters as a SQL request is processed through the database. As you can see, I’ve highlighted the joining columns and I’ve underlined some of the more interesting columns to look at. For example, we can get login, database name from “monProcess” table; we can get the ob-ject name, line number, statement number from “monProcessProcedures” table; and client hostname from “monProcessLookup” table. Finally, we can get the SQL text from “mon-ProcessSQLtext”.

    It’s important to know that these tables are ‘point in time’ or real-time only tables so you will need to capture this information and store it off somewhere in order to further analyze the data. In fact, one of the problems with these tables is that they are either point in time (like the four process tables) or they are

    maintained (stateful) over a certain amount of time. So you really can’t answer the ques-tion, “what happened yesterday between 2pm or 3pm?” or, “what happened last Wednesday, when you took that day off?”

    Response Time QueryHere is an example query to run against these tables in order to get the information to store off somewhere for further analysis. This is what DPA does. It polls these tables every second and then stores the information in a repository for further analysis. As you can see, there is not a great deal of magic needed to get this information. The real power of Response Time Analysis is in the presentation of all the information gathered. MDA Query Example:

    Here is an example of how DPA shows and presents response time information in what I call the ‘proactive’ view. To understand what might be going wrong in your instance, you need to be able to go back in time to see how this instance typically runs. In other words, you need to have some idea of what ‘normal’ times are, in order to be able to spot the ‘abnor-mal’ times.

    The DPA trend screen (Figure 3), shows seven days of response times analysis for the SAP ASE instance on a machine called ‘Gib-son’. Each bar represents the total time spent in the instance for one day, for the top 15 SQL statements running in a database. Each color in the bar represents a unique SQL statement or stored procedure call and quickly shows you which SQLs are taking the most time. The leg-

    http://my.isug.com/journal/e/home

  • First Published in the ISUG-TECH Journal, July 2014

    QUERY TUNING – GETTING IT RIGHT THE FIRST TIME

    Figure 4: Queries Running in Time Slice

    end to the right is ordered by the SQL with the highest amount of time spent. You can quickly see that the purple color – the one I’ve called ‘Sel Customers’ is the top SQL statement in this instance. On the left hand side, there is a vertical legend of accumulated hours that show the time spent in this instance. As you can see, August 10th was a particularly bad day with processes spending almost 50 hours of accumulated time spent on those top 15 SQL statements.

    Also, notice at the bottom of the screen, DPA goes through some initial analysis for you in the ‘Top Query Problem’ section. DPA basi-cally goes out every hour each day and tells you which SQL you should focus on. It even gives you some ideas on how to tune them by looking at the resource information. For this instance, DPA has warnings for three SQLs running on the August 12th. The first one is a blocking issue, which if you click on the ‘more’ link, it will give an in depth look at the SQL and give suggestions on how to tune it.

    Response time analysis is the best and quickest way to find the root cause of the problem when you are in ‘firefighting’ mode. The next screen shot (Figure 4) shows specific queries running in the database at a specific time frame.

    Notice that we are looking between 12:00pm and 12:05pm on Aug 10th – a five minute time slice. Each bar represents how much time each query spent in the database for this five min-utes. The different colors on each bar repre-sent what resource or wait event the query is

    spending most of its time on. Notice that my ‘Sel Customers’ query has spent most of its time waiting ‘on run queue’ and ‘buffer reads to complete’. The second query listed is primarily waiting on a semaphore or lock as is the fifth query, the ‘QA_insertCustomer…’ query. With this graphical view, you can easily spot which queries are having issues and what is causing the delay.

    It’s important to note, this view of combin-ing queries with the wait events they encoun-ter, is not easily found in SAP ASE today. You will need to collect this information or use some tool like DPA which shows it.

    Also, each wait event listed here give clues on how to go about tuning the SQL so you can reduce the response time on the wait event. In the case of this locking issue, it may not be a SQL tuning effort; instead it may be an application or concurrency issue. Also, notice that there is a ‘blockers’ tab on the right of the screen. By collecting this data, you can quickly get specific information about the blockers, the waiters and the impact that it has on your end users.

    Figure 5 shows the information that you can view while blocking is occurring. Notice that SPID 125 has caused 90 seconds or 1.5 minutes of wait time for other processes in the database. SPID 164 waited several times; first 22 seconds trying to run some dynamic SQL, and then it spent 15 seconds waiting to run line 13 of our ‘insertcustomer’ stored proce-dure. You can see that each session is waiting

    Figure 3: Response Times Analysis

    http://my.isug.com/journal/e/home

  • QUERY TUNING – GETTING IT RIGHT THE FIRST TIME

    DA

    TA

    MA

    NA

    GE

    ME

    NT

    First Published in the ISUG-TECH Journal, July 2014

    Figure 5: Blocking Details

    Figure 6: One Month’s Activity/Tuning Efforts

    for a semaphore and we can see what user, program and machine it’s coming from. At the bottom of the page, you can see what the block-er SPID (125) was running while it caused the blocking. It looks like the session was running a stored procedure called ‘createInvoice’.

    Now you can quickly tell why the blocking has occurred. The ‘create Invoice’ procedure was accessing the customer table while the ‘in-sertcustomer’ procedure was trying to change the table.

    Finally, Response Time Analysis shows ‘long term’ trends over time. This is important if workloads change or data grows as these often cause slowdowns for the end users. Also, it’s important to validate any of your tuning efforts. Did you really help improve perfor-mance by adding that index? Figure 6 shows approximately one month’s activity and my tuning efforts.

    You will notice that once I fixed the blocking issue, the top SQL disappeared from the chart. Also, notice that after applying several indexes on the August 18th, the 2nd and 3rd SQLs are spending much less time in the database. In fact, I reduced the time spent from 25 – 30 hours a day for the top 15 SQLs to less than one hour a day! I think I’ll save this chart to show my boss during my performance review!

    How to TuneOk, so now you’ve found the queries that are causing bottlenecks for end users, what do you do with them? This really speaks to the exper-tise that DBAs and Developers need in order to know how to tune SQL statements. Some-times, during my web conferences with DBAs or Developers, they want a tool that tells them exactly how to fix it. From my experience, I don’t think there is such a tool. Some tools, like DPA, will get you close, but at the end of the day you will need to know how the data is accessed, the different join methods and how to write good SQL. To tune, I use the resource or wait event information, and further com-bine that information with the query execution plan. From the execution plan, I can see how the data is being accessed and view each step to quickly see what or where the problem is.

    There are several ways to get the query execution plan for SQL in SAP ASE. One is to ‘set showplan_all on’ in the SQL editor before you execute the query. It’s important to note that there are many set options. For example, I usually use the ‘set statistics io on’ command by habit. If you think about it, from a server resource perspective, IO is one of the most expensive activities, so measuring this statistic as you tune will show you how successful you are. There are many statistics that you can gather but I would advise you to select only the statistics that are meaningful to you. If you turn them all on, you may get too much infor-mation where you’ll quickly get confused.

    Another way to view query plans is in Inter-active SQL. If you are particularly lazy, like I tend to be… you can use the ‘Plan Viewer’ to

    http://my.isug.com/journal/e/home

  • First Published in the ISUG-TECH Journal, July 2014

    QUERY TUNING – GETTING IT RIGHT THE FIRST TIME

    get the same information not only in a textual format, but a graphical view which I like. Plus you can get further advanced analysis as well, without worrying about the ‘set’ options. An-other way to collect query plans is to collect ab-stract query plans and metrics. You can do this by enabling metrics capture at the instance or session level with the following command: sp_configure 'enable metrics capture', 1.

    Finally, starting with version 15.0.2, SAP ASE gives you the ability to do application tracing which gives even more information (e.g. wait events) for each SQL. The problem with tracing is that you have to know when the problem is going to occur in order to capture the information. Otherwise, there’s too much information to wade through, and often too many details in order to determine the issue.

    Figure 7 shows an example of a query plan from the plan view in Interactive SQL. Basi-cally, this is the execution plan for the MDA query that I showed you earlier. To read execu-tion plans and follow the order of their steps, you need to read from left to right and from the bottom to the top. For example, the first table accessed in the query is the ‘monProcess’ table which is being further ‘restricted’ by the filter-ing in the ‘where’ clause.

    The result set is then sorted and joined to the sorted table, ‘monProcessLookup’, using the join method, ‘MergeJoin’. The subsequent result set is then sent to another ‘MergeJoin’, joining the sorted table ‘monProcessSQLText’, etc...

    I usually look at query plans, check the costs of each step and try to focus on the most costly steps first. I also review how the data is being accessed. As you can see in the example, the data access to all the tables is a ‘TableS-can’. A ‘TableScan’ sticks out to me because SAP ASE has to read the whole table which might be quite large. Now, if you’re reporting on the whole table that could be the best way to go after the data.

    However, if you only want 10% or less of the data, a TableScan could mean a lot of work has to happen to first find that data. At this point, I often reference the number of logical IOs that were performed to get the final result of the query. The reason for this is because SAP ASE always has to read data in memory before it re-turns a result set. It may or may not have to do physical IOs, since the data could already be in memory. So measuring the logical IOs is a good metric to use to see if you actually improved performance as you tune. I’ll show how this works as I dive into the case studies.

    Case Study #1We are now at the point where several case studies can help illustrate using Response Time Analysis (RTA) techniques to find the right queries to work on, and then use SQL di-agramming techniques in order to tune them. The first case study comes from one of our customers, a University that was encountering poor performance during their billing cycle. By utilizing RTA, they were able to quickly find the query that was spending the most time in the database. The query listed below, is trying to answer the business question of ‘Who regis-tered yesterday for a SQL Tuning class?’Figure 7: Query Plan Example

    “I believe it’s important to understand eve-

    rything about the tables being accessed in a

    query such as number of rows, column cardi-

    nality, indexes, and how the tables are related

    to each other.”

    http://my.isug.com/journal/e/home

  • QUERY TUNING – GETTING IT RIGHT THE FIRST TIME

    DA

    TA

    MA

    NA

    GE

    ME

    NT

    First Published in the ISUG-TECH Journal, July 2014

    SELECT s.FNAME, s.LNAME, r.SIGNUP_DATEFROM STUDENT s INNER JOIN REGISTRATION r ON s.STUDENT_ID = r.STUDENT_IDINNER JOIN CLASS c ON r.CLASS_ID = c.CLASS_IDWHERE c.NAME = 'SQL TUNING'AND r.SIGNUP_DATE BETWEEN @BegDate AND @EndDateAND r.CANCELLED = 'N‘

    Figure 8: Relationships of Tables

    Figure 9: Execution Plan and Logical IOs

    I believe it’s important to understand eve-rything about the tables being accessed in a query such as number of rows, column cardi-nality, indexes, and how the tables are related to each other. A quick way to gather some of this information is to quickly draw an entity relationship diagram (ERD) of only the tables in query. Figure 8 shows the relationship of the three tables in the billing query. Notice that the ‘registration’ table has two lookups into the ‘class’ and ‘student’ table and that each table has a primary key with supporting index.

    Figure 10: SQL Diagramming

    SQL DiagrammingA great technique to quickly find the best execution plan is SQL or Query Diagramming. Basically, you take the tables referenced in the query and draw an upside down tree with the detail table, ‘registration’ being at the top. Then you draw lines or links to any of the lookup or reference tables. In this example, the reference tables are ‘class and ‘student’, re-spectively. The final step will be to determine the best filtering criteria and join conditions. Figure 10 shows the SQL diagram of the bill-ing query.

    At this point, it’s important to get the cur-rent execution plan, wait events and initial logical IOs for the query so you’ll have some-thing to measure against after the tuning. Otherwise, you may not know whether you’ve tuned it or not.

    The billing query was spending 90% of its time on the wait event, ‘waiting for i/o (read or write) to complete’ and Figure 9 shows the execution plan and number of logical IOs per-formed. Our goal will be to reduce the cost of the TableScan on ‘registration’ and the number of logical IOs.

    The numbers listed at the ends of each line or link are approximately the number of rows from each table that should be joined to return a result. For example, for one student there are approximately five registration records as most college students take five hours of course work in a semester. For one class, there are

    http://my.isug.com/journal/e/home

  • First Published in the ISUG-TECH Journal, July 2014

    QUERY TUNING – GETTING IT RIGHT THE FIRST TIME

    approximately 30 records in registration due to the average size of a classroom. Therefore, the number of rows joined should be close to a 150 rows per student if there isn’t any other filter-ing critera.

    The underlined numbers represent the filtering criteria in the ‘where’ section of the query. In this example, there are two filters. The first filter is on the registration table where the ‘cancelled’ column must equal ‘N’, and ‘signup_date’ is yesterday. As you can see, there are a total of 79,999 rows in the registra-tion table and 4,141 rows or 5% of the table will be returned with that filter.

    The second filter is on the class table where the name column is equal to ‘SQL TUNING’. Only .2% of the rows in the class table will be returned. From this information, we can see that the class filter is the best filter to reduce the number of rows needed to be read. So, let’s put an index on the class table called, ‘cl_name’ and review the execution plan and our logical IOs.

    In Figure 11, you can see the execution plan has changed from a TableScan on class to an IndexScan but the logical IOs are still high. Also, why is a TableScan still being performed on the registration table?

    SELECT o.OrderID, c.FirstName, c.LastName, p.ProductID, p.Description, s.ActualShipDate, s.ShipStatus, s.ExpectedShipDateFROM SalesOrder oINNER JOIN OrderItem i ON i.OrderID = o.OrderIDINNER JOIN Customer c ON c.CustomerID = o.CustomerIDINNER JOIN Shipment s ON s.ShipmentID = i.ShipmentIDLEFT OUTER JOIN Product p ON p.ProductID = i.ProductIDLEFT OUTER JOIN Address a ON a.AddressID = s.AddressIDWHERE c.LastName LIKE ISNULL('Griffin','') + '%'AND o.OrderDate >= DATEADD(day, -30,getdate())AND s.ShipStatus 'C'order by LastName

    After adding the index, you can see that we’ve successfully reduced the number logical IOs from 1,677 to 87 and access to the registra-tion table is now an IndexScan. Also, notice that the number of physical IOs are reduced as well.

    Case Study #2The next tuning example deals with slow response from an order entry system. By using RTA, we found a lot of time was being spent on one query. The SQL listed below is trying to answer the business question of ‘What’s the shipment status from a caller on phone?’

    left-leading part of the index. SAP ASE will need to perform a full TableScan because it can’t directly access the class_id column in the clustered index. To fix this issue, let’s put a nonclustered index called ‘reg_alt’ on class_id in the registration table (Figure 12).

    Figure 12: Nonclustered Index on Registration

    Upon further investigation of the ERD, we can see that there is a clustered index on the registration table. The index contains the student_id, class_id, and signup_date columns in that specific order, so student_id is the

    Figure 11: Execution Plan with IndexScan

    http://my.isug.com/journal/e/home

  • QUERY TUNING – GETTING IT RIGHT THE FIRST TIME

    DA

    TA

    MA

    NA

    GE

    ME

    NT

    First Published in the ISUG-TECH Journal, July 2014

    Figure 13: Relationships of Tables

    Figure 14: RTA Information

    Figure 15: SQL Diagramming

    Notice, that the execution plan has several costly TableScans on salesorder and customer. To find the best execution plan, we will again use SQL diagramming techniques. Figure 15 shows the detail tables of salesorder (o), order-item (i) and shipment (s), with lines or arrows to the lookup tables of customer (c), product (p) and address (a). In this example, I’ve left off the join numbers as I really want to focus on the filtering criteria in order to find the best driving or initial table to start with.

    This query has three filters in the ‘where’ clause. The first filter is looking at the ‘Last-Name’ column on the customer table. To find an estimated number of rows returned with this filter, I look for my last name and .5% of the table will be returned. The next filter is on the ‘OrderDate’ column in salesorder table where it’s looking for orders within the last 30 days. We can see that 2% of the rows in the salesorder table will be read. Finally, the last filter is on the ‘ShipStatus’ column in the shipment table where 4% of the rows will be returned.

    From this example, we can see that our best filter is on the ‘LastName’ column of the customer table, so let’s put an index on that column.

    Figure 16 shows that after adding the in-dex, the execution plan performs an IndexScan on the customer table and significantly reduces the number of logical IOs from over 26,000 to

    Let’s begin tuning this SQL by gathering our initial measurements of current execu-tion plan, primary wait event and the number of logical IOs. In Figure 14, we can see that 26,603 logical IOs were performed and our RTA information showed that this SQL spent most of it’s time on a ‘wait for buffer read to complete’ wait event.

    There are five tables in this query, so let’s review the ERD to get acquainted with how they are related. In Figure 13, the salesorder table has a lookup into the customer table. The orderitem table has a lookup into the salesor-der, product and shipment tables. The ship-ment and customer tables have a lookup into the address table. It is also important to know the primary and foreign keys for each table and make sure that the foreign keys have sup-porting indexes.

    http://my.isug.com/journal/e/home

  • First Published in the ISUG-TECH Journal, July 2014

    QUERY TUNING – GETTING IT RIGHT THE FIRST TIME

    1,103. Also, it’s worth noting that there’s still a costly step on the salesorder table. At this point, we may want to consider adding an in-dex on the ‘OrderDate’ column as well.

    Figure 17 shows that an index on the salesorder table further reduces the number of logical IOs from 1,100 to just over 100 and speeds up the query even more.

    Often times, I’ll get asked, ‘How do I know if I’m done tuning?’ Especially in this example, where there was three filters and we only re-viewed and indexed the filters on customer and salesorders tables. There is another filter on the ‘ShipStatus’ column in the shipment table that is basically looking for status not equal to ‘C’ or complete status.

    We can see that there is already an IndexS-can happening on the shipment table but what index is it using? Also, what do we know about

    the ‘ShipStatus’ column? Are the values evenly distributed or is there data skew? In this case, the column is heavily skewed with 116,212 rows containing the value of ‘C’ and only 5,105 rows (or less than 4% of the table) containing the value of ‘I’ for incomplete status. In this case, we may want to add a histogram in order to give SAP ASE more information about this data skew.

    In Figure 18, we update the statistics on the ‘ShipStatus’ column and reduced the num-ber of Logical IOs even more. However, there isn’t a significant increase in performance, so at this point I generally stop tuning, yet I con-tinue to monitor it with RTA.

    Figure 16: Index on LastName

    Figure 17: Index on OrderDate

    Figure 18: Update Statistics on ShipStatus

    SummaryIn conclusion, tuning queries often gives you the “biggest bang for the buck” when trying to improve database performance. You will want to make sure that you are working on the right queries by using wait events and RTA to quick-ly find the SQLs spending the most time in the database. Once you’ve identified which SQL to tune, gather the ERD, baseline measurements, and execution plan.

    Also, you might consider using SQL Dia-gramming techniques so you can quickly identify the best execution plan. A great book to reference on SQL Diagramming is “SQL Tuning” by Dan Tow (http://www.singingsql.com). Finally, after tuning, continue to moni-tor using RTA to quickly identify your next tuning opportunity. ■

    http://my.isug.com/journal/e/home