Using End-Of-Time Date Semantics to Improve Performance
-
Upload
donald-bales -
Category
Documents
-
view
25 -
download
1
Transcript of Using End-Of-Time Date Semantics to Improve Performance
1HS2 Solutions confidential and proprietary.
9 / 1 5 / 2 0 1 6
By Donald Bales, Rails Practice Lead
USING END-OF-TIME ACTIVE DATE SEMANTICS TO IMPROVE PERFORMANCE
2HS2 Solutions confidential and proprietary.
Underlying almost every Rails application is a relational database management system. Let me show you how important it is to apply some fundamental time rules to your application's database in order to get the best response times possible.
ABSTRACT
3HS2 Solutions confidential and proprietary.
For 20 years, we have helped some of the best brand and eCommerce companies leverage the internet and digital marketing.
We value smart engineering and team members that collaborate well internally as well as with our clients and their agencies.
Leveraging Technology & Intelligence to Drive Results
WHO WE ARE
4HS2 Solutions confidential and proprietary.
HS2 HISTORY
1 9 9 4
Software DevelopmentHollyer & Schwartz (H&S)
1 9 9 9
eBusinessH&S acquired by XOR, Inc.
2 0 0 1
Precision Marketing
XOR merges with Seurot
2 0 0 3
HS2 FormedSeurat acquired by Fair Isaac. HS2
Solutions formed.
Hollyer & Schwartz was founded in 1994 as a software development and systems integration company. The core team has evolved and grown together for over 15 years into a full-service eBusiness and Precision Marketing company.
WHO WE WORK WITH
HS2 Solutions confidential and proprietary. 5
6HS2 Solutions confidential and proprietary.
WHAT WE DO
ECOMMERCE, WEB & MOBILE DEVELOPMENT
ANALYTICS & INSIGHTSEXPERIENCE DESIGN (UX/UI)
INTERACTIVE MARKETING
7HS2 Solutions confidential and proprietary.
“D O N A L D B A L E S
The obvious is always illusive
8HS2 Solutions confidential and proprietary.
• There’s not a bit of Ruby or Rails code
• Fundamentals
• Structured Query Language (SQL)
• Data Definition Language (DDL)
• Data Manipulation Language (DML)
DISCLAIMER
PROGRAMMER
EXPLICIT CONTENTADVISORY
9HS2 Solutions confidential and proprietary.
don=# create table test_integer (an_integer integer);
CREATE TABLE
don=# insert into test_integer (an_integer) values (1.5);
INSERT 0 1
don=# select * from test_integer;
an_integer
------------
?
WARM-UP EXERCISE QUESTION
10HS2 Solutions confidential and proprietary.
don=# select * from test_integer;
an_integer
------------
2
(1 row)
WARM-UP EXERCISE ANSWER
11HS2 Solutions confidential and proprietary.
where start_date <= CURRENT_DATE
and end_date >= CURRENT_DATE
V.
where start_date <= CURRENT_DATE
and (end_date >= CURRENT_DATE or end_date is NULL)
WHAT’S THE DIFFERENCE BETWEEN THESE?
12HS2 Solutions confidential and proprietary.
where CURRENT_DATE between start_date and end_date
V.
where start_date <= CURRENT_DATE
and (end_date >= CURRENT_DATE or end_date is NULL)
WHAT’S THE DIFFERENCE BETWEEN THESE?
13HS2 Solutions confidential and proprietary.
ANSWER: EFFICIENCY AND PERFORMANCE
14HS2 Solutions confidential and proprietary.
In this context we are talking about determining if something is active at some point in time by comparing that point in time against an item's start and end dates.
So if I have a time line:
WHAT DOES IT MEAN TO BE ACTIVE?
Jan 1
start
Mar 31
Active Period Inactive Period
PointIn Time
end
15HS2 Solutions confidential and proprietary.
We can say that at this point in time, that is, when the point in time is between the start and end dates, the item is active.
WHAT DOES IT MEAN TO BE ACTIVE?
Jan 1
start
Mar 31
Active Period Inactive Period
PointIn Time
end
16HS2 Solutions confidential and proprietary.
WHAT DOES IT MEAN TO BE ACTIVE?
Jan 1
start
Mar 31
Active Period Inactive Period
PointIn Time
end
where CURRENT_DATE between start_date and end_dateIn SQL:
17HS2 Solutions confidential and proprietary.
But what do we do if we don't know when an item will become inactive?
The typical and intuitive programming solution is not to specify an end date:
WHAT IF WE DON’T KNOW THE END DATE?
Jan 1
start
Active Period
PointIn Time
18HS2 Solutions confidential and proprietary.
Name Null? Type
------------------------------- -------- ----------------------
ID NOT NULL NUMBER(38)
CODE NOT NULL VARCHAR2(30)
DESCRIPTION VARCHAR2(4000)
START_DATE NOT NULL DATE
END_DATE DATE
WHAT IF WE DON’T KNOW THE END DATE?
19HS2 Solutions confidential and proprietary.
WHAT IF WE DON’T KNOW THE END DATE?
Jan 1
start
Active Period
PointIn Time
where start_date <= CURRENT_DATEand (end_date >= CURRENT_DATE or end_date is NULL)
In SQL:
20HS2 Solutions confidential and proprietary.
But using this semantic for “active” is wholly inefficient for queries against a database. Is there a more efficient yet equivalent way to represent no end date?
WHAT IF WE DON’T KNOW THE END DATE?
Jan 1
start
Active Period
PointIn Time
21HS2 Solutions confidential and proprietary.
Yes! We can substitute a code-able notion of the end of time: 12/31/9999
Now the item is still active, at this moment, and through the end-of-time, but it’s no longer NULL! And, that, makes all the difference.
USE A KNOWN VALUE TO REPRESENT THE END-OF-TIME
Jan 1
start
Active Period
PointIn Time
Dec 31, 9999
end-of-time
end
22HS2 Solutions confidential and proprietary.
• How about December 31, 9999 or 12/31/9999
• It works for all these:• DB2
• MariaDB/MySQL
• Microsoft SQL server
• Oracle
• PostgreSQL
• Sysbase
…
DEFINE AN END-OF-TIME FOR UNKNOWN END DATES
23HS2 Solutions confidential and proprietary.
Name Null? Type
------------------------------- -------- ----------------------
ID NOT NULL NUMBER(38)
CODE NOT NULL VARCHAR2(30)
DESCRIPTION VARCHAR2(4000)
START_DATE NOT NULL DATE
END_DATE NOT NULL DATE
USE A KNOWN VALUE TO REPRESENT THE END-OF-TIME
24HS2 Solutions confidential and proprietary.
USE A KNOWN VALUE TO REPRESENT THE END-OF-TIME
Jan 1
start
Active Period
PointIn Time
where CURRENT_DATE between start_date and end_date
Dec 31, 9999
end-of-time
In SQL:
25HS2 Solutions confidential and proprietary.
In the future, when someone wants to make the item truly inactive, they update the end date to a non-end-of-time value:
USE A KNOWN VALUE TO REPRESENT THE END-OF-TIME
Jan 1
start
Active Period
PointIn Time
Dec 31, 9999
end-of-time
end
Mar 31
Inactive Period
26HS2 Solutions confidential and proprietary.
Using end-of-time semantics for end date instead of no end date is extremely important if one is concerned about efficiency and performance.
Why?
That's what we will discuss in the remainder of this presentation.
WANT EFFICIENCY AND PERFORMANCE?
27HS2 Solutions confidential and proprietary.
LET’S REVIEW
28HS2 Solutions confidential and proprietary.
Determining if an entry is active is done by testing if the start date is in the past or the current moment and the end date, if it exists, is in the current moment or the future, and if it does not exist, that it is NULL. In SQL:
where CURRENT_DATE >= start_date
and (CURRENT_DATE <= end_date or end_date is NULL)
WITH A NULLABLE END DATE
29HS2 Solutions confidential and proprietary.
This is not an optimal situation, because a NULL value cannot be indexed, and accordingly, the database will have to do a full table scan, or index scan against the start date if it is indexed.
WITH A NULLABLE END DATE
30HS2 Solutions confidential and proprietary.
An optimal way to state that something is active is to populate both the start and end date. By setting the end date to the end of code-able time, say 12/31/9999, determining if an entry is active is done by testing if the start date is in the past or the current moment and the end date is in the current moment or the future. In SQL:
where CURRENT_DATE >= start_date
and CURRENT_DATE <= end_date
AN OPTIMAL APPROACH
31HS2 Solutions confidential and proprietary.
Another way to write it in SQL:
where CURRENT_DATE between start_date and end_date
AN OPTIMAL APPROACH
32HS2 Solutions confidential and proprietary.
• NULL values typically can’t be indexed
• This leads to full table scans or
• This leads to full index scans or
• This leads to partial index scans with partial table scans
• Full and partial table scans flush the database’s buffers (cache)
• Un-necessary work consumes capacity that may be needed elsewhere
• As table and index size grow, queries slow
WHY ARE NULL VALUES A PROBLEM?
33HS2 Solutions confidential and proprietary.
• An example using a dictionary• Find by Definition (full scan) – “free from legal or discriminatory restrictions”
• Start at the first page
• Read definitions
• Compare
• Match? You’ve found it, so you’re done, else go to next page…
FULL TABLE SCANS
Found It!
34HS2 Solutions confidential and proprietary.
• Find by Word (index scan) - “source”
• Start anywhere
• Read word
• Compare
• Match? You’ve found it! You’re done
• Less than, jump forward and compare again
• Greater than, jump backward and compare again
INDEX SCANS
Found It!
35HS2 Solutions confidential and proprietary.
PERFORMANCE AS A FACTOR OF SIZE
- 5,000,000 10,000,000 -
5,000 10,000 15,000 20,000 25,000 30,000 35,000
FULL TABLE SCAN
Full Scan
“As tables grow, queries slow.”
milliseconds
rows
36HS2 Solutions confidential and proprietary.
PERFORMANCE AS A FACTOR OF ORDER
- 5,000,000 10,000,000 -
5,000 10,000 15,000 20,000 25,000 30,000 35,000
UNCACHED INDEX RANGE SCANS
Full ScanStart-End UncachedStart-Null End Uncached
Look at how poorly that null-able end
date performs
37HS2 Solutions confidential and proprietary.
That is, using memory I/O instead of physical disk I/O
• NOTE: Full table scans flush cache, making it useless
PERFORMANCE AS A FACTOR OF CACHE
38HS2 Solutions confidential and proprietary.
PERFORMANCE AS A FACTOR OF CACHE
- 5,000,000 10,000,000 -
5,000 10,000 15,000 20,000 25,000 30,000 35,000
CACHED INDEX RANGE SCANS
Full ScanStart-End UncachedStart-Null End UncachedStart-End CachedStart-Null End Cached
A full table scan is faster than that
null-able end date
39HS2 Solutions confidential and proprietary.
LET’S TAKE A CLOSER LOOK
40HS2 Solutions confidential and proprietary.
PERFORMANCE AS A FACTOR OF SIZE
- 200 400 600 800 1,000 - 1 2 3 4 5 6 7
FULL TABLE SCAN
Full Scan
41HS2 Solutions confidential and proprietary.
PERFORMANCE AS A FACTOR OF ORDER
- 200 400 600 800 1,000 - 1 2 3 4 5 6 7
UNCACHED INDEX RANGE SCANS
Full ScanStart-End UncachedStart-Null End Uncached
42HS2 Solutions confidential and proprietary.
PERFORMANCE AS A FACTOR OF CACHE
- 200 400 600 800 1,000 - 1 2 3 4 5 6 7
CACHED INDEX RANGE SCANS
Full ScanStart-End UncachedStart-Null End UncachedStart-End CachedStart-Null End Cached
43HS2 Solutions confidential and proprietary.
AN EVEN CLOSER LOOK!
44HS2 Solutions confidential and proprietary.
PERFORMANCE AS A FACTOR OF CACHE
- 10 20 30 40 50 60 70 80 90 100 - 1 2 3 4 5 6 7
CACHED INDEX RANGE SCANS
Full ScanStart-End UncachedStart-Null End UncachedStart-End CachedStart-Null End Cached
45HS2 Solutions confidential and proprietary.
PERFORMANCE AS A FACTOR OF CACHE
- 5 10 15 20 25 - 1 2 3 4 5 6 7
CACHED INDEX RANGE SCANS
Full ScanStart-End UncachedStart-Null End UncachedStart-End CachedStart-Null End Cached
46HS2 Solutions confidential and proprietary.
PERFORMANCE CHARACTERISTICS AND TIME
47HS2 Solutions confidential and proprietary.
• In the beginning• Database is small
• Tables are small
• Indexes are small
• Temporal width is small
• Queries are fast
PERFORMANCE CHARACTERISTICS CHANGE OVER TIME
48HS2 Solutions confidential and proprietary.
“YO U R A P P L I C AT I O N U S E R S
Boy! This application is great!
49HS2 Solutions confidential and proprietary.
• As time goes by• Database gets larger
• Tables get larger
• Indexes get larger
• Temporal width gets larger
• Queries take longer
PERFORMANCE CHARACTERISTICS CHANGE OVER TIME
50HS2 Solutions confidential and proprietary.
“A N O N Y M O U S
*Sigh* This application sucks!
51HS2 Solutions confidential and proprietary.
• Now that you have index-able data, you need optimal indexes• Indexes speed up queries
• Always on the Primary Key
• Almost always on Foreign Keys
• As needed on temporal columns
• You can’t index every column• Indexes slow down inserts and affected updates
• An indexes value is proportional to its selectivity
MORE ON THAT OPTIMAL APPROACH
52HS2 Solutions confidential and proprietary.
• start_date, end_date? – the intuitive choice!
• end_date, start_date?
• How much history will you keep around?• A whole-heck-of-a-lot? Then end_date, start_date
• Only Recent? Then start_date, end_date
SELECTIVITY
53HS2 Solutions confidential and proprietary.
PERFORMANCE AS A FACTOR OF CACHE
- 5,000,000 10,000,000 -
5,000 10,000 15,000 20,000 25,000 30,000 35,000
CACHED INDEX RANGE SCANS
Full ScanStart-End CachedStart-Null End CachedEnd-Start CachedNull End-Start Cached
The combination of start date and null end date is off the chart!
54HS2 Solutions confidential and proprietary.
PERFORMANCE AS A FACTOR OF CACHE
1,000,000 5,500,000 10,000,000 -
50
100
150
200
250
300 CACHED INDEX RANGE SCANS
Full ScanStart-End CachedStart-Null End CachedEnd-Start CachedNull End-Start Cached
The combination of end date and start date performs the best!
55HS2 Solutions confidential and proprietary.
NULL END DATE, START DATE
-------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | -------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 16 | 10104 (1)| 00:00:01 | | 1 | SORT AGGREGATE | | 1 | 16 | | | | 2 | CONCATENATION | | | | | | |* 3 | INDEX RANGE SCAN| T10_000_000_NIA | 487K| 7621K| 3607 (1)| 00:00:01 | |* 4 | INDEX RANGE SCAN| T10_000_000_NIA | 849K| 12M| 6497 (1)| 00:00:01 | --------------------------------------------------------------------------------------
END-OF-TIME END DATE, START DATE
------------------------------------------------------------------------------------ | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ------------------------------------------------------------------------------------ | 0 | SELECT STATEMENT | | 1 | 16 | 5224 (1)| 00:00:01 | | 1 | SORT AGGREGATE | | 1 | 16 | | | |* 2 | INDEX RANGE SCAN| T10_000_000_IA | 1413K| 21M| 5224 (1)| 00:00:01 | ------------------------------------------------------------------------------------
WHAT’S HAPPENING HERE (AT TEN MILLION ROWS)?
end-of-time is ½ the cost
56HS2 Solutions confidential and proprietary.
PERFORMANCE AS A FACTOR OF CACHE
- 200 400 600 800 1,000 - 1 2 3 4 5 6 7
CACHED INDEX RANGE SCANS
Full ScanStart-End CachedStart-Null End CachedEnd-Start CachedNull End-Start Cached
57HS2 Solutions confidential and proprietary.
NULL END DATE, START DATE
--------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 16 | 4 (0)| 00:00:01 | | 1 | SORT AGGREGATE | | 1 | 16 | | | | 2 | CONCATENATION | | | | | | |* 3 | INDEX RANGE SCAN| T1_000_NIA | 45 | 720 | 2 (0)| 00:00:01 | |* 4 | INDEX RANGE SCAN| T1_000_NIA | 85 | 1360 | 2 (0)| 00:00:01 | ---------------------------------------------------------------------------------
END-OF-TIME END DATE, START DATE
------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 16 | 2 (0)| 00:00:01 | | 1 | SORT AGGREGATE | | 1 | 16 | | | |* 2 | INDEX RANGE SCAN| T1_000_IA | 134 | 2144 | 2 (0)| 00:00:01 | -------------------------------------------------------------------------------
WHAT’S HAPPENING AT ONE THOUSAND ROWS?
end-of-time is still ½ the cost
58HS2 Solutions confidential and proprietary.
PERFORMANCE AS A FACTOR OF CACHE
- 10 20 30 40 50 60 70 80 90 100 - 1 2 3 4 5 6 7
CACHED INDEX RANGE SCANS
Full ScanStart-End CachedStart-Null End CachedEnd-Start CachedNull End-Start Cached
59HS2 Solutions confidential and proprietary.
NULL END DATE, START DATE
----------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ----------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 16 | 1 (0)| 00:00:01 | | 1 | SORT AGGREGATE | | 1 | 16 | | | |* 2 | INDEX SKIP SCAN| T100_NIA | 11 | 176 | 1 (0)| 00:00:01 | -----------------------------------------------------------------------------
END-OF-TIME END DATE, START DATE
----------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ----------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 16 | 1 (0)| 00:00:01 | | 1 | SORT AGGREGATE | | 1 | 16 | | | |* 2 | INDEX RANGE SCAN| T100_IA | 11 | 176 | 1 (0)| 00:00:01 | -----------------------------------------------------------------------------
WHAT’S HAPPENING AT ONE HUNDRED ROWS?
end-of-time is the same cost
60HS2 Solutions confidential and proprietary.
PERFORMANCE AS A FACTOR OF CACHE
- 1 2 3 4 5 6 7 8 9 10 -
2
CACHED INDEX RANGE SCANS
Full ScanStart-End CachedStart-Null End CachedEnd-Start CachedNull End-Start Cached
61HS2 Solutions confidential and proprietary.
USING COST INSTEAD OF ELAPSED TIME
62HS2 Solutions confidential and proprietary.
A LOOK AT EXPLAIN PLAN COSTS
- 5,000,000 10,000,000 -
10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000
CACHED INDEX RANGE SCANS
Full ScanStart-Null EndEnd-Start
63HS2 Solutions confidential and proprietary.
A LOOK BACK AT ELAPSED TIMES
1,000,000 5,500,000 10,000,000 -
50
100
150
200
250
300 CACHED INDEX RANGE SCANS
Full ScanStart-EndStart-Null EndEnd-StartNull End-Start
The combination of end date and start date performs the best!
64HS2 Solutions confidential and proprietary.
START DATE, NULL END DATE
------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 16 | 78992 (1)| 00:00:04 | | 1 | SORT AGGREGATE | | 1 | 16 | | | |* 2 | INDEX RANGE SCAN| T10_000_000_ANI | 1337K| 20M| 78992 (1)| 00:00:04 | -------------------------------------------------------------------------------------
END-OF-TIME END DATE, START DATE
------------------------------------------------------------------------------------ | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ------------------------------------------------------------------------------------ | 0 | SELECT STATEMENT | | 1 | 16 | 5224 (1)| 00:00:01 | | 1 | SORT AGGREGATE | | 1 | 16 | | | |* 2 | INDEX RANGE SCAN| T10_000_000_IA | 1413K| 21M| 5224 (1)| 00:00:01 | ------------------------------------------------------------------------------------
WHAT’S HAPPENING HERE (AT TEN MILLION ROWS)?
end-of-time is 15 TIMES less costly!
65HS2 Solutions confidential and proprietary.
START DATE, NULL END DATE
-------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | -------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 16 | 10 (0)| 00:00:01 | | 1 | SORT AGGREGATE | | 1 | 16 | | | |* 2 | INDEX RANGE SCAN| T1_000_ANI | 130 | 2080 | 10 (0)| 00:00:01 | --------------------------------------------------------------------------------
END-OF-TIME END DATE, START DATE
------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 16 | 2 (0)| 00:00:01 | | 1 | SORT AGGREGATE | | 1 | 16 | | | |* 2 | INDEX RANGE SCAN| T1_000_IA | 134 | 2144 | 2 (0)| 00:00:01 | -------------------------------------------------------------------------------
WHAT’S HAPPENING AT ONE THOUSAND ROWS?
end-of-time is 5 TIMES less costly
66HS2 Solutions confidential and proprietary.
START DATE, NULL END DATE
------------------------------------------------------------------------------ | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ------------------------------------------------------------------------------ | 0 | SELECT STATEMENT | | 1 | 16 | 1 (0)| 00:00:01 | | 1 | SORT AGGREGATE | | 1 | 16 | | | |* 2 | INDEX RANGE SCAN| T100_ANI | 11 | 176 | 1 (0)| 00:00:01 | ------------------------------------------------------------------------------
END-OF-TIME END DATE, START DATE
----------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ----------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 16 | 1 (0)| 00:00:01 | | 1 | SORT AGGREGATE | | 1 | 16 | | | |* 2 | INDEX RANGE SCAN| T100_IA | 11 | 176 | 1 (0)| 00:00:01 | -----------------------------------------------------------------------------
WHAT’S HAPPENING AT ONE HUNDRED ROWS?
end-of-time is the same cost
67HS2 Solutions confidential and proprietary.
• Make end dates not null
• Define an end-of-time value to use everywhere
• Use the end-of-time value when you don’t know the end date
• Index in end date, start date order
• Eliminate full table scans
• Performance tests under load
• Performance tests with “real-life” data and table sizes
• Retest periodically
CONCLUSIONS
68HS2 Solutions confidential and proprietary.
• Move non-transactional data to another database
• Brush your teeth twice a day, and don’t forget to floss
• Listen to your spouse, I didn’t say obey did I?
OTHER GOOD HABITS
69HS2 Solutions confidential and proprietary.
The use of null values in a column that is queried often is a common performance problem that Oracle addressed years ago by allowing the creation of functional indexes. Using a functional index, one can work-around the null values by having the database calculate a replacement value using the nvl() function and storing the calculated value in the index instead. This means that every query against the table must use the same replacement value syntax if it is going to take advantage of the functional index. One caveat, you must use HINTS.
AN ORACLE-ENABLED WORKAROUND
70HS2 Solutions confidential and proprietary.
• You can use hints /*+ INDEX() */
• You can pin tables in cache (don’t do this with large tables)
ANOTHER ORACLE ENABLED WORKAROUNDS
71HS2 Solutions confidential and proprietary.
NULL VALUES ARE THE ZERO OF THE 21ST CENTURY
Known Unknown
Unknowable
NULL Values