AquaQ Analytics Kx Event - Datawatch Presentation
-
Upload
aquaq-analytics -
Category
Software
-
view
758 -
download
0
Transcript of AquaQ Analytics Kx Event - Datawatch Presentation
![Page 1: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/1.jpg)
Visual Data Discovery with and Datawatch
![Page 2: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/2.jpg)
![Page 3: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/3.jpg)
Jeremy Bentham
![Page 4: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/4.jpg)
• 28 Aug 2013 –
• Datawatch Completes Acquisition of Panopticon
![Page 5: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/5.jpg)
Datawatch History
• Founded in 1986, Public Since 1992 (NASDQ CM: DWCH)
• Global Operations and Support
US
EMEA: UK, Germany, France, Sweden
Asia Pac: Australia, Singapore, Hong Kong, India, Philippines
• Pioneer in Transforming All Types of Information
Structured (RDBMs, Data Warehouses)
Semi-Structured (PDF, Reports, Text …)
Unstructured (Log Files, EDI …)
• Over 40,000 customers worldwide
99 of the Fortune 100 & 487 of the Fortune 500
Large Number of SMB
Across All Verticals
![Page 6: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/6.jpg)
![Page 7: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/7.jpg)
What we do?
• Visual Data Discovery
Historically focussed on:
• Front & Mid Office
• Risk, Surveillance, Research, Sales & Trading
• For Buy & Sell Side, Regulators Exchanges & ECNs
Now Still Capital Markets plus:
• Energy & Utilities, Telco, Retail, Manufacturing, etc.
![Page 8: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/8.jpg)
Which Means?
• Reducing the time taken to understand your data.
Effectively:
• Find the Weird Stuff
![Page 9: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/9.jpg)
Using: Designer, Server & Web Client
![Page 10: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/10.jpg)
So From:
![Page 11: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/11.jpg)
To:
![Page 12: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/12.jpg)
Visual Data Display
![Page 13: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/13.jpg)
Time Series
![Page 14: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/14.jpg)
Producing
![Page 15: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/15.jpg)
Competing With
![Page 16: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/16.jpg)
How we’re Differentiated
• Assume data is never at rest
• Capital Markets Focus
• Real Time Streaming
• Time Series
• High Density Visuals
• Embed (Java & .NET SDKs)
• Java & .NET Servers
• Connectivity
![Page 17: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/17.jpg)
Kx Connectivity
![Page 18: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/18.jpg)
kx Connectivity
Synchronous: Request / Response
• Issue Q & Retrieve either: • Table, Dictionary, Vector or Value
Asynchronous Subscribe
• Subscribe to Service, Table & Symbols
• Keeping latest, or scrolling time window
![Page 19: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/19.jpg)
Request / Response Subscribekx Connectivity
![Page 20: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/20.jpg)
Kx – How to Query?
Either:
• Retrieve all into Memory
• Parameterise queries, and pull back subsets
• Dynamically query (auto-generating q selects)
Retrieve:
• Summaries & Detail
• Sampled Time series
• Down to individual Ticks
Passing through:
• Parameter Values & Vectors of Values
• Time Windows
• Zoom Bounds
![Page 21: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/21.jpg)
Problem vs. Competition
Assumed: Data in Motion
So Direct Data Access
• Implying Fast Data Access / Data Querying
So if the underlying data source is:
Slow
We appear:
Slow
![Page 22: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/22.jpg)
Solution = Caching
• If data is not time sensitive
• (e.g. Typical data warehouse)
• Populate Cache on a one-off, or scheduled basis.
• Dynamically Querying of Cache
• Approach taken by:
• Tableau, Tibco Spotfire & Qlikview
• Their In-Memory Db = Proprietary Cache
![Page 23: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/23.jpg)
Search for a Cache
We needed an in-memory cache that could:
• Load quickly
• Perform fast aggregation
• Perform fast filtering
• Work with big datasets
• Understand Time
• Small footprint
• Easy to OEM
• Windows & Linux
![Page 24: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/24.jpg)
Dataset Characteristics
• Typically Sparse Timeseries
• Sensor Data
• Sales/Revenue Transactions
• Latency Data
• Machine Data
• Market Data & Trade Data (Orders & Executions)
• Everywhere we look across verticals, data seems similar
to trades & quotes
![Page 25: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/25.jpg)
Way Forward
• Approached kx for OEM
• But our pricing ruled out usage within the Designer
Then:
• 2nd April – 32bit kx – Free for Commercial Use
![Page 26: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/26.jpg)
Next Datawatch Release – Cache Options
• Designer – 32bit kx.• View Single Workbook at a time
• Server –32bit or 64bit kx Cores• Host Multiple Workbooks
• Cache up to the memory in the machine (if using 64bit cores)
![Page 27: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/27.jpg)
Our Data Strategy
• If Fast underlying database.• Go Direct
• If Slowwwwww
• Cache into kx,
• Get the query performance that kx provides
![Page 28: AquaQ Analytics Kx Event - Datawatch Presentation](https://reader036.fdocuments.in/reader036/viewer/2022081519/559489b61a28ab0e7d8b4705/html5/thumbnails/28.jpg)
More Information
Peter Simpson
Visual Data Discovery
TEL: +44 (0) 798 464 6544