Unlocking the full potential of data storage technologies
-
Upload
brent-maxwell -
Category
Technology
-
view
180 -
download
1
Transcript of Unlocking the full potential of data storage technologies
1
Data Storage Technologies:
A Wealth of Potential!
Brent MaxwellHead of Data and Intelligence
The Iconic
2
Key topics
• Examining when and how to apply new data warehousing technologies
• Creating fast and convenient access to all of your data for advanced analytics
• Taking advantage of high performance data warehousing technologies to personalise the customer experience
3
THE ICONIC
• Founded in 2011• Leading the Australian online fashion market• Humongous growth in sales and the hearts of
Australians everywhere• 70% YOY data growth• Gone through 2 data warehouses• Gone through 3 BI tools
4
Data warehousing: context
What is notable about our architecture?Hint: The Operational Data Store (ODS) is in the
wrong place!!!…or is it?
5
What we’re doing that’s awesome
• Near-real-time data• Ultra-low maintenance• Self-documenting data warehouse
6
What does it mean for the business?
• Customer Service have rich data• Web/App Catalogue Sorting adapts to
events• Further Personalisation
7
The cloud is the way forward
Until recently it hasn’t been practical. What’s changed?• Source systems are in the cloud• Feature sets have been refined• The technologies have stabilized
It’s a good time to move your DWH to the cloud!
8
How did we do it?
9
How did we do it?
• Near-real-time processing where desired
• Test-driven database development• Dynamic schema creation• Hybrid Slowly Changing Dimensions• Re-runnable data loads• Cloud-based logging
10
What next?!?
• Continuous Integration• Just need to build the artefact!
• Improved Job Orchestration• Control-M• Rundeck• Any recommendations?