Divide and-conquer approach towards data analytics testing
16
Kokila Rudresh Devangana Khokhar Divide-and-Conquer Testing in Data Analytics Domain VodQA 2015
-
Upload
kokilarudresh -
Category
Software
-
view
140 -
download
1
Transcript of Divide and-conquer approach towards data analytics testing
Kokila Rudresh
Devangana Khokhar
Divide-and-Conquer Testing in Data Analytics Domain
Vo d Q A 2 0 1 5
Data Analytics: An Introduction
Collection
Processing Modelling Inference Visualization
Data Analytics: Use Cases
Business Intelligence
Social NetworksAstronomy and
Astrophysics
Robotics and Artificial Intelligence Life Sciences
Finance and Stock Market
Medical Imaging
Computer Graphics
Computer Vision
Energy Exploration
Data Analytics: Why Testing is Important
Volume
DomainComplexity
Variety
Computations
Testing
Thou shalt not leave the application untested!
Data Analytics: Testing Challenges
Data Validation
Model Implementation
Business Perspective
Data Analytics: Typical System Implementation
Extract
Transform
Load
Source Data
Simulation AggregationETL VisualizationRaw Data
Format
Consistency
Completeness
Divide-and-Conquer Testing
ExtractTransform
LoadSource Data
Pre-ETL Validations
Divide-and-Conquer Testing
ExtractTransform
LoadSource Data
Post-ETL Tests
Meta-data
Data transformation
Data quality checks
Business-specific validations
Divide-and-Conquer Testing
ExtractTransform
LoadSource Data
Simulation Validations
Model Validation
Implementation
Computation
Divide-and-Conquer Testing
ExtractTransform
LoadSource Data
Aggregation Validations
Data Hierarchy
Data Scope
Summarized Values
Divide-and-Conquer Testing
ExtractTransform
LoadSource Data
UI Validations
Information Representation
Data Format
Result Intuitiveness
Learnings
ANALYSE
CODETEST
Initial Data Flow• Pre defined data
template• Pre-ETL data validations
Domain Knowledge• KT Sessions involving SME’s• Core computations
Business Involvement• Test data closer to real
time data• User flows prioritization
Learnings
Implementation
• Alternate implementation• SME validation
Computation
• Addressing the right problem
• Computational Factors
ANALYSE
CODETEST
Learnings
Testing Process• Step wise data
validation• Defect investigation
Test Automation• Data combinations• Xml test data
Test Execution• CI test execution• Execution frequency
Test Data• Data distribution• Edge case data
Testing Tools• Spreadsheet gear• Excel macros
ANALYSE
CODETEST
Domain Context
Integrating Business
Use-cases
Design and Testing
Challenges
Testing Approach Learnings
Summary
