TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!
-
Upload
ca-technologies -
Category
Technology
-
view
99 -
download
0
Transcript of TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!
![Page 1: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/1.jpg)
World®’16
TechTalk:SometimesLessisMore–VisualizationCanReduceyourTestDatawhileEnhancingQuality!JamesWalker– PrincipalSoftwareEngineer– CATechnologies
DO5T06T
DEVOPS
![Page 2: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/2.jpg)
2 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
©2016CA.Allrightsreserved.Alltrademarksreferencedhereinbelongtotheirrespectivecompanies.
Thecontentprovidedinthis CAWorld2016presentationisintendedforinformationalpurposesonlyanddoesnotformanytypeofwarranty. The informationprovidedbyaCApartnerand/orCAcustomerhasnotbeenreviewedforaccuracybyCA.
ForInformationalPurposesOnlyTermsofthisPresentation
![Page 3: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/3.jpg)
3 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Abstract
Effectivetestingrequireshighqualitytestdata,butmostorganizationsstillrelyonproductiondatawhichprovidesjust10-20%functionalcoverage.Thisdataisdrawnfrom“businessasusual”scenariosthathaveoccurredinthepast,andsorarelyprovidethenegativescenariosandoutliersneededtorigorouslytestsoftware.Thedatathatcomesthickandfastintoproductionisfurthertoolargeandcomplicatedforanyhumanmindtoevaluate,sothatprofilingormodellingtechnologyisneeded.Toensurethattheyhavethequalitydataneededfortesting,organizationsneedtobeabletoevaluatewhichattributesexistinexistingdata,aswellashowtheycombine.Onlythencantheyevaluatewhichmissingattributesareneededtoexecutethetestsneededtodeliverqualitysoftware.
ThisTechTalkwillshowhowdatavisualizationprovidesaquickandreliablemethodtomeasurethetestcoverageprovidedbyexistingtestdata,spottinganymissingorinvaliddataataglance.Presentingdataattributesanddimensionsinpictorialformallowsuserstounderstandwhatdatatheyhave,howitsattributesrelate,andwhatdataismissing.TheaccuratemodelcreatedinCA’sDataVisualizationcanfurtherthenbefedintoCAAgileRequirementsDesignerandCATestDataManager,creatingthesmallestsetofdataneededtosatisfyeverypossibletestautomatically.
JamesWalker
CATechnologiesPrincipalSoftwareEngineer,CAAgileRequirementsDesigner
![Page 4: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/4.jpg)
4 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
AboutMe
§ BSc,MRes,PhD– SwanseaUniversity,Wales
§ ResearchinDataVisualisation/BigDataproblems
§ Grid-Tools– SoftwareEngineer(2012– 2015)
§ CA– LeadSoftwareEngineerARD
![Page 5: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/5.jpg)
5 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
MOTIVATION
INTRODUCTIONTODATAVISUALIZATION
DATAVISUALIZATIONFORTESTDATA
DEMO
1
2
3
4
Agenda
CONCLUDINGTHOUGHTS5
![Page 6: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/6.jpg)
6 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
TestDataChallenges
§ Productionisoftenviewedasthetruesourceofgoodtestdata(defaulttakeacopyofproductiondata-masking)
§ Productiondatahashighvolumesandlowvariance(edgecases)
§ Betterdataindevelopmentshouldbethegoal(subjective–whatisbetter?)
Todothisyouneedtobeabletoanswerawholebunchofquestions
![Page 7: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/7.jpg)
7 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
TestDataChallenges
“WhatdatadoIhave?”
“Whatdatadon’tIhave?”“DoIhave thedataIneed
formytests?”
“WhereamIundertesting?”
“WhereamIovertesting?”
“Howeffectiveismytestdata?”
“Whatismydatacoverage?”
![Page 8: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/8.jpg)
8 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
CustomSQL
Data Views/Cubes Off-the-shelf visualisationtool
OldWorldOrder…1.
2. 3.
![Page 9: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/9.jpg)
9 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
NewWorldOrder…TestingtotheBigDataFieldTechnologicaladvancementsoverthepastdecadehaveincreasedourabilitytocollectdatatopreviouslyunimaginablevolumes
Estimatedthatpeoplewillgenerate4.3exabytes ofdataintheirlifetime(1).
Datacontainshugeamountsofvalueforgaininginsight,understanding,decisionmaking,andprediction.
Virtuallyeveryfieldofscienceandindustryistakingadvantageofanalytics(medicine,sports,weather,finances,etc).
Testingislatetotheparty – Hugeopportunitiesforbigdatatechniquestohelpustestoursoftware,understandtheresults,gaininsightintoquality,makedecisions(shouldwerelease?),andeventuallypredictresultsbeforewe’veevenranasingletestcase...
![Page 10: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/10.jpg)
10 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
IntroductiontoDataVisualization
![Page 11: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/11.jpg)
11 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
“Thepurposeofcomputingisinsight,notnumbers”
Visualization:
§ Atoolthatallowstheusertogaininsightintodata
§ Toformamentalvision,image,orpictureof(somethingnotvisibleorpresenttothesightoranabstraction);tomakevisibletothemindorimagination[OxfordEnglishDictionary,1989]
RichardW.Hamming,1962
14 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
http://www.comm-dev.org/
![Page 12: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/12.jpg)
12 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
VisualizationisVeryOld
§ Oftenanintuitivesteptomakephenomenaclearere.g.agraph
§ Classical(easy)approachesknownfrombusinessgraphics(excel,etc)
§ Onlynowinthepastdecadeisthevaluestartingtobecomeprevalent
https://utah.com/parowan-gap
![Page 13: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/13.jpg)
13 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
DataSetsAreEver-increasinginSize– AGraphicalApproachIsNecessaryBefore – Simpletabulardata(verylownumberofdataanddimensions
Now – Distributedsystemscreatingmillionsofrowsasecond
![Page 14: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/14.jpg)
14 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
VisualizationisGoodfor:
§ Exploration– Findtheunknown,unexpected– Hypothesisgeneration
§ Analysis– Confirmorrejecthypotheses– Informationdrill-down
§ Presentation– Communicate/disseminateresults
![Page 15: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/15.jpg)
15 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
https://fluidi.wordpress.com
World®’16©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD15
![Page 16: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/16.jpg)
16 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
SoWhatisDataVisualization?
§ Datavisualizationistheprocessofcreatinggraphicalabstractionsofdata
§ Usevisualisationonthedailybasis(i.e.Tubemap,weatherreport,stockmarket,webtraffic…)
§ Techniqueshaveenormousvaluetoallaspectsoftheworldwelivein– todaywefocusontesting&testdata!
![Page 17: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/17.jpg)
17 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
CATestDataVisualizer
![Page 18: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/18.jpg)
18 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
LargeRelationalDatabases
Protein Data Bank – pdb.org
![Page 19: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/19.jpg)
19 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
DataCombinations
§ Itisimpossibletoconsider“AllCombinations”ofdata
n timesn timesn timesn =verylarge
Eachspinofthelockisadataattribute40possiblepositions4inputsrequired
40x40x40x40=2,560,00040x40x40x40x40=102,400,000
40x40x40x40x40x40=4,096,000,000
![Page 20: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/20.jpg)
20 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
DataConcepts
§ Dataconcepts– onlysomecombinationsarerelevantforatestcase(testrequirements).
§ Howdotestsrelatetothedata?
§ Notallcolumnsmatter,buttheircombinedeffectdoes
§ Wecreateameta-layeroftestdataattributes
![Page 21: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/21.jpg)
21 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
DataView– FlattentheDataandPickRelevantAttributes
![Page 22: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/22.jpg)
22 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Demo
![Page 23: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/23.jpg)
23 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Conclusion– TestDataVisualizer
§ Avisualizationtool– designedtoanalyse &andassistinbuilding‘better’testdata
§ Useadvancedspotdiagramsandparallelcoordinates– Comparedataforvalidandinvalidsetsofcombinations– Identifymissingcombinationsofdata– Indentify overandunder-testing– Compareenvironmentsforcoverage(QA1,QA2)– Measuredatacoverageaccurately– Reservedataamongstteammembers
![Page 24: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/24.jpg)
24 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
TestDataVisualizer
“WhatdatadoIhave?”
“Whatdatadon’tIhave?”“DoIhave thedataIneed
formytests?”
“WhereamIundertesting?”
“WhereamIovertesting?”
“Howeffectiveismytestdata?”
“Whatismydatacoverage?”
![Page 25: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/25.jpg)
25 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
RecommendedSessions
SESSION# TITLE DATE/TIME
DO5T17SCaseStudy:Nationwide'sCATestDataManagerSuccessStory 11/17/2016at1:45PM
DO5T07TTechTalk:WhatHappenedintheBackend?ThePowerofDBCompare 11/17/2016at3:00PM
DO5X42STechVision:TestDataonDemand:DeliveringtheRightData,totheRightPlace,attheRightTime 11/17/2016at4:30PM
![Page 26: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/26.jpg)
26 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Stayconnectedatcommunities.ca.com
Thankyou.
![Page 27: TechTalk: Sometimes Less is More –Visualization Can Reduce your Test Data while Enhancing Quality!](https://reader031.fdocuments.in/reader031/viewer/2022022414/587f03401a28abc26f8b4587/html5/thumbnails/27.jpg)
27 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
DevOps– ContinuousDelivery
FormoreinformationonDevOps– ContinuousDelivery,pleasevisit:http://cainc.to/PiTFpu