DSLsandOPEChristianColombo(workwithMarkMicallefandGordonPace)
DSLs
DSLsforTesting
CNLsforMonitoring
DSLs
DSLsforTesting
CNLsforMonitoring
DSLsfortesting• AgiledevelopmentèTestautomation
DSLsfortesting• AgiledevelopmentèTestautomation
Theneedfortestautomation
Existingtechnology:GherkinGiven I am a premium user When I place a bet on a football match And I win the bet Then I will win 10% more than the advertised odds for the match
Testautomation–2challenges• Nonskilledtesters(fortestautomation)• Timepressure
Proposedsolution:• MorestructuredDSL• Easyfornonskilledtesters• Savestime
Manytestingdomains• E-commerce(cart,stock,item,buy,sell,etc)• Games(bonus,points,bet,win,lose,etc)• GUIapps(button,progressbar,label,tap,swipe,etc)
DSLsfortesting
Well-DefinedDomains
UndefinedDomains
AndroidGUIApplications
E-CommerceApplications
GraphicalGames
DSLsfortesting
DSLsfortesting
GenerictestingDSL
ECommerceexample
StructuredDSLsvsGherkin1. Theapproachworksonceagrammarisinplace
a. (Automaticallygenerated)CodehelpersinIDEb. Consistentscripts
2. Morecumbersometoaddnewlanguagefeaturesa. Newfeaturesrequiregrammarmodificationb. Notionoftagging@manualtestslost
3. Specialisedskillsneededforgrammardefinitionandcompilation
4. Wecannot“fudge”anymore(isthisgoodorbad?)
DefinedvsUndefineddomains1. Easytoreachastablelanguagequicklywithwell-defineddomains
2. Youcanquicklygodowntotheleafnodeofthedomainhierarchy
Questions:• Whocurates/ownsthelanguage?• Whateffectwillachangeonthelanguagehaveonexistingscripts?• Howistheprocessoflanguageevolutioncontrolled?• Whomaintainscodegeneratorsandhow?
• Importanttohavemanagementonboard
DSLs
DSLsforTesting
CNLsforMonitoring
CNLsfordataprocessing• Non-technicalpeopleneedtoprocessdatabasedontheirexpertise• Theycannotprogram–evenDSLsmightstillfeeltootechnical
CNLsfordataprocessing• Non-technicalpeopleneedtoprocessdatabasedontheirexpertise• Theycannotprogram–evenDSLsmightstillfeeltootechnical
• Examples:• Businessintelligence
BusinessIntelligence• ThebusinessFacebookpage• Fastresponseexpected• “Alertmewhenacustomerhaspostedaquestiononmypageandhasnotbeenansweredinanhour”
BusinessIntelligence• ThebusinessFacebookpage• Fastresponseexpected• “Alertmewhenacustomerhaspostedaquestiononmypageandhasnotbeenansweredinanhour”
• Reviewers’pages• Badreviewsneedtobedamagecontrolled• “Alertmewhenapostonareviewpagementionsmybusinessandgetsmorethan5likes”
BusinessIntelligence• ThebusinessFacebookpage• Fastresponseexpected• “Alertmewhenacustomerhaspostedaquestiononmypageandhasnotbeenansweredinanhour”
• Reviewers’pages• Badreviewsneedtobedamagecontrolled• “Alertmewhenapostonareviewpagementionsmybusinessandgetsmorethan5likes”
• Advertisingpages• Competitorsarecontinuallyposting• “AlertmewhenacompetitorpostsonanadvertisingpageandIhavenotpostedanythingtoday”
BusinessIntelligence• ThebusinessFacebookpage• Fastresponseexpected• “Alertmewhenacustomerhaspostedaquestiononmypageandhasnotbeenansweredinanhour”
• Reviewers’pages• Badreviewsneedtobedamagecontrolled• “Alertmewhenapostonareviewpagementionsmybusinessandgetsmorethan5likes”
• Advertisingpages• Competitorsarecontinuallyposting• “AlertmewhenacompetitorpostsonanadvertisingpageandIhavenotpostedanythingtoday”
Offtheshelfsolutionsarenotso
flexible
Customsolutionsareexpensive
CNLsformonitoring• Non-technicalpeoplewanttocreatemonitorsbasedontheirexpertise• DSLsmightstillfeeltootechnical
• Examples:• Businessintelligence• Taxfraud
TaxfraudAuditor
“Findindividualswhodeclaredanaverageincomeoflessthan€3000
forany3sequentialyears”
Taxfraud1.Describearule
2.RuleInterpretation
3.Code
4.Reportfoundcases
5.Reviewfoundcases
DeveloperAuditor
Taxfraud1.Describearule
2.RuleInterpretation
3.Code
4.Reportfoundcases
5.Reviewfoundcases
DeveloperAuditor
Repeatuntilthefraudexpertissatisfied!
Taxfraud1.Describearule
2.RuleInterpretation
3.Code
4.Reportfoundcases
5.Reviewfoundcases
DeveloperAuditor
Repeatuntilthefraudexpertissatisfied!
Manyplaceswherethiscangowrong
Taxfraud1.Describearule
5.Reviewfoundcases
Auditor
Automatic
Solution?• CNLfordomainexpert
Approach:BI
CNLspecification
BIexpert
FBeventstream
Approach:BI
CNLspecification CNL-to-Monitortool
LARVAspecification Larva
BIexpert
FBeventstream
Approach:BI
CNLspecification CNL-to-Monitortool
LARVAspecification Larva
BusinessIntelligenceDashboard
BIexpert
FBeventstream
TaxfraudAuditor
CNLspecification CNL-to-Monitortool
LARVAspecification Larva
Spreadsheetwithsuspiciouscases
Taxdatastream
TaxdatastreamTaxfraudAuditor
CNLspecification CNL-to-Monitortool
LARVAspecification Larva
Spreadsheetwithsuspiciouscases
Inthelongtermmonitoringisfasterthandatabasequery
Lessonslearnt• UsersfounditquiteeasytoexpressthemselvesinCNL• Someexampleswereenoughtogetthemgoing
• GoodUIsupportmakesagreatdifference
• Taxfrauddomainwasmuchmoredifficulttocapture• Containsmorejargon• Canexpress(very)complexrules
OpenPaymentsEcosystemHorizon2020projectShaunAzzopardi,ChristianColombo,GordonJPace,andBrianVella
TravelAgency
TravelAgentCorporateCustomers
Airlines,Hotels,etc
TravelAgency
TravelAgentCorporateCustomers
Airlines,Hotels,etc
Pay-ments
Usingcorporatecreditcards
Flights
Hotel
Transport
Usingcorporatecreditcards
Flights
Hotel
Transport
Reconciliation
One-shotcards
Flights
One-shotcards
Flights
Easyreconciliation
Paymentprogrammesetupcosts
• Implementingcardprocesses• Agreementwithbank• Compliancetolegislation• Auditing• Disputeresolution• ...
Buildingapaymentapplication
TravelAgent
Buildingapaymentapplication
Paymentapplication
TravelAgent
Buildingapaymentapplication
Developer
Paymentapplication
TravelAgent
Buildingapaymentapplication
Serviceprovider(Bank)
Developer
Paymentapplication
TravelAgent
Buildingapaymentapplication
Serviceprovider(Bank)
Developer
Paymentapplication
Programme
(Application)
manager
TravelAgent
OpenPaymentsEcosystem
Serviceprovider
s
Developers
Paymentapplications
Programme
managers
Corporate
customers
OpenPaymentsEcosystem
Serviceprovider
s
Developers
Paymentapplications
Programme
managers
Corporate
customers
ExecutionEnvironment
OpenPaymentsEcosystem
Serviceprovider
s
Developers
Paymentapplications
Programme
managers
Corporate
customers
ExecutionEnvironment
DevelopmentEnvironment
Processgivesrisetoissues
• Istheapplicationlegal?
Processgivesrisetoissues
• Istheapplicationlegal?
• WhichServiceProviderwouldbewillingandabletorunit?
Processgivesrisetoissues
• Istheapplicationlegal?
• WhichServiceProviderwouldbewillingandabletorunit?
• Canapplicationviolateregulationsatruntime?
OpenPaymentsEcosystem+Compliance
Serviceprovider
s
Developers
Paymentapplications
Programme
managers
Corporate
customers
ExecutionEnvironment
Complianceengine
DevelopmentEnvironment
Compliance
1. Checkingcompliancetoregulations
Compliance
1. Checkingcompliancetoregulations
2. Matchingserviceprovidercapabilities
Compliance
1. Checkingcompliancetoregulations
2. Matchingserviceprovidercapabilities
3. Limitingriskforserviceproviders
Example
UKe-moneyregulationsstatethatfundsonfinancialinstrumentsshouldberedeemableatparvalue.
Example
UKe-moneyregulationsstatethatfundsonfinancialinstrumentsshouldberedeemableatparvalue.
1.IsthejurisdictiontheUK?
1. Compliancetoregulations2. Capabilitychecking3. Riskmitigation
Example
UKe-moneyregulationsstatethatfundsonfinancialinstrumentsshouldberedeemableatparvalue.
1.Doestheapplicationfallunderthedefinitionofe-money?
1. Compliancetoregulations2. Capabilitychecking3. Riskmitigation
Example
UKe-moneyregulationsstatethatfundsonfinancialinstrumentsshouldberedeemableatparvalue.
1.Arefundsredeemablethroughtheapplication?
1. Compliancetoregulations2. Capabilitychecking3. Riskmitigation
Example
UKe-moneyregulationsstatethatfundsonfinancialinstrumentsshouldberedeemableatparvalue.
2.Canserviceprovidersupporte-moneyapplications?
1. Compliancetoregulations2. Capabilitychecking3. Riskmitigation
Example
UKe-moneyregulationsstatethatfundsonfinancialinstrumentsshouldberedeemableatparvalue.
1.Iscorrectvaluegiventotheuser
1. Compliancetoregulations2. Capabilitychecking3. Riskmitigation
Example
UKe-moneyregulationsstatethatfundsonfinancialinstrumentsshouldberedeemableatparvalue.
3.Howmanyfundsareallowedoninstruments?
1. Compliancetoregulations2. Capabilitychecking3. Riskmitigation
OpenPaymentsEcosystem+Compliance
Serviceprovider
s
Developers
Paymentapplications
Programme
managers
Corporate
customers
ExecutionEnvironment
Complianceengine
DevelopmentEnvironment
Complianceupfront?
ComplianceChallenges
● Notallpropertiesarecheckableupfront Implication:SAnotenough
ComplianceChallenges
● Notallpropertiesarecheckableupfront
● Notallinformationisavailable–onlythemodeloftheapplication
Implication:SAnotenough
Implication:SAcanonlybedoneonmodel
ComplianceChallenges
● Notallpropertiesarecheckableupfront
● Notallinformationisavailable–onlythemodeloftheapplication
● Wecannottrusttheapplication(itisrunbythirdparty)
Implication:SAnotenough
Implication:SAcanonlybedoneonmodel
Implication:Wehavetoverifymodeladherenceatruntime
Model
Developersubmitsmodelofapplicationratherthanimplementation
Model
Developersubmitsmodelofapplicationratherthanimplementation
Staticcheckthemodel
Model-Implementation
Implementation
Runtimecheckimplementationagainst
model
Runtimeverification
Implementation
Runtimecheckimplementationagainst
model
Runtimeverifyremainingchecks,eg:limits,atparvalue,delays
Example
UKe-moneyregulationsstatethatfundsonfinancialinstrumentsshouldberedeemableatparvalue.
2.Canserviceprovidersupporte-moneyapplications?
1.IsthejurisdictiontheUK?
1.Doestheapplicationfallunderthedefinitionofe-money?
Staticcheckthemodel
1.Arefundsredeemablethroughtheapplication?
Example
UKe-moneyregulationsstatethatfundsonfinancialinstrumentsshouldberedeemableatparvalue.
1.Doestheapplicationfallunderthedefinitionofe-money?
StaticcheckthemodelRuntimecheck
implementationagainstmodel
Example
UKe-moneyregulationsstatethatfundsonfinancialinstrumentsshouldberedeemableatparvalue.
3.Howmanyfundsareallowedoninstruments?
1.Iscorrectvaluegiventotheuser
Staticcheckthemodel
Runtimeverifyremainingchecks,
eg:limits,atparvalue,delays
Whataboutthemaths?
CombiningStaticandDynamicAnalysis
CombiningStaticandDynamicAnalysis
Integratingthemodel
Integratingthemodel
Quotientoperatoronthemodel
Conclusions
• Loadsofworkahead!!
Top Related