Working with strongly typed data models in Lua for ... · – members may be assigned nil (e.g....
Transcript of Working with strongly typed data models in Lua for ... · – members may be assigned nil (e.g....
WorkingwithstronglytypeddatamodelsinLuaforbuildingIndustrialInternetofThings(IIoT)applica=ons
TheConnec=vityPlaAormfortheIndustrialInternetofThings™
RajiveJoshi,Ph.D.PrincipalSolu7onArchitect,Real-TimeInnova7ons,Inc.
LuaWorkshop2016SanFrancisco,CA
2016Oct14
Agenda
• Background• Problem• Solu=on• Tutorial|Demo• Applica=ons• NextSteps
©2015Real-TimeInnova=ons,Inc.
Background
Whatwedo?
TheIndustrialInternetofThingsIndustrialInternetofThings(IIoT)ConsumerInternetofThings(CIoT)
Cyber-PhysicalSystems(CPS)
TheIndustrialInternetofThingsIndustrialInternetofThings(IIoT)ConsumerInternetofThings(CIoT)
Cyber-PhysicalSystems(CPS)
RTIExcelsatIndustrialIoT(IIoT)• ~1000Projects
– Healthcare– Transporta=on– Communica=ons– Energy– Industrial– Defense
• 15+Standards&Consor=aEfforts– Interoperability– Mul=-vendorecosystems
ConnectAnything,Anywhere-Intelligently!
• Proximity• PlaAorm• Language
• Physicalnetwork• Transportprotocol• Networktopology
RTIConnextDDS“DataBus”
Seamlessdatasharingregardlessof:
ü Reliableü Real-Timeü Scalableü Availableü Resilientü Secureü Safeü Composable
RTIConnextPlaAormisbuiltontheDDSStandard
©2015Real-TimeInnova=ons,Inc.
hdp://portals.omg.org/dds/
ProblemDefini=on
“Datadominates.Ifyou'vechosentherightdatastructuresandorganizedthingswell,thealgorithmswillalmostalwaysbeself-evident.Datastructures,notalgorithms,arecentraltoprogramming.(SeeBrooksp.102.)”
- RobertPike
Data-CentricSystems:SharedDataSpace
©2015Real-TimeInnova=ons,Inc.
ComponentX1
Data-Objects(Topic) Component
Y1DataWriter
write()
DataReader
take()read()
Data-CentricSystems:SharedDataSpace
©2015Real-TimeInnova=ons,Inc.
ComponentX1
Data-Objects(Topic) Component
Y1
ComponentXN
ComponentYM
DataWriter
write()
DataReader
take()read()
Data-CentricSystems:SharedDataSpace
©2015Real-TimeInnova=ons,Inc.
ComponentX1
Data-Objects(Topic) Component
Y1
ComponentXN
ComponentYM
DataWriter
write()
DataReader
take()read()
C/C++ Java
.Net/C# Lua
Pickthelanguagemostsuitabletotherequirementsofacomponent
Mul=pleplaAorms,Mul=pleLanguages
Data-CentricSystems:DataModels
©2015Real-TimeInnova=ons,Inc.
DataModel(Datatype)
structShapeType{longx;longy;doubleshapesize;string<128>color;//@Key};
ComponentX1
ComponentY1
ComponentXN
ComponentYM
C/C++ Java
.Net/C# Lua
Data-CentricSystems:DataModels
©2015Real-TimeInnova=ons,Inc.
DataModel(Datatype)
structShapeType{longx;longy;doubleshapesize;string<128>color;//@Key};
ComponentX1
ComponentY1
ComponentXN
ComponentYM
C/C++ Java
.Net/C# Lua
InterfaceDefini7onLanguage(IDL)XMLwithEquivalentSchema
DDSX-TypesSpecifica=on
Data-CentricSystems:DataModels
©2015Real-TimeInnova=ons,Inc.
DataModel(Datatype)
structShapeType{longx;longy;doubleshapesize;string<128>color;//@Key};
ComponentX1
ComponentY1
ComponentXN
ComponentYM
C/C++ Java
.Net/C# Lua
struct { : }
public class{ : }
public class { : }
InterfaceDefini7onLanguage(IDL)XMLwithEquivalentSchema
DDSX-TypesSpecifica=on
?
DataModelConstrainstheData-Objects
©2015Real-TimeInnova=ons,Inc.
DataModel(Datatype)
structShapeType{longx;longy;doubleshapesize;string<128>color;//@Key};
Data-Objects(Topic)
Constrains
LanguageNeutral LanguageSpecificSta7c:C/C++,Java..NET/C#,Ada.…Dynamic:Lua,Python,Javascript,…
Compilerenforcesinasta5callytypedlanguage
DataModelConstrainstheData-Objects
©2015Real-TimeInnova=ons,Inc.
DataModel(Datatype)
structShapeType{longx;longy;doubleshapesize;string<128>color;//@Key};
Data-Objects(Topic)
Constrains
LanguageNeutral LanguageSpecificSta7c:C/C++,Java..NET/C#,Ada.…Dynamic:Lua,Python,Javascript,…
Howtoenforceindynamicallytypedlanguages?HowtoenforceinLua?
Solu=on
DDSL:DataDomainSpecificLanguage
DDSL:DataDomainSpecificLanguageAnalgebrafordefining,using,andmanipula5ngX-Types(inLua)
©2015Real-TimeInnova=ons,Inc.
DataModel(Datatype)
structShapeType{longx;longy;doubleshapesize;string<128>color;//@Key};
Data-Objects(Topic)
template(Lua)
instances(Lua)
DDSL
EnforcesConstraintsKeepsSynchronized
Create&Modify• Datatypes• InstancesIndexdatafields
Datamodelscanbeverycomplex….Primi7ve
• Atomic(long,double,string,etc…)
• Const• Enums
Structural• Structs• Unions• Sequences(mul=dimensional)• Arrays(mul=dimensional)
©2015Real-TimeInnova=ons,Inc.
Organiza7onal• Typedefs• Module• Annota=ons
WithoutDDSL…
• Thedatatypemustbemaintainedoutsidethescrip=nglanguageenvironment– notaccessibletoprogram(inthelanguage)
• Thedatatypemustbeknowntotheprogrammer,anditsstructurehard-codedinthescript– error-proneduetoduplica=on– bridletodatatypeevolu=onorlarge/complextypes
• Ascriptcannotuseorintroducenewtopicsorendpoints(notknowntotheprogrammer)– impossibletobuildgenericcomponents
©2015Real-TimeInnova=ons,Inc.
WithDDSL…
• Thedatatypecanbemaintainedinsidethescrip=ngenvironment– fullyaccessibletoprogram
• Thedatatypeneednotbeknowntotheprogrammer,anditsstructureneednotbehard-codedinthescript– safe,singleversionoftruth(nodatatypeduplica=on)– robusttodatatypeevolu=onorlarge/complextypes
• Ascriptcanuseorintroducenewtopicsorendpoints(notknowntotheprogrammer)– possibletobuildgenericcomponents
©2015Real-TimeInnova=ons,Inc.
QuickExample…
©2015Real-TimeInnova=ons,Inc.
DataModel-IDLstructShapeType{longx;longy;doubleshapesize;string<128>color;//@Key};
Data-Object-Luashape={x=50,y=30,shapesize=20,color=‘GREEN’}
QuickExample…
DataModel–DDSL(template)
localShapeType=xtypes.struct{ShapeType={{x={xtypes.long}},{y={xtypes.long}},{shapesize={xtypes.long}},{color={xtypes.string(128),xtypes.Key}},}}
Data-Object–DDSL(instance)
localshape=xtypes.new_instance(ShapeType)shape.x=50shape.y=30shape.shapesize=20shape.color='GREEN'
©2015Real-TimeInnova=ons,Inc.
DataModel-IDLstructShapeType{longx;longy;doubleshapesize;string<128>color;//@Key};
Data-Object-Luashape={x=50,y=30,shapesize=20,color=‘GREEN’}
DDSLopensupnewpossibili=es...• ADDSLdatatype
– Canbeexaminedandtraversedintheprogram– Canbedefinedandmanipulatedbytheprogram• create|read|update|modifyopera=ons
– Canbesynthesizedandmanipulatedbytheprogram– Canbeusedtoautomatecodeordatagenera=on
whileusingthefullpowerandflexibilityofLua• BringsdatatypestoeveryIoTplaAorm
– DDSLisplaAormindependent• iswrideninpureLuawithnoexternaldependencies(notevenDDS!)• canbeusedonanyplaAormonwhichLuacanbeused(prac=callyeverywhere)
– Evenweb-browsers(e.g.usinglua.vm.js)©2015Real-TimeInnova=ons,Inc.
DDSL
Analgebrafordefining,using,andmanipula=ngX-Types(inLua)
So,whatexactlyisDDSL?
1. ReplacementforIDL,byallowingdatatypesdefini=onsinLua2. Factoryofdata-objects(instances)thatadheretoadatatype3. Analgebratooperateondatatypesandmaintainthe
associateddata-objects(instances)4. Generatorofaccessorstringsforindexingmembersinadata-
objectstore5. Anextensibleframeworkfordatamodeling
©2015Real-TimeInnova=ons,Inc.
1.IDLReplacementinLua
• OMGDDSX-TypesSpecifica=oninLua– includingfeaturesnotyetsupportedinIDL• e.g.customannota=ons
• XML→DDSL– ImportdatatypesdefinedinXML
• DDSL→IDL(→XML)– ExportdatatypestoIDL
• Easytodefinenewkindsofdatatypes– Usefulforexperimen=ngwithnewX-Typesfeatures• e.g.choice
1.IDLReplacementinLua
localMAX_COLOR_LEN=xtypes.const{MAX_COLOR_LEN={xtypes.long,128}}localShapeType=xtypes.struct{ShapeType={{x={xtypes.long}},{y={xtypes.long}},{shapesize={xtypes.long}},{color={xtypes.string(MAX_COLOR_LEN),xtypes.Key}},}}
©2015Real-TimeInnova=ons,Inc.
localMAX_COLOR_LEN=xtypes.const{MAX_COLOR_LEN={xtypes.long,128}}localShapeType=xtypes.struct{ShapeType=xtypes.EMPTY}ShapeType[1]={x={xtypes.long}}ShapeType[2]={y={xtypes.long}}ShapeType[3]={shapesize={xtypes.long}}ShapeType[4]={color={xtypes.string(MAX_COLOR_LEN),xtypes.Key}}
Declara=vestyle(DatatypeConstructor)
Impera=veStyle(DatatypeCRUD)
2.Factoryofinstances
• Instancesareconstrainedbythedatatype– arbitrarymemberscannotbeaddedtoinstances– membersmaybeassignednil(e.g.op=onal);andlaterre-assignedanon-nilvalue– collec=on(arrayandsequence)boundsareenforced;nowallowedtoexceedthemaxcapacity
• Yet,instanceperformanceisonparwithamanuallydefinedinstance– nooverheadinsewng/gewngmembervalues
• itwasanexplicitdesigngoaltonotaddanyoverheadoverrawmemberfieldseders/geders
– onlystructuralconstraintsareenforced!• type-checkingisnotenforcedformembervalueassignment(althoughanapplica=oncouldeasilydosoifdesired)
©2015Real-TimeInnova=ons,Inc.
2.Factoryofinstances--shapeisequivalenttomanuallydefiningthefollowing--localshape_manual={x=50,y=30,shapesize=20,color='GREEN',}
©2015Real-TimeInnova=ons,Inc.
localshape=xtypes.new_instance(ShapeType)print("---Iteratethroughinstancemembers:unordered---")forrole,_inpairs(shape_manual)doshape[role]=shape_manual[role]print('\t',role,shape[role])end
--- Iterate through instance members : unordered --- color GREEN x 50 shapesize 20 y 30
⇒
3.DatatypeAlgebra
• Datatypescanbemutated(unlikeIDL/XML)– the“kind”(meta-datatype)isimmutableanddefinesthestructureandconstraints– allotheraspectsofadatatypecanbechanged,e.g.:name,members,annota=ons,inheritance,switch
• Datatypechangesarepropagatedtoinstances(unlikeIDL/XML)– addingamembertothedatatypealsoaddsittoalltheinstances(withadefaultvalue)– removingamemberfromathedatatypealsoremovesitfromalltheinstances– changingamember'sdatatypepropagatesthechangetoalltheinstances
©2015Real-TimeInnova=ons,Inc.
3.DatatypeAlgebra
ModelsandInstancesADDSLinstancehastwofaces(likeacoin):• anunderlyingdatatypemodel(theblueprint)• fieldswithrolenamesdictatedbythedatatype
Constructorsreturnaspecial“template”instance(e.g.MyType)asahandletothe
underlyingmodel.
Datatypemodelsaremanipulatedusingoperatorsoninstances.Anyoftheinstancescouldbeused,butitsidioma=ctousethetemplateinstance.Templateinstanceslinkmodelelementstoformadatamodelgraph.
©2015Real-TimeInnova=ons,Inc.
MyTypemember_role_1 str|tbl
:
member_role_M str|tbl
MyTypeModelOperatorsi-thmember MyType[i]={role={…}}
#members #MyType
name MyType[NAME]
namespace MyType[NS]
kind MyType[KIND]
() capacity|constvalue|alias
qualifiers|base|switch
MyType[QUALIFIER|BASE|SWITCH]
new_instance()myInstance
member_role_1 value|tbl
:
member_role_M value|tbl
metatable
template
3.DatatypeAlgebra
ModelsandInstancesADDSLinstancehastwofaces(likeacoin):• anunderlyingdatatypemodel(theblueprint)• fieldswithrolenamesdictatedbythedatatype
Constructorsreturnaspecial“template”instance(e.g.MyType)asahandletothe
underlyingmodel.
Datatypemodelsaremanipulatedusingoperatorsoninstances.Anyoftheinstancescouldbeused,butitsidioma=ctousethetemplateinstance.Templateinstanceslinkmodelelementstoformadatamodelgraph.
©2015Real-TimeInnova=ons,Inc.
MyTypemember_role_1 str|tbl
:
member_role_M str|tbl
MyTypeModelOperatorsi-thmember MyType[i]={role={…}}
#members #MyType
name MyType[NAME]
namespace MyType[NS]
kind MyType[KIND]
() capacity|constvalue|alias
qualifiers|base|switch
MyType[QUALIFIER|BASE|SWITCH]
new_instance()myInstance
member_role_1 value|tbl
:
member_role_M value|tbl
metatable
metatable
template
3.DatatypeAlgebra:ModelOperatorsprint('---addmemberz---')ShapeType[#ShapeType+1]={z={xtypes.string(),xtypes.Key}}print_datatype(ShapeType,'ShapeType:')print('---removememberx---')ShapeType[1]=nilprint_datatype(ShapeType,'ShapeType:')print('---redefinemembery---')ShapeType[1]={y={xtypes.double}}print_datatype(ShapeType,'ShapeType:')print('---addabasestruct---')localProperty=xtypes.struct{Property={{name={xtypes.string(MAX_COLOR_LEN)}},{value={xtypes.string(MAX_COLOR_LEN)}},}}ShapeType[xtypes.BASE]=Propertyprint_datatype(ShapeType,'PropertyShapeType:')print_instance(ShapeType,'PropertyShapeTypeaccessors:')
©2015Real-TimeInnova=ons,Inc.
4.GeneratorofAccessorStrings• Datatypeisaspecial‘template’instance
– whosemembersareini=alizedtotheaccessorstringsforaDDSDynamicDataobject– automa=callymaintainedtoconfirmwiththeunderlyingdatamodel– forefficiency,collec=onmemberelementsareallocated(atmostonce)ondemand(i.e.whenaccessed)
©2015Real-TimeInnova=ons,Inc.
--ShapeTypecanbeaccessedasifitwasdefinedas--ShapeType={x=‘x’,y=‘y’,shapesize=‘shapesize’,color=‘color’}
4.Anextensibleframeworkfordatamodeling
• Coreenginesupportsagenericmeta-model– Agnos=ctospecificdatamodelinglanguageconstrains– X-TypesmoduleextendstheddslenginetosupportDDSX-Types(OMGIDL)
• Easytoextendthemodelinglanguage– Newatomicdatatypes,annota=ons,composi=ons,…
• Easytoaddnewconstraints– Dataranges– Dataunits– Seman=cs
©2015Real-TimeInnova=ons,Inc.
DatatypeAlgebra
DDSLNomenclature
©2015Real-TimeInnova=ons,Inc.
MyTypemember_role_1 str|tbl
:
member_role_M str|tbl
MyTypeAPIi-thmember MyType[i]={role={…}}
#members #MyType
name MyType[NAME]
namespace MyType[NS]
kind MyType[KIND]
() capacity|constvalue|alias
qualifiers|base|switch
MyType[QUALIFIER|BASE|SWITCH]
new_instance()myInstance
member_role_1 value|tbl
:
member_role_M value|tbl
metatable
metatable
template
NomenclatureGeneralConcept DDSLTerminology
instance(a.k.a.data-object) instance
datatype template
member
membername role
memberdefini=on roledefini=on(roledef)
annota=on qualifier
DatatypeAlgebra-ModelConstructors
©2015Real-TimeInnova=ons,Inc.
Createadatatypetemplate:
MyType = xtypes.<kind>{...}
where,<kind> is one of: annotation,
const, enum, struct, union, module, typedef
and{...} dependson<kind> e.g. MyType = xtypes.struct{MyType=xtypes.EMPTY} Commondefini=onstyle:
{ <name> = { <definition> } }
DatatypeAlgebra-ModelOperators
©2015Real-TimeInnova=ons,Inc.
Createamember: MyType[i] = { role = {template, [array|sequence,] [[annotation,]...] } Deleteamember: MyType[i] = nil Namespace,Qualifiers,Base(structs),Switch(unions): MyType[xtypes.NS] = AnotherType MyType[xtypes.QUALIFIERS] = { annotation, ... } MyType[xtypes.BASE] = BaseType MyType[xtypes.SWITCH] = DiscriminatorType Name: MyType[xtypes.NAME] = ‘NewTypeName’ tostring(MyType) Kind(immutable): MyType[xtypes.KIND]
DatatypeAlgebra-ModelIterators
©2015Real-TimeInnova=ons,Inc.
structsIterateoverthemembers:
for i = 1, #MyType do local role, roledef = next(MyType[i]) print(role, table.unpack(roledef)) end
unionsIterateoverthemembers:
for i = 1, #MyType do local case = MyType[i] -- case discriminators: print(table.unpack(case)) -- member: local role, roledef = next(case, #case) print(role, table.unpack(roledef)) end
DatatypeAlgebra-InstanceConstructors
©2015Real-TimeInnova=ons,Inc.
Createaninstance: myInstance = xtypes.new_instance(MyType)
Createacollec=onofinstances: myCollection = xtypes.new_collection(MyType,
capacity) NOTE:Asequenceorarraymemberinaninstanceisacollec=on
DatatypeAlgebra-InstanceCollec7ons
©2015Real-TimeInnova=ons,Inc.
Isitacollec=on?if xtypes.is_collection(myCollection)then
print( ‘capacity’, myCollection() ) print( ‘length’, #myCollection )
end Iterateoveracollec=on’slength: for i = 1, #myCollection do print( myCollection[i] ) end Iterateoveracollec=on’scapacity: for i = 1, myCollection() do print( myCollection[i] ) end
DatatypeAlgebra-InstanceIterators
©2015Real-TimeInnova=ons,Inc.
Iterateoverinstancemembers(unordered): for role, value in pairs(myInstance) do print(role, value) end Iterateoverinstancemembers(ordered): for i = 1, #MyType do local element_i, role = MyType[i] role = next(element_i) -- struct
role = next(element_i, #element_i)-- union local value = myInstance[role] print(role, value, table.unpack(MyType(role))) end
DatatypeAlgebra–TemplateInstance
©2015Real-TimeInnova=ons,Inc.
IterateoverDDSDynamicDataaccessorstrings(unordered): for role, accessor in pairs(MyType) do print(role, accessor) end IterateoverDDSDynamicDataaccessorstrings(ordered): for i = 1, #MyType do local element_i, role = MyType[i] role = next(element_i) -- struct
role = next(element_i, #element_i)-- union local accessor = MyType[role] print(role, accessor) end
Fieldvaluesareaccessorstringsforindexingmembersinadata-objectstore
DatatypeAlgebra–MiscOperators
©2015Real-TimeInnova=ons,Inc.
Getthefullyqualifiednamewithinthenamespaces print( xtypes.nsname(MyType) ) Resolveanalias(e.g.typedef)toitsunderlyingdatatype print( xtypes.resolve(MyType) ) Getaninstance’sunderlyingdatatypetemplate print( xtypes.template(myInstance) )
DatatypeAlgebra–U7li7es
• ddsl.xtypes.u=ls– to_instance_string_table(instance)– to_idl_string_table(instance)– nslookup(name,ns)
• ddsl.xtypes.xml– filelist2xtypes(filename_array)– file2xtypes(filename)– string2xtypes(xmlstring)
• Commandline– runxml2ddsl[-d]<xml-file>[<xml-files>...]
©2015Real-TimeInnova=ons,Inc.
Tutorial
Runningthetutorial… git clone github.com/rticommunity/rticonnextdds-ddsl/
export DDSL_HOME=$(pwd -P)
export LUA_PATH+="${DDSL_HOME}tutorial/?.lua;”
cd tutorial/
lua ddsl_tutorial.lua
Applica=ons
©2015Real-TimeInnova=ons,Inc.
CurrentApplica=ons(thatIamawareof)
• DataEmulator– RTIExperimentalProduct• Emulatedata-centricinterfaces• Firstdroptocustomer(2015Jun30)
• DataGeneratorsforReac=veProgramming– ContribuedbySumantTambe– Usefulforsimula=onandtes=ng
Poten=alApplica=ons• Communityofopendatamodelsfor[Industrial]IoT• Dataserializa=onanddeserializa=on• Codegenera=on
– Integra=onwithtempla=nglibraries• Modelmappingandgenera=on– synthesizedatatypesfromseman7cmodels– synthesizedata-modelmappings
• Synthesisofdatamodels– machinelearning
• Experimenta=onwithNewDataModelingIdeas
©2015Real-TimeInnova=ons,Inc.
NextSteps
• GewngStartedGuide– README
• hdps://github.com/r=community/r=connextdds-ddsl
• UsersManual– API
• hdp://r=community.github.io/r=connextdds-ddsl/• UnitTests– test/(formoreadvancedexamples)• ddsl-xtypes-tester.lua
©2015Real-TimeInnova=ons,Inc.
Apache2.0
ThankYou!
rajive@r=.com|@RajiveJoshi
WeareHiring!www.r7.com/company/careers.html