Post on 14-Dec-2018
oneM2M Testing Activities
Prof. Song JaeSeungSejong University & KETIoneM2M TST WG Chair
jssong@sejong.ac.kr
WoT OpenDay (2)
Outlinen Interop & Conformance Testingn oneM2M Testing Activitiesn oneM2M Interop. Eventsn oneM2M Certification Program
WoT OpenDay (3)
n Tests (end-to-end) functionality between 2 or more products
n It shows, from the user's viewpoint, that functionality is accomplished (but not how)
n Scenario-based system testing
Interoperability testing
WoT OpenDay (4)
n Conformance testing concentrates on specific components in a system
n Conformance testing is applied over open interfaces and checks for conformance to the requirements in a base specification
n Unit testing
Conformance testing
WoT OpenDay (5)
n Both are complementary!n Product could happen to be conformant but not
interoperablen Interop testing is more appropriate when the standard
is in development phase . It helps to validate the standards
n Conformance testing is more appropriate for testing stable specifications and is for testing product and check if the standard is correctly implemented
Interoperability & Conformance testing
WoT OpenDay (6)
n Methodology: » TS-0015 : Testing Framework
n Interoperability Testing » TS-0013: Interop Testing
n Conformance Testing » Conformance on oneM2M primitives: PICS TS-0017,
TSS&TP TS-0018 and ATS TS-0019 (oneM2M TF-001 & ETSI STF 531)
» Security conformance: PICS TS-0027, TSS&TP TS-0028 and ATS TS-0029
n Definition of product profiles : TS-0025 n Developer Guides : Series of eight technical reports
oneM2M Testing Specifications
WoT OpenDay (7)
n Interoperability testing specification» R1 TS-0013 Interoperability testing à Stable and used in Interop events» R2 TS-0013 Interoperability testing à Under development with new
featuresn Release 2 Interop spec. features
» Semantic testing» Security testing» Interworking testing» Data model testing
n Two oneM2M Interop. Testing events every year
oneM2M Interoperability Testing
WoT OpenDay (8)
n Conformance Testing specifications » TS-0017, Implementation Conformance Statement (ICS)» TS-0018, Test Suite Structure & Test Purposes» TS-0019, ATS (TTCN-3)» TS-0025, Product profiles
n TTCN-3 test cases are developed for all R1 product profiles» oneM2M TF-001 and ETSI STF-531 collaborate to deliver TTCN-3 test
cases» Validation of the test cases with 2~3 implementations and 2~3 oneM2M
testing tools» Now all test cases are validated and the test cases are being used by
oneM2M CB and Testing Lab for testing» Interop events are being used to validate the test cases and tools
n Now Rel-2 & Rel-3 test cases are being developed
oneM2M Conformance Testing
WoT OpenDay (9)
TST
Features catalogue & Product profiles
• product profiles and feature catalogue– guidebook to my oneM2M product features – fills the gap btw. function specs. and test specs.
FeaturesCatalogue(TS-0031)
FunctionalArchitecture(TS-0001)
ProductProfiles(TS-0025)
glimpseofoneM2M
summarized
guidelineforproductplanning
preparetestcasesforproducts
referred
CoreProtocol(TS-0004)
TestPurposes(TS-0018)
AbstractTestSuite(TS-0019)
whichfeaturetobeimplemented?
whichfeatureneedstobetested?
WoT OpenDay (10)
Product profiles
• Rel-1 profiles define in total seven products
Profile DescriptionADN Profile 1 Constrained sensor as ADNADN Profile 2 Constrained actuator as ADNADN Profile 3 Normal sensor devicesADN Profile 4 Small originator device types of oneM2M servicesIN Profile 1 Server device type of oneM2M servicesASN Profile 1 Constrained actuator as ASNMN Profile 1 Gateway devices that support multiple different area
network technologies and connect devices
WoT OpenDay (11)
Developer guide series
n example scenarios and binding messages
Deliverable Title Examplesof
TR-0025 Applicationdeveloperguide HTTPbinding,XML/JSONserialization
TR-0034 TemperaturemonitoringexampleusingCoAPbinding
CoAPbinding,<pollingChannel>
TR-0035 DeveloperguideofDeviceManagement <mgmtObj>,OMADM,LwM2M,BBFTR-069
TR-0037 SmartFarmExampleusingMQTTBinding MQTTbinding
TR-0038 Developerguide- Implementingsecurityexample
Provisioning,SecurityAssociationEstablishment
TR-0039 Developerguide- SDTbasedimplementation SDTforhomeappliances
TR-0045 DeveloperGuide:ImplementingSemantics Semanticannotationanddiscovery
TR-0048 DeveloperGuideof3GPPInterworking (TBD)
WoT OpenDay (12)
n Testing and Test Control Notation Version 3 n Internationally standardized language developed specifically
for executable test specification» Specified by ETSI MTS Technical Committee » Is independent of a specific IUT or IUT interfaces » Is independent of a test execution environment » Standard available at portal.etsi.org via ETSI programme
n Allows unambiguous implementation of tests n Look and feel of a regular programming language n Good tool support (many commercial tools available) n Successfully deployed in different organizations and industry in
a variety of application domains » e.g., telecom, automotive, software, etc.
What is TTCN-3?
WoT OpenDay (13)
n Validation reveals problems/errors in » Standards and Products
n Validated standards give a higher chance of interoperable products » For standardisers gives assurance that they provide right functionality » For manufacturers and operators gives confidence to implement and go
to market n Provides an opportunity to correct errors in a controlled
manner » Late fixes in the product cycle are more expensive than early ones » Decreases time to market
Why Validate Standards
Standards can be validated by several means but one of the most practical and cost effective is by interop events
WoT OpenDay (14)
n Co-organized and funded by TTA and ETSIn Twice a year n Free of charge n Open to all companies with oneM2M implementations
(members and non-members) n Covered by NDA. No companies results are published n Important technical feedback provided to oneM2M
oneM2M Interop at a Glance
1st 2nd 3rd 4th
Regularevents
Verticals
2015 2016 2017
5th
SemanticInterop
2018
6th
WoT OpenDay (15)
oneM2M Interop #1 ~ #4Interop#1 Sophia-Antipolis (2015) Interop#2 Seoul (2016)
Interop#3 Kobe (2016) Interop#4 Taipei (2017)
WoT OpenDay (16)
n Co-organized by TTA and ETSIn When: 4th Dec (Mon) to 8th Dec (Fri) 2017, 5 daysn Where: Global IoT Certification Center, in Pangyo, Korean Scope:
» Interoperability and conformance testing (TS-0013, TS-0018, TS-0019)» Release 1 & 2 (TS-0001 & TS-0004 & Binding baselines from TP #31)
n Conformance Testing is the core of Interop5» In 5th Interop Event, Conformance session will be expanded to
• Validate Test Purpose, TTCN-3 (oneM2M ATS)• Validate Test Systems• Run Conformance Test on participants’ implementations• Semantic testing
» Semantic testing (an half-day)
oneM2M Interop #5
WoT OpenDay (17)
oneM2M Interop #5 – Semantic Testingn 4 companies implemented CSE or AE capabilitiesn 1 Four (4) hour session
» Tests developed based on Developer Guide• <semanticDescriptor>
• Semantic Discovery
n Information Session» Overview of oneM2M Semantic Features
n Planned topics for next Semantic Interop» Semantic Validation» Semantic Query» Semantic ACP» Partial Update of <semanticDescription>
WoT OpenDay (18)
oneM2M Semantics Features
• SemanticsannotationinR2(via<semanticDescriptor>)providesabasicsemanticsfunction,whichenablesothersemanticfeatures(e.g.semanticsresourcediscovery,semanticsquery,etc.)
• OntologymanagementinR3enablestoflexiblyleverageotherontologiesforsemanticsfeatures
• Moreadvancedsemanticsfeaturessuchassemanticsreasoningandsemanticsanalytics,whichhavebeenidentifiedasexistingrequirementsandcouldbespecifiedinR4.
oneM2MR2 oneM2MR3 oneM2MR4
• SemanticsAnnotation• SemanticsResource
Discovery• PartialTripleUpdate
usingSPARQL
• SemanticsQuery• SemanticsMashup• SemanticsACP• SemanticsValidation• OntologyManagement
• SemanticsReasoning• SemanticsAnalytics• …
WoT OpenDay (20)
Semantic Interop Scenario
Sensor#1IndoorTemperature
Fahrenheit
Sensor#2OutdoorTemperature
Celsius
Server
Smartphone#1Sensor#3
IndoorTemperatureCelsius
Sensor#4FridgeTemperature
Celsius
ADN-AE-2
ADN-AE-1
ADN-AE-3
ADN-AE-4
IN-CSE
ADN-AE-5
Mca
Mca
Mca
Mca
Mca
Jena
server
tempsensorae2
tempsensorae1
temperature
CI1
CI2
SD1
CIN
tempsensorae3
tempsensorae4
WoT OpenDay (21)
Semantic Interop Scenario
• CreateSDforAE1-4.• SemanticDiscovery
1. Naturallanguagequery:“GivemeallthesensorrelatedresourcesthatmeasurethetemperatureintheunitCelsius.”
2. Naturallanguagequery:“Givemeallthesensorrelatedresourcesthatmeasuretheindoortemperatureofahouse.”
3. Naturallanguagequery:“GivemeallthesensorrelatedresourcesthatmeasuretheindoortemperatureofahouseandprovidethemeasurementintheunitFahrenheit.”
WoT OpenDay (23)
oneM2M Certification program
• TTA was agreed the first oneM2M Certification Body in SC #33(Sep. 2016).
• oneM2M Certification Program was officially launched at Feb. 9, 2017.
• oneM2M Certification is intended to create an ecosystem of certified products that ensures interoperability among certified products.
www.onem2mcert.com
WoT OpenDay (24)
oneM2M CB Organization
SC
TP
TST
Certi. Ad-Hoc
oneM2M
oneM2M Certification Body (CB)
AuthorizedTestLab(ATL)
1st oneM2M Certification Body!
oneM2M - oneM2M CB
WoT OpenDay (25)
oneM2M Certified Products
Product Webpage Vendor ProductTypeGWP http://www.irexnet.co.kr IREXNET Endproduct(IN-CSE)
AiSOP http://www.irexnet.co.kr IREXNET Endproduct(IN-CSE)
Insator™ https://www.samsungsds.com SAMSUNGSDS Endproduct(IN-CSE)
HANDYPIAIoT Platform http://www.handysoft.co.kr/ HANDYSOFT,Inc. Endproduct(IN-CSE)
IoT HealthcarePlatform http://www.hconnect.co.kr/ HealthConnectCo.,Ltd Endproduct
ThingPlug https://www.thingplug.net SKTelecom Softwarecomponent
N-MAS http://www.ntels.com nTels Endproduct
IoTMakers Middleware http://iotmakers.olleh.com KT Softwarecomponent
IoTMakers http://iotmakers.olleh.com KT Softwarecomponent
e-IoT EnergyPlatform https://spin.kepco.co.kr KEPCO Endproduct
e-IoT EnergyGateway https://spin.kepco.co.kr KEPCO Endproduct
WoT OpenDay (26)
Global oneM2M Certification Program• The current certification program operated by TTA is now being
migrated to GCF for global program
• Industry advisor group is activated for this purpose• 1Q 2019 for Rel-2 and Rel-3
WoT OpenDay (28)
IoT Packet Analysern Simple user interface and operation process
• Packet Analysis - Show the packet structure of uploaded pcap file.
• Packet Compare - Same as above. Only the process of loading sample packets stored in the server is added.
Packet Analysis
Packet Compare
Web page
click
Click the button you want to perform
Browse…
File upload
Select the pcap file to upload from your computer
0000.packet
Select the packet you want to analyze
0001.packet0002.packet0003.packet0004.packet
….…. ok
A result page showing the structure of the packet.
All
oneM2M….…. ok
Select filter
WoT OpenDay (30)
Ttworkbench and TTCN-3 for oneM2M
oneM2M Functional Architecture – TTworkbench Test Access
FieldDomain
Infrastructure Domain #1
Application (AE)
Network (NSE)
Mcc
Mca
Mcn
Mca
Common Service (CSE)
Application (AE)
Network (NSE)
Mca
Mcn
Common Service (CSE)
Mca
Mcc Mcc
Optionally available via non-standard test API Standard Test API
Infrastructure Domain
Provider #2
Sensors, Actuators, Gateways
TTworkbench TTCN-3 Conformance test tool
WoT OpenDay (31)
Ontology Validatorn Developed by EGMn Basic aspects to be validated on semantic information
» Lexical validation» Syntactic validation» Semantic validation
Format FilenameextensionN-Quads .nqN-Triples .ntN3 .n3Turtle .ttlTriG .trigTriX .trixRDF/JSON .rjJSON-LD .jsonldBinaryRDF .brfRDF/XML .rdfOWL .owl
Popularreferenceontologies
areintegrated in
thetool
WoT OpenDay (32)
F-Interop H2020 Projectq www.f-interop.euq 1 November 2015 – 31 October 2018q develop and provide online interoperability and
performance test tools to support emerging technologies from research to standardization and market launch
q InitiallystandardsoftheIoT realmq Take,asastartingpoint,theETSIplugtests specificationsand
buildanarchitecturethatallowsthosetobedoneremotely
• CoAP• oneM2M• 6TiSCH