WP/PWP/EPUB4 Implementation Plans and Testing · 2017-06-22 · WP/PWP/EPUB4 Implementation Plans...

WP/PWP/EPUB4 Implementation Plans and

Testing

22June 2017

Ric Wright Readium Foundation

Readium Foundation, 2017

•  Implementation •  Testing •  Exit Criteria •  Testing Champions


Implementation

Implementa)on-ReadingSystems•  Readiumhasimplementedalmostallof3.0.1andpiecesof3.1

•  FullyintendtosupportEPUB4,PWPandWP–whatevertheyendupbeingJ

•  ReadiumhasalreadytakentheiniGaGvewithReadium-2toexplorewhatmightbebeyondanEPUBreadingsystem

•  Note:R2isNOTareplacementforR1•  R2isinsteadanevoluGonfromR1,withamoreplaMorm-specificandmodulardesign

•  It’snotcomplete,butgreatprogressisbeingmadeReadium Foundation, 2017

Implementa)on-ReadingSystems•  Readium-2isbeingdevelopedforserverdeployment,naGvemobileappsandthedesktop(whereitiswrappedinElectron)

•  ItistobeexpectedthatothermajorreadingsystemswillalsoprovideimplementaGons,e.g.•  VitalSource•  Google•  LibraryforAll•  andmanyothers...

•  ItwillbeinteresGngtoseehowotherReadingSystemsapproachthearchitectureandimplementaGon


Implementa)on-Authoring•  ThishasproventobeEPUB’sAchillesheel•  AuthoringEPUB2isfairlyeasy–therearemanytoolsoutthere–someofthemgood

•  EPUB3ismuchharder•  TherearefewtoolsouttherethatexportrealEPUB3(notjustEPUB2withEPUB3tags)

•  Noneofthemsupportallofit•  LiYleornosupportforJavaScriptorMediaOverlays

•  WPandEPUB4mightbeworse•  Howcanthatbeaddressed?



Testing

TestParameters•  Majordichotomy

•  ArewetesGngcontentitself?•  OrarewetesGngtherenderingofcontent?•  Orboth?•  Whatistheexitcriteriaforeach?

•  Whoexecutesthetests?•  IndependentorganizaGon?•  Self-interestedparGes?(i.e.implementers)

•  DisseminaGonoftheresults•  Publicorprivate?•  Website,database,web-service?


Tes)ngReadingSystems-Goals•  WhatarethegoalsofourtesGng?

•  AdherencetothespecificaGon•  CompletenessoftheimplementaGon•  InteroperabilitybetweenimplementaGons•  Quality(detecGngbugs)•  Performance

•  Memory•  Speed•  Reliability

•  Accessibility•  FailurecondiGons•  StresscondiGons•  RegressiontesGng


Tes)ngReadingSystems-Methods•  EPUBTestSuite

•  Usefulbutdoesn’tcovercomplexreal-worldproblems•  MorefortesGngcompletenessandprovidingayardsGckforexitcriteria

•  AutomatedTests•  Difficulttocreate,hardtomaintain•  Requiresdedicatedmanpower•  Can’ttestsomeaspects(e.g.accessibility)

•  UnitTestsandConGnuousBuilds•  VeryimportanttoReadingSystems•  VeryRS-specific


Tes)ng-ReadingSystems–– TestSuite•  EPUBTestSuiteShortcomings

•  Doesn’tcoverallof3.0.1andliYle(none?)of3.1•  Manytestsareoftheunderlyingbrowserengine•  TestsofJS,SVG,WebGLorinteracGvityareverylimited•  NoperformancetesGng•  NostresstesGng(size,complexity)•  Nofailurecases(e.g.malformedorillegalmarkup)•  LimitedandhardtoassesstestsofFXL•  LimitedandhardtoassesstestsofCJK,Arabic,Hebrew•  Notestsofotherscriptsandlanguages•  NolocalizaGon(allEnglish)•  And,ofcourse,newspecsfromEPUB4/WP


Tes)ngAuthoring-Goals•  AdherencetothespecificaGon

•  PassesEPUBCheck(orsimilar)•  Fidelity

•  Ofthecontent(author/editorintent)•  Acrossdifferentdevices

•  Quality(detecGngerrors)•  Performance

•  Memory•  Speed

•  Accessibility•  BestpracGcescheck


Tes)ngAuthoring-Methods•  EPUBCheckorsimilar

•  Simplesanitycheck,usefulbutonlyastart•  AutomatedTests

•  DifficultinpracGce?•  ManualTesGng

•  Labor-intensivebutnecessary•  Largenumberofdevicesandusagescomplicatesit

•  Workflows•  Publishingworkflowsvarywidelyacrosspublishers•  TesGngtendstobeveryspecifictotheworkflow

•  BestPracGcesTool


Tes)ngAuthoring–– BestPrac4ces•  Thisnotspecificallyaspecitem,butitisclearneed•  Needisforatoolthatflags

•  Useofillegalordeprecatedmarkup•  Inefficientmarkup•  Inefficientcontent(e.g.oversizeimages)•  Incompleteorincorrectmetadata•  Includingfontfileswithinsufficientrights•  PoorlystructuredEPUBs(toomanyortoofewspineitems,etc.)

•  Etc.•  Ideally,suchatoolcouldbeusedeitherinteracGvelyorinbatchmode



Exit Criteria

ExitCriteria––ReadingSystems•  Typically,theW3Crequires:

•  2independent,interoperableimplementaGonsofeachfeature•  Detailedtest-plan•  Accessibility,Privacy,i18nandSecurityassessmentsandplans

•  Giventhecomplexnatureoftheuse-casesandimplementaGonswhichusingmicroservicearchitectures,itcouldbechallengingtodecidewhata“completefeature”isandisitcomplete

•  Simplecases,yes,butallthoseedgecases...•  Thisemphasizestheneedforaclearsetoftestcases•  WhatconsGtutes“done”:

•  Performance?Ifafeatureisdog-slow,isitcomplete?•  Bugs?Howbaddoesabughavetobetorejectafeature?


ExitCriteria-Authoring•  Typically,theW3Crequires2implementaGonsthat

consumethecontent•  Shouldthatbebroadenedtoauthoringthecontent?•  IDPFdidn’thavesucharequirementandithasproved(IMO)aproblem,cf.lackofauthoringtools

•  OTOH,EPUB4/PWP/WPmaybesuchacomplexbeastandtheuse-casesequallycomplexthatdefiningsuchcriteriamaybeachallenge

•  Wecancoverthesimplecases,butwhataboutallthoseedgecases?

•  Butprobablyworththinkingabout-atleastasathought-experiment

•  Again,abest-pracGcesguidewouldbeexcellent



Testing Champion Ambassador

Tes)ngAmbassador––ReadingSystems•  GiventhepotenGalcomplexityofthespecandthefactthatatleastoneoftheimplementers(Readium)isopen-source(read:herdingcatsinthedark),itwillbeverychallenging

•  OTOH,evenintheseearlydays,enGGessuchasEDRLabandFeedbooksarededicaGngsignificantresourcestotheeffort

•  ValidaGngtheinteroperability,giventhecomplexityofthespecandtheuse/workflows,willbechallenging

•  TheroleoftheambassadorwillprobablybeacombinaGonofcoach,cheerleaderandcriGc


Tes)ngAmbassador-Authoring•  Similarly,validaGngtheinteroperabilitygiventhepotenGalcomplexityofthespecandtheuse/workflowswillbechallenging

•  TherewillalsobeaperceptualissueinthatasauthoringofEPUB3issGlldifficultandpoorlysupported,convincingpublisherstosupportanew,morecomplexspecwillbehard

•  Itmaybealmostimpossibleunlesswemandateauthoringsupport

•  ButthatisachickenandeggsituaGon•  WithoutRSsupport,howcanpublisherscommit?•  Withoutauthoringsupport,howcanRScommit?


Readium.org 22 June 2017



Appendix

Readium-2•  Highlymodular•  Microservicearchitecture•  BasedontheWebPubManifest

specificaGon•  RESTful•  LeveragesplaMormstrengths•  Designedtobeaccessible•  Bothclientandserver

implementaGons•  SupportsbothapplicaGons

andheadlessusage


WP/PWP/EPUB4 Implementation Plans and Testing · 2017-06-22 · WP/PWP/EPUB4 Implementation Plans...

Documents

Transcript of WP/PWP/EPUB4 Implementation Plans and Testing · 2017-06-22 · WP/PWP/EPUB4 Implementation Plans...