DB2 for Linux, Unix, and Windows - api.ning.com · PDF fileILOG, Maximo®,...
Transcript of DB2 for Linux, Unix, and Windows - api.ning.com · PDF fileILOG, Maximo®,...
DB2 for Linux, Unix, and Windows Version 11 Highlights
Actionable Insights Continuous Availability
Massive Scalability Outthink the Possible
DB2forLinux,Unix,andWindows:Version11HighlightsCopyright©2016byInternationalBusinessMachinesCorporation(IBM).Allrightsreserved.PrintedinCanada.ExceptaspermittedundertheCopyrightActof1976,nopartofthispublicationmaybereproducedordistributedinanyformorbyanymeans,orstoredinadatabaseorretrievalsystem,withoutthepriorwrittenpermissionofIBM,withtheexceptionthattheprogramlistingsmaybeentered,stored,andexecutedinacomputersystem,buttheymaynotbereproducedforpublication.Thecontentsofthisbookrepresentthosefeaturesthatmayormaynotbeavailableinthecurrentreleaseofanyproductsmentionedwithinthisbookdespitewhatthebookmaysay.IBMreservestherighttoincludeorexcludeanyfunctionalitymentionedinthisbookforthecurrentreleaseofDB211.1,orasubsequentrelease.Inaddition,anyclaimsmadeinthisbookarenotofficialcommunicationsbyIBM;rather,theyareobservedbytheauthorsinunauditedtestingandresearch.TheviewsexpressedinthisbookisthoseoftheauthorsandnotnecessarilythoseoftheIBMCorporation;botharenotliableforanyoftheclaims,assertions,orcontentsinthisbook.IBM'sstatementsregardingitsplans,directions,andintentaresubjecttochangeorwithdrawalwithoutnoticeandatIBM'ssolediscretion.Informationregardingpotentialfutureproductsisintendedtooutlineourgeneralproductdirectionanditshouldnotbereliedoninmakingapurchasingdecision.Theinformationmentionedregardingpotentialfutureproductsisnotacommitment,promise,orlegalobligationtodeliveranymaterial,codeorfunctionality.Informationaboutpotentialfutureproductsmaynotbeincorporatedintoanycontract.Thedevelopment,release,andtimingofanyfuturefeatureorfunctionalitydescribedforourproductsremainsatoursolediscretion.PerformanceisbasedonmeasurementsandprojectionsusingstandardIBMbenchmarksinacontrolledenvironment.Theactualthroughputorperformancethatanyuserwillexperiencewillvarydependinguponmanyfactors,includingconsiderationssuchastheamountofmultiprogrammingintheuser'sjobstream,theI/Oconfiguration,thestorageconfiguration,andtheworkloadprocessed.Therefore,noassurancecanbegiventhatanindividualuserwillachieveresultssimilartothosestatedhere.U.S.GovernmentUsersRestrictedRights-Use,duplicationordisclosurerestrictedbyGSAADPScheduleContractwithIBM.InformationinthiseBook(includinginformationrelatingtoproductsthathavenotyetbeenannouncedbyIBM)hasbeenreviewedforaccuracyasofthedateofinitialpublicationandcouldincludeunintentionaltechnicalortypographicalerrors.IBMshallhavenoresponsibilitytoupdatethisinformation.THISDOCUMENTISDISTRIBUTED"ASIS"WITHOUTANYWARRANTY,EITHEREXPRESSORIMPLIED.INNOEVENTSHALLIBMBELIABLEFORANYDAMAGEARISINGFROMTHEUSEOFTHISINFORMATION,INCLUDINGBUTNOTLIMITEDTO,LOSSOFDATA,BUSINESSINTERRUPTION,LOSSOFPROFITORLOSSOFOPPORTUNITY.
IBMproductsandservicesarewarrantedaccordingtothetermsandconditionsoftheagreementsunderwhichtheyareprovided.ReferencesinthisdocumenttoIBMproducts,programs,orservicesdoesnotimplythatIBMintendstomakesuchproducts,programsorservicesavailableinallcountriesinwhichIBMoperatesordoesbusiness.Informationconcerningnon-IBMproductswasobtainedfromthesuppliersofthoseproducts,theirpublishedannouncementsorotherpubliclyavailablesources.IBMhasnottestedthoseproductsinconnectionwiththispublicationandcannotconfirmtheaccuracyofperformance,compatibilityoranyotherclaimsrelatedtonon-IBMproducts.Questionsonthecapabilitiesofnon-IBMproductsshouldbeaddressedtothesuppliersofthoseproducts.IBMdoesnotwarrantthequalityofanythird-partyproducts,ortheabilityofanysuchthird-partyproductstointeroperatewithIBM'sproducts.IBMEXPRESSLYDISCLAIMSALLWARRANTIES,EXPRESSEDORIMPLIED,INCLUDINGBUTNOTLIMITEDTO,THEIMPLIEDWARRANTIESOFMERCHANTABILITYANDFITNESSFORAPARTICULARPURPOSE.Theprovisionoftheinformationcontainedhereinisnotintendedto,anddoesnot,grantanyrightorlicenseunderanyIBMpatents,copyrights,trademarksorotherintellectualpropertyright.IBM,theIBMlogo,ibm.com,Aspera®,Bluemix,BlueworksLive,CICS,Clearcase,Cognos®,DOORS®,Emptoris®,EnterpriseDocumentManagementSystem™,FASP®,FileNet®,GlobalBusinessServices®,GlobalTechnologyServices®,IBMExperienceOne™,IBMSmartCloud®,IBMSocialBusiness®,InformationonDemand,ILOG,Maximo®,MQIntegrator®,MQSeries®,Netcool®,OMEGAMON,OpenPower,PureAnalytics™,PureApplication®,pureCluster™,PureCoverage®,PureData®,PureExperience®,PureFlex®,pureQuery®,pureScale®,PureSystems®,QRadar®,Rational®,Rhapsody®,SmarterCommerce®,SoDA,SPSS,SterlingCommerce®,StoredIQ,Tealeaf®,Tivoli®,Trusteer®,Unica®,urban{code}®,Watson,WebSphere®,Worklight®,X-Force®andSystemz®Z/OS,aretrademarksofInternationalBusinessMachinesCorporation,registeredinmanyjurisdictionsworldwide.OtherproductandservicenamesmightbetrademarksofIBMorothercompanies.AcurrentlistofIBMtrademarksisavailableontheWebat"Copyrightandtrademarkinformation"at:www.ibm.com/legal/copytrade.shtml.AlltrademarksorcopyrightsmentionedhereinarethepossessionoftheirrespectiveownersandIBMmakesnoclaimofownershipbythementionofproductsthatcontainthesemarks.IntiialPublication:October23,2016
About the Authors GeorgeBaklarz,B.Math,M.Sc.,Ph.D.Eng.,hasspent31yearsatIBMworkingonvariousaspectsofdatabasetechnology.GeorgewaspartoftheteamthathelpedmovetheOS/2ESdatabasetoTorontotobecomepartoftheDB2familyofproducts.Hehasworkedonvendorenablement,competitiveanalysis,productmarketing,productplanning,andtechnicalsalessupport.Georgehaswritten10booksonDB2andotherdatabasetechnologies.GeorgeiscurrentlypartoftheWorldwideCoreDatabaseTechnicalSalesTeam.EnzoCialini,B.Sc.,isaSeniorTechnicalStaffMemberandMasterInventorintheWorldwideCoreDatabaseTechnicalSalesTeamandformerlytheChiefQualityAssuranceArchitectforDB2&PureDataintheIBMTorontoDB2DevelopmentTeam.HeisalsoapublishedbookauthorandwrittenvariouspapersonDB2.Enzohas25yearsofexperienceindatabasetechnology,softwaredevelopment,testing,support,competitiveanalysisandproductiondeployments.
Forward Theneweconomyischangingthewaywegatherinformation,managedata,gaininsights,reinventourbusinesses,anddosoquicklyanditeratively.Thedigitaltransformationofeverythingischangingtheinformationvaluechainnotjustfromyourownorganization,butfromstakeholdersandthirdpartydataproviders.Andwiththisnewinformationandincontextfromwhichthey'regathered,wegainadeeperunderstandingofclientbehaviorsandmarketconditions,andformulatefasterresponsestochangingcompetitivelandscape.Theadventofcloudandmobilecomputingfurtherencouragesthedistributionandconsumptionofinformationatarapidpace,whichinturngeneratesmoreinformationtoanalyze.Therefore,thisneweconomyrequiresarobustdatabasesoftwarethatsitsattheheartofanenterprisetosupportitstransformationinthisneweraofdigital,cloudandcognitive.DB2isaversatileandscalabledatabasethatcansupportyourtransactionalandanalyticalworkloadsfordifferentapplicationswhetheronpremisesorintheclouduptopetabytesinvolume.Itiseasytodeploy,upgradeandmaintainandatthesametimeitsimplifiesthemovefromOracledatabasesbyleveragingtheSQLCompatibilitycomponentalongwithyourexistingskillsandassetswithminimalchangestotheapplication.DB2'spureScalecapabilityensuresthatyourbusinessdataisalwaysavailableandaccessiblewithoutanyinterruptionstoyourbusinessprocesses,ensuringthatyourconsumersarenotimpactedwherevertheymaybe.WithBLUMPP,youleveragemultiparallelprocessingwithin-memorycolumnartechnologytoprovideextremeperformanceenhancementsforyouranalyticworkloads.
Finally,DB2providestheflexibilityofdeployingonpremiseoronthecloudwithintegratedsecuritytoensurethatyournewbusinessprocessapplicationsanddataarewheretheymakesense,nearyourconsumers.SeveralotherdistributeddatabaseofferingsareavailablefromIBMinamanagedenvironment,hostedenvironment,virtualized,usingyourowndatacenter,oronanappliance:
• dashDB-Anextgenerationhybriddatabasefamilyo dashDBforAnalyticso dashDBforTransactionalo dashDBLocal
• DB2onCloud• Informix• InformixonCloud• PureDataSystemforAnalytics• Cloudant
Nomatterwhatyourrequirementsare,IBMhasadatabaseofferingthatwillmeetyourneeds.
Packaging,Installation,andAdministration..................................................2EndofServiceandMarketing...............................................................................2NewIncludedFeatures.........................................................................................3
Federation.........................................................................................................3Encryption.........................................................................................................3pureScaleAdministrationFeature.....................................................................4
DB2ExpressandDB2ExpressCommunityEditions..............................................6DB2WorkgroupandEnterpriseEditions...............................................................7AdvancedWorkgroupandAdvancedEnterpriseEditions....................................8AdditionalProductsforDB2..................................................................................8
AdvancedRecoveryFeature..............................................................................8DB2PerformanceManagementOffering.........................................................9
DB2DirectEditions................................................................................................9OperatingSystemSupport..................................................................................10VirtualizationSupport.........................................................................................11UpgradeSupport.................................................................................................12AdministrationImprovements............................................................................13
RangePartitionReorganization......................................................................13ADMIN_MOVE_TABLE....................................................................................13RemoteStorage..............................................................................................14DB2HistoryFileBackup..................................................................................14HardwareBackupCompression......................................................................14
DB2BLUEnhancements..............................................................................17MassivelyParallelProcessingArchitecture.........................................................18FastCommunicationManager............................................................................19FasterSQLMERGEprocessing.............................................................................20NestedLoopJoinSupport...................................................................................20SortProcessingEnhancements...........................................................................21QueryRewriteEnhancements............................................................................22Push-downofOLAPfunctions.............................................................................23SQLFunctionsOptimizedforBLU.......................................................................23AutomaticDictionaryCreation............................................................................24
ContinuousAvailability................................................................................26pureScaleArchitecture........................................................................................26Deployments.......................................................................................................27Virtualization.......................................................................................................30Installation...........................................................................................................30WorkloadBalancing............................................................................................31Failoverbehavior.................................................................................................33Failbackbehavior................................................................................................34
Contents viii
Manageability......................................................................................................34Pre-Installation................................................................................................34Post-Installation..............................................................................................35
DisasterRecovery................................................................................................35HADR...............................................................................................................35GeographicallyDispersedDB2pureScaleCluster(GDPC)...............................36
CompatibilityFeatures................................................................................39OuterJoinOperator............................................................................................39CHARdatatypesizeincrease...............................................................................40BinaryDataTypes................................................................................................40SynonymDatatypes............................................................................................41FunctionNameCompatibility..............................................................................42
NetezzaCompatibility..................................................................................45SpecialOperatorCharacters...............................................................................45GROUPBYOrdinalLocation................................................................................46TRANSLATEFunction...........................................................................................47
SQLExtensions.............................................................................................50SimpleSQLwithFetchFirstClause.....................................................................50OFFSETExtension................................................................................................51FETCHFIRSTandOFFSETinSUBSELECTs.............................................................52AlternateSyntaxforFETCHFIRST.......................................................................54TableCreationwithSELECT.................................................................................55CREATEFUNCTIONforAggregateUDFs..............................................................56
DateFunctions.............................................................................................59ExtractFunction..................................................................................................59DATE_PARTFunction...........................................................................................60DATE_TRUNCFunction........................................................................................61ExtractingSpecificDaysfromaMonth...............................................................63DateAdditionFunctions......................................................................................64ExtractingWeeks,Months,Quarters,andYearsfromaDate............................65NextDayFunction...............................................................................................66BetweenDate/TimeFunctions............................................................................67MONTHS_BETWEENFunction.............................................................................68DateDurationFunctions.....................................................................................69OVERLAPSPredicate............................................................................................70UTCTimeConversions.........................................................................................71UsingUTCFunctions............................................................................................72
HexFunctions..............................................................................................78INTNFunctions....................................................................................................78TO_HEXFunction.................................................................................................79
Contents ix
RAWTOHEXFunction...........................................................................................79
RegularExpressions.....................................................................................81RegularExpressionExamples..............................................................................81RegularExpressionCommands...........................................................................82RegularExpressionFlagValues...........................................................................83RegularExpressionSearchPatterns....................................................................83AnchoringPatternsinaSearch...........................................................................84Matchingpatternsacrossmultiplelines.............................................................85LogicalOROperator............................................................................................86CombiningPatterns.............................................................................................87Matchingcharactertypes...................................................................................91SpecialPatterns...................................................................................................94NegatingPatterns................................................................................................95CapturingParenthesis.........................................................................................97
StatisticalFunctions...................................................................................101SamplingFunctions...........................................................................................101AdditionalAnalyticalFunctions.........................................................................102WIDTHBUCKETandHistogramExample..........................................................104
HashingFunctions......................................................................................107HASH4-Four-byteHashEncoding...................................................................107HASH8-Eight-byteHashEncoding...................................................................108HASH-VariableHashEncoding.........................................................................109
InternalDB2JSONRoutines......................................................................111DB2JSONFunctions..........................................................................................112PathStatementRequirements..........................................................................112CreatingTablesthatSupportJSONDocuments................................................113JSONDocumentRepresentation.......................................................................114JSON2BSON:InsertingaJSONDocument.........................................................114BSON2JSON:RetrievingaJSONDocument.......................................................115DeterminingDocumentSize..............................................................................116BSON_VALIDATE:CheckingtheFormatofaDocument....................................116RetrievingJSONDocuments..............................................................................117
SampleJSONTableCreation.........................................................................117AdditionalJSON_DEPTTable.........................................................................118
JSON_VAL:RetrievingDatafromaBSONDocument........................................118RetrievingAtomicValues..............................................................................120RetrievingArrayValues.................................................................................121RetrievingStructuredFields..........................................................................122DetectingNULLValuesinaField...................................................................122JoiningJSONTables.......................................................................................123
Contents x
JSONDataTypes................................................................................................124ExtractingFieldsUsingDifferentDataTypes....................................................126
JSONINTEGERSandBIGINT..........................................................................126JSONNUMBERSandFLOATINGPOINT.........................................................128JSONBOOLEANVALUES................................................................................130JSONDATE,TIME,andTIMESTAMPS............................................................131JSONStrings..................................................................................................132
JSON_TABLEFunction.......................................................................................133JSON_LENFunction...........................................................................................136JSON_GET_POS_ARR_INDEXFunction..............................................................136UpdatingJSONDocuments...............................................................................138IndexingJSONDocuments................................................................................139SimplifyingJSONSQLInsertsandRetrieval.......................................................141
Summary....................................................................................................144
ResourcestoBuildYourDB2Skills............................................................146IBMCertificationExams....................................................................................146IBMTraining......................................................................................................146BigDataUniversity.............................................................................................146InformationManagementBookstore...............................................................147IBMSupportforDB2.........................................................................................147IBMDataMagazine...........................................................................................147InternationalDB2UserGroup(IDUG)...............................................................147JointheConversation........................................................................................148AdditionaleBookMaterial................................................................................148
Acknowledgments
Wewanttothankthefollowingpeoplewho,inonewayoranother,contributedtothisbook:MichaelSpringgay,KellySchlamb,PaulBird,MattHuras,CalistoZuzarte,JohnHornibrook,GregStager,andAlanLeeWewouldalsoliketothankallthedevelopmentteamforhelpingtodeliverthisreleasegiventhetremendousdeadlinesandconstraintsthattheyhavebeenunder.ThecontentsofthiseBookaretheresultofalotofresearchandtestingbasedonthecontentsofourDB211.1KnowledgeCenter.Theauthorsofouronlinedocumentationdeservespecialthanksforgettingthedetailstousearlyintheproductdevelopmentcyclesowecouldbuildmuchofourmaterial.Forthemostup-to-dateinformationonDB211.1features,pleaserefertotheIBMKnowledgeCenter:http://www.ibm.com/support/knowledgecenter/SSEPGGThereyoucanselecttheVersion11.1.0releasefromthedrop-downmenuandgetmoredetailsonthefeaturesweexploreinthiseBook.
Introduction
AboutThisBookTheDB211.1releasedeliversseveralsignificantenhancementsincludingDatabasePartitioningFeature(DPF)forBLUcolumnartechnology,improvedpureScaleperformanceandfurtherHighAvailabilityDisasterRecovery(HADR)support,andnumerousSQLfeatures.Thisbookwaswrittentohighlightmanyofthenewfeaturesandfunctionsthatarenowavailableinthisrelease,withoutyouhavingtosearchthroughvariousforums,blogs,andonlinemanuals.WehopethatthisbookgivesyoumoreinsightintowhatyoucannowaccomplishwithDB211.1,andincludeitonyourshortlistofdatabasestodeploy,whetheritisonpremise,inthecloud,orahybridapproach.GeorgeandEnzo
HowThisBookIsOrganizedWeorganizedthisbookinto11chaptersthatcovermanyofthehighlightsandkeyfeaturesfoundintheDB211.1release.
• Chapter1givesyouanoverviewofthenewpackagingchangesinDB211.1,alongwithpre-requisitesforinstallingtheproduct
• Chapter2discussestheenhancementstoDB2BLU,includingsupportforBLUcolumnartablesinaDPF(DatabasePartitioningFeature)environment
• Chapter3examineswhatpureScalecapabilitiesprovideyouinaproductionenvironment,alongwithalltheenhancementsthathavebeenmadetosimplifyinstallationandmanagementofacluster
• Chapter4through10arefocusedonSQLandcompatibilityenhancements
Chapter1:Packaging,Installation,andAdministration 2
Packaging,Installation,andAdministration
DB211.1introducedseveralpricingandpackagingchangesthatwillaffectsomecustomers.Inaddition,theendofserviceofDB29.7andDB210.1wereannounced,whichmeansthatcustomerswitheitherofthosereleaseswillneedtostartmakingplanstoupgradetoanewversionofDB2overthenextyear.
EndofServiceandMarketingThreeadditionalannouncementsweremaderegardingtheendofmarketingandtheendofserviceforolderDB2releasesandfeatures.DB210.5endofmarketing,onSeptember30th,2016
• DB29.7andDB210.1endofservice,onSeptember30th,2017• DiscontinuedBusinessValueOfferings(BVO)
EndofmarketingmeansthatcustomerswillnotbeabletopurchaseDB210.5aftertheendofSeptember2016.ThismeansthatyoucanonlypurchaseDB211.1now,butyouarestillabletogetcopiesofDB210.5ifrequired.CustomerscontinuetogetserviceandsupportforDB210.5sinceendofserviceisnotbeingannouncednow.ForthosecustomersrunningeitherDB29.7orDB210.1,theywillneedtobeginplanningonupgradingtoeitherDB210.5orDB211.1.ExtendedservicecontractswillbeavailabletomaintainyourolddatabaseimagesafterSeptember2017,buttherewillbeanadditionalchargetodothat.ForthosecustomerswhopurchasedanyoftheBLUAccelerationBusinessValueOfferings,theywillbeupgradedtoanAdvancedlicenseofDB2dependingonwhichlicensetheyhadwhentheypurchasedtheBVO.TheEncryptionOfferingandtheBusinessApplicationContinuityOfferingarebundledwitheveryeditionofDB211.1,socustomersthatpurchasedthatfeaturewillnolongerhavetopayserviceandsupportonitwhenupgradingtoDB211.1.
Chapter1:Packaging,Installation,andAdministration 3
NewIncludedFeaturesOnesignificantchangetoDB2packagingisthatallDB2Editions(Workgroup,Enterprise,AdvancedWorkgroupandAdvancedEnterprise,DigitalStandard,DigitalAdvanced)willincludeEncryption,Federation,andpureScale.WhiletherearelimitationsontheuseofFederationandpureScaleinWorkgroup,Enterprise,andDigitalStandardEditions,theEncryptionfeatureisfullysupportedacrossalleditions.
Federation
DB211.1nowincludesdatafederationandvirtualizationthrougharobustSQLinterfaceandafullrelationaldatabaseenginedesignedforglobaloptimizationofdataaccess.Thiscapabilityoffershomogeneousorheterogeneousfederation(dependingontheDB2Edition)betweenselectIBMandnon-IBMdatasources.Youcanquicklyprototypeandimplementsolutionsinvolvingdisparatedatabyvirtualizingaccessandoptimizingperformance.Datacanremaininplaceforbothtransactionalaswellasanalyticalworkloads.
Figure1:FederationSupport
DB2AdvancedServerEditionsincludeallofthewrappersthatarepartofthefederationsupport.TheDB2WorkgroupandEnterpriseEditionproductsonlyincludesupportforconnectingtoDB2andInformixdatasources.
Encryption
DB2includedencryptionatthedatabaselevelinVersion10.5.However,theencryptionfeaturerequiredtheuseofalocalkeystoretocontainthe
Chapter1:Packaging,Installation,andAdministration 4
masterkeyforthedatabase.Thisrequiredthatthekeystoreitselfbebackedupandmanagedindependentlyfromthedatabasebackup.
Figure2:EncryptionSupport
Ratherthanmanageseveralseparatekeystoresmanually,customerscannowuseanyKMIP1.1centralizedkeymanagertomanagetheDB2keys.TheISKLMproduct(IBMSecurityKeyLifecycleManager)hasbeenvalidatedtoworkwithDB2,butanyKMIP1.1compliantmanagerissupported.ISKLMhelpsmeetregulationsandstandardssuchas:
• PaymentCardIndustryDataSecurityStandard(PCIDSS)• Sarbanes-Oxley• HealthInsurancePortabilityandAccountabilityAct(HIPAA)
UsingthePKCS#11protocol,DB2NativeEncryptionwillbeenhancedtosupportusingHardwareSecurityModules(HSM)asthekeystoreforthemasterkey.TheinitialsetofHSMproductsvalidatedwillincludeSafenetLunaSAandThalesnShieldConnect+,butanyPKCS#11complaintproductissupported.NotethattheHSMsupportiscurrentlyinpreviewmode,andhasnotbeenmadegenerallyavailableatthetimeofpublication.Forupdatesontheavailabilityofthisfeature,pleasecheckthelatestFixpacksummaryinformationon:http://www.ibm.com/support/knowledgecenter/SSEPGG_11.1.0/com.ibm.db2.luw.kc.doc/welcome.html
pureScaleAdministrationFeature
DB211.1Workgroup,Enterprise,andDigitalStandardEditionincludethepureScalefeature,limitedtotwomemberclusters.Customerswhodo
Chapter1:Packaging,Installation,andAdministration 5
notrequirethescalabilityofpureScalebutrequirethebenefitsofcontinuousavailability,candosonowatlowercosts.Thelimitedtwo-memberpureScaleclusterconfigurationenablesonemembertoprocessapplicationworkloads,andasecondmembertoprovidecontinuousavailabilitywhenthefirstmemberisundergoingplannedorunplannedoutages.Inaddition,thesecondmembercanalsobeusedtoperformadministrativetasksandutilities,thusoff-loadingthesetasksfromtheprimarymember.Thefirstmemberisfullylicensed,whilethesecondmemberonlyrequiresminimallicensingsimilartoanHADRenvironment(licensedaswarmstandby,e.g.100PVUsor1VPC).
Figure3:pureScale2-nodeconfiguration
Allapplicationworkloadsaredirectedtotheprimaryactivemember(s),sometimesreferredtoasthe"primary"memberwhileutilitiesandadministrativetasksareallowedonthesecondarymember.Thisconfigurationisidealforoff-loadingbackupsfromprimarymembersaswellasensuringcontinuousavailabilityintheeventofamemberfailure.
Chapter1:Packaging,Installation,andAdministration 6
Administrativetasksandutilities,however,areallowedtorunonthesecondarymember.Nothingispreventingthesameadministrativeworkfromrunningontheprimarymembersaswell,butthebestpracticeistokeepadministrativetaskstothesecondarymember-thus,freeingtheprimarymemberresourcesforprocessingtheworkload.Thisisanactive/activeclusterasworkisbeingdoneonALLmembersoftheclusterbutusingan'active/passive'licensingmodel.Thesame'active/passive'licensingmodelisavailabletobeappliedtothefullyscalableDB2pureScaleactive/activeconfigurationavailableinDB2AdvancedEditions,includingthenewDirectAdvancedEdition,allowingforoff-loadingofadministrationtasksandutilitiesinacontinuouslyavailableandscalable>2-membercluster.
DB2ExpressandDB2ExpressCommunityEditionsForthoseuserslookingtodevelop,deployanddistributeDB2atNOchargeshouldlookatDB2Express-C.DB2Express-CEditionisano-chargecommunityeditionofDB2availableforWindowsandLinuxplatforms,whichprovidesthecorefeaturesofDB2Editions.Formoreadvancedfeatures,suchasDB2pureScaleandBLUAcceleration,thesolutioncanbeupgradedwithoutrequiringmodificationtoyourapplications.DB2Express-Coffersthesefeaturesandbenefits:
• Encryptionfordataatrestandin-transittoprotectyourdata• UsesTimeTravelQuerytoprovidebi-temporaldatamanagement• DeliversanintegratedtoolsenvironmentwithIBMDataStudiofor
easiermanagementanddevelopment• ContainstheSQLcompatibilityfeaturetomoreeasilyrun
applicationswrittenforotherdatabasesonDB2• ContinuousDataIngest• Federation• pureXMLstorage• JSONNoSQL
Chapter1:Packaging,Installation,andAdministration 7
ThelicensedDB2ExpressEditionisnolongerofferedinDB211.1.ThosecustomersthatcurrentlyhaveDB2ExpresswillbeupgradedtotheDB2WorkgroupEdition.
DB2WorkgroupandEnterpriseEditionsBothDB2WorkgroupandEnterpriseEditionshavehadseveraladditionalfeaturesincludedintothepackaging:
• pureScaleStandbyMemberOption• Tablepartitioning,Encryption• Federation(DB2&Informix)
Inaddition,DB2WorkgroupEditionhashadfurtherfeaturesaddedwhicharealreadyincludedinDB2EnterpriseEdition
• Multi-dimensionalClustering• Multi-TempStorage• QueryParallelism
TherearesomeexclusionstowhatisincludedintheWorkgroupandEnterpriseeditions.Thelistincludes(butitnotlimitedto):
• DataPartitioning• SQLWarehouse(SQW)• MaterializeQueryTables(includedinEnterpriseEdition)• BLUAcceleration,Compression
TheonebigchangeinthelicensingofDB2WarehouseEditionisthatthereisnolongeracaponthesizeofdatabase.Thecorelimitremainsthesameat16cores,buttheamountofmainmemoryhasincreasedto128GB.Therearenolimitsonthecores,memoryordatabasesizewithDB2EnterpriseEdition.TherearetwooptionalproductsthatcustomersmaywanttopurchaseforWorkgroupandEnterpriseEdition:
• DB2PerformanceManagementOffering(WLM)whichispartoftheDataServerManagerEnterpriseEdition
• AdvancedRecoveryFeature
Theseproductsaredescribedinanothersection.
Chapter1:Packaging,Installation,andAdministration 8
AdvancedWorkgroupandAdvancedEnterpriseEditionsAswiththepriorreleaseofDB210.5,theAdvancedEditionsofWorkgroupandEnterpriseEditionincludeallfeaturesforonesingleprice.WhatisnewinDB211.1isthenewFederatedfeaturewhichincludesconnectivitytoDB2,Informix,andmanyotherrelationalandunstructureddatasources. TherearethreeproductsthatareincludedaspartoftheAdvancedEditionsthathavelimitationonusage.Theseproductsare:
• DB2ConnectincludedforusingSQWtoolingtoaccessDB2forzandDB2fori
• InfoSphereDataArchitectlimitedto10authorizedusers• CognosAnalyticslimitedto5authorizedusers
TheonlyoptionalfeaturethatcustomersmaywanttoconsideristheAdvancedRecoveryFeature.
AdditionalProductsforDB2InconjunctionwiththeDB211.1databaserelease,severaltoolswerereleasedthatareincludedintheAdvancedEditionsofDB2,andavailableforpurchasewiththeothereditions.
AdvancedRecoveryFeature
Thisfeatureconsistsofadvanceddatabasebackup,recovery,anddataextractiontoolsthatcanhelpyouimprovedataavailability,mitigaterisk,andacceleratecrucialadministrativetaskswhentimeisoftheessence.NewfeaturesofthetoolsincludesupportforDB211.1,includingusingthetoolswithDB2NativeEncryption.TheAdvancedRecoveryFeaturebundlecontainsthefollowingnewreleasesoftheincludedtools:
• DB2MergeBackupforLinux,UNIX,andWindowsV3.1• DB2RecoveryExpertforLinux,UNIX,andWindowsV5.1• OptimHighPerformanceUnloadforDB2forLinux,UNIX,and
WindowsV6.1
Chapter1:Packaging,Installation,andAdministration 9
DB2PerformanceManagementOffering
TheDB2PerformanceManagementOffering(WLM)ispartoftheDataServerManagerEnterpriseEdition.DB2PerformanceManagementOfferinghelpsorganizationsmanagetheirDB2databasesacrosstheenterprisethroughtoolingthathasacommonintegratedconsole,enhancedsimplicity,improvedperformanceandscalability,andsimplifiedworkflowwithactionablerecommendations.Organizationscandothefollowing:
• Simplifydatabaseadministrationandreduceerrorswithintegratedwebconsoletoviewandmonitorthehealthofdatabasesacrosstheenterprise.
• Identify,diagnose,prevent,andsolveperformanceproblemswithamorepredictabledatabaseserverexecutionenvironment.
• Trackandmanageclient/serveranddatabaseenvironmentefficientlywithcentralizedmanagement.
• Optimizedatabaseperformancewithexpertrecommendationsonquerydesign,uniqueindexes,andstatisticsquality.
DB2DirectEditionsDB211.1includesanewdeliverymechanismforlicensing.TheDB2DirectEditionsareintendedtomakeiteasiertoacquireDB2licensesforclouddeployments:
• Newlicensemetricstofacilitatehybridclouddeployments• Acquiretheproductdirectlyonline(PassportAdvantage)• Optiontodeployeitheron-premisesoroncloud
Itdoesn'tmatterwhereyouplaceyourDB2server,onpremise,oncloud,etc.YouarelicensedbythenumberofVirtualProcessorcorevCPUs(explainedonthenextpage).Thelicenseistotallyflexibleandcanbemovedbetweenplatformswithoutanychangeinlicensefees.NotethatthislicenseisanOPEX(OperatingExpense)license–youarerentingorleasingtheDB2license.Thepriceincludestheuseoftheproductaswellasserviceandsupport.Ifthelicenseisnotrenewed,thedatabasewillnothaveavalidlicense.TherearetwoDB2DirectEditions,dependingonwhatthedatabaserequirementsare:
Chapter1:Packaging,Installation,andAdministration 10
• DB2DirectStandardEdition11.1-HasallthedatabasefeaturesofDB2WorkgroupServerEdition
• DB2DirectAdvancedEdition11.1-HasallthedatabasefeaturesofDB2AdvancedEnterpriseServerEdition
BothlicensesaresoldasamonthlylicensechargeusingtheVirtualProcessorCore(VPC)soldasamonthlylicensecharge.Customerscanchoosetomaintainthesedatabaseswitheither:
• LongTermSupportRelease(LTSR)support• ContinuousDelivery(CD)support
LTSRsupportwillgivethecustomerfixpacksonlywithnoadditionalfunctionalityaddedtotheproduct.CDsupportwillcontainthefixesfoundintheLTSRstreamandmayalsocontainnewfunctionalityandenhancements.
OperatingSystemSupportAcustomerneedstobeawarethattherearenewoperatingsystems(O/S)requirementsforDB211.1.WhiletheirexistingDB2installationmaybesupportedatanolderO/Sreleaselevel,theywillnotbeabletoupgradetheirdatabaseunlesstheO/Shasbeenupdatedaswell.DB211.1introducessupportforPowerLinuxLE(LittleEndian)whichalsoincludessupportforpureScale.DB2supportsPowerLinuxLEwiththefollowingthreeoperatingsystems:
• RedHatEnterpriseLinux(RHEL)7.1+• SUSELinuxEnterpriseServer(SLES)12• Ubuntu14.04LTS
DB211.1hasalsoupdatedtheoperatingsystemsthatitsupports.Notethenewrequirementsinthefollowinglist.Supportfor32-bitserveroperatingsystemshasbeeneliminatedfornewinstancesinthisrelease,soWindows32-bitandLinux32-bitarenolongersupportedfortheservereditionsbutcontinuetobeavailableaspartofthedeveloperedition.32-bitclientscontinuetobesupportedhowever.
• Intel64-bito Windows7,8.1,10,WindowsServer2012R2o RedHatEnterpriseLinux(RHEL)6.7+,7.1+
Chapter1:Packaging,Installation,andAdministration 11
o SUSELinuxEnterpriseServer(SLES)11SP4+,12o Ubuntu14.04LTS
• AIXVersion7.1TL3SP5+• zLinux
o RedHatEnterpriseLinux(RHEL)7.1+o SUSELinuxEnterpriseServer(SLES)12
StartinginDB211.1,someoperatingsystems,Linuxdistributionsandarchitecturesarenolongersupported.CustomersneedtolookatalternativeplatformstoupgradetoDB211.1.ThefollowingarenolongerbeingsupportedinDB211.1.
• HP-UX,Solaris• PowerLinuxBE,InspurK-UX
However,customerswhoarecurrentlyonDB210.5willcontinuetoreceiveservicefortheseplatformsuntiltheEndofServicedateforDB210.5.
VirtualizationSupportDB211.1supportsanumberofvirtualizationenvironments.Virtualizationcanimprovetheutilizationofprocessor/machineresourcesandalsogivethecustomeranenvironmenttotestsystemswithouthavingtoinvestinentiresystems.ThefollowingisalistofvirtualizedenvironmentsthatDB2supports:
• IBMSystemzo IBMProcessorResource/SystemManagero z/VMandz/KVMonIBMSystemz
• IBMPowero IBMPowerVMandPowerKVMandIBMWorkload
PartitionsonIBMPowerSystems• LinuxX86-64Platforms
o RedHatKVMo SUSEKVM
• VMWareESXi• Dockercontainersupport–Linuxonly• Microsoft
o Hyper-V
Chapter1:Packaging,Installation,andAdministration 12
o MicrosoftWindowsAzureonx86-64WindowsPlatformsonly
• pureScalesupportonPowerVM/KVM,VMWare,andKVM
YouwillnotethatpureScaleissupportedinavirtualizedenvironmentandthatROCEAdapterscannowbevirtualizedinaVMWareenvironment(initially)sothatmultiplememberscansharethesamehardwareadapter.
UpgradeSupportUpgradingtoDB211.1issupportedfromDB210.5,DB210.1,orDB29.7.IfyouhaveanearlierversionofDB2,youmustfirstupgradetoDB210.5,DB210.1,orDB29.7beforeupgradingtoDB211.1.UpgradingtoaDB211.1non-rootinstallationissupportedfromaDB210.5,DB210.1,orDB29.7non-rootinstallation.UpgradingtoaDB211.1non-rootinstallationfromapre-DB211.1rootinstallationisnotsupported.InstancebitsizeisdeterminedbytheoperatingsystemwhereDB211.1isinstalled,andsupportfor32-bitkernelsand64-bitkernelshaschanged.UpgradingfromasystemwithmultiplecopiesofDB210.5,DB210.1,orDB29.7issupported.OnWindowsoperatingsystems,youmustbeawareoftherestrictionsoncoexistenceofpreviousversionsoftheDB2databaseproducts.Upgradingfromapartitioneddatabaseenvironmentwithmultipledatabasepartitionsissupported.Restoringfulldatabaseofflinebackupsfrompre-DB211.1copiesissupported.However,rollingforwardoflogsfromapreviouslevelisnotpossible.ReviewBackupandrestoreoperationsbetweendifferentoperatingsystemsandhardwareplatformsforcompletedetailsaboutupgradesupportusingtheRESTOREDATABASEcommand.
Chapter1:Packaging,Installation,andAdministration 13
AdministrationImprovementsDB211.1includesanumberofadministrativeenhancementsthatcustomerswillfinduseful.Fiveofthemthatarehighlightedhereinclude:
• RangePartitionTableReorganization• ADMIN_MOVE_TABLEimprovements• RemoteStorageOption• DB2HistoryFileBackup• BackupCompressionHardwareAcceleration
RangePartitionReorganization
ThemanageabilityoflargerangepartitionedtableshasbeenimprovedinDB211.1.AsinglepartitionofarangepartitionedtablecannowbereorganizedwiththeINPLACEoptionif:
• thetablehasnoglobalindex(i.e.non-partitionedindexes)• ONDATAPARTITIONisspecified
Thereorganizationcanonlyoccurononedatapartitionatatime,andthetablemustbeatleastthreepagesinsize.Thebiggestrestrictionofthisfeatureisthatthetablecannothaveanyglobalindexes.Ifaglobalindexexists,thereorgcommandwillnotrun.
ADMIN_MOVE_TABLE
TherearetwonewoptionsintheADMIN_MOVE_TABLEcommand:
• REPORT• TERM
TheREPORToptioncanbeusedtomonitortheprogressoftablemoves.Thecommandcalculatesasetofvaluestomonitortheprogressofsingleormultipletablemoves.ThefocusisontheCOPYandREPLAYphaseofarunningtablemove.TheREPORToptionrequiresatableschemaandnametogetinformationonaspecifictablemove.IfthesevaluesareleftblankorNULL,informationonalloutstandingtablemovesisretrieved.TheTERMoptioncanbeusedtoterminateatablemoveinprogress.TERMwillforceofftheapplicationrunningthetablemove,rollbackall
Chapter1:Packaging,Installation,andAdministration 14
opentransactionsandsetthetablemovetoawell-definedoperationalstatus.Fromhere,thetablemovecanbecancelledorcontinued.
RemoteStorage
DB211.1deliversmoreflexibilityandoptionsforacquiring,sharingandstoringdatafilesandbackupimages,byallowscustomerstouseremotestorageforseveralDB2utilities:
• INGEST,LOAD,BACKUP,andRESTORE
DB2supportsremotestorageusingstoragealiasesfor:
• IBM®SoftLayer®ObjectStorage• AmazonSimpleStorageService(S3)
DB2HistoryFileBackup
TheDB2historyfilecontainsinformationaboutlogfilearchivelocation,logfilechainetc.Ifyouusesnapshotbackupsyouwanttohavecurrentinformationaboutlogfilelocationtoperformpointintimerecovery(RECOVERcommand).RECOVERneedsacurrentversionofthehistoryfiletobeavailable.TheNOTABLESPACEbackupoptionallowsyoutocreateacurrentandconsistentbackupofthehistoryfileinaconvenientway.ANOTABLESPACEbackupdoesnotcontaintablespaces,onlythehistoryfile.Anotablespacebackupisusedtoquicklybackupthehistoryfileandcanbedoneatdifferentfrequencythanadatabasebackup.RestorethehistoryfilebyusingtheexistingHISTORYFILEoptionwiththeRESTOREDATABASEcommand.
HardwareBackupCompression
DB2backupandlogarchivecompressionnowsupporttheNX842hardwareacceleratoronPOWER7+andPOWER8processors.DB2BACKUPsrequiretheuseofaspecificNX842librarytotakeadvantageofthehardwareacceleration.backup database <dbname> compress comprlib libdb2nx842.a
TheregistryvariableDB2_BCKP_COMPRESSIONcanbesettoNX842forthedefaultBACKUPcommandtousethehardwareacceleration.
Chapter1:Packaging,Installation,andAdministration 15
Allowingforexistingcustomersthatareusingthebuilt-inDB2backupcompression(i.e.softwarecompression)tobenefitwithoutanychangestoscripts/jobs.Oncetheregistryvariableisset,usingthefollowingbackupcommandformatwillbenefitfromthehardwarecompressionaccelerator:backup database <dbname> compress
LogarchivecompressionisalsosupportedandcanbeconfiguredbyupdatingthedatabaseconfigurationparameterLOGARCHCOMPR1orLOGARCHCOMPR2toNX842:update database configuration for <dbname> using LOGARCHCOMPR1 NX842
Chapter2:DB2BLUEnhancements 17
DB2BLUEnhancements
TherehasbeensignificantenhancementsandcapabilitiesaddedtoDB2BLUtablesfromarchitecturetoperformanceimprovementsinthecolumnarengine.DB211.1addssupportforcolumn-organizedtablesinadatabasepartitionedenvironmentprovidedbyDB2DatabasePartitionFeature(DPF)allowingforpetabytescalingofBLUtablesinamassivelyparallelprocessing(MPP)architecture.InadditiontoMPPenablementofcolumnartables,DB211.1deliversadvancesinthecoreBLUAccelerationtechnologyforcolumn-organizedtables.TheseadvancesincludeNestedLoopJoin(NLJN)support,queryrewrite,fasterSQLMERGE,enhancementstomemorymanagement,furtherSIMDadvances,industryleadingparallelsort,improvedsortheaputilization,increasedSQLparallelismandwidevarietyofpushdownfunctionalityintotheBLUcolumnarengine.OtherenhancementsincludeBLUAccelerationsupportforIDENTITYandEXPRESSIONgeneratedcolumns,EuropeanLanguagesupportandNOTLOGGEDINITIALLYsupportforcolumn-organizedtables.Furtherinformationonsomeoftheseenhancementsarecoveredbelow.Theimprovementsnotonlyimproveperformanceofcolumnartables,butalsoincreaseconcurrentintra-parallelismdeliveringincreasedqueriesperhour,asseeninFigure4,DBInsightqueryperformance.
Figure4:DBInsightQueryPerformance
DB2 V10.5 FP5 DB2 V11.1QpH 703.85 955.82
0
200
400
600
800
1000
1200
Que
ries
Per H
our
Query Throughput BD Insights (800GB)
1.36x
Chapter2:DB2BLUEnhancements 18
MassivelyParallelProcessingArchitectureWithDB211.1,youcancreatecolumn-organizedtablesinapartitioneddatabaseenvironmentwiththeDatabasePartitioningFeatureofDB2.ThisallowsyoutoleverageBLUenhancementsinaMPPenvironment.InaDPFshared-nothingdatabase,dataispartitionedacrossmultipleserversorpartitionswitheachpartitionhavingitsowndedicatedresources–memory,processorsandstorage–toprocessdatalocallyandparallel.DB2allowsmultiplelogicalpartitionstobedeployedonthesamephysicalserver.Eachpartitionisresponsibleforasingleportion(i.e.partition)ofthedatainatableandthequeriesarebrokenupacrossallpartitions,withpartitionsworkinginparallel,leveragingthecomputepowerofallserversforfastqueryprocessing.Thepartitioningofthetableisdoneusingadistributionkeytodistributetabledataacrossasetofdatabasepartitions.Adistributionkeyconsistsofoneormorecolumnsofthetable.Tocreateapartitionedcolumn-organizedtableisthesameasforrow-organizedtables,viatheDISTRIBUTEBYclauseoftheCREATETABLEstatement,asbelow. CREATE TABLE sales(…) ORGANIZE BY COLUMN DISTRIBUTE BY (C1,C2)
IfDISTRIBUTEBYHASHisspecified,thedistributionkeysarethekeysexplicitlyincludedinthecolumnlistfollowingtheHASHkeyword(C1andC2intheexampleabove).Thecolumnsspecifiedmustbeasubsetofanyuniqueorprimarykeyonthetable.ImplicitselectionofdistributionkeyoccursiftheDISTRIBUTEBYHASHclauseisomittedandthetableisdefinedinadatabasepartitioninggroupwithmultiplepartitionsorifDISTRIBUTEBYRANDOMclauseisused.TherearetwomethodsthatDB2usestodistributedatawhenDISTRIBUTEBYRANDOMisspecified:
• Randombyunique:Ifthetableincludesauniqueorprimarykey,thecolumnsoftheuniqueorprimarykeyareusedasthedistributionkey.
• Randombygeneration:DB2willincludeanIMPLICITLYHIDDENcolumninthetabletogenerateandstoreageneratedvaluetouseinthehashingfunction.Thevalueofthecolumnwillbe
Chapter2:DB2BLUEnhancements 19
automaticallygeneratedasnewrowsareaddedtothetable.Bydefault,thecolumnnameisRANDOM_DISTRIBUTION_KEY.Ifitcollideswiththeexistingcolumn,anon-conflictingnamewillbegeneratedbyDB2.
FastCommunicationManagerTheFastCommunicationManager(FCM)communicationinfrastructureofDB2DPFhasbeenoptimizedforcolumnardataexchange.Thus,dataexchangeduringdistributedjoinsandaggregationprocessingoccursentirelywithintheBLUruntimeinnativecolumnarformat.Thefollowingspecificenhancementspertaintocolumn-organizedtablesinapartitionedenvironment:
• MPPawarequeryplanningandoptimizationforthecolumnvectorprocessingengine
• Anoptimizedvectorformatforcolumnardataexchangebetweenpartitions
• Acommontabledictionaryallowingdatatoremaincompressedacrossthenetwork
• Anoptimizedcommunicationsinfrastructuredesignedformulti-coreparallelism
BuiltonDB2'sDPFcapabilitiesmeansBLUtablesareabletoscale-outwellbeyondthatofasingleserverforimprovedresponsetime,handlinglargerdatasetsinthemulti-petabyterangeandmeansthatexistingDB2MPPdatawarehousescaneasilyleveragethein-memoryoptimizedcolumnartechnology.
Chapter2:DB2BLUEnhancements 20
FasterSQLMERGEprocessingTheMERGEstatementupdatesaTableorViewandusesdatafromasource(resultofatablereference).Rowsinthetargettablethatmatchthesourcecanbedeletedorupdatedasspecifiedandrowsthatdonotexistinthetargetcanbeinserted.Whenthetargetisaview,updates,deletesorinsertsoccurtotherowinthetable(s)onwhichtheviewisdefined.ThefollowingSQLisanexampleofaMERGEstatement.MERGE INTO archive ar USING (SELECT activity, description FROM activities) ac ON (ar.activity = ac.activity) WHEN MATCHED THEN UPDATE SET description = ac.description WHEN NOT MATCHED THEN INSERT (activity, description) VALUES (ac.activity, ac.description);
MERGEhasalreadybeensupportedagainstcolumnar-originatedtables,howeverDB211.1deliversperformanceimprovementsbypushingtheMERGEoperationdownintothecolumnarenginewhichmeansitoperatesinnativecolumnarformat.Thisalongwithafastersortengineimprovesperformancealso.
NestedLoopJoinSupportDB211.1deliversNestedLoopJoinsupportforcolumnar-orientedtables.ANestedLoopJoinisoneofthreetechniquesthattheDB2Optimizerusestoperformajoinbetweentwotables.HASHjoinistheotherjointechniquethatalreadysupportscolumnar-orientedtables.Anested-loopjoinisperformedbyscanningtheinnertableforeachaccessedrowoftheoutertableorbyperforminganindexlookupontheinnertableforeachaccessedrowoftheoutertable.NestedLoopJoinsupportoncolumnar-orientedtablesmeansthatevaluationsandactivitiescanbedonenativelyinBLU,therebyimprovingperformancedramatically.Thereisnoexternalenablementorsettingtobenefitfromthisnewcapability.ThisperformancebenefitisrealizedautomaticallywhentheDB2OptimizerselectsaNestedLoopJoinasthejointechnique.
Chapter2:DB2BLUEnhancements 21
SortProcessingEnhancementsDB211.1includessortinnovations,PARADIS,fromtheIBMTJWatsonResearchdivision.PARADISisanefficientparallelalgorithmforin-placeRadixsort.Inadditiontodeliveryindustryleadingparallelsort,DB211.1isalsoabletosortcompressedandencodeddata.ThisallowsmoreefficientandimprovedperformanceasprocessingisperformedwithintheBLUengine.DB211.1BLUsortenhancementscanincreaseBLUperformancebyasmuchas13.9x.PerformanceisbasedonmeasurementsandprojectionsusingstandardIBMbenchmarksinacontrolledenvironment.Theactualthroughputorperformancethatanyuserwillexperiencewillvarydependinguponmanyfactors,includingconsiderationssuchastheamountofmultiprogrammingintheuser'sjobstream,theI/Oconfiguration,thestorageconfiguration,andtheworkloadprocessed.Therefore,noassurancecanbegiventhatanindividualuserwillachieveresultslikethosestatedhere.MoredetailsonPARADIScanbefoundintheIBMResearchPaper:http://www.vldb.org/pvldb/vol8/p1518-cho.pdfTheDB2Explainfacilitycanbeutilizedtoexaminetheassociatedqueryaccessplans.TheaccessplanswillshowwhatpartoftheprocessingisdoneintherowenginevstheBLUengine.Allpartsbelowthe"CTQ"evaluatoraredoneintheBLUengine.InFigure5,wecanseethatbypushingdowntheSORTevaluatorwekeepthedataprocessinginBLUonlywhichcontributestothe4xspeedupobservedforthisquery.
Chapter2:DB2BLUEnhancements 22
Figure5:ColumnarEngineNativeSORTExample
QueryRewriteEnhancementsTherehavebeenenhancementstoQueryRewritewhendealingwithCorrelatedSubqueries.Theenhancementincludesthedecorrelationofcorrelatedsubqueries.Figure6isanexampleofacorrelatedsubquery.TheprocessingissuchthatthequerymatchestherowsintheLookuptableforaparticularKeyintheFacttable.Onceamatchisfound,weselectallrowsfromLookuptableforthatparticularKey.ThisprocessrepeatsforeachrowintheFacttable.
Figure6:CorrelatedSubqueryExample
Chapter2:DB2BLUEnhancements 23
Decorrelationallowsforbulkjoinprocessing,viaHASHjoinexploitation,insteadofarowatatime.Thereareseveralscenarioswhicharedecorrelatedsuchas:
• UPDATEstatementswithcorrelatedsubqueriesintheSETstatement
• CASEexpressionscontainingcorrelatedsubqueries• ORpredicatescontainingcorrelatedsubqueries• SubqueriescontainingGROUPBY
ThecorrespondingDB2QueryRewritegenerateddecorrelatedqueryforthecorrelatedsubqueryexampleinFigure6isasfollows:SELECT F.key FROM fact F LEFT OUTER JOIN lookup L ON F.key = L.key WHERE F.flag = 1 OR F.key IS NOT NULL;
Anested-loopjoinisthejoinmethodthatbestdealswithtruecorrelatedsubqueries,whichcannowalsobechosenasofDB211.1forquerieswheredecorrelationisnotpossible.
Push-downofOLAPfunctionsSQLOLAPfunctionsarepushed-downintotheBLUengineforprocessingandaredeliveredfordeeperin-databaseanalyticswithcolumn-organizedtables.Theseimprovementsincludefunctionssuchas:
• RANK,ROW_NUMBER,DENSE_RANK• FIRST_VALUE• AVG,COUNT,COUNT_BIG• MAX,MIN,SUM• FIRST_VALUE,RATIO_TO_REPORT• ROWS/RANGEBETWEENUNBOUNDEDPRECEDINGAND
UNBOUNDEDFOLLOWING• ROWSBETWEENUNBOUNDEDPRECEDINGANDCURENTROW
SQLFunctionsOptimizedforBLUSomeSQLsupportforBLUColumnartablesthatweremissinginearlierversionhasnowbeenincludedinDB211.1.WhichmeansincreasedperformanceasprocessingoccursnativelyintheBLUengineforcolumn-
Chapter2:DB2BLUEnhancements 24
organizedtables.Supportforcolumn-organizedtablesinclude,widerowsupport,allowingyoutocreateatablewhereitsrowlengthcanexceedthemaximumrecordlengthforthepagesizeofthetablespace.Othersupportincludes,logicalcharactersupport(CODEUNITS32),EuropeanLanguagesupport(Codepage819),IDENTITYandEXPRESSIONgeneratedcolumns,NOTLOGGEDINITIALLY,RowandColumnAccessControl(RCAC)andDeclaredGlobalTemporaryTable(DGTT)support,includingparallelinsertintonot-loggedDGTTfromBLUsource.
Inaddition,thereareaseriesofString,Numeric,Date&Timefunctionsoptimizedforcolumn-organizedtablesasnotedbelow:
• StringFunctionso LPAD/RPADo TO_CHARo INITCAP
• NumericFunctionso POWER,EXP,LOG10,LNo TO_NUMBERo MODo SIN,COS,TAN,COT,ASIN,ACOS,ATANo TRUNCATE
• DateandTimeFunctionso TO_DATEo MONTHNAME,DAYNAME
AutomaticDictionaryCreationDB211.1improvestheautomaticdictionarycreationofcolumn-organizedtables.ThecompressiondictionaryoccursduringINSERT,INGEST,IMPORTandisnowavailableearlierinprocessing.Theautomaticdictionarycreationimprovementsprovidebettercompressionrates,improvedqueryperformance,increasedthroughputforhighconcurrencyworkloadsandhigherPCTENCODED(percentencoded)valuesinSYSCAT.COLUMNS.
Chapter3:pureScaleContinuousAvailability 26
ContinuousAvailability
DB211.1introducessomeadditionalcapabilitiestoDB2pureScalethatwillprovidesimplificationwithinstallation&configuration,virtualization,workloadmanagement,manageabilityandadditionalDisasterRecovery(DR)configurations.DB2pureScalehasbeenprovidingscalabilityandcontinuousavailabilitythroughplannedandunplannedeventssinceversion9.8.TransparentscalabilitymeansthatasMembersareaddedtotheDB2pureScaleInstance,applicationscanscalewithoutanychangesorawarenessattheapplicationlevel.Bothscalabilityandavailabilityhavebeendesignedintothearchitectureandpossibleusingthehighlyreliableclustercachingfacility(CF)-capabilityportedfromthegoldstandardofavailability,DB2zOSSysplex.BeforewegetintothenewcapabilitiesdeliveredinDB211.1,let'stakeaquickreviewofthepureScalearchitecture.
pureScaleArchitectureThearchitectureofaDB2pureScalecluster(alsoreferredtoasapureScaleinstance),asshowninFigure7,isanactive/activedata-sharingenvironmentinwhichmultiplenodes–calledDB2members–handlethetransactionalworkloadofthesystemandtheyalsohaveequalandsharedaccesstoasinglecopyofthedatabaseondisk.AmemberisalogicalcomponentandessentiallyasingleinstanceofDB2Serverwithitsownbufferpool,memoryregion,logfiles,etc.ThesinglecopyofthedatabaseaswellasthelogfilesforeachmemberarestoredwithinthesamesharedstorageusingthehighlyscalableIBMSpectrumScale(GPFS)filesystem.Clientscanconnectintoanyofthemembersandneedonlyknowofoneastherearedefaultworkloadbalancingcapabilitiesthatwillautomaticallydistributetheworkloadacrossalltheactivemembers.Ifamemberfails,clientconnectionsareautomaticallyreroutedtohealthymembers.Acriticalelementofthearchitectureisthatthereisnomembertomembercommunications.ThememberscommunicatewiththeclusterCachingFacility(CF)forcentralizedlockingandcachemanagement.Foravailability,thereis
Chapter3:pureScaleContinuousAvailability 27
supportfortwoCFs,aprimaryandstandby.Thiscommunicationtakesplaceoverahighspeed,lowlatencyinterconnectviatheRDMA(RemoteDirectMemoryAccess)protocol.Alternatively,theuseofTCP/IPsocketsfortheinterconnectinsteadofanRDMA-basedinterconnectisalsosupported.TheDB2ClusterServices(CS)providesintegratedfailuredetection,recoveryautomationandtheclusteredfilesystem.AllthecomponentsarefullyintegratedintoDB2pureScale,inthattheyareallinstalledtogetheraspartofasingleinstallationprocess,theyareconfiguredaspartofinstallation,andtheyaremanagedandmaintainedtogetherallwithinpureScale.
Figure7:pureScaleArchitecture
DeploymentsDB2pureScalearchitectureallowsforhighlyflexibledeploymenttopologiesduetothelogicalaspectsofthemembersandCFs.ThisallowsforthemembersandCFstobedeployedinanynumberofcombinations,beitvirtual(LPAR,VM,KVM)orphysicalservers.Followingaresome
Clients
Shared Storage
Database
Logs Logs LogsLogs
Cluster Interconnect
MemberCS
MemberCS
MemberCS
MemberCS
Primary CF
CFCSSecondary CF
CFCS
DB2 pureScale Cluster (Instance)
Chapter3:pureScaleContinuousAvailability 28
exampledeploymentoptionsformembersandCF.Fortheexamples,wereferto'server'whichrepresentseithervirtualorphysicalimplementation.TheminimalrequirementforpureScaleisonememberandoneCF.However,therecommendeddeploymentconfiguration,asshowninFigure8,istwomembersandtwoCFsrunningacrosstwoservers.AsthisprovidesthecontinuouslyavailableconfigurationforwhichpureScaleisarchitected.
Figure8:TwoServers-TwoMembersTwoCFs
WecantakethesametwomembersandtwoCFconfigurationanddeployitoverthreeservers,asinFigure9,orfourserversasinFigure10.
Figure9:ThreeServers-TwoMembersTwoCFs
Figure10:FourServers-TwoMembersTwoCFs
Withthreeservers,youcanalsodeploythreemembersandtwoCFs,asshowninFigure11andwithfourservers,youcanhavefourmembersandtwoCFs(Figure12):
Member
CFp
Member
CFs
Chapter3:pureScaleContinuousAvailability 29
Figure11:ThreeServers-ThreeMembersTwoCFs
Figure12:FourServers-FourMembersTwoCFs
Thedecisiononwhichdeploymenttoutilizecomesdowntoresourcesontheserverandimpactonserverfailure.TakingFigure8andFigure9asanexample,aserverfailureinFigure8wouldresultinonememberandoneCFgoingofflineandtheworkloadcontinuesontheexistingmemberandCF.However,inFigure9withthreeservers,afailurewouldresultineitheraMemberorCFgoingofflineorbothamemberandCF.Ifit'sonlytheprimaryCF,thenbothmembercontinuetoprocesstransactionswiththesameprocessingcapacity.IfthememberorthememberandCFserverfail,thentheworkloadcontinuesontheexistingmemberandCFthesameastheFigure8failureexample.AsyouincreasethenumberofmembersyouincreasethescalabilityprovidedbyDB2pureScaleandminimizeimpactduringafailurescenario.Inadditiontothefullyactive/activeDB2pureScaledeployments,youcanalsodeploypureScaleusingan'active/passivelicensingmodel',asshownearlierinFigure3inChapter1.The'active/passive'isfromtheapplicationworkloadperspectiveandnotthecontinuouslyavailableaspect,asthisremains.Allapplicationworkloadsaredirectedtotheprimaryactivemember(s),sometimesreferredtoasthe"primary"memberwhileutilitiesandadministrativetasksareallowedonthesecondarymember.Thisisconfigurationisidealforoff-loadingbackupsandotheradministrativetasksfromprimarymembersaswellasensuringcontinuousavailableintheeventofamemberfailure.
Chapter3:pureScaleContinuousAvailability 30
VirtualizationDB2pureScalehassupportedavarietyofvirtualizationtechnologiessuchasIBMPowerVM,PowerKVM,RedHatKVMandVMWareESXi.DB211.1enhancesthevirtualizationsupportforVMWarebynowdeliveringRDMAoverConvergedEthernet(RoCE)supportinVMWarethroughRoCESingle-RootI/OVirtualization(SR-IOV)forRHEL7.2.SR-IOVstandardenablesonePCIExpress(PCIe)adaptertobepresentedasmultipleseparatelogicaldevices(VirtualFunctions)tovirtualmachines.ThisallowsthevirtualmachinestorunnativeRoCEandachievenearwirespeedperformance.ThiscapabilitycanbeenablewithDB2pureScalewhenusingMellanoxConnectX-3/ConnectX-3Pro/ConnectX-3VPIadaptersforEthernet.
InstallationTheDB2pureScaleinstallationprocessinDB211.1hasbeensimplifiedwithsmarterdefaults,intuitiveoptions,parallelandquickpre-deploymentvalidationacrosshosts.Allowinguserstobeupandrunningwithinhoursofstartingtheinstallationprocess.Therehasbeenre-engineeringeffortsininstallwhichreducesthecomplexitybyatleast40%forsocketsandtakesa30-stepnativeGPFSsetupdowntoasimple4-stepDB2installprocess.DB211.1providessimplificationinthesetupandconfigurationofGPFSreplication.GPFSreplicationisusedforcustomerswhowanttoprovideprotectionagainstacompleteSANstoragesubsystemfailure.TheGPFSreplication,Figure13,iscompletedwithafewdb2clustercommandsandisdefinedacrosstwoseparatestoragesubsystems.
Chapter3:pureScaleContinuousAvailability 31
Figure13:FourServers-FourMembersTwoCFsandGPFSReplication
WorkloadBalancingTherehasbeenadditionalcapabilityincludedinDB2pureScalewithworkloadbalancingwhenusingmembersubsets.MembersubsetsallowclientstoconnectandbebalancedacrossasubsetofthememberscomprisingtheDB2pureScaleinstance.DB211.1allowsyoutoexplicitlydefinethememberstobeusedasalternatesintheeventofprimarymemberfailures.AnalternatemembercanbeanyothermemberofthepureScaleInstance.Analternatememberofasubsetbehavesasanyothermemberandprocessingtransactionsbutisconsidereddormantforthatsubsetuntilafailureoccurstoaprimarymemberofthesubset.Uponafailure,thealternatememberwillbeassignedworkloadsfortherespectivesubsetinadditiontootherworkloadsthathasbeenassigned/runningonthatmember.DB2willattempttomaintainthesamenumberofprimarymembersintherespectivesubset,ifthereareenoughalternatesavailable.
Chapter3:pureScaleContinuousAvailability 32
AlternatemembersareassignedtoasubsetusingthenewlyaddedFAILOVER_PRIORITYoptionoftheWLM_ALTER_MEMBER_SUBSETDB2procedure.Aspartofthisenhancement,anewcolumn,FAILOVER_PRIORITY,wasaddedtotheSYSCAT.MEMBERSUBSETMEMBERSsystemcatalogview,asseeninTable1,toallowyoutoquicklyseewhichmembersareprimary,howmanyandorderingofalternatemembersforeachsubsetdefined.Table1:SYSCAT.MEMBERSUBSETMEMBERSCatalogView
Failoverpriorityhasvaluesfrom0-254.Memberswithfailoverpriorityof0,thedefaultifnotspecified,areconsideredprimarymembersofthesubset.Memberswithfailoverpriorityof1-254areconsideredalternatemembersofthesubset.Ifaprimarymemberfails,analternatememberisautomaticallyselectedtoservicethemembersubsetworkload.Thedeterminationofwhichalternatemembertoassigntothesubsetismadebychoosingthealternatememberwiththelowestfailoverpriority.Iftwoalternatemembershavethesameprioritydefined,thenthealternatememberwiththelowestmembernumberischosen.Ifaprimarymemberfails,analternatememberisautomaticallyselectedtoservicethemembersubsetworkload,fromalowerfailoverpriority.CreatingandalteringmembersubsetsisaDB2serversidedynamicallymanagedonlineoperation.
Assumingwealreadyhaveamembersubsetovermembers0and1calledBATCH,thefollowingexampleassignsmember2and4asanalternatememberwithfailoverpriorityof1and2,respectively,tothesubsetwithname'BATCH':CALL SYSPROC.WLM_ALTER_MEMBER_SUBSET ('BATCH', NULL, '(ADD 2 FAILOVER_PRIORITY 1, ADD 4 FAILOVER PRIORITY 2)');
Chapter3:pureScaleContinuousAvailability 33
Thefollowingquerycanbeusedtoviewthealternatememberinformationforsubsets: SELECT CAST(SUBSETNAME AS CHAR(10)) AS SUBSETNAME, MEMBER, FAILOVER_PRIORITY FROM SYSCAT.MEMBERSUBSETS ms, SYSCAT.MEMBERSUBSETMEMBERS msm WHERE ms.SUBSETID = msm.SUBSETID; ResultoftheabovequeryfortheBATCHexamplegiven:SUBSETNAME MEMBER FAILOVER_PRIORITY ---------- ------ ----------------- BATCH 0 0 BATCH 1 0 BATCH 2 1 BATCH 3 2 Inadditiontothecatalogview,thedb2pdcommandcanbeusedtoseethealternatememberinformation:db2pd –db <dbname> –membersubsetstatus TopreservethepriorDB211.1behaviorofmembersubsets,duringtheupgrade,thevalueofFAILOVER_PRIORITYattributeissetto0forallmembers.PriormembersubsetbehaviorcanalsobemaintainedfornewlycreatedsubsetsinDB211.1pureScale,bysettingaFAILOVER_PRIORTYof0forallthemembersintherespectivesubset.
TheDB211.1enhancementtomembersubsetsallowscustomerscurrentlyusingclientaffinitytomovetheirconfigurationtousingmembersubsetsandfailoverprioritysothattheycanexploitthenewbenefitssuchasdynamicserversidereconfiguration.SimplificationtosettingupclientaffinitywithhavingcontrolattheServervsclient–noneedtodb2dsdriver.cfg.
FailoverbehaviorDuringaplannedorunplannedeventtoamemberofasubset,theapplicationwillautomaticallyberoutedtothealternatememberwiththelowestfailoverpriorityandlowestmembernumber,whenmultiplealternatesaredefinedwiththesamefailoverpriority.Shouldasubsequenteventoccur,applicationswillberoutedtothenextalternatemember.
Chapter3:pureScaleContinuousAvailability 34
Ifthemembersubsetisdefinedasaninclusivesubsetandtherearenotenoughalternatememberstofailover,thentheapplicationusestheotheravailablemembersintheclusterthatarenotpartofthesubset
FailbackbehaviorOnceaprimarymemberoramemberwithlowerfailoverprioritybecomesavailable,applicationsfromthememberwiththehighestfailoverpriorityfailback.Thismeansthatallnewapplicationsconnecttothememberwithlowerprioritythatbecameavailableandexistingapplicationsconnectedtothealternatemember,withhigherfailoverpriority,cancompleteprocessingbeforefailingbacktothelowerprioritymember.
Manageability
Pre-Installation
Aspartoftheimprovementstoinstallation,DB211.1addsanewoption,adapterlist,tothedb2prereqcheckcommand.ThisoptionisusedtoverifythenetworkconnectivitythatallthehostsintheDB2pureScaleconfigurationarepingableusingRDMA.Theoptionrequiresaninputfiletobespecifiedwhichcontainsthelistofhostname,netnameandadapternamesforeachhosttobeverified.db2prereqcheck –adapter_list <adapter_list_filename>
Where<adapter_list_filename>specifiesthefilenamethatcontainsthelistofhostname,netname,andadapternamesforeachofthehosttobeverified.Theinputfilemusthavethefollowingformat:Table2:Adapterlistfilenameexample
Linesthatareprecededby#areconsideredcommentsandskipped.
#Hostname Netname Interface-Adapterhostname1 netname1 devicename-1hostname2 netname2 devicename-2hostname3 netname3 devicename-3
Chapter3:pureScaleContinuousAvailability 35
Post-Installation
Inadditiontotheenhancementstopre-installationabove,DB211.1deliversapost-installationunifiedhealthcheckoptiontothedb2clustercommand.Thedb2cluster–verifycommandperformsacomprehensivelistofcheckstovalidatethehealthoftheDB2pureScalecluster.Accordingly,analertisraisedforeachfailedcriterionandisdisplayedintheinstancemonitoringcommanddb2instance-list.Thevalidationsperformedinclude,butarenotlimitedto,thefollowing:
• ConfigurationsettingsinpeerdomainandGPFScluster• CommunicationsbetweenmembersandCFs(includingRDMA)• Replicationsettingforeachfilesystem• Statusofeachdiskinthefilesystem
DisasterRecoveryTherehavebeenenhancementstoDB2pureScaleinHADR(HighAvailabilityDisasterRecovery)andGDPC(GeographicallyDispersedDB2pureScaleCluster).BothprovidezerodatalossDisasterRecoverysolutionsintegratedanddeliveredaspartofDB2pureScale.
HADR
DB211.1enablesHADRsynchronous(SYNC)andnear-synchronous(NEARSYNC)supporttopureScaledeployments.ThisenhancementcombinesthecontinuousavailabilityoftheDB2pureScalefeaturewiththerobustdisasterrecoverycapabilitiesofHADRprovidinganintegratedzerodataloss(i.e.RPO=0)disasterrecoverysolution.
Figure14:HADRandpureScale
Chapter3:pureScaleContinuousAvailability 36
ThecompleteHADRfeaturessupportedwithpureScaleinDB211.1include:
• SYNC,NEARSYNC,ASYNCandSUPERASYNCmodes• Timedelayedapply• Logspooling• Bothnon-forced(roleswitch)andforced(failover)takeovers
GeographicallyDispersedDB2pureScaleCluster(GDPC)
DB211.1addsimprovedhighavailabilityforGeographicallydispersedDB2pureScaleclusters(GDPC)forbothRoCE&TCP/IP.pureScaleallowsformultipleadapterportspermemberandCFtosupporthigherbandwidthandimprovedredundancyattheadapterlevel.
Figure15:GeographicallyDispersedDB2pureScaleCluster
Inaddition,dualswitchescanbeconfiguredateachsiteinaGDPCconfigurationtoprovidegreateravailabilityintheeventofaswitchfailure.
Chapter3:pureScaleContinuousAvailability 37
InDB211.1apureScaleGDPCconfigurationexhibitstheequivalentfailurebehaviortoanon-GDPCpureScalecluster.Inaddition,thefour-switchconfigurationcansustainadditionalconcurrentfailurescenarios,suchasanytwoor3switchfailures,withoutanoutage.
Chapter4:CompatibilityFeatures 39
CompatibilityFeatures Movingfromonedatabasevendortoanothercansometimesbedifficultduetosyntaxdifferencesbetweendatatypes,functions,andlanguageelements.DB2alreadyhasahighdegreeofcompatibilitywithOraclePLSQLalongwithsomeoftheOracledatatypes.DB211.1introducessomeadditionaldatatypeandfunctioncompatibilitythatwillreducethemigrationeffortrequiredwhenportingfromothersystems.TherearesomespecificfeatureswithinDB2thataretargetedatNetezzaSQLandthatisdiscussedinaseparatesection.
OuterJoinOperatorDB2allowstheuseoftheOracleouter-joinoperatorwhenOraclecompatibilityisturnedonwithinadatabase.InDB211.1,theouterjoinoperatorisavailablebydefaultanddoesnotrequiretheDBAtoturnonOraclecompatibility.DB2supportsstandardjoinsyntaxforLEFTandRIGHTOUTERJOINS.However,thereisproprietarysyntaxusedbyOracleemployingakeyword:"(+)"tomarkthe"null-producing"columnreferencethatprecedesitinanimplicitjoinnotation.The(+)appearsintheWHEREclauseandreferstoacolumnoftheinnertableinaleftouterjoin.
Forinstance:SELECT * FROM T1, T2 WHERE T1.C1 = T2.C2 (+);
Isthesameas:SELECT * FROM T1 LEFT OUTER JOIN T2 ON T1.C1 = T2.C2;
Inthefollowingexample,wegetalistofdepartmentsandtheiremployees,aswellasthenamesofdepartmentswhohavenoemployees.
ThisexampleusesthestandardDB2outerjoinsyntax.SELECT DEPTNAME, LASTNAME FROM DEPARTMENT D LEFT OUTER JOIN EMPLOYEE E ON D.DEPTNO = E.WORKDEPT;
Chapter4:CompatibilityFeatures 40
Theresultsfromthequeryare:DEPTNAME LASTNAME -------------------- --------------- SPIFFY COMPUTER SERV HAAS PLANNING THOMPSON INFORMATION CENTER KWAN SUPPORT SERVICES GEYER MANUFACTURING SYSTEM STERN ADMINISTRATION SYSTE PULASKI OPERATIONS HENDERSON SOFTWARE SUPPORT SPENSER SPIFFY COMPUTER SERV LUCCHESSI SPIFFY COMPUTER SERV O'CONNELL
Thisexampleworksinthesamemannerasthepreviousone,butusesthe"(+)"signsyntax.TheformatisalotsimplertorememberthanOUTERJOINsyntax,butitisnotpartoftheSQLstandard.SELECT DEPTNAME, LASTNAME FROM DEPARTMENT D, EMPLOYEE E WHERE D.DEPTNO = E.WORKDEPT (+);
CHARdatatypesizeincreaseTheCHARdatatypewaslimitedto254charactersinpriorreleasesofDB2.InDB211.1,thelimithasbeenincreasedto255characterstobringitinlinewithotherSQLimplementations. CREATE TABLE LONGER_CHAR ( NAME CHAR(255) );
BinaryDataTypesDB211.1introducestwonewbinarydatatypes:BINARYandVARBINARY.Thesetwodatatypescancontainanycombinationofcharactersorbinaryvaluesandarenotaffectedbythecodepageoftheserverthatthevaluesarestoredon.ABINARYdatatypeisfixedandcanhaveamaximumlengthof255bytes,whileaVARBINARYcolumncancontainupto32,672bytes.EachofthesedatatypesiscompatiblewithcolumnscreatedwiththeFORBITDATAkeyword.
Chapter4:CompatibilityFeatures 41
TheBINARYdatatypewillreducetheamountofconversionrequiredfromotherdatabases.AlthoughbinarydatawassupportedwiththeFORBITDATAclauseonacharactercolumn,itrequiredmanualDDLchangeswhenmigratingatabledefinition.Thisexampleshowsthecreationofthethreetypesofbinarydatatypes.CREATE TABLE HEXEY ( AUDIO_SHORT BINARY(255), AUDIO_LONG VARBINARY(1024), AUDIO_CHAR VARCHAR(255) FOR BIT DATA );
InsertingdataintoabinarycolumncanbedoneusingBINARYfunctions,ortheuseofX'xxxx'modifierswhenusingtheVALUEclause.Forfixedstrings,youusetheX'00'formattospecifyabinaryvalueandBX'00'forvariablelengthbinarystrings.Forinstance,thefollowingSQLwillinsertdataintotheprevioustablethatwascreated.INSERT INTO HEXEY VALUES (BINARY('Hello there'), BX'2433A5D5C1', VARCHAR_BIT_FORMAT(HEX('Hello there')));
HandlingbinarydatawithaFORBITDATAcolumnwassometimestedious,sotheBINARYcolumnswillmakecodingalittlesimpler.Youcancompareandassignvaluesbetweenanyofthesetypesofcolumns.ThenextSQLstatementwillupdatetheAUDIO_CHARcolumnwiththecontentsoftheAUDIO_SHORTcolumn.ThentheSQLwilltesttomakesuretheyarethesamevalue.UPDATE HEXEY SET AUDIO_CHAR = AUDIO_SHORT; SELECT COUNT(*) FROM HEXEY WHERE AUDIO_SHORT = AUDIO_CHAR; 1 ----------- 1
SynonymDatatypesDB2hasthestandarddatatypesthatmostdevelopersarefamiliarwith,likeCHAR,INTEGER,andDECIMAL.ThereareotherSQLimplementations
Chapter4:CompatibilityFeatures 42
thatusedifferentnamesforthesedatatypes,soDB211.1nowallowsthesedatatypesassynonymsforthebasetypes.Thesenewdatatypesare:Table3:DataTypeSynonyms
Type DB2EquivalentINT2 SMALLINTINT4 INTEGERINT8 BIGINTFLOAT4 REALFLOAT8 FLOATThefollowingSQLwillcreateatablewiththesedatatypes.CREATE TABLE SYNONYM_EMPLOYEE ( NAME VARCHAR(20), SALARY INT4, BONUS INT2, COMMISSION INT8, COMMISSION_RATE FLOAT4, BONUS_RATE FLOAT8 );
Whenyoucreateatablewiththeseotherdatatypes,DB2doesnotusethese"types"inthesystemcatalog.WhatDB2willdoisusetheDB2typeinsteadofthesesynonymtypes.Ifyoudescribethecontentsofatable,youwillseetheDB2typesdisplayed,notthesesynonymtypes.DESCRIBE TABLE SYNONYM_EMPLOYEE; COLNAME TYPESCHEMA TYPENAME LENGTH SCALE NULLABLE --------------- ---------- -------- ------ ----- -------- NAME SYSIBM VARCHAR 20 0 Y SALARY SYSIBM INTEGER 4 0 Y BONUS SYSIBM SMALLINT 2 0 Y COMMISSION SYSIBM BIGINT 8 0 Y COMMISSION_RATE SYSIBM REAL 4 0 Y BONUS_RATE SYSIBM DOUBLE 8 0 Y
FunctionNameCompatibilityDB2hasawealthofbuilt-infunctionsthatareequivalenttothoseusedbyotherdatabasesystems,butwithadifferentname.InDB211.1,thesealternatefunctionnamesaremappedtoanequivalentDB2functionsothatnore-writeofthefunctionnamerequired.
Chapter4:CompatibilityFeatures 43
ThefollowingtableliststhefunctionnamesandtheequivalentDB2functionthattheyaremappedto.Table4:FunctionSynonyms
Function DB2EquivalentCOVAR_POP COVARIANCESTDDEV_POP STDDEVVAR_POP VARIANCEVAR_SAMP VARIANCE_SAMPISNULL ISNULLNOTNULL ISNOTNULLLOG LNRANDOM RANDSTRPOS POSSTRSTRLEFT LEFTSTRRIGHT RIGHTBPCHAR VARCHAR(Castingfunction)DISTRIBUTEON DISTRIBUTEBY
Chapter5:NetezzaCompatibility 45
NetezzaCompatibility
DB2providesfeaturesthatenableapplicationsthatwerewrittenforaNetezzaPerformanceServer(NPS)databasetobeusedagainstaDB2database.TheSQL_COMPATglobalvariableisusedtoactivatethefollowingoptionalNPScompatibilityfeatures:
• Double-dotnotation-WhenoperatinginNPScompatibilitymode,youcanusedouble-dotnotationtospecifyadatabaseobject.
• TRANSLATEparametersyntax-ThesyntaxoftheTRANSLATEparameterdependsonwhetherNPScompatibilitymodeisbeingused.
• Operators-WhichsymbolsareusedtorepresentoperatorsinexpressionsdependsonwhetherNPScompatibilitymodeisbeingused.
• GroupingbySELECTclausecolumns-WhenoperatinginNPScompatibilitymode,youcanspecifytheordinalpositionorexposednameofaSELECTclausecolumnwhengroupingtheresultsofaquery.
• RoutineswritteninNZPLSQL-WhenoperatinginNPScompatibilitymode,theNZPLSQLlanguagecanbeusedinadditiontotheSQLPLlanguage.
SpecialOperatorCharactersDB2andNetezzabehavedifferentlywhenusingspecialoperatorcharacters.Thetablebelowliststhespecialcharactersandhoweachproductinterpretsthem.Table5:DB2SpecialCharacters
Character DB2Equivalent NetezzaI/&/~ LogicalOR,AND,Complement LogicalOR,AND,Complement^ LogicalXOR Power(also**)# None LogicalXOR
Chapter5:NetezzaCompatibility 46
ThefollowingSQLwilldisplaytheDB2interpretationofthespecialcharacters.SET SQL_COMPAT = 'DB2'; WITH SPECIAL(OP, DESCRIPTION, EXAMPLE, RESULT) AS ( VALUES (' ^ ','XOR ', '2 ^ 3 ', 2 ^ 3), (' # ','NONE ', ' ',0) ) SELECT * FROM SPECIAL; OP DESCRIPTION EXAMPLE RESULT ---- ----------- --------- ----------- ^ XOR 2 ^ 3 1 # NONE 0
IfweturnonNPScompatibility,the^operatorbecomesa"power"operator,andthe#becomesanXORoperator.SET SQL_COMPAT = 'NPS'; WITH SPECIAL(OP, DESCRIPTION, EXAMPLE, RESULT) AS ( VALUES (' ^ ','POWER ', '2 ^ 3 ', 2 ^ 3), (' # ','XOR ', '2 # 3 ', 2 # 3) ) SELECT * FROM SPECIAL; OP DESCRIPTION EXAMPLE RESULT ---- ----------- --------- ----------- ^ POWER 2 ^ 3 8 # XOR 2 # 3 1
GROUPBYOrdinalLocationTheGROUPBYcommandbehavioralsochangesinNPSmode.ThefollowingSQLstatementgroupsresultsusingthedefaultDB2syntax:SET SQL_COMPAT='DB2'; SELECT WORKDEPT,INT(AVG(SALARY)) FROM EMPLOYEE GROUP BY WORKDEPT; WORKDEPT 2 -------- ----------- A00 70850 B01 94250 C01 77222 D11 58784 D21 51240 E01 80175 E11 45306 E21 47087
Chapter5:NetezzaCompatibility 47
Ifyoutryusingtheordinallocation(similartoanORDERBYclause),youwillgetanerrormessage.SELECT WORKDEPT, INT(AVG(SALARY)) FROM EMPLOYEE GROUP BY 1; SQL0119N An expression starting with "WORKDEPT" specified in a SELECT clause, HAVING clause, or ORDER BY clause is not specified in the GROUP BY clause or it is in a SELECT clause, HAVING clause, or ORDER BY clause with a column function and no GROUP BY clause is specified. SQLSTATE=42803
IfNPScompatibilityisturnedon,thenthenyoucanusetheGROUPBYclausewithanordinallocation.SET SQL_COMPAT='NPS'; SELECT WORKDEPT, INT(AVG(SALARY)) FROM EMPLOYEE GROUP BY 1;
TRANSLATEFunctionThetranslatefunctionsyntaxinDB2is: TRANSLATE(expression, to_string, from_string, padding)
TheTRANSLATEfunctionreturnsavalueinwhichoneormorecharactersinastringexpressionmighthavebeenconvertedtoothercharacters.Thefunctionconvertsallthecharactersinchar-string-expinfrom-string-exptothecorrespondingcharactersinto-string-expor,ifnocorrespondingcharactersexist,tothepadcharacterspecifiedbypadding.Ifnoparametersaregiventothefunction,theoriginalstringisconvertedtouppercase.InNPSmode,thetranslatesyntaxis:TRANSLATE(expression, from_string, to_string)
Ifacharacterisfoundinthefromstring,andthereisnocorrespondingcharacterinthetostring,itisremoved.IfitwasusingDB2syntax,thepaddingcharacterwouldbeusedinstead.Note:IfORACLEcompatibilityisONthenthebehaviorofTRANSLATEisidenticaltoNPSmode.
Chapter5:NetezzaCompatibility 48
Thisfirstexamplewilluppercasethestring.SET SQL_COMPAT = 'NPS'; VALUES TRANSLATE('Hello'); 1 ----- HELLO
Inthisexample,theletter'o'willbereplacedwithan'1'.VALUES TRANSLATE('Hello','o','1'); 1 ----- Hell1
Notethatyoucouldreplacemorethanonecharacterbyexpandingboththe"to"and"from"strings.Thisexamplewillreplacetheletter"e"witha"2"aswellas"o"with"1".VALUES TRANSLATE('Hello','oe','12'); 1 ----- H2ll1
Translatewillalsoremoveacharacterifitisnotinthe"to"list.VALUES TRANSLATE('Hello','oel','12'); 1 ----- H21
Chapter6:SQLExtensions 50
SQLExtensions
DB2hastheabilitytolimittheamountofdataretrievedonaSELECTstatementthroughtheuseoftheFETCHFIRSTnROWSONLYclause.InDB211.1,theabilitytooffsettherowsbeforefetchingwasaddedtotheFETCHFIRSTclause.
SimpleSQLwithFetchFirstClauseTheFETCHfirstclausecanbeusedinavarietyoflocationsinaSELECTclause.Thisfirstexamplefetchesonly5rowsfromtheEMPLOYEEtable.SELECT LASTNAME FROM EMPLOYEE FETCH FIRST 5 ROWS ONLY; LASTNAME --------------- HAAS THOMPSON KWAN GEYER STERN
YoucanalsoaddORDERBYandGROUPBYclausesintheSELECTstatement.NotethatDB2stillneedstoprocessalltherecordsanddotheORDER/GROUPBYworkbeforelimitingtheanswerset.So,youarenotgettingthefirst5rows"sorted".Youaregettingtheentireanswersetsortedbeforeretrievingjust5rows.SELECT LASTNAME FROM EMPLOYEE ORDER BY LASTNAME FETCH FIRST 5 ROWS ONLY; LASTNAME --------------- ADAMSON ALONZO BROWN GEYER GOUNOT
HereisanexamplewiththeGROUPBYstatement.ThisfirstSQLstatementgivesusthetotalanswerset-thecountofemployeesbyWORKDEPT.
Chapter6:SQLExtensions 51
SELECT WORKDEPT, COUNT(*) FROM EMPLOYEE GROUP BY WORKDEPT ORDER BY WORKDEPT; WORKDEPT 2 -------- ----------- A00 5 B01 1 C01 4 D11 11 D21 7 E01 1 E11 7 E21 6
AddingtheFETCHFIRSTclauseonlyreducestherowsreturned,nottherowsthatareusedtocomputetheGROUPingresult.SELECT WORKDEPT, COUNT(*) FROM EMPLOYEE GROUP BY WORKDEPT ORDER BY WORKDEPT FETCH FIRST 5 ROWS ONLY; WORKDEPT 2 -------- ----------- A00 5 B01 1 C01 4 D11 11 D21 7
OFFSETExtensionTheFETCHFIRSTnROWSONLYclausecanalsoincludeanOFFSETkeyword.TheOFFSETkeywordallowsyoutoretrievetheanswersetafterskipping"n"numberofrows.ThesyntaxoftheOFFSETkeywordis:OFFSET n ROWS FETCH FIRST x ROWS ONLY
TheOFFSETnROWSmustprecedetheFETCHFIRSTxROWSONLYclause.TheOFFSETclausecanbeusedtoscrolldownananswersetwithouthavingtoholdacursor.Forinstance,youcouldhavethefirstSELECTcallrequest10rowsbyjustusingtheFETCHFIRSTclause.Afterthatyoucouldrequestthefirst10rowsbeskippedbeforeretrievingthenext10rows.Theonethingyoumustbeawareofisthatthatanswersetcouldchangebetweencallsifyouusethistechniqueofa"moving"window.Ifrowsareupdatedoraddedafteryourinitialqueryyoumaygetdifferentresults.
Chapter6:SQLExtensions 52
ThisisduetothewaythatDB2addsrowstoatable.IfthereisaDELETEandthenanINSERT,theINSERTedrowmayendupintheemptyslot.Thereisnoguaranteeoftheorderofretrieval.Forthisreason,youarebetteroffusinganORDERBYtoforcetheorderingalthoughthistoowon'talwayspreventrowschangingpositions.Herearethefirst10rowsoftheemployeetable(notordered).SELECT LASTNAME FROM EMPLOYEE FETCH FIRST 10 ROWS ONLY; LASTNAME --------------- HAAS THOMPSON KWAN GEYER STERN PULASKI HENDERSON SPENSER LUCCHESSI O'CONNELL
Youcanspecifyazerooffsettobeginfromthebeginning.SELECT LASTNAME FROM EMPLOYEE OFFSET 0 ROWS FETCH FIRST 10 ROWS ONLY;
Nowwecanmovetheanswersetaheadby5rowsandgettheremaining5rowsintheanswerset.SELECT LASTNAME FROM EMPLOYEE OFFSET 5 ROWS FETCH FIRST 5 ROWS ONLY; LASTNAME --------------- PULASKI HENDERSON SPENSER LUCCHESSI O'CONNELL
FETCHFIRSTandOFFSETinSUBSELECTsTheFETCHFIRST/OFFSETclauseisnotlimitedtoregularSELECTstatements.Youcanalsolimitthenumberofrowsthatareusedina
Chapter6:SQLExtensions 53
subselect.Inthiscase,youarelimitingtheamountofdatathatDB2willscanwhendeterminingtheanswerset.Forinstance,sayyouwantedtofindthenamesoftheemployeeswhomakemorethantheaveragesalaryofthe3rdhighestpaiddepartment.(Bytheway,therearemultiplewaystodothis,butthisisoneapproach).Thefirststepistodeterminewhattheaveragesalaryisofalldepartments.SELECT WORKDEPT, AVG(SALARY) FROM EMPLOYEE GROUP BY WORKDEPT ORDER BY AVG(SALARY) DESC; WORKDEPT 2 -------- ------------------------------------------ B01 94250 E01 80175 C01 77222.5 A00 70850 D11 58783.63636363636363636363636363636 D21 51240 E21 47086.66666666666666666666666666667 E11 45305.71428571428571428571428571429
Weonlywantonerecordfromthislist(thethirdone),sowecanusetheFETCHFIRSTclausewithanOFFSETtogetthevaluewewant(Note:weneedtoskip2rowstogettothe3rdone). SELECT WORKDEPT, AVG(SALARY) FROM EMPLOYEE GROUP BY WORKDEPT ORDER BY AVG(SALARY) DESC OFFSET 2 ROWS FETCH FIRST 1 ROWS ONLY; WORKDEPT 2 -------- ------------------------------------------ C01 77222.5
Andhereisthelistofemployeesthatmakemorethantheaveragesalaryofthe3rdhighestdepartmentinthecompany. SELECT LASTNAME, SALARY FROM EMPLOYEE WHERE SALARY > ( SELECT AVG(SALARY) FROM EMPLOYEE GROUP BY WORKDEPT ORDER BY AVG(SALARY) DESC OFFSET 2 ROWS FETCH FIRST 1 ROW ONLY ) ORDER BY SALARY;
Chapter6:SQLExtensions 54
LASTNAME SALARY --------------- ----------- GEYER 80175.00 SPENSER 86150.00 HENDERSON 89750.00 THOMPSON 94250.00 PULASKI 96170.00 KWAN 98250.00 HAAS 152750.00
AlternateSyntaxforFETCHFIRSTTheFETCHFIRSTnROWSONLYandOFFSETclausecanalsobespecifiedusingasimplerLIMIT/OFFSETsyntax.TheLIMITclauseandtheequivalentFETCHFIRSTsyntaxareshownbelow.Table6:DataTypeSynonyms
Syntax EquivalentLIMITx FETCHFIRSTxROWSONLYLIMITxOFFSETy OFFSETyROWSFETCHFIRSTxROWSONLYLIMITy,x OFFSETyROWSFETCHFIRSTxROWSONLYWecanusetheLIMITclausewithanOFFSETtogetthevaluewewantfromthetable.SELECT WORKDEPT, AVG(SALARY) FROM EMPLOYEE GROUP BY WORKDEPT ORDER BY AVG(SALARY) DESC LIMIT 1 OFFSET 2; WORKDEPT 2 -------- ------------------------------------------ C01 77222.5
Andhereisthelistofemployeesthatmakemorethantheaveragesalaryofthe3rdhighestdepartmentinthecompany.NotethattheLIMITclausespecifiesonlytheoffset(LIMITx)ortheoffsetandlimit(LIMITy,x)whenyoudonotusetheLIMITkeyword.OnewouldthinkthatLIMITxOFFSETywouldtranslateintoLIMITx,ybutthatisnotthecase.Don'ttrytofigureouttheSQLstandardsreasoningbehindthesyntax!
Chapter6:SQLExtensions 55
SELECT LASTNAME, SALARY FROM EMPLOYEE WHERE SALARY > ( SELECT AVG(SALARY) FROM EMPLOYEE GROUP BY WORKDEPT ORDER BY AVG(SALARY) DESC LIMIT 2,1 ) ORDER BY SALARY; LASTNAME SALARY --------------- ----------- GEYER 80175.00 SPENSER 86150.00 HENDERSON 89750.00 THOMPSON 94250.00 PULASKI 96170.00 KWAN 98250.00 HAAS 152750.00
TableCreationwithSELECTTheCREATETABLEstatementcannowuseaSELECTclausetogeneratethedefinitionandLOADthedataatthesametime.ThesyntaxoftheCREATEtablestatementhasbeenextendedwiththeAS(SELECT...)WITHDATAclause:CREATE TABLE <name> AS (SELECT ...) [ WITH DATA | DEFINITION ONLY]
ThetabledefinitionwillbegeneratedbasedontheSQLstatementthatyouspecify.ThecolumnnamesarederivedfromthecolumnsthatareintheSELECTlistandcanbechangedbyspecifyingthecolumnsnamesaspartofthetablename:EMP(X,Y,Z,...)AS(...).Forexample,thefollowingSQLwillfailbecauseacolumnlistwasnotprovided:CREATE TABLE AS_EMP AS (SELECT EMPNO, SALARY+BONUS FROM EMPLOYEE) DEFINITION ONLY; SQL0153N The statement does not include a required column list. SQLSTATE=42908
YoucannameacolumnintheSELECTlistorplaceitinthetabledefinition.CREATE TABLE AS_EMP AS (SELECT EMPNO, SALARY+BONUS AS PAY FROM EMPLOYEE) DEFINITION ONLY;
Chapter6:SQLExtensions 56
YoucanDESCRIBEthetabletoseethedefinition.DESCRIBE TABLE AS_EMP; COLNAME TYPESCHEMA TYPENAME LENGTH SCALE NULLABLE ------- ---------- --------- ------ ----- -------- EMPNO SYSIBM CHARACTER 6 0 N PAY SYSIBM DECFLOAT 16 0 Y
TheDEFINITIONONLYclausewillcreatethetablebutnotloadanydataintoit.AddingtheWITHDATAclausewilldoanINSERTofrowsintothenewlycreatedtable.Ifyouhavealargeamountofdatatoloadintothetable,youmaybebetteroffcreatingthetablewithDEFINITIONONLYandthenusingLOADorothermethodstoloadthedataintothetable.CREATE TABLE AS_EMP AS (SELECT EMPNO, SALARY+BONUS AS PAY FROM EMPLOYEE) WITH DATA;
TheSELECTstatementintheCREATETABLEbodycanbeverysophisticated.Itcandoanytypeofcalculationorlimitthedatatoasubsetofinformation. CREATE TABLE AS_EMP(LAST,PAY) AS ( SELECT LASTNAME, SALARY FROM EMPLOYEE WHERE WORKDEPT='D11' FETCH FIRST 3 ROWS ONLY ) WITH DATA;
YoucanalsousetheOFFSETclauseaspartoftheFETCHFIRSTONLYtogetchunksofdatafromtheoriginaltable.CREATE TABLE AS_EMP(DEPARTMENT, LASTNAME) AS (SELECT WORKDEPT, LASTNAME FROM EMPLOYEE OFFSET 5 ROWS FETCH FIRST 10 ROWS ONLY ) WITH DATA;
CREATEFUNCTIONforAggregateUDFsTheupdatedCREATEFUNCTIONstatementallowsyoutocreateyourownaggregationfunctions.Anaggregatefunctionreturnsasinglevaluethatistheresultofanevaluationofasetoflikevalues,suchasthoseinacolumnwithinasetofrows.FunctionslikeSUM,AVG,andCOUNTareexamplesofbuilt-incolumnfunctions.
Chapter6:SQLExtensions 57
Aggregationfunctionscanbecreatedwithavarietyofprogramminglanguagesandmustcoverfourstagesoftheaggregationprocess.
• INITIATE–CalledwhenbeforetheSQLreturnsindividualrows.Theinitializationstepisusedtozerooutcountersandothervaluesthatareusedintheaggregationprocess.
• ACCUMULATE–Aseachrowisretrieved,theaccumulatestepiscalledtoincrementorprocessthevalueinsomeway.
• MERGE–InaDPF(partitioned)environment,thedatafromeachpartitionneedstobeconsolidatedtocomeupwithafinalresult.
• FINALIZE–ThefinalizestepwillcomputethefinalvaluethatthefunctionshouldreturntotheSQLstatement.
Chapter7:DateFunctions 59
DateFunctions
ThereareplentyofnewdateandtimefunctionsfoundinDB211.1.Thesefunctionsallowyoutoextractportionsfromadateandformatthedateinavarietyofdifferentways.WhileDB2alreadyhasanumberofdateandtimefunctions,thesenewfunctionsallowforgreatercompatibilitywithotherdatabaseimplementations,makingiteasiertoporttoDB2.
ExtractFunctionTheEXTRACTfunctionextractsanelementfromadate/timevalue.ThesyntaxoftheEXTRACTcommandis: EXTRACT( element FROM expression )
ThisisaslightlydifferentformatfrommostfunctionsthatyouseeinDB2.Elementmustbeoneofthefollowingvalues:Table7:DateElements
ElementName DescriptionEPOCH Numberofsecondssince1970-01-0100:00:00.00.Thevaluecanbe
positiveornegative.MILLENNIUM(S) Themillenniumistobereturned.CENTURY(CENTURIES) Thenumberoffull100-yearperiodsrepresentedbytheyear.DECADE(S) Thenumberoffull10-yearperiodsrepresentedbytheyear.YEAR(S) Theyearportionistobereturned.QUARTER Thequarteroftheyear(1-4)istobereturned.MONTH Themonthportionistobereturned.WEEK Thenumberoftheweekoftheyear(1-53)thatthespecifieddayis
tobereturned.DAY(S) Thedayportionistobereturned.DOW Thedayoftheweekthatistobereturned.Notethat"1"represents
Sunday.DOY Theday(1-366)oftheyearthatistobereturned.HOUR(S) Thehourportionistobereturned.MINUTE(S) Theminuteportionistobereturned.SECOND(S) Thesecondportionistobereturned.MILLISECOND(S) Thesecondoftheminute,includingfractionalpartstoone
thousandthofasecondMICROSECOND(S) Thesecondoftheminute,includingfractionalpartstoonemillionth
ofasecond
Chapter7:DateFunctions 60
ThisSQLwillreturneverypossibleextractvaluefromthecurrentdate.TheNOWkeywordisasynonymforCURRENTTIMESTAMP.WITH DATES(FUNCTION, RESULT) AS ( VALUES ('EPOCH', EXTRACT( EPOCH FROM NOW )), ('MILLENNIUM(S)', EXTRACT( MILLENNIUM FROM NOW )), ('CENTURY(CENTURIES)', EXTRACT( CENTURY FROM NOW )), ('DECADE(S)', EXTRACT( DECADE FROM NOW )), ('YEAR(S)', EXTRACT( YEAR FROM NOW )), ('QUARTER', EXTRACT( QUARTER FROM NOW )), ('MONTH', EXTRACT( MONTH FROM NOW )), ('WEEK', EXTRACT( WEEK FROM NOW )), ('DAY(S)', EXTRACT( DAY FROM NOW )), ('DOW', EXTRACT( DOW FROM NOW )), ('DOY', EXTRACT( DOY FROM NOW )), ('HOUR(S)', EXTRACT( HOURS FROM NOW )), ('MINUTE(S)', EXTRACT( MINUTES FROM NOW )), ('SECOND(S)', EXTRACT( SECONDS FROM NOW )), ('MILLISECOND(S)', EXTRACT( MILLISECONDS FROM NOW )), ('MICROSECOND(S)', EXTRACT( MICROSECONDS FROM NOW )) ) SELECT * FROM DATES; FUNCTION RESULT ------------------ --------------------------- EPOCH 1474090894.000000 MILLENNIUM(S) 2.000000 CENTURY(CENTURIES) 20.000000 DECADE(S) 201.000000 YEAR(S) 2016.000000 QUARTER 3.000000 MONTH 9.000000 WEEK 38.000000 DAY(S) 17.000000 DOW 7.000000 DOY 261.000000 HOUR(S) 5.000000 MINUTE(S) 41.000000 SECOND(S) 34.578000 MILLISECOND(S) 34578.000000 MICROSECOND(S) 34578000.000000
DATE_PARTFunctionDATE_PARTisliketheEXTRACTfunctionbutitusesthemorefamiliarfunctionsyntax: DATE_PART(element, expression)
InthecaseoftheDATE_PARTfunction,theelementmustbeplacedinquotes,ratherthanasakeywordintheEXTRACTfunction.inaddition,theDATE_PARTalwaysreturnsaBIGINT,whiletheEXTRACTfunctionwill
Chapter7:DateFunctions 61
returnadifferentdatatypedependingontheelementbeingreturned.Forinstance,theSECONDsoptionforEXTRACTreturnsaDECIMALresultwhiletheDATE_PARTreturnsatruncatedBIGINT. WITH DATES(FUNCTION, RESULT) AS ( VALUES ('EPOCH', DATE_PART('EPOCH' ,NOW )), ('MILLENNIUM(S)', DATE_PART('MILLENNIUM' ,NOW )), ('CENTURY(CENTURIES)', DATE_PART('CENTURY' ,NOW )), ('DECADE(S)', DATE_PART('DECADE' ,NOW )), ('YEAR(S)', DATE_PART('YEAR' ,NOW )), ('QUARTER', DATE_PART('QUARTER' ,NOW )), ('MONTH', DATE_PART('MONTH' ,NOW )), ('WEEK', DATE_PART('WEEK' ,NOW )), ('DAY(S)', DATE_PART('DAY' ,NOW )), ('DOW', DATE_PART('DOW' ,NOW )), ('DOY', DATE_PART('DOY' ,NOW )), ('HOUR(S)', DATE_PART('HOURS' ,NOW )), ('MINUTE(S)', DATE_PART('MINUTES' ,NOW )), ('SECOND(S)', DATE_PART('SECONDS' ,NOW )), ('MILLISECOND(S)', DATE_PART('MILLISECONDS' ,NOW )), ('MICROSECOND(S)', DATE_PART('MICROSECONDS' ,NOW )) ) SELECT * FROM DATES; FUNCTION RESULT ------------------ -------------------- EPOCH 1474090894 MILLENNIUM(S) 2 CENTURY(CENTURIES) 20 DECADE(S) 201 YEAR(S) 2016 QUARTER 3 MONTH 9 WEEK 38 DAY(S) 17 DOW 7 DOY 261 HOUR(S) 5 MINUTE(S) 41 SECOND(S) 34 MILLISECOND(S) 34809 MICROSECOND(S) 34809000
DATE_TRUNCFunctionDATE_TRUNCcomputesthesameresultsastheDATE_PARTfunctionbutthentruncatesthevaluedown.Notethatnotallvaluescanbetruncated.Thefunctionsyntaxis: DATE_TRUNC(element, expression)
Theelementmustbeplacedinquotes,ratherthanasakeywordintheEXTRACTfunction.NotethatDATE_TRUNCalwaysreturnsaBIGINT.
Chapter7:DateFunctions 62
Theelementsthatcanbetruncatedare:Table8:DATE_TRUNCElements
ElementName DescriptionMILLENNIUM(S) Themillenniumistobereturned.CENTURY(CENTURIES) Thenumberoffull100-yearperiodsrepresentedbytheyear.DECADE(S) Thenumberoffull10-yearperiodsrepresentedbytheyear.YEAR(S) Theyearportionistobereturned.QUARTER Thequarteroftheyear(1-4)istobereturned.MONTH Themonthportionistobereturned.WEEK Thenumberoftheweekoftheyear(1-53)thatthespecifieddayis
tobereturned.DAY(S) Thedayportionistobereturned.HOUR(S) Thehourportionistobereturned.MINUTE(S) Theminuteportionistobereturned.SECOND(S) Thesecondportionistobereturned.MILLISECOND(S) Thesecondoftheminute,includingfractionalpartstoone
thousandthofasecondMICROSECOND(S) Thesecondoftheminute,includingfractionalpartstoonemillionth
ofasecond
WITH DATES(FUNCTION, RESULT) AS ( VALUES ('MILLENNIUM(S)', DATE_TRUNC('MILLENNIUM',NOW )), ('CENTURY(CENTURIES)', DATE_TRUNC('CENTURY' ,NOW )), ('DECADE(S)', DATE_TRUNC('DECADE' ,NOW )), ('YEAR(S)', DATE_TRUNC('YEAR' ,NOW )), ('QUARTER', DATE_TRUNC('QUARTER' ,NOW )), ('MONTH', DATE_TRUNC('MONTH' ,NOW )), ('WEEK', DATE_TRUNC('WEEK' ,NOW )), ('DAY(S)', DATE_TRUNC('DAY' ,NOW )), ('HOUR(S)', DATE_TRUNC('HOURS' ,NOW )), ('MINUTE(S)', DATE_TRUNC('MINUTES' ,NOW )), ('SECOND(S)', DATE_TRUNC('SECONDS' ,NOW )), ('MILLISEC(S)', DATE_TRUNC('MILLISECONDS ',NOW )), ('MICROSEC(S)', DATE_TRUNC('MICROSECONDS ',NOW )) ) SELECT * FROM DATES; FUNCTION RESULT ------------------ -------------------------- MILLENNIUM(S) 2000-01-01 00:00:00.000000 CENTURY(CENTURIES) 2000-01-01 00:00:00.000000 DECADE(S) 2010-01-01 00:00:00.000000 YEAR(S) 2016-01-01 00:00:00.000000 QUARTER 2016-07-01 00:00:00.000000 MONTH 2016-09-01 00:00:00.000000 WEEK 2016-09-12 00:00:00.000000 DAY(S) 2016-09-17 00:00:00.000000 HOUR(S) 2016-09-17 05:00:00.000000 MINUTE(S) 2016-09-17 05:41:00.000000 SECOND(S) 2016-09-17 05:41:35.000000 MILLISEC(S) 2016-09-17 05:41:35.049000 MICROSEC(S) 2016-09-17 05:41:35.049000
Chapter7:DateFunctions 63
ExtractingSpecificDaysfromaMonthTherearethreefunctionsthatretrievedayinformationfromadate.
• DAYOFMONTH–returnsanintegerbetween1and31thatrepresentsthedayoftheargument
• FIRST_DAY–returnsadateortimestampthatrepresentsthefirstdayofthemonthoftheargument
• DAYS_TO_END_OF_MONTH–returnsthenumberofdaystotheendofthemonth
ThefollowingexamplesassumethatthecurrentdateisSeptember17th,2016.VALUES NOW; 1 -------------------------- 2016-09-17 05:41:35.229000
DAYOFMONTHreturnsthedayofthemonth.VALUES DAYOFMONTH(NOW); 1 ----------- 17
FIRST_DAYwillreturnthefirstdayofthemonth.YoucouldprobablycomputethiswithstandardSQLdatefunctions,butitisaloteasierjusttousethisbuilt-infunction.VALUES FIRST_DAY(NOW); 1 -------------------------- 2016-09-01 05:41:35.399000
Finally,DAYS_TO_END_OF_MOTNHwillreturnthenumberofdaystotheendofthemonth.AZerowouldbereturnedifyouareonthelastdayofthemonth.VALUES DAYS_TO_END_OF_MONTH(NOW); 1 ----------- 13
Chapter7:DateFunctions 64
DateAdditionFunctionsThedateadditionfunctionswilladdorsubtractdaysfromacurrenttimestamp.Thefunctionsthatareavailableare:
• ADD_YEARS–Addyearstoadate• ADD_MONTHS–Addmonthstoadate• ADD_DAYS–Adddaystoadate• ADD_HOURS–Addhourstoadate• ADD_MINUTES–Addminutestoadate• ADD_SECONDS–Addsecondstoadate
Theformatofthefunctionis:ADD_DAYS ( expression, numeric expression )
ThefollowingSQLwilladdone"unit"tothecurrentdate.WITH DATES(FUNCTION, RESULT) AS ( VALUES ('CURRENT DATE ',NOW), ('ADD_YEARS ',ADD_YEARS(NOW,1)), ('ADD_MONTHS ',ADD_MONTHS(NOW,1)), ('ADD_DAYS ',ADD_DAYS(NOW,1)), ('ADD_HOURS ',ADD_HOURS(NOW,1)), ('ADD_MINUTES ',ADD_MINUTES(NOW,1)), ('ADD_SECONDS ',ADD_SECONDS(NOW,1)) ) SELECT * FROM DATES; FUNCTION RESULT --------------- -------------------------- CURRENT DATE 2016-09-17 05:41:35.669000 ADD_YEARS 2017-09-17 05:41:35.669000 ADD_MONTHS 2016-10-17 05:41:35.669000 ADD_DAYS 2016-09-18 05:41:35.669000 ADD_HOURS 2016-09-17 06:41:35.669000 ADD_MINUTES 2016-09-17 05:42:35.669000 ADD_SECONDS 2016-09-17 05:41:36.669000
Chapter7:DateFunctions 65
Anegativenumbercanbeusedtosubtractvaluesfromthecurrentdate.WITH DATES(FUNCTION, RESULT) AS ( VALUES ('CURRENT DATE ',NOW), ('ADD_YEARS ',ADD_YEARS(NOW,-1)), ('ADD_MONTHS ',ADD_MONTHS(NOW,-1)), ('ADD_DAYS ',ADD_DAYS(NOW,-1)), ('ADD_HOURS ',ADD_HOURS(NOW,-1)), ('ADD_MINUTES ',ADD_MINUTES(NOW,-1)), ('ADD_SECONDS ',ADD_SECONDS(NOW,-1)) ) SELECT * FROM DATES; FUNCTION RESULT --------------- -------------------------- CURRENT DATE 2016-09-17 05:41:35.749000 ADD_YEARS 2015-09-17 05:41:35.749000 ADD_MONTHS 2016-08-17 05:41:35.749000 ADD_DAYS 2016-09-16 05:41:35.749000 ADD_HOURS 2016-09-17 04:41:35.749000 ADD_MINUTES 2016-09-17 05:40:35.749000 ADD_SECONDS 2016-09-17 05:41:34.749000
ExtractingWeeks,Months,Quarters,andYearsfromaDateTherearefourfunctionsthatextractdifferentvaluesfromadate.Thesefunctionsinclude:
• THIS_QUARTER–returnsthefirstdayofthequarter• THIS_WEEK–returnsthefirstdayoftheweek(Sundayis
consideredthefirstdayofthatweek)• THIS_MONTH–returnsthefirstdayofthemonth• THIS_YEAR–returnsthefirstdayoftheyear
ThefollowingSQLwillreturnallthecurrent(THIS)valuesofthecurrentdate.WITH DATES(FUNCTION, RESULT) AS ( VALUES ('CURRENT DATE ',NOW), ('THIS_WEEK ',THIS_WEEK(NOW)), ('THIS_MONTH ',THIS_MONTH(NOW)), ('THIS_QUARTER ',THIS_QUARTER(NOW)), ('THIS_YEAR ',THIS_YEAR(NOW)) ) SELECT * FROM DATES;
Chapter7:DateFunctions 66
FUNCTION RESULT --------------- -------------------------- CURRENT DATE 2016-09-17 05:41:35.879000 THIS_WEEK 2016-09-11 00:00:00.000000 THIS_MONTH 2016-09-01 00:00:00.000000 THIS_QUARTER 2016-07-01 00:00:00.000000 THIS_YEAR 2016-01-01 00:00:00.000000
ThereisalsoaNEXTfunctionforeachofthese.TheNEXTfunctionwillreturnthenextweek,month,quarter,oryeargivenacurrentdate. WITH DATES(FUNCTION, RESULT) AS ( VALUES ('CURRENT DATE ',NOW), ('NEXT_WEEK ',NEXT_WEEK(NOW)), ('NEXT_MONTH ',NEXT_MONTH(NOW)), ('NEXT_QUARTER ',NEXT_QUARTER(NOW)), ('NEXT_YEAR ',NEXT_YEAR(NOW)) ) SELECT * FROM DATES; FUNCTION RESULT --------------- -------------------------- CURRENT DATE 2016-09-17 05:41:35.979000 NEXT_WEEK 2016-09-18 00:00:00.000000 NEXT_MONTH 2016-10-01 00:00:00.000000 NEXT_QUARTER 2016-10-01 00:00:00.000000 NEXT_YEAR 2017-01-01 00:00:00.000000
NextDayFunctionTheprevioussetoffunctionsreturnedadatevalueforthecurrentweek,month,quarter,oryear(orthenextoneifyouusedtheNEXTfunction).TheNEXT_DAYfunctionreturnsthenextday(afterthedateyousupply)basedonthestringrepresentationoftheday.Thedatestringwillbedependentonthecodepagethatyouareusingforthedatabase.Thedate(fromanEnglishperspective)canbe:Table9:DaysandShortForm
Day ShortformMonday MONTuesday TUEWednesday WEDThursday THUFriday FRISaturday SATSunday SUN
Chapter7:DateFunctions 67
ThefollowingSQLwillshowyouthe"day"afterthecurrentdatethatisMondaythroughSunday.WITH DATES(FUNCTION, RESULT) AS ( VALUES ('CURRENT DATE ',NOW), ('Monday ',NEXT_DAY(NOW,'Monday')), ('Tuesday ',NEXT_DAY(NOW,'TUE')), ('Wednesday ',NEXT_DAY(NOW,'Wednesday')), ('Thursday ',NEXT_DAY(NOW,'Thursday')), ('Friday ',NEXT_DAY(NOW,'FRI')), ('Saturday ',NEXT_DAY(NOW,'Saturday')), ('Sunday ',NEXT_DAY(NOW,'Sunday')) ) SELECT * FROM DATES; FUNCTION RESULT --------------- -------------------------- CURRENT DATE 2016-09-17 05:41:36.139000 Monday 2016-09-19 05:41:36.139000 Tuesday 2016-09-20 05:41:36.139000 Wednesday 2016-09-21 05:41:36.139000 Thursday 2016-09-22 05:41:36.139000 Friday 2016-09-23 05:41:36.139000 Saturday 2016-09-24 05:41:36.139000 Sunday 2016-09-18 05:41:36.139000
BetweenDate/TimeFunctionsThesedatefunctionscomputethenumberoffullseconds,minutes,hours,days,weeks,andyearsbetweentwodates.Ifthereisn'tafullvaluebetweenthetwoobjects(likelessthanaday),azerowillbereturned.Thesenewfunctionsare:
• HOURS_BETWEEN–returnsthenumberoffullhoursbetweentwoarguments
• MINUTES_BETWEEN–returnsthenumberoffullminutesbetweentwoarguments
• SECONDS_BETWEEN–returnsthenumberoffullsecondsbetweentwoarguments
• DAYS_BETWEEN–returnsthenumberoffulldaysbetweentwoarguments
• WEEKS_BETWEEN–returnsthenumberoffullweeksbetweentwoarguments
• YEARS_BETWEEN–returnsthenumberoffullyearsbetweentwoarguments
Chapter7:DateFunctions 68
Theformatofallofthesefunctionsis:DAYS_BETWEEN( expression1, expression2 )
ThefollowingSQLwilluseadatethatisinthefuturewithexactlyoneextrasecond,minute,hour,day,weekandyearaddedtoit.CREATE VARIABLE FUTURE_DATE TIMESTAMP DEFAULT(NOW + 1 SECOND + 1 MINUTE + 1 HOUR + 8 DAYS + 1 YEAR); WITH DATES(FUNCTION, RESULT) AS ( VALUES ('SECONDS_BETWEEN',SECONDS_BETWEEN(FUTURE_DATE,NOW)), ('MINUTES_BETWEEN',MINUTES_BETWEEN(FUTURE_DATE,NOW)), ('HOURS_BETWEEN ',HOURS_BETWEEN(FUTURE_DATE,NOW)), ('DAYS BETWEEN ',DAYS_BETWEEN(FUTURE_DATE,NOW)), ('WEEKS_BETWEEN ',WEEKS_BETWEEN(FUTURE_DATE,NOW)), ('YEARS_BETWEEN ',YEARS_BETWEEN(FUTURE_DATE,NOW)) ) SELECT * FROM DATES; FUNCTION RESULT --------------- -------------------- SECONDS_BETWEEN 32230861 MINUTES_BETWEEN 537181 HOURS_BETWEEN 8953 DAYS BETWEEN 373 WEEKS_BETWEEN 53 YEARS_BETWEEN 1
MONTHS_BETWEENFunctionYoumayhavenoticedthattheMONTHS_BETWEENfunctionwasnotinthepreviouslistoffunctions.ThereasonforthisisthatthevaluereturnedforMONTHS_BETWEENisdifferentfromtheotherfunctions.TheMONTHS_BETWEENfunctionreturnsaDECIMALvalueratherthananintegervalue.Thereasonforthisisthatthedurationofamonthisnotaspreciseasaday,weekoryear.Thefollowingexamplewillshowhowthedurationisadecimalvalueratherthananinteger.Youcouldalwaystruncatethevalueifyouwantaninteger.WITH DATES(FUNCTION, RESULT) AS ( VALUES ('0 MONTH ',MONTHS_BETWEEN(NOW, NOW)), ('1 MONTH ',MONTHS_BETWEEN(NOW + 1 MONTH, NOW)), ('1 MONTH + 1 DAY',MONTHS_BETWEEN(NOW + 1 MONTH + 1 DAY, NOW)), ('LEAP YEAR ',MONTHS_BETWEEN('2016-02-01','2016-03-01')), ('NON-LEAP YEAR ',MONTHS_BETWEEN('2015-02-01','2015-03-01')) ) SELECT * FROM DATES;
Chapter7:DateFunctions 69
FUNCTION RESULT --------------- --------------------------------- 0 MONTH 0.000000000000000 1 MONTH 1.000000000000000 1 MONTH + 1 DAY 1.032258064516129 LEAP YEAR -1.000000000000000 NON-LEAP YEAR -1.000000000000000
DateDurationFunctionsAnalternatewayofrepresentingdatedurationsisthroughtheuseofanintegerwiththeformatYYYYMMDDwheretheYYYYrepresentstheyear,MMforthemonthandDDfortheday.Datedurationsareeasiertomanipulatethantimestampvaluesandtakeupsubstantiallylessstorage.Therearetwonewfunctions.
• YMD_BETWEENreturnsanumericvaluethatspecifiesthenumberoffullyears,fullmonths,andfulldaysbetweentwodatetimevalues
• AGEreturnsanumericvaluethatrepresentsthenumberoffullyears,fullmonths,andfulldaysbetweenthecurrenttimestampandtheargument
ThisSQLstatementwillreturnvariousAGEcalculationsbasedonthecurrenttimestamp.WITH DATES(FUNCTION, RESULT) AS ( VALUES ('AGE+ 1 DAY ',AGE(NOW - 1 DAY)), ('AGE+ 1 MONTH ',AGE(NOW - 1 MONTH)), ('AGE+ 1 YEAR ',AGE(NOW - 1 YEAR)), ('AGE+ 1 DAY + 1 MONTH ',AGE(NOW - 1 DAY - 1 MONTH)), ('AGE+ 1 DAY + 1 YEAR ',AGE(NOW - 1 DAY - 1 YEAR)), ('AGE+ 1 DAY + 1 MONTH + 1 YEAR',AGE(NOW - 1 DAY - 1 MONTH - 1 YEAR)) ) SELECT * FROM DATES; FUNCTION RESULT ------------------------------ ----------- AGE + 1 DAY 1 AGE + 1 MONTH 100 AGE + 1 YEAR 10000 AGE + 1 DAY + 1 MONTH 101 AGE + 1 DAY + 1 YEAR 10001 AGE + 1 DAY + 1 MONTH + 1 YEAR 10101
TheYMD_BETWEENfunctionisliketheAGEfunctionexceptthatittakestwodatearguments.WecansimulatetheAGEfunctionbysupplyingtheNOWfunctiontotheYMD_BETWEENfunction.
Chapter7:DateFunctions 70WITH DATES(FUNCTION, RESULT) AS ( VALUES ('1 DAY ',YMD_BETWEEN(NOW,NOW - 1 DAY)), ('1 MON ',YMD_BETWEEN(NOW,NOW - 1 MONTH)), ('1 YR ',YMD_BETWEEN(NOW,NOW - 1 YEAR)), ('1 DAY + 1 MON ',YMD_BETWEEN(NOW,NOW - 1 DAY - 1 MONTH)), ('1 DAY +1 YR ',YMD_BETWEEN(NOW,NOW - 1 DAY - 1 YEAR)), ('1 DAY+1 MON +1 YR',YMD_BETWEEN(NOW,NOW - 1 DAY - 1 MONTH - 1 YEAR)) ) SELECT * FROM DATES; FUNCTION RESULT ------------------------------ ----------- 1 DAY 1 1 MONTH 100 1 YEAR 10000 1 DAY + 1 MONTH 101 1 DAY + 1 YEAR 10001 1 DAY + 1 MONTH + 1 YEAR 10101
OVERLAPSPredicateTheOVERLAPSpredicateisusedtodeterminewhethertwochronologicalperiodsoverlap.ThisisnotafunctionwithinDB2,butratheraspecialSQLsyntaxextension.Achronologicalperiodisspecifiedbyapairofdate-timeexpressions.Thefirstexpressionspecifiesthestartofaperiod;thesecondspecifiesitsend.(start1, end1) OVERLAPS (start2, end2)
Thebeginningandendvaluesarenotincludedintheperiods.Thefollowingsummarizestheoverlaplogic.Forexample,theperiods2016-10-19to2016-10-20and2016-10-20to2016-10-21donotoverlap.Thefollowingintervaldoesnotoverlap.VALUES CASE WHEN (NOW, NOW + 1 DAY) OVERLAPS (NOW + 1 DAY, NOW + 2 DAYS) THEN 'Overlaps' ELSE 'No Overlap' END; 1 ---------- No Overlap
Chapter7:DateFunctions 71
Ifthefirstdaterangeisextendedbyoneday,thentherangewilloverlap.VALUES CASE WHEN (NOW, NOW + 2 DAYS) OVERLAPS (NOW + 1 DAY, NOW + 2 DAYS) THEN 'Overlaps' ELSE 'No Overlap' END; 1 ---------- Overlaps
Identicaldaterangeswilloverlap.VALUES CASE WHEN (NOW, NOW + 1 DAY) OVERLAPS (NOW, NOW + 1 DAY) THEN 'Overlaps' ELSE 'No Overlap' END; 1 ---------- Overlaps
UTCTimeConversionsDB2hastwofunctionsthatallowyoutotranslatetimestampstoandfromUTC(CoordinatedUniversalTime).
• TheFROM_UTC_TIMESTAMPscalarfunctionreturnsaTIMESTAMPthatisconvertedfromCoordinatedUniversalTimetothetimezonespecifiedbythetimezonestring.
• TheTO_UTC_TIMESTAMPscalarfunctionreturnsaTIMESTAMPthatisconvertedtoCoordinatedUniversalTimefromthetimezonethatisspecifiedbythetimezonestring.
Theformatofthetwofunctionsis:FROM_UTC_TIMESTAMP( expression, timezone)
TO_UTC_TIMESTAMP( expression, timezone)
Thereturnvaluefromeachofthesefunctionsisatimestamp.The"expression"isatimestampthatyouwanttoconverttothelocaltimezone(orconverttoUTC).Thetimezoneisanexpressionthatspecifiesthetimezonethattheexpressionistobeadjustedto.Thevalueofthetimezone-expressionmustbeatimezonenamefromtheInternetAssignedNumbersAuthority(IANA)timezonedatabase.Thestandard
Chapter7:DateFunctions 72
formatforatimezonenameintheIANAdatabaseisArea/Location,where:
• AreaistheEnglishnameofacontinent,ocean,orthespecialarea'Etc'
• LocationistheEnglishnameofalocationwithinthearea;usuallyacity,orsmallisland
Examples:
• "America/Toronto"• "Asia/Sakhalin"• "Etc/UTC"(whichrepresentsCoordinatedUniversalTime)
Forcompletedetailsonthevalidsetoftimezonenamesandtherulesthatareassociatedwiththosetimezones,refertotheIANAtimezonedatabase.DB211.1usesversion2010coftheIANAtimezonedatabase.Theresultisatimestamp,adjustedfrom/totheCoordinatedUniversalTimezonetothetimezonespecifiedbythetimezone-expression.Ifthetimezone-expressionreturnsavaluethatisnotatimezoneintheIANAtimezonedatabase,thenthevalueofexpressionisreturnedwithoutbeingadjusted.
UsingUTCFunctionsOneoftheapplicationsforusingtheUTCistotakeatransactiontimestampandnormalizeitacrossallsystemsthataccessthedata.YoucanconvertthetimestamptoUTConinsertandthenwhenitisretrieved,itcanbeconvertedbacktothelocaltimezone.Thisexamplewilluseseveraltechniquestohidethecomplexityofchangingtimestampstolocaltimezones.ThefollowingSQLwillcreateourbasetransactiontable(TXS_BASE)thatwillbeusedthroughouttheexample.CREATE TABLE TXS_BASE ( ID INTEGER NOT NULL, CUSTID INTEGER NOT NULL, TXTIME_UTC TIMESTAMP NOT NULL );
Chapter7:DateFunctions 73
TheUTCfunctionswillbewrittentotakeadvantageofalocaltimezonevariablecalledTIME_ZONE.Thisvariablewillcontainthetimezoneoftheserver(oruser)thatisrunningthetransaction.InthiscaseweareusingthetimezoneinToronto,Canada.CREATE OR REPLACE VARIABLE TIME_ZONE VARCHAR(255) DEFAULT('America/Toronto');
TheSETCommandcanbeusedtoupdatetheTIME_ZONEtothecurrentlocationwearein.SET TIME_ZONE = 'America/Toronto';
Toretrievethevalueofthecurrenttimezone,wetakeadvantageofasimpleuser-definedfunctioncalledGET_TIMEZONE.ItjustretrievesthecontentsofthecurrentTIME_ZONEvariablethatwesetup.CREATE OR REPLACE FUNCTION GET_TIMEZONE() RETURNS VARCHAR(255) LANGUAGE SQL CONTAINS SQL RETURN (TIME_ZONE);
TheTXSviewisusedbyallSQLstatementsratherthantheTXS_BASEtable.ThereasonforthisistotakeadvantageofINSTEADOFtriggersthatcanmanipulatetheUTCwithoutmodifyingtheoriginalSQL.NotethatwhenthedataisreturnedfromtheviewthattheTXTIMEfieldisconvertedfromUTCtothecurrentTIMEZONEthatwearein.CREATE OR REPLACE VIEW TXS AS ( SELECT ID, CUSTID, FROM_UTC_TIMESTAMP(TXTIME_UTC,GET_TIMEZONE()) AS TXTIME FROM TXS_BASE );
AnINSTEADOFtrigger(INSERT,UPDATE,andDELETE)iscreatedagainsttheTXSviewsothatanyinsertorupdateonaTXTIMEcolumnwillbeconvertedbacktotheUTCvalue.Fromanapplicationperspective,weareusingthelocaltime,nottheUTCtime.CREATE OR REPLACE TRIGGER I_TXS INSTEAD OF INSERT ON TXS REFERENCING NEW AS NEW_TXS FOR EACH ROW MODE DB2SQL BEGIN ATOMIC INSERT INTO TXS_BASE VALUES (
Chapter7:DateFunctions 74 NEW_TXS.ID, NEW_TXS.CUSTID, TO_UTC_TIMESTAMP(NEW_TXS.TXTIME,GET_TIMEZONE()) ); END / CREATE OR REPLACE TRIGGER U_TXS INSTEAD OF UPDATE ON TXS REFERENCING NEW AS NEW_TXS OLD AS OLD_TXS FOR EACH ROW MODE DB2SQL BEGIN ATOMIC UPDATE TXS_BASE SET (ID, CUSTID, TXTIME_UTC) = (NEW_TXS.ID, NEW_TXS.CUSTID, TO_UTC_TIMESTAMP(NEW_TXS.TXTIME,TIME_ZONE) ) WHERE TXS_BASE.ID = OLD_TXS.ID ; END / CREATE OR REPLACE TRIGGER D_TXS INSTEAD OF DELETE ON TXS REFERENCING OLD AS OLD_TXS FOR EACH ROW MODE DB2SQL BEGIN ATOMIC DELETE FROM TXS_BASE WHERE TXS_BASE.ID = OLD_TXS.ID ; END /
Atthispointintime(!)wecanstartinsertingrecordsintoourtable.WehavealreadysetthetimezonetobeToronto,sothenextinsertstatementwilltakethecurrenttime(NOW)andinsertitintothetable.Forreference,hereisthecurrenttimewhentheexamplewasrun.VALUES NOW; 1 -------------------------- 2016-09-17 05:53:18.956000
Wewillinsertonerecordintothetableandimmediatelyretrievetheresult.INSERT INTO TXS VALUES(1,1,NOW); SELECT * FROM TXS; ID CUSTID TXTIME ----------- ----------- -------------------------- 1 1 2016-09-17 05:53:19.056000
Chapter7:DateFunctions 75
Notethatthetimestampappearstobethesameaswhatweinsert(plusorminusafewseconds).WhatactuallysitsinthebasetableistheUTCtime.SELECT * FROM TXS_BASE; ID CUSTID TXTIME_UTC ----------- ----------- -------------------------- 1 1 2016-09-17 09:53:19.056000
Wecanmodifythetimethatisreturnedtousbychangingourlocaltimezone.ThestatementwillmakethesystemthinkweareinVancouver.SET TIME_ZONE = 'America/Vancouver';
Retrievingtheresultswillshowthatthetimestamphasshiftedby3hours(Vancouveris3hoursbehindToronto).SELECT * FROM TXS; ID CUSTID TXTIME ----------- ----------- -------------------------- 1 1 2016-09-17 02:53:19.056000
So,whathappensifweinsertarecordintothetablenowthatweareinVancouver?INSERT INTO TXS VALUES(2,2,NOW); SELECT * FROM TXS; ID CUSTID TXTIME ----------- ----------- -------------------------- 1 1 2016-09-17 02:53:19.056000 2 2 2016-09-17 05:53:19.436000
ThedataretrievedreflectsthefactthatwearenowinVancouverfromanapplicationperspective.LookingatthebasetableandyouwillseethateverythinghasbeenconvertedtoUTCtime.SELECT * FROM TXS_BASE; ID CUSTID TXTIME_UTC ----------- ----------- -------------------------- 1 1 2016-09-17 09:53:19.056000 2 2 2016-09-17 12:53:19.436000
Finally,wecanswitchbacktoTorontotimeandseewhenthetransactionsweredone.YouwillseethatfromaTorontoperspective
Chapter7:DateFunctions 76
thatthetransactionsweredonethreehourslaterbecauseofthetimezonedifferences.SET TIME_ZONE = 'America/Toronto'; SELECT * FROM TXS; ID CUSTID TXTIME ----------- ----------- -------------------------- 1 1 2016-09-17 05:53:19.056000 2 2 2016-09-17 08:53:19.436000
Chapter8:BinaryManipulation 78
HexFunctions
SeveralnewHEXmanipulationfunctionshavebeenaddedtoDB211.1.Thereareaclassoffunctionsthatmanipulatedifferentsizeintegers(SMALL,INTEGER,BIGINT)usingNOT,OR,AND,andXOR.Inadditiontothesefunctions,thereareseveralfunctionsthatdisplayandconvertvaluesintohexadecimalvalues.
INTNFunctionsTheINTNfunctionsarebitwisefunctionsthatoperateonthe"two'scomplement"representationoftheintegervalueoftheinputargumentsandreturntheresultasacorrespondingbase10integervalue.Thefunctionnamesallincludethesizeoftheintegersthatarebeingmanipulated:Table10:ArgumentSizes
Identifier(N) IntegerSize2 SMALLINT–2bytes4 INTEGER–4bytes8 BIGINT–8bytesTherearefourfunctions:
• INTNAND-PerformsabitwiseANDoperation• INTNOR-PerformsabitwiseORoperation• INTNXORPerformsabitwiseexclusiveORoperation• INTNNOT-PerformsabitwiseNOToperation
ThisexamplewillshowthefourfunctionsusedagainstINT2datatypes. WITH LOGIC(EXAMPLE, X, Y, RESULT) AS ( VALUES ('INT2AND(X,Y)',1,3,INT2AND(1,3)), ('INT2OR(X,Y) ',1,3,INT2OR(1,3)), ('INT2XOR(X,Y)',1,3,INT2XOR(1,3)), ('INT2NOT(X) ',1,3,INT2NOT(1)) ) SELECT * FROM LOGIC; EXAMPLE X Y RESULT ------------ ------ ------ ------ INT2AND(X,Y) 1 3 1 INT2OR(X,Y) 1 3 3 INT2XOR(X,Y) 1 3 2 INT2NOT(X) 1 3 -2
Chapter8:BinaryManipulation 79
YoucanmixandmatchtheINT2,INT4,andINT8valuesinthesefunctionsbutyoumaygettruncationifthevalueistoobig.
TO_HEXFunctionTheTO_HEXfunctionconvertsanumericexpressionintoacharacterhexadecimalrepresentation.Forexample,thenumericvalue255representsx'FF'.ThevaluereturnedfromthisfunctionisaVARCHARvalueanditslengthdependsonthesizeofthenumberyousupply.VALUES TO_HEX(255); 1 -------- ff
RAWTOHEXFunctionTheRAWTOHEXfunctionreturnsahexadecimalrepresentationofavalueasacharacterstring.Theresultisacharacterstringitself.VALUES RAWTOHEX('00'); 1 ---- 3030
Thestring"00"convertstoahexrepresentationofx'3030'whichis12336inDecimal.So,theTO_HEXfunctionwouldconvertthisbacktotheHEXrepresentation. VALUES TO_HEX(12336); 1 -------- 3030
Chapter9:RegularExpressions 81
RegularExpressions
DB211.1introducedsupportforregularexpressions.Regularexpressionsallowyoutodoverycomplexpatternmatchingincharacterstrings.NormalSQLLIKEsearchesarelimitedtoveryspecificpatterns,butregularexpressionshavearichsyntaxthatgivesyoumuchmoreflexibilityinsearching.
RegularExpressionExamplesTheexamplesinthissectionareusingalistofsomeoftheundergroundstationsofLondonUndergroundCentralline(existingstationsonly,nothistoricalones!).Thistablewillbeusedforalloftheexampleswithinthissection.CREATE TABLE CENTRAL_LINE ( STATION_NO INTEGER GENERATED ALWAYS AS IDENTITY, STATION VARCHAR(31), UPPER_STATION VARCHAR(31) GENERATED ALWAYS AS (UCASE(STATION)) );
Thelistofstationsis:Table1:LondonCentralLine(Subset)
WestRuislip RuislipGardens SouthRuislip NortholtGreenford Perivale HangerLane EalingBroadwayWestActon NorthActon EastActon WhiteCityShepherd'sBush HollandPark NottingHillGate QueenswayLancasterGate MarbleArch BondStreet OxfordCircusTottenhamCourtRoad
Holborn ChanceryLane St.Paul's
Bank LiverpoolStreet BethnalGreen MileEndStratford Leyton Leytonstone WansteadRedbridge GantsHill NewburyPark BarkingsideFairlop Hainault GrangeHill ChigwellRodingValley Snaresbrook SouthWoodford WoodfordBuckhurstHill Loughton Debden TheydonBoisEpping
Chapter9:RegularExpressions 82
RegularExpressionCommandsTherearesixregularexpressionfunctionswithinDB2including:
• REGEXP_COUNT–Returnsacountofthenumberoftimesthataregularexpressionpatternismatchedinastring.
• REGEXP_EXTRACT–Returnsoneoccurrenceofasubstringofastringthatmatchestheregularexpressionpattern.
• REGEXP_INSTR–Returnsthestartingorendingpositionofthematchedsubstring,dependingonthevalueofthereturnoptionargument.
• REGEXP_LIKE–ReturnsaBooleanvalueindicatingiftheregularexpressionpatternisfoundinastring.Thefunctioncanbeusedonlywhereapredicateissupported.
• REGEXP_MATCH_COUNT–Returnsacountofthenumberoftimesthataregularexpressionpatternismatchedinastring.
• REGEXP_REPLACE–Returnsamodifiedversionofthesourcestringwhereoccurrencesoftheregularexpressionpatternfoundinthesourcestringarereplacedwiththespecifiedreplacementstring.
• REGEXP_SUBSTR–Returnsoneoccurrenceofasubstringofastringthatmatchestheregularexpressionpattern.
Eachoneofthesefunctionsfollowsasimilarcallingsequence: REGEXP_FUNCTION(source, pattern, flags, start_pos, codeunits)
Theargumentstothefunctionare:
• Source–stringtobesearched• Pattern–theregularexpressionthatcontainswhatweare
searchingfor• Flag–settingsthatcontrolhowmatchingisdone• Start_pos–wheretostartinthestring• Codeunits–whichtypeunitofmeasurementstart_posrefersto
(forUnicode)
ThesourcecanbeanyvalidDB2stringincludingCHAR,VARCHAR,CLOB,etc.Start_posisthelocationinthesourcestringthatyouwanttostartsearchingfrom,andcodeunitstellsDB2whethertotreatthestart_posasanabsolutelocation(thinkbytelocation)oracharacterlocationwhichtakesintoaccounttheUnicodesizeofthecharacterstring.
Chapter9:RegularExpressions 83
CodeunitscanbespecifiedasCODEUNITS16,CODEUNITS32,orOCTETS.CODEUNITS16specifiesthatstartisexpressedin16-bitUTF-16codeunits.CODEUNITS32specifiesthatstartisexpressedin32-bitUTF-32codeunits.OCTETSspecifiesthatstartisexpressedinbytes.Patternandflagvaluesarecomplexandsoarediscussedinthefollowingsections.
RegularExpressionFlagValuesRegularexpressionfunctionshaveaflagspecificationthatcanbeusedtochangethebehaviorofthesearch.TherearesixpossibleflagsthatcanbespecifiedaspartoftheREGEXPcommand:Table11:RegularExpressionFlags
Flag Meaningc Specifiesthatmatchingiscase-sensitive(thedefaultvalue)I Specifiesthatmatchingiscaseinsensitivem Specifiesthattheinputdatacancontainmorethanoneline.Bydefault,the
'^'inapatternmatchesonlythestartoftheinputstring;the'$'inapatternmatchesonlytheendoftheinputstring.Ifthisflagisset,"^"and"$"alsomatchesatthestartandendofeachlinewithintheinputstring.
n Specifiesthatthe'.'characterinapatternmatchesalineterminatorintheinputstring.Bydefault,the'.'characterinapatterndoesnotmatchalineterminator.Acarriage-returnandline-feedpairintheinputstringbehavesasasingle-lineterminator,andmatchesasingle"."inapattern.
s Specifiesthatthe'.'characterinapatternmatchesalineterminatorintheinputstring.Thisvalueisasynonymforthe'n'value.
x Specifiesthatwhitespacecharactersinapatternareignored,unlessescaped.
RegularExpressionSearchPatternsRegularexpressionsusecertaincharacterstorepresentwhatismatchedinastring.Thesimplestpatternisastringbyitself.
Chapter9:RegularExpressions 84
SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'Ruislip'); STATION -------------------- West Ruislip Ruislip Gardens South Ruislip
Thepattern'Ruislip'willlookforamatchofRuislipwithintheSTATIONcolumn.Notethatthispatternwillalsomatch'WestRuislip'or'RuislipGardens'sinceweplacednorestrictiononwherethepatterncanbefoundinthestring.Thematchwillalsobeexact(casematters).ThistypeofsearchwouldbeequivalenttousingtheSQLLIKEstatement:SELECT STATION FROM CENTRAL_LINE WHERE STATION LIKE '%Ruislip%'; STATION -------------------- West Ruislip Ruislip Gardens South Ruislip
Ifyoudidn'tplacethe%atthebeginningoftheLIKEstring,onlythestationsthatstartwithRuislipwouldbefound.SELECT STATION FROM CENTRAL_LINE WHERE STATION LIKE 'Ruislip%'; STATION -------------------- Ruislip Gardens
IfyouwanttomatchRuislipwithupperorlowercasebeingignored,youwouldaddthe'i'flagaspartoftheREGEXP_LIKE(oranyREGEXPfunction).SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'RUISLIP','i'); STATION -------------------- West Ruislip Ruislip Gardens South Ruislip
AnchoringPatternsinaSearchBydefault,apatternwillbematchedanywhereinastring.OurpreviousexampleshowedhowRuislipcouldbefoundanywhereinastring.To
Chapter9:RegularExpressions 85
forceamatchtostartatthebeginningofastring,thecaratsymbol^canbeusedtoforcethematchtooccuratthebeginningofastring. SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'^Ruislip'); STATION -------------------- Ruislip Gardens
Tomatchapatternattheendofthestring,thedollarsign$canbeused.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'Ruislip$'); STATION -------------------- West Ruislip South Ruislip
Toforceanexactmatchwithastringyouwoulduseboththebeginningandendanchors.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'^Leyton$'); STATION -------------------- Leyton
Notethatifwedidn'tusetheendanchor,wearegoingtogetmorethanoneresult.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'^Leyton'); STATION -------------------- Leyton Leytonstone
MatchingpatternsacrossmultiplelinesSofartheexampleshavedealtwithstringsthatdonotcontainnewlinecharacters(orcarriagefeeds).Insomeapplications,datafromaninputpanelmayincludemultiplelineswhichmaycontainhardlinefeeds.Whatthismeansisthattherearemultiplelinesinthedata,butfromadatabaseperspective,thereisonlyonelineintheVARCHARfield.YoucanmodifythebehavioroftheRegularExpressionsearchbyinstructingittohonortheCRLFcharactersaslinedelimiters.
Chapter9:RegularExpressions 86
ThefollowingSQLwillinsertasinglelinewithmultipleCRLFcharactersinittosimulateamulti-linetextstring.CREATE TABLE LONGLINE (NAME VARCHAR(255)); INSERT INTO LONGLINE VALUES 'George' || CHR(10) || 'Katrina';
SearchingforKatrinaatthebeginningandendofstringdoesn'twork.SELECT COUNT(*) FROM LONGLINE WHERE REGEXP_LIKE(NAME,'^Katrina$'); 1 ----------- 0
WecanoverridetheregularexpressionsearchbytellingittotreateachNL/CRLFastheendofastringwithinastring.SELECT COUNT(*) FROM LONGLINE WHERE REGEXP_LIKE(NAME,'^Katrina$','m'); 1 ----------- 1
LogicalOROperatorRegularexpressionscanmatchmorethanonepattern.TheORoperator(|)isusedtodefinealternativepatternsthatcanmatchinastring.Thefollowingexamplesearchesforstationsthathave"ing"intheirnameaswellas"hill".SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'way|ing'); STATION -------------------- Ealing Broadway Notting Hill Gate Queensway Barkingside Roding Valley Epping
Somethingstobeawareofwhencreatingthesearchpattern.Spacesinthepatternsthemselvesaresignificant.Iftheprevioussearchpatternhadaspaceinoneofthewords,itwouldnotfindit(unlessofcoursetherewasaspaceinthestationname).
Chapter9:RegularExpressions 87
SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'way| ing'); STATION -------------------- Ealing Broadway Queensway
Usingthe"x"flagwillignoreblanksinyourpattern,sothiswouldfixissuesthatwehaveinthepreviousexample.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'way| ing','x'); STATION -------------------- Ealing Broadway Notting Hill Gate Queensway Barkingside Roding Valley Epping
Bracketscanbeusedtomakeitclearwhatthepatternisthatyouaresearchingforandavoidtheproblemofhavingblanksintheexpression.Bracketsdohaveaspecificusageinregularexpressions,buthereweareusingitonlytoseparatethetwosearchstrings.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(way)|(ing)'); STATION -------------------- Ealing Broadway Notting Hill Gate Queensway Barkingside Roding Valley Epping
CombiningPatternsAswefoundoutintheprevioussection,thereisanORoperatorthatyoucanusetoselectbetweentwopatterns.Howdoyourequestthatmultiplepatternsbepresent?Firstwemustunderstandhowmatchingoccurswhenwehavemultiplestringsthatneedtobematchedthathaveanunknownnumberofcharactersbetweenthem.
Chapter9:RegularExpressions 88
Forinstance,howdowecreateapatternthatlooksfor"ing"followedby"way"somewhereinthestring?Regularexpressionsrecognizethe"."(period)characterasmatchinganything.Followingthepattern,youcanaddamodifierthatspecifieshowmanytimesyouwantthepatternmatched:Table12:LondonCentralLine(Subset)
Char Meaning* Matchzeroormoretimes? Matchzerooronetimes+ Matchoneormoretimes{m} Matchexactlymtimes{m,} Matchasleastaminimumofmtimes{m,n} Matchatleastaminimumofmtimesandnomorethanntimes
Thefollowingregularexpressionsearchesforapatternwith"ing"followedbyanycharactersandthen"way".SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(ing)*.(way)'); STATION -------------------- Ealing Broadway Queensway
Thepreviousanswergaveyoutworesults(EalingBroadwayandQueensway).Whytwo?Thereasonisthatweusedthe*inthewrongplace(asinglecharacterinawrongplacecanresultinverydifferentresults!).Whatwereallyneededtodowasplacea.*afterthe(ing)tomatch"ing"andthenanycharacters,beforematching"way".Whatourquerydidabovewasmatch0ormoreoccurrencesof"ing",followedbyanycharacter,andthenmatch"way".Hereisthecorrectquery.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(ing).*(way)') STATION
--------------------Ealing Broadway
Findingatleastoneoccurrenceofapatternrequirestheuseofthe+operator,orthebracketoperators.Thisexamplelocatesatleastoneoccurrenceofthe"an"stringinstationnames.
Chapter9:RegularExpressions 89
SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(an)+'); STATION -------------------- Hanger Lane Holland Park Lancaster Gate Chancery Lane Bank Wanstead Gants Hill Grange Hill
Ifwewanttofindanexactnumberofoccurrences,weneedtousethe{}notationtotelltheregularexpressionmatcherhowmanywewanttofind.Thesyntaxofthe{}matchis:
• {m}–Matchexactlymtimes• {m,}–Matchasleastaminimumofmtimes• {m,n}–Matchatleastaminimumofmtimesandnomorethann
times
Sothe"+"symbolisequivalenttothefollowingregularexpressionusingthe{}syntax.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(an){1,}'); STATION -------------------- Hanger Lane Holland Park Lancaster Gate Chancery Lane Bank Wanstead Gants Hill Grange Hill
Ifwewanttomatchexactly2'an'patternsinastring,wewouldthinkthatchangingtheexpressionto{2}wouldwork.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(an){2}'); No records found.
Sadly,wegetnoresults!Thiswouldappeartobethewrongresult,butit'sbecausewegotluckywithourfirstsearch!Thebestwaytofigureout
Chapter9:RegularExpressions 90
whatwematchedintheoriginalqueryistousetheREGEXP_EXTRACTandREGEXP_INSTRfunctions.
• REGEXP_EXTRACT–Returnsoneoccurrenceofasubstringofastringthatmatchestheregularexpressionpattern.
• REGEXP_INSTR–Returnsthestartingorendingpositionofthematchedsubstring,dependingonthevalueofthereturnoptionargument.
ThefollowingSQLgivesusacluetowhatwasfoundwiththe(an)pattern.SELECT STATION, REGEXP_INSTR(STATION,'(an)') AS LOCATION, REGEXP_EXTRACT(STATION,'(an)') AS EXTRACT FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(an)'); STATION LOCATION EXTRACT -------------------- ----------- -------------------- Hanger Lane 2 an Holland Park 5 an Lancaster Gate 2 an Chancery Lane 3 an Bank 2 an Wanstead 2 an Gants Hill 2 an Grange Hill 3 an
Whatyoushouldseeinthepreviousresultisthelocationwherethe"an"patternwasfoundinthestring.Notethatallwematchedwasthe"an"pattern,nothingelse.Sowhycan'tIfindtwo"an"patternsinthestring?Thereasonisthat(an){2}means"an"followedbyanother"an"!Wedidn'ttellthepatterntomatchanythingelse!Whatweneedtodoismodifythepatterntosaythatitcanmatch"an"followedbyanythingelse.Thepatternneedstobemodifiedto(an).*wherethe".*"meansanycharacterfollowingthe"an".Inordertotelltheregularexpressionfunctiontousethisentirepattern(an).*twice,weneedtoplacebracketsarounditaswell.Thefinalpatternis((an).*){2}.
Chapter9:RegularExpressions 91
SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'((an).*){2}'); STATION -------------------- Hanger Lane Chancery Lane
Youshouldfindthattwostationsmatchthepattern.ThefollowingSQLshowswhichpatternismatchedfirstintheSTATIONSnames.SELECT STATION, REGEXP_INSTR(STATION,'((an).*){2}') AS LOCATION, REGEXP_EXTRACT(STATION,'((an).*){2}') AS EXTRACT FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'((an).*){2}'); STATION LOCATION EXTRACT -------------------- ----------- -------------------- Hanger Lane 2 anger Lane Chancery Lane 3 ancery Lane
MatchingcharactertypesAsidefrommatchingentirestrings,youcanalsouseregularexpressiontolookforpatternsofcharacters.Thesimplestmatchingpatternistheperiod(.)whichmatchesanycharacter.Matchingastringofarbitrarylengthisthepattern".*".The"+"and"?"characterscanalsobeusedtomodifyhowmanycharactersyouwantmatched.Whataboutsituationswhereyouwanttocheckforcertainpatternsorcharactersinastring?Agoodexamplewouldbeasocialsecuritynumber,orcreditcardnumber.Therearecertainpatternsthatyouwouldfindfortheseobjects.Assumewehaveasocialsecuritynumberintheformatxxx-xx-xxxx.ItispossibletocreatearegularexpressionthatwouldreturntrueiftheSSNmatchedthepatternabove(itdoesn'ttellyouiftheSSNitselfisvalid,onlythatithastheproperformat).Regularexpressionsallowyoutocreatealistofcharactersthatneedtobematchedinsomethingcalledabracketexpression.Abracketexpressionhastheformat:[a-z] [A-Z] [0-9] [a-zA-z]
Chapter9:RegularExpressions 92
Theexamplesaboverepresentthefollowingsearchpatterns:
• [a-z]-matchanyseriesoflowercasecharactersbetweenaandz• [A-Z]-matchanyseriesofuppercasecharactersbetweenAandZ• [0-9]-matchanyvaliddigits• [a-zA-Z]-matchanylower-oruppercaseletters
Youcanalsoenumerateallofthecharactersyouwanttomatchbylistingthembetweenthebracketslike[abcdefghikjlmnopqrstuvwxyz].Theshortforma-ziseasiertoreadandlesspronetotypingerrors!ThefollowingexamplechecksforstationnamesthatbeginwiththeletterP-R.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'^[P-R]'); STATION -------------------- Ruislip Gardens Perivale Queensway Redbridge Roding Valley
IfyouwantedtoincludeallstationsthathavetheletterP-Rorp-e,youcouldaddtheconditionwithinthebrackets.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'[p-rP-R]'); STATION -------------------- West Ruislip Ruislip Gardens South Ruislip Northolt Greenford Perivale Hanger Lane Ealing Broadway North Acton Shepherd's Bush
BacktoourSSNquestion.Canaregularexpressionpatternbeusedtodeterminewhetherthestringisinthecorrectformat?ThedatawillbeintheformatXXX-XX-XXXXsotheregularexpressionneedstofindthethreenumericvaluesseparatedbydashes.
Chapter9:RegularExpressions 93
Thenumberpatterncanberepresentedwiththebracketexpression[0-9].Tospecifythenumberofcharactersthatneedtobefound,weusethebraces{}tospecifytheexactnumberrequired.Forthethreenumbersinthepatternwecanuse[0-9]{3},[0-9]{2},and[0-9]{4}.Addinginthedashesgivesusthefinalpattern.TheSQLbelowcheckstoseeifaSSNiscorrect.VALUES CASE WHEN REGEXP_LIKE('123-34-1422','[0-9]{3}-[0-9]{2}-[0-9]{4}') THEN 'Valid' ELSE 'Invalid' END; 1 ------- Valid
TheSSNisvalidintheexampleabove.Herearesomeotherexamplestoshowwhetherornottheregularexpressionpicksupalloftheerrors.WITH SSNS(SSN) AS ( VALUES '123-34-1322', 'ABC-34-9999', 'X123-44-0001', '123X-Y44-Z0001', '111-222-111' ) SELECT SSN, CASE WHEN REGEXP_LIKE(SSN,'[0-9]{3}-[0-9]{2}-[0-9]{4}') THEN 'Valid' ELSE 'Invalid' END FROM SSNS; SSN 2 -------------- ------- 123-34-1322 Valid ABC-34-9999 Invalid X123-44-0001 Valid 123X-Y44-Z0001 Invalid 111-222-111 Invalid
Ifyoucheckclosely,oneofthestringswasmarkedasvalid,althoughitisnotcorrect(X123-44-0001).Thereasonthisoccurredisthatthepatternwasfoundafterthe"X"anditwascorrect.Topreventthisfromhappening,weneedtoanchorthepatternatthebeginningtoavoidthissituation.Abetterpatternwouldbetoanchorbothendsofthepattern
Chapter9:RegularExpressions 94
sothereisnopossibilityofothercharactersbeingatthebeginningorendofthepattern.WITH SSNS(SSN) AS ( VALUES '123-34-1322', 'ABC-34-9999', 'X123-44-0001', '123X-Y44-Z0001', '111-222-111' ) SELECT SSN, CASE WHEN REGEXP_LIKE(SSN,'^[0-9]{3}-[0-9]{2}-[0-9]{4}$') THEN 'Valid' ELSE 'Invalid' END FROM SSNS; SSN 2 -------------- ------- 123-34-1322 Valid ABC-34-9999 Invalid X123-44-0001 Invalid 123X-Y44-Z0001 Invalid 111-222-111 Invalid
SpecialPatternsThepreviousexampleusedthe[0-9]syntaxtorequestthatonlynumbersbefoundinthepattern.Therearesomepredefinedpatternsthatdefinethesecommonpatterns.ThefirstargumentisPosixformat(ifitexists),thesecondistheescapecharacterequivalent,andthefinaloneistherawpatternitrepresents.Table13:RegularExpressionSpecialPatterns
Posix Escape Pattern Meaning[:alnum:] [A-Za-z0-9] Alphanumericcharacters \w [A-Za-z0-9_] Alphanumericcharactersplus"_" \W [^A-Za-z0-9_] Non-wordcharacters[:alpha:] \a [A-Za-z] Alphabeticcharacters[:blank:] \s,\t Spaceandtab \b Wordboundaries[:cntrl:] [\x00-\x1F\x7F] Controlcharacters[:digit:] \d [0-9] Digits \D [^0-9] Non-digits[:graph:] [\x21-\x7E] Visiblecharacters[:lower:] \l [a-z] Lowercaseletters[:print:] \p [\x20-\x7E] Visiblecharactersandthespacecharacter
Chapter9:RegularExpressions 95
[:punct:] [][!"#$%&'()*+,./:;<=>?@\^_`{|}~-]
Punctuationcharacters
[:space:] \s [\t\r\n\v\f] Whitespacecharacters \S [^\t\r\n\v\f] Non-whitespacecharacters[:upper:] \u [A-Z] Uppercaseletters[:xdigit:] \x [A-Fa-f0-9] HexadecimaldigitsForinstance,thefollowingthreestatementswillproducethesameresult.WITH SSNS(SSN) AS ( VALUES '123-34-1322', 'ABC-34-9999', 'X123-44-0001', '123X-Y44-Z0001', '111-222-111' ) SELECT 'Original', SSN, CASE WHEN REGEXP_LIKE(SSN,'^[0-9]{3}-[0-9]{2}-[0-9]{4}$') THEN 'Valid' ELSE 'Invalid' END FROM SSNS UNION ALL SELECT 'Posix', SSN, CASE WHEN REGEXP_LIKE(SSN,'^[:digit:]{3}-[:digit:]{2}-[:digit:]{4}$') THEN 'Valid' ELSE 'Invalid' END FROM SSNS UNION ALL SELECT 'Escape', SSN, CASE WHEN REGEXP_LIKE(SSN,'^\d{3}-\d{2}-\d{4}$') THEN 'Valid' ELSE 'Invalid' END FROM SSNS; 1 SSN 3 -------- -------------- ------- Original 123-34-1322 Valid Original ABC-34-9999 Invalid Original X123-44-0001 Invalid Original 123X-Y44-Z0001 Invalid Original 111-222-111 Invalid Posix 123-34-1322 Valid Posix ABC-34-9999 Invalid Posix X123-44-0001 Invalid Posix 123X-Y44-Z0001 Invalid Posix 111-222-111 Invalid
NegatingPatternsUptothispointintime,thepatternsthathavebeenusedarelookingforapositivematch.Insomecases,youmaywanttofindvaluesthatdonot
Chapter9:RegularExpressions 96
match.TheeasiestwayistonegatetheactualREGEXP_LIKEexpression.Thefollowingexpressionfindsallofthestationsthatstartandendwith"West".SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'^West$'); No records found.
AddingtheNOTmodifierinfrontoftheREGEXPfunctiongivesusthestationsthatdonotbeginwithWest.SELECT STATION FROM CENTRAL_LINE WHERE NOT REGEXP_LIKE(STATION,'^West$'); STATION -------------------- West Ruislip Ruislip Gardens South Ruislip Northolt Greenford Perivale Hanger Lane Ealing Broadway West Acton North Acton
Youcanalsonegatesomeofthesearchesinapatternbyusingthe[^...]syntaxwherethe^tellstheregularexpressionnottomatchthefollowingcharacters.Theexpression[^0-9]wouldmeanmatchanycharacterswhicharenotnumeric.However,regularexpressionshavesomethingcallednegativelookaroundswhichbasicallymeanfindthepatternwhichdoesnotmatch.Youcreatethispatternbyaddingthe(?!..)atthebeginningofthestring.Thesamequery(findingstationsthatdon'tstartwithWest)wouldbewrittenwiththislookaroundlogicfoundintheSQLbelow.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'^(?!West)'); STATION -------------------- Ruislip Gardens South Ruislip Northolt Greenford Perivale Hanger Lane Ealing Broadway
Chapter9:RegularExpressions 97
North Acton East Acton White City
CapturingParenthesisThepreviousexampleusedsomethingcalledanegativelookaroundwithcapturingparenthesis.Whenyouplaceapatternwithinasetofbrackets(...)thestringthatmatchesthispatternis"remembered".Thestringsthatarematchedcanbeusedinsubsequentpartsofyourregularexpression.Thisallowsaformofprogrammingwithinyourregularexpression!Eachsetofparenthesesthatarematchedareassociatedwithanumber,startingatoneandincrementingforeachsubsequentpatternmatch.Forinstance,thefollowingpatternwillhavethreematches:^([0-9]{3})-([0-9]{3})-([0-9]{3})$
ThisisliketheSSNexampleusedearlieroninthissection.Thedifferenceinthisexampleisthateachblockofnumbersisthesame(3digits).Thispatternwillmatchanysequenceofnumbersintheformat123-456-789.WITH SSNS(SSN) AS ( VALUES '123-456-789', '123-555-123', '890-533-098', '123-456-456' ) SELECT SSN, CASE WHEN REGEXP_LIKE(SSN,'^([0-9]{3})-([0-9]{3})-([0-9]{3})$') THEN 'Valid' ELSE 'Invalid' END FROM SSNS; SSN 2 ----------- ------- 123-456-789 Valid 123-555-123 Valid 890-533-098 Valid 123-456-456 Valid
Thesenumbersfitthepatternandshouldbevalid.Whenoneofthecapturingparenthesismatches,itwillrememberthestringthatitmatched.Forinstance,inthefirstexample(123-456-789),thefirstmatchwillfindthestring'123'.Thesecondmatchwillfind'456'andsoon.Wecanrefertothesematchedstringswiththespecialcontrolcharacters\n
Chapter9:RegularExpressions 98
wherenrepresentsthecapturingparenthesislocation.So\1wouldrefertothe'123'thatwasfound.\2wouldbefor'456'.Theregularexpressionwillbeupdatedsothatthelastpartofthepatternneedstobenumeric(0-9)butcan'tbesameasthefirstmatch.WITH SSNS(SSN) AS ( VALUES '123-456-789', '123-555-123', '890-533-098', '123-456-456' ) SELECT SSN, CASE WHEN REGEXP_LIKE(SSN,'^([0-9]{3})-([0-9]{3})-(?!\1)([0-9]{3})$') THEN 'Valid' ELSE 'Invalid' END FROM SSNS; SSN 2 ----------- ------- 123-456-789 Valid 123-555-123 Invalid 890-533-098 Valid 123-456-456 Valid
Inmanycases,itmaybeeasiertofindthepatternsthatmatchandthennegatetheREGEXPstatement!The(?...)syntaxisusedforavarietyofpurposesinregularexpressions:Table14:CapturingParentheses
Pattern Description Details(?:...) Non-capturing
parenthesesGroupstheincludedpattern,butdoesnotprovidecapturingofmatchingtext.Moreefficientthancapturingparentheses.
(?>...) Atomic-matchparentheses
Firstmatchoftheparenthesizedsubexpressionistheonlyonetried.Ifitdoesnotleadtoanoverallpatternmatch,backupthesearchforamatchtoapositionbeforethe"(?>"
(?#...) Free-formatcomment
(?#comment)
(?=...) Look-aheadassertion
Trueiftheparenthesizedpatternmatchesatthecurrentinputposition,butdoesnotadvancetheinputposition.
(?!...) Negativelook-aheadassertion
Trueiftheparenthesizedpatterndoesnotmatchatthecurrentinputposition.Doesnotadvancetheinputposition.
Chapter9:RegularExpressions 99
(?<=...) Look-behindassertion
Trueiftheparenthesizedpatternmatchestextthatprecedesthecurrentinputposition.Thelastcharacterofthematchistheinputcharacterjustbeforethecurrentposition.Doesnotaltertheinputposition.Thelengthofpossiblestringsthatismatchedbythelook-behindpatternmustnotbeunbounded(no*or+operators.)
(?<!...) NegativeLook-behindassertion
Trueiftheparenthesizedpatterndoesnotmatchtextthatprecedesprecedingthecurrentinputposition.Thelastcharacterofthematchistheinputcharacterjustbeforethecurrentposition.Doesnotaltertheinputposition.Thelengthofpossiblestringsthatismatchedbythelook-behindpatternmustnotbeunbounded(no*or+operators.)
Forefficiencyinmatching,thebestapproachistoplacestringsthatyouaresearchingforinnon-capturingparentheses(?:...)ratherthanthegeneric()parenthesis.Thefollowingexamplefindsallstationswith"West"inthename.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(West)'); STATION -------------------- West Ruislip West Acton
ThefollowingSQLisequivalent,exceptthatthematchedpatternisnotkeptforfutureuseinmatching.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(?:West)'); STATION -------------------- West Ruislip West Acton
Chapter10:StatisticalFunctions 101
StatisticalFunctions
DB2alreadyhasavarietyofStatisticalfunctionsbuiltin.InDB211.1,anumberofnewfunctionshavebeenaddedincluding:
• COVARIANCE_SAMP–Samplecovarianceofasetofnumberpairs• STDDEV_SAMP–Samplestandarddeviation• VARIANCE_SAMP–Samplevarianceofasetofnumbers• CUME_DIST-CumulativeDistribution• PERCENT_RANK-Percentilerank• PERCENTILE_DISCandPERCENTILE_CONT–Percentiles• MEDIAN• WIDTH_BUCKET
SamplingFunctionsThetraditionalVARIANCE,COVARIANCE,andSTDDEVfunctionshavebeenavailableinDB2foralongtime.Whencomputingthesevalues,theformulaeassumethattheentirepopulationhasbeencounted(N).Thetraditionalformulaforstandarddeviationis:
𝜎 = 1𝑁
𝑥' − 𝜇 *+
',-
Nreferstothesizeofthepopulationandinmanycases,weonlyhaveasample,nottheentirepopulationofvalues.Inthiscase,theformulaneedstobeadjustedtoaccountforthesampling.
𝑠 = 1
𝑁 − 1𝑥' − 𝑥 *
+
',-
Chapter10:StatisticalFunctions 102
ThethreefunctionsthatuseasamplingalgorithmareCOVARIANCE_SAMP,STDDEV_SAMP,andVARIANCE_SAMP.COVARIANCE_SAMPreturnsthesamplecovarianceofasetofnumberpairs. SELECT COVARIANCE_SAMP(SALARY, BONUS) FROM EMPLOYEE WHERE WORKDEPT = 'A00'; 1 ------------------------------------------ 5428750
STDDEV_SAMPreturnsthesamplestandarddeviation(divisionby[n-1])ofasetofnumbers.SELECT STDDEV_SAMP(SALARY) FROM EMPLOYEE WHERE WORKDEPT = 'A00'; 1 ------------------------------------------ 46863.03180546474203969595277486485
VARIANCE_SAMPreturnsthesamplevariance(divisionby[n-1])ofasetofnumbers.SELECT VARIANCE_SAMP(SALARY) FROM EMPLOYEE WHERE WORKDEPT = 'A00';
1 ------------------------------------------ 2196143750
AdditionalAnalyticalFunctionsDB211.1alsoincludesseveralanalyticfunctionsthatDataScientistswillfinduseful.ThesefunctionsincludeMEDIAN,CUME_DIST,PERCENT_RANK,PERCENTILE_DISC,PERCENTILE_CONT,andWIDTH_BUCKET.
Chapter10:StatisticalFunctions 103
TheMEDIANcolumnfunctionreturnsthemedianvalueinasetofvalues.SELECT MEDIAN(SALARY) AS MEDIAN, AVG(SALARY) AS AVERAGE FROM EMPLOYEE WHERE WORKDEPT = 'E21'; MEDIAN AVERAGE ----------------------------- ------------------------------------ 41895 47086.66666666666666666666666666667
CUME_DISTreturnsthecumulativedistributionofarowthatishypotheticallyinsertedintoagroupofrows.SELECT CUME_DIST(47000) WITHIN GROUP (ORDER BY SALARY) FROM EMPLOYEE WHERE WORKDEPT = 'A00'; 1 ------------------------------------------ 0.5
ThePERCENT_RANKcolumnfunctionreturnstherelativepercentilerankofarowthatishypotheticallyinsertedintoagroupofrows.SELECT PERCENT_RANK(47000) WITHIN GROUP (ORDER BY SALARY) FROM EMPLOYEE WHERE WORKDEPT = 'A00'; 1 ------------------------------------------ 0.4
ThePERCENTILE_DISCreturnsthevaluethatcorrespondstothespecifiedpercentilegivenasortspecificationbyusingdiscrete(DISC)orcontinuous(CONT)distribution.SELECT PERCENTILE_DISC(0.75) WITHIN GROUP (ORDER BY SALARY) FROM EMPLOYEE WHERE WORKDEPT = 'E21'; 1 ----------- 45370.00 SELECT PERCENTILE_CONT(0.75) WITHIN GROUP (ORDER BY SALARY) FROM EMPLOYEE WHERE WORKDEPT = 'E21'; 1 ---------------------- 4.49875000000000E+004
Chapter10:StatisticalFunctions 104
WIDTHBUCKETandHistogramExampleTheWIDTH_BUCKETfunctionisusedtocreateequal-widthhistograms.UsingtheEMPLOYEEtable,thisSQLwillassignabuckettoeachemployee'ssalaryusingarangeof35000to100000dividedinto13buckets.SELECT EMPNO, SALARY, WIDTH_BUCKET(SALARY, 35000, 100000, 13) FROM EMPLOYEE ORDER BY EMPNO; EMPNO SALARY 3 ------ ----------- ----------- 000010 152750.00 14 000020 94250.00 12 000030 98250.00 13 000050 80175.00 10 000060 72250.00 8 000070 96170.00 13 000090 89750.00 11 000100 86150.00 11 000110 66500.00 7 000120 49250.00 3
Wecanplotthisinformationbyaddingsomemoredetailstothebucketoutput.WITH BUCKETS(EMPNO, SALARY, BNO) AS (SELECT EMPNO, SALARY, WIDTH_BUCKET(SALARY, 35000, 100000, 13) aS BUCKET FROM EMPLOYEE ORDER BY EMPNO) SELECT BNO, COUNT(*) FROM BUCKETS GROUP BY BNO ORDER BY BNO ASC; BNO 2 ----------- ----------- 0 1 1 10 2 3 3 8 4 2 5 2 6 2 7 5 8 2 10 1
Chapter10:StatisticalFunctions 105
Plottingthedataresultsinthefollowingbarchart.
Figure16:HistogramofEmployeeSalaries
Chapter11:HashingFunctions 107
HashingFunctions
Hashingfunctionsaretypicallyusedtotakelargertextstringsandmapintoasmaller"hashed"value.Thesehashvaluescanbe4byte(HASH4),8byte(HASH8)orupto64byteslong(HASH).ThereasonforusingvariousHASHalgorithmsdependsonthesizeofthedataandhowmanycollisionsyouexpecttoget.AHASHfunctiondoesnotnecessarilyresultinauniquevalueforeverystringthatishashed.Toavoidcollisions,alargeHASHvalueisoftenused.ThetradeoffistheamountofcomputepowerrequiredtogeneratetheHASHvalueaswellasthesizeoftheHASHkey.HASHvaluescanbeusedforavarietyofpurposes.HASHvaluescanbeusedtocreateartificialpartitionkeysforatable,createachecksumfortransportingthedatatoanapplicationoranotherdatabase,orforindexingphrases.Theapplicationmustbeawarethatduplicatescouldoccursoadditionalcodingmayberequired.
HASH4-Four-byteHashEncodingHASH4isthesimplestofthehashingfunctions,butalsohasthepotentialforthemostcollisionswhencomputinghashvalues.TheHASH4takestwoarguments:thestringtobehashed,andthealgorithmthatistobeusedforthehash. HASH4(string, algorithm)
TheHASH4functionreturnsa4-byteintegervalue.Thealgorithmvaluecanbeeither0or1.Zero(0)isthedefaultvalueandusestheAdleralgorithm.One(1)usestheCRC32algorithm.TheAdleralgorithmprovidesafasterchecksumhash;however,ithaspoorcoveragewhenthestringsarelessthanafewhundredbytesinsize.Poorcoveragemeansthattwodifferentstringshashtothesamevalue,referredtoasa"collision".Inthiscase,usetheCRC32algorithm,orswitchtoHASH8instead.
Chapter11:HashingFunctions 108
ThefollowingSQLwillHASHaphraseusingthedefault(Adler)algorithm.VALUES HASH4('Hello there'); 1 ----------- 409338925
ChangingthephraseslightlyresultsinadifferentHASHvalue.VALUES HASH4('Hello therf'); 1 ----------- 409404462
TheSQLismodifiedtousetheCRC32algorithm.VALUES HASH4('Hello there',1); 1 ----------- -342989177
HASH8-Eight-byteHashEncodingTheHASH8functionisidenticaltotheHASH4functionexceptthatitreturnsan8-byte(BIGINT)value.TherearetwoargumentstotheHASH8function.Thefirstisthestringtobehashedandthesecondisthealgorithmtouse.WiththisversionofDB2,thereisonlyonevalueallowed(0)whichtellsthefunctiontousetheJenkinsalgorithm. HASH8(string, algorithm)
TheHASH8functionwillhaveahigherprobabilityofreturninguniquevaluesforallstrings,butthedownsideisthatitwillrequiremorecyclestocomputethehashvalue.VALUES HASH8('Hello there'); 1 -------------------- -844407019926684877
Chapter11:HashingFunctions 109
Supplyinganythingotherthanazeroforthesecondargumentwillresultinanerrorbeingreturned.VALUE HASH8('Hello there',1); SQL0104N An unexpected token "VALUE HASH8" was found following "BEGIN-OF-STATEMENT". Expected tokens may include: "<create_index>". SQLSTATE=42601
HASH-VariableHashEncodingTheHASHfunctionisthemostflexibleofallthehashingfunctions.IthasaformatidenticaltotheothertwoHASHfunctions: HASH(string, algorithm)
Therearefourchoicesforalgorithm:Table15:HASHAlgorithmOptions
# Algorithm Length Bytes ReturnValues0 MD5 128 16 2**1281 SHA1 160 20 2**1602 SHA2_256 256 32 2**2563 SHA2_512 512 64 2**512TheresultofthefunctionisaVARBINARYfield.NotethattheMD5andSHA1algorithmshavehadsecurityflawsidentifiedwiththemsoyoumaywanttoconsiderusingahigherstrengthalgorithmifyouareusingthisforsecuritypurposes.VALUES HASH('Hello there',0), HASH('Hello there',1), HASH('Hello there',2), HASH('Hello there',3); 1 ------------------------------------------------------------------ E8EA7A8D1E93E8764A84A0F3DF4644DE 726C76553E1A3FDEA29134F36E6AF2EA05EC5CCE 4E47826698BB4630FB4451010062FADBF85D61427CBDFAED7AD0F23F239BED89 567683DDBA1F5A576B68EC26F41FFBCC7E718D646839AC6C2EF746FE952CEF4CBE6DEA...
Chapter12:ExperimentalJSONFunctions 111
InternalDB2JSONRoutines
ThecurrentDB211.1releaseincludesadriver-basedJSONsolutionthatembracestheflexibilityoftheJSONdatarepresentationwithinarelationaldatabase.UserscanuseaJSONprogrammingparadigmthatismodeledaftertheMongoDBdatamodelandquerylanguageandstorethedatanativelyinDB2.TheJSONdataisstoredinabinary-encodedformatcalledBSON(BinaryJSON).BSONisdesignedtobelightweight,easilytraversedandveryefficientlyencodedanddecoded.UserscaninteractwithJSONdatainmanyways.
• TheycanadministerandinteractivelyqueryJSONdatausingacommandlineshell.
• TheycanprogrammaticallystoreandquerydatafromJavaprogramsusinganIBMprovidedJavadriverforJSONthatallowsuserstoconnecttotheirJSONdatathroughthesameJDBCdriverusedforSQLaccess.
• TheycanuseanydriverthatimplementsportionsoftheMongoDBprotocol.ThisallowsthemtoaccesstheirDB2JSONstorefromavarietyofmodernlanguages,includingnode.js,PHP,Python,andRuby,aswellasmoretraditionallanguagessuchasC,C++,andPerl.
TheseapproachestomanipulatingJSONwithinDB2useavarietyofuser-definedfunctions(UDFs).TheseUDFsarecataloguedwheneveraJSONdatastoreisdefinedwithinDB2.DevelopersdonothaveaccesstotheseSQL-basedroutinessincetheywereoriginallydesignedtobeusedonlybythesespecificJSONinterfaces.However,manycustomerswouldliketostoreandqueryaJSONcolumnthatisdefinedaspartofaDB2table.Whiletheseroutineshavenotbeenofficiallypublished,theyareavailableforcustomeruse.TheseroutinesarenotexternalizedintheDB2documentationbecausetheywereonlyusedbytheinternalAPI'sforqueryingandmanagingJSON.Forthisreason,theseroutineswerenotoriginallyconsidered"supported".Thischaptergivesyoudetalsofthesefunctionsandtheyarenowavailabletouseinyourapplications.NotethattheinternaluseoftheseroutinesbyDB2hasveryspecificusagepatterns,whichmeans
Chapter12:ExperimentalJSONFunctions 112
thatitispossiblethatyoumaygenerateasetofSQLthatmaynotbehandledproperly.Note:ThesefunctionsmaychangeinthefuturetoconformtotheSQLstandard.
DB2JSONFunctionsThereisonebuilt-inDB2JSONfunctionandseveralotherfunctionsthatmustberegisteredwithinDB2beforetheycanbeused.Thenamesofthefunctionsandtheirpurposearedescribedbelow.
• JSON_VAL–ExtractsdatafromaJSONdocumentintoSQLdatatypes
• JSON_TABLE–Returnsatableofvaluesforadocumentthathasarraytypesinit
• JSON_TYPE–ReturnsthedatatypeofaspecificfieldwithinaJSONdocument
• JSON_LEN–Returnsthecountofelementsinanarraytypeinsideadocument
• BSON2JSON–ConvertBSONformatteddocumentintoJSONstrings
• JSON2BSON–ConvertJSONstringsintoaBSONdocumentformat• JSON_UPDATE–Updateafieldordocumentusingsetsyntax• JSON_GET_POS_ARR_INDEX–Findavaluewithinanarray• BSON_VALIDATE–CheckstomakesurethataBSONfieldina
BLOBobjectisinacorrectformat
AsidefromtheJSON_VALfunction,allotherfunctionsinthislistmustbecataloguedbeforefirstbeingused.Ascriptthatcanbeusedtocatalogthesefunctions(DB2-V11-JSON-Examples.sql)isavailableinthedownloadfolderforthiseBook.Inaddition,alltheexamplesinthischapterarealsoincludedinthisscript.Pleaserefertotheappendixfordetailsonaccessingthisfileandotherusefulinformation.
PathStatementRequirementsAlltheDB2JSONfunctionshavebeenplacedintotheSYSTOOLSschema.Thismeansthattoexecuteanyofthesecommands,youmustprefixthe
Chapter12:ExperimentalJSONFunctions 113
commandwithSYSTOOLS,asinSYSTOOLS.JSON2BSON.Toremovethisrequirement,youmustupdatetheCURRENTPATHvaluetoincludeSYSTOOLSaspartofit.TheSQLbelowwilltellyouwhatthecurrentPATHis. VALUES CURRENT PATH; 1 ------------------------------------------------- "SYSIBM","SYSFUN","SYSPROC","SYSIBMADM","BAKLARZ"
UsethefollowingSQLtoaddSYSTOOLStothecurrentpath. SET CURRENT PATH = CURRENT PATH, SYSTOOLS;
Fromthispointonyouwon'tneedtoaddtheSYSTOOLSschemaonthefrontofanyofyourSQLstatementsthatrefertotheseDB2JSONfunctions.
CreatingTablesthatSupportJSONDocumentsTocreateatablethatwillstoreJSONdata,youneedtodefineacolumnthatcancontainbinarydata.TheJSONcolumnmustbecreatedasaBLOB(binaryobject)datatypebecausethenewVARBINARYdatatypewillnotworkwithanyoftheJSONfunctions.JSONDocumentsarealwaysstoredinBLOBSwhichcanbeaslargeas16M.Toensuregoodperformance,youshouldhaveaBLOBspecifiedasINLINEifpossible.Ifalargeobjectisnotinlined,orgreaterthan32Kinsize,theresultingobjectwillbeplacedintoalargetablespace.TheresultisthatBLOBobjectswillnotbekeptinbufferpools(whichmeansadirectreadisrequiredfromdiskforaccesstoanyBLOBobject)andthattwoI/Osarerequiredtogetanydocument–onefortherowandasecondforthedocument.ByusingtheINLINEoptionandkeepingtheBLOBsizebelowthepagesize,youcanimprovetheretrievalperformanceofJSONcolumns.ThisSQLwillcreateacolumnthatissuitableforstoringJSONdata: CREATE TABLE TESTJSON ( JSON_FIELD BLOB(4000) INLINE LENGTH 4000 );
Chapter12:ExperimentalJSONFunctions 114
NotethatweareassumingthatthesizeoftheJSONobjectwillnotexceed4000charactersinsize.
JSONDocumentRepresentationDB2storesJSONobjectsinamodifiedbinary-encodedformatcalledBSON(BinaryJSON).Asmentionedatthebeginningofthischapter,BSONisdesignedtobelightweight,easilytraversedandveryefficientlyencodedanddecoded.AlltheJSONfunctionsexecuteagainstBSONobjects,nottheoriginalJSONtextstringsthatmayhavebeengeneratedbyanapplication.Tostoreandretrieveanentiredocumentfromacolumninatable,youmustuse:
• BSON2JSON–ConvertaBSONformatteddocumentintoaJSONstring
• JSON2BSON–ConvertJSONstringsintoaBSONdocumentformat
YoucanalsoverifythecontentsofadocumentthatisstoredinacolumnbyusingtheBSON_VALIDATEfunction:
• BSON_VALIDATE–CheckstomakesurethataBSONfieldinaBLOBobjectisinthecorrectformat
YoucannotuseJSONtoBSONconversionfunctionsthatareavailableinsomeprogramminglanguages.TheBSONformatusedbyDB2hassomeextensionstoitthatwillnotbehandledbytheseroutines,soyoumustusetheDB2BSON2JSONandJSON2BSONfunctionsexclusively.
JSON2BSON:InsertingaJSONDocumentInsertingintoaDB2columnrequirestheuseoftheJSON2BSONfunction.TheJSON2BSONfunction(andBSON2JSON)areusedtotransferdatainandoutofatraditionalDB2BLOBcolumn.InputtotheJSON2BSONfunctionmustbeaproperlyformattedJSONdocument.IfthedocumentdoesnotfollowproperJSONrules,youwillgetanerrorcodefromthefunction. INSERT INTO TESTJSON VALUES ( JSON2BSON('{Name:"George"}'));
Chapter12:ExperimentalJSONFunctions 115
ApoorlyformattedJSONdocumentwillreturnanerror. INSERT INTO TESTJSON VALUES ( JSON2BSON('{Name:, Age: 32}')); SQL0443N Routine "JSON2BSON" (specific name "") has returned an error SQLSTATE with diagnostic text "JSON parsing error for: {Name:, Age: 32}, error code: 4 ". SQLSTATE=22546
BSON2JSON:RetrievingaJSONDocumentSincethedatathatisstoredinaJSONcolumnisinaspecialbinaryformatcalledBSON,selectingfromthefieldwillonlyresultinrandomcharactersbeingdisplayed. SELECT * FROM TESTJSON; JSON_FIELD -------------------- 0316000000024E616D65
IfyouwanttoextractthecontentsofaJSONfield,youmustusetheBSON2JSONfunction. SELECT BSON2JSON(JSON_FIELD) FROM TESTJSON; 1 -------------------- {"Name":"George"}
OnethingthatyouwillnoticeisthattheretrievedJSONhasbeenmodifiedslightlysothatallthevalueshavequotesaroundthemtoavoidanyambiguity.ThisisduetotheconversiontoBSONformatandbacktoJSON.Notethatwedidn'tnecessarilyrequirethequoteswhenweinsertedthedata.Forinstance,ouroriginalJSONdocumentcontainedthefollowingfield: {Name:"George"}
Whatgetsreturnedisslightlydifferent,butstillconsideredtobethesameJSONdocument. {"Name":"George"}
Youmustensurethatthenamingofanyfieldsisconsistentbetweendocuments."Name","name",and"Name"areallconsidereddifferentfields.Oneoptionistouselowercasefieldnames,ortousecamel-case(firstletteriscapitalized)inallyourfielddefinitions.Theimportantthing
Chapter12:ExperimentalJSONFunctions 116
istokeepthenamingconsistentsoyoucanfindthefieldsinthedocument.
DeterminingDocumentSizeThesizeofBLOBfieldwillbedependentonthesizeofdocumentsthatyouexpecttostoreinthem.ConvertingaJSONdocumentintoaBSONformatusuallyresultsinasmallerdocumentsize.ThesampleJSON_EMPtable,with42documentscontains11,764bytesofdata,andwhenconvertedtoBSONcontains11,362bytes.TherewillbeoccasionswheretheBSONformatmaybelargerthantheoriginalJSONdocument.Thisexpansioninsizeoccurswhendocumentshavemanyintegerornumericvalues.Anintegertypetakesup4bytesinaBSONdocument.Convertingacharactervaluetoanativeintegerformatwilltakeupmorespacebutresultinfasterretrievalandcomparisonoperations.
BSON_VALIDATE:CheckingtheFormatofaDocumentDB2hasnonativeJSONdatatype,sotherenovalidationdoneagainstthecontentsofaBLOBcolumnwhichcontainsJSONdata.IftheJSONobjectisunderprogramcontrolandyouareusingtheJSON2BSONandBSON2JSONfunctions,youareprobablynotgoingtorunacrossproblemswiththedata.Ifyoubelievethatadocumentiscorruptedforsomereason,youcanusetheBSON_VALIDATEtomakesureitisokay(ornot!).Thefunctionwillreturnavalueof1iftherecordisokay,orazerootherwise.TheonerowthatwehaveinsertedintotheTESTJSONtableshouldbeokay. SELECT BSON_VALIDATE(JSON_FIELD) FROM TESTJSON; 1 ----------- 1
ThefollowingSQLwillinjectabadvalueintothebeginningoftheJSONfieldtotesttheresultsfromtheBSON_VALIDATEfuntion. UPDATE TESTJSON SET JSON_FIELD = BLOB('!') || JSON_FIELD;
Chapter12:ExperimentalJSONFunctions 117
SELECT BSON_VALIDATE(JSON_FIELD) FROM TESTJSON; 1 ----------- 0
RetrievingJSONDocumentsThelastsectiondescribedhowwecaninsertandretrieveentireJSONdocumentsfromacolumninatable.ThissectionwillexploreseveralfunctionsthatallowaccesstoindividualfieldswithintheJSONdocument.Thesefunctionsare:
• JSON_VAL–ExtractsdatafromaJSONdocumentintoSQLdatatypes
• JSON_TABLE–Returnsatableofvaluesforadocumentthathasarraytypesinit
• JSON_TYPE–ReturnsthedatatypeofaspecificfieldwithinaJSONdocument
• JSON_LEN–Returnsthecountofelementsinanarraytypeinsideadocument
• JSON_GET_POS_ARR_INDEX–Retrievetheindexofavaluewithinanarraytypeinadocument
SampleJSONTableCreation
ThefollowingSQLwillloadtheJSON_EMPtablewithseveralJSONobjects.TheserecordsaremodelledaroundtheSAMPLEdatabaseEMPLOYEEtable. CREATE TABLE JSON_EMP ( SEQ INT NOT NULL GENERATED ALWAYS AS IDENTITY, EMP_DATA BLOB(4000) INLINE LENGTH 4000 );
ThefollowingcommandwillloadtherecordsintotheJSON_EMPtable.OnlythefirstINSERTisdisplayed,butthereisatotalof42recordsincluded.ADB2scriptfile(DB2-V11-JSON-Examples.sql)containingalltheJSONexamplescanbefoundinthesamedirectoryasthiseBook.SeetheAppendixformoredetailsonhowtogetacopyofthisfile.
Chapter12:ExperimentalJSONFunctions 118
INSERT INTO JSON_EMP(EMP_DATA) VALUES JSON2BSON( '{ "empno":"000070", "firstnme":"EVA", "midinit":"D", "lastname":"PULASKI", "workdept":"D21", "phoneno":[7831,1422,4567], "hiredate":"09/30/2005", "job":"MANAGER", "edlevel":16, "sex":"F", "birthdate":"05/26/2003", "pay": { "salary":96170.00, "bonus":700.00, "comm":2893.00 } }');
AdditionalJSON_DEPTTable
InadditiontotheJSON_EMPtable,thefollowingSQLwillgenerateatablecalledJSON_DEPTthatcanbeusedtodeterminethenameofthedepartmentanindividualworksin. CREATE TABLE JSON_DEPT ( SEQ INT NOT NULL GENERATED ALWAYS AS IDENTITY, DEPT_DATA BLOB(4000) INLINE LENGTH 4000 ); INSERT INTO JSON_DEPT(DEPT_DATA) VALUES JSON2BSON('{"deptno":"A00", "mgrno":"000010", "admrdept":"A00", "deptname":"SPIFFY COMPUTER SERVICE DIV."}'), JSON2BSON('{"deptno":"B01", "mgrno":"000020", "admrdept":"A00", "deptname":"PLANNING" }'), JSON2BSON('{"deptno":"C01", "mgrno":"000030", "admrdept":"A00", "deptname":"INFORMATION CENTER" }'), JSON2BSON('{"deptno":"D01", "admrdept":"A00", "deptname":"DEVELOPMENT CENTER" }'), ... JSON2BSON('{"deptno":"J22", "admrdept":"E01", "deptname":"BRANCH OFFICE J2" }');
JSON_VAL:RetrievingDatafromaBSONDocumentNowthatwehaveinsertedsomeJSONdataintoatable,thissectionwillexploretheuseoftheJSON_VALfunctiontoretrieveindividualfieldsfromthedocuments.Thisbuilt-infunctionwillreturnavaluefromadocumentinaformatthatyouspecify.Theabilitytodynamicallychange
Chapter12:ExperimentalJSONFunctions 119
thereturneddatatypeisextremelyimportantwhenweexamineindexcreationinanothersection.TheJSON_VALfunctionhastheformat: JSON_VAL(document, field, type);
JSON_VALtakes3arguments:
• document–BSONdocument• field–Thefieldwearelookingfor(searchpath)• type–Thereturntypeofdatabeingreturned
Thesearchpathandtypemustbeconstants–theycannotbevariablessotheiruseinuser-definedfunctionsislimitedtoconstants.AtypicalJSONrecordwillcontainavarietyofdatatypesandstructuresasillustratedbythefollowingrecordfromtheJSON_EMPtable. { "empno":"200170", "firstnme":"KIYOSHI", "midinit":"", "lastname":"YAMAMOTO", "workdept":"D11", "phoneno":[2890], "hiredate":"09/15/2005", "job":"DESIGNER", "edlevel":16, "sex":"M", "birthdate":"01/05/1981", "pay": { "salary":64680.00, "bonus":500.00, "comm":1974.00 } }
Therearenumberoffieldswithdifferentformats,includingstrings(firstnme),integers(edlevel),decimal(salary),date(hiredate),anumberarray(phoneno),andastructure(pay).JSONdatacanconsistofnestedobjects,arraysandverycomplexstructures.TheformatofaJSONobjectischeckedwhenusingtheJSON2BSONfunctionandanerrormessagewillbeissuedifitdoesnotconformtotheJSONspecification.
Chapter12:ExperimentalJSONFunctions 120
TheJSON_VALfunctionneedstoknowhowtoreturnthedatatypebackfromtheJSONrecord,soyouneedtospecifywhattheformatshouldbe.Thepossibleformatsare:
Code Formatn DECFLOATi INTEGERI BIGINT(noticethisisalowercaseL)f DOUBLEd DATEts TIMESTAMP(6)t TIMEs:n AVARCHARwithasizeofnbeingthemaximumb:n ABINARYvaluewithnbeingthemaximumu Anintegerwithavalueof0or1.
RetrievingAtomicValues
Thisfirstexamplewillretrievethenameandsalaryoftheemployeewhoseemployeenumberis"200170" SELECT JSON_VAL(EMP_DATA,'lastname','s:20'), JSON_VAL(EMP_DATA,'pay.salary','f') FROM JSON_EMP WHERE JSON_VAL(EMP_DATA,'empno','s:6') = '200170'; 1 2 -------------------- ------------------------------------------ YAMAMOTO 64680
Ifthesizeofthefieldbeingreturnedislargerthatthefieldspecification,youwillgetaNULLvaluereturned,notatruncatedvalue. SELECT JSON_VAL(EMP_DATA,'lastname','s:7') FROM JSON_EMP WHERE JSON_VAL(EMP_DATA,'empno','s:6') = '200170'; 1 -------------------- -
Inthecaseofcharacterfields,youmayneedtospecifyalargerreturnsizeandthentruncateittogetasubsetofthedata.
Chapter12:ExperimentalJSONFunctions 121
SELECT LEFT(JSON_VAL(EMP_DATA,'lastname','s:20'),7) FROM JSON_EMP WHERE JSON_VAL(EMP_DATA,'empno','s:6') = '200170'; 1 ------- YAMAMOT
RetrievingArrayValues
Selectingdatafromanarraytypewillalwaysgiveyouthefirstvalue(elementzero).Theemployeesallhaveextensionnumbersbutsomeofthemhavemorethanone.Someoftheextensionsstartwithazerososincethecolumnisbeingtreatedasanintegeryouwillgetonly3digits.It'sprobablybettertodefineitasacharacterstringratherthananumber! SELECT JSON_VAL(EMP_DATA, 'phoneno', 'i') FROM JSON_EMP; 1 ----------- 3978
Ifyouspecify":na"afterthetypespecifier,youwillgetanerrorifthefieldisanarraytype.HopefullyyoualreadyknowtheformatofyourJSONdataandcanavoidhavingtochecktoseeifarraysexist.Whatthisstatementwilltellyouisthatoneoftherecordsyouwereattemptingtoretrievewasanarraytype.Infact,allthephoneextensionsarebeingtreatedasarraytypeseventhoughtheyhaveonlyonevalueinmanycases. SELECT JSON_VAL(EMP_DATA, 'phoneno', 'i:na') FROM JSON_EMP; SQL20556N The operation failed because multiple result values cannot be returned from a scalar function "SYSIBM.JSON_VAL". SQLSTATE=22547
Ifyouneedtoaccessaspecificarrayelementinafield,youcanusethe"dot"notationafterthefieldname.Thefirstelementstartsatzero.Ifweselectthe2ndelement(.1)alltheemployeesthathaveasecondextensionwillhaveavalueretrievedwhiletheoneswhodon'twillhaveanullvalue.
Chapter12:ExperimentalJSONFunctions 122
RetrievingStructuredFields
Structuredfieldsareretrievedusingthesamedotnotationasarrays.Thefieldisspecifiedbyusingthe"field.subfield"formatandthesefieldscanbeanarbitrarynumberoflevelsdeep.Thepayfieldintheemployeerecordismadeupofthreeadditionalfields. "pay": { "salary":64680.00, "bonus":500.00, "comm":1974.00 }
Toretrievethesethreefields,youneedtoexplictlynamethemsinceretrievingpayalonewillnotwork. SELECT JSON_VAL(EMP_DATA,'pay.salary','i'), JSON_VAL(EMP_DATA,'pay.bonus','i'), JSON_VAL(EMP_DATA,'pay.comm','i') FROM JSON_EMP WHERE JSON_VAL(EMP_DATA,'empno','s:6') = '200170'; 1 2 3 ----------- ----------- ----------- 64680 500 1974
Ifyouattempttoretrievethepayfield,youwillendupwithaNULLvalue,notanerrorcode.ThereasonforthisisthattheJSON_VALfunctioncannotformatthefieldintoanatomicvaluesoitreturnstheNULLvalueinstead.
DetectingNULLValuesinaField
Todeterminewhetherafieldexists,orhasanullvalue,youneedusethe"u"flag.Ifyouusethe"u"flag,thevaluereturnedwillbeeither:
• 1–Thefieldexists,andithasavalue(notnulloremptystring)• 0–Thefieldexists,butthevalueisnullorempty• null–Thefielddoesnotexist
IntheJSON_EMPtable,thereareafewemployeeswhodonothavemiddlenames.Thefollowingquerywillreturnavalueor1,0,orNULLdependingonwhetherthemiddlenameexistsforarecord.
Chapter12:ExperimentalJSONFunctions 123
SELECT JSON_VAL(EMP_DATA,'lastname','s:30'), JSON_VAL(EMP_DATA,'midinit','u') FROM JSON_EMP;
Theresultscontain40employeeswhohaveamiddleinitial,andtwothatdonot.Theresultscanbemisleadingbecauseanemployeecanhavethemidinitfielddefined,butnovalueassignedtoit: { "empno":"000120", "firstnme":"SEAN", "midinit":"", "lastname":"O''CONNELL",... }
Inthiscase,theemployeedoesnothaveamiddlename,butthefieldispresent.Todeterminewhetheranemployeedoesnothaveamiddlename,youwillneedtocheckforaNULLvalue(thefielddoesnotexist,orthefieldisempty)whenretrievingthemiddleinitial(9rows): SELECT COUNT(*) FROM JSON_EMP WHERE JSON_VAL(EMP_DATA,'midinit','s:40') IS NULL;
Ifyouonlywanttoknowhowmanyemployeehavethemiddleinitialfield(midinit)thatisempty,youneedtoexcludetherecordsthatdonotcontainthefield(7rows): SELECT COUNT(*) FROM JSON_EMP WHERE JSON_VAL(EMP_DATA,'midinit','s:40') IS NULL AND JSON_VAL(EMP_DATA,'midinit','u') IS NOT NULL;
JoiningJSONTables
YoucanjointableswithJSONcolumnsbyusingtheJSON_VALfunctiontocomparetwovalues: SELECT JSON_VAL(EMP_DATA,'empno','s:6') AS EMPNO, JSON_VAL(EMP_DATA,'lastname','s:20') AS LASTNAME, JSON_VAL(DEPT_DATA,'deptname','s:30') AS DEPTNAME FROM JSON_EMP, JSON_DEPT WHERE JSON_VAL(DEPT_DATA,'deptno','s:3') = JSON_VAL(EMP_DATA,'workdept','s:3') FETCH FIRST 5 ROWS ONLY; EMPNO LASTNAME DEPTNAME ------ -------------------- ------------------------------ 000010 HAAS SPIFFY COMPUTER SERVICE DIV. 000020 THOMPSON PLANNING 000030 KWAN INFORMATION CENTER
Chapter12:ExperimentalJSONFunctions 124
000050 GEYER SUPPORT SERVICES 000060 STERN MANUFACTURING SYSTEMS
YouneedtoensurethatthedatatypesfrombothJSONfunctionsarecompatibleforthejointoworkproperly.Inthiscase,thedepartmentnumberandtheworkdepartmentarebothreturnedas3-bytecharacterstrings.Ifyoudecidedtouseintegersinsteadorasmallerstringsize,thejoinwillnotworkasexpectedbecausetheconversionwillresultintruncatedorNULLvalues.IfyouplanondoingjoinsbetweenJSONobjects,youmaywanttoconsidercreatedindexesonthedocumentstospeedupthejoinprocess.Moreinformationontheuseofindexesisfoundattheendofthischapter.
JSONDataTypesIfyouareunsureofwhatdatatypeafieldcontains,youcanusethetheJSON_TYPEfunctiontodeterminethetypebeforeretrievingthefield.TheJSON_TYPEfunctionhastheformat: ID = JSON_TYPE(document, field, 2048);
JSON_TYPEtakes3arguments:
• document–BSONdocument• field–Thefieldwearelookingfor(searchpath)• searchpathsize–2048istherequiredvalue
The2048specifiesthemaximumlengthofthefieldparameterandshouldbeleftatthisvalue.WhenqueryingthedatatypeswithinaJSONdocument,thefollowingvaluesarereturned.ID TYPE ID TYPE1 Double 10 Null2 String 11 RegularExpression3 Object 12 Futureuse4 Array 13 JavaScript5 Binarydata14 Symbol6 Undefined15 Javascript(withscope)
Chapter12:ExperimentalJSONFunctions 125
ID TYPE ID TYPE7 Objectid 16 32-bitinteger8 Boolean 17 Timestamp9 Date 18 64-bitintegerThenextSQLstatementwillcreateatablewithstandardtypeswithinit. CREATE TABLE TYPES (DATA BLOB(4000) INLINE LENGTH 4000); INSERT INTO TYPES VALUES JSON2BSON( '{ "string" : "string", "integer" : 1, "number" : 1.1, "date" : {"$date": "2016-06-20T13:00:00"}, "boolean" : true, "array" : [1,2,3], "object" : {type: "main", phone: [1,2,3]} }');
ThefollowingSQLwillgeneratealistofdatatypesandfieldnamesfoundwithinthisdocument. SELECT 'STRING',JSON_TYPE(DATA, 'string', 2048) FROM TYPES UNION ALL SELECT 'INTEGER',JSON_TYPE(DATA, 'integer', 2048) FROM TYPES UNION ALL SELECT 'NUMBER',JSON_TYPE(DATA, 'number', 2048) FROM TYPES UNION ALL SELECT 'DATE',JSON_TYPE(DATA, 'date', 2048) FROM TYPES UNION ALL SELECT 'BOOLEAN', JSON_TYPE(DATA, 'boolean', 2048) FROM TYPES UNION ALL SELECT 'ARRAY', JSON_TYPE(DATA, 'array', 2048) FROM TYPES UNION ALL SELECT 'OBJECT', JSON_TYPE(DATA, 'object', 2048) FROM TYPES; 1 2 --------- ----------- ARRAY 4 BOOLEAN 8 DATE 9 NUMBER 1 INTEGER 16 STRING 2 OBJECT 3
Chapter12:ExperimentalJSONFunctions 126
ExtractingFieldsUsingDifferentDataTypesThefollowingsectionswillshowhowwecangetatomic(non-array)typesoutoftheJSONdocuments.Wearenotgoingtobespecificwhichdocumentswewantasidefromthefieldwewanttoretrieve.AtemporarytablecalledSANDBOXisusedthroughouttheseexamples: CREATE TABLE SANDBOX (DATA BLOB(4000) INLINE LENGTH 4000);
JSONINTEGERSandBIGINT
IntegerswithinJSONdocumentsareeasilyidentifiedasnumbersthatdon'thaveadecimalplacesinthem.TherearetwodifferenttypesofintegerssupportedwithinDB2andareidentifiedbythesize(numberofdigits)inthenumberitself.
• Integer–asetofdigitsthatdonotincludeadecimalplace.Thenumbercannotexceed−2,147,483,648to2,147,483,647
• Bigint–asetofdigitsthatdonotincludeadecimalplacebutexceedthatofaninteger.Thenumbercannotexceed–9,223,372,036,854,775,808to9,223,372,036,854,775,807
Youdon'texplicitystatethetypeofintegerthatyouareusing.Thesystemwilldetectthetypebasedonitssize.TheJSON_TYPEfunctionwillreturnavalueof16forintegersand18foralargeinteger(BIGINT).Toretrieveavaluefromanintegerfieldyouneedtousethe"i"flagand"l"(lowercaseL)forbigintegers.ThisfirstSQLstatementwillcreatearegularintegerfield. INSERT INTO SANDBOX VALUES JSON2BSON('{"count":9782333}');
TheJSON_TYPEfunctionwillverifythatthisisanintegerfield(Type=16). SELECT JSON_TYPE(DATA,'count',2048) AS TYPE FROM SANDBOX; TYPE ----------- 16
Chapter12:ExperimentalJSONFunctions 127
Youcanretrieveanintegervaluewitheitherthe'i'flagorthe'l'flag.ThisfirstSQLstatementretrievesthevalueasaninteger. SELECT JSON_VAL(DATA,'count','i') FROM SANDBOX; 1 ----------- 9782333
WecanaskthatthevaluebeinterpretedasaBIGINTbyusingthe'l'flag,soJSON_VALwillexpandthesizeofthereturnvalue. SELECT JSON_VAL(DATA,'count','l') FROM SANDBOX; 1 -------------------- 9782333
ThenextSQLstatementwillcreateafieldwithaBIGINTsize.Notethatwedon'tneedtospecifyanythingotherthanhaveaverybignumber! DELETE FROM SANDBOX; INSERT INTO SANDBOX VALUES JSON2BSON('{"count":94123512223422}');
TheJSON_TYPEfunctionwillverifythatthisisabigintegerfield(Type=18). SELECT JSON_TYPE(DATA,'count',2048) AS TYPE FROM SANDBOX; TYPE ----------- 18
WecanchecktoseethatthedataisstoredinthedocumentasaBIGINTbyusingtheJSON_TYPEfunction. SELECT JSON_TYPE(DATA,'count',2048) FROM SANDBOX; 1 ----------- 18
Returningthedataasanintegertype'i'willfailsincethenumberistoobigtofitintoanintegerformat.Notethatyoudonotgetanerrormessage-aNULLvaluegetsreturned.
Chapter12:ExperimentalJSONFunctions 128
SELECT JSON_VAL(DATA,'count','i') FROM SANDBOX; 1 ----------- -
Specifyingthe'I'flagwillmakethedatabereturnedproperly. SELECT JSON_VAL(DATA,'count','l') FROM SANDBOX; 1 -------------------- 94123512223422
SincewehaveanintegerintheJSONfield,wealsohavetheoptionofreturningthevalueasafloating-pointnumber(f)orasadecimalnumber(n).Eitheroftheseoptionswillworkwithintegervalues. SELECT JSON_VAL(DATA,'count','n') AS DECIMAL, JSON_VAL(DATA,'count','f') AS FLOAT FROM SANDBOX; DECIMAL FLOAT ---------------------------------- ---------------------- 94123512223422 9.41235122234220E+013
JSONNUMBERSandFLOATINGPOINT
JSONnumbersarerecognizedbyDB2whenthereisadecimalpointinthevalue.FloatingpointvaluesarerecognizedusingtheExxspecifierafterthenumberwhichrepresentsthepowerof10thatneedstobeappliedtothebasevalue.Forinstance,1.0E01isthevalue10.TheJSONtypefornumbersis1,whetheritisinfloatingpointformatordecimalformat.TheSQLstatementbelowinsertsasalaryintothetable(usingthestandarddecimalplacenotation). INSERT INTO SANDBOX VALUES JSON2BSON('{"salary":92342.20}');
TheJSON_TYPEfunctionwillverifythatthisisanumericfield(Type=1). SELECT JSON_TYPE(DATA,'salary',2048) AS TYPE FROM SANDBOX; TYPE ----------- 1
Chapter12:ExperimentalJSONFunctions 129
Numericdatacanberetrievedineithernumber(n)formant,integer(i-notethatyouwillgettruncation),orfloatingpoint(f). SELECT JSON_VAL(DATA,'salary','n') AS DECIMAL, JSON_VAL(DATA,'salary','i') AS INTEGER, JSON_VAL(DATA,'salary','f') AS FLOAT FROM SANDBOX; DECIMAL INTEGER FLOAT ------------------------------ ----------- ---------------------- 92342.199999999997 92342 9.23422000000000E+004
Youmaywonderwhynumberformat(n)resultsinananswerthathasafractionalcomponentthatisn'texactly92342.20.ThereasonisthatDB2isconvertingthevaluetoDECFLOAT(34)whichsupportsahigherprecisionnumber,butcanresultinfractionsthatcan'tbeaccuratelyrepresentedwithinthebinaryformat.CastingthevaluetoDEC(9,2)willproperlyformatthenumber. SELECT DEC(JSON_VAL(DATA,'salary','n'),9,2) AS DECIMAL FROM SANDBOX; DECIMAL ----------- 92342.20
Afloating-pointnumberisrecognizedbytheExxspecifierinthenumber.TheBSONfunctionwilltagthisvalueasanumbereventhoughyouspecifieditinfloatingpointformat.ThefollowingSQLinsertsthefloatingvalueintothetable. INSERT INTO SANDBOX VALUES JSON2BSON('{"salary":9.2523E01}');
TheJSON_TYPEfunctionwillverifythatthisisafloating-pointfield(Type=1). SELECT JSON_TYPE(DATA,'salary',2048) AS TYPE FROM SANDBOX; TYPE ----------- 1
Thefloating-pointvaluecanberetrievedasanumber,integer,orfloatingpointvalue.
Chapter12:ExperimentalJSONFunctions 130
SELECT JSON_VAL(DATA,'salary','n') AS DECIMAL, JSON_VAL(DATA,'salary','i') AS INTEGER, JSON_VAL(DATA,'salary','f') AS FLOAT FROM SANDBOX; DECIMAL INTEGER FLOAT ------------------------- ----------- ---------------------- 92.522999999999996 92 9.25230000000000E+001
JSONBOOLEANVALUES
JSONhasadatatypewhichcanbetrueorfalse(boolean).DB2doesn'thaveanequivalentdatatypeforboolean,soweneedtoretrieveitasanintegerorcharacterstring(true/false).TheJSONtypeforbooleanvaluesis8.TheSQLstatementbelowinsertsatrueandfalsevalueintothetable. INSERT INTO SANDBOX VALUES JSON2BSON('{"valid":true, "invalid":false}');
Wewilldouble-checkwhattypethefieldisintheJSONrecord. SELECT JSON_TYPE(DATA,'valid',2048) AS TYPE FROM SANDBOX; TYPE ----------- 8
Toretrievethevalue,wecanaskthatitbeformattedasanintegerornumber. SELECT JSON_VAL(DATA,'valid','n') AS TRUE_DECIMAL, JSON_VAL(DATA,'valid','i') AS TRUE_INTEGER, JSON_VAL(DATA,'invalid','n') AS FALSE_DECIMAL, JSON_VAL(DATA,'invalid','i') AS FALSE_INTEGER FROM SANDBOX; TRUE_DECIMAL TRUE_INTEGER FALSE_DECIMAL FALSE_INTEGER ------------ ------------ ------------- ------------- 1 1 0 0
Youcanalsoretrieveabooleanfieldasacharacterorbinaryfield,buttheresultsarenotwhatyouwouldexpectwithbinary.
Chapter12:ExperimentalJSONFunctions 131
SELECT JSON_VAL(DATA,'valid','s:5') AS TRUE_STRING, JSON_VAL(DATA,'valid','b:2') AS TRUE_BINARY, JSON_VAL(DATA,'invalid','s:5') AS FALSE_STRING, JSON_VAL(DATA,'invalid','b:2') AS FALSE_BINARY FROM SANDBOX; TRUE_STRING TRUE_BINARY FALSE_STRING FALSE_BINARY ----------- ----------- ------------ ------------ true 0801 false 0800
JSONDATE,TIME,andTIMESTAMPS
ThisfirstSQLstatementwillinsertaJSONfieldthatusesthe$datemodifier. INSERT INTO SANDBOX VALUES JSON2BSON('{"today":{"$date":"2016-07-01T12:00:00"}}');
QueryingthedatatypeofthisfieldusingJSON_VALwillreturnavalueof9(datetype). SELECT JSON_TYPE(DATA,'today',2048) FROM SANDBOX; 1 ----------- 9
Ifyoudecidetouseacharacterstringtorepresentadate,youcanuseeitherthe"s:x"specificationtoreturnthedateasastring,oruse"d"tohaveitdisplayedasadate.ThisfirstSQLstatementreturnsthedateasastring. INSERT INTO SANDBOX VALUES JSON2BSON('{"today":"2016-07-01"}'); SELECT JSON_VAL(DATA,'today','s:10') FROM SANDBOX; 1 ---------- 2016-07-01
Usingthe'd'specificationwillreturnthevalueasadate. SELECT JSON_VAL(DATA,'today','d') FROM SANDBOX; 1 ---------- 2016-07-01
Chapter12:ExperimentalJSONFunctions 132
Whatabouttimestamps?Ifyoudecidetostoreatimestampintoafield,youcanretrieveitinavarietyofways.ThisfirstsetofSQLstatementswillretrieveitasastring. INSERT INTO SANDBOX VALUES JSON2BSON('{"today":"' || VARCHAR(NOW()) || '"}'); SELECT JSON_VAL(DATA,'today','s:30') FROM SANDBOX; 1 ------------------------------ 2016-09-17-06.27.00.945000
RetrievingitasaDatewillalsowork,butthetimeportionwillberemoved. SELECT JSON_VAL(DATA,'today','d') FROM SANDBOX; 1 ---------- 2016-09-17
Youcanalsoaskforthetimestampvaluebyusingthe'ts'specification.Notethatyoucan'tgetjustthetimeportionunlessyouuseaSQLfunctiontocastit. SELECT JSON_VAL(DATA,'today','ts') FROM SANDBOX; 1 -------------------------- 2016-09-17 06:27:00.945000
Toforcethevaluetoreturnjustthetimeportion,eitherstorethedataasatimevalue(HH:MM:SS)stringorstoreatimestampandusetheTIMEfunctiontoextractjustthatportionofthetimestamp. SELECT TIME(JSON_VAL(DATA,'today','ts')) FROM SANDBOX; 1 -------- 06:27:00
JSONStrings
Forcharacterstrings,youmustspecifywhatthemaximumlengthis.Thisexamplewillreturnthesizeofthelastnamefieldas10characterslong.
Chapter12:ExperimentalJSONFunctions 133
SELECT JSON_VAL(DATA, 'lastname', 's:10') FROM JSON_EMP; 1 ---------- HAAS
Youmustspecifyalengthforthe's'parameterotherwiseyouwillgetanerrorfromthefunction.Ifthesizeofthecharacterstringistoolargetoreturn,thenthefunctionwillreturnanullvalueforthatfield. SELECT JSON_VAL(DATA, 'lastname', 's:8') FROM JSON_EMP; 1 -------- HAAS
JSON_TABLEFunctionThefollowingqueryworksbecausewedonottreatthefieldphonenoasanarray: SELECT JSON_VAL(DATA, 'phoneno', 'i') FROM JSON_EMP; 1 ----------- 3978
Bydefault,onlythefirstnumberofanarrayisreturnedwhenyouuseJSON_VAL.However,therewillbesituationswhereyoudowanttoreturnallthevaluesinanarray.ThisiswheretheJSON_TABLEfunctionmustbeused.TheformatoftheJSON_TABLEfunctionis: JSON_TABLE(document, field, type)
Theargumentsare:
• document–BSONdocument• field–Thefieldwearelookingfor• type–Thereturntypeofdatabeingreturned
JSON_TABLEreturnstwocolumns:TypeandValue.Thetypeisoneofapossible18valuesfoundinthetablebelow.TheValueistheactualcontentsofthefield.
Chapter12:ExperimentalJSONFunctions 134
ID TYPE ID TYPE1 Double 10 Null2 String 11 RegularExpression3 Object 12 Futureuse4 Array 13 JavaScript5 Binarydata14 Symbol6 Undefined15 Javascript(withscope)7 Objectid 16 32-bitinteger8 Boolean 17 Timestamp9 Date 18 64-bitinteger
TheTYPEfieldisprobablysomethingyouwouldn'trequireaspartofyourqueriessinceyouarealreadyspecifyingthereturntypeinthefunction.TheformatoftheJSON_TABLEfunctionislikeJSON_VALexceptthatitreturnsatableofvalues.YoumustusethisfunctionaspartofFROMclauseandatablefunctionspecification.Forexample,toreturnthecontentsofthephoneextensionarrayforjustoneemployee(000230)wecanusethefollowingJSON_TABLEfunction. SELECT PHONES.* FROM JSON_EMP E, TABLE( JSON_TABLE(E.EMP_DATA,'phoneno','i') ) AS PHONES WHERE JSON_VAL(E.EMP_DATA,'empno','s:6') = '000230'; TYPE VALUE ----------- -------------------- 16 2094 16 8999 16 3756
TheTABLE(...)specificationintheFROMclauseisusedfortablefunctions.TheresultsthatarereturnedfromtheTABLEfunctionaretreatedthesameasatraditionaltable.Tocreateaquerythatgivesthenameofeveryemployeeandtheirextensionswouldrequirethefollowingquery. SELECT JSON_VAL(E.EMP_DATA, 'lastname', 's:10') AS LASTNAME, PHONES.VALUE AS PHONE FROM JSON_EMP E, TABLE( JSON_TABLE(E.EMP_DATA,'phoneno','i') ) AS PHONES; LASTNAME PHONE ---------- -------------------- HAAS 3978 THOMPSON 3476 THOMPSON 1422
Chapter12:ExperimentalJSONFunctions 135
KWAN 4738 GEYER 6789 STERN 6423 STERN 2433 PULASKI 7831 PULASKI 1422 PULASKI 4567
Onlyasubsetoftheresultsisshownabove,butyouwillseethattherearemultiplelinesforemployeeswhohavemorethanoneextension.TheresultsofaTABLEfunctionmustbenamed(AS...)ifyouneedtorefertotheresultsoftheTABLEfunctionintheSELECTlistorinotherpartsoftheSQL.YoucanuseotherSQLoperatorstosortororganizetheresults.Forinstance,wecanusetheORDERBYoperatortofindoutwhichemployeeshavethesameextension.NotehowtheTABLEfunctionisnamedPHONESandtheVALUEScolumnisrenamedtoPHONE. SELECT JSON_VAL(E.EMP_DATA, 'lastname', 's:10') AS LASTNAME, PHONES.VALUE AS PHONE FROM JSON_EMP E, TABLE( JSON_TABLE(E.EMP_DATA,'phoneno','i') ) AS PHONES ORDER BY PHONE; LASTNAME PHONE ---------- -------------------- THOMPSON 1422 PULASKI 1422 SCHNEIDER 1422 O'CONNELL 1533 MEHTA 1533 ALONZO 1533 SCOUTTEN 1682 ORLANDO 1690
Youcanevenfoundouthowmanypeoplearesharingextensions!TheHAVINGclausetellsDB2toonlyreturngroupingswherethereismorethanoneemployeewiththesameextension. SELECT PHONES.VALUE AS PHONE, COUNT(*) AS COUNT FROM JSON_EMP E, TABLE( JSON_TABLE(E.EMP_DATA,'phoneno','i') ) AS PHONES GROUP BY PHONES.VALUE HAVING COUNT(*) > 1 ORDER BY PHONES.VALUE PHONE COUNT -------------------- ----------- 1422 3 1533 3
Chapter12:ExperimentalJSONFunctions 136
1793 2 2103 2 2167 2 2890 2 3332 2 3780 2
JSON_LENFunctionThepreviousexampleshowedhowwecouldretrievethevaluesfromwithinanarrayofadocument.Sometimesanapplicationneedstodeterminehowmanyvaluesareinthearrayitself.TheJSON_LENfunctionisusedtofigureoutwhatthearraycountis.TheformatoftheJSON_LENfunctionis: count = JSON_LEN(document,field)
Theargumentsare:
• document–BSONdocument• field–Thefieldwearelookingfor• count–NumberofarrayentriesorNULLifthefieldisnotanarray
Ifthefieldisnotanarray,thisfunctionwillreturnanullvalue,otherwiseitwillgiveyouthenumberofvaluesinthearray.Inourpreviousexample,wecoulddeterminethenumberofextensionsperpersonbytakingadvantageoftheJSON_LENfunction. SELECT JSON_VAL(E.EMP_DATA, 'lastname', 's:10') AS LASTNAME, JSON_LEN(E.EMP_DATA, 'phoneno') AS PHONE_COUNT FROM JSON_EMP E; LASTNAME PHONE_COUNT ---------- ----------- HAAS 1 THOMPSON 2 KWAN 1 GEYER 1 STERN 2 PULASKI 3 HENDERSON 1 SPENSER 1
JSON_GET_POS_ARR_INDEXFunctionTheJSON_TABLEandJSON_LENfunctionscanbeusedtoretrieveallthevaluesfromanarray,butsearchingforaspecificarrayvalueisdifficultto
Chapter12:ExperimentalJSONFunctions 137
do.OnewaytosearcharrayvaluesistoextracteverythingusingtheJSON_TABLEfunction. SELECT JSON_VAL(E.EMP_DATA, 'lastname', 's:10') AS LASTNAME, PHONES.VALUE AS PHONE FROM JSON_EMP E, TABLE( JSON_TABLE(E.EMP_DATA,'phoneno','i') ) AS PHONES WHERE PHONES.VALUE = 1422;
AneasierwaytosearchanarrayisbyusingtheJSON_GET_POS_ARR_INDEXfunction.ThisfunctionwillsearcharrayvalueswithouthavingtoextractthearrayvalueswiththeJSON_TABLEfunction.TheformatoftheJSON_GET_POS_ARR_INDEXfunctionis: element = JSON_GET_POS_ARR_INDEX(document, field)
Theargumentsare:
• document–BSONdocument• field–Thefieldwearelookingforanditsvalue• element–Thefirstoccurrenceofthevalueinthearray
Theformatofthefieldargumentis"{field:value}"anditneedstobeinBSONformat.ThismeansyouneedstoaddtheJSON2BSONfunctionaroundthefieldspecification. JSON2BSON( '{"field":"value"}' )
Thisfunctiononlytestsforequivalenceandthedatatypeshouldmatchwhatisalreadyinthefield.Thereturnvalueisthepositionwithinthearraythatthevaluewasfound,wherethefirstelementstartsatzero.InourJSON_EMPtable,eachemployeehasoneormorephonenumbers.ThefollowingSQLwillretrieveallemployeeswhohavetheextension1422: SELECT JSON_VAL(EMP_DATA, 'lastname', 's:10') AS LASTNAME FROM JSON_EMP WHERE JSON_GET_POS_ARR_INDEX(EMP_DATA, JSON2BSON('{"phoneno":1422}')) >= 0; LASTNAME ---------- THOMPSON PULASKI SCHNEIDER
Chapter12:ExperimentalJSONFunctions 138
Ifweusedquotesaroundthephonenumber,thefunctionwillnotmatchanyofthevaluesinthetable.
UpdatingJSONDocumentsThereareacoupleofapproachesavailabletoupdatingJSONdocuments.OneapproachistoextractthedocumentfromthetableinatextformusingBSON2JSONandthenusingstringfunctionsorregularexpressionstomodifythedata.TheotheroptionistousetheJSON_UPDATEstatement.ThesyntaxoftheJSON_UPDATEfunctionis: JSON_UPDATE(document, '{$set: {field:value}}')
Theargumentsare:
• document–BSONdocument• field–Thefieldwearelookingfor• value–Thevaluewewanttosetthefieldto
TherearethreepossibleoutcomesfromusingtheJSON_UPDATEstatement:
• Ifthefieldisfound,theexistingvalueisreplacedwiththenewone
• Ifthefieldisnotfound,thefield:valuepairisaddedtothedocument
• Ifthevalueissettothenullkeyword,thefieldisremovedfromthedocument
Thefieldcanspecifyaportionofastructure,oranelementofanarrayusingthedotnotation.ThefollowingSQLwillillustratehowvaluescanbeaddedandremovedfromadocument.Asinglerecordthatcontains3phonenumberextensionsareaddedtoatable: INSERT INTO SANDBOX VALUES JSON2BSON('{"phone":"[1111,2222,3333]"}');
Toaddanewfieldtotherecord,theJSON_UPDATEfunctionneedstospecifythefieldandvaluepair.
Chapter12:ExperimentalJSONFunctions 139
UPDATE SANDBOX SET DATA = JSON_UPDATE(DATA,'{ $set: {"lastname":"HAAS"}}');
Retrievingthedocumentshowsthatthelastnamefieldhasnowbeenaddedtotherecord. SELECT BSON2JSON(DATA) FROM SANDBOX; 1 ---------------------------------------------- {"phone":"[1111,2222,3333]","lastname":"HAAS"}
Ifyouspecifyafieldthatisanarraytypeanddonotspecifyanelement,youwillendupreplacingtheentirefieldwiththevalue. UPDATE SANDBOX SET DATA = JSON_UPDATE(DATA,'{ $set: {"phone":"9999"}}'); SELECT BSON2JSON(DATA) FROM SANDBOX; 1 ---------------------------------- {"phone":"9999","lastname":"HAAS"}
RunningtheSQLagainsttheoriginalphonedatawillworkproperly. UPDATE SANDBOX SET DATA = JSON_UPDATE(DATA,'{ $set: {"phone.0":9999}}'); SELECT BSON2JSON(DATA) FROM SANDBOX; 1 ------------------------------------------------------------ {"phone":"[9999,2222,3333]"}
IndexingJSONDocumentsDB2supportscomputedindexes,whichallowsfortheuseoffunctionslikeJSON_VALtobeusedaspartoftheindexdefinition.Forinstance,searchingforanemployeenumberwillresultinascanagainstthetableifnoindexesaredefined: SELECT JSON_VAL(EMP_DATA, 'lastname', 's:20') AS LASTNAME FROM JSON_EMP WHERE JSON_VAL(EMP_DATA, 'empno', 's:6') = '000010'; LASTNAME -------------------- HAAS
Chapter12:ExperimentalJSONFunctions 140
Cost = 13.628412 Rows Operator (ID) Cost 1.68 RETURN ( 1) 13.6284 | 1.68 TBSCAN ( 2) 13.6283 | 42 Table: BAKLARZ JSON_EMP
Tocreateanindexontheempnofield,weusetheJSON_VALfunctiontoextracttheempnofromtheJSONfield. CREATE INDEX IX_JSON ON JSON_EMP (JSON_VAL(EMP_DATA,'lastname','s:20'));
RerunningtheSQLresultsinthefollowingexplainplan: Cost = 6.811412 Rows Operator (ID) Cost 1.68 RETURN ( 1) 6.81141 | 1.68 FETCH ( 2) 6.8113 / \ 1.68 42 IXSCAN Table: ( 3) BAKLARZ 0.00484089 JSON_EMP | 42 Index: BAKLARZ IX_JSON
Chapter12:ExperimentalJSONFunctions 141
DB2cannowusetheindextoretrievetherecord.
SimplifyingJSONSQLInsertsandRetrievalFromadevelopmentperspective,youalwaysneedtoconvertdocumentstoandfromJSONusingtheBSON2JSONandJSON2BSONfunctions.Therearewaystohidethesefunctionsfromanapplicationandsimplifysomeoftheprogramming.OneapproachtosimplifyingtheconversionofdocumentsbetweenformatsistouseINSTEADOFtriggers.Thesetriggerscanintercepttransactionsbeforetheyareappliedtothebasetables.Thisapproachrequiresthatwecreateaviewontopofanexistingtable.ThefirststepistocreatethebasetablewithtwocopiesoftheJSONcolumn.OnewillcontaintheoriginalJSONcharacterstringwhilethesecondwillcontaintheconvertedBSON.Forthisexample,theJSONcolumnwillbecalledINFO,andtheBSONcolumnwillbecalledBSONINFO.TheuseoftwocolumnscontainingJSONwouldappearstrangeatfirst.ThereasonforthetwocolumnsisthatDB2expectstheBLOBcolumntocontainbinarydata.Youcannotinsertacharacterstring(JSON)intotheBSONcolumnwithoutconvertingitfirst.DB2willraiseanerrorsotheJSONcolumnistheretoavoidanerrorwhiletheconversiontakesplace.Fromadebuggingperspective,wecankeepboththeCLOBandBLOBvaluesinthistableifwewant.ThetriggerwillsettheJSONcolumntonullaftertheBSONcolumnhasbeenpopulated. CREATE TABLE BASE_EMP_TXS ( SEQNO INT NOT NULL GENERATED ALWAYS AS IDENTITY, INFO VARCHAR(4000), BSONINFO BLOB(4000) INLINE LENGTH 4000 );
TouseINSTEADOFtriggers,aviewneedstobecreatedontopofthebasetable.NotethatweexplicitlyusetheSYSTOOLSschematomakesurewearegettingthecorrectfunctionusedhere. CREATE OR REPLACE VIEW EMP_TXS AS (SELECT SEQNO, BSON2JSON(BSONINFO) AS INFO FROM BASE_EMP_TXS);
Chapter12:ExperimentalJSONFunctions 142
AtthispointwecancreatethreeINSTEADOFtriggerstohandleinsert,updatesanddeletesontheview.OnINSERTtheDEFAULTkeywordisusedtogeneratetheIDnumber,theJSONfieldissettoNULLandtheBSONcolumncontainstheconvertedvalueoftheJSONstring. CREATE OR REPLACE TRIGGER I_EMP_TXS INSTEAD OF INSERT ON EMP_TXS REFERENCING NEW AS NEW_TXS FOR EACH ROW MODE DB2SQL BEGIN ATOMIC INSERT INTO BASE_EMP_TXS VALUES ( DEFAULT, NULL, SYSTOOLS.JSON2BSON(NEW_TXS.INFO)
);END
OnUPDATES,thesequencenumberremainsthesame,andtheBSONfieldisupdatedwiththecontentsoftheJSONfield. CREATE OR REPLACE TRIGGER U_EMP_TXS INSTEAD OF UPDATE ON EMP_TXS REFERENCING NEW AS NEW_TXS OLD AS OLD_TXS FOR EACH ROW MODE DB2SQL BEGIN ATOMIC UPDATE BASE_EMP_TXS SET (INFO, BSONINFO) = (NULL, SYSTOOLS.JSON2BSON(NEW_TXS.INFO)) WHERE BASE_EMP_TXS.SEQNO = OLD_TXS.SEQNO; END
Finally,theDELETEtriggerwilljustremovetherow. CREATE OR REPLACE TRIGGER D_EMP_TXS INSTEAD OF DELETE ON EMP_TXS REFERENCING OLD AS OLD_TXS FOR EACH ROW MODE DB2SQL BEGIN ATOMIC DELETE FROM BASE_EMP_TXS WHERE BASE_EMP_TXS.SEQNO = OLD_TXS.SEQNO; END
ApplicationswillonlydealwiththeEMP_TXSview.AnyinsertswillusethetextversionoftheJSONandnothavetoworryaboutusingthe
Chapter12:ExperimentalJSONFunctions 143
JSON2BSONfunctionsincetheunderlyingINSTEADOFtriggerwilltakecareoftheconversion.ThefollowinginsertstatementonlyincludestheJSONstringsincethesequencenumberwillbegeneratedautomaticallyaspartoftheinsert.INSERT INTO EMP_TXS(INFO) VALUES ( '{ "empno":"000010", "firstnme":"CHRISTINE", "midinit":"I", "lastname":"HAAS", "workdept":"A00", "phoneno":[3978], "hiredate":"01/01/1995", "job":"PRES", "edlevel":18, "sex":"F", "birthdate":"08/24/1963", "pay" : { "salary":152750.00, "bonus":1000.00, "comm":4220.00} }');
SelectingfromtheEMP_TXSviewwillreturntheJSONinareadableformat: SELECT SEQNO, CAST(LEFT(INFO,50) AS VARCHAR(50)) FROM EMP_TXS;
SEQNO 2 ----------- --------------------------------------------------- 1 {"empno":"000010","firstnme":"CHRISTINE","midinit"}
ThebasetableonlycontainstheBSONbuttheviewtranslatesthevaluebackintoareadableformat.Anupdatestatementthatreplacestheentirestringworksasexpected. UPDATE EMP_TXS SET INFO = '{"empno":"000010"}' WHERE SEQNO = 1; SELECT SEQNO, CAST(LEFT(INFO,50) AS VARCHAR(50)) FROM EMP_TXS; SEQNO 2 ----------- -------------------------------------------------- 1 {"empno":"000010"}
IfyouwanttomanipulatetheBSONdirectly(saychangetheemployeenumber),youneedtorefertotheBASEtableinstead.
Chapter12:ExperimentalJSONFunctions 144
UPDATE BASE_EMP_TXS SET BSONINFO = JSON_UPDATE(BSONINFO, '{$set: {"empno":"111111"}}') WHERE SEQNO = 1;
Andwecancheckitusingouroriginalview. SELECT SEQNO, CAST(LEFT(INFO,50) AS VARCHAR(50)) FROM EMP_TXS;
SEQNO 2 ----------- -------------------------------------------------- 1 {"empno":"111111"}
Summary
ThecurrentDB211.1release(andDB210.5)includesseveraluser-definedfunctions(UDFs)thatwereoriginallydesignedtobeusedbyinternalJSONinterfaces.Basedonfeedbackfromseveralcustomers,we'vedocumentedwhatisusedinternallywithinDB2.Whiletheseroutineshavenotbeenofficiallypublished,theyareavailableforcustomeruse.Note:RememberthatthesefunctionsmaychangeinthefuturetoconformtotheSQLstandard.
AppendixA:AdditionalResourcesforDB2 146
ResourcestoBuildYourDB2Skills
RelyonthewiderangeofIBMexperts,programs,andservicesthatareavailabletohelpyoutakeyourInformationManagementskillstothenextlevel.ParticipateinouronlinecommunitythroughdeveloperWorks.Findtutorials,articles,whitepapers,videos,demos,bestpractices,DB2Expressdownloads,andmore.Visitibm.com/developerworks/data.
IBMCertificationExamsFindindustry-leadingprofessionalcertificationexams,includingnewcertificationsforDB210.5withBLUAccelerationandDB211.1:
• DB210.5DatabaseAdministrationUpgradeExam(Exam311)• DB211.1DBAforLUW(Exam600)• DB210.5FundamentalsforLUW(Exam615)
Visitibm.com/certifyformoreinformationandexamavailability.
IBMTrainingIBMiscommittedtohelpingourclientsachievetheskillsandexpertisetotaketheircareerstothenextlevel.Weofferacomprehensiveportfoliooftechnicaltrainingandeducationservicesdesignedforindividuals,companies,andpublicorganizationstoacquire,maintain,andoptimizetheirITskillsinIBMSoftwareandIBMSystems.Visitibm.com/software/data/educationfordetailsandcourseavailability.
BigDataUniversityLearnaboutDB2andvariousbigdatatechnologiesatyourpaceandatyourplace.BigDataUniversityoffershelpfulonlinecourseswithinstructionalvideosandexercisestohelpyoumasternewconcepts.Coursecompletionismarkedwithafinalexamandacertificate.Visitbigdatauniversity.com.
AppendixA:AdditionalResourcesforDB2 147
InformationManagementBookstoreFindthemostinformativeDB2booksonthemarket,alongwithvaluablelinksandofferstosaveyoumoneyandenhanceyourskills.Visithttp://bit.ly/DB2_Books.
IBMSupportforDB2AccesstheIBMSupportPortaltofindtechnicalsupportinformationforDB211.1,includingdownloads,notifications,technicaldocuments,flashes,andmore.Visitibm.com/support.Wantaglimpseintothefuture?Checkoutthenewsupportexperiencebeta-https://ibm.biz/support-pilot.LookforanswerstoyourDB2questions?PleasetrydWAnswersforum-http://ibm.biz/dwAnswersDB2.
IBMDataMagazineThemagazine'smissionistodeliversubstantive,high-qualitycontentonthelatestdatamanagementdevelopmentsandIBMadvances,aswellascreateastrongcommunityoftheworld'stopinformationmanagementprofessionals.IBMDatamagazinevividlydemonstrateshowthesmartuseofdataandinformationadvancesbroadbusinesssuccess,providingthecontextthatenablesdatamanagementprofessionalsatalllevelstomakemoreinformedchoicesandcreateinnovative,synchronized,agilesolutions.Seemoreat:ibmdatamag.com.
InternationalDB2UserGroup(IDUG)IDUGisallaboutcommunity.Througheducationevents,technicalresources,uniqueaccesstofellowusers,productdevelopersandsolutionproviders,theyofferanexpansive,dynamictechnicalsupportcommunity.IDUGdeliversqualityeducation,timelyinformationandpeer-drivenproducttrainingandutilizationthatenableDB2userstoachieveorganizationalbusinessobjectives,andtodrivepersonalcareeradvancement.Visitidug.org.
AppendixA:AdditionalResourcesforDB2 148
JointheConversationStaycurrentasDB211.1evolvesbyusingsocialmediasitestoconnecttoexpertsandtocontributeyourvoicetotheconversation.Visitoneormoreofthefollowing:
• https://twitter.com/IBM_DB2• https://facebook.com/DB2community• http://bit.ly/BLUVideos• http://linkd.in/DB2Professional• https://twitter.com/ibmbigdata• http://www.planetdb2.com/
AdditionaleBookMaterialTheDB2DEMOprogramletsyoutryoutmanyofthefeaturesthatarefoundinthiseBook.TheprogramandthecorrespondinginstallationmanualarefoundinthesamedirectorythatthiseBookisfoundin:https://ibm.ent.box.com/v/DB2v11eBookTouseDB2DEMO,youmustberunningonaWindowsworkstationandbeabletoconnecttoaDB2server.IfyouwanttotryDB2onyourworkstation,youcandownloadthefreeDB2Express-C(CommunityEdition)from:http://www-03.ibm.com/software/products/en/db2expresscAnyupdatestotheeBookanddemonstrationsoftwarewillbefoundinthisdirectory.Ifyouhaveanycommentsorsuggestions,pleasecontacttheauthorsat:GeorgeBaklarz:[email protected]:[email protected]
TheDB211.1releasedeliversseveralsignificantenhancementsincludingDatabasePartitioningFeature(DPF)forBLUcolumnartechnology,improvedpureScaleperformanceandHighAvailabilityDisasterRecovery(HADR)support,andnumerousSQLfeatures.Thisbookwaswrittentohighlightmanyofthenewfeaturesandfunctionsthatarenowavailableinthisrelease,withoutyouhavingtosearchthroughvariousforums,blogs,andonlinemanuals.WehopethatthisbookgivesyoumoreinsightintowhatyoucannowaccomplishwithDB211.1,andincludeitonyourshortlistofdatabasestodeploy,whetheritisonpremise,inthecloud,orinvirtualizedenvironments.CoverageIncludes:
• AnoverviewofthenewpackagingchangesinDB211.1,alongwithpre-requisitesforinstallingtheproduct
• EnhancementstoDB2BLU,includingsupportforBLUcolumnartablesinaDPF(DataPartitioningFeature)environment
• pureScalecapabilitiesthatprovidecontinuousavailabilityinaproductionenvironment,alongwithalltheenhancementsthathavebeenmadetosimplifyinstallationandmanagementofacluster
• SQLandcompatibilityenhancements
GeorgeBaklarz,B.Math,M.Sc.,Ph.D.Eng.,hasspent31yearsatIBMworkingonvariousaspectsofdatabasetechnology.Georgehaswritten10booksonDB2andotherdatabasetechnologies.GeorgeiscurrentlypartoftheWorldwideCoreDatabaseTechnicalSalesTeam.EnzoCialini,B.Sc.,isaSeniorTechnicalStaffMemberandMasterInventorintheWorldwideCoreDatabaseTechnicalSalesTeamandformerlytheChiefQualityAssuranceArchitectforDB2&PureDataintheIBMTorontoDB2DevelopmentTeam.HeisalsoapublishedbookauthorandwrittenvariouspapersonDB2.Enzohas25yearsofexperienceindatabasetechnology,softwaredevelopment,testing,support,competitiveanalysisandproductiondeployments.
DB2 for Linux, Unix, and Windows Version 11 Highlights George Baklarz and Enzo Cialini