Single Master HDR Architecture

1Feb2017GaryDemosPresentationtoHPA

SingleMasterHDRArchitecture

GaryDemos,ImageEssenceLLC

HPA

Feb2017


HDRLandscape

•Camerashaveever-wideningrangeandever-loweringnoisefloor

•Displayshavealsogreatlyexpandedbrightrange,andsomedisplaytechnologieshavegreatlyexpandeddarkrange

•Sometestscenesbeingshownattradeshows,especiallyinthe400nitto1000nitrange,aregreattolookat

•Sometestscenesbeingshownattradeshows,mostlyabove1500nits,appearharshanddifficulttowatch(althoughalsofascinating)

•GainingcontroloftheaestheticsofHDRcontentisaprimarygoal

•ReproductionoftheappearanceoftheHDRmasterisalsoaprimarygoal


•IfusingPQ,pixelbrightnessabovethemasteringdisplay’srangewillnotbevisible(e.g.1000nitand2000nitpixelsside-by-sidelookidenticalona1000nitdisplay)

•ColoroutsideP3gamutwith2020containerhasasimilarissue,(e.g.apixeloutsideP3andattheP3boundarycanlookidenticalonaP3display)

•Onlyonesurroundistypicallyutilizedduringmastering(e.g.darksurround,5nits,20nits,?nits).Presentationmaydifferintheconsumerhome.

•Masterscreatedthiswayshouldbe“inverted”intoaneutralformwhichisindependentofthespecificdisplayandsurroundinordertobeutilizedondisplayshavingdifferentbrightordarkrange,and/ordifferentgamut,and/ordifferentsurround.

•Suchinversionmaynotalwaysbepossible,ormaybeflawed

Dangersof“GradingtoGlass”toMasterHDR


•A“neutralform”isrequiredfortheHDRmaster

•Mypreferenceisscene-referredlinearlight

•ThesinglemasterHDRsystem’sgoalistoworkoutthemechanismsthatprovideaconsistentappearance,correspondingtothecreativeintentforthemasteredappearance

•Thismustbedonewithoutbeing(strictly)displayreferred

Forthesereasons,display-referredHDRmastersareproblematic


StrawmanRequirementforHDR/Single-MasterSystemArchitectures:

MustATLEASTbeabletoreproducemasteredappearanceover150nit-1200nit(threestops)rangeandP3gamut

Everythingelseis“extra”capability

• MaybepossibletogotolowerbrightnesswithasingleHDRmaster,butthatremainsanopenissue.

•Theassumptionisthat50nits(e.g.DCinema)and100nits(e.g.Rec709)probablyrequireaseparategradeforoptimumresults(e.g.using“powerwindows”).

•ThedemonstrationhereatHPA2017goesdownto150nits.

TheSDRendofthings:


ExposureInvariance Display/ProjectorInvariance

LinearScene-ReferredLightLinearDisplay/Projector-ReferredLight

(bothcanbereaddirectlywithaSpectralRadiometer)


LinearScene-ReferredLightLinearDisplay/Projector-ReferredLight

(bothcanbereaddirectlywithaSpectralRadiometer)

SceneLightDisplay/ProjectorLight

ThisistheFundamentalTransform(aka“OOTF”,OpticaltoOpticalTransferFunction)

•BT.2100andACESpursuethedefinitionofan“OOTF”

•TheOOTFisthekeytoallformsof“colorscience”

•ThecenterpieceoftheOOTFisalsoknownas“colorrendering”


History

•Insearchoflinearized systemarchitecture

•prototypeACES-likeversions:

•Inthelineageof“UchidaTune”RRTs,usingtheDC22lookreference

•TheseACESversionsbrightenanddarkenwhileretainingfixedchromaticities

ACESOct2012Emmy

AdozenotherACEScontributorsnotshownhere,suchasDougWalker,Iwaki-san,JohnMcElvain

Uchida-san

BackrowsLtoR:?(Fuji?),GeorgeJoblove,M.Uchida,ScottDyer,AlexForsythe,HaraldBrendel,JosephGoldstone,Floiran Kainz,LarsBorg,GaryDemos

FrontRow:JackHolm,JimHouston,RayFeeney,AndyMaltz


•“gd10”developedasaprototypeACES-likeversionin2013

- modular

- renderingtonecurvedownthroughzero,tonegativevalues

- rendering“nugget”,inoutputportionofsplitcolorrenderingmodule

- newtheory,“onesizefitsall”maynotbeworkableforrenderingappearance

- onefavoredrenderedappearancemodule,“full”withtwobrightbehaviors:•brightendopenedupandsaturation-preservingvs.•brightendrolledoffwithdesaturation(bestforfaces)

- “full”appearancebasedonACESnon-HDR“DC22”lookreference(note:ACESv0.1x,v0.2x,andv0.7xallusedtheDC22lookreference)

ModularACES-likeversion


LADmaps0.18to0.10(thispluswhitepointsetsnumericscale)

Slope0.02(downthroughzero)

Slope1.0(out=in)

v0.22tonecurve

v0.22tonecurve

RobustforHDR

OptimalNoiseProperties

Fundamental brightness renderingS-curve(keytoend-to-endOOTF)


2015ASCworkontheHDR_System

•RadiometriccapabilitiesarenewinHDRSingle-MasterSystem,asareroomambientcompensation,andfor“colorfulness”compensation(theHunteffect)

•EmphasisonMDR/HDR(andnoton50nit/100nitDCinema/Rec709)

•renderingsplitintooptionalaestheticrenderingpreamble(s),andnugget

•display-typetransformsplitintodisplay/projectorappearanceattributes,and“codingonthewire”attributes(whichhavenoaffectonappearance)

•Topendofrenderingtonecurveopeneduptounityslope(out=in)

•Topendreductionanddesaturationmovedintooptionalpreambles


ModularArchitecture


Master

RGBratio-preservingrenderingnugget

IntendedLookIsHere

Device-specific&viewingadapted

* *

However,AnythingGoes

Scene-ReferredOftenAvailable

DeviceIndependent

ColorsPreciselyDefined&archival

ModularArchitecture (detail)


Master


IntendedLookIsHere

*

DeviceIndependent

ColorsPreciselyDefined&archival

Archive

scene-referredScenePhysics

(realand/orartificial)

(needsappearancemodule,e.g.“full”)

“full”isactivemid-curve,atmid-saturation

“full”dials-outfordarkandbright

curveappliedtoNorm(R,G,B)




IntendedLookIsHere Device-

specific&viewingadapted

*

DeviceIndependent

ParametricAppearance

Compensation

DisplayCharacteristics/Settings

ViewingSurround

DisplayInputWireFormatting

(noappearancechanges)



DisplayInputWireFormatting

(noappearancechanges)

DisplayRemoves(inverts-out)Formatting

(aka“transferfunction”,EOTF)

Caveats:•TopendofPQmaybedifficult•Gamutboundarysoft-clip(ifpresent)•Bitdepth<->perceptualthreshold

end-to-endshouldbeano-op(codingonthewire)

(withrespecttoappearance)

Example:inverse-gamma2.4gamma2.4

•Darkbehavior(nearblack)•Largerbrightareadarkening•Function<->perceptualthreshold



DisplayInputWireFormatting(none)

NoUn-FormattingUtilized(LinearLightIsLiteral)

Configuration:•Gamutinsignalmatchesdisplaygamut(e.g.P3)•1.0isdisplaymaximumforeachofR,G,B,andW•Linearlightrepresentedashalf-float(aka“FP16”),willbebelowperceptualthreshold•Everydifferentvalueinthe0.0to1.0rangeforR,G,andBarevisuallydistinct

end-to-endaliteralno-op

Example:half-floathalf-float

MyPreference:

anotherviewpoint:thisalldisappears!


CameraRaw

SingleMasterHDRArchitectureComparisonvsACES

IDT

ReferenceRendering

Transform(RRT)

AestheticRenderingPreamble(s)



ACES_RGB

Rendering“Nugget”

(tonecurve)

ColorRenderingisModularOnlyNuggetisMandatory

RRTisMandatoryACES

SingleMasterHDRParametricAppearance

Compensation

DisplayFormatting(ifneeded)

OutputDeviceTransform(ODT)

HDRODTsHDRODTsHDRODTsHDRODTs

Split Split

ACESMasterisHere

MasterisHere

OutputDeviceTransform(ODT)OutputDeviceTransform(ODT)OutputDeviceTransform(ODT)

ACES’(prime)grading,CGI,LMTs,

etc.

ChromaticityPreservation


SingleMasterHDR_System approach

•Aestheticrenderingpreambles,andgrading,arenotnormative.TheresultoftheseisembodiedintheradiometricHDR_System master.

•TheuseandmeaningoftheHDR_System masterisnormative

•TheentirenormativeprocessingfortheHDR_System canmoveintodisplay/projectorelectronics,iftheHDRmasterisdirectlydistributed(e.g.usingafloatingpointcodec).Thisisamigrationpath.

• TheHDR_System mastermaintainsthelinearlightcharacteristicsofthemaster,eventhroughgrading(asdemonstratedbytheLustre gradingtests).Nolimitationsonfutureuseofsuchalinearlight(“aces-prime”)gradedmaster.

•ThisenablestheHDR_System tosupportsingle-masterHDR(withintheHDR_System’s rangeandgamut).

• TheresultingHDR_System masterobtainsradiometricintegrity,whichincludeslong-termdefinedmeaningofanHDRmaster


BasisoftheSingleMasterHDRSystem

Keydesignprincipleforthecorrectpresentationappearance fortheHDR_System master:

•Brightnessmoves

•Chromaticitydoesnot*

* (exceptforHunteffectsaturationcompensationonhighbrightnessdisplays)


CameraRaw

SingleMasterHDRArchitecture

IDT




ACES_RGB


(tonecurve)

ColorRenderingisModular(andoptional)

SingleMasterHDR ParametricAppearance

Compensation

DisplayFormatting(ifneeded)

MasterisHere

ACES’(prime)grading,CGI,etc.

ChromaticityPreservation(aka“Radiometric”)

OnlyNuggetisMandatory

Infofromdisplay


Discovery DuringLustre v0.7gradevs.originalICAS

-originalandgradelookthesameexceptforintentionaladjustments

- thegradedimageisthusinlinearscene-referredlight

- nolossofhigh-endcolorsaturation

- nonoisebuildupindarkareas(noratanybrightnessnorcolor)

- anHDRmasterinthisformisunassailablecannotbequestionedregarding“gradeability”

•Similartofilmnegative/digital-film-densityintermediate

•Fullcamerarangeandgamutpreserved(ifdesired)

•Creativeadjustmentsingradingprocessarenottiedtospecificdisplays(neitheratmasteringnoratdistribution/presentation)


• Nouseofmetadatainthisexamplearchitecture(fornow)

•Masterscanbecreatedinfullrangeandgamutlikecameraoriginals(similartofilmnegativebeingamaster,buthavingevenmorerange)

•Scenelightlinearphysicsisthenfundamentaltothedefinition/specificationofthearchivalmaster

- definedusingD60orD65whitepoint,andLAD- fullgamut(e.g.ACESRGBorXYZ,addingspectralinfoifavailable)

•Eachdistributiondisplayshouldhaveamechanismtocompensateforitsowninherentproperties:

- Brightness(includinglargeareabrightnessdimming)- Gamut- Ambientsurround- Bitdepth(ditheringdownfromhigherbitdepthsource(s),whenavail.)

•Independentof“codingonthewire”attributes(e.g.EOTFnulledout)

ArchitecturalImplicationsofthisSingle-MasterHDRArchitecture


Pixel-independent(notregional)appearanceeffects:

*Abney:Constanthuevs.saturationgoesalongcurvedlines

*Stevens:Constrast increaseswithluminance

*Hunt:Colorfulnessincreaseswithluminance

x Bezold-Brucke:Huechangeswithluminance

xHelmholtz-Kohlrausch:Brightnessincreaseswithcolorsaturation

*modeledandcompensated

x tobeexplored

AppearanceCompensation


Regionalandtime-sensitiveappearanceeffects:

*Bartleson-Breneman:Contrastincreaseswithsurroundluminance

xAppearanceeffectinvolvingcontrastchangingwithsubtendedangle?

- Surroundwhiteadaptationwhendifferentfromdisplayedwhite

- Colorandbrightnesspersistenceeffects(includingcone-bleaching)

- Regionalopticalillusions(oftenrelatedtolocalizedsurroundadaptation)

*modeledandcompensated

x tobeexplored

- notsurewhatcanbedone

AppearanceCompensation (continued)


•LAD(LaboratoryAimDensity,mid-grey)- LADwasthemainbrightnessanchorforSDRfilm

•LADneedstovarysomewherein-betweenabsolutenits(e.g.PQ)andpercentofmax(e.g.SDRLADissetat10%ofMax)

•LAD(mid-grey)wouldbe1000nits(10%ofMax)ifscalingupSDRtomax10,000!

•LADwouldbedeep0.1%blackifheldatSDR’s10nitsvsPQmax10,000!

•Thereisavastregionbetween0.1%ofpeakand10%ofpeak- 0.1%to10%is100-to-1range!- LAD(mid-grey)belongssomewhereinthisrange- However,LADshouldnotbefixedaspercentageofMax,norabsolutenits

•LADneedstomovewithrespecttodisplaymaximum,andambientsurround

LADMustMove!


LowPortionisStraightLineToZero•nos-curveatbottom•lifteddisplayblackcompensationrequiresbetterthancrudeLb• darkappearanceandcompensationisanopentopic

HighEndofCurveAsymptotictoDisplayMaximum(usingnorm)

LADVariable LADVariableLADVariable

AppearanceBrightnessCompensationBuiltOnVariableAsymptote(thus,LADMoves):

BrightnessCompensation


DiffuseWhitevs.LightsandHighlights

•Diffusewhite(e.g.awhiteshirtinthescene)needstomovesimilartoLAD

•LightsandHighlightscanvaryinbrightness- butnottoomuch- mustthinkaboutthe3stopmaxrangeweareconsidering:

- 150nitsto1200nits- consider100nitdiffusewhiteis1%of10,000nits!- 1%blackinSDRismid-black(between½%blackand2%highblack)

- needtobalancemaxbrightcapabilitywithnaturalappearance- theambientsurroundlevelalsoisafactor

•Whenmastering,needtospecifyallowedvariabilityfordiffusewhite,lights,andhighlights

- Lockdownthediffusewhiteallowedvariabilitywhenmakingthemaster- Lockdowntheshinyreflectivehighlightbehavioralso- Lockdownthebehaviorofwhiteandcolorful(e.g.neon)lights- Enforceconsistentbrightregionsbetweendifferentcompositeelements- Managebrightnessofhighlightglintsontheeyesofactorsinclose-ups


HuntEffect(colorfulness):

•Functionofabsolutebrightness• Appliesequallytosaturatedanddesaturatedcolors•Appliesmoretobrighterpartswithinaframe•Compensationviadesat needstofollowconstant-huecurvesandappliesluminancebrightnessadjustment(e.g.desat bluedarkens)•Seemstoaffectcolorfulnessoververylargedarktobrightrange

Colorfulnessquestionstoexplore:

•Dopeoplepartiallyadapttosaturationlevelsathigherbrightnesses?•Ifso,issuchadaptationaffectedbythesurroundappearance?•Isitusefultodarkenbrightsaturatedcolors?

- HLGassertsthis- Shoulddesaturationanddarkeningofbrightcolorsbecombined?

•Isasimpleoverall(allbrightnessesforagivenpeak)desaturationsometimesadequate?Overwhatrangeifso?

BrightandTooColorful


Summary

•Scenereferred

•Builton“radiometric”(chromaticity-preserving)spine

Thisenables:- colorfulnesscompensation- ambientsurroundcompensation- unambiguousmasters- end-to-endcalibration- appearanceattributescreatedintomaster,reproduciblesubsequentlyoverMDR/HDRrangeandgamut

•Master,afterappearancemodule/gradingdefinedbylinearscenelight,LAD,andD60(orD65)

•Worknotcomplete,butreasonablyfaralong


ViewDemonstrationintheInnovationZone

SMPTE(smpte.org):education->on-demand-webinars->23April2015“HDRIntermediate”

Alsocheckout:

[email protected]

(ThisArchitectureWorks!)

Note:appendicesfollow,FYI


Appendices:

• DemonstrationDescriptions

•FurtherInvestigations

•TipsforHDRMastering

•AvoidingStructuralLimitations

•TheFilmDensityModel

• PixelMeanings

• Meaningof”Color”


Demonstrations


TestMDRP3 TestHDRP3

DemonstrationofMDRandHDRmatching(150nitto1200nits)

•Leftimage(s)coveredwith0.3NDFilter(onestop)

•Displaybrightnesscompensation

•“colorfulnesscompensation”fortheHunteffect

0.3ND


Demonstrations (continued)

- Side-by-side0.7Lustre gradeofICASvs.F65camera-native(viaSonyRawViewer)

- Facehighlightdesaturationdemonstration

- Brightregiondesat forHuntEffectcompensation,alongconstant-huecurves

- Near60fpsrealtime 10bitUHDdecodingwith257-cubed3DLUTinGPU

- 24fpsrealtime 12bit4kdecodingrunning“full”algorithmsinGPU

- MacPro Laptop,500nits,P3gamut,w/companionLG5k27”Thunderbolt-3display

- On-screengraduatedneutralsurroundforD60andD65(forusewhenHDRmastering)

- Behind-screenadjustableambientsurround



• Radiometricrendering

- Nopreamble,nuggetonly

- Chromaticity-preservingappearancecompensationonly

- Brighteninganddarkeningasafunctionofhue:

- RedandYellowbrighter

- Green,Cyan,Blue,andMagentadarker

- End-to-endchromaticitypreserving(i.e.hueandsaturationpreserving)

- NoColorfulnesscompensation(butcouldbeaddedusingHLG-styledarkening)

•Notfullyaesthetic(e.g.notDC-2.2look),butnotbad(havealook)



ComparisonofAppearancevs.ACES1.0.2

•Comparisonof“Full_nobr_fc”aestheticrenderingpreamblevs.ACES1.0.2RRTwith2020gamut/2084PQ_1000nit_EOTFHDRODT

•ACES1.0.2RRTandHDRODTfromBlackMagic DaVinci Resolve


CameraRaw

SingleMasterHDRAppearanceComparisonvsACES1.0.2

IDT

RRTACES1.0.2ReferenceRenderingTransform

Fullnobr_fcAestheticRenderingPreamble

ACES_RGB


(tonecurve)

ACES

SingleMasterHDR ParametricAppearance

CompensationSetFor1000nitsLowAmbient

DisplayFormatting

HDRODT2020gamut

2084/PQ-1000nitEOTF


FurtherInvestigations


InvestigationsandVerificationsofCurrentVersion:

•Yellow-biasednorm

•Tonecurvesectionfitrefinement- lowlinear,.02slope,downthrough0.0- midtriestohitLAD(.18mapsto.10)- highlinear,in=out

•MatrixfromACESRGBtoP3RGBgamutreductionissues- matrixhas~2.0red-to-redterm- negativetermscantakeallpositiveinputtonegativeoutput- inputcurrentlylimitedtoallpositive- howtosoft-clipgamuttopandbottomonhuecurves/luminance?

•D60(ACESwhitepoint)vs.D65(HLG/BT.2100andPQ/BT.2020whitepoints)

•On-screenambientand/orbehind-screenambientduringmastering


InvestigationsandVerificationsofCurrentVersion(cont.):

•Specificfunctionofmaxbrightness

•Specificfunctionofambientsurround

•Currentlymaxandsurroundarecombinedinasingleadjustment

•Activepreambles:- Null(radiometric)- Full- Fullrolled-off-desaturatedbrights- gd9desaturatedbrights- gd9rolled-off-desaturatedbrights

•Inactivepreambles:- gamma_and_mat,simple,moderate,double,some“looks”

•Feasiblepreamble:ACES1.0(maynotbeorthogonaltoall1.0ODTs)

Reproducesdesiredappearancebaseduponscenelightenergy


TipsforUsingtheHDRMasteringDisplay


•Testvarioussurroundlevels(e.g.adjustingonscreenappearancevs.surroundbrightness,usingthemethodsdescribedhere)

•Liftdarksbytwoorthreestopstoinspect

•Dropbrights bytwoorthreestopstoinspect

•InspectHDRcompositeslifted,darkened,desaturated,andatreducedcontrast

•Attempttoinspecteverymasteredpixelvaluewithindisplay’srange/gamut

•Alternatively,putmasteringdisplayintoamodewhereeverypixelvalueisvisiblydistinct,andlimitthemastertothatrange/gamut

- e.g.P3gamut10/12bitgamma2.6w/smallrange/gamutedgemargin

- Notsurehowtoobtainaneutralform(e.g.scene-referred)doingthis

HelpfulproceduresforHDRdisplayduringmastering:


AvoidingStructuralLimitations


OrangeDesaturatedHuePushedTowardRed

OrangeBoostedSaturationHuePushedTowardRed

OrangeBoostedSaturationHuePushedTowardYellow

OrangeDesaturatedHuePushedTowardYellow

HueandSaturationVaryWithBrightness(objectschangecolorasbrightnessvaries)

WhenR,G,andBindependently“ride”anS-Curve

ThisisOKforanoptional“Look”,butcreatesdifficultywhenmandatory---

Inherentlychallengingtoattemptbuildingasingle-masterHDRsystemfromthis

brightsdesaturated

darksdesaturated

mid-tonesboostedsaturation


WhatthemandatorypartsofthisHDRarchitecturedoNOTdo:

•R,G,andBdonot ridethetonecurveindependently(yieldinghueandsaturationvaryingasafunctionofbrightness)

•NoLb (blackoffset,e.g.BT1886)crudeprocessing

•Norendered-indisplayappearancecompensationprocessing

•Nomandatorymodificationofhueandsaturation

• Nomandatorydesaturationofdarks

•Nomandatory desaturationofbrights

•Nofixingtoabsolutenits(cd/m2)No!

AvoidingStructuralLimitations


TheFilmDensityModel


FilmDensity

FilmPrint

FilmCameraNegative

DMin (orange)

DMax (black)

DMax (black)

LADMoves,RGBshifts,when“timing”Neg toPrint(defaultOOTF.18->.10)

LADLAD

DMin (clear)

•DC22wasACESfilm-densitymodel(neg +printOOTF)

•PrintDMin (clear)limitedto14flLamp/Lens•NotHighDynamicRange•Darksandlightsdesaturate•Everythingisnonlinear(straightestnearLAD)


PixelMeanings


PixelLinearandNonlinearMeaningsandtheirProperties

• Gamma2.22+linearblacktoe(Rec709,BT1886),OETF

•Gamma2.4+/- Lb (BT1886),EOTF(Lb iscrudeblackoffset)

• Mis-useof“linear”,meaningeitherorbothoftheabove

•Gamma2.6(DCinema/DCI/DC-28),EOTF,noblackcompensation

•Correctuseof“linear”,gamma1.0- Scenereferredlinearistheradiometryphysicsofthescene- Displayreferredlinearistheradiometryphysicsofthedisplay

•PerceptualQuantizer (BT.2084),EOTF,HLG(BT.2100),OOTF+

•FilmDensity(ameasureofnegativedensity)

•Variousquasi-logsoptimizedforcameras(S-Log,Log-C,Filmstream,etc.)


•Pixel”coverage”isinherentlylinear(anti-aliasing),butmayinteractviasub-pixelspecificorientations

• Pixel”transparency”anddensityisinherentlylogarithmic(.3ND+.3ND=.6NDinlogarithmicdensity=0.5x0.5=.25linear

•Allnon-linearformatswillaliasduringprocessingandcompositing

•Increasingresolutionhelps(lessprominentaliasingartifacts)

• Increasingdynamicrangehurts(moreprominentaliasingartifacts)

• Aliasingandcross-talkissuesareinherentin“Non-Constant-Luminance”

•4:2:0isusuallyappliedinnon-linearspaces,e.g.HEVCMain-10

PixelRepresentationProperties(cont.)


Meaningof“Color”


MultipleMeaningsof“Color”

- Asperceived(appeareance)

- Asameasurable“chromaticity”

- Chromaticities implycolormatchingfunctions(“CMF’s)

- TherearemultiplerelevantCMF’s(e.g.19312deg,196410deg,170-1)

- Aslinearweightsofsensingoremissivespectra,measurablewithaspectralradiometer

- Asthecombinedmeaningofilluminatinglightspectratogetherwithsurfacespectralabsorption/reflectance

- Assurfacespectralabsorptionitself

- Emissivecolorlights(e.g.neon),etc.


<end>

Single Master HDR Architecture

Documents

Transcript of Single Master HDR Architecture