NV_path_renderingFunctional Improvements

Mark KilgardNVIDIA

March 25, 2014(Updated July 9, 2014)

Motivation for Improvements

• Make NV_path_rendering extension a better match for web usage– For Skia, Chromium browser, and web

standards generally

• Better performance– Driven by profiling & usage experience

• OpenGL ES support– NV_path_rendering intended for non-subset

OpenGL functionality (full OpenGL 4.4)

New Functionality

• More efficient GL commands– One command for “stencil then cover”– 2D matrix commands

• New path commands– Rational quadratics– Rounded rectangles

• Path objects from font’s glyph indices

• OpenGL ES support

Single Command“Stencil then Cover”

• glStencilThenCover{Fill,Stroke}PathNV– Does glStencil{Fill,Stroke}PathNV, then

glCover{Fill,Stroke}PathNV– On the same path object

• Also instanced versions– glStencilThenCover{Stroke,Stroke}PathInstancedNV

• 100% functionally identical to just calling equivalent stencil & cover commands sequentially– (Only different if there is erroneous usage)

• Any error ignores steps of the command

Single Command“Stencil then Cover” Details

• Minor performance advantage– Halves object locking and name lookup overhead– Instanced versions “display list” well since only one

set of per-path transform values

• Advice to programmers– Easy to adopt if you keep your glStencil*Path* and

glCover*Path* commands in the same function– Easy to emulate for older driver

• Just register helper function that calls old commands sequentially

Single Command“Stencil then Cover” API

• void glStencilThenCoverStrokePathNV (GLuint path, GLenum StrokeMode, GLuint mask, GLenum coverMode);

• void glStencilThenCoverStrokePathNV (GLuint path, GLint reference, GLuint mask, GLenum coverMode);

common path object

common path objectstencil params

cover params

stencil params

cover params

Single Command“Stencil then Cover” Equivalents

• glStencilThenCoverStrokePathNV ==– glStencilStrokePathNV(path, StrokeMode, mask);– glCoverStrokePathNV(path, coverMode);

• glStencilThenCoverStrokePathNV ==– glStencilStrokePathNV(path, StrokeMode, mask);– glCoverStrokePathNV(path, coverMode);

Single Command“Stencil then Cover” Instanced API

• void glStencilThenCoverFillPathInstancedNV(GLsizei numPaths, GLenum pathNameType, const GLvoid *paths, GLuint pathBase,

GLenum fillMode, GLuint mask, GLenum coverMode, GLenum transformType, const GLfloat *transformValues);

• void glStencilThenCoverStrokePathInstancedNV(GLsizei numPaths, GLenum pathNameType, const GLvoid *paths, GLuint pathBase,

GLint reference, GLuint mask, GLenum coverMode, GLenum transformType, const GLfloat *transformValues);

array of pathsstencil paramscover params

per-path arrayelement transforms

array of pathsstencil paramscover params

per-path arrayelement transforms

More Compact/Efficient 2D Matrix Updates

• Existing OpenGL matrix load and multiply assume 4x4

• But most path rendering relies on simpler 2D matrix forms– 3x2 affine, 3x3 projective

• Advantages of Compact 2D Matrix Commands– Easier to process and analyze– Smaller in display lists– Selector-free, no glMatrixMode dependency

• Based on EXT_direct_state_access approach

• Easy to emulate for older drivers– Substitute with 4x4 EXT_direct_state_access functions

Compact/Efficient 2D Matrix Update Commands

• Affine 2D matrix load and multiply commands– void glMatrixLoad3x2fNV (GLenum mode, const GLfloat *m);– void glMatrixMult3x2fNV (GLenum mode, const GLfloat *m);

• Projective 2D matrix load and multiply commands– void glMatrixLoad3x3fNV (GLenum mode, const GLfloat *m);– void glMatrixLoadTranspose3x3fNV (GLenum mode, const

GLfloat *m);– void glMatrixMult3x3fNV (GLenum mode, const GLfloat *m);– void glMatrixMultTranspose3x3fNV (GLenum mode, const

GLfloat *m);

Compact Matrix Commands by APIStandard Type Component order Corresponding Load command

Skia SkMatrix [0,1,2]

[3,4,5]

[6,7,8]

glMatrixLoadTranspose3x3fNV

SkScalar [6] [0,2,4]

[1,3,5]

glMatrixLoad3x2fNV

Cairo cairo_matrix_t [0,2,4]

[1,3,5]

glMatrixLoad3x2fNV

Qt QMatrix [0,2,4]

[1,3,5]

glMatrixLoad3x2fNV

OpenVG VGfloat [9] [0,3,6]

[1,4,7]

[2,5,8]

glMatrixLoad3x3fNV

Direct2D D2D_MATRIX_3X2_F [0,2,4]

[1,3,5]

glMatrixLoad3x2fNV

Rational Quadratic Path Commands

• Inspired by Skia support– Rational quadratics allow circular, elliptical arcs, and hyperbola

• Also known as conic curves• Instead of just parabolas

• Easy to support, just 2 new paths commands– GL_CONIC_CURVE_TO_NV (0x1A)

• 5 coordinates– [ x, y, w, x, y ]– First (x,y,w) is extrapolating control point– Second (x,y) is interpolating end point

– GL_RELATIVE_CONIC_CURVE_TO_NV (0x1B)• 5 coordinates

– [ dx, dy, w, dx, dy ]

GL_CONIC_CURVE_TO_NV Details

• Normal GL_QUADRATIC_CURVE_TO_NV– Takes four scalar components

• (P1x,P1y,P2x,P2y)– With end-point of the last path command, these

provide three control points:• (P0x,P0y), (P1x,P1y), (P2x,P2y)

• Rational quadratic version supports quadratics– Call this command GL_CONIC_CURVE_TO_NV

• Google wants this, and for good reason– Takes five scalar components

• (P1x,P1y,P1w,P2x,P2y)– Make 3 homogeneous control points

• (P0x,P0y,1), (P1x,P1y,P1w), (P2x,P2y,1)

Partial Circular and Elliptical Arcs

• Rational quadratic Bezier curves can be drawn if we support perspective rendering of quadratic Bezier segments

neat!

should extendto ellipses

Conics Details

• If the interpolating control points (P0 & P2) have unit W values– Meaning if P0w and P2w are both 1.0

• Then P1w controls the type of conic generated– P1w = 1 generates a parabolic segments

• Basically a normal integral (non-rational) quadratic Bezier segments

– P1w > 1 generates a hyperbolic segment• A portion of a hyperbola

– P1w < 1 generates a partial elliptical or circular arc

• To maintain the convex hull property, we need to avoid negative values for P1w– So use the absolute value of the coordinate for P1w

Conic Rendering Example

w=1.0

w=2.0

w=0.5

works with filling, stroking, and dashing

w=∞

Rounded Rectangles

• Very common primitive on web pages– Standardized by “

CSS Backgrounds and Borders Module Level 3” specification– Google cites rounded rectangles as the only “non-trivial” paths in

most web pages

• Already have GL_RECT_NV path command– PDF standardizes this path command

• Rationale for first-class support– Common in web content– Faster to bake– Faster to render– Easy parameterization, more compact to specify

• Represents sequence of 4 lines and 4 arcs

Rounded RectanglePath Commands

• New rounded rectangle path commands– Uniform circular radius: GL_ROUNDED_RECT_NV– Uniform elliptical radius: GL_ROUNDED_RECT2_NV– Per-corner circular radii: GL_ROUNDED_RECT4: NV– Per-corner elliptical radii:

GL_ROUNDED_RECT8_NV• Relative versions too

– GL_RELATIVE_RECT_NV, GL_RELATIVE_ROUNDED_RECT_NV, GL_RELATIVE_ROUNDED_RECT2_NV, GL_RELATIVE_ROUNDED_RECT4_NV, GL_RELATIVE_ROUNDED_RECT8_NV

Rounded RectangleCircular Corners

• Rounded rect = 8 segment spline– Four linear segments for edges– Four 90 degree arcs at rounded corners

(x,y)

height

width

corner radius

corner radiuscorner radius

corner radius

radius canbe per-rectangleor per-corner

Rounded RectangleElliptical Corners

• Rounded rect = 8 segment spline– Four linear segments for edges– Four 90 degree arcs at rounded corners

(x,y)

height

width

y radius

x & y ellipticalradii canbe per-rectangleor per-corner

x radius

y radius

x radius

x radius

x radius

y radius y radius

Rounded Rectangle Rendering Examples

odd cases, butpossiblefromparameterization

Glyph Index Font SupportMotivation

• Problem– Existing NV_path_rendering allows glyphs to be created from a font

based on Unicode character point– Very handy/useful for “toy” or “simple” text layout

• Advantage of Unicode approach: multiple font faces can “overlap” to provide fallback to some supported glyph

– BUT quite insufficient for sophisticated text layout• Browsers definitiely need sophisticated text layout

• Solution– Add glyphs created based on font glyph index– Facilities sophisticated text layout

• Awareness: Application request a specific font face and have avaialble the font’s metrics and shading rules– Note: NV_path_rendering does NOT provide this information– Apps are expected to get it from the font directly

• Natural to use FreeType 2 + Harfbuzz for this

Glyph Index-based Path Specification API

• Two commands– GLenum glPathGlyphIndexArrayNV(

GLuint firstPathName, GLenum fontTarget, const void *fontName, GLbitfield fontStyle, GLuint firstGlyphIndex, GLsizei numGlyphs, GLuint pathParameterTemplate, GLfloat emScale);

• Mimic glPathGlyphRangeNV but indexing of the font face is glyph indices rather than Unicode character points

– GLenum glPathMemoryGlyphIndexArrayNV( GLuint firstPathName, GLenum fontTarget, GLsizeiptr fontSize, const void *fontData, GLsizei faceIndex, GLuint firstGlyphIndex, GLsizei numGlyphs, GLuint pathParameterTemplate, GLfloat emScale);

• Like glPathGlyphIndexArrayNV but allows for in-memory font file for font face

Glyph Index-based Path Specification Discussion

• App or library decides how glyph indices are arranged in path object name space– Example: Glyph indices can be arranged to match Skia’s

fallback font ordering of glyph indices

• Assumes app has a-priori knowledge of the number of glyphs in the font face

• glPathMemoryGlyphIndexArrayNV: in-memory font data needed is copied– At least the font data needed to maintain the glyph outlines– Not referenced– Works like FreeType’s FT_New_Memory_Face

• Expectation is app will use FT_New_Memory_Face for it’s a-prior knowledge of glyph count, metrics, etc.

Extra Glyph Index Range with Allocation Command

• Generates range of glyphs for a particular font– Good if you don’t know how many glyph indices there

might be in a font• (But you probably know that if you are shaping your own text)

– GLenum glPathGlyphIndexRangeNV ( GLenum fontTarget, const GLvoid *fontName, GLbitfield fontStyle, GLuint pathParameterTemplate, GLfloat emScale, GLuint baseAndCount[2]);

• Like calling glGenPathsNV for the font’s glyph index count, and then populating the range with glPathGlyphIndexArrayNV

first path object ID &count returned here

Examples of NV_path_rendering Glyph Index Rendering

Text shownshaped with

HarfBuzz library

More Example Scripts

Text shownshaped with

HarfBuzz library

Rotating and Scaling Just Works

OpenGL ES 2/3 Support

• NV_path_rendering– Assumes modelview/projection matrix– Assumes fragment shader only operates during “cover” step– Relies on generating varying fragment values on linear function

of object-space path coordinates• Similar to fixed-function glTexGen’s operation

• Problem: Required vertex shader in ES– ES2 eliminates fixed function– ES2 requires vertex & fragment shaders to be paired

• No fragment shaders alone• No separate shader objects (is an EXT extension though)

• ES2 & ES3 eliminates API machinery NV_path_rendering relies on for path transformation and shading

OpenGL ES 2/3 Support Goals

• Invent as little new API as possible– Leverage existing extensions

• EXT_separate_shader_objects• EXT_direct_state_access• ARB_program_interface_query

– Reuse token values and match prototypes for matrix commands

• Same NVpr code for ES, Core Profile, and Compatibility Profile

• Rationale– Developers don’t like “same, only different” APIs– Testing effort easier– Re-use is less controversial than invention

Proposed NVpr for ES Solution

• Add back into ES 2.0 and 3.x– Transformation state

• Modelview and projection transforms

– Rely on selector-free matrix routines from EXT_direct_state_access for this

– Clip planes operating in eye-space

• Leverage (assume/require!) separate shader objects (SSO) functionality– Currently ES 2.0/3.0 extension

• EXT_separate_shader_objects

– Slated for inclusion in ES 3.1 standard

Varying Fragment Input Generation for “Cover” Step

• ES lacks fixed-function fragment inputs– So NVpr’s glPathTexGen*NV, etc. APIs cannot drive

inputs

• So add generation command for GLSL fragment inputs– void glProgramPathFragmentInputGenNV(

GLuint program, GLint location, GLenum genMode, GLint components, const GLfloat *coeffs);

ES “Cover” Fragment Shader Usage Notes

• Use “layout” qualifiers to assign locations to fragment inputs– Example: layout(location=0) int vec2 st;– SSO introduces layout qualifiers for every shader domain

• Prior to SSO, just layout locations just allowed for the vertex shader

• ARB_program_interface_query is useful related extension– Allows programmatic querying of locations for fragment program

inputs• When fragment shader is “first” domain in the SSO pipeline program

– Provides introspection when explicit layout locations not used in the shader or shader is from an “outside” source

ES “Cover” Fragment Shader Example

• Complete example for radial gradient:– #version 300 es

layout(location=0) in vec2 st;uniform sampler1D ramp;uniform vec4 color;void main() { gl_FragColor = color*texture(ramp, length(st));}

• Driving the “st” varying– GLfloat params[3][3] = {

/* radial gradient linear coefficients */ };glProgramPathFragmentInputGenNV(radGradProg, /*location*/0, GL_OBJECT_LINEAR, 2, &params[0][0]);

– Generation state is per-program• (not per-GL context)

ES Matrix Additions• Require EXT_direct_state_access (DSA) matrix functions be available

– When NV_path_rendering exposed on ES– (Doesn’t require DSA itself)

• Matrix commands:– glMatrixLoadfEXT, glMatrixLoaddEXT, glMatrixMultfEXT, glMatrixMultdEXT,

glMatrixLoadIdentityEXT, glMatrixRotatefEXT, glMatrixRotatedEXT, glMatrixScalefEXT, glMatrixScaledEXT, glMatrixTranslatefEXT, glTranslateMatrixdEXT, glMatrixOrthoEXT, glMatrixFrustumEXT, glMatrixPushEXT, glPopMatrixEXT

• Matrix values should be GL_PATH_PROJECTION_NV or GL_PATH_MODELVIEW_NV

– Aliases (same token values) for GL_MODELVIEW and GL_PROJECTION– No other matrices included (texture matrix, color matrix)

• Includes matrix stack for both these matices– Important to commands can be rendered with respect to other views

• Unresolved– Reintroduce built-in variables for these so non-NVpr rendering can reference the same

matrices?• Likely YES, otherwise hard for non-NVpr rendering to be consistent• gl_PathModelviewMatrixNV, gl_PathProjectionMatrixNV

– Reintroduce a glPathClipPlane (and glPathClipRect) for clip plane usage?• Likely YES

Approach to Additions

• Simply extend exist NV_path_rendering with new functions and tokens– No new extension; documented as minor revisions in

specification– Detect minor update by querying if new command functions exist– Or detecting errors from new tokens

• Rationales– Just adding functionality; fully backward compatible– Few actual users of NV_path_rendering today– None of these additions tied to actual hardware features– Stuff NV_path_rendering really “should have always had”– Relative to emulate the lack of these features

Availability

• In latest drivers (now public)– Single command “StencilThenCover”– 2D matrix commands

• Glyph index support– Implemented

• Conic curves & round rectangles– Implemented

• OpenGL ES support– Implemented

• In 337.88 public drivers circa June 2014

Appendix material

Skia Bugs

• skbug.com/2051 clip planes

• skbug.com/2042 enable nvpr by default

• skbug.com/2034 glStencilThenCover

• skbug.com/2061 NV_blend_equation_advanced

• skbug.com/2062 3x3 matrix API

API Details: Path Commands• #define GL_CONIC_CURVE_TO_NV 0x1A• #define GL_RELATIVE_CONIC_CURVE_TO_NV 0x1B

• #define GL_ROUNDED_RECT_NV 0xE8• #define GL_RELATIVE_ROUNDED_RECT_NV 0xE9• #define GL_ROUNDED_RECT2_NV 0xEA• #define GL_RELATIVE_ROUNDED_RECT2_NV 0xEB• #define GL_ROUNDED_RECT4_NV 0xEC• #define GL_RELATIVE_ROUNDED_RECT4_NV 0xED• #define GL_ROUNDED_RECT8_NV 0xEE• #define GL_RELATIVE_ROUNDED_RECT8_NV 0xEF

• #define GL_RELATIVE_RECT_NV 0xF7

API Details: Font-related Tokens

• #define GL_FONT_GLYPHS_AVAILABLE_NV 0x9368• #define GL_FONT_TARGET_UNAVAILABLE_NV 0x9369• #define GL_FONT_UNAVAILABLE_NV 0x936A• #define GL_FONT_UNINTELLIGIBLE_NV 0x936B• #define GL_STANDARD_FONT_FORMAT_NV 0x936C

• #define GL_FONT_NUM_GLYPH_INDICES_BIT_NV 0x20000000

API Details: Glyph Index Font Commands

• GLenum glPathGlyphIndexArrayNV (GLuint firstPathName, GLenum fontTarget, const GLvoid *fontName, GLbitfield fontStyle, GLuint firstGlyphIndex, GLsizei numGlyphs, GLuint pathParameterTemplate, GLfloat emScale);

• GLenum glPathMemoryGlyphIndexArrayNV (GLuint firstPathName, GLenum fontTarget, GLsizeiptr fontSize, const GLvoid *fontData, GLbitfield fontStyle, GLuint firstGlyphIndex, GLsizei numGlyphs, GLuint pathParameterTemplate, GLfloat emScale);

• GLenum glPathGlyphIndexRangeNV (GLenum fontTarget, const GLvoid *fontName, GLbitfield fontStyle, GLuint pathParameterTemplate, GLfloat emScale, GLuint *baseAndCount);

API Details: Glyph Index Font Command Typedefs

• typedef GLenum (GLAPIENTRYP PFNGLPATHGLYPHINDEXARRAYNVPROC) (GLuint firstPathName, GLenum fontTarget, const GLvoid *fontName, GLbitfield fontStyle, GLuint firstGlyphIndex, GLsizei numGlyphs, GLuint pathParameterTemplate, GLfloat emScale);

• typedef GLenum (GLAPIENTRYP PFNGLPATHMEMORYGLYPHINDEXARRAYNVPROC) (GLuint firstPathName, GLenum fontTarget, GLsizeiptr fontSize, const GLvoid *fontData, GLbitfield fontStyle, GLuint firstGlyphIndex, GLsizei numGlyphs, GLuint pathParameterTemplate, GLfloat emScale);

• typedef GLenum (GLAPIENTRYP PFNGLPATHGLYPHINDEXRANGENVPROC) (GLenum fontTarget, const GLvoid *fontName, GLbitfield fontStyle, GLuint pathParameterTemplate, GLfloat emScale, GLuint *baseAndCount);

API Details: “Stencil, then Cover” Commands

• void glStencilThenCoverFillPathNV (GLuint path, GLenum fillMode, GLuint mask, GLenum coverMode);

• void glStencilThenCoverStrokePathNV (GLuint path, GLint reference, GLuint mask, GLenum coverMode);

• void glStencilThenCoverFillPathInstancedNV (GLsizei numPaths, GLenum pathNameType, const GLvoid *paths, GLuint pathBase, GLenum fillMode, GLuint mask, GLenum coverMode, GLenum transformType, const GLfloat *transformValues);

• void glStencilThenCoverStrokePathInstancedNV (GLsizei numPaths, GLenum pathNameType, const GLvoid *paths, GLuint pathBase, GLint reference, GLuint mask, GLenum coverMode, GLenum transformType, const GLfloat *transformValues);

API Details: Compact Matrix Commands

• void GLAPIENTRY glMatrixLoad3x2fNV (GLenum mode, const GLfloat *m);

• void GLAPIENTRY glMatrixLoad3x3fNV (GLenum mode, const GLfloat *m);

• void GLAPIENTRY glMatrixLoadTranspose3x3fNV (GLenum mode, const GLfloat *m);

• void GLAPIENTRY glMatrixMult3x2fNV (GLenum mode, const GLfloat *m);

• void GLAPIENTRY glMatrixMult3x3fNV (GLenum mode, const GLfloat *m);

• void GLAPIENTRY glMatrixMultTranspose3x3fNV (GLenum mode, const GLfloat *m);

API Details: Compact Matrix Commands Typedefs

• typedef void (GLAPIENTRYP PFNGLMATRIXLOAD3X2FNVPROC) (GLenum mode, const GLfloat *m);

• typedef void (GLAPIENTRYP PFNGLMATRIXLOAD3X3FNVPROC) (GLenum mode, const GLfloat *m);

• typedef void (GLAPIENTRYP PFNGLMATRIXLOADTRANSPOSE3X3FNVPROC) (GLenum mode, const GLfloat *m);

• typedef void (GLAPIENTRYP PFNGLMATRIXMULT3X2FNVPROC) (GLenum mode, const GLfloat *m);

• typedef void (GLAPIENTRYP PFNGLMATRIXMULT3X3FNVPROC) (GLenum mode, const GLfloat *m);

• typedef void (GLAPIENTRYP PFNGLMATRIXMULTTRANSPOSE3X3FNVPROC) (GLenum mode, const GLfloat *m);

API Details: GLSL Path Fragment Varying Generation

• void GLAPIENTRY glProgramPathFragmentInputGenNV (GLuint program, GLint location, GLenum genMode, GLint components, const GLfloat *coeffs);

• typedef void (GLAPIENTRYP PFNGLPROGRAMPATHFRAGMENTINPUTGENNVPROC) (GLuint program, GLint location, GLenum genMode, GLint components, const GLfloat *coeffs);