1

DIRSIG5: Next Generation Remote Sensing Dataand Image Simulation Framework

Adam Goodenough and Scott BrownDigital Imaging and Remote Sensing LaboratoryChester F. Carlson Center for Imaging Science

Rochester Institute of TechnologyRochester, NY 14623

Abstract—The Digital Imaging and Remote Sensing ImageGeneration model (DIRSIG) is a physics-based, image and datasimulation model that is primarily used to generate syntheticimagery across the visible to thermal infrared regions using engi-neering driven descriptions of remote sensing systems. The modelrecently went through a major redesign and reimplementationeffort to address changes in user requirements and numericalcomputation trends that have emerged in the 15 years since thelast major development effort. The new model architecture adoptssome of the latest light transport algorithms matured by thecomputer graphics community and features a framework thatis easily parallelized at the micro-scale (multi-threading) andmacro-scale (cluster-based computing). A detailed descriptionof the framework is provided, including a novel method forefficiently storing, evaluating, integrating and sampling sphericaland hemispherical datasets appropriate for the representationof modeled or measured bidirectional scattering, reflectance andtransmission distribution functions. The capabilities of the modelare then briefly demonstrated and cross-verified with scenariosof interest to the remote sensing community.

Index Terms—DIRSIG, Data modeling, Modeling and simula-tion, Scene simulation, Sensors, Simulations, Software

I. INTRODUCTION

The initial development of the Digital Imaging and RemoteSensing Image Generation (DIRSIG) model was begun atRochester Institute of Technology (RIT) in the late 1980s as a3-D simulation environment to explore the phenomenology ofimagery collected by long-wave infrared (LWIR) systems. Themodel synthesized imagery utilizing ray tracing techniques toidentify objects in a virtual scene viewed by a virtual camera.The temperature at locations intersected by camera probe rayswere computed using a simple 1-D slab model[1] and thesurface leaving radiance was computed using Planck black-body emission and material specific emissivity values. Theray tracer was also utilized to compute the accumulated solarloading for the location, by tracing rays to the sun locationas a function of time to generate a shadowing history. Thesurface leaving radiance would then be propagated through theatmosphere to the sensor using path transmission and radianceterms computed by LOWTRAN[2].

Expansion of the model into the mid-wave infrared requiredthe need to handle increasingly lower emissivity (reflective)materials and the ray tracer was used to identify incidentflux from a specularly reflected direction. Over time, thereflected radiance calculation was improved to better sample

the incident light field onto a surface and to increase thenumber of bounces. By the mid 1990’s, the model had evolvedto cover the visible through infrared (0.2 to 20.0 micron)region of the spectrum utilizing a robust Whitted-style raytracing[3] method. The model has undergone a series of majorrewrites since it’s inception, with DIRSIG version 5 (referredto as DIRSIG5 for the remainder of this paper) being the latest.In addition to the improved numerical radiometry frameworkat the focus of this paper, this latest version is focused onintegrating with the latest versions of the modeling toolsutilized by the remote sensing including MODTRAN6[4] andSystems Toolkit (STK)[5].

There are numerous commercial and open-source renderingprograms that focus on producing conventional frame-orientedimagery in the visible region of the electro-magnetic spectrumfor applications ranging from scientific data visualization toarchitectural previewing to entertainment. Through the devel-opment of faster computing and better algorithms, many of theavailable rendering environments are now employing physics-driven approaches rather than approximations that dominatedcomputer graphics a decade ago. Although red-green-blue(RGB) rendering programs are the most common, a fewsystems capable of operating spectrally (more channels thansimply RGB) have emerged (e.g. NVIDIA’s Iray[6]). However,there are a limited number of actively developed tools thatfocus on the visible through long-wave infrared wavelengths(e.g. MCScene[7] and MATISSE[8]) and we believe thatDIRSIG is the only tool that focuses on sensor-driven EO/IRsimulation of remote sensing imaging systems that do notnecessarily utilize 2-D array sensors.

The primary goal of the DIRSIG modeling effort is togenerate datasets that enable the user to perform hardwaredesign trade studies and to evaluate algorithm performance.The model is designed to generate passive broad-band, multi-spectral, hyper-spectral, low-light, polarized and active laserradar datasets through the integration of a suite of physics-driven components. The model is sometimes described as a“hardware simulator” because the user defines the imagingsensors using engineering level descriptions including phys-ical focal plane dimensions, detailed timing and clockinginformation and vehicle command data. As an example, theDIRSIG4 model was used to simulate level-0 class data forLandsat8 prior to launch[9]. These simulated datasets includedvarious focal plane calibration anomalies (variation in the

2

spectral filters and gain characteristics across sub-chip assem-blies (SCAs)) and the pixel-to-pixel, band-to-band, module-to-module and instrument-to-instrument registration offsetspresent in operational data so that the ground processingpipeline could be exercised and tested prior to launch.

II. BACKGROUND

The design and capabilities of the next generation architec-ture introduced in the DIRSIG5 model were driven by the con-tinued evolution of computing hardware and changes in enduser requirements. Since 2001, when DIRSIG4 was designedand implemented, multi-core central processing units (CPUs)were introduced (Intel’s first dual-core chip, the Pentium D,was introduced in 2005) and the ensuing proliferation of multi-core CPUs enabled multi-threaded (parallel) programming toemerge from specialized applications into general practice.In addition, the introduction of general purpose graphicalprocessing units (GPGPUs) produced another avenue for ac-cessing low cost, parallel computing. During this same periodof growth in computational power, the size and cost of 1-Dand 2-D detector arrays improved significantly. This fueleda shift in the DIRSIG user community from performing asmall number of specific simulations of smaller format systemsto modeling hundreds or thousands of collections of systemsutilizing comparatively massive focal planes.

Due to the underlying architecture of the DIRSIG4 software,updating the code to utilize multi-threaded and/or GPU accel-erated approaches was not practical. Instead, the requirementfor faster computation throughput was addressed through theimplementation of a new radiometry engine at the core ofDIRSIG5. This new core leverages a numerical approach thatis inherently more efficient at computing accurate radiancesand that is inherently better suited for micro-scale (multi-threaded) and macro-scale (cluster) parallelization on moderncomputing hardware.

A. DIRSIG4 Numerical Radiometry Approach

The numerical radiometry approach employed in DIRSIG4was centered around the concept of radiometry solvers, whichwere surface (or volume) leaving radiance algorithm objectsthat could be assigned to geometric elements in the scene ona per material basis. These algorithms used a Whitted-styleapproach to ray tracing by spawning one or more rays at eachbounce to sample the incident illumination. For example, ahighly specular (e.g. mirror) surface might use an algorithmthat traces a single ray into the specular direction. In contrast, adiffuse surface might use an algorithm that traces rays into theentire hemisphere. Each of these sampling rays might interactwith another scene element, which would trigger anotherradiometry solver to sample the incident illumination at thatlocation. Because this approach would attempt to sample theentire incident light field at each interaction point, we referto it as a “breadth first” approach (c.f. Figure 1). In somescenarios, this would lead to nearly exponential growth in thecalculations as a function of bounces.

A suite of these solver algorithms was developed to allowfor specialization of the calculation for different material

types. The most advanced and flexible radiometry solverswould adaptively sample the scattering phase function toinsure that more rays were directed into the incident directionsthat would contribute most to the overall flux. Althoughthis modular approach made the DIRSIG4 radiometry enginevery flexible, the correct selection and configuration of theradiometry solvers resulted in a system that was complex andrequired significant user education. In addition, the numberof calculations required by each initial detector ray variedgreatly, which made efficient parallelization of the calculationsdifficult.

...

a single sensor ray spawns generations of new rays – the

total number of calculations can vary greatly between pixels

Fig. 1. “Breadth first”, Whitted-style ray tracing used in DIRSIG4

B. DIRSIG5 Numerical Radiometry ApproachIn contrast to the approach used in DIRSIG4, DIRSIG5

utilizes a unified, path tracing approach [10], where a back-ward ray from the camera is used to construct a single paththrough the scene to an originating source or terminating event(absorption). Each node along the path is a scattering event ata surface or in a medium. We refer to the construction ofthese paths as path generation, which utilizes stepwise raytracing and surface or medium interactions. Once the pathhas been constructed, the radiance arriving at the start of thepath is computed in a step we refer to as path processing,which entails stepwise propagation of radiance through thechain of events along the path. Because this method prioritizescreation of single path connections through the scene, we referto it as a “depth first” approach to finding multi-bounce lightpaths between sources and the sensor (c.f. Figure 2). Thisis a more efficient physics-based rendering method than thebranching style utilized in DIRSIG4, and due to the uniformityof computations, it can be efficiently parallelized on moderncomputing hardware. Furthermore, the algorithm avoids thecomplexity and correct configuration risks of the radiometrysolver approach employed in DIRSIG4. The implementationof the path tracing algorithm is discussed in detail later in thispaper.

III. DESCRIPTION

This section will introduce the architecture of the DIRSIG5model in greater detail. The description will focus on the role

3

...

many sensor rays generate similar paths for relatively constant computation time

across threads

Fig. 2. “Depth first” path tracing used in DIRSIG5

of 3 major components in the model; (a) the scene data model,(b) the sensor model plugins and (c) the scheduler (c.f. Figure3). The core radiometry component is where the path tracingalgorithm resides that computes the radiance from a requesteddirection. Due to the complexity of this component, it will beseparately described in Section IV.

State Changes (spectral, temporal)

Sensor Plugin A

Scheduler

Inte

rnal

/Pub

lic P

lug-

in A

PIs

Scene Data Model

Geometry Models

Dynamic Motion Models

Optical Property Models

Medium Models

Whole Earth Models

Temperature Models

Sensor Plugin N

Radiometry Core

Ray Tracer

Path Generator

Path Processor

…

Fig. 3. A diagram illustrating the major components of the model and howthey relate to one another.

A. Scene Data Model

The path tracer in the radiometry core interacts with oneor more scene data models, which contain the 3-D geometryand materials descriptions describing the virtual scene to beimaged. The ability to utilize multiple scene data models in thecontext of a single simulation allows the user to partition largeregions into smaller pieces that can be dynamically loadedjust-in-time. It also allows for easily generating variants ofscenes. For example, a baseline city scene containing the

static geometry can be combined with different dynamic trafficpattern scenes based on the time of day.

1) Geometry: The base geometry in the scene can be com-posed of 3-D polygon objects using either an older DIRSIG-specific description or the ubiquitous Alias/Wavefront OBJ fileformat. Polygon geometry available in other formats can beeasily converted to one of these two formats via 3rd party com-mercial and open-source tools. The polygon geometry modelsupports normal shading, where the normal is continuouslyvaried across planar surfaces by spatially interpolating normalsdefined at the vertexes. In addition to polygon descriptions,the model supports a set of parametrically defined primitivesincluding planes, disks, boxes, spheres, cones, cylinders andsinusoids. Finally, the model supports voxelized descriptions(a 3-D grid of cubic elements) in order to represent objectsincluding clouds, plumes, water, etc.

The scene description allows the base geometry to beassembled into a hierarchy to assist organizationally andcomputationally. For example, a large urban scene might beorganized into “neighborhoods” and within a “neighborhood”by “streets”. Some of this hierarchy is leveraged by the raytracing algorithms to help minimize ray intersection tests.Because of the Euclidean methods at the heart of all rayintersection algorithms, the scene is defined and assembledusing a local Cartesian coordinate system that is then posi-tioned on the Earth by specifying the geolocation of the localscene origin. The orientation of the scene coordinate systemfollows an XYZ East-North-Up correspondence (hence, thisscene coordinate system is frequently referred as a Scene ENUsystem). Significantly large scenes must have the Earth curva-ture represented in the Scene ENU coordinate system becausethe tangent Cartesian coordinate system is not automaticallywarped to follow the curve of the Earth.

Each base geometry object can be used multiple times inthe scene by supplying a minimal geometric description ofan instance of the base object. Instance descriptions includethe translation and rotation of the base geometry as a dualquaternion and allow the base geometry to be duplicatedwithin the scene with a minimal memory and computationaloverhead per instance. An arbitrary depth of instancing issupported.

Instance descriptions can be fixed in time (e.g. the locationof a building) or dynamic with time (e.g. the location and ori-entation of a car driving down the street). A variety of dynamicinstance models can be utilized including a set of data-drivenmechanisms (e.g. location and orientation as a function of timein either the scene or a geolocated coordinate system), para-metric models (e.g. altitude, heading and speed or rotationalrates) and algorithmic models (e.g. an SGP4 orbital propagatorcombined with orientation constraint models). Data-drivendynamic instances are temporally interpolated from the list ofdual quaternions using a screw linear interpolation (ScLERP).The instance dual quaternions can be described in either theScene ENU coordinate system or in a variety of geolocatedcoordinate systems, which are converted into the Scene ENUcoordinate system using GeographicLib[11].

Rather than utilize the in-house, uniform subdivision raytracing acceleration methods written for DIRSIG4, DIRSIG5

4

utilizes highly optimized, 3rd party ray tracing engines. For thepure CPU implementation (no GPU acceleration) the modelemploys the Intel Embree[12] ray tracing engine.

2) Materials: A material description is composed of a set ofsurface and/or volume optical properties and an optional set ofthermodynamic properties (if thermal region modeling is de-sired). Surface optical properties can include either reflectance,emissivity and transmission descriptions or any combinationthereof. These surface optical properties can be characterizedthrough a variety of unidirectional and bidirectional models. Ina similar fashion, a material can also describe a volume suchas a portion of the atmosphere, a cloud, a gas plume, water,etc through a combination of scattering, absorption and/orextinction properties. With the exception of some specializedproperties, all of the surface and bulk optical properties aredescribed spectrally. A novel data-driven model is introducedin Section IV that is capable of efficiently storing and evalu-ating spherical and hemispherical datasets appropriate for therepresentation of modeled or measured bidirectional scattering,reflectance and transmission distribution functions.

Each material description has a unique alpha-numeric iden-tifier, which is used to assign the material to geometry. Thespatial scale of material assignment can range from per-polygon, per-primitive or per-voxel to sub-geometry scalesusing a variety of 2-D and 3-D image-driven mappings. Forexample, a terrain might be represented by a triangular irreg-ular network (TIN) of polygons but fine-scale, sub-polygonmaterial assignments can be introduced at a nearly arbitraryscale using a 2-D classmap (a single, 100% fill material at eachlocation) or a mixture map (multiple, partial fill materials ateach location).

DIRSIG also supports a set of image-driven tools that canintroduce spatially and spectrally correlated variation withina given material. These methods include the direct inges-tion of calibrated multi-spectral or hyper-spectral reflectanceimagery and a unique spatial-spectral technique developedto fuse high spatial resolution, multi-spectral imagery withhigh spectral resolution point reflectance measurements[13].There are also a set of map-driven geometry manipulatorsthat spatially manipulate the surface normal to simulate highresolution surface variations (e.g. bump maps). Any of thegeometry, material or property maps can be projected ontogeometry using either a vertical “rubber sheet” projection orUV projection (a commonly used 3-D to 2-D projection systemexplicitly generated for an object).

3) Validation and Storage: A scene is constructed througha hierarchy of files that help the user structure and organizethe scene assets. The myriad of input files used to describe thescene geometry and material properties are assembled from avariety of different sources and each input file needs to un-dergo a series of format specific checks to insure data integrity.In addition, some of the input files store data in an format thatis not optimal for direct use by the DIRSIG model. In previousgenerations of the model, the data validation and reformattingsteps were performed at the start of each simulation and couldaccount for a significant amount of the run-time for complexscenes. Therefore, a scene preparation stage was introduced inDIRSIG5 that performs these validation and formatting tasks

and then stores the resulting data model in a Hierarchical DataFormat (HDF)[14]. The resulting scene HDF file contains allthe scene static and dynamic content and it does not needto be updated unless the scene creator modifies the scene.In addition to making it easier to share and revision controlscenes, the new HDF storage mechanism utilizes a fractionof the storage space and allows the model to load a complexscene into the model nearly 50x faster than DIRSIG4 could.

B. Sensor Model

The DIRSIG5 framework uses an application program inter-face (API) to define the basic interface between the radiometrycore, the scheduler and one or more sensors being modeledduring a given simulation. The scheduler will call a specificset of functions in the sensor plugins to acquire data aboutthe radiometry calculations to be performed and notify theplugin when those calculations have completed and can beprocessed. The sensor API is openly documented to allowend users to implement their own codes for modeling specificor unique sensors. However, the DIRSIG5 model comes witha flexible sensor plugin that was implemented to providebackward compatibility with DIRSIG4 sensor descriptions.

The plugin utilizes a vehicle-centric (e.g. van, unmannedaircraft system (UAS), airplane, satellite, etc.) description andallows the user to define one or more imaging sensors ormeta-data collection instruments (e.g. platform location andorientation) that attach to the vehicle at various locations.Imaging sensors can feature multiple parametrically definedfocal planes with data-driven spectral response and point-spread function (PSF) descriptions.

Flexible timing (read out) and integration options allowthe user to model common remote sensing payloads rangingfrom 2-D framing array cameras with rolling shutters usedin UASs and small satellite applications to modular, sub-chipassembly (SCA) pushbrooms used in larger satellites includingLandsat8, IKONOS and WorldView 2/3/4.

The vehicle motion can be supplied using a set of data-driven descriptions (e.g. location and orientation as a functionof time in either the scene or a geolocated coordinate sys-tem), parametric models (e.g. altitude, heading and speed) oralgorithmic models (e.g. an SGP4 orbital propagator combinedwith orientation constraint models). The “unprogrammed mo-tion” or “jitter” can be independently defined for all sixdegrees of freedom at the platform level (XYZ location andXYZ orientation) and two degrees of freedom at the platform-relative attachment level (along-track and across-track point-ing) using either a temporally uncorrelated model (normallydistributed deviates from a user-supplied mean and standarddeviation) or a temporally correlated model driven by a user-supplied power spectral density (PSD).

The key capability of a sensor model plugin is the im-plementation of the function that generates the rays usedto initiate paths in the path tracer. This function will becalled by the radiometry core to start new paths until theconvergence criteria specified by the sensor plugin is meet.The resulting spectral radiance is then processed by the sensorplugin (e.g. converted into signal) and written to an output

5

data stream. The pixel sampling function in the default pluginuses a variety of spatial and temporal sampling techniquesto introduce common artifacts observed in remote sensingsystems. Sample rays are constructed by randomly samplingthe pixel area and offsetting those locations by importancesampling the (optional) point spread function (PSF). Thetemporal integration of the detectors is modeled by distributingthe sample ray times across the integration time window. Eachunique ray time will result in a different origin of the ray dueto platform gross scale motion (velocity) and fine scale motion(jitter). Those distributed ray times will also be used to positiondynamic content in the scene. A sufficient number of pixel raysemploying these spatial and temporal sampling techniques hasbeen shown to reproduce effects including optical PSF blur,motion blur (due to platform and in-scene motion), rollingshutter artifacts and time-delayed integration (TDI) [15].

C. Scheduler

The scheduler is the glue between the low-level radiancecomputation capabilities of the radiometry core and the sensorsthat need radiance calculations performed (c.f. Figure 3). Thescheduler interacts with sensor plugins configured for thesimulation to acquire data about the radiometry calculations tobe performed and notifying the plugin when those calculationshave completed and can be used. As a result, the sensor pluginsare the primary driver of the simulation, with the schedulerin charge of optimally delegating the required radiometrycalculations to the core.

The radiometry core is primed to compute light paths forspecific time windows and specific wavelengths described by astate. Specifying a limited time window (a temporal state) al-lows the scene geometry model to temporally filter and preparedynamic geometry present in the scene in an effort to optimizethe ray tracing that will be utilized to assemble light pathsfrom the sensors to sources in the scene. Specifying a limitedset of wavelengths (a spectral state) allows the scene materialmanager to optimize surface and volume optical properties sothat rays can be reflected, scattered or absorbed at interactionnodes using efficient techniques (e.g. sampling driven by aBRDF). Performing these state optimization calculations atdiscrete points in the execution enables faster calculationsof all the light paths modeled within that state. However,a temporal or spectral state change can involve expensivecalculations including the rebuilding of ray tracer accelerationstructures due to scene motion or the generation of BSDFsampling models. Therefore, one of the scheduler’s jobs is tominimize these state changes.

The details of the sensor API are beyond the scope of thisdiscussion, but the following high-level description will ex-plain how the scheduler collects the required data required forthe simulation and feeds it to the radiometry core processingpipeline to perform the radiometry calculations. At the startof the simulation, the scheduler will collect the spectral states(a wavelength window and a reference wavelength used forsampling) from all the sensor plugins. The scheduler will thenask all the sensor plugins to describe a one or more tasks,which are time windows during which a given sensor will be

collecting data (e.g. the period during which a video camerais recording). For each task, the scheduler will ask the sensorplugin to describe one or more captures (e.g. a specific framewithin the video). Each capture can include a time windowthat describes a temporal state for the radiometry core (forexample, the time window during which the focal plane isintegrating). This time window defines the range of times thatwill be assigned to rays initiating paths in the radiometry core.

A capture is decomposed into a set of radiometric problems,which are generally analogous to pixels. The radiance for apixel is generated in the radiometry core as an accumulation ofradiances computed by the core path tracer. The sensor plugincan specify the minimum number of paths, maximum numberof paths and convergence criteria (stability of the radiance atthe reference wavelength) for the core to utilize on a givenpixel problem. This allows the core to use a range of pathsfor a given pixel depending on the complexity of the geometryand light field imaged by the pixel.

To execute the simulation, the scheduler iterates over allthe capture events submitted by the sensor plugins. For eachcapture event it will trigger a temporal state change in thecore based on the time window associated with that capture.It will then acquire the set of spectral states associated withthe capture and iterate across those. The core radiometrypipeline is multi-threaded, and one of the responsibilities of thescheduler is to divide a given capture into blocks or “chunks”of problems that can computed by the pipeline in parallel.During the simulation, the scheduler iterates through theseblocks of pixel problems, feeds them to the core radiometrypipeline and routes the computed pixel radiances back to thesensor model for processing.

In order for the radiometry core to initiate paths in the pathtracer for each problem, the scheduler provides the pipelinewith access to the sensor plugin’s ray generation callbackfunction. This callback function will generate initial path raysfor the pixel indicated by a unique pixel problem index.In addition to the geometric ray, the sensor plugin callbackfunction will also supply the initial time. Variations in thistime are commonly used by the sensor plugins to performtemporal sampling in order to model the integration time fora focal plane.

In addition to routing radiance calculation requests betweenthe sensor plugins and the radiometry core, the scheduler alsomanages “truth” requests made by the sensor plugin. For eachproblem, the core can collect a variety of metrics includingthe number of paths used in the problem, the average locationof the first node in the light path, material statistics for thefirst node, etc. This truth data can be communicated back tothe user by the sensor plugin using a variety of techniques(e.g. an auxiliary raster image where each band contains truthmeta-data for the same pixel in the primary output).

IV. RADIOMETRY CORE

The job of the radiometry core is to compute radiancesarriving at a specified location from a specified direction.This is achieved in DIRSIG5 using a path tracing approach,where the ray tracer is used by the path generator to construct

6

paths through the scene and the path processor computes theradiance propagated along that path. To facilitate parallel com-putations, the radiometry core uses a pipelined computationimplementation that allows the core to perform path genera-tion and path processing operations on multiple problems inparallel.

A. Radiometry PipelineWhen the radiometry core is asked to compute a set of

problems, the computation node (e.g. a CPU in a cluster) hasbeen primed with a particular spectral and temporal state bythe scheduler. This state is fixed, meaning that temporal varia-tions are guaranteed to occur at a fine scale (no modificationsto the ray tracing acceleration structures) and all the internalspectral storage has been pre-sized. These constants obviatethe need for expensive tests and updates that might otherwisebe performed in response to individual problems.

The radiometry pipeline is structured to be as “wide” asthe number of problems in a chunk. Each pass through thepipeline takes a single initial ray from each problem and findsthe corresponding radiance returned along the path of the ray.A single radiance subtending an infinitesimally small solidangle is a poor representation of the potential distribution ofradiances into the solid angle of a problem, so this processis repeated until a maximum number of problem samplesis reached or the average radiance within the solid angleconverges to a user-defined change threshold.

A pass through the pipeline consists of generating a pathin the scene for each initial ray. A path is made up of aseries of nodes (scattering interactions with the scene, usuallyat surfaces) connected by segments representing the passageof radiance through a medium (radiance may be attenuated oradded to along each segment). This path-based approach toradiometry is called path tracing and is based on a numericalMonte Carlo integral radiometry solution[10].

Paths have a maximum size (the number of consecutivescattering events allowed) and are generally well-balancedbetween parallel problems within the pipeline. This is incontrast to the approach taken in DIRSIG4, where the compu-tational loads between individual radiance calculations couldvary significantly. A brief summary of the overall process isgiven below.

1 request problem sample from sensor plugin callback2 initialize path3 construct path

a) attempt to propagate sample ray to a scene– account for path; reinitialize at scene boundary

b) trace ray against scene geometry, or Earth model– if no intersection, proceed to 4

c) generate a segment and node for traced rayd) trace ray to sampled source for Le

e) if node count reached maximum, proceed to 4f) generate a new, sampled ray at nodeg) go to step a

4 initialize path weights5 process path

a) get next nodeb) apply current path weight to node Le

c) weight path by next sample’s weight, Wd) if path continues, go to step ae) apply path weight to sampled Li if it existsf) add radiance result to accumulation buffer

6 if radiance has not met convergence criteria, go to 1

B. Path tracing

Scattered radiometry at surfaces, i.e. the calculation ofsurface-leaving radiance from the interaction of externalsources with the material, is governed by the integration of thebidirectional reflectance and transmittance distribution func-tions (BRDF and BTDF, respectively). This can be describedby the sum of the two contributors on either side of thesurface in a single bidirectional scattering distribution function(BSDF):

Lλ(νo) =

∫Ω

BSDFλ(νo, νi)Lλ(νi)[νi · n]dω, (1)

where the transmittance and/or the reflectance componentscan be zero. The i and o subscripts refer to incoming oroutgoing vectors, ν, respectively, and n is the normal vectorperpendicular to the surface.

The surface leaving radiance equation can be generalizedto a radiance weighting operator, w, acting on all possibleincoming radiance directions, νi:

Lλ(νo) =

∫Ω

wλ(νo, νi)Lλ(νi)dω, (2)

noting that the projected area scaling (the dot product inEquation 1) has been incorporated into the weighting function.

Using the fact that the weighting can be split into itsprobability density function, pw, scaled by its integral, W,we are left with the expected value of the incoming radiancescaled by the weighting integral for the outgoing direction(dropping the spectral notation for brevity):

L(νo) = W (νo)

∫Ω

L(νi)pw(νo, νi)dω (3)

≈ W (νo)

N

N∑j=1

L(νi)j , (4)

where the expected radiance is found by sampling theunderlying probability density function. If the incident ra-diance is direct at each sample (e.g. the sampled radiancecorresponds to the sky) then the radiance sample stops there.Incident radiance originating from another surface turns theequation into a recursive function, substituting the surface-leaving radiance for each step (i.e. node along the path). Theresult is a cumulative weight from each each node acting onthe final non-scattered radiance from a “source” element ofthe scene. The new direction taken at each node continuesto be sampled from the local weighting function, forming aneffective, aggregate probability density function without everknowing the exact form of the multi-scattered solution thatpotentially encompasses the whole scene.

C. Practical considerations

Since paths are inherently spectral (weighting functionmagnitude and shape at each node can be dependent onwavelength) and we do not want to split paths in the pipelineto keep the computation loads as consistent as possible, itis necessary to make a choice about what spectral weightingfunction should be used to drive the sampled direction. Atthe same time we want to compute as many wavelength

7

samples as possible if their directional coherence is high.The spectral states (generated by the sensors) provide thereference wavelength (in addition to the wavelengths at whichthe radiometry should be evaluated) used to drive that decision– i.e. the spectral weighting function that is used for thesampling is based on the reference wavelength. As a result,highly dispersive scene simulations should use many narrowwavelength windows while spectrally consistent scenes requirefar fewer. Note that the reference wavelength mechanism onlyaffects “spatial” consideration (e.g. the choice of direction)and the corresponding math is performed spectrally across thestate wavelengths.

The path tracing approach described above can be very effi-cient since it tends to prioritize the most important interactionswithin the scene while minimizing the time spent on paths thathave little impact on the final radiance. However, a naive im-plementation will perform poorly when non-uniform (discrete,angularly limited) sources are present in the scene. Althougha moderately uniform, cloud-filled sky scenario avoids thischallenge, most simulations relevant to the remote sensingcommunity where the sun, moon, or man-made sources arethe primary contributors to the scene illumination. To getaround this, we can modify Equation 4 into include explicitand indirect contributions,

L(νo) ≈ Le +W (νo)

N

N∑j=1

Li(νi)j , (5)

where Le is any explicitly calculated surface-leaving radiance(e.g. a reflected sun term). The only constraint is that anyexplicitly added source cannot be incorporated (again) whensampling for the indirect radiance term. Note that the explicitradiance becomes part of the expansion under recursion andis part of the processing of each node along a path (thecumulative weighting corresponding to the node depth isapplied).

When we have multiple explicit radiances that all need to beadded at each node we need to avoid the pitfall of unbalancingthe pipeline by significantly increasing the computational loadfor certain paths. This is accomplished by only adding one,or a small subset, of the sources each time, weighting by theinverse probability of choosing the sources that we included.In aggregate (over many sample paths) this is effectively thesame as including each one every time, i.e.

Le =1

N

N∑j=1

nj

(Ljepj

), (6)

where nj is the number of times that explicit source radiance jwas chosen out of N samples and pj was the probability of thatchoice. For large N , nj should be equal to pjN , reducing theaggregate to a simple sum of the sources. Secondary sources(street lights, headlights, etc..) within a scene are handled thisway, but with an additional step that culls far away and/or dimsources across sample generations (primary, coherent samplesonly).

Unfortunately including the explicit term usually means thatthe weighting function has to be known for any arbitrarycombination of view and source vectors. For something like

the reflectance we need to know the directional hemisphericalreflectance (i.e. the view dependent weighting integral referredto as the DHR), the cosine weighted function itself (thecosine weighted bidirectional reflectance distribution function,or BRDF), and the PDF (for sampling the BRDF) – all atas many wavelengths as we have spectral states. This is anon-trivial requirement since most materials vary to somedegree based on source and view angles and with wavelength.Handling materials (and their weighting functions) effectivelytherefore requires highly capable models of the underlyingoptical properties they represent.

D. Optical property models

Our previous description of the path tracing algorithm es-tablished that optical property models should provide efficientmechanisms for:

direct evaluation for finding the weights applied to explic-itly calculated radiance contributions at each node (such asthe sun)

integration for the integrated weights used for the recursiveradiance contributions along the path (e.g. scattered sky-light)

sampling the solution in Equation 4 requires that the under-lying probability function be sampled exactly

storage the storage of the data should be efficient so thatmany materials can be modeled at once without massivememory requirements

DIRSIG4 took an open-ended approach to surface opticalproperties – individual models were independently imple-mented and were only required to respond to basic modelqueries. This approach was particularly well suited for anacademic environment since new models could be brought indirectly and easily, often by simply translating equations inpapers into code. However, that same open-endedness meantthat the level of initialization, general computational com-plexity, and the potential ability to parallelize varied widelydepending on the user’s choice of model. More complexoptical property models [16] also needed to be appropriatelypaired with radiometry solvers (the specific methods used tocompute a surface-leaving radiance) – often requiring fine-tuning of many detailed runtime parameters to achieve bothradiometric accuracy and optimal speed.

The approach in DIRSIG5 is to unify the diversity of opticalproperty model options to eliminate the burden of an open-ended set of optical properties. At this time, the model sup-ports four weighting models that can be used for reflectance,emissivity, transmission and scattering: (1) a perfectly diffuse(Lambertian) model, (2) a perfectly specular (delta) model,(3) a parametric, diffuse+specular model, and (4) a data-drivenmodel capable of capturing arbitrary distributions.

1) The Diffuse (Lambertian) Model: The Lambertian modelhas no directional variability, so it is fully defined by thetotal (integrated) spectral reflectance, R, divided by π, andthe corresponding weighting function, wL, simply adds thecosine term for the surface-projected irradiance,

wL(νo, νi) =R

π[νi · n]. (7)

8

Sampling of the Lambertian weighting function involves aninversion of the cumulative distribution function to find:

θ = arcsin(√

ξ1

), (8)

φ = 2πξ2, (9)

or the zenith and azimuth angles for constructing the samplevector in surface-local space (i.e. relative to the source andnormal vectors). The ξ terms correspond to independent drawsfrom a uniform distribution from zero to one. Despite beingsimplistic, the Lambertian model can be well suited forapproximately describing many surfaces where the effectiveGSD (ground sample distance) is much larger than the scaleof the underlying “roughness” if it is also somewhat uniformlyrandom and self-shadowing is not significant[17].

2) The Specular (Delta) Model: The delta model representsthe other extreme in terms of the shape of the function, but itis just as simple in its implementation,

wδ(νo, νi) = R(νo)δ(νi − ν′i)[νi · n], (10)

where δ is the spherical form of the Dirac delta function:

δ(ν) =1

sin θνδ(θν − θ′ν)δ(φν − φ′ν). (11)

ν′ is dependent on the process being modeled (e.g. a specularvector for an ideal mirrored reflection or a refracted vector forFresnel transmittance). There is only one possible sampleddirection (the νi corresponding to νo).

3) The Diffuse+Specular Model: The real-world is largelycomposed of materials that do not fall into the first twocategories. For those cases, the first option available is aparametric Ward BRDF [18] model that combines a diffusecomponent and non-zero width specular lobe. Although adetailed treatment of the Ward model is beyond the scopeof this paper, the model is fully supported in DIRSIG5 withoptimal evaluation, integration and sampling functions.

4) The Data-driven Model: Some materials might be ini-tially described by a parameterized function, a weighted sumof convenient basis functions, or as raw, measured data. Thesemethods each have their pros and cons with respect to ourfour requirements. For example, a parameterized functioncan be efficiently evaluated and it’s storage requirements arevery low. However, not every parametric function supportsclosed-form integration and very few have efficient analyticalsampling mechanisms. Some basis functions are very efficientat representing smooth shapes but can require a large numberof terms to accurately fit complex shapes (which impactsstorage). In addition, efficient direct sampling methods donot exist for most basis function systems. Measured datais considered a gold standard by many, but highly sampledmeasurements require significant storage, require interpolationschemes for arbitrary evaluation and have no inherent mecha-nism for efficient sampling. In order to simultaneously satisfyall four of our modeling requirements for arbitrary weightingfunctions, we are introducing a novel data-driven approach thatleverages an adaptive, geometric structure to store spherical (orhemispherical) data. This flexible weighting function approachis used by DIRSIG5 to emulate the dozens of parameterizedBRDF and scattering phase functions available in DIRSIG4

and provides the end-user with an open mechanism to add an“unsupported” BRDF model or to import measured data. Theconstruction and usage of this data-driven weighting functionis described in the following sections.

To satisfy the direct evaluation requirement, a data-drivenmodel needs to store the weighting data in a structure that iseasily searched and can quickly translate a set of query vectorsinto the appropriate weight by finding the closest data pointsand interpolating appropriately. The integral requirement canbe easily pre-generated, storing the spectral integrals for a setof outgoing vectors and interpolating as needed. Sampling thefunction, however, is a non-trivial task. The data model needsto know the effective solid angle of each data point and itsrelative contribution to the entire PDF of the function, then beable to select one of those weighted solid angles by samplingthe cumulative distribution function (CDF) and then samplingagain within the solid angle of a data point. Getting to arepresentation of the PDF is straightforward – we are alreadystoring the integrated values and can normalize the data pointsaccordingly so they integrate to one as needed (and we can stillsatisfy direct queries by multiplying the new PDF values bythe appropriate magnitude). Efficient sampling is the biggestchallenge.

Ideally we want to have a set of data points that represent anequidistant distribution of points on the hemisphere or spheresuch that the solid angles corresponding to the points (formedby their Voronoi cells) are identical both in size and shape. Insuch a structure every data point has equal weight and weneed only store the sampled PDF value at each point andintegrating any portion of the sphere is simply a matter ofsumming the data points and scaling by the (constant) solidangle. Simultaneously, we can use the same values for directqueries without having to account for (and store) a solid angleadjustment at each point.

Unfortunately there is no (known) exact solution to theproblem of distributing points in such a way across a sphere,but there are several approximate solutions. We have chosento use one based on the Hierarchical Equal Area isoLatitudePixelization (HEALPix) partition of the sphere[19][20]. TheHEALPix partition does a fairly good job of achieving theideal distribution, especially when the number of points ishigh (areas across the sphere only vary slightly and the shapesare more or less diamonds around every point). ExampleHEALPix point distributions are shown in Figure 4. However,it has the added benefit that it is based on a quad-tree hierarchyof cells (the next order of the partition is found by splittingeach cell into four equal child cells). To be more accurate, itis based on 12 quad-trees based off of an initial partition ofthe sphere into 12 equal-area regions. The size of the data setis 12(4d), where d is the depth (order) of the partition.

With this partitioning scheme at the core of our data-drivenoptical property, we refer to the underlying geometric storagestructure as a Spherical Quad Tree, or SQT. This hierarchicaldata structure makes traversing and sampling the data set veryefficient. Rather than store a single partition of the spherefor one order, we store a set of partitions for orders 1..N ,where N the maximum depth. This is chosen to be eightso that the minimum solid angle of 4π/(12 × 48) is about

9

Fig. 4. Distributions of data points on the sphere for HEALPix orders 2(a:192 points), 3 (b:768 points), 4 (c:3,072 points), and 5 (d:12,288 points)

1.6e−5 steradians or roughly a quarter the average solid anglesubtended by the sun (i.e. we can have about 4 uninterpolatedsamples for the solar disk, which is a good practical limit).

Populating and optimizing the SQT data structure withmeasured or model generated data is a multi-step process. Thedetails of this process are described here using data sampledfrom the Henyey-Greenstein (HG) [21] phase function withan average cosine parameter of 0.9. Visualizations of the SQTdata structure during the process are illustrated in Figure 5(note that the HG phase function is a spherical function andthe figure depicts only the forward scattering hemisphere).

The initial, empty SQT is uniformly constructed to themaximum depth. The known values are then inserted into theSQT at the appropriate angular locations. In this case, theHG function was sampled with constant angular sampling,which produces varying solid angles. As a result, the SQTis sparsely populated at solid angles near the perimeter due toit’s inherent constant solid angle sampling (c.f. Figure 5(a)).The next step in the process is to average nodes that containmultiple values and aggregate empty child nodes into theirparent partition until every leaf node contains a value. Theresult of this process is shown in Figure 5(b), where theSQT features lower order (larger) nodes in regions that wereoriginally sparsely filled with data. The final step optimizes theSQT by reducing it to a minimum number of required nodesbased on the magnitude of the values stored at the nodes. Afamily of 2 x 2 child nodes that contain similar values can beaggregated into the parent with minimal loss of information.In other words, areas of the weighting function that have littleor no variation can be consolidated until the overall impactnecessitates angular variation. Figure 5(c) shows an optimizedSQT that captures the angular variation of the original datathrough adaptive sampling within the geometric structure. Theoriginal 64,800 samples (one degree samples) that were seededinto the high order SQT structure are now represented bya 612-node SQT. This 100x reduction of the original data(for this HG example) into a structure that is driven byangular variation is the first of two methods used to achieve

compression the data. The second compression method is theapplication of a lossless data compression algorithm (currently,LZMA [22] is used) to the binary representation of the SQTdata structure.

To facilitate rapid and efficient sampling, the leaf nodes ofthe final tree are each assigned an index. Generating a sample(in this case, computing the scattering direction of an incidentphoton) is accomplished by drawing a single, uniformly dis-tributed random number in the range 0 to M −1, where M isthe number of leaf nodes. Finding the randomly selected nodeis performed by comparing the random index to the range ofchild indexes in each parent node and traversing down thetree until reaching the appropriate leaf node. Although everyleaf node has the same probability of being selected by theuniformly distributed random number generator, areas of theweighting function with higher magnitudes contain smaller(denser) nodes. Hence, these regions will be sampled at ahigher rate than lower probability regions containing largernodes.

The shape of the distribution of a weighting function (thePDF) may not vary at the same spectral frequency as themagnitude (integral) of the same function (if at all). To accountfor this we allow the spectral sampling for the integrals to beat a higher rate than the PDFs. When evaluating a direct querybetween two stored PDFs we simply interpolate the shape andscale the entire function so the integral matches the higherfrequency, integral samples.

Since each SQT only contains data for a single view vectorand a single wavelength (i.e. one set of hemispherical orspherical data), the final SQT storage file is an aggregateof individual SQTs for a range of view vectors and spectralsamples representing coverage of the original data source. Thedata structure is “streamed” as binary data directly in and outof storage memory and no processing needs to be done whenit is loaded for a simulation (the only processing that occursis to resample the spectral magnitude to match the currentspectral state).

V. DEMONSTRATIONS

We have chosen three demonstrations to illustrate some ofthe concepts and capabilities related to DIRSIG5. The first isa performance test of the SQT optical property model con-ducted by importing an external database of diverse materials.The second demonstration is a subset of an extensive multi-scattered radiometry verification in the context of environmen-tal remote sensing. The final shows the usage of the defaultDIRSIG5 sensor plugin to model different sources of temporalartifacts and their effect on a test scene.

A. SQT representation of the MERL database

The Mitsubishi Electric Research Laboratories (MERL)BRDF database[23] publishes reflectance functions for 100measured materials ranging from fabrics and woods to metalsand plastics. As such it provides a range of spectrally varyingshapes and magnitudes that are well suited for testing the SQT-based data model (though limited three spectral samples in the

10

(a) Known values (from querying a parameterized modelor a measurement) are stored into the appropriate leafnodes of a high order tree. The resulting uniform ordertree will contain some nodes with one or more values (red)and some nodes with none (black).

(b) Nodes with multiple measurements are averaged andempty nodes are aggregated into the parent. The resultingtree contains values at every node (nodes are colored basedon the value) and aggregated nodes will be at a lower order(note the larger size nodes in regions that lacked data).

(c) A final pass through the tree finds all sets of childnodes with small or similar values and aggregates thoseareas into the parent. At this point, the depth (order) ofthe tree mirrors the magnitude of the slope of the data atthat node.

Fig. 5. A 2-D projection of a Spherical Quad-Tree (SQT) hemisphere as it is filled with regularly sampled values from the Henyey-Greenstein[21] phasefunction (with an average cosine parameter of 0.9) and then optimized. The original 64,800 samples (one degree steps) are used to create a 612-cell SQTwith a maximum depth of six.

Fig. 6. A zoom of the specular lobe in an SQT generated from one of theMERL BRDF datasets. The original function had a highly peaked specularlobe at 20 degrees. Note the asymmetric shape of the specular lobe and howthe adaptive nature of the SQT structure captures those angular variations.The logarithm of the BRDF at each measured point is shown to make theasymmetric structure apparent.

visible region of the spectrum). The MERL database includescode that can be used to query the provided datasets.

To construct data models for the MERL database we gener-ated a dense set of random, uniformly distributed samples overthe hemisphere (representing the incidence vectors) for a rangeof view vectors and feed those into the supplied MERL codefor sampling the BRDF data. The sampled BRDF values foreach material where then imported into DIRSIG5 by fittingthe sample data with an SQT model. Figure 6 shows theasymmetric shape of the specular lobe (commonly observed inmeasured data) for one of the materials and how the SQT usesprogressively higher orders (denser nodes) to capture the gra-

Fig. 7. An RGB rendering of the 100 materials in the MERL databaseassigned to spheres over a diffuse background. Note the wide range of specularlobe shape and size as well as variations in diffuse portions of the BRDF.

dients in the BRDF. It should be noted that parametric modelsgenerally employ symmetric representations and struggle tocapture the asymmetry observed in this dataset. A DIRSIG5rendering of the 100 imported materials is shown in Figure 7on a 10 x 10 array of spheres.

To judge the performance of the model we evaluate twometrics. First, the original database (100 binary files storingthe data) is 3.3 gigabytes of data (each material is roughly33 megabytes in size). The same database represented bythe SQTs is 34 megabytes, where the individual file sizesrange from 136 to 388 kilobytes (the disparity coming fromthe adaptive consolidation of the SQT for differently shaped

11

Fig. 8. Root mean squared errors between the 100 materials in the original MERL database and the SQT approximations based on random sampling of inputvectors (1e6 samples used). Results are shown on a log scale.

(a) chrome-steel (b) aluminium

Fig. 9. Representative plots of two materials from the MERL database that demonstrated significant RMS error. Plots show a cross section of the BRDF inthe principal plane corresponding to an incident vector at 20 [deg] zenith angle.

functions). That means that we are storing the data in almost1/100th the amount of memory while also eliminating anyprocessing needed to use the data.

The second metric to consider is the quality of the fittedSQT models. The total reflectance for the given incidence(i.e. the integrated BRDF for all view directions, or DHR)is pre-computed based on the input data, before any compres-sion/reduction of the data takes place (that total is stored and isused to scale the final PDF representation). Hence, the qualityof the DHR is based solely on the input sampling and does notrepresent the performance of the SQT data structure. Instead,we want to focus on the quality of the shape captured by theSQT. To evaluate this we extracted the BRDF value from theoriginal data and the corresponding SQT representation forrandom (uniformly distributed) samples for the input vectorsand computed the root mean squared error (RMSE) betweenthe two representations. The RMSE BRDF results for all 100

materials are shown in Figure 8.

Overall, the errors are fairly low, with the majority ofmaterials below 1e−2 [1/sr] for materials that have peakBRDF values of ≈ 1e+2 [1/sr]. However, there are significantexceptions, which we found generally corresponded to highlyspecular materials. We explored the nature of this discrepancyby analyzing the fit for two materials: chrome-steel andaluminium. Figures 9(a) and 9(b) contain plots of the originaland SQT BRDF along the principal plane (vertically orientedplane aligning the source and view vectors) for an incidenceangle of 20 degrees. The original data sets are dominated by anarrow, high-magnitude specular lobe. In both cases, the widthof the SQT lobe is a close match, however there is mismatchin the peak (the SQT is 20% higher for the chrome-steel caseand 65% lower for the aluminium case). The fact that the SQTis higher or lower than the MERL data in the cross-sectionssuggests that neighboring areas outside the principal plane (but

12

still in the same SQT cells) are getting averaged within cellsto produce the resulting shape.

Although the SQT is not an exact replica of the originaldata in these extreme cases at high resolution angles (i.e. itisn’t capturing the inherent noise or high resolution featuresin the data), it is a faithful representation of the characterof the BRDF. Furthermore, it could be argued that the SQTrepresentation smooths out what appears to be a physicallyinconsistent features in the original data (a noisy dip withinthe specular lobe for the chrome-steel material and a spikein the aluminium that would be noise-prone in a simulationas the peak nears a bright source). If preserving that detail isdesired, then the SQT construction parameters can be modifiedto be more conservative in terms of sampling convergenceand consolidation thresholds (presumably at the cost of lessefficient storage) and non-random sampling can be used tocapture specific features.

B. Multi-scatter radiometry verificationExcept in mostly trivial cases, there is no practical closed-

form solution for multiple-scatter radiometry. Instead, we needto find another means of determining whether the performanceof this core capability in DIRSIG5 is acceptable. Luckily,the RAdiation transfer Model Intercomparison (or RAMI)initiative[24] was developed to answer these types of questionsin the context of forestry research. It is a collection of three-dimensional scenes of forest canopies (both abstract represen-tations and measured geometry) that provide a testbed for awide range of independent, physics-based radiative transfercodes. Each code that participates in the initiative simulatescharacteristic optical properties for each scene—such as theblack sky albedo (DHR) or the bidirectional reflectance factor(the BRF, which is analogous to the BRDF).

Covering even one of the dozens of scenes provided byRAMI in full is beyond the scope of this paper (a more detailedverification will be published elsewhere). Instead we showselected results from a single scene of an abstract canopy.The scene is part of the third generation (RAMI3) data setsand is made up of abstracted leaves (simple disks) distributedin a volume with orientations that are characteristic of anerectophile leaf angle distribution (i.e. leaves are, on average,are in the up-down orientation). A visualization of the leafgeometry is shown in Figure 10.

Two spectral regions are modeled. The first is in the visibleregime (nominally “RED” wavelengths), which is dominatedby reflectance (total leaf reflectance and transmittance are0.0546 and 0.0149, respectively). The second models near in-frared conditions (“NIR”) where the characteristic chlorophyllproperties in the leaves equalize reflectance and transmittanceat high values (0.4957 and 0.4409, respectively), and theresults are dominated by multiple scattering between leaves.For both cases the respective optical properties are governedby a Lambertian (angularly constant) property. Canopies areintended to provide cyclic boundary conditions – this isapproximated by simply putting multiple instances of the scenein a grid.

Results of DIRSIG5 simulations were uploaded to theRAMI On-line Model Checker (ROMC) for comparison

Fig. 10. Visualization of the RAMI3 HOM03 abstract tree canopy composedof uniformly sized disks distributed at various orientations.

against reference simulations from the original contributors ina “debug” mode (a validation mode also exists that providespreviously unpublished scenes for comparison and will beapplied to the final DIRSIG5 release). Comparisons for theRED case at 20 (Figure 11) and 50 (Figure 12) as well asthe NIR case (Figures 13 and 14) are shown.

Based on the comparison, especially the higher orderscattering from the NIR simulation, we can conclude thatDIRSIG5 is consistent with dedicated forest canopy modelsfor multiple-scatter radiometry. Our simulations also revealthat the RED regime need an average path length of three,while the NIR simulations use five successive scattering eventson average.

While it would have been possible to produce the radi-ances for the BRF measurements using the standard sensorplugin, this demonstration was an opportunity to exploit theflexibility of the sensor API. The RAMI experiment pluginwas implemented such that each required measurement wastreated like a capture with individual, fixed collection angles(samples averaged over the entire area of the scene) formingthe problems. This enabled efficient collection of data thatcould then be internally processed and output directly in theformat required for submission to the ROMC. The flexibilityof the sensor API and the DIRSIG model as a whole can behighlighted by considering a multi-scale modeling approach.A variant of this tool could be used to generate data to useas input into a simulation at a coarser scale. For instance asimulation of a (more detailed) tree canopy could be used todirectly generate an SQT based BRDF in another scene wheretree canopy geometry would have been sub-pixel.

C. Simulating a UAS framing array system

The DIRSIG model is primarily used to simulate imagerycaptured by remote sensing systems and this final demonstra-tion focuses on that use case. The platform for the scenario isa small, unmanned aircraft system (UAS) with a 1280 x 720panchromatic framing array camera. The system is circlingaround a scene at the edge of a wooded area while staringat a series of objects and calibration targets, including astandard USAF tri-bar target. A series of simulated imagesare presented below that illustrate how the sensor model

13

Fig. 11. ROMC generated comparison between DIRSIG5 and RAMI initiativeparticipants. Data shown is for the principal plane of the BRF for RED opticalproperties and a source zenith angle of 20 degrees.

Fig. 12. ROMC generated comparison between DIRSIG5 and RAMI initiativeparticipants. Data shown is for the principal plane of the BRF for RED opticalproperties and a source zenith angle of 50 degrees

can produce several common motion induced image artifactsthrough the generation of the rays used to initiate paths in theradiometry core.

The image in Figure 15 is an instantaneous radiance imagecreated by assigning all the rays the same timestamps andthe location of each pixel doesn’t change. The foregroundobjects (the tree branch), midground targets (the tri-bar target)and background objects (distant trees) are qualitatively sharp.The image in Figure 16 was produced by modeling the

Fig. 13. ROMC generated comparison between DIRSIG5 and RAMI initiativeparticipants. Data shown is for the principal plane of the BRF for NIR opticalproperties and a source zenith angle of 20 degrees

Fig. 14. ROMC generated comparison between DIRSIG5 and RAMI initiativeparticipants. Data shown is for the principal plane of the BRF for NIR opticalproperties and a source zenith angle of 50 degrees

integration time of the imaging array by assigning randomtimestamps within the integration period to each pixel ray andcomputing the change in pixel location (due to the motion) forthose corresponding timestamps. During the integration, theplatform swings around the scene enough that objects in theforeground (the tree branch) and background (the distant trees)are impacted by motion blur. However, the tri-bar target at thecenter of the scene remains sharp due to the lack of motionnear the stare point. The image in Figure 17 was produced

14

(a) Full 1280 x 720 frame (b) Zoom

Fig. 15. Simulated imagery without temporal integration.

(a) Full 1280 x 720 frame (b) Zoom

Fig. 16. Simulated imagery with temporal integration and smooth motion.

(a) Full 1280 x 720 frame (b) Zoom

Fig. 17. Simulated imagery with temporal integration and jittered motion.

by modeling the integration time and introducing jitter to theplatform through a frequency spectrum associated with thevertical location of the platform. This introduces a verticalmotion (due to the jitter) to the existing horizontal motion (dueto the flight of the platform) to the detector location duringthe integration time. As a result, the scene features a distinctvertical blur across the entire scene, which is best observed inthe appearance of the calibration targets in the zoomed area.

It should be emphasized that the blurring present in Figures

16 and 17 are not created via direct image convolution ofthe instantaneous radiance image in Figure 15. These effectsare produced by the sensor plugin as a result of generatingpixel sampling rays with different origins and timestamps,which captures how the pixel views the scene from a spatiallyextended area (due to platform motion) during the integration.

VI. SUMMARY

A new radiometric core has been introduced that allowsthis next generation version of the DIRSIG model to take

15

advantage of hardware and conceptual advancements madesince the inception of last version, almost 15 years ago. Thetheme of this revision has been to unify and optimize thedisparate elements of DIRSIG4 that allowed for open-endedflexibility at the cost of efficiency and gather the new modelsunder as single radiative transfer algorithm that is suited forparallelization. The burden placed on the user for balancingradiometric accuracy with runtime computation costs on a per-material/object basis has been replaced with a single, universalfidelity metric.

We have also introduced a new, data-driven method for stor-ing, evaluating, integrating and sampling BRDF, BDTF and/orBSDF type data using an equi-area, hierarchical data structure.The process for loading and optimizing this spherical quad-tree (SQT) data structure was outlined and the use of SQTto import measured BRDF data into the model was shown.The use of SQT for spherical and hemispherical datasets withDIRSIG5 means that specialized integration and samplingsolutions do not need to be created for each parametric BRDF,BDTF and/or BSDF model. In the near future, the DIRSIG5model will leverage the flexibility of the SQT data model tocharacterize other data sources, including the angular intensitydistributions of in-scene sources (e.g. street lights, head lights,etc.).

At the same time, we’ve maintained and expanded whatmakes DIRSIG unique to the remote sensing community. Thefocus of development has been on establishing a powerful andhighly adaptable core radiometry engine and to provide theability to use that engine through detailed sensor models andplugins. The focus is on accurate, spectrally based radiom-etry for dynamic simulations. To this end the architectureof DIRSIG5 has been geared towards spectral and temporalstate computations that can be distributed on a wide rangeof hardware environments. The radiometry engine changeshave made multiple-scatter solutions easily accessible andcommunity based verification of the path tracing approachhas been presented as the basis of on-going verification andvalidation work.

REFERENCES

[1] DCS Corporation, “Airsim thermal signature and prediction analysis tooldefinition and analysis of object inputs,” DCS Technical Note 9090-002-004, December 1990.

[2] F.X. Kneizys, E.P. Shettle, and W.O. Gallery, “Atmospheric transmit-tance and radiance: The LOWTRAN 5 code,” 1981, vol. 0277, pp.116–124.

[3] T. Whitted, “An improved illumination model for shaded display,”Communications of the ACM, vol. 23, no. 6, pp. 343–349, June 1980.

[4] A. Berk, P. Conforti, R. Kennett, T. Perkins, F. Hawes, and van denBosch J., “MODTRAN6: a major upgrade of the MODTRAN radiativetransfer code,” SPIE, June 2014, vol. 9088.

[5] Analytical Graphics Incorporated, “Systems toolkit (STK),”http://www.agi.com/products/stk, Accessed: 2017-01-11.

[6] NVIDIA Corporation, “NVIDIA IRAY,”http://www.nvidia.com/object/nvidia-iray.html, Accessed: 2017-01-11.

[7] S.M. Adler-Golden, S.C. Richtsmeier, A. Berk, and J.W. Duff, “Fastmonte carlo-assisted simulation of cloudy earth backgrounds,” in RemoteSensing of Clouds and the Atmosphere XVII. SPIE, 2012, vol. 8534.

[8] C. Schweitzer, K. Stein, N. Wendelstein, L. Labarre, K. Caillaut,S. Fauqueux, C. Malherbe, A. Roblin, B. Rosier, and Simoneau P.,“Validation of the background simulation model MATISSE: comparingresults with MODIS satellite images,” in Optics in AtmosphericPropagation and Adaptive Systems XIV. SPIE, 2011, vol. 8178.

[9] J.R. Schott, A.D. Gerace, S.D. Brown, M.G. Gartley, M. Montanaro,and D. Reuter, “Simulation of image performance characteristics ofthe Landsat Data Continuity Mission (LDCM) Thermal Infrared Sensor(TIRS),” Remote Sensing, vol. 4, no. 8, pp. 2477–2491, August 2012.

[10] J.T. Kajiya, “The rendering equation,” Proceedings of SIGGRAPH ’86,vol. 20, no. 4, pp. 88–95, 1986.

[11] C. Karney, “GeographicLib,” http://geographiclib.sourceforge.net, Up-dated: 2015-07-30, Accessed: 2017-01-11.

[12] Intel Corporation, “Embree,” http://embree.github.io, Updated: 2015-07-30, Accessed: 2017-01-11.

[13] J.R. Schott, C. Salvaggio, S.D. Brown, and R.A. Rose, “Incorporationof texture in multispectral synthetic image generation tools,” in Char-acterization and Representation, Models and Generators. SPIE, 1995,vol. 2469, pp. 189–196.

[14] The HDF Group, “Hierarchical Data Format (HDF),”https://www.hdfgroup.org, Accessed: 2017-01-11.

[15] S.D. Brown and A. Goodenough, “Plugin-driven sensor modeling ofremote sensing imaging systems,” in Proceedings of the 2017 IEEEInternational Geoscience and Remote Sensing Symposium (IGARSS).IEEE, 2017, Submitted.

[16] J. Shell, Polarimetric Remote Sensing in the Visible to Near Inrared,Ph.D. thesis, Rochester Institute of Technology, 2005.

[17] J.R. Schott, Remote Sensing, An Image Chain Approach, 2nd Edition,Oxford Press, 2005.

[18] G.J. Ward, “Measuring and modeling anisotropic reflection,” Proceed-ings of SIGGRAPH ’92, vol. 26, no. 2, pp. 265–272, 1992.

[19] K. M. Gorski, E. Hivon, A. J. Banday, B. D. Wandelt, F. K. Hansen,M. Reinecke, and M. Bartelmann, “HEALPix: A Framework for High-Resolution Discretization and Fast Analysis of Data Distributed on theSphere,” The Astrophysical Journal, vol. 622, pp. 759–771, Apr. 2005.

[20] B.F. Roukema and B. Lew, “A solution to the isolatitude, equi-area,hierarchical pixel-coordinate system,” Submitted to: Astron. Astrophys.,2004.

[21] L.G. Henyey and J.L. Greenstein, “Diffuse radiation in the galaxy,”Astrophysical Journal, vol. 93, pp. 70–83, 1941.

[22] I. Pavlov, “Lempel-Ziv-Markov chain algorithm (LZMA),”http://www.7-zip.org/sdk.html, Accessed: 2017-01-11.

[23] W. Matusik, H. Pfister, M. Brand, and L. McMillan, “A data-drivenreflectance model,” ACM Transactions on Graphics, vol. 22, no. 3, pp.759–769, July 2003.

[24] J-L. Widlowski, M. Taberner, B. Pinty, V. Bruniquel-Pinel, M. Dis-ney, R. Fernandes, J.-P. Gastellu-Etchegorry, N. Gobron, A. Kuusk,T. Lavergne, S. Leblanc, P. Lewis, E. Martin, M. Mottus, P. J. R. North,W. Qin, M. Robustelli, N. Rochdi, R. Ruiloba, C. Soler, R. Thompson,W. Verhoef, M. M. Verstraete, and D. Xie, “The third RAdiation transferModel Intercomparison (RAMI) exercise: Documenting progress incanopy reflectance modelling,” Journal of Geophysical Research, vol.112, 2006.