Research Article Reconstructing 3D Tree Models Using...

Hindawi Publishing CorporationInternational Journal of Computer Games TechnologyVolume 2013 Article ID 363160 11 pageshttpdxdoiorg1011552013363160

Research ArticleReconstructing 3D Tree Models Using MotionCapture and Particle Flow

Jie Long and Michael D Jones

Department of Computer Science Brigham Young University Provo UT 84604 USA

Correspondence should be addressed to Jie Long may109gmailcom

Received 8 May 2013 Revised 27 August 2013 Accepted 18 September 2013

Academic Editor Alexander Pasko

Copyright copy 2013 J Long and M D Jones This is an open access article distributed under the Creative Commons AttributionLicense which permits unrestricted use distribution and reproduction in any medium provided the original work is properlycited

Recovering tree shape frommotion capture data is a first step toward efficient and accurate animation of trees in wind usingmotioncapture data Existing algorithms for generating models of tree branching structures for image synthesis in computer graphics arenot adapted to the unique data set provided by motion capture We present a method for tree shape reconstruction using particleflow on input data obtained from a passive optical motion capture system Initial branch tip positions are estimated from averagedand smoothed motion capture data Branch tips as particles are also generated within a bounding space defined by a stack ofbounding boxes or a convex hull The particle flow starting at branch tips within the bounding volume under forces creates treebranches The forces are composed of gravity internal force and external force The resulting shapes are realistic and similar to theoriginal tree crown shape Several tunable parameters provide control over branch shape and arrangement

1 Introduction

Reconstruction of tree shape from motion capture datais an important step in replaying motion capture of treesunder external forces such as natural wind Motion captureprovides a fast and easy way to collect the locations over timeof retroreflectivemarker locations placed on an object In thispaper we address the problem of creating 3D tree shape frommotion capture data We also discuss the best practices forcollectingmotion data from a treeThis researchwill focus onreconstructing static 3D tree shape with branching skeletonsfrom data collected by motion capture system

Solutions to the motion capture problem for trees canbe applied to problems in visual effects and the study oftree motion Motion capture is a potential solution becausemotion capture data includes wind effects which are difficultto model in simulation such as variable branch stiffnessnonuniform variation in size and emergent effects due to leafdeformation

Tree shape modeling has long been studied on computergraphics L-systems [1ndash3] generate branching structures usingpredefined rules Parametric models [4] embed treersquos biologyinformation into growth and shape using a parametric set

Approaches based on photographs [5ndash7] or videos [8 9]create tree shapes in 3D space by image processing methodsLaser scanning has been employed for collecting informationof 3D tree shapes [10ndash12] Particle systems [6 13ndash15] representeach branch as a result from particle flow simulation

Most of these methods result in satisfying tree shapes butdo not leverage the 3D positions of motion capture markerswhich are recorded as part of a motion capture sessionbut require a different set of inputs Image- or video-basedapproaches convert a set of 2D input images into 3D treemodels by filling in the missing dimension Motion capturesystems can record tree shape in 3D with high precision(using similar techniques for converting a set of 2D imagesto a 3D model) Prior work in reconstructing tree shapefrom motion capture data uses either exact measurementsor markers placed within the crown The approach based onphysical measurements can reconstruct the tree shape fora single measured tree but our method scales to producemultiple tree shapes from the measured data In the previousapplication of recording tree shape from motion capturemarkers are placedwithin the crown and the visibility of thesemarkers could be occluded easily with tree leaves We solvethe problem by designing a new process for motion capture

2 International Journal of Computer Games Technology

in which we place markers on the edge of the tree crown tomaximize the visibility of markers

We reconstruct 3D tree shape using particle flow withmotion capture data as input Passive optical motion capture(httpwwwnaturalpointcom) records locations of reflec-tive markers in the capture arena We place markers only atbranch tips on the edge of the tree crown Approximately 30markers cover the crown shape of amedium sized treeWe donot put markers on each branch tip because passive opticalmotion capture systems cannot reliably track more than 70markers due to device limitation A particle flow algorithmgenerates branching structures starting at the recorded tippositions Additional starting points are defined within theestimated volume of the tree crown using a vertical stackof bounding boxes or a convex hull The bounding spaceapproximates the tree crown and bounds particle flow andcreation The step length of a particlersquos flow varies with thedistance to the nearest trunk point The direction of particlemotion is a combination of three forces which are gravityinternal force and external force In our research we definea force called shape-format as an instance of internal forceand apply wind force as external force The shape-formatguides the particles to preserve the original tree shape Thedominant wind direction is recorded during the motioncapture process We also introduce two vectors with onepointing to the nearest trunk point and the other pointing to aconstant predefined attractor point These vectors are factorsfor the direction and magnitude of the forces

Compared with the existing work we present a new pro-cedure of extracting 3D tree shape with branching structuresfrom motion capture system and prove the following advan-tages over the existing work First the functions guidingparticle flow are classified into three categories which areextensible for adding more features and are not limited tofunctions that the paper is presenting to produce various treeshapes as desired Theoretic knowledge including biologyand dynamic mechanism about trees is embedded into thefunctions Second our approach does not require preknowl-edge of rules such as the rule-based approach or parametricmodels Third motion capture provides more informationabout the original natural tree shapes than the existing datacollection process Photographs provide 2D data for the orig-inal tree shape while other approaches such as the rule-basedapproach or parametric models might not start from naturaltree shapes With more information available we producebetter visually plausible tree shapes to the original treesFourth our method improves accuracy of the positionalinformation of tree branches as well as improves efficiencyof the procedure to generate 3D tree shapes Photographs- orvideo-based approaches usually require techniques related toimage processing or camera calibration which are difficultto make accurate By applying motion capture system wedirectly use the industrial standard of these techniquesIn addition our approach extends the research objects ofmotion capture frompopular rigid bodies to nonrigid bodiesExisting research applying motion capture to trees assumesthat trees are rigid bodies or semirigid bodies In our researchwe treat trees as complete nonrigid bodies which better fitstreesrsquo nature

In this paper we propose a new particle flow method forreconstructing tree shape fromonlymotion capture dataOurprimary contributions are

(i) a simplified particle flow algorithm for constructingtree shapes from a sparse set of branch tip positionsas collected as part of motion capture

(ii) a method for deploying passive optical motion cap-ture to reconstruct the shape and motion of trees inthe laboratory

This paper extends our prior work on 3D tree modelingfrommotion capture [16] In this extended paper we general-ize the forces that guide particle flow The generalized forcescan be customized in the framework of forces to producedifferent kinds of 3D tree models from the same motioncapture data In addition we implement the convex hull as analternative approach to bounding all the particles andmarkerlocations on a tree This paper includes 3D models of treesbuilt using these extensions and the models are compared toprior work

The combination of motion capture with a particle flowmethod provides an innovative way of creating 3D treeshapes The resulting tree shapes are similar to the originaltrees as shown in Figure 1 and can be used to replay motionsimilar to the capturedmotionWith the flexibilities of tuningthe forces we can produce several tree shapes besides theoriginal shape Figure 2 shows various tree shapes generatedfrom the same set of motion capture data and demonstratesthe flexibility of our forces for guiding particle flow

2 Related Work

Our work is most closely related to prior work in treemodeling and applied motion capture Our purpose is toinvestigatemethods for creating tree shapes onwhichmotioncapture data can be replayed Compared to prior work inmodeling trees we introduce new equipment for collectingshape information of trees and improve the particle flowprocess by creating a new set of functions to computeattracting factors Specially the particle flow process fits wellwith the data collection process Compared to prior work inmotion capture we design a new data collection process fornonrigid bodies and reconstruct realistic 3D models out ofthe data

21 Tree Modeling L-systems [1] generate a treersquos branchingstructure using axioms and rules in a concurrent context-free rewriting system L-systems have been extended inmanywaysThe extensionmost relevant to our work is [3] in whichL-systems are enriched with partial differential equations andcan be parameterized to reconstruct the shape of a specifictree or plant However L-systems require understanding ofthe rewriting system and the according parametric systemsWith different rules and parameters the resulting shapes oftrees could greatly vary Most of the time L-systems build atree model from trunk to branches For our work we collectpositions of branch tips for describing tree shape Startingfrom these positions of branch tips back to the trunk it

International Journal of Computer Games Technology 3

(a) (b) (c)

Figure 1 Maple model (a) Is a photo of the original tree (b) Is the 3D tree model from the result (c) Demonstrates the branching structureand locations (in grey cubes) where markers are placed

Figure 2 Various tree shapes generated from the same set ofmotioncapture data

becomes harder to combine the collected data from motioncapture with L-system techniques for generating 3D treeshapes

Laser scanning can be used to record 3D tree shapes Priorwork such as [10ndash12] collects tree shape information fromlaser scanning and reconstructs 3D tree shapes Livny et al[11] start from the position of a treersquos root and recursivelyrefine a BSG graph (Branch-Structure Graph) given a setof global parameters Later Livny et al [12] further developan approach which creates a lobe-based representation ofthe tree crown and adds texture using a predefined branchlibrary for different shapes and species The texture basedtree models improve the visual quality of 3D tree modelsbut make it difficult to create tree animations because it ishard to animate trees with texture clusters Xu et al [10] startfrom the root and use a shortest paths based approach tocreate branching structures Laser scanning records about50000 points per second and thus collects more completeinformation about tree shapes It also has the advantageof recording bigger trees However laser scanning recordsleaves and branches together in the same point cloud Thatmixes positional information and requires extra effort toseparate leaves and branches Leaf occlusion makes it hard torecord branching structures when leaves have high densityespecially in the crown area or in certain tree species Withmore positional information the point cloud gets big andrequires more computational resources during processing Itis hard to keep track of a single point in the point cloudamong frames and this limits its application in recording treeanimation In our research we place markers on the edgeof the tree crown which preserves the tree crown shape

As laser scanning could not address the exact branchingstructure even with more available information about thetreersquos shape our research is not aimed at getting the exactbranching structure but instead we provide flexibility tousers who adjust the parameter values for generating variousbranching structures from the same set of recorded dataMocap records less information about tree shape but the datais very accurate For moving trees we could easily match apoint position in its neighboring recorded frame because therecording rate is about 100 frames per second The designallows researchers to further explore the usage of mocap inanimating trees and this is where we contribute a method tomodel 3D trees frommocap as the first step of tree animation

More recentwork in tree shapemodeling involves particlesystems With the exception of Palubicki et alrsquos work [15]particle systems approximate tree shapes obtained fromphotographs [5ndash7] Palubicki et al devised a particle flowwhich approximates bud fate models This research extendsthe Borchert-Honda (BH) model and combines with theconcept of self-organizing trees The simulated amount oflight guides the particle flow process and results in differenttree shapes Comparing to Palubickirsquos work in our researchwe designed a new particle flow process which fits with thepositional data collected from motion capture to build atree model The particle flow formats the treersquos branchingstructure Instead of the light amount computed in Palubickirsquoswork we drive the flow using three forces gravity internalforce and external force In addition the factors for internalforce and external force can be defined differently in differentcircumstances Thus our model has better extensibilities tocreate trees with various shapes

These methods are not directly applicable to motioncapture data because these methods use photographs or envi-ronmental conditions that are not collected during motioncapture Photographs of the tree could be taken duringmotion capture (and indeed are taken during optical motioncapture) but we focus on taking only marker positions asinput because this simplifies data collection and processingby reusing the background removal and image alignmentperformed as part of marker position calculation

22 Applied Motion Capture Motion capture has beenmainly used in rigid bodies such as human motion It


produces positions for points on an object over timewith verysmall measurement errors Ourmain contribution regardingmotion capture is that we extend the usage ofmotion capturefrom the popular rigid bodies to nonrigid bodies In additionwe design an innovative data collection process for describingtree shapes using motion capture

Motion capture systems have beenwidely used for humanor animal motion capture [17ndash19] Kirk et al [20] automat-ically generate rigid skeletons from optical motion capturesystems by preserving a constant distance for each rigid partThese algorithms assume that the distance between markerson the same bone is invariant and cannot be directly appliedto nonrigid bodies such as natural trees because the distancebetween markers is not invariant as the object deforms

Prior work in motion capture for nonrigid bodiesincludes several approaches to facial motion (including [2122]) These methods are based on domain-specific featuresof the facial structure or patterns Obviously these domain-specific features do not apply to tree shape reconstruction ormotion capture

The uniform branching structure of human bodies hasled to well-understood processes for deploying markers on ahuman The number and placement of markers is a criticalpart of successful motion capture using a passive opticalsystem Trees have a more complex and less predictabletopology and this will require a different approach to markerplacement Previously [23] researchers attempted to directlyreplay themotion of a tree inwind following the exactmotionpaths collected for all branches by putting markers on everybranch of the tree Leaves on the tree create marker occlusionwhich results in poor data In addition manually definingbranch topology to exactly match the subject tree is laborintensive

In this paper we design a data collection and treemodeling process to overcome these difficulties by building asimilar but not exact copy of the branching structure froma partial collection of branch tip positions Ongoing workfocuses on replaying collected motion such that the motionlooks natural on an approximate copy of the branchingstructure

3 Motion Capture of Trees

In this section we describe how to collect data from treesusing a motion capture system such that the data supportsreconstruction of tree shape The data will be collectedindoors on trees with height less than 25 meters The datacollection process results in an unindexed set of markerlocations over time for a small set of instrumented tree branchtips

We use a passive optical motion capture system (Opti-Track V100 by NaturalPoint) to collect data The passiveoptical capture system strikes balances conflicting featuresPassive optical systems can reliably track up to 70 markersand some markers weigh only a few grams This is ideal forworking with tree crowns Active optical and active magneticsystems use heavier markers and cannot track more than 20markers at once The magnetic systems have the advantage

Figure 3 Marker placement on the crown periphery avoids occlu-sion while generating data that can be used to generate crownstructure

that they do not have visibility occlusions and can trackposition and rotation but are more expensive than passiveoptical systems and can track fewer branch tips Magneticsystem markers are heavier and may alter branch tip motion

Collecting data from a natural tree is challenging forpassive optical motion capture systems because trees arenot rigid bodies and partially self-occluding The followingmethod for deploying a passive opticalmotion capture systemcollects data from which tree shape and motion can beinferred

Twelve or more cameras are arranged in a circle aroundthe tree with 6 cameras located approximately 08 metersabove the ground and another 6 cameras located 33 metersabove the ground For each camera field of view is adjustedto include the entire tree About 30 markers are placedon branch tips throughout the crown Markers are squareretroreflective markers with surface area of about 1 cm andadhesive backing Markers are placed to cover branch tipson each major branch from the stem and to provide nearlyuniform coverage of the crown On these branch tips thesquare shaped markers are wrapped to cover the whole tipso these markers are visible to most of the cameras fromdifferent view angles Uniform coverage improves both shapereconstruction and motion capturing Placing markers suchthat their motion paths overlap complicates algorithms forextracting motion paths from unindexed marker positionsFor a medium sized tree the number of branch tips exceedsthe number of markers so that not every branch tip is coveredby a marker Leaves around the marker are removed toimprove marker visibility So in the resulting 3D tree shapethe shape of the original tree crown will be preserved Whilerecording tree motion we use an electric fan to create windaround the tree because data is collected indoors The winddirection is inferred from the location where it sits relativeto the tree Other statistic methods such as PCA (principlecomponent analysis) can also compute the dominant winddirection after motion data of branch tips are processed

Photographs in Figure 3 show the arrangement of themotion capture cameras and the reflective markers asdeployed on an indoor pine treeMarkers are placed at branchtips shown as white dots in the image The right side imageshows marker location in the red box on the left side image

Our design requires less user intervention producescleaner motion capture data and may support animation oftree motion


4 Data Processing

Simply inferring positions from one frame of captured datais not adequate due to noise These branch tips with markersare called ldquorecordedrdquo or ldquocapturedrdquo tips However the initiallocations of recorded branch tips may contain errors due tonoise in either the system or the capture environment

A clustering algorithm approximates a single initialposition from a collection of captured initial positions forrecorded branch tipswhileminimizing error from themotioncapture system The clustering algorithm analyzes positionsover many frames of motion captured data and eliminatesgaps and noise Gaps occur when a marker is not present ina frame This algorithm uses forward differencing to predicta position for a marker at frame 119872 based on positions inprevious frames The closest marker in the next frame isadded to the motion trace for the marker if there is a markerlocated close enough to the predicted position If there isno marker position recorded close enough to the predictedposition that marker is marked with no data that is a gapfor that frame Gaps are repaired using interpolation over themotion path for a marker If the number of marker positionsrecorded for a marker over time is less than 14 of the totalframes that partial marker trace is marked as noise andeliminated

If the capture process includes several hundred frames ofdata in which the tree is not moving the averaging branchtip positions over these frames result in precise estimatesof initial positions In most cases the number of markersinferred from the clustering process matches the number ofmarkers placed on the tree

5 Generating 3D Tree Shape

Particle flow is a well-studied approach to generating 3Dbranching structure for trees [6 13ndash15]We adapt the methodtomotion capture dataThe 3D tree crown boundary inferredfrom motion capture data constrains the particle rangeand preserves the original 3D tree silhouette We use astack of bounding boxes or a convex hull to represent thecrown boundary Particles either generated randomly in theboundary or locations of branch tips recorded by motioncapture are moving towards trunk nodes Three forces fromgravity internal force and external force drive the particleflowprocessThepaths of the particle flowproduce branchingstructure By attaching leaves to the branching structure wegenerate a 3D tree model that has similar appearance as thenatural tree shape

We synthesize a trunk in the center of the crown shapeas shown in Figure 4 Figure 4(a) shows a photograph of pinetree with markers placed on the crown Figure 4(b) containsa bounding box of these marker locations and a straightvertical line representing trunk shapeThe length of the line isscaled by the crown height of the bounding box On this linesegment we generate about 10 trunk nodes Random offsetsto these nodes in 119909 and 119911 directions are added as shown inFigure 4(c)

A particle represents a branch tip One group of particlesis 3D positions of branch tips recorded frommotion capturewhich is described in Section 4 Figure 4(a) shows a photo-graph of a pine tree withmarkers placed on the crown All thebranch tip locations recorded by motion capture are labeledin white dots These particles are shown in black circles inFigures 5(c) and 6

Because of limitations with the motion capture devicersquoscapability we cannot record location information for everybranch tip on the tree Another group of particles is randomlygenerated within the bounded space of all the recordedbranch tip positions In Figures 5(c) and 6 these particles arein green color

Instead of using one single bounding box for the wholetree crown as shown in Figure 5(a) we create a vertical stackof bounding boxes as shown in Figure 5(b) where the crownheight is evenly divided The number of bounding boxesvaries to match the original tree shape and required levelof detail In the case of the pine tree shown in Figure 5 weuse four parts which can closely match the tree shape wellParticles are randomly generated inside the smaller boundingboxes as shown using green dots in Figure 5(c) The totalnumber of branch tips including motion capture branchtips and randomly generated particles is set to be similarto the original tree The number of green dots in each boxis proportional to the number of black dots Therefore wemaintain a similar total amount and distribution of naturaltree branch tips

Alternatively we use a convex hull for all the particlesas shown in Figure 6 A convex hull provides more precisebounding space than the stack of bounding boxes Howeverit requires a (slightly) more complex boundary detectionscheme and more implementation details To pursue betterfit for the tree shape other bounding methods such as alphashapes [24] could also be considered

For the pine treersquos trunk we generate 9 nodes in a straightvertical line and add random offsets in 119909 and 119911 directions tothese trunk nodes The length of the line is scaled about 12times of the pine treersquos crown height

Trunk nodes contain a root attractor point and a nearesttrunk point as shown in Figure 7 The root attractor pointshown in green is the trunk node closest to the lowerbound of the bounding box for the entire crown The nearesttrunk point which is shown in blue for the red particle inFigure 7(a) is the closest trunk node to a single particle

Particles move under directions of three forces gravityshape-format and dominant wind In Table 1 we describedirection and magnitude of each force Gravity points ver-tically down to the ground We assume that a particle hasbigger mass when it is closer to the trunk This assumptionfollows the observation that when closer to the trunk abranch has a bigger radius For a particle with bigger massit has bigger magnitude of gravity and moves faster in thevertical downwards direction Under this assumption weset the magnitude of gravity proportional to the distancebetween a particle position 119878119901 and the nearest trunk point 119878119905

We call the second force shape-format Arborists dis-tinguish styles of growth habit of trees using differentclassifications such as excurrent and decurrent The force


(a) A pine tree with markers

Boundingbox

xy

(b) A straight line represents the trunk

xy

(c) Adding random offsets looks morenatural

Figure 4 A simple trunk model is added to the collection of branch tip positions

x

y

(a) Single bounding box

x

y

(b) A stack of bounding boxes

x

y

(c) New particles (green)generated

Figure 5 New particles are added in a vertical stack of bounding boxes

Table 1 Forces gravity shape-format and dominant wind

Force Direction MagnitudeGravity (0 minus 1 0) 10038161003816100381610038161003816

119878119901 minus 119878119905

10038161003816100381610038161003816

Shape-format 997888997888997888997888997888119878119901 minus 119878119903 120572(

1198781

119901119910+ sdot sdot sdot + 119878

119899

119901119910

119899)

Wind 11986511990810038161003816100381610038161198651199081003816100381610038161003816

119878119901 position of a particle 119878119901119910 position of a particle in119910 direction 119878119905 positionof the nearest trunk point 119878119903 position of the root attractor point 119865119908 windmeasured from motion capture setup 120572 scaling weight in range of [0 1] 119899number of particles

tries to guide particles flow to follow the growth patternof the original tree Simulating different growth patternsrequires different definitions of the shape-format force Inthis research we provide a simple example of shape-formatdefinition The shape-format force shown in Table 1 guides

the particle flow process by height and depth of a tree crownThe direction of the force is pointing to root attractor point119878119903 from particle location 119878119901 The magnitude of the force isthe average height of all the particles with a weight parameter120572 The higher the center of all the particles the stronger theshape-format force is pointing to the root attractor point 119878119903This force is a precomputed global force which is a constantfor all the particles The direction of the force ensures that aparticle finally merges to trunk nodes and therefore all thebranches grow towards the trunk

The dominant wind direction is recorded from themotion capture setup as described in Section 3 Wind forceis a special case of external force acting on treersquos branchingshapes and structures While doing motion capture werecord the location of the electrical fan Because we only useone fan to create wind that is the only source of explicit exter-nal force Alternatively the dominant wind direction can beinferred from tree movements recorded in motion captured


Figure 6 A convex hull for all the particles Particles in black color are from motion capture and particles in green color are randomlygenerated inside the convex hull

Root attractor point

Nearest trunk point

x

y

(a) Flow direction depends on root attractorand nearest trunk point positions

B

A

C

Sr

St

(b) Root attractor point 119878119903and nearest trunk point 119878119905

A

A1B

C

B1

C1

AB

(c) Particle flow endswhen a particle mergeswith a neighbor or withthe trunk

Figure 7 A simple particle flow results in crown branching structure

data Statistic methods such as principal component analysis(PCA) can estimate the dominant wind direction

The flow of particles starts at branch tips Some particlesmerge in the flow process while others eventually reach andmerge to the trunk At each time step we compute step sizeand direction of a particle The step size 119871 is

119871 = 120573 lowast10038171003817100381710038171003817119878119901 119878119905

10038171003817100381710038171003817 (1)

where 120573 is a tunable parameter in range of [01 05] for mostof our tree models The step direction combines all the threeforces Figure 7(b) shows the computation of the direction forparticle 119894 which is shown as particle 119860 in Figure 7(b) Eachforce has a weighting parameter 120596 which tunes the relative

importance of each vector and provides flexibility in creatingbranching shapes The particle step direction 119894 is given by

119894 =1205961 lowast 119894 + 1205962 lowast

119878119894 + 1205963 lowast 119894

3 (2)

where 120596 is weight of direction 119894 is direction of gravity 119878119894is direction of shape-format and 119894 is wind direction forparticle 119894 where 119894 isin [1 119899]

Using step size 119871 and direction the new particleposition 119881(119898) is given by

119881 (119898) = 119881 (119898 minus 1) + 119871 lowast 119894 (3)

where119898 is current time step and119881(119898minus1) is particle positionat the last time step


At each time step after updating all the particle positionsparticles might be mergedWhen the distance between a pairof particles is less than a predefined merging threshold thoseparticles are combinedWhen the distance between a particleand a trunk point falls below a predefinedmerging thresholdthat particle is merged with the trunk In Figure 7(c) wedemonstrate the paths of particle flow Particles 119860 119861 and119862 are moved using the step and direction Particles 119860 and119861 are merged at point 119860119861 and finally merged to the trunkParticle 119862 merged to the trunk point after two time steps ofmovement

6 Adding Leaves

Tree leaves are visually important to 3D tree models Afterthe branching structure is generated leaves will be attachedWe use predefined leaf shapes growth pattern density andsize Also because leaves do not always start growing at thebeginning of a branch we set a parameter called the leafstarting point This parameter is proportional to a branchlength For example when the parameter is 03 leaf thenstarts after the point whose distance to the branch startingpoint is 03 times the branch length

7 Results

Results are given for multiple trees including maple and pinetrees The maple tree is instrumented with 24 markers placedon the periphery of the crown at branch tips and thw pine treeis instrumented with 35 markers also located on branch tipsWe collect the stationary locations of these markers for about20 seconds at the capture rate of 100 framessec

Figure 8(a) shows all the recorded locations as red dotsfor a single frame and Figure 8(b) shows averaged locationsfrom each cluster of marker positions for clusters in whichthe number of frames with a position in that cluster is 14of the total frame countThe data is recorded when the tree isstationary Notice that a point in the blue box in Figure 8(a) isidentified as noise and removed by the clustering algorithmFor the maple tree after clustering and averaging only 24markers remain and this matches the actual number ofmarkers placed on the tree

Although initial marker positions are collected beforewind is applied when the tree is stationary the data containa small amount of noise which can be removed to create asingle initial marker position In Figure 9 we show markerpositions over time for two markers during the stationaryphase The horizontal axis shows time while the vertical axisshows a marker location in 3D space In the first imagemovement in each direction spans 10minus3 meters In additionin the second image there is apparently a small periodicmotion for the stationary branch tip The range of motion isalso within 10minus3meters In both cases averaging removes thissmall motion and estimates initial marker positions based onthe average rather than a single position in a single frame

A vertical stack of bounding boxes is a better approxima-tion for the crown volume than a single bounding box andresults in better crown shapes Each bounding box contains

(a) Original collected data

(b) Remaining branch tip locations after clustering

Figure 8 Clustering removes noise that leads to spurious markerpositions

(a) Close to white noise

(b) Noise with apparent periods

Figure 9 Marker positions over time contain a small amount ofspurious motion which is removed by averaging

branch tips and additional branch tips are added to eachbox Figures 10(a) and 10(b) illustrate the difference betweentree crowns created with and without a stack of boundingboxes for a pine tree Random particles placed uniformlywithin a single bounding box result in a cube shaped treePlacing particles in a vertical stack of bounding boxes betterapproximates the original crown shape Future work mightinclude investigating nonuniform distributions of randomlyinserted points instead of using bounding boxes

In Figures 10(c) and 10(d) we demonstrate the differenceof tree shapes using stacked box bounding volume andconvex hull bounding volume The bounding box approachprovides a looser bounding condition and allows morerandom factors in the final tree shape The convex hullapproach more closely approximates the original tree crownshape


(a) A single big boundingbox

(b) Stacked bounding box (c) Stacked bounding box (d) Convex hull

Figure 10 Bounding volumes affect tree shape

Figure 11 Pine tree model 3D tree models created from branch tip positions are similar to the shapes of the trees from which branch tippositions were collected

Figure 12 Different branching structures under different shape-format forces

Particle flow starting from branch tips using our simpli-fied algorithm results in tree crown shapes that mimic theshape of the tree from which data was collected In Figure 11we show the results from reconstructing a pine tree

Besides replaying the original tree shape our approachhas flexibility to produce different tree shapes based on thesame set ofmotion capture dataThree forces with their scalescreate particlesrsquo moving paths which represents branchingstructures Figure 12 demonstrates that shape-format forcesets the global attracting trunk node which controls theconverging direction of particle flow The resulting treemodels display the treersquos growth style in terms of beingexcurrent and decurrent

Figure 13 shows the impact of gravity on the treersquos shapeswith various values of the weighting factor When the factoris set to be negative we create a special willow-like tree shape

Figure 14 shows tree models with different weightingfactors of wind force A bigger value of the factor producesbranches bending more toward the wind direction Noticethat the differences among these tree models are smallespecially when compared to the influence from the othertwo forces This is because with a global wind direction thevalue of force is set to be much smaller than the other twoforces Otherwise when wind force dominates the directionof particlemovement the particlesmight not be able tomergeto a treersquos trunk and violate the natural treersquos shape


(a) Weight 001 (b) Weight 020 (c) Weight minus020

Figure 13 Tree shapes with different weight factors for gravity

(a) Weight 001 (b) Weight 007 (c) Weight 015

Figure 14 Tree shapes with different weight factors for wind force

Figure 15 Various tree shapes

In Figure 15 we generate 3D tree models with differentparameters for particle flow with the same set of motioncaptured data These results demonstrate that our approachproduces visually plausible tree shapes which is scalable formore extended shapes by tweaking parameters of the threeforces

8 Discussion and Future Work

Placing retroreflective markers on branch tips evenly spacedthroughout the crown on trees located in a passive opticalmotion capture arena results in data that can be used toreconstruct tree shape and which may be usable for replaying


branch motionThis can be done using a particle flow systemstarting from recorded branch tip positions supplementedwith additional random branch tip positions within bound-ing space We bound the space using a stack of boxes or aconvex hull The procedure of particle flow involves threemain forces With various parameter values of forces wegenerate different shapes of trees A new data collectionprocess designed for trees may extend the use of motioncapture to include trees and eventually other networks ofnonrigid bodies

Future work includes extending the process to large treesoutdoors as well as improving methods for animating theresulting tree models using the motion capture data Wehave reconstructed an approximate tree crown branchingstructure Replaying the captured motion data will requirecare to ensure that the motion of the approximate branchingstructure does not include uncorrelated motion for branchtips which share a common parent

References

[1] A Lindenmayer ldquoMathematical models for cellular interactionin developmentmdashparts I and IIrdquo Journal of Theoretical Biologyvol 18 pp 280ndash315 1968

[2] B Lintermann and O Deussen ldquoInteractive modeling ofplantsrdquo IEEE Computer Graphics and Applications vol 19 no1 pp 56ndash65 1999

[3] P Prusinkiewicz L Mundermann R Karwowski and B LaneldquoThe use of positional information in the modeling of plantsrdquoin Proceedings of the 28th Annual Conference on ComputerGraphics and Interactive Techniques (SIGGRAPH rsquo01) pp 289ndash300 ACM August 2001

[4] J Weber and J Penn ldquoCreation and rendering of realistic treesrdquoinProceedings of the 22ndACMAnnual Conference onComputerGraphics and Interactive Techniques (SIGGRAPH rsquo95) pp 119ndash126 Los Angeles Calif USA August 1995

[5] A Reche-Martinez I Martin and G Drettakis ldquoVolumetricreconstruction and interactive rendering of trees from pho-tographsrdquo in Proceedings of the ACM International Conferenceon Computer Graphics and Interactive Techniques (SIGGRAPHrsquo04) pp 720ndash727 ACM Los Angeles Calif USA August 2004

[6] B Neubert T Franken and O Deussen ldquoApproximate image-based tree-modeling using particle flowsrdquo in Proceedings ofthe ACM International Conference on Computer Graphics andInteractive Techniques (SIGGRAPH rsquo07) ACM SanDiego CalifUSA August 2007

[7] P Tan G Zeng J Wang S B Kang and L Quan ldquoImage-based tree modelingrdquo in Proceedings of the ACM InternationalConference on Computer Graphics and Interactive Techniques(SIGGRAPH rsquo07) ACM San Diego Calif USA August 2007

[8] C Li O Deussen Y-Z Song P Willis and P Hall ldquoModelingand generating moving trees from videordquo in Proceedings of theSIGGRAPH Asia Conference (SA rsquo11) pp 1ndash12 ACM HongKong December 2011

[9] J Diener L Reveret and E Fiume ldquoHierarchical retargettingof 2D motion fields to the animation of 3D plant modelsrdquo inProceedings of the ACM SIGGRAPHEurographics Symposiumon Computer Animation (SCA rsquo06) pp 187ndash195 EurographicsAssociation Aire-la-Ville Switzerland 2006

[10] H Xu N Gossett and B Chen ldquoKnowledge and heuristic-based modeling of laser-scanned treesrdquo ACM Transactions onGraphics vol 26 no 4 2007

[11] Y Livny F Yan M Olson B Chen H Zhang and J El-SanaldquoAutomatic reconstruction of tree skeletal structures from pointcloudsrdquoACMTransactions onGraphics vol 29 no 6 article 151pp 1ndash8 2010

[12] Y Livny S Pirk Z Cheng et al ldquoTexture-lobes for treemodellingrdquoACMTransactions onGraphics vol 30 no 4 article53 pp 1ndash10 2011

[13] W T Reeves ldquoParticle systemsmdasha technique for modeling aclass of fuzzy objectsrdquo ACM Transactions on Graphics vol 2pp 359ndash376 1983

[14] A Runions B Lane and P Prusinkiewicz ldquoModeling treeswith a space colonization algorithmrdquo in Proceedings of theEurographics Workshop on Natural Phenomena 2007

[15] W Palubicki K Horel S Longay et al ldquoSelf-organizingtree models for image synthesisrdquo in Proceedings of the ACMInternational Conference on Computer Graphics and InteractiveTechniques (SIGGRAPH rsquo09) pp 1ndash10 ACM 2009

[16] J Long and M D Jones ldquo3D tree modeling using motioncapturerdquo inProceedings of the 4th IEEE International Symposiumon Plant Growth Modeling Simulation Visualization and Appli-cations (PMA rsquo12) pp 242ndash249 2012

[17] H Lou and J Chai ldquoExample-based humanmotion denoisingrdquoIEEE Transactions on Visualization and Computer Graphics vol16 no 5 pp 870ndash879 2010

[18] S Rajko and G Qian ldquoReal-time automatic kinematic modelbuilding for optical motion capture using a markov randomfieldrdquo in Proceedings of the IEEE International Conference onHuman-Computer Interaction (HCI rsquo07) pp 69ndash78 SpringerBeijing China 2007

[19] G Wen Z Wang S Xia and D Zhu ldquoFrom motion capturedata to character animationrdquo in Proceedings of the 13th ACMSymposium on Virtual Reality Software and Technology (VRSTrsquo06) pp 165ndash168 ACM Limassol Cyprus November 2006

[20] A G Kirk J F OrsquoBrien and D A Forsyth ldquoSkeletal parameterestimation from optical motion capture datardquo in Proceedings ofthe IEEE Computer Society Conference on Computer Vision andPattern Recognition (CVPR rsquo05) pp 782ndash788 June 2005

[21] W-C Ma A Jones J-Y Chiang et al ldquoFacial performancesynthesis using deformation-driven polynomial displacementmapsrdquo ACM Transactions on Graphics vol 27 no 5 article 121pp 1ndash10 2008

[22] E Sifakis I Neverov and R Fedkiw ldquoAutomatic determinationof facial muscle activations from sparse motion capture markerdatardquoACMTransactions on Graphics vol 24 pp 417ndash425 2005

[23] J Long C Reimschussel O Britton A Hall and M JonesldquoMotion capture for a natural tree in the windrdquo in Proceedings ofthe 3rd International Conference on Motion in Games (MIG rsquo10)pp 158ndash169 Springer Utrecht The Netherlands 2010

[24] H Edelsbrunner and E P Mucke ldquoThree-dimensional alphashapesrdquo ACM Transactions on Graphics vol 13 no 1 pp 43ndash72 1994

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Active and Passive Electronic Components

Control Scienceand Engineering

Journal of



RotatingMachinery


Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design



Shock and Vibration


Civil EngineeringAdvances in

Acoustics and VibrationAdvances in



Electrical and Computer Engineering

Journal of

Advances inOptoElectronics


Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of


Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Chemical EngineeringInternational Journal of Antennas and

Propagation




Navigation and Observation



DistributedSensor Networks



in which we place markers on the edge of the tree crown tomaximize the visibility of markers

We reconstruct 3D tree shape using particle flow withmotion capture data as input Passive optical motion capture(httpwwwnaturalpointcom) records locations of reflec-tive markers in the capture arena We place markers only atbranch tips on the edge of the tree crown Approximately 30markers cover the crown shape of amedium sized treeWe donot put markers on each branch tip because passive opticalmotion capture systems cannot reliably track more than 70markers due to device limitation A particle flow algorithmgenerates branching structures starting at the recorded tippositions Additional starting points are defined within theestimated volume of the tree crown using a vertical stackof bounding boxes or a convex hull The bounding spaceapproximates the tree crown and bounds particle flow andcreation The step length of a particlersquos flow varies with thedistance to the nearest trunk point The direction of particlemotion is a combination of three forces which are gravityinternal force and external force In our research we definea force called shape-format as an instance of internal forceand apply wind force as external force The shape-formatguides the particles to preserve the original tree shape Thedominant wind direction is recorded during the motioncapture process We also introduce two vectors with onepointing to the nearest trunk point and the other pointing to aconstant predefined attractor point These vectors are factorsfor the direction and magnitude of the forces

Compared with the existing work we present a new pro-cedure of extracting 3D tree shape with branching structuresfrom motion capture system and prove the following advan-tages over the existing work First the functions guidingparticle flow are classified into three categories which areextensible for adding more features and are not limited tofunctions that the paper is presenting to produce various treeshapes as desired Theoretic knowledge including biologyand dynamic mechanism about trees is embedded into thefunctions Second our approach does not require preknowl-edge of rules such as the rule-based approach or parametricmodels Third motion capture provides more informationabout the original natural tree shapes than the existing datacollection process Photographs provide 2D data for the orig-inal tree shape while other approaches such as the rule-basedapproach or parametric models might not start from naturaltree shapes With more information available we producebetter visually plausible tree shapes to the original treesFourth our method improves accuracy of the positionalinformation of tree branches as well as improves efficiencyof the procedure to generate 3D tree shapes Photographs- orvideo-based approaches usually require techniques related toimage processing or camera calibration which are difficultto make accurate By applying motion capture system wedirectly use the industrial standard of these techniquesIn addition our approach extends the research objects ofmotion capture frompopular rigid bodies to nonrigid bodiesExisting research applying motion capture to trees assumesthat trees are rigid bodies or semirigid bodies In our researchwe treat trees as complete nonrigid bodies which better fitstreesrsquo nature

In this paper we propose a new particle flow method forreconstructing tree shape fromonlymotion capture dataOurprimary contributions are

(i) a simplified particle flow algorithm for constructingtree shapes from a sparse set of branch tip positionsas collected as part of motion capture

(ii) a method for deploying passive optical motion cap-ture to reconstruct the shape and motion of trees inthe laboratory

This paper extends our prior work on 3D tree modelingfrommotion capture [16] In this extended paper we general-ize the forces that guide particle flow The generalized forcescan be customized in the framework of forces to producedifferent kinds of 3D tree models from the same motioncapture data In addition we implement the convex hull as analternative approach to bounding all the particles andmarkerlocations on a tree This paper includes 3D models of treesbuilt using these extensions and the models are compared toprior work

The combination of motion capture with a particle flowmethod provides an innovative way of creating 3D treeshapes The resulting tree shapes are similar to the originaltrees as shown in Figure 1 and can be used to replay motionsimilar to the capturedmotionWith the flexibilities of tuningthe forces we can produce several tree shapes besides theoriginal shape Figure 2 shows various tree shapes generatedfrom the same set of motion capture data and demonstratesthe flexibility of our forces for guiding particle flow

2 Related Work

Our work is most closely related to prior work in treemodeling and applied motion capture Our purpose is toinvestigatemethods for creating tree shapes onwhichmotioncapture data can be replayed Compared to prior work inmodeling trees we introduce new equipment for collectingshape information of trees and improve the particle flowprocess by creating a new set of functions to computeattracting factors Specially the particle flow process fits wellwith the data collection process Compared to prior work inmotion capture we design a new data collection process fornonrigid bodies and reconstruct realistic 3D models out ofthe data

21 Tree Modeling L-systems [1] generate a treersquos branchingstructure using axioms and rules in a concurrent context-free rewriting system L-systems have been extended inmanywaysThe extensionmost relevant to our work is [3] in whichL-systems are enriched with partial differential equations andcan be parameterized to reconstruct the shape of a specifictree or plant However L-systems require understanding ofthe rewriting system and the according parametric systemsWith different rules and parameters the resulting shapes oftrees could greatly vary Most of the time L-systems build atree model from trunk to branches For our work we collectpositions of branch tips for describing tree shape Startingfrom these positions of branch tips back to the trunk it


(a) (b) (c)

























4 Data Processing

















Boundingbox

xy


xy



x

y


x

y


x

y





119878119901 minus 119878119905

10038161003816100381610038161003816


1198781

119901119910+ sdot sdot sdot + 119878

119899

119901119910

119899)

Wind 11986511990810038161003816100381610038161198651199081003816100381610038161003816








Nearest trunk point

x

y


B

A

C

Sr

St


A

A1B

C

B1

C1

AB





119871 = 120573 lowast10038171003817100381710038171003817119878119901 119878119905

10038171003817100381710038171003817 (1)



119894 =1205961 lowast 119894 + 1205962 lowast

119878119894 + 1205963 lowast 119894

3 (2)



119881 (119898) = 119881 (119898 minus 1) + 119871 lowast 119894 (3)




6 Adding Leaves


7 Results



































References



























RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










(a) (b) (c)

























4 Data Processing

















Boundingbox

xy


xy



x

y


x

y


x

y





119878119901 minus 119878119905

10038161003816100381610038161003816


1198781

119901119910+ sdot sdot sdot + 119878

119899

119901119910

119899)

Wind 11986511990810038161003816100381610038161198651199081003816100381610038161003816








Nearest trunk point

x

y


B

A

C

Sr

St


A

A1B

C

B1

C1

AB





119871 = 120573 lowast10038171003817100381710038171003817119878119901 119878119905

10038171003817100381710038171003817 (1)



119894 =1205961 lowast 119894 + 1205962 lowast

119878119894 + 1205963 lowast 119894

3 (2)



119881 (119898) = 119881 (119898 minus 1) + 119871 lowast 119894 (3)




6 Adding Leaves


7 Results



































References



























RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation

























4 Data Processing

















Boundingbox

xy


xy



x

y


x

y


x

y





119878119901 minus 119878119905

10038161003816100381610038161003816


1198781

119901119910+ sdot sdot sdot + 119878

119899

119901119910

119899)

Wind 11986511990810038161003816100381610038161198651199081003816100381610038161003816








Nearest trunk point

x

y


B

A

C

Sr

St


A

A1B

C

B1

C1

AB





119871 = 120573 lowast10038171003817100381710038171003817119878119901 119878119905

10038171003817100381710038171003817 (1)



119894 =1205961 lowast 119894 + 1205962 lowast

119878119894 + 1205963 lowast 119894

3 (2)



119881 (119898) = 119881 (119898 minus 1) + 119871 lowast 119894 (3)




6 Adding Leaves


7 Results



































References



























RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation











Boundingbox

xy


xy



x

y


x

y


x

y





119878119901 minus 119878119905

10038161003816100381610038161003816


1198781

119901119910+ sdot sdot sdot + 119878

119899

119901119910

119899)

Wind 11986511990810038161003816100381610038161198651199081003816100381610038161003816








Nearest trunk point

x

y


B

A

C

Sr

St


A

A1B

C

B1

C1

AB





119871 = 120573 lowast10038171003817100381710038171003817119878119901 119878119905

10038171003817100381710038171003817 (1)



119894 =1205961 lowast 119894 + 1205962 lowast

119878119894 + 1205963 lowast 119894

3 (2)



119881 (119898) = 119881 (119898 minus 1) + 119871 lowast 119894 (3)




6 Adding Leaves


7 Results



































References



























RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation












Nearest trunk point

x

y


B

A

C

Sr

St


A

A1B

C

B1

C1

AB





119871 = 120573 lowast10038171003817100381710038171003817119878119901 119878119905

10038171003817100381710038171003817 (1)



119894 =1205961 lowast 119894 + 1205962 lowast

119878119894 + 1205963 lowast 119894

3 (2)



119881 (119898) = 119881 (119898 minus 1) + 119871 lowast 119894 (3)




6 Adding Leaves


7 Results



































References



























RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation











6 Adding Leaves


7 Results



































References



























RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation































References



























RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation











RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation









Research Article Reconstructing 3D Tree Models Using...

Documents

Transcript of Research Article Reconstructing 3D Tree Models Using...