Artículo sobre XC publicado en la revista de Kitware.

7/31/2019 Artculo sobre XC publicado en la revista de Kitware.

1/24

S O F T W A R E D E V E L O P E R S Q U A R T E R L Y Isse 15 Oct 2

The Kitware Source contains articles related to the develop-ment o Kitware projects in addition to a myriad o sotware

updates, news and other content relevant to the open sourcecommunity. In this issue, Brad King, Marcus Hanwell and BillHoman discuss the DCVS tools Kitware is using to supportcollaborative, distributed development teams. Sophie Chenprovides a tutorial or creating your own VolView and V3Dplugins. David Gobbi provides an overview o the recentoverhaul to VTK's wrappers. David Cole introduces a new aeature in CDash which allows users to request project buildsdone on any client. And David DeMarle, Jonathan Woodringand James Ahrens explain the addition o adaptive stream-ing algorithms in ParaView and VTK.

Readers will also notice a new section in this issue called the"Community Spotlight" highlighting community memberswho use Kitware tools in their own development environ-

ments. Cyrus Aidun, Daniel Reasor, Jonathan Clausen andJingshu Wu share their experiences using ParaView to to sim-ulate thousands o deormable particles. And Luis Prez Tatodiscusses how his company, Iturribizia, utilizes VTK to solvestructural analysis problems. To be eatured as a CommunitySpotlight in the uture, email [email protected].

The Source is part o a suite o products and services oeredto assist developers in getting the most out o our open sourcetools. Project websites include links to ree resources suchas: mailing lists, documentation, FAQs and Wikis. Kitwaresupports its open source projects with textbooks, consultingservices, support contracts and training courses. For moreinormation, please visit our website at www.kitware.com.

Editors Note ........................................................................... 1

Recent Reeases ..................................................................... 1

Distrited Version Contro: The Ftre o Histor ............ 2

Insiht Tookit P-ins: VoView and V3D ....................... ... 6

VTK Wrapper Oveha 2010 .................................................. 9

The CDash "@Home" Cod ................................................ 12

Mti-Resotion Streamin in VTK and ParaView ........... 13

Commnit Spotiht .......................................................... 15

Kitware News ...................................................................... 20

ITKV4 DEVElOPMENTOver the last quarter, the ITK development team releasedthe rst two Alpha versions o ITKv4. ITKv4 is a major reactoring o the ITK toolkit that introduces improved wrappingor other languages, a modular architecture and revisions tomany o ITKs components. These two releases were intendedto perorm a general code clean up, dropping the tricks tosupport now-deunct compilers used in the past while pavingthe way or major reactoring activities to commence. The

next decade o ITK has begun.

Detais

One o the most signicant operational changes is that thesource code o ITK was moved to a Git repository and a newdevelopment workfow has been put in place in order tointegrate the teams that are collaborating in this new versiono the toolkit. The major changes introduced in these tworeleases are described below.

ITKv4-Apha-01

The ollowing compilers were deprecated: Borland 5.5, VisuaStudio 6.0 and 7.0, SGI CC, Sun CC 5.6, Metrowerks. Source

code that was intended solely to support these compilerswas also removed. The original Statistics Framework wasremoved and replaced with the one that was reactored in2007. Multi-threaded implementations o the image registration metrics, mean squares and Mattes mutual inormationreplaced those which were preexisting.

Source code marked as deprecated in previous releases waremoved, classes implemented to consolidate morphologywere integrated into the toolkit and the consistent computation o pixel-centered coordinates is now enorcedAdditionally, several CMake options were removed; in particular, those intended to alternate between the changesmade above and pre-existing code.

ITKv4-Apha-02

For Alpha version 2, the source code was processed usingUncrustiy to reormat the coding style, the openjpeg librarywas updated to openjpeg-v2 version rom July 2010 and the

jpeg library was updated rom version 6b to version 8b.

Additionally, GDCM (the library that provides DICOM supportin ITK) was updated to GDCM 2.0.16 and an ImageIO classspecialized on managing JPEG2000 les was added rom anInsight Journal paper contribution.

Details on these changes can be ound in the ITK Wiki. Pleasecheck itk.org and the Wiki or continuous updates on ITKv4


2/24

2

DISTRIbuTED VERSION CONTROl:THE FuTuRE OF HISTORy

Collaborative sotware development, an approach Kitwareand open source communities have used or many years, isnow a mainstream development model [1] [2]. Developerteams are oten distributed geographically and may evenwork or dierent organizations. Kitware requently teams

with its customers to develop sotware solutions. New tooland processes are becoming available to manage this collaborative development model. Distributed Version ControSystems (DVCS) are especially exciting. According to JoeSpolsky [3], "This is possibly the biggest advance in sotwaredevelopment technology in the ten years Ive been writingarticles here."

Over the last nine months at Kitware we have transitionedmost o our work to Git, a popular DVCS tool [10]. Althoughwe have not yet ully realized all o the benets, this tech-nology will address important issues. In particular, it will helpus urther engage our customers and the open source com-munity, and will improve our release process.

DVCS tools engage our customers and users as welcomeparticipants in our open source sotware developmentmodel. They acilitate code reviews, allow easy incorpora-tion o contributed modications and shorten release cyclesFurthermore, "social coding" sites such as GitHub [7] andGitorious [8] allow everyone to share their work outside thecentral repositories.

We are very excited to improve our release process usingDVCS capabilities. Our new process allows all developers totake part in release preparation by deciding what changesare release-ready as they are developed. A release-readyversion o the sotware grows in parallel with the develop-ment version. We avoid building up an inventory o partially

nished work in an otherwise releasable version. Thisstreamlines the release process and allows or stable andrequent releases.

A bRIEF HISTORy OF VERSION CONTROlPrior to 1986, two le locking version control systems whichstored the changes or a le existed: SCCS, developed at BelLabs, and RCS. I a developer wanted to change a le, he oshe had to issue the "edit" command on the le and, i noother developer was working on that le, she would get awritable version o the le to edit. Ater completing edits onthat le, the developer would check the changes in, and thiswould release the lock.

Having a history or each le was helpul and locking the

les made sense in order to prevent two or more developersrom making the same or conficting changes. However, inpractice this did not scale very well beyond a ew developers. When a developer let a le in edit mode and went onvacation or was away rom the oce, the edit locks wereorceully and "hackishly" taken by others on the team sothat work could continue. The hand merges that wouldhappen ater the developer returned were oten painul andtime consuming.

In 1986, the Concurrent Versions System (CVS) was created toaddress many o the shortcomings o previous version controsystems. CVS oered concurrent edits on les managed bya central repository using a client/server model. Changes to

VTK 5.6.1 RElEASEDKitware is pleased to announce the release o VTK 5.6.1, apatch release over 5.6.0 with a small set o improvements.

VTK now includes NetCDF 4.1.1 allowing users to take advan-tage o the latest enhancements to NetCDF. Additionally, theC++ version o the library is also included. A new le ormat,MPAS, has been added; it is a netcd-based reader or oceanand atmosphere datasets.

Charts have signicant improvements in axes layout and the

ability to display multiple data series (up to our) in the sameplot. Initial support or linked selection has been added tothe charts in VTK and ParaView. Support or alpha blendingin parallel coordinate plots has also been added.

We've cleaned up VTK's build and install rules so that develop-ers don't have to worry about 'rpath' anymore. Additionally,"make install" has been vastly improved on all platorms.

As always, we value your eedback. Please report any issueson the VTK mailing list or the VTK bug tracker.

PARAVIEW 3.8.1 RElEASEDKitware is pleased to announce the release o ParaView 3.8.1,a patch release over 3.8.0 with a small set o improvements.

ParaView now comes HDF5 1.8.5. This enables developersto easily create readers and plugins or ParaView that usenewer versions o HDF5. We are now including the C++ andthe high level (hl) versions o the library or those who wishto use their newer API.

Among the improvements are two new le ormats. Therst is a netcd based reader or MPAS ormat ocean andatmosphere datasets. The second, available in source ormonly, is a reader plugin that allows ParaView to read multi-resolution wavelet compressed VAPOR data ormat les.Thanks to John Clyne and Dan Legreca or contributing theVAPOR plugin.

We have improved the support or animation scriptingthrough Python. The new API is more coherent with the resto the ParaView Python API, avoiding the need or scriptwriters to know minute details regarding how to createanimations using proxies. Tracing capabilies or animationwere also revised.

The GPU volume ray cast mapper or voxel datasets nowworks in tile display mode. In keeping with our ongoingimprovements o ParaView's charting capabilities, we haveincluded a ew xes regarding labels and axes placements.

ParaView build and install rules have been cleaned up sodevelopers no longer have to worry about 'rpath' and "makeinstall" has been improved on all platorms.

This release includes a material interace extraction lterthat takes a cell data volume raction and generates a poly-data surace. It also perorms connectivity on the particlesand generates a particle index as part o the cell data othe output. The lter computes the volume o each particlerom the volume raction and is accessible rom the MaterialAnalysis category in the Filters menu.

As always, we rely on your eedback to make ParaViewbetter. Please use http://paraview.uservoice.com/ or click onthe "Tell us what you think" link on paraview.org to leaveyour eedback and vote or new eatures.


3/24

3

les were integrated together during an update o a check-out prior to a commit to the repository by the last person tocommit his/her changes on that le. This encouraged prema-ture commits that oten orced unnished changes on thewhole team. However, the model was certainly much betterthan the le-locking days o SCCS and RCS.

When CVS rst came out it was a hard sell to the develop-ment community. Developers elt secure with systems basedon le locks. The idea that two people could work on onele at the same time and that changes would be integrated

automatically was "just plain crazy". However, over timeCVS became the accepted norm or sotware development.In 2000, the Subversion project (SVN) was created to replaceCVS. SVN provided atomic whole-tree commits, versionedle properties and a more solid implementation o CVSscentralized repository model.

A new model or version control, Distributed Version ControlSystems (DVCS), is now unseating centralized systems as thestandard or sotware development. These systems oerconcurrent repositories, network-ree access to local-diskrepositories (Figure 1) and they enable new collaborationmodels with non-linear history and multiple shared reposi-tories (Figures 2 and 5). In this article we describe the power

o DVCS as a version control system and explain how we areusing it to improve our collaborative development model.

Figure 1 Access Inormation at All Times

NOTATION & TERMINOlOgyThis article uses conventions and terminology rom Git, butthe discussion applies to DVCS in general. Our gures denotehistory using a directed acyclic graph as shown in Figure 2.Nodes (circles) represent versions and directed edges pointat prior versions. The subgraph reachable rom any givenversion represents its history.

Figure 2 Visualizing History

COllAbORATION TASKSWe divide collaborative development into three basic versioncontrol tasks:

Create: Record a new version with meta-data describingthe changes;

Share: Publish new versions or other developers to see;

Integrate: Combine changes rom multiple versions intoanother new version.

The ollowing reviews each task in more detail.

Create

Figure 3 shows the basic workfow we each ollow to createnew versions. First, we checkouta version rom the repository on which to base changes. Then, we editthe content inour work tree to produce a new version. Finally, we commithe new version to the repository. All version control systemprovide this basic workfow by denition.

Figure 3 Basic Workow

Share

We share versions through repositories. Figure 4 shows thetraditional model in which each developer has only a worktree. The checkout and commit operations share all versionsthrough a single repository.

Figure 4 Single Centralized Repository

Figure 5 shows the distributed model in which each developer has a private repository and work tree. The checkoutand commit operations occur locally. Sharing occurs arbi-trarily among repositories through separate operations suchaspush, etch, andpull.


4/24

4

and how releases are prepared and maintained. Traditionaversion control systems severely limit possible workfows byinseparably combining all three version control tasks undethe commit operation in a single repository. Distributedversion control systems provide separate operations or thethree tasks and thus support many workfows [4].

Rease v. Mere Workfows

No matter who creates changes or where they are sharedthey eventually need to be integrated together. At this point

workfows are distinguished by their approach to integrationeither rebase or merge. Figure 7 illustrates both approacheswith a series o commits belonging to three topics: eaturea, eature b, and a bug fx. It also includes the creation andmaintenance o a release branch.

Figure 7 Collaboration Workow by Rebase or Merge

In the rebase workfow, each commit is rebased on top owhatever happens to be the latest version at the time it ispublished; there is no record o the original base on which itwas developed. The commits rom all topics are intermixedwith one another in history. The history o the release branchhas no indication that its second commit is a copy o the bugx changes.

In the merge workfow, each commit is preserved on itsoriginal base. The commits belonging to each topic appeacontiguously in history. There is an explicit merge commirecording the integration o each topic. The second commit

on the release branch explicitly merges the bug x topic.

Traditional version control systems automatically rebaseevery commit, and thereore support only the rebase work-fow. Distributed version control systems support both rebaseand merge workfows.

A "branch" Workfow

In the past, when using a central repository with a singledevelopment branch, the authors have seen work come toa halt when some bad code was committed. No other workcould continue until the issue was xed. With the use o DVCSdevelopers can start new work rom a stable working codebase and no longer base new work on a moving target.

Figure 5 Concurrent Distributed Repositories

Interate

Figure 6 shows a case in which work has divergedbecausetwo developers independently created versions C and Dbased on version B. Assume version D has been published andwe must integrate its changes with those made in version Cto produce a new version.

Figure 6 Integrate by Rebase or Merge

The gure illustrates two approaches. One wherein we iden-tiy the changes made by C, originally based on B, and rebasethem on D to create a new version C. And another wherewe merge versions Cand D together into a new version, M,that combines their changes.

Both approaches integrate the same changes into a singlenew version, either C or M, but record dierent historybehind said version.

In traditional version control systems the commit operationautomatically rebases changes in new versions on the latestversion in the repository. I the rebase ails it asks the userto update the work tree and integrate the changes beorecommitting. In distributed version control systems, rebaseand merge are explicit operations separate rom commit.

COllAbORATION WORKFlOWSAll developer communities establish a workow involvingthese three version control tasks in order to collaborativelydevelop and maintain their projects. A workfow determineswhen new eatures are introduced, how bugs are xed


5/24

5

The use o merge commits to integrate work providesgreater fexibility and motivates the use o a "branchy"workfow with DVCS tools [5]. This workfow denes twotypes o branches: topic and integration. Topic branchescontain a contiguous sequence o commits developing a spe-cic eature or bug x. Integration branches contain mergecommits to integrate topic branches together.

In the bottom o Figure 7 each o eature a, eature b, andbug ixis a topic branch, and masterand release are integra-tion branches. Integration branch heads are published in a

designated ocial repository. Topic branch heads are notnamed explicitly in the ocial repository, but appear in thehistory o integration branches that merge them.

Each integration branch has a specic purpose such asmaintenance o a current release (typically called maintorrelease), preparation o a uture release (typically calledmaster) or bleeding-edge development (typically callednext, as in the next topics to evaluate). Each topic branchstarts rom the most stable integration branch to which itwill be merged. Typically this is masteror new eatures andrelease or bug xes.

Figure 8 Multiple Integration Branches

Figure 8 shows the use o two integration branches, masterand next, while developing a topic branch, my-topic. Thehead o master is commit(0) when the topic starts. Commits(1) and (2) develop the topic. Merge commit (3) integratesthe topic into nextwhich is then published or evaluationand testing by others. Later, when the topic is deemedstable, merge commit (4) integrates the topic into masteror publication.

Throughout this workfow the masterbranch remains stablebecause topics are merged into it only ater they have beenevaluated on next. Since new topics are started rom masterthey have a stable base version on which to work regardlesso whether unstable topics have been merged to next. A

new stable topic may be merged back to masterat any timeindependent o other (unstable) topics.

MANAgINg RElEASESRelease management has two parts: preparing a new releaseand maintaining an existing release. In the past, Kitwareplaced responsibility or both parts on a release manager.We prepared new releases using a "reeze" period duringwhich the release manager had exclusive commit access tostabilize the trunk, oten by cleaning up unnished work,beore creating the release. The release manager then main-tained the release branch by manually copying bug xes romthe development trunk. With this approach, developmentstalled during reeze periods and maintenance o releases

became increasingly burdensome on the release manager athe trunk diverged over time.

Kitware's new release process is based on the DVCS branchyworkfow in Figure 8. New topics are merged to next oevaluation and testing and only stable topics are merged intomaster, keeping it release-ready at all times. This approachamortizes the cost o release preparation over the develop-ment cycle, distributes the workload to all developers andseparates release scheduling rom eature development.

We now manage releases as shown in Figure 9 (or simplicitywe omit nextrom the gure but we use it to test changesbeore merging to master).

Figure 9 Release Maintenance

The release manager tags a new release directly on maste(0) and development proceeds normally. A developer starts abug-x topic (1) rom the released version and merges it intomaster (2), making the "release+x" version (1) availablewithout any action by the release manager. Then the releasemanager merges the x into the release branch (3), tags apatch release, and merges release into master (4), makingthe tag reachable. The process repeats with another bug-fx(5 and 6), merge to master(7), merge to release (8), tag andmerge o release into master(9).

This maintenance approach relieves the release manager o

all tasks except trivial merge and tag operations. Release tagsare always reachable rom masterso there is no need or aseparate named branch or every new release. DVCS allowsdevelopers to commit bug xes directly on the releases thatneed them.

CONCluSIONSome DVCS concepts may seem overly complex and irrel-evant to many developers, but are invaluable to others. Notonly are they here to stay, but they'll improve productivityand reduce waste in the sotware development process. Aswith the advent o centralized concurrent version systems, aperiod o education and exploration is required to ully takeadvantage o this new technology.

Kitware has standardized on Git, our DVCS o choiceAlthough powerul, some consider Git to be complicatedto learn. It was even introduced at the now-amous GoogleTech Talk by Linus Torvalds as "expressly designed to makeyou eel less intelligent than you thought you were" [9]. Weencourage the reader to stay the course and hang in there asthe benets really are numerous.

Kitware is not alone in embracing DVCS. There are manyother sotware companies exploring the eatures o DVCSFor example, we are investigating use o Gerrit, a code reviewtool developed by Google engineers to acilitate Androiddevelopment [6]. It provides tight integration with Git anda web interace or perorming online code reviews. The


6/24

6

INSIgHT TOOlKIT Plug-INS:VOlVIEW AND V3D

VolView and V3D are applications or visualization andanalysis o three-dimensional images. They both have toolswhich allow users to lter and analyze image data. The twoapplications serve two dierent niches: VolView was createdwith radiologists in mind, while V3D caters primarily tomicroscopists. However, a powerul part o both biomedical imaging tools is support or user dened extensions viacustom plug-ins. This support allows users to extend howinput data is ltered. This quick guide will help you getstarted with your own VolView and/or V3D plug-in.

Sotware applications such as Slicer, SNAP, Analyze, VolViewSCIRun, GoFigure and V3D use ITK lters as plug-ins to adddesirable additional capability to their image analysis application o choice, thus removing the need to rewrite existingalgorithms or each new piece o sotware while stampingout the hassles o requesting usage-permission.

ITKs qualications or use in scientic research makes itimportant or developers to make the most o ITKs imaging

tools and oer tailored combinations to those who desirethem. The ollowing table compares some o the main ea-tures o VolView [1] and V3D [2].

Creator

VoView

Kitware, Inc.

Windows XP, VistaMacLinux

3D image analysis ormedical, scientifc andengineering researchcommunities. Interactively

render, reormat, annotate,measure, animate andcapture/print volumetricand image data.

http://www.kitware.com/products/volviewdownload.html

V3D

Howard Hughes MedicalInstitute: Janelia Farm ResearchCampus, Hanchuan Peng, et al.

Windows XP, VistaMacLinux

3D image analysis such as cellsegmentation, neuron tracing,brain registration, annotation,quantitative measurement, data

management and more.

http://penglab.janelia.org/proj/v3d

Patorms

Intent

Downoad

Strctre o P-ins

Typically, a plug-in or V3D and VolView consists o sourcecode compiled and packaged as a shared library. The plug-inname determines the name o the shared library used ordeployment as well as the name o the plug-in initializationunction. The shared library is copied into a specic directoryo the VolView or V3D binary installation. No libraries rom

VolView or V3D are required in the process. Plug-in devel-opers only need a set o source code headers dening theplug-in API oered by the application. This is essentially theset o data structures and unction calls by which the plug-incommunicates with the application.

A V3D Plug-INThe development o ITK plug-ins or V3D serves two purposes: 1) exposing ITK unctionalities to researches whoanalyze microscopy data and 2) uncovering the areas inwhich ITK requires improvements in order to better servethe microscopy community. ITK lter plug-ins were added toV3D via a collaborative eort between Kitware and JaneliaFarm (HHMI).

ITK project is using it experimentally, and several develop-ers are evaluating it internally. Gerrit can combine a humanelement with automated testing and multiple integrationbranches to provide us with a very eective workfow tocollaboratively develop complex sotware with contributorsrom around the world.

Other online tools such as Gitorious and GitHub are alreadybeing used by developers outside Kitware to develop bugxes and new eatures or Kitware-hosted projects like VTK,ParaView, ITK and CMake.

We hope to have armed you with some basic concepts andterminology associated with DVCS, while presenting someo the exciting new workfows that DVCS makes possible.Kitware is very excited about this technology and is lookingorward to reaping the ull benets.

To our customers and co-developers: we look orward toreleasing our work requently and incorporating your con-tributions quickly and reliably.

REFERENCES[1] http://en.wikipedia.org/wiki/Collaborative_sotware_devel-

opment_model

[2] http://www-01.ibm.com/sotware/ino/eatures/collabora-tion/main.html

[3] "Distributed Version Control is here to stay, baby", Joel onSotware, March 17, 2010

http://joelonsotware.com/items/2010/03/17.html

[4] "Distributed Workfows", Pro Git, ch. 5, http://progit.org/book/ch5-1.html

[5] "Git help workfows",

http://www.kernel.org/pub/sotware/scm/git/docs/gitwork-fows.html

[6] "Gerrit Code Review", http://code.google.com/p/gerrit/

[7] "GitHub Social Coding", http://github.com/

[8] "About Gitorious", http://gitorious.org/about

[9] "Linus Torvalds on git", Google Tech Talk, May 3, 2007,

https://git.wiki.kernel.org/index.php/LinusTalk200705Transcript

[10] "Git the ast version control system", http://git-scm.com/

Marcus Hanwell is an R&D engineer in thescientifc visualization team at Kitware, Inc.He joined the company in October 2009, andhas a background in open source, Physicsand Chemistry. He spends most o his timeworking with Sandia on VTK, Titan andParaView.

Bill Hoffman is currently Vice President andCTO or Kitware, Inc. He is a ounder o

Kitware, a lead architect o the CMake cross-platorm build system and is involved in thedevelopment o the Kitware QualitySotware Process and CDash, the sotwaretesting server.

Brad King is a technical developer in KitwaresCliton Park, NY ofce. He led Kitware'stransition to distributed version control,converted many o our project histories toGit, and conducted training sessions.


7/24

7

The simplest way to implement a plug-in is to copy and pastean existing V3D plug-in and modiy two methods. Moreadvanced plug-ins, typically those requiring more than onelter, may need to be modied urther. For our V3D plug-inexample, we will use the existing itkSigmoidImageFilter,which can be ound under the Intensity Transormationdirectory to create another plug-in, itkLogImageFilter. ForV3D, the V3DPluginCallback is used to get data structuresand callbacks.

Create a V3D P-inCopy and paste existing plug-in header and source lesto the binary directory where plug-ins are set up in yoursystem. An example path is: src/ITK-V3D-Plugins/Source/IntensityTransormations. Change the le names to corre-spond to the goal image lter.

Then nd instances in les where lter reerences and namesought to be replaced. In the SetupParameters section, adjustyour lters parameters, or i the section does not exist, reerto the SetUpParameters() below. I you're uncertain about

a deault value use your best judgment or communicatewith an appropriate end-user or a general value. The valuechosen should refect some noticeable changes in an imageupon testing.

void Execute(const QString &menu_name, Qwidget 50 *parent){this->Compute();}

virtual void ComputeOneRegion(){

this->m_Filter->SetInput( this->GetInput3DImage() );

if( !this->ShouldGenerateNewWindow() ){}

this->m_Filter->Update();}

virtual void SetupParameters(){//// These values should actually be provided by// the Qt Dialog...// just search the respective .h file for the// itkSetMacro for parametersthis->m_Filter->SetFullyConnected( true );

this->m_Filter->SetBackgroundValue( 0 );this->m_Filter->SetForegroundValue( 100);this->m_Filter->SetNumberOfObjects( 3 );this->m_Filter->SetReverseOrdering( false );this->m_Filter->SetAttribute( 0 );}

A VOlVIEW Plug-INFor this example, a plug-in named vvITKGradientMagnitudewill be deployed in a shared library: libvvITKGradientMag

nitude.so in Unix/OsX and vITKGradientMagnitude.dll onMS Windows. The initialization unction is vvITKGradientMagnitudeInit(). The result o the example will be animplementation o a simple ITK based lter with only oneGUI parameter. The example may be adapted to most othetoolkits or C/C++ implementation.

Given the similar structure o V3D, the directions in thisVolView example should be generic enough to be applied toa V3D plug-in with the respective style/naming dierences.

Communication between the plug-in and the application isacilitated by a public header le that denes the data andGUI structures. The plug-in developer simply implements themethods that are dened within the header le.

Initiaiation nction

A plug-ins initialization unction must conorm to a particular API. For our particular example, this would be:

extern "C"{

void VV_PLUGIN_EXPORT vvITKGradientMagnitudeInit(vtkVVPluginInfo *info)

{}

}

where the symbol VV_PLUGIN_EXPORT and the structurevtkVVPluginIno are dened in the public header le

vtkVVPluginAPI.h. This initialization unction will be invokedby VolView at start-up -- ater the shared library has beendynamically loaded.

Content

Below is the typical content o the public header le,vtkVVPluginAPI.h.

Call macro vvPluginVersionCheck() to veriy the plug-in/APconorms to current version o VolView's binary distributionA plug-in cannot be executed i the versions do not matchand VolView displays an error message at run-time to indi-cate this when necessary.

vvPluginVersionCheck();

Inormation Structure is initialized. Setup Inormation doesnot change.

// Setup Information

ProcessData is set to the pointer o the unction that wilperorm the computation on the input data. This allowsor reedom in the implementation o the unction. This isurther covered in the next section.

info->ProcessData = ProcessData;


8/24

8

completing the plug-in processing and or determining i theundo inormation can be kept. Note that this estimate is notbased on the size o the nal dataset produced as output,but on the total amount o memory required or intermediate processing. In other words, it should provide the peak omemory consumption during the plug-in execution.

The ProcessData() Fnction

The ProcessData() unction perorms the lter computationon the data. The unction signature o ProcessData() is:

static int ProcessData( void *inf,vtkVVProcessDataStruct *pds)

where the rst argument is a pointer to a vtkVVPluginInostructure which can be downcast to a vtkVVPluginInopointer using:

vtkVVPluginInfo *info = (vtkVVPluginInfo *) inf;

In this assignment, the right hand side is a structure, vtkVVProcessDataStruct, that carries inormation on the data set tobe processed. This inormation includes: the actual buer ovoxel data, the number o voxels along each dimension inspace, the voxel spacing and the voxel type.

The vtkVVProcessDataStruct also contains the membersinData and outData, which are pointers to input and outputdata sets, respectively. ProcessData() extracts the data romthe inData pointer, processes it, and stores the nal results inthe outData buer.

ProcessData() Startin Code

The typical ProcessData() starting code o this unctionextracts meta inormation about the data set rom thevtkVVProcessDataStruct and vtkVVPluginIno structuresFor example, the ollowing code shows how to extract thedimensions and spacing o the data.

First, set up a data structure.

SizeType size;IndexType start;double origin[3];double spacing[3];

size[0] = info->InputVolumeDimensions[0];size[1] = info->InputVolumeDimensions[1];size[2] = pds->NumberOfSlicesToProcess;

for( unsigned int i=0; iInputVolumeOrigin[i];spacing[i] = info->InputVolumeSpacing[i];start[i] = 0;}

Image data can be imported into an ITK image using theitkImportImageFilter.

RegionType region;region.SetIndex( start );region.SetSize( size );m_ImportFilter->SetSpacing( spacing );m_ImportFilter->SetOrigin( origin );m_ImportFilter->SetRegion( region );m_ImportFilter->SetImportPointer( pds->inData,totalNumberOfPixels, false );

The output o the import lter is then connected as theinput o the ITK data pipeline and the pipeline execution istriggered by calling Update() on the last lter.

Similarly, UpdateGUI is also set to a unction pointer.

info->UpdateGUI = UpdateGUI;

SetProperty() is used to dene general properties o theplug-in some o these properties are simply inormativetext that is displayed on the GUI (i.e. textual name o theplug-in, terse and extended documentation). Properties areidentied by tags to urther enorce the decoupling betweenthe internal representation o inormation in VolView and

the structure o code in the plug-in. Other non-GUI proper-ties are also set with this method.

//Setup Information - SetProperty()

The tag VVP_NAME species that the string being passed asthird argument o the SetProperty() method should be usedor the text label o the plug-in in the GUI. VVP_GROUPspecies the grouping o the lter within the plug-in menu,and VVP_TERSE_DOCUMENTATION provides a short descrip-tion o the plug-in.

info->SetProperty( info, VVP_NAME, "GradientMagnitude IIR (ITK)");

info->SetProperty( info, VVP_GROUP, "Utility");

info->SetProperty( info, VVP_TERSE_DOCUMENTATION,"Gradient Magnitude GaussianIIR");

The tag VVP_FULL_DOCUMENTATION species the completedescription string.

info->SetProperty( info, VVP_FULL_DOCUMENTATION,"This filter applies IIR filters to compute theequivalent of convolving the input image with theerivatives of a Gaussian kernel and then computingthe magnitude of the resulting gradient.");

Other tags are used to speciy:Whether this lter is can perorm in-place processing;

info->SetProperty( info,VVP_SUPPORTS_IN_PLACE_PROCESSING, "0");

Whether this lter supports data streaming (processing inchunks);

info->SetProperty( info,VVP_SUPPORTS_PROCESSING_PIECES, "0");

And other inormation about the lter implementation.

info->SetProperty( info, VVP_NUMBER_OF_GUI_ITEMS,"1");

info->SetProperty( info, VVP_REQUIRED_Z_OVERLAP,"0");

Memory consumption is an important consideration or pro-cessing. By providing an estimated number o bytes to beused per voxel o the input dataset:

info-> SetProperty( info,VVP_PER_VOXEL_MEMORY_REQUIRED, "8");

Memory consumption can be estimated. VolView will usethis actor to ensure that the system has enough memory or


9/24

9

m_FilterA->SetInput( m_ImportFilter->GetOutput() );m_FilterB->SetInput( m_FilterA->GetOutput() );m_FilterC->SetInput( m_FilterB->GetOutput() );m_FilterD->SetInput( m_FilterC->GetOutput() );m_FilterD->Update();

Finally the output data can be copied into the pointer pro-vided by VolView. This is typically done using an ITK imageiterator that will visit all the voxels.

outputImage = m_Filter->GetOutput();

typedef itk::ImageRegionConstIterator< OutputImageType > OutputIteratorType;OutputIteratorType ot( outputImage,outputImage->GetBufferedRegion() );

OutputPixelType * outData =static_cast< OutputPixelType * >( pds->outData );

ot.GoToBegin();while( !ot.IsAtEnd() ){*outData = ot.Get();++ot;++outData;}

When memory consumption is critical, it is more convenientto actually connect the output memory buer provided by

VolView to the output image o the last lter in the ITKpipeline. This can be done by invoking the ollowing lines ocode beore executing the pipeline.

m_FilterD->GetOutput()->SetRegions(region);m_FilterD->GetOutput()->GetPixelContainer()->SetImportPointer(static_cast< OutputPixelType * >( pds->outData ),totalNumberOfPixels, false);

m_Filter->GetOutput()->Allocate( );

The current distribution o ITK provides support or notrewriting this same code or each new plug-in. A templatedclass containing this code is available in the current distribu-tion o ITK in the InsightApplications module. New plug-ins

only need to dene their own ITK pipelines and invoke themethods o the base class in the appropriate order.

Rereshin the guI

Ater the source code has been packaged into a sharedlibrary, it should be deposited into the plug-ins directory:VolView 3.2/bin/Plugins. In order or the plug-in to load,the GUI needs to be rereshed by re-scanning all plug-ins. Acircular arrow next to the lters selection menu will rereshthe lters list.

Image processing algorithms can take considerable time toexecute on 3D data sets. It is important to provide user eed-back as to how the processing is progressing and to allowthe user to cancel an operation i the total execution timeis excessively long. Calling the UpdateProgress() unction othe vtkVVPluginIno structure rom within the ProcessData()unction accomplishes this:

float progress = 0.5; // 50% progressinfo->UpdateProgress( info, progress,"half data set processed");

This unction provides eedback to the VolView GUI allow-ing VolView to update the progress bar and set the statusbar message. The requency with which the UpdateFunctionshould be called should be well balanced. I it is invokedtoo oten, it will negatively impact the perormance o theplug-in -- a considerable amount o time will be spent in GUI

rereshing. I it is not called oten enough, it may producethe impression that the processing is ailing and that theapplication is no longer responding to user commands.

A detailed skeleton plug-in and a more contextual version othis guide can be ound starting on page 45 in the VolViewUsers Manual, available at kitware.com/volview.

REFERENCES[1] Download VolView rom: kitware.com/volview

[2] Download V3D rom: http://penglab.janelia.org/proj/v3d

Sophie Chen recently completed her secondsummer as a Kitware intern where sheworked under Luis Ibez, Wes Turner andHarvey Cline on programming algorithms,ITK and VTK. Sophie is a senior at RPI where

she is working toward an IT degree inManaging Inormation Systems.

VTK WRAPPER OVERHAul 2010

The VTK wrappers received their last major overhaul in1998 and have done well to serve the VTK community. Inrecent years, however, the wrappers have started to showsome cracks, particularly as new types such as vtkVariantand vtkUnicodeString have been introduced to VTK buthave not been made available in the wrappers. One reasonor the wrappers slow development compared to the resto VTK is undoubtedly the intimidation actor o thewrapper-generator code, which is very complex and lackingin documentation.

VTK wrappers were recently overhauled again. The ourmain goals o overhaul project were as ollows:

Cleaning up the wrapper-generator code by removinghard-coded hexadecimal constants, reducing the use oglobal variables, and improving code documentation;

Proper wrapping o vtkStdString, which is a crucial interace type that is only partly wrapped;

Wrapping vtkVariant and other new VTK types in Python;

And eliminating the need or BTX/ETX unwrappablesection markers in the code.

The overarching goal is to provide a new oundation orthe wrapper-generators that will make it easier to movethe wrappers orward.These changes have been made whilemaintaining backwards compatibility with existing TclPython, and Java programs which use VTK. The old wrappergenerator code has not been replaced; it has only beencleaned up and enhanced.

WRAPPER PRIMERTo provide some background, the wrapper-generator codeconsists o a ront-end parser that reads the VTK headerles and stores the class declarations in simple data structures, and three back-ends that read those data structuresand generate the wrapper code or each o the wrapperlanguages. Most o this project has ocused on the ront endbut enhancements have been added to the back end as wellparticularly the Python back end.

The parser ront-end can be urther subdivided into a lextokenizer (which also does rudimentary preprocessing) and


10/24

10

The new parser also eatures a preprocessor, something thatwas conspicuously absent beore. The preprocessor storesdened macros and provides conditional parsing based on#i directives, eliminating yet another previous use o theBTX/ETX markers. Unlike a traditional preprocessor, theparser stores macros but does not expand them. This is bydesign, since several VTK macros have special meaning to thewrappers that would be lost i those macros were expandedThe parser can query the macros to get their value.

HIERARCHIES FOR AllA undamental addition or the new wrappers is a collectiono hierarchy les, one per source directory, that list the ulgenealogy o all the VTK classes in each directory. The levtkCommonHierarchy.txt, or example, lists all classes denedin the Common directory. The le structure is simple:

vtkArray : vtkObject ; vtkArray.h ; ABSTRACTvtkArraySort ; vtkArraySort.h ; WRAP_EXCLUDE

The name o the class comes rst, ollowed by any super-classes, then the header le, and nally any o the CMakefags WRAP_EXCLUDE, WRAP_SPECIAL or ABSTRACT whichapply to the class. Classes that have a name that is dierentrom the name o their header le are automatically labelledwith VTK_WRAP_EXCLUDE. Note that the le ormat mightchange in the uture, so anyone who is interested in usingthis le should always use the unctions dened in VTK/Wrapping/vtkParseHierarchy.h to read the le, instead owriting their own code to do so.

In addition to classes, the hierarchy les also include altypedes and enum types encountered in the header lesin order to provide a comprehensive list o all types denedin VTK. The rationale behind the hierarchy les is as ollowspreviously, the wrappers would assume that any type witha vtk prex was derived rom vtkObjectBase, exceptingtypes like vtkIdType that were specically caught by theparser. This is the other reason that BTX/ETX had to be used

so oten (in addition to the aorementioned limitations othe parser), since methods that used new types like vtkVariant or vtkUnicodeString had to be BTX'd because these typewere misidentied by the wrappers. By using the hierarchyles, the wrappers can identiy any type dened in VTK.

WRAPPER COMMAND lINE ARguMENTSThe command line or the wrapper-generators has beenmodied, and this will make it easier to invoke the wrapperby hand. The old calling convention still works, in order tosupport older CMake scripts. The new command line is aollows:

vtkWrapPython [options] input_file output_file

--concrete tell wrappers that class is concrete--abstract tell wrappers that class is abstract--vtkobject class is derived from vtkObjectBase--special class not derived from vtkObjectBase--hintsspecify a hints file--typesspecify a hierarchy file-I add an include directory-D define a preprocessor macro

All o these arguments are optional. For instance, thewrapper-generators will automatically guess that any classwith pure virtual methods is an abstract class. However, theconcrete option is needed in cases where an abstract classis being wrapped that will be replaced by a concrete actorygenerated class at run time.

a yacc parser that understands the C++ grammar. Thesetwo pieces are the oundation o the wrappers, or less gen-erously, they are the bottleneck. The wrappers are only ableto wrap the class and method denitions the parser can pullrom the header les. Because o the parser's importance,it has received more attention during this update than anyother part o the wrappers.

An important eature o the VTK wrappers is that they wrapthe VTK classes one class at a time using only the header leor that class, with a minimal amount o hinting. When com-

bined with CMake, this approach is easily scalable to a verylarge number o classes in dierent directories or even indierent packages. The new wrappers urther enhance thisapproach by automatically generating hierarchy les thatdescribe all types dened in any particular VTK source direc-tory. These les are discussed in detail later in this article.

A SHINy NEW PARSERThe new parser is a signicant improvement to the oldparser code. The original code took a minimalist approach toparsing the VTK header les. It looked or the rst VTK classdened in the header, and then extracted only the methodsdened in that class. The rest o the le would be ignored,but since the parser lacked a ull C++ grammar, it was not

always successul in skipping over parts it was supposed toignore. These troublesome patches o code had to be sur-rounded by BTX/ETX exclusion markers so that they could beremoved at the preprocessing stage.

The new parser code reverses this minimalist approach: itreads all the declarations in the header le and stores themall or use in the wrappers. This means that typedes, tem-plates, constant denitions, enum types, operator overloadsand namespaces are all available to the wrapper-generators.Each piece o inormation rom the header le is stored in aC struct, with the most-used struct being the ValueIno structthat is used or method arguments, variables, constants andtypedes. The ollowing is rom the vtkParse.h header le:

struct ValueInfo{parse_item_t ItemType; /* var, typedef, etc */parse_access_t Access; /* public, private, etc. */const char *Name;const char *Comment;const char *Value; /* value or default val */unsigned int Type; /* see vtkParseType.h */const char *Class; /* type as a string */int Count; /* for arrays */int NumberOfDimensions;const char **Dimensions;FunctionInfo *Function; /* for function ptrs */int IsStatic;/* class variables only*/int IsEnum; /* for constants only */

};

One should note that in contrast to the old parser, arrayscan now be multi-dimensional. The dimensions are stored asstrings, in case the dimensions are symbolic values (e.g., tem-plate parameters) that cannot be evaluated until compiletime. The product o the dimensions is stored as an int iall the dimensions are integer literals.

Similar structs provide inormation or unctions, classes,namespaces, templates, etcetera. The FunctionIno andClassIno structs have backward compatibility sections thatprovide their ino in the old wrapper ormat, so that wrap-pers-generators based on the old structs can easily be madeto work with the new parser.


11/24

11

The --hints option provides a way o speciying the hintsle that has been used by the wrappers since day one. Thehints le is the only part o the wrappers that still useshexadecimal literals to describe types. Support or it will bemaintained indenitely to ensure backwards compatibility.The new --types option gives the wrappers access to thenew hierarchy les that were described above.

The -I and -D options are orwarded to the preprocessor,so that i the le being wrapped includes important headerles like vtkCongure.h, the preprocessor can read those

les and use the macro values dened within them.

STRINgS AND uNICODEThe vtkStdString type was introduced several years ago,and when it was added to the wrappers, it was wrapped byidentiying it as const char * in the parser, with its c_str()method used to do the conversion. This caused problems,because most VTK methods return vtkStdString by value,resulting in the creation o a temporary string object on thestack, or which the char pointer returned by c_str() is like-wise only temporarily valid. Because o this, only methodsthat returned vtkStdString by reerence could saely bewrapped, and methods that passed vtkStdString by valuewere blocked with BTX/ETX. One o the reasons that vtk-StdString was wrapped this way is that the old parser hadonly 15 slots available to identiy undamental VTK types,compared to nearly 255 slots or the new parser.

The new Tcl, Python and Java wrappers have all beenmodied so that they correctly identiy vtkStdString as astd::string subclass and wrap it appropriately, allowing theBTX/ETX markers to be removed rom methods that pass vtk-StdString by value. The vtkUnicodeString type has also beenadded to the parser as a new type, and the Python wrappershave been modied to transparently convert between vtkU-nicodeString and Python's own unicode type.

WRAPPINg VARIANTS IN PyTHON

There are two approaches that could have been used towrap vtkVariant. The rst approach would have been tohave the Python wrappers implicitly convert Python types toand rom vtkVariant, so that i a C++ VTK method returneda variant, the wrapped Python method would automaticallyextract and return the value stored in the variant. This wouldhave been convenient, but also would have resulted in losso inormation. For example, any integer type betweenunsigned char and long would automatically be con-verted into a Python integer, and Python users would notbe able to discover the original C++ type. For this reason, anapproach in which vtkVariant is wrapped as its own distinctPython type was used instead.

The technique used to wrap vtkVariant is similar to thetechnique used to wrap vtkObjectBase-derived objects.The main dierences are the way memory managementis done and the way that constructors are handled. UnlikevtkObjects, the vtkVariant is not reerence-counted, so i avariant is passed as a method argument, it is copied just asi it was an int or any other basic type. Wrapping the manyconstructors or vtkVariant was a challenge, because the oldPython wrapper code or resolving overloaded methods wasinadequate or this task: it would simply try each overloadin turn until one o the overloads could be called withoutgenerating an argument type error. Hence, calling vtkVari-ant(10) rom Python would create an unsigned charvariant since vtkVariant(unsigned char) is the rst construc-

tor in the header le. For this wrapper update project, codewas added to the Python wrappers so that they compare alpassed arguments against those o the various method signatures, and call the overload that provides the best matchThis new code is used or all Python method calls, not just orconstructors, so there might be some small backwards compatibility problems since the old code always called the rstmatched method, while the new code calls the best match.

Another eature that was added to the Python wrappers isautomatic argument conversion via constructors. In C++, i

a method requires a vtkVariant argument, but is passed asan int, the vtkVariant(int) constructor will automaticallybe called to convert the int. No such automatic conver-sion exists in Python; it is instead the responsibility o themethod writer to have it explicitly convert the argumentsFortunately, since all the VTK/Python methods are gener-ated by the wrapper-generator code it only required somecreative programming to have the Python wrappers auto-matically look through the constructors o all wrapped typesand do conversions as necessary.

bEyOND VARIANTSThe new wrappers handle the Python-wrapping o vtkVari-ant automatically, and a ew other special VTK types are

similarly wrapped. These types are wrapped by markingthem with VTK_WRAP_SPECIAL in the CMakeLists.txt leand they must also have a public copy constructor and an= operator. Other special types that have been wrappedinclude the vtkArray helper types: vtkArrayCoordinatesvtkArrayExtents, vtkArrayRange, and vtkArrayWeights.

Even though the parser now recognizes all operator methodsat this point in time the Python wrappers only wrap the comparison operators , and thestream output operator


12/24

12

request a test run o the project on any client that is presently available. Right now, only project administrators havethe privilege o scheduling builds. It should eventually evenbe possible to have CDash observe whenever a commit imade to a projects repository and immediately start dolingout build instructions to waiting clients.

See the CDash Wiki page or more (and evolving) inormation on build management. This eature is still in its inancyand we could use your help to shape its uture. Try it out andsend us eedback on the CDash mailing list with the subjec

CDash Schedule Builds Feedback.

SCHEDulINg buIlDS FOR yOuR CDASH PROJECTFirst, you have to coordinate with your CDash server administrator to set $CDASH_MANAGE_CLIENTS to 1 in either thecdash/cong.php or cdash/cong.local.php le.

Next, as project administrator, go to the Edit Project page andcheck on the settings on the Repository tab. The Repositoryand Repository Viewer Type settings need to be set. I youdont really have a repository viewer, choose a type romthat list that matches your repository anyway. CDash usethat setting to generate the correct checkout and updatecommands in the scripts that it sends to volunteer clients.

You can see a part o the script that CDash will send to theclients on the Clients tab o the Edit Project page. You maeven customize it by modiying the script or adding to it anecessary or your project.

Next, make sure the CTestCong le checked into your repository is correct. You may download an initial copy rom CDashi you dont already have one. Click on the Miscellaneoustab and look or the CTestCong download link.

Next, set up some volunteers running looping scripts. Oneach volunteer client machine, you should create a "Clientdirectory, and then two sub-directories in Client named"base" and "tmp"

In the Client directory, we need a site description xml le anda looping script. On my client machines, I name the xml le$SITE.cdash.xml and I name the script CDashClient.ctestThe xml le should contain contents like this:

MacSnowLeopard64/Users/davidcole/Client/base

THE CDASH @HOME ClOuD

Imagine running a little program on your computer to volun-teer your computing power to help keep your avorite opensource project in tip-top shape. Extend that to a swarmingcloud o volunteers running builds at your request and sub-mitting results to your projects dashboard. Soon, with the

help o CTest and CDash, youll be able to do just that.Weve created a eature in CDash called build management.This eature allows users to request that a project build bedone on any client that has volunteered its resources. Youcan try it out today i you have a checkout o CDash rom svntrunk. This eature is not ully implemented in the currentCDash 1.6 release, so youll have to use the bleeding edgecode to try it out.

The eature works through a polling protocol. A clientmachine runs a CTest script that perorms the ollowingsteps. First, the client machine submits an xml le describingits resources to CDash (submitino). The client then queriesCDash to get its corresponding site id (getsiteid). The

client then starts executing a loop, in which it queries CDashor a job to do (getjob). I the client is given a job to perormby CDash it does the job and reports the results, and thencontinues the loop. Otherwise, the client sleeps or a whileand then continues the loop.

With several client machines connected and looping, query-ing CDash or jobs to do, CDash can match build requests towaiting clients.

Ater clients are set up and looping, a project administra-tor may go to his My CDash page and schedule a buildby clicking on the Schedule Build icon. I no clients arecurrently connected, then CDash will not allow you to sched-ule builds. I clients are connected, you should see a list o

available clients, what OS each is running, and what compil-ers each has available. As the build scheduler, you choose aclient to run the build on and click the Schedule button atthe bottom o the page.

Ater you schedule a build, the next time a client asks ora job to do, CDash assigns it a job, sending a CTest scriptback as the result. The client then executes that script, whichyields a new submission to the dashboard rom that client.

As part o scheduling a build, you may: speciy an alternaterepository rom which to clone or checkout, speciy initialCMakeCache.txt entries, add a build name sux to identiyan experiment in the dashboard results and choose whichconguration to build and test. Other settings are alsoavailable in the CDash schedule build interace on the man-ageClient.php page.

When submitting an xml le to CDash, use the normalctest_submit unction in the CTest script. To query CDash orinormation, use le(DOWNLOAD in the CTest script.

At present, running CDash as a build management systeminvolves setting things up on the server (server admin andproject admin) and on each client machine (volunteers) thatwill be participating, as well as scheduling builds manually(project admin). However, the vision or this eature movingorward, is to make running CDash as a build managementsystem as automatic as possible. Anybody should be able to


13/24

13

gcc4.2.1Unix Makefiles

2.8.2/Applications/CMake 2.8-2.app/Contents/bin/

cmake

As you can see, this xml le gives CDash sucient inorma-tion to know what OS your client is running, what compilerand CMake version it has available and what CMake genera-tor to use or conguring the build.

The ull looping script is about 65 lines o CTest scriptingcode, Ill post this script to the Kitware Blog so we donthave to waste space here; the only part thats dierent rommachine to machine looks like this:

set(CDASH_SITENAME "qwghlm.kitware")set(CDASH_SYSTEMNAME "Mac-SnowLeopard-64bits")set(CDASH_SITE_CONFIG_FILE "/Users/davidcole/Client/qwghlm.cdash.xml")set(CDASH_TEMP_DIRECTORY "/Users/davidcole/Client/tmp")set(CTEST_EXECUTABLE "/Applications/CMake 2.8-2.app/Contents/bin/ctest")set(CTEST_DROP_SITE "www.yourcompany.com")

Ater the looping scripts are running on a client or two (ormore!), you may go to the Schedule Builds interace oCDash and request builds rom them. Go to the My CDashpage, click on the Schedule Builds icon it looks like afoppy disk, you should see a page like this:

By deault, you should be able to choose a client rom theSite: list and click on the Schedule >> button at thebottom. However, you may wish to enter inormation intoany o the elds on that page to control various aspects othe build.

A list o projects with Schedule Build icons

Let us know what needs improvement. Especially let us knowi youd like to sponsor uture work on this topic.

David Cole is an R&D Engineer in KitwaresCliton Park ofce. David has contributed

code to the VTK, CMake, ITK, ParaViewKWWidgets and gccxml opensource projectsand to Kitwares proprietary products including ActiViz and VolView.

MulTI-RESOluTION STREAMINg INVTK AND PARAVIEW

ParaView's ability to process and visualize massive datasetis based on VTK's streaming unctionality. In ParaView, iden

tical copies o the visualization pipeline are run on manymachines and each pipeline is asked to process a dierensmall portion, or piece, o the input data. Together themachines process the entire dataset simultaneously, and nomachine ever exceeds the capacity o its local RAM.

Refnement at work in the analysis o a 3600x2400x42 chlorouo-

rocarbon (CFC) concentration simulation study, being perormed

on a 32-bit laptop PC. Yellow outlines identiy pieces. At this

point, ParaView has progressed seven levels down into a nine leve

deep piece refnement tree. Blue outlines show individual cells. At

the lowest level cells are already at sub-pixel resolution.


14/24

14

//the next call is very fastdouble priority = filtersExec->ComputePriority();

if (priority > 0.0){//the next call is potentially very slowaFilter->Update();vtkDataObject *data =aFilter->GetOutputDataObject(port);

vtkInformation* dataInfo = data->GetInformation();double resultResolution = dataInfo->Get(vtkDataObject::DATA_RESOLUTION());

}

ParaView 3.8 also includes AdaptiveParaView, anotheexperimental application which exercises this new multiresolution capability. Multi-resolution streaming begins byrendering a single piece that covers the entire domain atminimum resolution with the very rst update giving theuser valuable eedback about the entire domain. This is agreat advantage over standard streaming in which globainormation sometimes becomes apparent only when theentire domain is covered at ull resolution and ater a muchlonger delay.

Refnement o isosuraces in the CFC data o the coast o

Newoundland. A laptop computer is able to show the ull resolu-

tion data since areas o-screen are ignored and only a ew pieces

stay resident in memory at any given instant.

The adaptive streaming algorithm then recursively splitspieces, increasing the resolution with each split, to showthe data in greater detail. As it processes pieces it gathersmeta-inormation (world extent and data ranges) anduses this inormation to improve its importance estimatesThroughout the algorithm, prioritization guides the choiceo which pieces need to be rened immediately, which canbe deerred, and which can removed rom urther consider

ation. The algorithm eventually converges to processing justthe important eatures o the data at the ullest resolution.

THE NEED FOR CACHINgUnortunately streaming adds overhead. Every pipelineupdate can take a signicant amount o time. In standardVTK, the pipeline internally caches intermediate results sothat data processing time can be amortized over many lateupdates i those do not invalidate the results. Unortunatelychanging the requested piece or resolution invalidates theinternal caches. We minimize the problem by aggressivelycaching results at the end o the pipeline. In particularwhenever the visible geometry is small enough, our cacheallows all o the data to be re-rendered in a single pass

VTK's ability to break up data is called streaming, becausewhen the data can be divided it is also possible to iterateover the pieces. In data parallel processing you stretch theproblem over a larger amount o RAM whereas in stream-ing you stretch the problem over a longer time. Iterativestreamed rendering proceeds, or example, by renderingeach piece in turn without clearing the color or depthbuer, using the persistent Z-buer to resolve occlusion orindividual triangles both within a piece and across pieces.In practice, streaming and data parallel processing are

orthogonal and can be combined by dividing the probleminto P sets o I pieces.

In many computational processes and especially rendering, itis oten the case that only a small raction o the entire datacontributes to the end result. Prioritized streaming processesonly those pieces that contribute to the nal result (ignor-ing pieces that are o-screen, or example) and processesthem in a most important (i.e. nearest to the camera) toleast important order. This variation o streaming has greatbenets including eliminating unnecessary processing andIO [1] and providing more immediate eedback to the userto speed the data discovery process [2]. Prioritized streamingis the basis or the experimental branded application calledStreamingParaView. StreamingParaView was rst introducedin ParaView 3.6.

VTK's streaming has an interesting eature in that its pos-sible to ask or data in arbitrarily small chunks. Streaming isdriven by asking or dierent pieces at the downstream endo the pipeline (ex vtkPolyDataMapper::SetPiece()). One asksor smaller pieces by asking or pieces out o a larger set (ex vtkPolyDataMapper::SetNumberOPieces()). What is interest-ing about this is that as the chunk size decreases - assumingprioritization is in eect - the work done to produce a givenvisualization approaches the minimal amount necessary.

We recently added the ability to ask or data at diering res-olution levels to VTK's streaming support. It is now possibleto not only ask or arbitrary pieces, but also to ask or themat arbitrary resolutions. The mechanics are similar to VTK'stemporal support [3]. One asks the pipeline to provide dataat a given requested resolution. This request is a dimension-less number that ranges rom 0.0 meaning lowest resolutionto 1.0 meaning ull resolution. I unspecied, ull resolutionis assumed. The request travels up the pipeline to the reader,which decides how to interpret this number. Structureddata sources use the resolution to choose how coarsely tosubsample in the i, j and k dimensions and adjust their x, yand z spacing to compensate accordingly. As in the temporalpipeline, the result is ree to vary rom what was requested.To support this, the reader inserts a resolution answer keyinto the data it produces, which then travels back down the

pipeline to the requester.The ollowing is a code example rom VTK which asks thepipeline to: compute the priority or a particular piece ata particular resolution, conditionally update the pipeline toproduce the requested piece and then examine the returnedresolution result.

vtkStreamingDemandDrivenPipeline * filtersExec =vtkStreamingDemandDrivenPipeline::SafeDownCast(aFilter->GetExecutive());

filtersExec->SetUpdateResolution(port, requestedResolution);

filtersExec->SetUpdateExtent(port, pieceNum, numPieces, ghostLevel);


15/24

15

In this situation, camera movement proceeds as ast asit does in non-streaming ParaView. Despite caching, theconvergence process itsel can take a signicant amount otime, thereore AdaptiveParaView has controls that allowthe user to pause and restart, limit, or manually control therenement process.

CONSERVATIVE PRIORITIzATIONA key point is that its generally impossible to know a prioriwhat pieces contribute the most or least without executing

the pipeline. This is unortunate because the goal o priori-tization is to avoid executing the pipeline on unimportantdata. Consider culling pieces that all outside o the viewrustum. Many VTK readers can determine the initial world-space bounding box o any requested piece. However ltersexist to change data, and any lter along the pipeline mighttransorm the data arbitrarily changing the bounding boxbeore the data is rendered. In order to nd which piecesare not visible, one must rst know what the world spacebounding box o each piece is ater that piece is processedby the pipeline.

To solve this chicken and egg problem, a acility or perpiece meta-inormation propagation was added to the VTKpipeline. Readers and sources can provide piece level meta-

inormation, and lters can describe what types o datamodications they do. With inormation about what is pro-vided and what is potentially changed, the pipeline is betterable to provide a conservative estimate o the importance oany piece without doing any time consuming computations.When either type o inormation is missing, the pipelinealls back to the non-prioritized behavior o iterativelyprocessing every piece. See VTK/Rendering/Testing/Cxx/TestPriorityStreaming.cxx or a demonstration.

FuTuRE WORKWe continue to "rene" our implementation o adaptivestreaming. Our most immediate challenge is to improve theclient parallel server implementations o both streaming

applications. Currently, pure streaming requires too requentcommunication with the server to be ecient and adaptivestreaming has only been implemented to work with serialmode ParaView runs.

Next, there are airly major robustness and usability limi-tations o our preliminary experimental prototypes. Ourcurrent work is available in source ormat only and there areoutstanding unsolved issues regarding how to deliver globalup-to-date meta-inormation to the end user as computa-tion progress.

Lastly, we are working to extend the work to be compatiblewith more data ormats. Our rst streaming capable readerreads simple raw structured data in either preprocessed or

raw ormat. For this data type, changing resolution is easilyachieved by sub-sampling [4]. We have since extended theramework to handle cloud data, o which the LANL cos-mology ormat was our rst target. For this we devised animportance sampling mechanism that chooses representa-tive samples and limits the resolution so as not to overll thedisplayed image resolution.

We anticipate extending the ramework to work on AMRdata, which has multi-resolution inormation written inby the simulation codes that generate it; and to waveletcompressed image data, as exemplied by NCAR's VaporData Format les. Extending the ramework to handle non-preprocessed unstructured data types is a long-term goal.

REFERENCES[1] Childs, H., Brugger,E., Bonnell, K., Meredith, J., Miller, M.

Whitlock, B., Max, N. "A Contract Based System For LargeData Visualization." Proceedings o the IEEE VisualizationConerence 2005.

[2] Ahrens, J., Desai, N., McCormick, P., Martin, K., WoodringJ. "A modular extensible visualization system architectureor culled prioritized data streaming." Proceedings o theSPIE, 2007.

[3] Biddiscombe, J., Geveci, B., Martin, K., Moreland, K., andThompson, D. "Time Dependent Processing in a Paralle

Pipeline Architecture." IEEE Transactions on Visualizationand Computer Graphics, 2007.

[4] Ahrens J., Woodring J., DeMarle D., Patchett J., MaltrudM. "Interactive Remote Large-Scale Data Visualization viaPrioritized Multi-resolution Streaming." Proceedings o theUltraScale Visualization Workshop, 2009

David DeMarle is a member o the R&D teamat Kitware where he contributes to bothParaView and VTK. He requently teachesKitware's proessional development andtraining courses or these product applications and enjoys putting puns in KitwareSource articles. Dave's research interests are

in systems level aspects o visualization, in particular memoryoptimizations or parallel visualization o large datasets.

Jonathan Woodring is a new sta scientist inCCS-7 Applied Computer Science at Los

Alamos National Laboratory. He received hisPhD in Computer Science rom The OhioState University in 2009. His research interests include scientifc visualization andanalysis, high perormance supercomputing

and data intensive supercomputing.

James Ahrens is a team leader in the AppliedComputer Science Group at Los AlamosNational Laboratory. His research ocuses on

large-data visualization and scientifc datamanagement. Ahrens received his Ph.D. inComputer Science rom the University oWashington and is member o the IEEEComputer Society.

COMMuNITy SPOTlIgHT

The ollowing articles highlight how some o our commu

nity members are utilizing Kitwares open-source toolkits

and products in their own development environment. I you

would you like to be eatured as a Community Spotlight, send

your article ideas and materials to [email protected]

VISuAlIzINg DEFORMAblE SuSPENSIONSIMulATIONS IN PARAVIEW

Computational simulation o blood fow and densesuspensions o deormable particles is only possible onhigh perormance computing resources using a highlyscalable computational approach such as the hybrid lattice-Boltzmann / nite element (LB-FE) method. This methodoriginally developed or rigid suspensions, was extended todeormable red blood cells (RBCs), particles, capsules [1] and


16/24

16

VISuAlIzATION OF DATA WITH PARAVIEWWe have shown the ability to simulate thousands o deormable particles and capture non-Newtonian characteristicsthat agree well with experimental observations [1]. Ourapproach has shown the ability to scale on as many as 65,536computational cores o Argonne National Laboratorys IBMBlue Gene/P [5]. When computing the dynamics o 1,000s to100,000s o deormable particles suspended in fuid, datasetcan easily reach 1 terabyte or greater or a single simulationAs an example, one such simulation with 100,000 deorm

able spheres (each with 254 triangular suraces) requires a LBfuid domain in excess o 2 billion lattice grid points.

When fuid and solid data sets rom CFD codes are O(100GB), the use o distributed memory architectures is requiredor post-processing. Through the use o virtual networkcomputing (VNC) and access to visualization clusters likeLonghorn and Spur at the Texas Advanced ComputingCenter (TACC) via NSFs TeraGrid HPC network, visualizationnot possible on shared memory resources can be perormedVisualization clusters like those at TACC oer multiple CPUmultiple GPU nodes that are created specically or theseapplications. The ollowing is an instantaneous snapshot oa dense suspension o 2472 RBCs in a wall-bounded shear.

FuTuRE WORKRecent additions [6] have shown the ability to simulate densesuspensions o RBCs in micro-vessel sized tubes at high shearates, in which large RBC deormations extend beyond theassumptions o linear theory. This work is based on couplingthe LB method with a spectrin-link model [7] or the RBCmembrane.

REFERENCES[1] R. M. MacMeccan, J. R. Clausen, G. P. Neitzel, and C. K

Aidun, Simulating deormable particle suspensions usinga coupled lattice-Boltzmann and nite-element method,J. Fluid Mech., 618:13-39, 2009.

[2] J. Wu and C. K. Aidun, Simulating 3D deormable particlesuspensions using lattice-Boltzmann method with discreteexternal boundary orce, Inter. J. Num. Meth. Fluids62:765-783, 2009.

[3] J. Wu and C. K. Aidun, A method or direct simulation ofexible ber suspensions using lattice Boltzmann equationwith external boundary orce, Inter. J. Multiphase Flow36:202-209, 2010.

[4] C. K. Aidun and J. R. Clausen, Lattice-Boltzmann Methodor Complex Flows, Annual Rev. Fluid Mech., 42:439-4722010.

bers [2, 3]. The LB-FE method resolves the fuid phase, bothinterior and exterior to the fuid-lled particles, with the LBmethod. A recent review article [4] highlights the easibilityo using the LB method or complex fows including suspen-sion o deormable, particles, capsules and bers.

We examine the dynamics o deormable capsules and howthese dynamics aect the rheology o dense suspensionsusing the LB-FE method. O particular interest are the par-ticle pressure and a ull characterization o normal stresses,which are dicult to measure experimentally. The ollow-ing gure demonstrates a novel problem where the particlemicrostructure aects the viscosity, normal stresses, and par-ticle pressure o a suspension. The maximum viscosity andrst normal stress dierence is obtained when particles arealigned in a cluster along the compressional axis.

Red blood cells (RBCs), the most numerous constituent oblood, infuence continuum-level measurements by alteringthe suspension at microscopic scales. In moderately-large-sized arteries, the nature o blood can be described as amacroscopic non-Newtonian fuid; however, fow in smallervessels requires treating human blood as a multiphase fuidcontaining 40-45% RBCs by volume. Simulations o blood

at a cellular level provide a tool that allows exploration othe rheology, stress, and diusion o individual suspendedcells. In our approach, the blood plasma and hemoglo-bin are treated as Newtonian fuids with viscosities o 1.2and 6 cP, respectively. The multiphase nature o the bloodcreates non-Newtonian fow eects, such as shear thinning,commonly observed in experiments. An image o multipleclusters orming in Hagen-Poiseuille fow is shown below.


17/24

17

[5] J. R. Clausen, D. A. Reasor, and C. K. Aidun, ParallelPerormance o a Lattice-Boltzmann/Finite Element CellularBlood Flow Solver on the IBM Blue Gene/P Architecture,Comp. Phys. Comm., 181:1013-1020, 2010.

[6] D. A. Reasor, J. R. Clausen, and C. K. Aidun, Coupling thelattice-Boltzmann and spectrin-link methods or the directnumerical simulation o cellular blood fow, submitted toInter.l J. Num. Meth. Fluids, June 2010.

[7] J. Li, M. Dao, C. T. Lim, and S. Suresh, Spectrin-LevelModeling o the Cytoskeleton and Optical TweezersStretching o the Erythrocyte, Biophys. J., 88:3707-3719,

2005.

Dr. Cyrus Aidun joined the Woodru Schoolo Mechanical Engineering at the GeorgiaInstitute o Technology as a Proessor in

2003. He began at Tech in 1988 as an AssistantProessor at the Institute o Paper Scienceand Technology. Prior, he was at BattelleResearch Laboratories and was Senior

Research Consultant at the National Science Foundation'sSupercomputer Center at Cornell University.

Daniel Reasor received his Bachelor's andMaster's degree in Mechanical Engineeringrom the University o Kentucky (2006, 2007).

He is currently a Ph.D. student at the GeorgiaInstitute o Technology perorming researchocused on the direct numerical simulationo cellular blood ow. He is unded by the

U.S. Department o Deense through the ASEE SMART el-lowship.

Dr. Jonathan Clausen received a Bachelor'sdegree in Mechanical Engineering romClemson University in 2004 and a Ph.D. romGeorgia Institute o Technology in 2010. HisPh.D. research ocused on the rheology andmicrostructure o deormable capsule sus-

pensions. Since July 2010, Jonathan Clausen

has been a Senior Member o Technical Sta at SandiaNational Laboratories.

Dr. Jingshu Wu has a Master's degree inAerospace Engineering (2004) and a Ph.D. inMechanical Engineering (2010) rom theGeorgia Institute o Technology. His Ph.D.research ocused on the development omethods or multiphase ow simulation.Since June 2010, he has been an aeroacoustic

engineer at Vestas Technology R&D Americas.

ITuRRIbIzIA ON HOW XC uTIlIzES VTKXC is a program designed at our company, Iturribizia [1], tosolve structural analysis problems utilizing a nite elementmethod. The program can solve various types o problems,rom simple linear analysis to complex nonlinear simula-tions. It has a library o nite elements which can be used tomodeling various geometries and multiple materials or usein various areas o structural analysis.

MOTIVATIONSomeone said that, when the French climber Lionel Terraywas asked about his reason to climb a mountain, he simplysaid because it was there. Something similar happened

with the development o this program. I began the studyo the nite element method ater studying the analyti-cal solutions to elastic problems (so limited) and I becamegreatly interested in their use in structural problems. Thiscoupled with my love or computer science, made me decideto develop a nite element program that would be useul tocalculate structures and could be modied and expanded inany way the user wanted.

First I wrote a Pascal version o the program which onlyworked with bar-type elements. Then I wrote a C++ version

rom scratch that was never able to solve any nontriviaproblem. Finally, I discovered the possibilities oered by thecalculation core o Opensees and decided to modiy it tobe suitable or an industrial environment (as opposed toacademic use).

To achieve this objective, several signicant modications tothe original code were required. Algorithms or generatingnite element mesh were incorporated, allowing the modeleto create structured grids rom the description o geometryby means o points, lines, suraces and solids. Graphics weregenerated using the VTK library (we give more details onthis later.) A new macro language was developed to makeit possible to obtain the results produced by the calculation

without having to extract them rom predened listings. Thisprovides the program with the ability to interpret a sentencelike get the ratio between the vertical displacement o thenode closest to the center o the beam and the total span othe beam. Utilities or the construction and calculation odesign load combinations prescribed by the building codes(EHE, ACI 318, EAE, Eurocodes, etc.) were implemented toacilitate the verication o design requirements. The abilityto activate and deactivate elements was introduced toenable the analysis o structures built in phases, geotechnical problems, and the strengthening o existing structuresMacros were written to veriy the structure and its elementsaccording to the criteria prescribed by building codes (e.gaxial and bending capacity, shear reinorcement). The code

was changed to link with standard linear algebra libraries(e.g. BLAS, Arpack, LAPACK, SuperLU), eliminating the needto include in the program ad-hoc versions o these libraries. Finally, the material models were modied to supportprescribed strains, making it possible to solve problemsinvolving thermal and rheological actions.

DESIgN guIDElINESWith the experience obtained rom previous developmentworks we arrived at the ollowing conclusions: test, testcareully and test again; do not reinvent the wheel do notwaste time in developing an elegant GUI; and while buildingcodes change, physical laws do not.

TestinFor each unctional element o the program a test should bewritten to veriy the correctness o the code. Also ater eachcode modication it must be veried that every one o thetests still execute successully. Once an error is detected andcorrected we must write a test to veriy that the error wilnot happen again.

Reinventin the Whee

Rather than starting rom scratch every time you need tointroduce something to your code, seek out open sourcelibraries. Weve learned to use open source libraries (such asVTK) as much as possible. Make sure that these libraries areeasily accessible via the Internet.


18/24


19/24

19

strains, loads, etc., or design a command language whichwould be a means through which users could dene thegraphical output.

The rst approach is commonly used in some matrix structuralanalysis computer programs and other programs oriented tospecic tasks such as slope stability analysis or solving planeelasticity problems. The main disadvantage o this solutionis that its very dicult to apply to situations that were notpreviously considered by the programmer.

The second solution is used in the majority o the generalpurpose nite element codes (ANSYS, Abacus). Its unda-mental diculty is the design o a command language thatis fexible enough and easy to use. During analysis o thissolution we studied other program manuals (ANSYS, Abacus,Solvia, Calculix Graphix) to determine the eatures neededor the command language.

Figure 2: View o a cone generated with the script rom Table 1

Ater studying both solutions we decided that the rst wasalmost unworkable i we wanted the program to be able totreat multiple types o problems (e.g. analysis o buildings,bridges, reservoirs).

The second was more appropriate to deal with the problemsince it provides the users with the tools they need to displayresults and, in some way, transer the task to them. Once thesolution was chosen, we had to design a command languagepowerul enough to generate graphics allowing the use oall options (transparency, lighting, textures) that moderncomputer graphics oer. Here the potential o VTK cameinto play.

The VTK library is written in C++ (the same language used orthe remainder o the program), making it quite easy to callit rom the rest o the code. The availability o some scriptlanguages (Tcl, Java and Python) in VTK made us realize thatthe command language design was already done (the VTKAPI itsel) and all we had to do was move it to our own scriptlanguage used by the rest o the program. Proceeding inthis manner, we enabled the use o VTK or any purpose theuser desires as opposed to just making the code available ornite element model representations. Figure 2 (which willbe recognizable to all those amiliarized with the examplessupplied with the VTK package) shows the results o themacro shown in Table 1. A more complex example can beseen in Figure 3.

Figure 3: Shell fnite element model o a bridge deck

\vtk{\define["vtkConeSource","cone"]{ \altura{3.0} \radio{1.0} \resol{10} }

\define["vtkPolyDataMapper","coneMapper"]{ \set_input{"cone"} }

\define["vtkActor","coneActor"]{ \set_mapper{"coneMapper"} }

\define["vtkRenderer","ren1"]{\add_actor{"coneActor"}\set_background{0.1,0.2,0.4}

}\define["vtkRenderWindow","renWin"]{ \add_renderer{"ren1"} \set_size{1024,768} }

\define["vtkRenderWindowInteractor","iren"]{ \set_render_window{"renWin"} }

\define["vtkInteractorStyleTrackballCamera","style"]

{}\iren{\set_interactor_style{"style"}}\iren{\initialize{} \start{}}

}

Table 1: Script to display the cone on fgure 2

ACKNOWlEDgEMENTSMany thanks to all the people who contribute to opensource, without their eort this work would not have beenpossible. Thanks to Proessor Filip C. Filippou and FrankMackenna rom the Department o Civil and EnvironmentaEngineering at the University o Caliornia, Berkeley ortheir email correspondence. Thank you to my amily or theipatience as I worked on the development o this sotwarepackage. And thank you to my riends Raul Hernandez andTomas Sanz or their encouragement.

REFERENCES[1] Iturribizia may be downloaded here:

http://www.iturribizia.com/descarga sotware.html.

[2] http://www.gnuplot.ino

Luis C. Prez Tato was born in Madrid (Spain)in 1965. Studied civil engineering at the"Universidad Politcnica de Madrid". Hiswork has been related to computer programming since 1988 and to structuraanalysis since 1992. Has developed programsin the felds o structural analysis, geographi

inormation systems an dam instrumentation.


20/24

20

The team will also continue to develop and apply cuttingedge research into the hard problems in descriptors, indexingiterative query renement, and descriptor usion. One o themany research thrusts in Phase II will be to incorporate morecomputed and supplied scene knowledge directly into thedescriptors, and to more eectively leverage available metadata such as view point and sun angle. Augmenting systemcapabilities to allow or additional video sources, both romUAVs and ground cameras, wil

Artículo sobre XC publicado en la revista de Kitware.

Documents

Transcript of Artículo sobre XC publicado en la revista de Kitware.