Sketch-based Image-independent Editing of 3D Tumor Segmentations using Variational Interpolation

Toward Automated Validation ofSketch-based 3D Segmentation Editing ToolsFrank Heckel1, Momchil I. Ivanov2, Jan H. Moltz1, Horst K. Hahn1,21 Fraunhofer MEVIS, Bremen, Germany, 2 Jacobs University, Bremen, Germany

18th Scandinavian Conference on Image Analysis, Espoo, Finland, June 2013 Fraunhofer MEVIS Fraunhofer MEVISNr. / 15MotivationSegmentation is one of the essential tasks in medical image analysisMany sophisticated automatic segmentation algorithms exist which might fail in some casesLow contrast, noise, biological variability, What is segmentation editing and why isnt it trivial?

Solution Results Outlook Conclusion What to do? Fraunhofer MEVISNr. / 15,1MotivationIntuitive interaction in 2D Estimate the users intention in 3DAs few interactions as possibleThe segmentation problems are typically hard

What is segmentation editing and why isnt it trivial?Solution Results Outlook Conclusion Locally correct the error until it satisfies the specific needs

F. Heckel et al., 3D contour based local manual correction of tumor segmentations in CT scans, SPIE Medical Imaging: Image Processing, 2009F. Heckel et al., Sketch-based Image-independent Editing of 3D Tumor Segmentations using Variational Interpolation, Eurographics Workshop on Visual Computing for Biology and Medicine, 2012 Fraunhofer MEVISNr. / 15,2Motivation

What is segmentation editing and why isnt it trivial?Solution Results Outlook Conclusion F. Heckel et al., Sketch-based Image-independent Editing of 3D Tumor Segmentations using Variational Interpolation, Eurographics Workshop on Visual Computing for Biology and Medicine, 2012 Fraunhofer MEVISNr. / 15,3MotivationThe segmentation editing processSolution Results Outlook Conclusion

intended result that the user only has in mind visually performed by the user Fraunhofer MEVISNr. / 15,4MotivationStatic quality measurements existVolume overlap / dice coefficientAverage / maximum surface distanceInteractive segmentation processMeasuring the quality of the final result only is not enoughAcceptance suffers from bad intermediate resultsAdditional quality factors like number of steps, computation time, User is mandatoryHigh effort New evaluations after algorithmic changesBad reproducibilityThe difficulty in validation of segmentation editing toolsSolution Results Outlook Conclusion Fraunhofer MEVISNr. / 15,5SolutionThe automatic validation processResults Outlook Conclusion Motivation

once generated by an expertuse common quality measurements Fraunhofer MEVISNr. / 15,6SolutionStep 1: Find the most probably corrected errorSubtract intermediate from reference segmentation3D connected components define errorsSelect an error to be correctedLargest volume + compactness

Largest Hausdorff distance

Editing simulationResults Outlook Conclusion Motivation

reference segmentationintermediate segmentation

Fraunhofer MEVISNr. / 15,7SolutionStep 2: Select slice and view where the error is most probably correctedLargest area + compactness

Largest Hausdorff distanceEditing simulation

Results Outlook Conclusion Motivation

Slice 48

Slice 44

Slice 52

Fraunhofer MEVISNr. / 15,8SolutionStep 3: Generate user-input for sketchingGet surface of errorRemove voxels that are on the surface of the intermediate segmentation as wellStep 4: Apply editing algorithmEditing simulation

Results Outlook Conclusion Motivation

Fraunhofer MEVISNr. / 15,9ResultsVolume-based strategyOutlook ConclusionSolution

Fraunhofer MEVISNr. / 15,10ResultsDistance-based strategyOutlook ConclusionSolution

Fraunhofer MEVISNr. / 15,11ResultsOutlook ConclusionSolution

Volume-based strategyDistance-based strategy Fraunhofer MEVISNr. / 15,12OutlookSolve current limitations (e.g., correction of holes)Extend simulationModel inaccuracy in drawing sketchesModel more correction strategiesFinish an error before moving to the nextPerform correction in one of the errors first slicesInvestigate how the quality of editing tools is measured bestApply simulation-based validation to a larger database

ConclusionResults Fraunhofer MEVISNr. / 15,13ConclusionSegmentation editing:Is an indispensable step in the segmentation processEfficient editing in 3D is challengingValidation of 3D editing algorithms:Needs to consider the dynamic nature of such toolsUser studies are time consuming and lack reproducibilityProposed solution: Simulate the userAllows objective and reproducible validation and comparisonAllows better quality assessment

Outlook Fraunhofer MEVISNr. / 15,14

frank.heckel@mevis.fraunhofer.de

Thank you! Fraunhofer MEVISNr. / 15AppendixComputation time

Volume-based strategyDistance-based strategy Fraunhofer MEVISNr. / 15,16IntermediateSegmentation

Reference Segmentation

Satisfying?

User / Simulation

Static Validation

Segmentation Editing

no

yes

Stop

Start

User Input

Results

IntermediateSegmentation

Reference Segmentation

Satisfying?

User

Static Validation

Segmentation Editing

no

yes

Stop

Start

Program flow

Data flow

User Input

IntermediateSegmentation

Reference Segmentation

Satisfying?

User

Static Validation

Segmentation Editing

no

yes

Stop

Start

Program flow

Data flow

User Input

IntermediateSegmentation

Reference Segmentation

Satisfying?

Simulation

Static Validation

Segmentation Editing

no

yes

Stop

Start

Program flow

Data flow

User Input

Results

IntermediateSegmentation

Reference Segmentation

Satisfying?

Simulation

Static Validation

Segmentation Editing

no

yes

Stop

Start

Program flow

Data flow

User Input

IntermediateSegmentation

Reference Segmentation

Satisfying?

User / Simulation

Static Validation

Segmentation Editing

no

yes

Stop

Start

User Input

Results