Full Body 3D Scanning Team KIPPA Sam Calabrese, Abhishek Gandhi, Changyin Zhou {smc2171, asg2160,...

Full Body 3D ScanningFull Body 3D ScanningTeam KIPPA

Sam Calabrese, Abhishek Gandhi, Changyin Zhou

{smc2171, asg2160, cz2166}@columbia.edu

OutlineOutline

BackgroundBackgroundMotivationMotivationOur PlanOur PlanData CaptureData CaptureData ProcessingData ProcessingResult ComparisonResult ComparisonDiscussionDiscussion

BackgroundBackground Laser Scanner using TOF

High precision, long range, slow Laser Scanner using triangulation method

High precision, smaller range, occlusion problem, slow Image-based method using triangulation method

(motion, stereo, focus/defocus, shading…) Relatively low precision and resolution Pose assumption to the surface Fast (can be real-time)

.. With Structured Light Improve the precision and resolution Indoor

Background | Motivation | Our Plan | Data Capture | Data Processing | Result Comparison

MotivationMotivation

Lots of people dream to have an accurate 3D Lots of people dream to have an accurate 3D model of themselvesmodel of themselves

Lots of applications with the real 3D model Lots of applications with the real 3D model (Animation, Augmented Realistic and even (Animation, Augmented Realistic and even Clothes Design…)Clothes Design…)

DifficultiesDifficulties Laser scanner – too slow to scan a live person Laser scanner – too slow to scan a live person

(moving and non-rigid)(moving and non-rigid) Image-based method – not enough resolution and Image-based method – not enough resolution and

precision (even with structured light)precision (even with structured light) Laser may hurt the eyesLaser may hurt the eyes


Our PlanOur Plan

Image-based method to model the headImage-based method to model the headLaser scanner to capture the bodyLaser scanner to capture the body

Proper experiment settings to minimize the Proper experiment settings to minimize the model movement during the scanningmodel movement during the scanning

Proper post-process to tolerant slight Proper post-process to tolerant slight movementmovement

Software to merge them togetherSoftware to merge them togetherMap skins and do animation afterwardMap skins and do animation afterward


Data Capture - HeadData Capture - Head

FaceGen: www.facegen.comUse face symmetric, a large set of 3D Face models

(gender, ages, races..)


Data Capture -BodyData Capture -Body A professional model + a large private room + 4 hoursA professional model + a large private room + 4 hours Tripods to rest the arms (10 cm lower than the shoulders)Tripods to rest the arms (10 cm lower than the shoulders) Mark the feet positionMark the feet position A camera to track the movementA camera to track the movement Scanner is as high as the shoulderScanner is as high as the shoulder The distance to scanner ranges from 2.5m to 3mThe distance to scanner ranges from 2.5m to 3m


Data Capture –Body (cont’d)Data Capture –Body (cont’d) Four scan points (Frontal/Back x Left/Right) Four scan points (Frontal/Back x Left/Right) Ten targets around for further registrationTen targets around for further registration Precision set to 2mmPrecision set to 2mm ≈≈10 min for each scan, but a long time to move the scanner10 min for each scan, but a long time to move the scanner 90 – 130 K vertices for each range data90 – 130 K vertices for each range data


Data Capture –Body (cont’d)Data Capture –Body (cont’d)

Registration with CycloneRegistration with Cyclone The body movement causes The body movement causes

big problemsbig problems we need more stronger non-we need more stronger non-

rigid mesh merging methodsrigid mesh merging methods


MeshLabMeshLab

Initially used to Initially used to convert PTX to PLYconvert PTX to PLY

Problems arose with Problems arose with the original PTX files the original PTX files from Cyclonefrom Cyclone

Used MeshLab to Used MeshLab to reorientreorient

Used again to clean Used again to clean data and resurface data and resurface after VRip surfacingafter VRip surfacing


ScanalyzeScanalyze

Used to re-register the Used to re-register the Range scans following Range scans following the initial errorsthe initial errors

Prepares for VRip directly Prepares for VRip directly by saving out .conf by saving out .conf and .xf files which VRip and .xf files which VRip reads to orient the reads to orient the different scans to each different scans to each other while maintaining other while maintaining their original orientationtheir original orientation


VRipVRip

Used to create a surface from the registered point cloudUsed to create a surface from the registered point cloud Uses view direction and ramp weights to get confidence Uses view direction and ramp weights to get confidence

about each vertexabout each vertex Sampled at .002m in each direction per voxelSampled at .002m in each direction per voxel Used ramp weight of .004m with slight increase in Used ramp weight of .004m with slight increase in

standard weights standard weights


PlyCrunch and Meshlab (again)PlyCrunch and Meshlab (again)

PlyCrunch is Packaged with VRipPlyCrunch is Packaged with VRipDecimated the Mesh from 3.5 million Decimated the Mesh from 3.5 million

polygons to just over 10,000polygons to just over 10,000Full of holes because Plycrunch deleted Full of holes because Plycrunch deleted

Triangles, but left proper vertices.Triangles, but left proper vertices.

Meshlab used to clean and resurface Meshlab used to clean and resurface resulting mesh, filled most of the holes resulting mesh, filled most of the holes


3dsMax3dsMax

Used to finalize mesh by capping the remaining Used to finalize mesh by capping the remaining holes and smoothing the result.holes and smoothing the result.

Used to attach the head from facegenUsed to attach the head from facegen Attached a skin shader for Mental Ray, one of Attached a skin shader for Mental Ray, one of

the built in Renderers which used Sub-Surface the built in Renderers which used Sub-Surface Scattering for realism, modified to match the Scattering for realism, modified to match the texture from FaceGentexture from FaceGen

Rigged with Biped objectRigged with Biped object Animated with stock Motion Capture DataAnimated with stock Motion Capture Data Rendered into animationsRendered into animations


Result ComparisonResult Comparison

We compared our output with that of two We compared our output with that of two professional companies :professional companies : Headus – based in AustraliaHeadus – based in Australia AvatarMe (initially developed by University of Surrey)AvatarMe (initially developed by University of Surrey)

Criteria for Comparison :Criteria for Comparison : Resolution of the outputResolution of the output Accuracy of scanned dataAccuracy of scanned data Error prevention during scanningError prevention during scanning Cost/Ease of setupCost/Ease of setup


Resolution of the outputResolution of the output

We made way for Quality via QuantityWe made way for Quality via Quantity KIPPAKIPPA

Number of Polygons in original scanNumber of Polygons in original scan Scan 1 : 168,855Scan 1 : 168,855 Scan 2 : 179,868Scan 2 : 179,868 Scan 3 : 174,686Scan 3 : 174,686 Scan 4 : 101,010Scan 4 : 101,010 Total number of points : 500,000Total number of points : 500,000

Data Density : 2mmData Density : 2mm

OthersOthers Data Density 4 mm (Headus)Data Density 4 mm (Headus) Average number of points : 300,000Average number of points : 300,000


Resolution of the outputResolution of the output

AvatarMeAvatarMe KIPPA KIPPA


Accuracy of Scanned DataAccuracy of Scanned Data(smoothness, details captured)(smoothness, details captured)

Used more than just one good software –Used more than just one good software –

Vrip- 3,858,996

MeshLab-3,858,783

PlyCruch-10,019

MeshLab-15,056

3ds Max-15,080

93,108


Error prevention during scanning &Error prevention during scanning &Cost/Ease of setupCost/Ease of setup

KIPPAKIPPA We didn’t have a pre-defined model/shape against We didn’t have a pre-defined model/shape against

which to map our data.(explain)which to map our data.(explain) Cheap/Simple set-upCheap/Simple set-up

Separate scanning for head and bodySeparate scanning for head and body Limited body movement below the neckLimited body movement below the neck

OthersOthersWBX Platform(explain)WBX Platform(explain)

(movable platform, cables, etc.)(movable platform, cables, etc.)http://www.cyberware.com/documentation/digisize/www/info/http://www.cyberware.com/documentation/digisize/www/info/

WBXPlatform.htmlWBXPlatform.html


Discussion & ConclusionDiscussion & Conclusion Reduction and dealing with motion…Reduction and dealing with motion… Repositioning the scanner was difficult…Repositioning the scanner was difficult… Problems during transformation…Problems during transformation… Efficient use of markers…Efficient use of markers… Ignored hair…Ignored hair… Missed details of hands and toes…Missed details of hands and toes…

To conclude, To conclude,

3D human scan was a very interesting problem to deal with. We 3D human scan was a very interesting problem to deal with. We managed to get a satisfying output in a very efficient manner.managed to get a satisfying output in a very efficient manner.

It was enjoyable learning and using softwares such as Vrip, It was enjoyable learning and using softwares such as Vrip, Cyclone, MeshLab, PlyCrunch and 3ds Max.Cyclone, MeshLab, PlyCrunch and 3ds Max.

References & ResourcesReferences & Resources ALLEN, P., 2007. 3d photography 2007 fall. Class notes on Active 3D Sensing. ALOIMONOS, Y., AND SPETSAKIS, M. 1989. A unified theory of structure from motion. BERALDIN, J., BLAIS, F., COURNOYER, L., GODIN, G., AND RIOUX, M. 2000. Active 3D Sensing. Modelli

E Metodi per lo studio e la conservazione dell’architettura storica, 22–46. BLAIS, F., PICARD, M., AND GODIN, G. 2004. Accurate 3d acquisition of freely moving objects. In 3DPVT04,

422–429. BLAIS, F. 2004. Review of 20 years of range sensor development. Journal of Electronic Imaging 13, 231. DHOND, U., AND AGGARWAL, J. 1989. Structure from stereo-a review. Systems, Man and Cybernetics,

IEEE Transactions on 19, 6, 1489–1510. DU, H., ZOU, D., AND CHEN, Y. Q. 2007. Relative epipolar motion of tracked features for correspondence in

binocular stereo. In IEEE International Conference on Computer Vision (ICCV). NAYAR, S., WATANABE, M., AND NOGUCHI, M. 1996. Realtime focus range sensor. IEEE Transactions on

Pattern Analysis and Machine Intelligence 18, 12, 1186–1198. RUSINKIEWICZ, S., HALL-HOLT, O., AND LEVOY, M. 2002. Real-time 3D model acquisition. Proceedings of

the 29th annual conference on Computer graphics and interactive techniques, 438–446. SCHECHNER, Y., AND KIRYATI, N. 2000. Depth from Defocus vs. Stereo: How Different Really Are They?

International Journal of Computer Vision 39, 2, 141–162. WATANABE, M., AND NAYAR, S. 1998. Rational Filters for Passive Depth from Defocus. International

Journal of Computer Vision 27, 3, 203–225. ZHANG, R., TSAI, P., CRYER, J., AND SHAH, M. 1999. Shape from shading: A survey. IEEE Transactions

on Pattern Analysis and Machine Intelligence 21, 8, 690–706. Facegen: http://www.facegen.com, Meshlab: http://meshlab.sourceforge.net/ Vrip, Scanalyze, Plycrunch: http://graphics.stanford.edu/software/vrip/, 3Ds Max: www.autodesk.com/3dsmax Prometheus : http://personal.ee.surrey.ac.uk/Personal/A.Hilton/research/PrometheusResults/index.htmlPrometheus : http://personal.ee.surrey.ac.uk/Personal/A.Hilton/research/PrometheusResults/index.html Headus : Headus : www.www.headusheadus.com/au/3D_.com/au/3D_scanscans/index.html s/index.html TC Square : http://www.tc2.com/what/bodyscan/index.htmlTC Square : http://www.tc2.com/what/bodyscan/index.html Cornell University Body Scan : http://www.bodyscan.human.cornell.edu/scene0037.htmlCornell University Body Scan : http://www.bodyscan.human.cornell.edu/scene0037.html

Thanks to Prof. Allen, Karan, Matei and Paul for your kindly help and support.

Special thanks to our model, Daniel, for his professional, passion and great cooperation.

Any Questions…Any Questions…

~Thank You

(team KIPPA)

Full Body 3D Scanning Team KIPPA Sam Calabrese, Abhishek Gandhi, Changyin Zhou {smc2171, asg2160,...

Documents

Transcript of Full Body 3D Scanning Team KIPPA Sam Calabrese, Abhishek Gandhi, Changyin Zhou {smc2171, asg2160,...