Key Laboratory of Machine Perception (MOE), Peking University Applications of 3D Digital...
-
date post
23-Jan-2016 -
Category
Documents
-
view
227 -
download
3
Transcript of Key Laboratory of Machine Perception (MOE), Peking University Applications of 3D Digital...
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Applications of Applications of 3D Digital Technologies 3D Digital Technologies in Archaeology in Archaeology and Digital Heritageand Digital Heritage
Hongbin Zha ( 査紅彬)3D Visual Computing and Robotics Gro
upKey Lab on Machine Perception (MOE)
Peking University, China
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
OutlineOutline
IntroductionIntroduction Task definition of 3D digitizationTask definition of 3D digitization ApplicationsApplications Process pipeline and challengesProcess pipeline and challenges
Applications in Digital HeritageApplications in Digital Heritage Digital Longmen Grottos Digital Longmen Grottos (竜門石窟)(竜門石窟) ProjecProjec
tt Computer-assisted line drawingComputer-assisted line drawing Digital restorationDigital restoration
Maijishan Grottos Maijishan Grottos (麦積山石窟)(麦積山石窟) Project and Project and OthersOthers
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
3D Modeling of Real 3D Modeling of Real Objects Objects
mini-statue
mesh rendered mesh
Copying machineCopying machine
3D digital copy3D digital copy
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Objectives of 3D DigitizationObjectives of 3D Digitization
Geometry Digitization:Geometry Digitization: Cover the whole surfaces Cover the whole surfaces
of an objectof an object Describe accurate shape Describe accurate shape
and structure detailsand structure details Digitization of Other Digitization of Other
Attributes:Attributes: Textures and colorsTextures and colors Transparency and Transparency and
glossiness glossiness …… ……
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Characteristics of 3D Modeling Technologies
OObjects:bjects: WWith complex geometrical and topological ith complex geometrical and topological
structures, with dynamic deformation or structures, with dynamic deformation or movements -- architectures, human bodies movements -- architectures, human bodies or outdoor scenes. or outdoor scenes.
GGeometry Processing: eometry Processing: MModel simplification, data compression odel simplification, data compression
and transmission for large-scale scenes.and transmission for large-scale scenes.
AAutomatic Modeling:utomatic Modeling: 3D laser scanning, hybrid imaging 3D laser scanning, hybrid imaging
systems, systems, ddata acquisition using mobile ata acquisition using mobile platplatformsforms..
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Break down Wall between Real and Cyber Worlds
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Digital Museum and Digital Digital Museum and Digital HeritageHeritage
Digitization for museum Digitization for museum and heritage objectsand heritage objects
Interactive walkthrough Interactive walkthrough of heritage sitesof heritage sites
�Digital recording and Digital recording and documentation documentation �Digital exhibitionDigital exhibition�Digital reconstruction, Digital reconstruction, measurement and analysismeasurement and analysis�Remote visiting and Remote visiting and educationeducation
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Digital recording and Digital recording and duplication of heritage articlesduplication of heritage articles
Digital restoration of heritage Digital restoration of heritage relicsrelics
Behavior analysis of life forms Behavior analysis of life forms based on skeleton fossilsbased on skeleton fossils
Digital recording of Digital recording of archaeological excavation sitesarchaeological excavation sites
Virtual reconstruction of Virtual reconstruction of historic scenerieshistoric sceneries
3D Digitization Assisted 3D Digitization Assisted Archaeology and Heritage Archaeology and Heritage
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
NRC Tylosaurus sp.Mosasaur fossils
Sta
nfo
rd
David
IBM Pietà Tokyo Buddha
Heritage Related WorkHeritage Related Work
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Pipeline for 3D Imaging and Modeling
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
3D Scanners3D Scanners
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
3D Photography or 3D TV
Develop devices that can capture 3D imageDevelop devices that can capture 3D images or videos as general camerass or videos as general cameras image-based modelingimage-based modeling camera arrays or domescamera arrays or domes hybrid imaging systems hybrid imaging systems
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Complexity in 3D shapes Complexity in 3D shapes and topological and topological structuresstructures
Discontinuities in data Discontinuities in data parameterization and parameterization and irregularities in samplingirregularities in sampling
Well-established digital Well-established digital processing tools (FFT, processing tools (FFT, Wavelet analysis etc.) can Wavelet analysis etc.) can not be taken into good not be taken into good use.use.
Difficulties in 3D Digital Difficulties in 3D Digital Geometry ProcessingGeometry Processing
sound
image
video
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Big Sizes of 3D Models
3D models become 3D models become larger and larger: larger and larger:
High requirements for High requirements for geometrical and other geometrical and other attribute detailsattribute details
Complexities of objects Complexities of objects to be modeled to be modeled
Advance in 3D data Advance in 3D data acquisition technologiesacquisition technologies
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
OutlineOutline
IntroductionIntroduction Task definition of 3D digitizationTask definition of 3D digitization ApplicationsApplications Process pipeline and challengesProcess pipeline and challenges
Applications in Digital HeritageApplications in Digital Heritage Digital Longmen Grottos Digital Longmen Grottos (竜門石窟)(竜門石窟) ProjecProjec
tt Computer-assisted line drawingComputer-assisted line drawing Digital restorationDigital restoration
Maijishan Grottos Maijishan Grottos (麦積山石窟)(麦積山石窟) Project and Project and OthersOthers
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Digital HeritageDigital Heritage
19
Display Fundamental Fundamental ResearchResearch
An input imageReconstruction result
• Modeling targets with complex geometry• Computation on 3D models• ……
Transformation
Digitization Projects Virtual Tourism of Maiji Grottoes (Collaborated with Top
pan) Surveying and Mapping Caves (Collaborated with Longm
en Grottoes) ……
Data Acquisition
Laser Scanners
Laser ScannerGPS IMU
Video Camera
Laser Scanner
Geometric reconstruction & Texture generation
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Projects
Beijing
XiangTang Shan Grottoes
LongMen GrottoesMaiJiShan Grottoes
Five-Layer Tower
ChongQing DiaoYu Town
YangCan Tomb
NanYueWang Museum
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Joint Projects with Longmen Grottoes
Computer assisted line draComputer assisted line drawing for archaeological reswing for archaeological researchearch
Virtual restoration of heritVirtual restoration of heritage sites and relicsage sites and relics
Digitization standards for gDigitization standards for grotto scanning and modelirotto scanning and modelingng
““Digital Longmen GrottoeDigital Longmen Grottoes” projects” project
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Surveying and Mapping in Surveying and Mapping in Archaeology ResearchArchaeology Research
The purpose of surveying and mapping is to measure The purpose of surveying and mapping is to measure and record accurate, readable and parallel-projection and record accurate, readable and parallel-projection feature curvesfeature curves of cultural heritage. of cultural heritage.
Archeological reports: Archeological reports: Photos: record current situations and information to Photos: record current situations and information to
infer knowledge such as its producing dateinfer knowledge such as its producing date Texts: describe all explicit Texts: describe all explicit
known informationknown information line drawings: reflect line drawings: reflect
accurate, vivid and feature accurate, vivid and feature emphasized figuresemphasized figures
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Line Drawing in Archeology Traditional methods:Traditional methods:
�Set baseline and grid, then measure using rulersSet baseline and grid, then measure using rulers�Disadvantages:Disadvantages:
Low accuracyLow accuracyTime-consumingTime-consumingDifficult to draw lines on multiple sections and Difficult to draw lines on multiple sections and
large areaslarge areas
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
3D Digitizing 3D Digitizing PipelinePipeline
Pre-Processing
Post-Processing
Registration
Merging
Display
Measurin
g
Restoratio
n
Application
s
+ =
……
3D Scanning
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
3D Scanners in Use3D Scanners in Use
Cyrax 2500
RIEGL LMS-Z210i
OKIO
VIVID 910
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
3D scanner
Data Acquisition
Get 3D range information of surface
s.
Scan an object from different view p
oints.
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Pre-processing
Denoising
Decimation
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Registration
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Data Merging
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Post-processing
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Line Drawing
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
3D Model Aided Line Drawing
Photo
Rendered 3D ModelLine Drawing Based on
3D Model AdvantageAdvantage
�Precise measurementPrecise measurement�FastFast�Parallel projection, no distortionParallel projection, no distortion�Multiple sections from any positionMultiple sections from any position�Line drawing for large scale scenesLine drawing for large scale scenes
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Sections from MultipleSections from Multiple PositionsPositions
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
LeiGuTai, LongMen Grottoes
Pho
to
8.5m
20m
Traditional Traditional methodmethod
480 hours480 hours
Our Our methodmethod
8 hours (for modeling) 8 hours (for modeling) ++
24 hours (for line 24 hours (for line drawing)drawing)
Saved Saved timetime
448 hours448 hours
Para
llel projection m
ap
Line drawing
Efficiency ofEfficiency of 3D 3D Model Aided Model Aided Line DrawingLine Drawing
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Buddha statue in the north cave
Photo Parallel projection map Line drawing
2.55m
0.9m
1.3m
1.8m
TraditioTraditional nal methodmethod
360 hours360 hours
Our Our methodmethod
4 hours (for 4 hours (for modeling)+32 hours modeling)+32 hours (for line drawing)(for line drawing)
Saved Saved timetime
324 hours324 hours
Efficiency ofEfficiency of 3D 3D Model Aided Model Aided Line DrawingLine Drawing
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Para
llel p
roje
ction
m
ap
Lin
e d
raw
ing
Ph
oto
Bodhisattva statue in the middle cave
0.65m
0.4m
0.5m
1.85m
TraditioTraditional nal methodmethod
96 hours96 hours
Our Our methodmethod
4 hours (for 4 hours (for modeling)+40 modeling)+40 hours (for line hours (for line drawing)drawing)
Saved Saved timetime
52 hours52 hours
Efficiency ofEfficiency of 3D 3D Model Aided Model Aided Line DrawingLine Drawing
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
0.65m
0.40m
0.5m
1.85m
Automatic Line DrawingAutomatic Line Drawing
image from 3D dataimage from 3D dataautomatically automatically
produced sketchproduced sketchmodel assisted drawingmodel assisted drawing
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
A large scene in Longmen Grottoes
Automatic Line DrawingAutomatic Line Drawing
A Buddha statue
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Digital Restoration
Budda in GuYang Cave, LongMen Grottoes
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Bodhisattva in HuoDing Cave, LongMen Grottoes
Digital Restoration
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
LongMen Grottoes
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
MaiJiShan Grottoes A joint project: Toppan Printing Co., Wased
a Univ., Peking Univ. Objective: to develop advanced 3D modelin
g, rendering, displaying and interaction methods for digital heritage
A case study: Maijishan Grottos – production of 3D video for the Cave #123
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
MaiJiShan Grottoes
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Budda in South Cave, XiangTangShan Grottoes
Digital Restoration
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Statues in South Cave, XiangTangShan Grottoes
Digital Restoration
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
XiangTangShan Grottoes
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
ConclusionsConclusions
3D digital technology is very useful for 3D digital technology is very useful for preservation, research, and knowledge preservation, research, and knowledge dissemination for cultural heritagesdissemination for cultural heritages
It requires collaboration among different It requires collaboration among different disciplines: archaeology, cultural heritage, disciplines: archaeology, cultural heritage, information science and technology, …information science and technology, … Well-defined objectivesWell-defined objectives High-quality resultsHigh-quality results Useful in some aspects for the applicationUseful in some aspects for the application
How to find research issues from the How to find research issues from the application work?application work?
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Thank You!Thank You!
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
TombTomb� 400m400m22
ObjectsObjects� 2020
GuangZhou NanYueWang Museum
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Digital Archeology Report
Photo
Database managementDatabase management�Photos, 3D models, and relevant textsPhotos, 3D models, and relevant texts�From large scenes to single statuesFrom large scenes to single statues�Basic operations Basic operations 3D measurement and analysis 3D measurement and analysis Section from any positionSection from any position
3D View Measurement
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Digital Museum
3D scanning and picturing
3D modeling
Texture mapping Virtual display
Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University
Other Cases