MOMIRAS Kick-Off Meeting, 28/7/2014 1 Center for Research and Technology Hellas / Information...

MOMIRAS Kick-Off Meeting, 28/7/20141

Center for Research and Technology Hellas / Information Technologies Institute (http://www.iti.gr)

MOMIRAS Project Overview

Dr. Dimitrios TzovarasCERTH/ITI

http://www.iti.gr/

http://www.iti.gr/


Summary of Project - MOMIRAS at a Glance

MOMIRAS aims to provide a highly novel programming environment that will exploit recent advances in mobile 3D sensing and will enable processes like 3D registration, object detection, optimized AR rendering and 3D content-bases search.

MOMIRAS will provide a complete set of tools that from one hand will improve current supported 3D features in Augmented Reality and on the other will further expand them by utilizing novel technologies (based on range image sensing) and algorithms (adaptive scene interactions, interaction simulation engine utilizing audio-visual cues for optimized immersive experience).

MOMIRAS aims to reinforce all key stakeholders in the AR value chain, such as Augmented Reality application developers, content and cultural heritage providers, industrial players e.g. from automotive and manufacturing domains in utilizing new solutions for better marketing and disseminating their assets and products to the end-user community.

Real environment

Augmented reality

Augmented virtuality Virtual

environment

Reality-virtuality continuum

Mixed reality


MOMIRAS Objectives

Objective 1: To develop new AR algorithms that exploit depth sensing for efficient annotation and analysis of the interacting environment.

3D Scene Reconstruction by combining depth and visual cues Continuous and real-time analysis of the interacting environment

Main Partners Involved: CERTH, RTC

Objective 2: Development of a multi-layer software architecture that enables robust object recognition and tracking and efficient object search and retrieval.

Lightweight Processing components (mobile/embedded devices) High Level Processing components (cloud-based)

Main Partners Involved: INTRACOM, CERTH


MOMIRAS Objectives

Objective 3: Advanced User interaction and scene manipulation.

Handled by a simulation engine according to real world physics Basic manipulation functions such as add, remove and move of both

real and virtual ones objectsMain Partners Involved: UPAT, RTC

Objective 4: Head-coupled perspective augmented reality solutions.

Realistic rendering of both real and synthetic objects fully exploiting depth information

Support for different viewpoints including user’s head motion analysis

Main Partners Involved: UPAT, CERTH


MOMIRAS Objectives

Objective 5: Image and 3D geometric in-painting for real world object manipulation.

Real-world objects detection, registration and tracking Implementation of in-painting algorithms (2D and 3D) for supporting

basic functionalities like virtual deletion of a real object

Main Partners Involved: CERTH, RTC

Objective 6: Optimization of AR experience through processing and rendering of multimodal cues.

Audio/visual environment capture and analysis Perform synthetic visual and audio rendering to improve existing AR

rendering algorithms

Main Partners Involved: UPAT, INTRACOM


MOMIRAS Objectives

Objective 7: Demonstration of MOMIRAS in challenging application scenarios.

Cultural Heritage (CH) *o Solution for both personnel of cultural heritage sites and end-

users (visitors)o Advanced tour of a cultural collection

Mobile Value Added Services (M-VAS)*o Expanded AR-enabled City Guide Service (part of the mobile

VAS platform of Intracom)o Explore benefits for Social Networking, Smart Calendar and

Advertising

* Use Cases shall be early refined and agreed by Consortium partners (also to identify potential content providers)


MOMIRAS – Gantt Chart (refined)

Main Changes Performed in duration of WPs/Tasks

• All tasks ended after M15 in the original submission now end on M15

• All reports and milestones up to M15 kept without changes whenever possible

• Total 10 Milestones and 20 Deliverables are anticipated in total by the Consortium

MS1.1 MS2.1 MS2.2

MS3.1

MS3.2

MS5.1

MS4.1MS6.1 MS6.2

MS6.3


MOMIRAS Architecture

WP5

WP2

WP3 WP4

Conceptual Architecture of MOMIRAS mapped to core development Work Packages

• WP2 Leader – RTC• WP3 Leader – RTC

• WP4 Leader – UPAT

• WP5 Leader – CERTH/ITI


MOMIRAS – Pert Diagram


MOMIRAS Anticipated Efforts (highlighted up to M6)

Deliverable Responsible Date Nature/Level

D1.1 “Service Definition and System Specification”

ICOM Month 5 (November 2014)

Report

D1.2 “System Design and Concept Architecture ”

CERTH/ITI Month 9 (March 2015)

Report

WP1

Milestone Responsible Date Comments

MS1.1 “Availability of system specification and provisional system reference architecture”

CERTH/ITI Month 6 (December 2014)

Initial specifications available using UML diagrams




D2.1 “Object detection and segmentation module ”

RTC M12 (June 2015)

Report + S/W

D2.2 “Lightweight compact feature descriptors”

CERTH/ITI Month 12 (June 2015)

Report + S/W

D2.3 “3D Pose estimation, registration and 3D reconstruction”

RTC Month 12 (June 2015)

Report + S/\W

D2.4 “Client SDK infrastructure” RTC Month 14 (August 2015)

Report + S/\W

WP2


MS2.1 “Initial version of the WP2 components available”

RTC M9 algorithms available for testing

MS2.2 “Final version of the WP2 components available”

RTC M12 All algorithms available




D3.1 “Interactions simulation engine” RTC M14 (Aug. 2015)

Report + S/W

D3.2 “Generation and interaction with augmented world data

CERTH/ITI Month 14 (Aug. 2015)

Report + S/W

D3.3 “User interactions and manipulations module ”

RTC Month 14 (June 2015)

Report + S/\W

WP3


MS3.1 “First version of the simulation and interaction engine ready ”

RTC M12 (June 15)

Initial integrated version

MS3.2 “Availability of WP3 Results for Integration ”

RTC M14 (Aug. 15)

All modules tested properly




D4.1 “AR user aware rendering engine”

UPAT M14 (Aug. 2015)

Report + S/W

D4.2 “Algorithms to improve experience and content visualization “

UPAT Month 14 (Aug. 2015)

Report + S/W

D4.3 “In painting module” RTC Month 14 (Aug 2015)

Report + S/\W

WP4


MS4.1 “First version of the AR rendering engine ready”

UPAT M10 (April 15) Including in-painting




D5.1 “Artefact import toolkit ” CERTH/ITI M10 (April 2015)

Report + S/W

D5.2 “Feature extraction algorithms from 3D objects and the MOMIRAS 3D search and retrieval engine”

CERTH/ITI Month 14 (Aug. 2015)

Report + S/W

D5.3 “WEB Service Infrastructure” ICOM Month 14 (Aug 2015)

Report + Prototype

WP5


MS5.1 “Availability of the WEB Service Infrastructure for deployment”

ICOM M14 (Aug 15) AR Services available




D6.1 “Communication Plan including Dissemination and Exploitation”

ICOM Month 3 (Sept. 2014) Report

D6.2 “Use case deployment plans and evaluation”

CERTH/ITI Month 15 (Sept. 2015)

Report (prelim. version by M6?)

D6.3 “Final SDK end-to-end platform” ICOM/RTC Month 15 (Sept 2015)

Report + Prototype

D6.4 “Report on Achieving Dissemination and Exploitation objectives”

ICOM M15 (Sept 2015) Report

D6.5 “Business Model) ICOM M15 (Sept 2015) Report

WP6

Milestone Responsible Date CommentsMS6.1 “Availability of the Communication Plans ”

ICOM Month 3 (Sept. 2014)

Web site, dissemination material, Event/Workshop?

MS6.2 “Availability of the Integrated SDK platform”

ICOM/RTC M15 (Sept. 15)

All components working

MS6.3 “Availability of the Business Model”

ICOM M15 (Sept. 15)

Business/Exploitation Model defined


MOMIRAS – Main Tasks assigned to Partners

INTRACOM Project coordination, including administrative, financial and technical leadership. Aligning the proposed system with expressed user needs (T1.1) towards production

of technical specifications and conceptual system architecture (T1.2). Efficient deployment of the results from WP2 on embedded platforms and mobile

devices as well as means of exploiting cloud infrastructures for enhancing the capabilities and the performance of the MOMIRAS system (T3.1).

In WP4 Intracom will perform the development of the WEB service support infrastructure (T4.4).

Intracom will also provide a back-office support (T5.2) and will lead the exploitation and disemination activties (T5.5).


MOMIRAS – Role of Partners

UPAT Leader of WP4 on AR optimization Active involvement in WP2 and WP3 activities in the field of AR optimized

rendering, head coupled perspective rendering, interaction and simulation engine.

CERTH/ITI Leader of the architecture definition activities (WP1) and of the cloud-

based AR services (WP5) R&D on novel algorithms for 3D object annotation, indexing and retrieval running

on the cloud as services (WP4) Development of low-level feature descriptors for 3D Object recognition and

matching in resource-limited devices such as smart phones and tablets (Task 2.2) Exploit the development of lightweight algorithms enabling compact feature

extraction of 3D objects in non-controlled environments.


MOMIRAS – Main Tasks assigned to Partners

RTC Leader of WP2 : “AR client SDK” and WP3: “Advanced world interaction” Design and development of the SDK infrastructure and state of the are algorithms

for object detection, tracking and segmentation, pose estimation and 3D reconstruction

o Lightweight and for use in android platforms Co-development with UPAT the simulation engine, and also the generation of the

augmented world data, and the user interaction module. In WP4, RTC will develop the in-painting module. RTC will also work on the system specification and final integration and

deployment


MOMIRAS – Expected Outcomes (per WP)

WP2: Lightweight object descriptors for enabling 3D artifact search and retrievalNovel object detection, registration and reconstruction components residing in resource-limited devices (smart phones, etc)WP3: Delivery of innovative Interaction and simulation engines allowing user to interact with content (including virtual content) residing in the real environment WP4: AR rendering algorithms taking into account the environmental context and the motion of the observer2D/3D in-painting algorithms to optimize the rendering of partial 3D content WP5: Unified artefact descriptors comprising of multi-layer information including semantics, quality indicators and multiple 3D extracted featuresEfficient and Effective 3D search engine running on a cloud infrastructureOpen and Modular Reference Web service infrastructure to accommodate the MOMIRAS AR-enabled services.

Technical Development & Innovation(WP2-WP4)

Cloud Infrastructure& Services Deployment

(WP5-WP6)

AR SDK Design & Specs – WP1

MOMIRAS Validation & Assessment

(WP6)

MOMIRAS Kick-Off Meeting, 28/7/2014 1 Center for Research and Technology Hellas / Information...

Documents

Transcript of MOMIRAS Kick-Off Meeting, 28/7/2014 1 Center for Research and Technology Hellas / Information...