1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.
-
Upload
jordan-bradford -
Category
Documents
-
view
214 -
download
0
Transcript of 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.
![Page 1: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/1.jpg)
1
AE@SG Media Grid Initiative
By
A/Prof. Bu-Sung Lee, Francis
Nanyang Technological University
![Page 2: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/2.jpg)
2
Content
BackgroundArchitectureApplications
Maya(with plug-in)PovRayMental Ray
Resource Matching
![Page 3: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/3.jpg)
3
Objectives
Illustration of an ICT application encompassing diverse components in applications, portal & middleware
Application has potential to benefit digital media industry
Complements promotion of Grid Computing in digital media industry
![Page 4: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/4.jpg)
4
Background
Project duration: Nov’2004 till Nov 2006
Initiative of Infocomm Development Authority(iDA) Singapore Hewlett Packard(HP)
Participants School of Information Systems, Singapore
Management University School of Computer Engineering, Nanyang
Technological University Institute of High Performance, A*STAR
![Page 5: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/5.jpg)
5
Background(2)
![Page 6: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/6.jpg)
6
Portal
Media Grid Architecture
iHPC Meta-scheduler
ESCSubmit job
SOAP ESC
ESC
Platform LSF Meta-scheduler
NGOLSF Grid
Submit job
WWW Browser
Maya
submit
iMac
ssh
http
ssh
animator
Toolkits- Data collection- Job monitoring- Results preview- Fault-redundancy (Missing job resubmission)- Input data preprocessing/uncompressing
Applications:- Pixie- Povray- Modeler- MentalRay
Resource Matchmaker
Mental Ray/others
ESC
![Page 7: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/7.jpg)
7
Use scenarioData
Media Portal
Charging model
QM
MAYA/3DS,etc.
Infoservice
Jobservice
othersvcs
Virtualization/Middleware
Layer
data
data
data
data
data
![Page 8: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/8.jpg)
8
Work Packages
WP1 - Point of access for users. Provide real-time collaborative environment & Media Workbench
WP2 - Gridification of animation & image rendering applications
WP3 - Generic middleware for job execution & management
WP3 - Expanding the scope of HP – Quartermaster tool from UDC to Grid for resource virtualization & allocation
WP4 - Globus standards. No development undertaken WP5 - Aims to model usage patterns using NGO/GOG
data & work on market based resource allocations models/biz models
![Page 9: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/9.jpg)
9
Multiple Interfaces for Job Submission
![Page 10: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/10.jpg)
10
Step 1 – Create Model in Maya
![Page 11: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/11.jpg)
11
Step 2 – Job Submission from Maya to the Grid
Each frame is associated with one Grid job.
![Page 12: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/12.jpg)
12
Step 3 – Output Data Collection from Portal
The package contains all resultant TIF files.
![Page 13: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/13.jpg)
13
Speedup Sample
The time to render a RIB file is about 40 seconds.
351 RIB files need about 4 hours for rendering in a single computer.
Clusters hpc-pdpm has 20 CPUs in 10 machines. Cluster melon has 20 CPUs in 10 machines.
By using LSF to submit the jobs to hpc-pdpm and melon, now rending 351 RIB files only need about 14 minutes including the time for data stage in/out. The speedup is about 17.
![Page 14: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/14.jpg)
14
Job Submission from Portal to LSF
The uploaded package includes all rib files, textures and shaders.
![Page 15: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/15.jpg)
15
Check Status for the Most Recently Submitted Job
Progress
![Page 16: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/16.jpg)
16
Portlet of Checking Status for the Rendering Jobs Submitted from Maya and from Portal
![Page 17: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/17.jpg)
17
2. POV-Ray Interactive Rendering
GoalsUse Grid to render complex photo-realistic
objects when a single computer is not sufficient.
Deliver the application through the rich media portal directly to the users, without installing additional software
Perform on-the fly visualization of the rendering process to give the user more control.
![Page 18: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/18.jpg)
18
Architecture overview
Web interface
GRID
Cluster A
Nodes
Cluster B
Nodes
Portal
Proxies
Rendering is performed in parallel for every web user
![Page 19: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/19.jpg)
19
POV-Ray Interactive Rendering
![Page 20: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/20.jpg)
20
POV-Ray Interactive Rendering
![Page 21: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/21.jpg)
21
POV-Ray Interactive Rendering
Rendering in progress
![Page 22: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/22.jpg)
22
POV-Ray Interactive Rendering
![Page 23: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/23.jpg)
23
3. Interactive Shape Modeling App.
Goals Use Grid to work with 3D shapes when a single
computer computational power is not sufficient. Deliver the application through the rich media portal
directly to the users, without installing additional software
Methods Java3D web-based interface to receive user
commands and visualize the model Real-time communication link between computing
nodes and the interface
![Page 24: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/24.jpg)
24
Architecture overview
Web interface
GRID
Cluster A
Nodes
Cluster B
Nodes
PortalWeb clients
Proxies
Rendering is performed in parallel for every web user
![Page 25: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/25.jpg)
25
Task distribution
1. Object is split into several parts
2. The parts are distributed among the processing nodes
3. The results are sent back and combined
4. Users are able to visualize the modelIn their browsers interactively
![Page 26: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/26.jpg)
26
Interactive Modeling App.
![Page 27: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/27.jpg)
27
Interactive Modeling App
![Page 28: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/28.jpg)
28
Interactive Modeling App
![Page 29: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/29.jpg)
29
4. Mental Ray
![Page 30: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/30.jpg)
30
Mental Ray Arch.
Melon020Mentalray client
Melon001Mentalray server
Melon002Mentalray server
Melon003Mentalray server
Melon004Mentalray server
![Page 31: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/31.jpg)
31
Web-Service Enabled Resource Management
Understanding User Requirements
Formal Representation of Grid Resources
page 3August 31, 2003 Grid definition
![Page 32: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/32.jpg)
32
Web-Service Enabled Resource Management
Discovery and Matching of Grid Resources
•Problems Countered in Integration •Information Service
![Page 33: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/33.jpg)
33
Conclusion
Maya Application plug-in and portal submission
AchievementsMental Ray submission through portal.
Integration from Application to
Information management.Integration with LSF
![Page 34: 1 AE@SG Media Grid Initiative By A/Prof. Bu-Sung Lee, Francis Nanyang Technological University.](https://reader035.fdocuments.in/reader035/viewer/2022062517/56649f305503460f94c4aad7/html5/thumbnails/34.jpg)
34
The challenge
Thanks!