Distributed Video Rendering using Blender, VirtualBox, and BOINC. Christopher J. Reynolds.
description
Transcript of Distributed Video Rendering using Blender, VirtualBox, and BOINC. Christopher J. Reynolds.
Distributed Video Rendering using Blender, VirtualBox,and BOINC. Christopher J. Reynolds.Centre for Parallel Computing,University of Westminster.
Material:
• The general concept and motivations.
• Choosing a system virtualization technology.
• Interactions with the hypervisor.
• Final comments.
Background: University of Westminster
• Approx 1600 node Windows & BOINC based DG.
• Runs applications from Enabling Desktop Grids for E-Science (EDGeS).
• 20 node GPU cluster under construction to run further EDGeS/DEGISCO applications.
• For most of 2009 we ran a public video/animation rendering service on a dedicated cluster (~256 Nodes).
Blender: http://www.blender.org
• Rastorization based rendering of frames; a Bag of Tasks (BoT).
• Fits the DG model so move from cluster to DG.
• Uses Blender, a well known animation/image rendering suite.
• Q: What are Blender’s requirements? • A: Python.
• Q: What other considerations are required for a public service?• A: Python scripts can be embedded within Blender input files (Security)
• Q: How to address these issues?• A: Virtualization
Existing Virtualization Solutions
• Existing solutions use either process virtualization or system virtualization.
• Process virtualization (using Entropia VM) proposed by Calder et al [1] is tied to Windows as the host OS.
• VM Wrapper: Generic but more complex and incomplete (Windows Host Support).
• Marosi et al described proof of concept using QEMU based on features [2] .
• Q: QEMU/KQEMU now obsolete on Windows, so what next? • A: Look at latest feature set and choose something else for a
Lightweight solution.
QEMU/KQEMU vs VirtualBox
Feature VirtualBox QEMU KQEMU
X86 guest only. Yes No Yes
Requires admin privileges to run.
No – Add user to allow execution of
VBox DCOM service.
No No
Requires admin to install.
Yes No Yes
CPU performance overhead (x86) [3]
20%(SP)10%(FP)
Huge (emulation) 100%(SP)30%(FP)
Supports disk overlay/cow images.
Yes Yes Yes
Guest to Host File I/O (Without Network)
Yes (Guest Additions)
No No
Monitor/Hypervisor interface.
COM Sockets Sockets
Windows support Yes Yes (Partial) Obsolete
Solution: A Lightweight VM Controller.
Host/DG Node
DG (BOINC) Server
BOINC Core Client
VM (Guest)
Disk image (guest OS), VM config file, inputs.
Instantiate VMand transfer
inputs
React to messages: suspend, resume,
abort
Transfer outputs
VM Controller (standard BOINC job)
Start
1
2
3
45
6
VBox Hypervisor
Transfer outputs
VirtualBox Hypervisor/Monitor ControlVM Controller (C++ Boinc application)
• Win32 CreateProcess() or fork() to VboxManage cmdln tool.
• COM preferable.
(COM)
VM Controller
• Two Threads.• Worker launches unzipper (lzma), prepares VM config files, creates I/O
shared directories, polls for Job completion.• Monitor thread deals with messages and VM control.
VM Controller (C++ Boinc application)
Worker Thread
Monitor Thread
VM Worker Script: Blender
echo 15 > /proc/$$/oom_adjecho "Setting oom_adj to 15 for PID=$$"
case "$1" in start)
mkdir /mnt/sharemount -t vboxsf Shared /mnt/share
mkdir /mnt/share/Outputs
swapon /dev/sdaswapon /dev/sdb
START_FRAME=`cat /mnt/share/input.txt | head -1 `END_FRAME=`cat /mnt/share/input.txt | head -2 |tail -1 `
sudo –u blender_user sh -c "blender -b /mnt/share/input.blend -o /mnt/share/Outputs/Frame-# -F PNG -x 0 -s $START_FRAME -e $END_FRAME –a”
echo $? > /mnt/share/Outputs/blender_return.log
echo "Finished Rendering Application";;
Further Comments
• We adhere to the “one VM per Workunit” model.
• Our BOINC user account has restricted permissions.
• This means VirtualBox.xml and MachineDefinition.xml are loaded from C:\Windows\system32\config\systemprofile\.VirtualBox
• Single xml file location restricts us to one WU per node.
• Proof of concept, though proposed roll out soon (late 2010).
References
[1] Calder, B., Chien, A., & Yang, J. W. D. (2005). The entropia virtual machine for desktop grids. In VEE ’05: Proceedings of the 1st ACM/USENIX international conference on Virtual execution environments (pp. 186–196).
[2] Marosi, A., Kacsuk, P., Fedak, G., & Lodygensky, O. (2010). Sandboxing for Desktop Grids using Virtualization. In The 18th Euromicro International Conference on Parallel, Distributed and Network-Based Computing.
[3] P. Domingues and F. Araujo and L. Silva. (2009) Evaluating the performance and intrusiveness of virtual machines for desktop grid computing. In IPDPS '09:Proceedings of the 2009 IEEE International Symposium on Parallel & Distributed Processing.
Questions?