Hybrid Cluster Technology Whitepaper
-
Upload
vl-ricketts -
Category
Documents
-
view
224 -
download
0
Transcript of Hybrid Cluster Technology Whitepaper
-
8/8/2019 Hybrid Cluster Technology Whitepaper
1/7
Hybrid Web Cluster Technology Overview
A brief introduction to the technologies in Hybrid Web Cluster.
Hybrid Web Cluster
Technology OverviewA brief introduction to the technologies
present in Hybrid Web Cluster.
Document version: 1.0Date: 5th August 2010
http://www.hybrid-cluster.com/
Page 1 of 7
http://www.hybrid-cluster.com/http://www.hybrid-cluster.com/ -
8/8/2019 Hybrid Cluster Technology Whitepaper
2/7
Hybrid Web Cluster Technology Overview
A brief introduction to the technologies in Hybrid Web Cluster.
Table of Contents
Hybrid Web Cluster Technology Overview.........................................1System Overview.................................................................................3Redundancy..........................................................................................4Scalability.............................................................................................5Management Tools...............................................................................6Contact Information.............................................................................7
Document version: 1.0Date: 5th August 2010
http://www.hybrid-cluster.com/
Page 2 of 7
http://www.hybrid-cluster.com/http://www.hybrid-cluster.com/ -
8/8/2019 Hybrid Cluster Technology Whitepaper
3/7
Hybrid Web Cluster Technology Overview
A brief introduction to the technologies in Hybrid Web Cluster.
System OverviewAt the heart of each Hybrid Web Cluster in-
stallation is an advanced system administrator
which automatically manages the cluster, re-
sponding to hardware or network issues auto-
matically and within seconds so that yourwebsites stay online .Each cluster can be comprised of individual
nodes of either real physical servers or virtualcloud server instances. New cloud server in-
stances can be automatically brought online
by the software to cope with peaks in de-
mand. Websites are automatically and trans-
parently load balanced around the cluster
such that no single server is ever overloaded
and near-live backups are stored on an administrator-configurable number of servers.
This means that as your hosting requirements grow in terms of storage capacity, performance
management and fault tolerance, you dont need to think about how to scale either vertically
or horizontally. Weve done the thinking for you.
C omplete fault tolerance means that if a hard disk,
a server, or even an entire data centre fails, the
software will recognise this, restore data from a
backup no more than 15 seconds old and continue
serving websites as if nothing had happened.
Hybrid Web Cluster is designed to run on FreeBSD
and provide the standard Apache, MySQL and PHP
stack, so normal LAMP applications will run
unmodified and scale from zero resource usage to
requiring a dedicated web and database server allto themselves automatically and instantly.
Through the use of multi-master database replication the system is able to scale beyond this
limit provided that users modify their code to only store variable state data in the database.
Document version: 1.0Date: 5th August 2010
http://www.hybrid-cluster.com/
Page 3 of 7
http://www.hybrid-cluster.com/scalability/http://www.hybrid-cluster.com/scalability/http://www.hybrid-cluster.com/redundancy/http://www.hybrid-cluster.com/http://www.hybrid-cluster.com/redundancy/http://www.hybrid-cluster.com/http://www.hybrid-cluster.com/scalability/http://www.hybrid-cluster.com/scalability/ -
8/8/2019 Hybrid Cluster Technology Whitepaper
4/7
Hybrid Web Cluster Technology Overview
A brief introduction to the technologies in Hybrid Web Cluster.
RedundancyIn order to achieve military-grade fault tolerance,
Hybrid Web Cluster allows you to specify exactly
how many redundant copies of each website are
kept starting at 2 for basic business redundancy.
This means that each website will be stored on
two servers at any one time, so if either one of
them fails you will not experience any downtime.
However, the redundancy level (or redundancyinvariant) is fully configurable and limited only
by the number of available cluster nodes you
could elect to have each website continuously backed-up to every single node in the cluster.
Hybrid Web Cluster has no single point of failure. The redundancy invariant guarantees that
data is sufficiently replicated at all times to cope with the failure of many nodes, and upon
failure the nodes act quickly to re-establish the redundancy. In other words, your data is always
safe, and failures are dealt with quickly and automatically.
Hybrid Web Cluster makes use of the ability of the
ZFS filesystem to take a snapshot of a filesystemand then send only the changes that have been
made over the network to another machine and
have the filesystem rebuilt. Each website or
database is divided onto its own ZFS filesystem,
and these filesystems are monitored for changes
when a change happens (at most every 15
seconds), our filesystem daemon creates a
snapshot of the filesystem and sends it to each of
the other cluster nodes that hold a backup copy of that particular website or database.
This means that when a failure occurs, a snapshot of each website on the failed server should be
available with data no more than 15 seconds old. The cluster uses this data to automatically and
instantly restore service with seconds of a failure rather than hours.
Document version: 1.0Date: 5th August 2010
http://www.hybrid-cluster.com/
Page 4 of 7
http://www.hybrid-cluster.com/http://www.hybrid-cluster.com/ -
8/8/2019 Hybrid Cluster Technology Whitepaper
5/7
Hybrid Web Cluster Technology Overview
A brief introduction to the technologies in Hybrid Web Cluster.
ScalabilityBecause Hybrid Web Cluster can run in a cloud
computing environment, the software can decide to
automatically spin-up new servers from the
underlying cloud providers infrastructure, if it sees
that a particular website needs to scale. The cluster
will reconfigure itself within seconds of a spike in
traffic so that two big servers (one web, one
database) are dedicated to hosting your popularwebsite, and the other servers carry on sharing all your
other sites between them.
The fact that the cluster can dynamically change its
own size and rapidly cope with changes in demand in
this way is all down to Hybrid Web Cluster's unique
'Site-Juggling' load balancing algorithm.
Site Juggling works by transparently moving websites
between servers to cope with changes in demand.
When a new node is added to the cluster, the rest ofthe cluster reacts by gradually moving over some of
the websites it's hosting until the load is evenly
balanced again.
Responsible for this load balancing and dynamic sizing of the cluster is the StateMachine this
is our name for our home-grown cluster management daemon. This program runs on each
cluster node and monitors the load on each website or database, decisions about when to move
a website from one node to another are made on the nodes themselves based on global state
information which is broadcast over Spread.
The limitation of this approach is that the smallest unit of work is either a single website orsingle database this means that whilst maintaining compatibility with existing LAMP web
applications, the system cannot scale beyond a single website requiring its own separate
dedicated web and database servers. In order to scale beyond this limitation we use multi-
master database technology, although this requires applications be modified to only store
variable state data in the database.
Document version: 1.0Date: 5th August 2010
http://www.hybrid-cluster.com/
Page 5 of 7
http://www.hybrid-cluster.com/cloud-computing/http://www.hybrid-cluster.com/cloud-computing/http://www.hybrid-cluster.com/http://www.hybrid-cluster.com/cloud-computing/http://www.hybrid-cluster.com/cloud-computing/http://www.hybrid-cluster.com/cloud-computing/http://www.hybrid-cluster.com/ -
8/8/2019 Hybrid Cluster Technology Whitepaper
6/7
Hybrid Web Cluster Technology Overview
A brief introduction to the technologies in Hybrid Web Cluster.
Management ToolsAt the core of Hybrid Web Cluster is a
web hosting control panel responsible
for adding sites and databases,
managing e-mail settings, automatic
installation ofcommon applications
such as WordPress. The control panel
offers a single point of access allowing
you to make configuration changesaffecting the whole cluster in what
appears to be one centralised location
(of course the control panel is
resistant to failure as any website
hosted on the cluster would be).
Although Hybrid Web Cluster itself is almost completely self-managing, it is still useful and
interesting to be able to see whats going on in the cluster, live, as it happens. Within the
cluster administration control panel, a powerful set of visual representations of the cluster
allow you to see which servers are hosting which websites, how much load each individual node
or website is experiencing at any given time, as well as allowing you to actually watch as
websites are sitejuggledaround the cluster.
The web cluster control panel is a modern
PHP web application using jQueryUI and
AJAX to provide a responsive and easy-to-
use interface, all changes to the cluster are
made via StateMachine, or YakD (the
automated system administration daemon)
these two programs together are
responsible for rolling out configuration
changes to cluster nodes and keeping
configurations up-to-date as websites are
juggled around the cluster.
Document version: 1.0Date: 5th August 2010
http://www.hybrid-cluster.com/
Page 6 of 7
http://www.hybrid-cluster.com/application-support-list/http://www.hybrid-cluster.com/application-support-list/http://www.hybrid-cluster.com/load-balancing/http://www.hybrid-cluster.com/http://www.hybrid-cluster.com/application-support-list/http://www.hybrid-cluster.com/load-balancing/http://www.hybrid-cluster.com/ -
8/8/2019 Hybrid Cluster Technology Whitepaper
7/7
Hybrid Web Cluster Technology Overview
A brief introduction to the technologies in Hybrid Web Cluster.
Contact InformationMike Smithson
+44 (0)845 508 1116
http://www.hybrid-cluster.com/contact/
Document version: 1.0Date: 5th August 2010
http://www.hybrid-cluster.com/
Page 7 of 7
http://www.hybrid-cluster.com/contact/mailto:[email protected]://www.hybrid-cluster.com/http://www.hybrid-cluster.com/contact/mailto:[email protected]://www.hybrid-cluster.com/