Hosting Millennium Project AIUG Lesley Sharp Manager, Online Strategy 16 November 2012.
-
Upload
lyndsey-cowee -
Category
Documents
-
view
214 -
download
0
Transcript of Hosting Millennium Project AIUG Lesley Sharp Manager, Online Strategy 16 November 2012.
Hosting Millennium ProjectAIUG
Lesley SharpManager, Online Strategy
16 November 2012
Background
SA Government StateNet
SLSA
Issues - 2008-2009
• Millennium server• 7 years old and needed to be replaced• Not compliant with SA Government Security
requirements (database and web server on same server)
• DMZ environment• Even if the web serving module was deployed and
application and database servers separated, approval for SSH access would be refused
• III Support Requirements vs StateNet Support Requirements• III preferred support method = SSH (Port 22)• StateNet = VPN, two-factor authentication and no
split tunnelling• Testing showed that two-factor authentication process
raised other issues for III, and couldn’t realistically be achieved (clash of IP numbers)
INTERNETINTERNET
PATRON
SLSASLSA DMZA
DMZAIII
IIIDMZA
DMZA
SLSASLSA
HSP Co-located
Hosting
HSP Co-located
Hosting HSP Dedicated
Hosting (VS)
HSP Dedicated
Hosting (VS)
HSP Dedicated Hosting (VS)
HSP Dedicated Hosting (VS)
INTERNETINTERNET
STAFF
1. 2. 4.3. 5. 6. 7.Role
Based DMZ
Role Based DMZ
8.
Option 4
• October 2009 – commenced discussions with III about their willingness to host our Millennium system in Emeryville
• Commenced a one month trial from early December 2009 of hosting Millennium with III
Show-stopper
• Serial check-in card!
• Drawing the 84-box check-in card on the screen consistently took 20 seconds
• III staff testing in Emeryville – instantaneous
• III staff tested from Ohio and the response time reduced to 9-14 seconds
• Investigations with Dimension Data to check if packet-shaping of data through StateNet was causing this problem
Show-stopper cont’d
• Tested from different parts of the network with no change in the delay time
• Dimension Data traced the data and advised there was no packet loss and the delay was caused by latency while traversing networks and/or the application itself may be the cause
• End result that Hosting Millennium with III in the USA was not a viable option
• Project closed 23 March 2010
Option 5 – Co-Located Hosting
• SA Government Hosting Panel established 2008
• SLSA issued RFQ in June 2010 for “Un-managed Co-Location” services i.e. Place our new server in rented rack space outside of StateNet and have SA Government DCSS vendor manage the server for us
• No formal response to this RFQ, general lack of rack space in Adelaide at that time
Option 6 – Virtual Server - Hosted
• 9 July 2010 – Issued RFQ for a Virtual Private Server in a dedicated hosting environment
• One response – from Internode
• 19 August 2010 – Customer Agreement established
• 16 September 2010 – Virtual Private Server commissioned
• Hooray – all our problems are over!
Migration
• Opened a new project with III, engaged a project manager
• Held a series of planning teleconferences
• Migration date of 26 October 2010 agreed
• Outage notifications were issued a week prior to staff and customers – advising up to 72 hours outage, with possibility of early release
Issues
• Required to convert from Method I indexing to Method II indexing for Advanced Word Searching before the migration could occur
• III Project Manager unavailable
• III had forgotten that the data would need to be converted from Sun to Linux
• Release 2009A and Linux 64 bit create performance problems
• Unplanned immediate upgrade to Release 2009B• Caused issues with acquisitions data
Issues cont’d
• Firewall not set up correctly, needed changes
• Host name not set up correctly, not fully qualified
• Issues with testing the Enterprise Backup API
• Communications time lag between USA and SA
• SLSA Information Desk staff provided access to the catalogue despite an official “outage” still being in place
Issues cont’d
• When testing commenced, not all staff had the most recent Millennium client
• Staff did not follow the test plan• Other than seeing how quickly the serials check-in
card rendered
• Found it was impossible to get the required number of people testing at the same time
• We were too far down the track to consider rolling back
Issues cont’d
• And as we segued from testing to normal operations we started to encounter
Performance Issues
Disconnections
Issues cont’d
• PC power settings adjusted
• Higher than acceptable network collision rate
• Commumications cupboard re-cabled
• Millennium clients freezing; response times slow
• Upgraded server with memory and CPU
• Ongoing performance issues
• III check VMWare settings
Issues cont’d
• Internode add VMWare Tools
• Internode apply Resource Groups
18 February 2011
Performance issues resolved
Issues cont’d
• Disconnections• Dimension Data - Packet tracing
• StateNet Intrusion Prevention System – change required to bypass rules
10 March 2011
Disconnect problem declared ‘solved’
Other issues
• 16 March 2011• Kernel problems commence• Related to install of Encore Reporter?• Resolved by booting into “old” kernel
• 25 March 2011 (a Friday of course)• Performance issues – slowness and Server Not
Responding messages• This turns out to be caused by an incident at Internode,
unrelated to anything else (phew!)
• No other outages caused by hosting environment
Benefits of hosting
No more StateNet!
Better backup regime
Ability to increase server resources quickly and simply
III can provide support using their preferred method
Have been able to deploy our backlog of modules (RSS, WAM, Encore, Encore Reporter, ERM, Content Cafe)
Can now plan upgrades such as to Sierra with more confidence
Thank you
Lesley SharpManager, Online [email protected]
State Library of South Australia, North Terrace, Adelaide SA 5000Phone: (08) 8207 7250 - www.slsa.sa.gov.au