m04res02

EMC Avamar Installation and Configuration Lab Guide

Copyright 2010 EMC Corporation. All Rights Reserved. Version E Page 1 of 26

Lab Exercise 4-1: Node Replacement

Purpose:

In this lab, you will incorporate a Spare storage node as an active member of the Avamar server to replace a node that failed.

Objectives: In this lab you will Fail a node Perform data recovery Decommission the failed node Add a replacement data node

References: EMC Avamar Installation and Configuration Student Guide

DISCLAIMER: The node replacement procedures in this lab are provisional and should not be used at a customer site.



Desired Architecture Simulate storage4 failure. Add storage5 as a replacement node.

utilitynode

192.168.100.2

Avamar Utility Node

Avamar 1X4 Server Configuration

storage0

192.168.100.3

Avamar Data Node

storage1

192.168.100.4

Avamar Data Node

storage2

192.168.100.5

Avamar Data Node

storage3

192.168.100.6

Avamar Data Node

storage4

192.168.100.7

Avamar Data Node

(Spare)



Lab Exercise 4-1 Part 1: Fail a node

Step Action

1 Open a PuTTY session to your utility node by clicking the PuTTY icon.

Login as user root with password: changeme.



Step Action

2 Type: nodenumbers Make note of the output

3 Start anAvamar Administrator in Windows, click the Avamar Administrator shortcut on the desktop or from the Windows Start menu. Enter the following information: User Name: MCUser Password: MCUser1 Domain Name: / Administrator Server: utilitynode Then, click Log On. Accept any JRE warnings.



Step Action

4 Click on Server. Observe that all nodes are online.

5 Close all Avamar Administrator windows

6 Open a PuTTY session to storage3. Accept any warnings and login as user root

password: changeme.

7 Kill the storage nodes gsan process in order to cause the node to fail. Type: killall gsan

8 Close the PuTTY session to the storage node.



Step Action 9 To start Avamar Administrator in Windows, click the Avamar Administrator

shortcut on the desktop or from the Windows Start menu. Enter the following information: User Name: MCUser Password: MCUser1 Domain Name: / Administrator Server: utilitynode Then, click Log On. Accept any JRE warnings.

Note: It may take several minutes for the log on; please be patient.



Step Action 10 You will see an Avamar Alert popup that informs you that a node has gone offline.

Click View Details.

Study the information in the Event Detail window. Then, click Close.

Click OK on the Avamar Alert popup to close the window.



Step Action 11 From the Avamar Administrator menu, click Server.

The following screen appears. Note that Node 0.3 is red (offline).

Close the Avamar Administrator Server window. Close the Avamar Administrator menu.

End of Lab 4-1 Part 1.



Lab Exercise 4-1 Part 2: Perform Recovery When Avamar Server is Running in Degraded Mode Step Action

1 On your client machine, open Windows Explorer and navigate to C:\WINDOWS\Fonts.

Delete the ariblk.TTF file.



Step Action

2 Launch Avamar Administrator. Click Backup & Restore.



Step Action

3 Expand the tree in the left pane and select your client. Click the Select for Restore tab in the right pane. Browse through your backup and select the C:\WINDOWS\Fonts\ariblk.ttf file.



Step Action

4 Select Actions > Restore Now from the menu bar. Then, click OK to begin the restore.

Click OK to confirm.



Step Action

5 Monitor the progress of the restore by clicking Navigation > Activity. Verify that the restore completes successfully.

6 Verify that the ariblk.ttf file was restored by checking the contents of the C:\WINDOWS\Fonts folder on your client.

End of Lab 4-1 Part 2.



Lab Exercise 4-1 Part 3: Perform Node Replacement

Step Action

1 Open a PuTTY session to your utility node and login as user root password changeme. Type: nodenumbers and study the output.



Step Action

2 In Avamar Administrator, select Navigation Server, and click the Checkpoint Management tab.

Ensure that a checkpoint that is acceptable as a suitable rollback point (if required) exists and has been validated (hfs checked).

3 Back in the PuTTy session for the utilitynode, change users to admin. Type: su - admin Load the admin SSH keys. Type: ssh-agent bash ssh-add ~/.ssh/admin_key passphrase: P3t3rPan

Suspend all Avamar client activities. Type: avmaint suspend Suspend all maintenance jobs. Type: avmaint sched suspend --avamaronly



Step Action

4 Type the following to display information about the spare node: nodedb print --nodes=0.spare.0

5 Update the /usr/local/avamar/var/probe.xml file using the nodedb command as shown below. Verify that the info on node 0.3 in the probe.xml file matches the node that has failed: nodedb print --nodes=0.3 Remove the entry for the spare node: nodedb delete node --node=0.spare.0 Replace the failed node with a storage node using the same information as the spare node: nodedb add node --addr=192.168.100.7 --replace --node=0.3 --type=storage



Step Action

6 Verify that the probe.xml file contains information reflecting the replaced node: nodedb print

7 Exit out of the admin user shell (twice).



Step Action

8 Change users to user dpn. Type: su - dpn Load the dpn SSH keys, type: ssh-agent bash ssh-add ~/.ssh/dpnid export SYSPROBEUSER=root

9 To synchronize time on the new storage node, type: asktime

Enter y to use the timeservers specified.



Step Action 10 Enter n to the question Do you have a dedicated Avamar Subnet?

11 Enter y to use the previously specified time zone.

12 Enter y to configure the default set of nodes.



Step Action 13 Enter y to proceed with the installation.

14 Check the time displayed and confirm by entering y.



Step Action 15 Enter y to watch for time synchronization.

16 This next step takes about 10 minutes. After completion, enter y to see the results.



Step Action 17 Results:

(0.3) ssh -x [email protected] '/usr/sbin/ntpq -pn 2>&1' remote refid st t when poll reach delay offset jitter ============================================================================== *10.127.175.1 10.127.210.201 5 u 22 64 377 99.884 30.536 41.489 +10.127.175.2 10.127.175.1 6 u 13 64 377 99.964 21.053 36.499 192.168.100.2 .INIT. 16 u 16 64 0 0.000 0.000 4000.00 192.168.100.3 .INIT. 16 u 16 64 0 0.000 0.000 4000.00 Using /usr/local/avamar/var/probe.xml (0.s) ssh -x [email protected] '/bin/date' (0.0) ssh -x [email protected] '/bin/date' (0.1) ssh -x [email protected] '/bin/date' (0.2) ssh -x [email protected] '/bin/date' (0.3) ssh -x [email protected] '/bin/date' Thu Mar 18 14:41:46 EDT 2010 Thu Mar 18 18:41:46 UTC 2010 Thu Mar 18 18:41:46 UTC 2010 Thu Mar 18 18:41:46 UTC 2010 Thu Mar 18 18:41:46 UTC 2010 -------------------------------------------------------- Done. NOTES: * Time on this host is: Thu Mar 18 14:41:46 EDT 2010

18 Exit out of the dpn user shell (twice).



Step Action 19 Change user to admin and load the admin SSH keys. Type:

su - admin ssh-agent bash ssh-add ~/.ssh/admin_key passphrase: P3t3rPan

20 Note: This task must be performed as the admin user.

Rebuild the new node using the rebuild.node command. Type: rebuild.node --nodes=0.3 --clean --tag=0.3



Step Action 21 Launch a new PuTTY session to your utility node and login as user root password

changeme. Monitor the status of the new node by typing: status.dpn

Ensure that the new node comes online.



Step Action 22 Go back to the SSH session running as the admin user and type: nodenumbers

23 In the PuTTY session to your utility node (host 1) logged on as root, monitor the status of your nodes by typing: status.dpn



Step Action 24 Restart server operations and scheduled cron jobs. In the PuTTY session running as the

admin user, type: avmaint resume avmaint sched resume --avamaronly

25 Exit all PuTTY sessions.

26 View the Server Monitor in Avamar Administrator to see that the new node is online.

End of Lab 4-1 Part 3 End of Lab 4-1.

m04res02

Documents

Transcript of m04res02