03_Aggr_Defrag_v2

Module 3. Defragmentation

Defragmentation & Reallocatation

Module Objectives

Upon completion of this module, you will be able to:– Describe the new defragmentation feature and

associated commands– Describe the differences between a normal and full

reallocation– Set an allocation schedule on a file– Set, quiesce, restart and view the status with the

new reallocate command

New WAFL Defragmenter

Prior to DOT 7.0– wafl scan measure_layout– wafl scan reallocate

DOT 7.0– Integrated fragmentation/reallocation management– CLI and FilerView– Scheduled defragmentation/reallocation– Status/Inquiry – Improved performance

Reduce amount of data moved during defragmentation.Reallocation scan adjusts speed based on system load

Normal vs. Full Reallocation

Normal Reallocation– Use reallocate

• Coalesces fragmented data into larger chunks• Improves read performance• Lightweight, scheduled• Performed on large files or LUNs

Full Reallocation– Use reallocate –f– Or wafl scan reallocate

• Rearranges data more aggressively• Performed on files, LUNs or Volumes• One-time operation• e.g. when adding a new RAID group

Operational Comparison, Full

New Full Reallocation, unfragmented

Data Data

Data Data Data Data Data

Old Reallocation, unfragmented

Data Data

The reallocate command

Manages reallocation jobs– reallocation jobs are persistent, scheduled by default.

• Start new job: reallocate start• Stop existing job: reallocate stop• Check status: reallocate status• Manage schedules: reallocate schedule

Available in FilerViewUser creates a LUN, wants automatic scheduled reallocation– CLI example:

lun create –s 50g /vol/myvol/lun1reallocate start /vol/myvol/lun1

Scan/Reallocate/Rescan

Three phases to a reallocate job1. Scan to measure layout/optimization2. If too fragmented, perform reallocation3. Rescan to see how we did

– Repeat as necessary (If the lun/file is heavily used, the above process could result in a continuous scanner (This is OK)» If fragmentation is below threshold, go to

interval or schedule

Reallocation Scheduling

Reallocation jobs are scheduled to run periodically.Three kinds of job scheduling– Interval (default)

• Default = 1 day between scans– Specific schedule

• Use “*” for all, “-” for range, “,” for list– Do not use “*” for minutes

– None (once only)• reallocate start –f <path>• wafl scan reallocate <path>

Reallocate Scheduling

Scan Saturday and Sunday at 9:00 P.M.

0,6*210

Scan M-F at 11:00 P.M.1-5*230

Scan every 10 minutes***0,10,20,30,40,50

Scan at 10:00 A.M. on the 15th of the month

*15100

Scan at 11:00 P.M. every Saturday

6*230

ResultDay of week

Day of month

HourMinute

Reallocate command

reallocate start /vol/flex1/abc

reallocate status -v

By default, starting a reallocation on a file or LUN will use an interval of 1 day, and set the fragmentation threshold to 4

Fri Nov 5 20:17:11 GMT [wafl.scan.start:info]: Starting WAFL layout measurement on volume flex1.Reallocation scan will be started on '/vol/flex1/abc'.Monitor the system log for results.

Reallocation scans are on/vol/flex1/abc:

State: IdleFlags: maybe_realloc,repeat

Threshold: 4Schedule: n/aInterval: 1 day

Optimization: 1

To change the default interval, you can use the –i <#> option.

To change the default threshold value, you can use the –t <#> option.

Use the –o option if you want to run the job only once. This has the same action as using the

‘reallocate start –f <path>’ command.

If you do not want to check the optimization of the file against the threshold value, you can use

the –n option to run the command regardless if the file is fragmented or not.

Note that if you want to show all reallocation schedules, you can use the –v command. You can

also view specific schedules by adding the <path> to the end of the command.

Reallocate command

reallocate schedule -s "0 23 * 6" /vol/flex1/abc


Notice that the interval is now set to N/A, as there is now a scheduled reallocation time for the file.

Reallocation for '/vol/flex1/abc' will be scheduled for '0 23 * 6'.

/vol/flex1/abc:State: IdleFlags: maybe_realloc,repeat

Threshold: 4Schedule: 0 23 * 6Interval: n/a

Optimization: n/a

Reallocate command

reallocate schedule –d /vol/flex1/abc


Now we’ve deleted the schedule and returned to using an interval value for our schedule.

Reallocation schedule for '/vol/flex1/abc' will removed.

Reallocation scans are on/vol/flex1/abc:

State: IdleFlags: maybe_realloc,repeat

Threshold: 4Schedule: n/aInterval: 1 day

Optimization: n/a

Reallocate command

reallocate stop /vol/flex1/abc


Now we’ve deleted the reallocation itself from this jobs.

Fri Nov 5 20:37:12 GMT [wafl.reallocate.scan.remove:info]: Reallocation scan for '/vol/flex1/abc' is being removed.Reallocation will be stopped on '/vol/flex1/abc'.Monitor the system log for results.

Reallocation scans are onNo reallocation status.

Reallocate command

reallocate start –f /vol/flex1


Fri Nov 5 21:07:27 GMT [wafl.scan.start:info]: Starting file reallocating on volume flex1.Reallocation scan will be started on '/vol/flex1'.Monitor the system log for results.

Reallocation scans are on/vol/flex1:

State: Reallocating: Inode596, block 0 of 1168

Flags: doing_force,whole_vol

Threshold: 4Schedule: n/aInterval: n/a

Optimization: n/a

You can only start a one-time job on volumes.

You can stop a one-time job on a file, LUN, or volume by using the ‘reallocate stop <path>’

command. The command itself doesn’t say that you are able to stop a volume reallocation, but it

will allow you to do this.

Enable/Disable/Restart jobs

Global enable/disable:– reallocate off|on

Disable a single defrag job:– reallocate quiesce /vol/myvol/lun1

Restart a quiesced or idle job:– reallocate restart /vol/myvol/lun1

Usage Suggestions

Set up a scheduled reallocate job when a LUN/file is created– e.g. Oracle via NFS with a few Large files

Avoid fragmentation by having a regular scheduled reallocation job.Schedule reallocate job to finish before major operations (like Exchange backup/verify)Avoid full reallocation of snapshot data– If data is aggressively moved, this will result in a lot

fragmented data being held in snapshots

Troubleshooting

Use reallocate status –v firstAlso wafl scan status (ADV or DIAG)View scanner speed with wafl scan speed– Hidden command

Most actions EMS logged

Other Points

Works with Traditional or Flexible VolumesScheduled reallocation job is independent of the file/LUN (need to create/destroy independently.)Renaming a volume renames the reallocation job paths.Destroying a volume destroys all reallocation jobs in that volume.CFO takeover/giveback handles scheduled jobs also.

Internal Information

Troubleshooting

Log measure values to file via setflag:– setflag wafl_scan_frag_log 1– If set to 1

• logged to /etc/WAFL_layout.XXXX– Will show what system thinks of

fragmentation

Reallocation jobs stored in registry• state.reallocate.XXXX

Topic Review

What command would you use to defragment your file system?What are the differences between a normal and full reallocation?When would you set an allocation schedule?What command would you use to view a reallocation job?

Exercises

Exercise: Defrag and Reallocation

Objective

When you have completed this module, you will be able to do the following:

• Set a interval schedule on a lun

• Set a interval schedule on a file

• Change the normal interval setting to a specific schedule.

• Quiesce and restart a reallocation job

• Start a full reallocation on a volume.

Exercise Overview

This exercise is to familiarize you with the new .reallocation command.

Time Estimate

20 Minutes

Required Hardware, Software, and Tools

Hardware

• Standard class setup

Software

• DOT 7.0

Start of Exercise

Set a normal reallocation schedule with the default interval.

Step Action

1. At the command prompt, enter the following:

lun create –s 10m –t solaris /vol/flexvol/lun1

(If your volume is named something different, use the appropriate name).


reallocate start /vol/flexvol/lun1



reallocate status

View the status and note the schedule is blank, but the interval is set to 1 day. This is the default. .

Create a custom schedule for a file

Step Action

1. Open a window to the adminhost and create a 10 Megabyte file named file1 at the root of vol0.


reallocate start /vol/vol0/file1



reallocate status

View the status and note the schedule is blank, but the interval is set to 1 day. This is the default.


reallocate schedule -s "* * * *" /vol/vol0/file1


reallocate status

view the status and note the schedule should be * * * * and the interval should be N/A.

6. Wait at least one minute and you should see a console message similar to the following:

filer> Thu Oct 14 09:08:00 PDT [wafl.scan.start:info]: Starting WAFL layout measurement on volume vol0

* * * * sets the reallocate schedule to do a reallocate every minute of every day. This would not be done in normal practice. It is only for demonstration purposes.


reallocate schedule -s "* * - -" /vol/vol0/file1

8. Wait one minute, console messages that say starting wafl scan…

No, you have set the schedule to every minute every hour, however, you have not set the day of month or the day of the week.

Full reallocation on a volume

Step Action


filer> reallocate start –f /vol/vol0.

(If your volume is named something different, use the appropriate name)

2. Type the following:

filer> reallocate status

You should see the schedules for the above file and lun, as well as the reallocation on the volume.


filer> reallocate status -v

View the status notice the difference in the two commands.

Quiesce a reallocation job

Step Action



Pick one of the reallocation jobs from the list.


filer> reallocate quiesce /vol/vol0/file1

view the status and note that the reallocation job is quiesced..



View the status.

Restart a reallocation job

Step Action



Pick one of the reallocation jobs from the list.


filer> reallocate restart /vol/vol0/file1



view the status and note that the reallocation job is active.

This page intentionally left blank

03_Aggr_Defrag_v2

Documents

Transcript of 03_Aggr_Defrag_v2