Intelligent Archiving Strategies: Toward ILM
description
Transcript of Intelligent Archiving Strategies: Toward ILM
![Page 1: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/1.jpg)
Intelligent Archiving Strategies: Toward ILMArun Taneja, Founder and Consulting Analyst, Taneja GroupAlex Gorbansky, Senior Analyst, Taneja Group
![Page 2: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/2.jpg)
Agenda A Bit of Historical Perspective Why Archive? What to Archive? The ILM Panacea Developing an Operational Archival Strategy Key Considerations Representative Vendors and Solutions Conclusions
![Page 3: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/3.jpg)
Archival ≠ BackupBACKUP
Copying production data to an alternative medium for restorability in the event of data loss, corruption, or unavailability.
ARCHIVAL
Retention of historical data for future access for business reasons such as audits, customer issues, or litigation.
![Page 4: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/4.jpg)
Some History On Archiving
3000 BCE
Ancient Egypt:• Library of Alexandria• Engravings
Middle Ages 1600s 1789 1884
Shift from Feudalism To Nation State:• Records• Property rights
American colonists• Births • Marriages• Businesses
French Revolution• Property records
American Historical Association• Archival standards • Marriages• Businesses
![Page 5: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/5.jpg)
Archival Business Drivers TodayREGULATORY COMPLIANCE
REQUIREMENTS
EXPLOSIVE DATA GROWTH
APPLICATION PERFORMANCE DEGRADATION
RISING COSTS
![Page 6: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/6.jpg)
What to Archive? Structured Data:
• ERP/CRM DB tiers• Business transactions
Unstructured Data: • Documents• X-Rays• Check Images• Voice recording
Semi-structured Data:• Email • Instant Messaging
![Page 7: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/7.jpg)
ILM…ShmILM
“ILM” is an abstract framework for describing the processes and technology used to manage information throughout its life according to its business value.
“ILM” is NOT the panacea for your storage management challenges.
![Page 8: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/8.jpg)
Archival is a key component of what vendors are calling “ILM
Applications: ERP, CRM, Email, Call Recording, Image Access
Application Data: Structured, Unstructured, Semi-Structured
Policies and RulesBusiness ContextReferential Integrity Regulatory Compliance
Snapshots HSMData Movement Technologies
Replication Backup Archival
PrimaryStorage Infrastructure Tiers
Secondary Tertiary
![Page 9: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/9.jpg)
Developing an Archival StrategyPLAN When/How Data Classification Requirements
2. DESIGN
3. IMPLEMENT
4. REPORT &
TEST
![Page 10: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/10.jpg)
Why Plan and When to Start Upfront Planning will Result in Significant Benefits in
Future Phases. Develop an Archival Strategy as part of your
application design and development process. Engage Key Stakeholders:
• Application Owners• Business Decision Makers: Compliance Officers, Legal
Identify Key Archival Business Drivers:• Regulatory Compliance• Other: Data Growth, Increasing Costs, Poor Performance
![Page 11: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/11.jpg)
The Data Classification PuzzleAssess the application data in your shop
according to the following categories:• Structured: database• Unstructured: files, videos, images• Semi-structured: email
Identify specific data sets impacted by regulatory compliance:• Examples: Email, Medical Records, Call Recordings
![Page 12: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/12.jpg)
Requirements DefinitionEngage Application OwnersCompliance not the ONLY archival driver Separate requirements processes for
applications impacted by compliance.Compliance-specific:
• Retention period• Media characteristics• Data restorability rates• Access control policies• Data availability/DR
General archival:• Data Access Patterns• Restore time requirements• Application performance• Cost structure• Access control policies• Data availability/DR
![Page 13: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/13.jpg)
Taming the Compliance Monster1. Understand the Regulations: Significant Variance
by Industry
2. Assess/Communicate Requirements to Key Business Stakeholders
3. Judge Products for Yourself – Just because a vendor says a solution is “Compliant” doesn’t make it so.
4. Stay abreast of changes in regulatory mandates.
![Page 14: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/14.jpg)
Defining Key Archival Metrics
Archive Distribution Percentages Across:• Online: Disk, Object-based storage• Near-line: Optical, Tape (local)• Off-line: Off-site vaults
Number of data copies• Local• Remote
![Page 15: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/15.jpg)
Designing an Archival SolutionRequires an application specific
assessment – look for commonality in application requirements
Wholly enterprise-wide strategies will be difficult to build and sustain
Evaluate alternative solutions based on application requirements and metrics
![Page 16: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/16.jpg)
Don’t Ignore the Organizational Dynamics
Archival Touches Multiple Organizations:• IT – Applications• IT – Infrastructure• Legal• Users
Consequences of mistakes are enormous:• Fines• Litigation
Consider organizing a cross-functional team led by an archival champion with a combination of technical and business expertise
![Page 17: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/17.jpg)
Comprehensive Application Assessment
Data Classification Exercise Data Set Size and Historical and Predicted Data Growth Rates based
on business drivers Is Regulatory Compliance an Issue? Data Valuation over Time:
• Access patterns of data of 90 days old and beyond.• Cost of data loss
Going it alone can be difficult Available resources:
• Services organizations: GlassHouse, Accenture, EDS, Storage Vendor• Application Management Tools: File-Level SRM, Precise
Budgetary Requirements
![Page 18: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/18.jpg)
Components of the Archival Stack
Application Specific ModuleDiscovery and analysis of data assets
Business rules and policies definitions
Identification and movement of specific data to
appropriate storage medium
Management, indexing of data and metadata
Access control mechanism
Application Data
Storage InfrastructurePhysical archive repository
Data Preservation and Protection
Indexing Technologies for Retrieval
Management
& Control
Physical
Repository
Data Flow
![Page 19: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/19.jpg)
Structured Data Archival Challenges to Investigate
ERP deployments are still very nascent Preventing application downtime during archival Preserving referential data integrity:
• Archival of core data and associated data in other tables
Enforcing single read-only state across related data Delivering transparent access to archived/combined data via
native app UI• Maintaining performance of remote queries and union views.
Update process:• Restate vs. entire reload
![Page 20: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/20.jpg)
Unstructured Data ConsiderationsScalability Sustained performance with data growth• Hierarchical file-systems limited at large scales
Content Access and VisibilityMeta data use to intelligently manage and
maintain archive addresses traditional file system limitations
Scalability of Index (Content addresses)
![Page 21: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/21.jpg)
Email Archival Challenges
Stringent regulations: SEC Rule 17A-4• Non-rewriteable, non-reusable media• Verification of writes• Serialize units of media
Solution Requirements• Server-based capture• Support for multiple distributed Email Servers
![Page 22: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/22.jpg)
Meta Data Holds Real Value
• Digital asset tied to specific infrastructure• No value outside of infrastructure context
• Self-describing attributes for digital asset• Enables powerful policy-based data
movement applications
Traditional File Systems
Object-based systems
Meta Data is data about data
Object Age and creation date Object Change History Associated application/users Access control Priority/Criticality Data Access/Frequency
![Page 23: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/23.jpg)
Choosing the Right Storage Medium
Amount of D
ata
Probability of Reuse
D2D Systems
Libraries
Drives
< Seconds Minutes Hours to Days
1 WeekLife Expectancy
1 Month 3 Months 18 – 30 Years1 Year
Disk Systems
Recovery Time
Object Storage
![Page 24: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/24.jpg)
Key Considerations for Storage Media
Cost Access time Application access method:
• NFS/CIFS• Application-specific API
Reliability/Availability Data Preservation Capability Scalability Archival solution integration
![Page 25: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/25.jpg)
Storage Media ConsiderationsPros Cons
Primary Storage No risk of data lossInstantaneous access
Exorbitant costsPerformance degradation
Secondary Storage (SATA)
Cost effective Solid access time
IntegrationEnforcing preservationManagement
Object Storage Fit for large unstructured files Elimination of data redundancy WORM-like preservation
Price premiumPerformance scalability with index growth
Tape Most cost effective Removable Integrated WORM
Access timeReliability
![Page 26: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/26.jpg)
Shifting towards an On-line Model
Primary
ObjectStorage
Tape
SATA
![Page 27: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/27.jpg)
Representative VendorsStructured Data Email Unstructured
Archival Solutions
OuterBay, Princeton Softech, Applimation, Ixos,
Legato, KVS, Assentor Documentum, FileNet, NICE
SATA Object TapeStorage Platforms
CLARiiON, STK, IBM, Nexsan COPAN, Centera, Archivas, Permabit, DCT
STK, Quantum, ADIC, IBM
Start with your application vendor
![Page 28: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/28.jpg)
Trust But Verify
Develop processes to periodically access historical data to test:• Data integrity• Access time
Manage capacity growth using vendor-supplied reporting tools
![Page 29: Intelligent Archiving Strategies: Toward ILM](https://reader035.fdocuments.in/reader035/viewer/2022070420/56815e61550346895dccdd39/html5/thumbnails/29.jpg)
SummaryArchival is not backup and is not just about
complianceSuccessful strategy requires application-centric
approachEngage with key corporate stakeholders to
define requirements and select solutionsLook for automated and interoperable software
and hardware modules.Be Paranoid!