E-mail archiving and DB2 CommonStore for Lotus Domino

IBM INFORMATION ON DEMAND COMES TO YOUFebruary 5, 2008Bangkok, Thailand

2

Agenda

CSLD - Overview of base functions Sub bullet

CSLD - Overview of advanced functions

CSLD - Summary and highlights

3

Extract e-mails from the messaging system– Option: together with meta data (like subject, sender, date, …)

… and store them in an electronic archive system (repository)– On hard drive, tape, or optical disks (WORM)– For many years

Access archived e-mails (retrieval)– Through the Domino client– Through the client of the electronic archive

What is e-mail archiving?

Messaging systemElectronic archive

system (repository)

Client

Archive

Retrieve

Retrieve

4

CommonStore

Electronic archive and content management solution

Designed for– SAP (CSSAP)– Lotus Notes and Domino (CSLD)– Microsoft Exchange (CSX)

IBM Content ManagerIBM Content Manager OnDemandTivoli Storage Manager

Mail system

CommonStore

Electronic Archive

5

What drives e-mail archiving today?

Growth of the messaging system

– More users– Larger mail boxes– More expensive operation

Requirement– Reduce the growth of the

messaging system– Offload old e-mails from

production messaging system– Keep old e-mails in near on-

line access

Legal requirements or industry regulations

– E-mails treated as business records (based on procedures)

– Provide e-mails on demand during audit or investigation

Example 1– User selects e-mails that should

be business records

Example 2– All e-mails of a certain user group

have to be captured and retained for a certain number of years

– Example: Securities Exchange Commission (SEC), regulations for broker and dealers

6

The Architecture

Application or User

DB

Lotus DominoLotus DominoEnvironmentEnvironment

DB2 CommonStore for Lotus Domino (CSLD)

Archiveoptions

Notes user

Job DB

CM Agent

archpro

Notes API

Archive specific APIs

Config. DB

CM iSeries

CM v8

CMOD

TSM

CM v8Agent

CM iSeriesAgent

CMODAgent

TSMAgentHTTP

dispatcher

Domino dispatcher

CSLD tasks

CSLD crawler

Notes API

archint.ini

csmimes.properties

ARCHWIN dispatcher

Local CSLD archive DB

7

Click to edit Master title style

DB2 CommonStore for Lotus Domino

Overview of base functions

8

CommonStore base functions

Archive

Retrieve

Search

Update

Delete

9

Click to edit Master title style

Base function - Archive

10

Archiving Types

Entire messages– Mail header (To, From, Subject, ...) can be kept as a

placeholder (“stub“) in Notes views or folders– Mail becomes very small or is removed completely

Attachments only– Only the attachments are archived, – Text in mail body remains unmodified

Convert format of message– Line data, Rich text– Any other format (like TIFF, PDF) through program exit

11

Archiving Type - Entire messages

Highlights– 100 % document fidelity (no format conversion)– Includes Notes specific Rich text elements

• Database links, buttons, sections, …

– Works for messages in Notes Richtext and MIME format– Works for signed and encrypted messages

Several options for mail placeholder– Text only (configurable)– Retrieval hotspot– Intelligent abstract of message

12

Example: Archived message with retrieval hotspot

Retrieval hotspot

New

13

Archiving Type - Entire messages (cont.) Intelligent abstracting (optional)

– A short summary of the mail is inserted in the mail body– Intelligent text analysis based on IBM’s Text Analysis Framework

(TAF) identifies most relevant sentences (special training for e-mails)

– Length (number of phrases) can be configured– Mail placeholder remains small (around 3 KB)

Benefit– Better search results for user since the abstract is included in the

Domino full-text index– No additional license cost for intelligent text analysis

14

Example: Archived message in iNotes with abstract

Intelligent abstract

New

15

Archiving Type - Attachment

Highlights– Archives only the attachment– Works for messages in Notes Richtext and MIME format– Works even for attachments outside of mail body – Works for signed messages– Mail body remains intact– Can save 60-75 % of entire message storage space

Attachment placeholder– In the exact location (not at end of message)

• Option 1: Text with hidden URL for Web retrieval

• Option 2: Text only (no URL)New

16

Example: Attachment archiving

Attachment placeholders

17

Archiving Method – Automatic archiving (Crawler)

Rules (policies)– Size of documents– Age of documents (absolute or relative)– Size of mail database– Notes formula (maximum flexibility)

New

18

Example: Policy with Notes users and groups

New

19

Archiving Method - User-driven archiving

Archive action in Notes folder(Notes Client)

Archive action in open document

(Web Mail)

20

Click to edit Master title style

Base function - Retrieve

21

Retrieval Types

Simple click on a button for reload on Domino server– No Client code installed on user work station

• Retrieval button added as action to mail template or

• Retrieval hotspot inserted in mail body

– Temporary restoration of message• Periodic clean-up of reloaded messages

– For reply or forward to external recipients– Works for both entire message and attachment archiving

CommonStore web viewing– Simple click on web link (URL) created during archiving– Attachments can be viewed individually– No reload on Domino server– Works for attachment archiving only

22

Example: Reload on Domino Server

Notes Client- Retrieval button- Retrieval hotspot

Web Mail- Retrieval button- Retrieval hotspot

23

1. Click on CommonStore Web link (URL)2. View archived attachment in Web Browser

Example: Web retrieval of an archived attachment

24

Click to edit Master title style

Base function - Search

25

Search and retrieve of archived messages

Search– Based on attributes

• Subject, To, From, Date, ...

• Stored in relational database of archive

• Requires mapping during archiving

– Full-text (requires Content Manager v8.2)• On a text attribute

• On the content of the archived item

Environment– Notes Client or Domino Web Access

• Entire messages are restored in original state to Domino

– Content Manager Client (or Content Manager OnDemand Client)• Access point to Enterprise Content Management (ECM)

26

Example: Search in Notes Client

1. Enter a search2. Get a hit list (results)3. Click to open or view

document

27

Example: Search in Content Manager pClient

1. Enter a search2. Get a hit list (results)3. Click to open

document

28

Click to edit Master title style

Base function – Update and Delete

29

Other features: Update and delete in archive Update

– Attributes stored in repository

Delete– Trigger removal in the archive from Notes interface– Important for keeping documents in Domino in synch with archived

items (no orphans)– Important for implementing document retention policies

30

DB2 CommonStore for Lotus Domino

Overview of advanced functions

31

Single-instance store (SIS) for CM v8.2

Archiving of entire Notes folders

Other Domino applications

CommonStore advanced functions

32

Single-instance store (CM v8.2)

Feature– Identical messages are automatically recognized and

archived just once (based on intelligent hash code algorithm)– Example: 1 e-mail is sent to 50 recipients

• 51 copies in Domino, only 1 copy archived by CommonStore

– Optional setting– Work when archiving entire messages

Benefit– Minimizes storage space in archive– Removal from Notes in archive supported

New

33

Single-instance store (CM v8.2) – cont’d

1 item linked to several mail owners– Multi-value attributes in CM v8.2 (child component)– Any attribute that is different for messages with same

content is part of the child component

New

34

Archiving of entire Notes folders Archive

– Notes folder and all subfolders and all contained documents stored with just one archiving request

– CommonStore stores the Notes folder information in the archive– Recommended: archive entire message without leaving document

stubs Restore

– Select root folder and restore all subfolder and contained documents with just one retrieval request

Benefit– One folder becomes the placeholder for many documents and

folders– Useful for user-driven archiving in Domino applications

35

Other Domino applications

All CommonStore functions available– Automatic and user driven archiving– Retrieve, search, update, delete, …– Any attribute mappings– Any Notes forms

Template changes– Based on CommonStore Script libraries– Easy customization

36

Example: Customer & Co application

37

DB2 CommonStore for Lotus Domino

Summary and highlights

38

Highlights

Central, rule-based, automatic archiving– Makes distributed local archives obsolete

Ease of use– Available in Notes, Domino Web Mail, and iNotes– User driven archiving– Flexible retrieval (without reload on Domino server)– Fast retrieval for mobile users (local archive copy)

Simple implementation and administration– No code install on Client workstation– Domino database for system administration– Tightly integrated in Notes/Domino

39

Highlights – cont’d

100 % document fidelity– Preserves Notes format (links, sections, signature, …)

Flexibility– Several archiving options (attachments only, entire

message, format conversion, ...)– Arbitrary meta data (attributes) stored in archive– Many additional features (update, delete …)– Can be used for other Domino applications

Part of an Enterprise Content Management (ECM) solution– No point solution– Choice of archive platform (Windows, Unix, iSeries, zOS)– Mature repository with proven scalability

• 10,000+ archive installs world-wide