Dasish workshop on Audit and Certification 2014-b sierman
-
Upload
barbara-sierman -
Category
Presentations & Public Speaking
-
view
358 -
download
0
description
Transcript of Dasish workshop on Audit and Certification 2014-b sierman
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
ISO standards and Audit & Certification
Barbara Sierman, KB National Library of the
Netherlands
Dasish Meeting 17-10-2014, The Hague
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Audit & Certification: introduction
– History of the standards for audit and certification
– The ISO standards 16363 and 16919
– The APARSEN test audits
– Final remarks and further reading
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Audit & Certification: what is it and what not
• Audit:
planned and documented investigation by an
independent qualified group of the compliance of an
organization against a certain standard
Not a simple Yes or No, but recommendation for
improvements
• Certification:
Confirmation that organization meets the requirements of the
standard to which it is audited
Temporarily: regularly revised
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The history
: Infrastructure and Security Risk Management : Infrastructure and Security Risk Management
2002
• OAIS ISO 14721 published (updated 2012)
• Par. 1.5: standard(s) for accreditation of archives.
2005
• Checklist for Certification of Trusted Digital Repositories (RLG/NARA)
• Testaudits performed by RLG
2007
• DRAMBORA (2007), NESTOR (2006)
• Trusted Repositories Audit and Certification final report.
• (Input for Repositories Audit and Certification Working Group (RAC-WG)
2012-
• ISO 16363 Audit and Certification of Trustworthy Digital Repositories (RAC-WG)
• Draft ISO 16919 Requirements for bodies providing Audit and Certification for candidate trustworthy repositories (RAC-WG)
• Primary Trustworthy Digital Repository Authorisation Body (PTAB)
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Audit & Certification : European Framework
3 Levels of Certification
• Basic Certification (based on DSA)
• Extended Certification (self-assessment based on DSA plus self-audit
based on ISO 16363 or DIN 31644)
• Formal Certification (self-assessment based on DSA plus full external
audit of ISO 16363 or DIN 31644)
This Framework is supported by and coordinated with the help of the
European Commission
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The standard 16363
• ISO 16363- 2012 Audit and Certification of Trustworthy
Digital Repositories
: Infrastructure and Security Risk Management : Infrastructure and Security Risk Management
Organisational
Infrastructure
Digital Objects
Management
Infrastructure and
Security Risk Mgmt.
Metrics
• Statement of requirement
• Supporting text
• Examples: repository demonstrates it is meeting this requirement
• Discussion
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The standard 16363
: Infrastructure and Security Risk Management : Infrastructure and Security Risk Management
The ISO standard follows the chapters in TRAC and
distinguished 3 areas of auditing:
Organisational Infrastructure
Digital Objects Management
Infrastructure and Security Risk Management
as this one is also dealt with in other IT-related
standards, only the specific requirements in relation to
preservation are mentioned.
The standard consitis of metrics
In comparison with the TRAC document the explanation of the “
metrics” is extended
Statement of requirement
Supporting text
Examples: repository demonstrates it is meeting this requirement
Discussion part to explain various points of view in relation to the
statement or “metric”
• TRAC 2005
• TRAC 2007
• ISO 16363
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The standard 16363: example
: Infrastructure and Security Risk Management : Infrastructure and Security Risk Management
The ISO standard follows the chapters in TRAC and
distinguished 3 areas of auditing:
Organisational Infrastructure
Digital Objects Management
Infrastructure and Security Risk Management
as this one is also dealt with in other IT-related
standards, only the specific requirements in relation to
preservation are mentioned.
The standard consitis of metrics
In comparison with the TRAC document the explanation of the “
metrics” is extended
Statement of requirement
Supporting text
Examples: repository demonstrates it is meeting this requirement
Discussion part to explain various points of view in relation to the
statement or “metric”
Metric: “3.3.1 The repository shall have defined its
Designated Community and associated
knowledge base(s) and shall have these
definitions appropriately accessible”
Evidence: “A written definition of the Designated
Community. “
Discussion:
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The standard 16363: example
: Infrastructure and Security Risk Management : Infrastructure and Security Risk Management
The ISO standard follows the chapters in TRAC and
distinguished 3 areas of auditing:
Organisational Infrastructure
Digital Objects Management
Infrastructure and Security Risk Management
as this one is also dealt with in other IT-related
standards, only the specific requirements in relation to
preservation are mentioned.
The standard consitis of metrics
In comparison with the TRAC document the explanation of the “
metrics” is extended
Statement of requirement
Supporting text
Examples: repository demonstrates it is meeting this requirement
Discussion part to explain various points of view in relation to the
statement or “metric”
Metric: 3.3.2 The repository shall have Preservation Policies in
place to ensure its Preservation Strategic Plan will be met.
Evidence: Preservation Policies; Repository Mission Statement.
Discussion:
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The standard 16363
• ISO 16363- 2012 Audit and Certification of Trustworthy
Digital Repositories
• Guidance for auditors
• Other standards also applicable (security)
• Dependent on auditors experience
Consistency!
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The standard 16919
• ISO : standards of good auditing practices , accreditation of auditors
• Basis: ISO/IEC 17021
– Standard requirements for A&C general management systems
– Adapted for Trustworthy Digital Repositories (TDR)
Reference to OAIS
Reference to ISO 16363 as the set of criteria
Dealing with sensitive collection / confidentiality
List of competencies (normative in annex)
• PTAB group created new standard
• ISO 16919-2014 Requirements for bodies providing Audit and Certification
for candidate trustworthy digital repositories
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The standard 16919
Process of accreditation
ISO
CASCO: Committee on
Conformity Assesment:
advice
IAF: International
Accreditation Forum Assessors,
Training/Accreditation Group
National standards bodies
Monitoring &
Approving
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The standard 16919
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
What to expect from an auditor?
In general:
• Impartiality,
• Competence,
• Responsibility,
• Openness,
• Confidentiality,
• Responsiveness to complaints
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
What to expect from an auditor?
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The APARSEN test audits: what?
“Trust” is one of the pillars in APARSEN
2011: Testing of practical use of (draft) standards
• Metrics understandable and usable
• How much effort and time is needed for a repository
• Consistency in evaluation of the evidence
• Is the standard ISO 16363 applicable on different
kind of repositories?
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
The APARSEN test audits
Europe
Data Archiving and Networked Services (DANS),
UK Data Archive (UKDA),
Centre Informatique National de l’Enseignement Supérieur:
Département Archivage et Diffusion (CINES-DAD, France),
German National Library (DIN 31644 standard)
United States
Socio-economic Data and Applications Center (SEDAC),
National Space Science Data Center (NSSDC)
Kentucky Department for Libraries and Archives (KDLA).
International Group of “test-auditors”
Members of the RAC-WG
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Test Audit preparations
How much time will it take?
• Greater effort than expected to prepare the audit.
Preparation varied between 1.5 to 3 months
• Time spent on:
– Internal discussions about the standard
– Writing documentation that was not there yet
– Collecting existing documentation
– Improving existing documentation
• “Difficult to evaluate level of compliance”
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Test Audit procedure
• Expectations document: test-audit!
• Two Stages:
– 1. Repositories completed a Self-Audit template (Checklist
based on 16363)
Checklist plus documentation returned to audit team to prepare
audit
– 2. Site visit (2 days)
Verbal feedback with first impressions
Detailed audit report: areas for improvement
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Test audits: benefit quotes
Benefits as stated in the APARSEN report:
• DNB: “to have their own processes and documentation reviewed,
scrutinized, and ideally approved by some external professionals. “
• DANS: “it sheds a clear light on what the strengths and the weaknesses
are in the archiving activities of our institute. It gave us confidence that
we are well on our way to fulfil the requirements.
• CINES-DAD : [it] certainly helped them to evaluate the progress made
since the previous audits and the relevance of the actions taken over
the past couple of years
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Audits: benefits for organisations
• 3rd Party view of qualified people
• Better understanding of requirements
• Identification of areas for improvement
• Incentive to take action
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Test audits: benefits for organisations
• In line with report of 4C project:
– “To improve work processes
– To meet contractual obligation
– Publicly understandable statement of quality and reliability”
• In line with experience self-assesment SB Denmark:
– Improvement common vision organisation
– Competency development
– Organisational awareness digital preservation
– Good overview available documentation
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Audit & Certification : costs
• Factor costs often discussed
• 4C project showed:
– The only figures we have are of the APARSEN test audits
– Distinguish
Procurement of standards (preparation)
Staff costs
Certification costs
• Audit and certification will cost time and money
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Audit & Certification : risks
• Digital preservation is pioneering area
• Need for qualified auditors
• Growth path in audit and certification
Barbara Sierman, KB-NL
Dasish, The Hague 17-10-2014
Co-funded by the European Union under FP7-ICT-2009-6
aparsen.eu #APARSEN
Further information
• APARSEN : Report on peer review of Digital Repositories http://bit.ly/1jxRorz
• 4C project on audit & certification: http://bit.ly/1yGDpvc
• iPRES 2014 G. Elstroem & J. Junge:
Self-assessment of the Digital Repository at the State and University
Library, Denmark - a Case Study
• Blogposts David Rosenthal about recent TRAC audit http://bit.ly/1vyLzEI
• PTAB group: http://www.iso16363.org/
– News and updates about these standards
– Self-Assessment Template