Cisco Con!dential1 2010 Cisco and/or its affiliates. All rights reserved. Auditing in a Virtual World (Team, Tools, & Techniques) Sheronda Je"riesTL 9000 Program Manager Cisco Con!dential2 The workplace is evolving From the Future of Work Research Program Workforce Survey *Drop-in workstations or cubicles or other facility at their company (e.g.,conference room, training room, cafeteria, employee lounge), and co-working spaces or third places such as cafes, libraries, airports, or hotels. In-O"ce Workers Work from Home 32% In-O"ce 35% Drop-In / Co-Working* 33% Cisco Con!dential3 Standards are evolving Nigel Croft ISO TC 176 SC 2 Chair Cornish presentation to US TAGs For MSS Liaison Group Teleconference Cisco Con!dential4 CD available 2013 DIS available 05/2014 FDIS expected05/2015 ISO 9001:2015 to be published09/2015 3-Year Transition Agreed ISO Standards Development Process IAF Informative Document - Transition Planning Guidance for ISO 9001:2015 Published ISO 9001:2015 Status Update Cisco Con!dential5 Cisco Con!dential6 For information visit: questforum.org Cisco Con!dential7 4 Context of organization Understanding of the organization and its context Needs & Expectations of interested parties Scope of the QMS QMS and its Processes 5 Leadership Leadership and commitment Quality policy Roles, responsibilities and authorities 6 Planning Actions to address risk and opportunity Quality objectives 7 Support Resources Competence Awareness Communication 8 Operation Operation planning and control Determination of requirements for products/ services Design and development Control of externally providedproducts and services 9 Performance evaluation Monitoring, measurement, analysis and evaluation Internal audit Management review 10 Improvement PLANDOCHECKACT Documented information Planning of changes Production & serviceprovision Release of products and services Control of Nonconforming process outputs,products and services (Slide Courtesy of Alan Daniels; Boeing) Continual improvement Nonconformity and corrective action (C) Nigel H Croft 2014 - All rights reserved Structure of ISO 9001:2015 Cisco Con!dential8 !Supports globalization !Drives innovation, creativity and better decisions !More critical now due to new normal for technologiesVirtual technology is critical Cisco Con!dential9 Typical Challenges to Virtual Auditing Technology Lack of trustLogistics - time zones/holidaysAgainst the rules Other challenges? Cisco Con!dential10 Its NOT against the rules ISO 19011:2011 Guidelines for Auditing Management Systems Cisco Con!dential11 Its NOT against the rules IAF MD 4:2008 Use of Computer Assisted Auditing Techniques ("CAAT") for Accredited Certi!cation of Management Systems provides guidance Computer Assisted Auditing Techniques (CAAT) may include Teleconferencing Web meetings Interactive web-based communications Remote electronic access to the management system documentation and/or management system processes NOTE:New MDs will include the term Virtual Site- An on-line environment allowing persons from di!erent physical locations to execute processes. Cisco Con!dential12 Its NOT against the rules Rules may vary for example: AerospaceISO-9104-1 was revised in 2012 to include allowances in audit day rules for CAAT Information and Communication Technologies (ICT) Auditor(s) and auditee(s) must be $uent in electronic data transferring Rules: The following areas cannot be audited using e-audit techniques: Laboratorieswarehousing-distribution centersmanufacturing sitesrepair (also know as service) centers Cisco Con!dential13 Its NOT against the rules APG Electronic-Based Management Systems Cisco Con!dential14 Logistics can be managed Cisco Con!dential15 Logistics can be managed Malaysia China Australia US Canada DecOctSepAugJulyJuneAprilMarchFebJanMayNov Italy India From World Time Buddy http://www.worldtimebuddy.com/ Cisco Con!dential16 Atcollaboration and the use of virtual technology is a part of our culture our competitive advantage employees 71K+ countries 165+ o"ces 380+ Cisco is globally certi!ed to ISO 9001 -includes all aspects - manufacturing, sales, Cisco Services and engineering Cisco Con!dential17 Logistics can be managed Planning is required and should consider Normal business/working hours for each physical location Holidays Organizational protocols and norms Dont forget basic etiquette Cisco Con!dential18 Logistics can be managed For a Virtual Meeting Checklist, please visit:http://www.virtualworkswell.com/citrix

Cisco Con!dential19 What about TRUST? Cisco Con!dential20 Use of TechnologyTRUSTWhat about TRUST? Cisco Con!dential21 21 Culture Process Technology Cisco uses collaboration and virtual technology to change the way we work. We use these technologies ourselves we are our own laboratory. Requires equal parts Process, Culture and Technology What about TRUST? Cisco Con!dential22 Virtual Team Considerations blah blah blah Cisco Con!dential23 Work is now anytime, anywhereCisco Con!dential24 Technology Options "Options range from Webcam to TelePresence + Advantages:!Allows everyone to sit at the same table Telepresence / Video-conferencing Data-conferencing "Options includeCisco WebEx Microsoft O"ce Live Meeting GoTo Meeting, etc. Cisco Con!dential25 Technology Options "SKYPE "FaceTime "ooVoo "Yahoo Messenger, Google+, . There are many FREE / Low Cost Options Cisco Con!dential26 Technology, Training & Auditing Use technology to replicate & simulate WHICH may require more planning but does it really work? Cisco Con!dential27 !O%ering an e%ective training curriculum using an evolving blended learning approach that supports our shareholders professional development via business-focused training. !Explaining the basics of understanding ISO 9001, TL 9000 & ISO 14001, and e%ective auditing strategies for maintaining Ciscos certi!cations while promoting continual business improvement.!Delivering worldwide training using multiple tools (TelePresence, WebEx Training, Vovici now known as Verint EFM) !Positively impacting Auditee Satisfaction & OFI Adoption metrics Internal Quality Auditor Training supports Cisco by: Cisco Con!dential28 Collaboration & Internal Quality Auditor Training A Timeline On-line survey tool used to administer course exams and end-of-course survey

Q4CY2008

Q1-Q2 CY2009

Q3-Q4CY2009

Q1-Q2 CY2010

Q3-Q4 CY2010

CY2012 - 2013 WebEx Connect Spaces used to store transition training material, WebEx recording,communicate exam url and end-of-course survey url ISO 9001 Internal Auditor Training piloted using WebEx FLIP video used for ISO 9001 Case Studies & Brown Bag Level 3 Training Evaluation V.1 conducted ISO 9001 Internal Auditor Training piloted using TelePresence Partnered with LDSGto update TL 9000 GA TL 9000 Internal Auditor Training piloted using TelePresence On-line surveys updated to comply with internal policy changes TL 9000 Internal Auditor Training piloted using WebEx New Auditor Application moved to on-line survey tool;Quality Champion learning path enhanced

CY2011 Update Level 3 TrainingEvaluation to V.2 Auditor Trainingstabilized/matured to annual updates

CY2014- 2015 Enhanced Learning Sessions Update End-of-Training Survey Cisco Con!dential 2010 Cisco and/or its a"liates. All rights reserved.29 Course was very good. Definitely this training helps each one of us to look at our quality policy and apply that in our daily job. It was a very good training course.Would de!nitely recommend to other peersHere is what learners are saying about internal quality auditor Training!Superb knowledge of subject.Great Class! Material was presented in a fun and interesting way, that kept us on the ball and helped make the information relevantLoved the high energy and ability to keep the whole class engaged especially through WebEx.Never have I had an instructor make a class fun while being effective in getting us to learn detailed material.GREAT TRAINER The training was very well presented via WebEx to a small group of students. The session was very interactive with frequent pop quiz type refreshers.training is very good Class was Fantastic. Engagement was fantastic.is an expert on the subject and explained the implementation of ISO at Cisco beautifully.I really appreciate the hard work, motivation and outstanding manner in which the training was conducted. The training was good, helpful and enjoyable.a great experience I would definitely recommend this class to my colleagues.Very lively, kept me awake till after midnight my time :) That is a great achievement. Very good SME and fun to learn with.Cisco Con!dential30 6months+ Post Training Survey of ISO 9001 Internal Quality Auditors for CY2013 through 07/2014 Summary More individual contributors (96%) than managers (4%) completed ISO 9001 Internal Quality Auditor training for the period versus 85% and 15% for previous yearThe survey response rate for CY2013 07/2014 is 57% versus 31.4% for the previous year which shows a significant increase29% of the respondents had not participated in an internal audit as of the survey date which matches previous year 50% had participated in an internal audit within 6 months of training completion versus 41% for previous year 83% of the respondents found the training Extremely or Very Useful in their roles as Individual Contributors/Managers at Cisco which almost matches 84% for the previous year 100% would continue to recommend ISO 9001 Internal Quality Auditor training to others based upon 57% survey response rate! Cisco Con!dential31 Are Virtual Audits E"ective? How will you measure the e%ectiveness of virtual audits? Are auditors able to identify nonconformities and OFIs?Do your internal audit results align with your Certi!cation Bodys results? Do you get the same responsiveness from management to issues even though the auditor may not bephysically on-site?Are auditees and auditors satis!ed with the use of virtual techniques?Do the savings in travel balance against any extra time required in planning/conducting virtual audits? Cisco Con!dential32 TIARA Dashboard !Operational Bene!ts: Management & ISO PMs can now better manage the global ISO 9001, TL 9000 and ISO 14001 audit programs.Progress against planMonitoring for aged and overdue audit !ndingsTracking status at a detailed level Wow. thats a lot of OFIs! Is the business improving over time? Nope! Timeliness ofaudit completionCisco Con!dential33 Tips & Techniques 1.Take extra time in planning and communicating - consider the length of each audit interview & schedule breaks 2.Do as much pre-work as possible -pre-load agenda, pre-visit urls, ask auditees to bring laptops/webcams 3.Allocate additional time for audit interviews.. 4.Resist inviting more people to each audit interview 5.For large groups, use an on-site host to address issues and minimize side conversations/chats/IMs 6.Be familiar with the features of the tools sharing desktops and taking screenshots is simple/e%ective 7.Be prepared to familiarize others with the tools 8.Know how and whom to contact for tools help 9.Have back-up plans in place! Remember YOU are the LEADER! Cisco Con!dential34 Other challenges Cisco Con!dential35 Questions? Many thanks to Leading High Performing Teams in a Global and Virtual World presented at ProjectWorld & World Congress for Business Analysts, November 2011by Rebekah Thayer Cisco Con!dential36 Sheronda Je"riesTL 9000 Program Manager (919) 641-0608 mobile sje"rie@cisco.com Questions? Thank you.