CSIS 3600 Systems Analysis and Design – Fall 2001 Section 1 System Maintenance and Evaluation.

CSIS 3600 Systems Analysis and Design – Fall 2001 Section 1

System Maintenance and Evaluation

Key Ideas• Post-implementation activities include

providing:– System support, such as help desks

– Systems maintenance, fixing bugs and providing improvements

– Project assessment, learning how to improve from project experiences

PowerPoint Presentation for Dennis, Wixom & Tegardem Systems Analysis and Design

Copyright 2001 © John Wiley & Sons, Inc. All rights reserved.

POST-IMPLEMENTATION ACTIVITIES



Institutionalization of the System

• Provide support– Assistance in using the system

• Provide maintenance– Repair or fix discovered bugs or errors

– Add minor enhancements to provide added value

• Assess the project– Analyze what was done well

– Discover what activities need improvement in the future



Types of System Support

• On-demand training at time of user need

• Online support– Frequently asked questions (FAQ)

• Help desk– Phone service for known issues– Level 2 Support



System Maintenance

• While much of the emphasis on systems development is focused at developing new systems or replacement systems, the truth is the largest expenditure for systems development in most organizations is systems maintenance.

• In fact, more programmers work on systems maintenance than on new development.

– And here is something for you to think about: "This disproportionate distribution of maintenance programmers is interesting since software does not wear out in a physical manner as do buildings and machines" (Hoffer, et al, 1998, page 805).

Why is Maintenance such a Major Activity?

• There are several reasons why maintenance is such an important area.– But one of the major factors is that many times

it is more cost effective to evolve a current system than to replace it with a new system.

– Other reasons include, adding additional functionality, fixing errors, and adding new processes and technologies as they are identified.

What Does System Maintenance Include?

• Obtaining maintenance requests

• Transforming requests into changes

• Designing Changes

• Implementing Changes

Sources of Change Requests

1. Problem reports from the operations group

2. Requests for enhancements from users

3. Requests from other systems development projects

4. Change requests from senior management



Types of Maintenance

• Corrective– Repair design and programming errors

• Adaptive– Modify system to environmental changes

• Perfective– Evolve system to solve new problems or take

advantage of new opportunities

• Preventive– Safeguard system from future problems

Measuring Maintenance Effective

System effectiveness should be measured by:

– Number of failures

– Time between failures

– Type of Failures

MTBF• Mean Time Between Failures (MTBF) is a

measurement of error occurrences that can be tracked over time to indicate the quality of a system.

• Often used for hardware devices – but also for software. Some times Mean Time Between Faults is used.

• Tracking maintenance activities tells you how well the system is doing and provides knowledge on how the system can be improved.

Meaning of MTBF

• Would like to see the MTBF go up or down?– You should expect the MTBF value to

rapidly increase after a few months of use.– A MTBF that does NOT rapidly increase

over time, signals a problem with the system.

Costs of Maintenance

• Maintenance costs are a significant expenditure.• Some organizations allocate as much as 80% of

their IS budget to maintenance.• Factors that affect costs:

– Defects - # of unknown defects– Customers - # of different customers supported– Documentation - quality of technical documentation and

test cases– Personnel - # of personnel dedicated to support– Tools - software development tools-debuggers,etc.– Software Structure - structure and maintainability of the

code

Year Programmers on New Programs

Programmers on Maintenance

1950 90 10 1960 8,500 1,500 1970 65,000 35,000 1980 1,200,000 800,000 1990 3,000,000 4,000,000 2000 4,000,000 6,000,000

Programmers assigned to systems maintenance:

Maintenance and the SDLC

• If you look closely, you will quickly see that these steps are not that different from those used in the global Systems Development Life Cycle.

• In fact many software vendors use the SDLC process for each new release of an existing application.

• Four Stage Plan (the short version):– I. Systems Planning

– II. Analysis– III. Systems Design– IV. Systems Implementation

– Evaluation

– One missing from this model is Maintenance

– Book refers to it as post-implementation

Systems Development Life Cycle



Evaluation and Assessment

• The point of the evaluation is to critique the project, its methods, its deliverables and its management.

• There are two categories of evaluation summative and formative. – Summative evaluation occurs after a project is

complete. – Formative evaluation takes place as the project

occurs.

Formative Evaluation Techniques • Within a systems development project, evaluation

is an ongoing process.• Much of it may even be informal.

– But it is helpful to have set procedures for how this ongoing evaluation is to occur.

• Some organizations use guidelines and ask project participants to reflect on certain aspects of the project at various milestones.

• This reflection might be a checklist or a few simple narrative questions (What is going well, what might work better, etc.)

Formal Formative Evaluation

Test Plans

They assist in evaluating the physical design of the system including application code and the database model.

Features Team • For each new release of an application or for the

development of a new project, a group of people are selected and assigned to serve as a team to oversee the systems development project.

• The group is made up of developers, application specialists, end users, management, trainers, etc.

• All project deliverables (which are assigned through the systems project plan that follows the organization's systems development methodology) are presented to the Features Team for approval.

• Approval is done by majority vote.

• Deliverables not approved must be revised and resubmitted to the Features Team.

Summative Evaluation Techniques

• Summative evaluation is usually a formal process that occurs after the project is closed.

• Common instruments used are: – end user surveys – interviews– system audits.

System Review

• Examine the extent to which the costs and benefits of the system are realized

• Use this information to help in more accurately estimating costs and benefits for future projects



System Audits

• System audits involve comparing the original system requirement statements with the final system functionality. (Sometimes system audits occur in-house; other times end users are involved. Used properly, they provide a way to establish long term relationships with end users.)

End User Surveys

• Besides just end users, those involved in the project should also be surveyed and it is often helpful to ask them to reflect on the project methodology.

Project Team Review• Each member prepares 2-3 page document

regarding her or his actions during the project• Focus on improvement not penalties• Excellent behaviors are acknowledged and

diffused to others• Team leader summarizes and distributes lessons

learned



Process Improvement though Evaluation

• Data collected from summative evaluations can often be used to identify additional system functionality needs as well as ways to improve the systems development process.

Evaluation Techniques

• This site lists some evaluation techniques - they were designed for evaluating multimedia projects but have some good ideas that could be applied to information systems development projects:

• http://mime1.marc.gatech.edu/MM_Tools/evaluation.html

http://mime1.marc.gatech.edu/MM_Tools/evaluation.html





Mid-Semester Survey Results

Class Activities

• I really like the notes on the web. It allows me to go back and review and it helps me concentrate on what you are saying not on writing.

• Your lectures were well informed and easy to follow. • The in-class exercises we did for the use case, … diagrams helped a lot.• The notes are too general in my opinion.• The lecture notes on the Web page should be concise. They have too

much information. • I like the real life examples to show an actual application of the

information in the notes.• The tests are consistent with the work covered on the slides. This is

very helpful.• CSIS3600 is the only class in CSIS I have enjoyed so far.

ExtremelyHelpful %

Helpful %

Not reallyHelpful %

NC N

WeightedScale

CLASS ACTIVITIES In-class handouts: 12 57.14 8 38.10 1 4.76 21 2.52Lecture notes on Web: 16 76 5 23.81 0 0.00 21 2.76Lectures: 9 42.86 12 57.14 0 0.00 21 2.43

Group Project

• I wish the material needed for the deliverables was talked about a little sooner to give more time to work on it, not wait to the last minute.

• All other comments were related to in-class group time

ExtremelyHelpful %

Helpful %

Not reallyHelpful %

NC N

WeightedScale

GROUP PROJECTBroken into deliverables throughout the semester: 16 76.19 5 23.81 0 0.00 21 2.76

Feedback and ability to use feedback to revise project: 19 90.48 2 9.52 0 0.00 21 2.90Simulating the SDLC: 12 57.14 8 38.10 0 0.00 1 21 2.48

In-class Group Time

• All but three said ‘yes’– Others said it depends on the schedules of the students

in the group

• It gives us a chance to ask you to assist with any difficulties we are experiencing with the project

• Was not helpful when members did not show up for in-class time (it was posted in the syllabus)

Overall Effectiveness• I’m really uncertain. It seems to be

extremely similar to project management with the addition of forming tables.

• On personal ventures, this is a great help, benefit, assistance and a must.

• Learned more in this class and covered more material.

• I am learning a lot for this course. Having everything laid out on the Web and always available is very helpful.

• Many classes have touched on these subjects but knowing them has made those classes more meaningful. I wished I could have taken this class sooner.

• Effective to the point where I think of specializing in systems development.

• I think you know the material very well which helps - it is very easy for us to learn.

• Very good in helping explain SA.• Pretty good effectiveness.

• I believe this course has so far been very informative. At first, I honestly didn’t see the point, but now I have a clearer understanding about what the course is, I feel it will possible be one of the most useful courses I have had.

• The instructor is quite organized and gives insight into systems development that can be taken to fields other than IS or CS.

• Very good. I feel that I am progressing through the course.

• I think it has been very effective to learn from someone who has so much real-world experience (and presents examples of this in –class).

• This course has helped me thus far in understanding the steps taken in designing a system. I can apply this stuff in the real world very easily.

• This class seems like it would be a very good one to an IS major but I don’t think CS majors should be required to take it.

Changes• Make the class more interactive. Feedback between the

student and the teacher.

• .NET again

• This course should have a follow-up for those who have a deep interest in systems development… or is these another course?

• I would have learned faster (maybe) if there was a specific example running throughout the class lectures that applied the principles learned instead of generalizing them to every project that people may be doing.

• An overview of how the different things we learned relate to each other would be nice (ex. If you put something on one diagram, you have to put something else on another).

CSIS 3600 Systems Analysis and Design – Fall 2001 Section 1 System Maintenance and Evaluation.

Documents

Transcript of CSIS 3600 Systems Analysis and Design – Fall 2001 Section 1 System Maintenance and Evaluation.