Examining the use of Software Engineering by
Computer Science Researchers
Andre Oboler
School of Computer Science and Software Engineering,
Monash University, Melbourne, Australia
6-25-03
Outline1. Introduction
– Computer Science
– Computer Science Research
– Software Engineering
2. Software Engineering in the University– Teaching and Student Use
– Staff and Postgraduate Student Use
– Impact of current practice
3. A new system for research– RAISER / RESET a new SDLC to support the creation
and development of research software.
Computer Science
Problem solving with the aid of computers or the study of how this can be achieved
Careers:
Allows entry into any part of the IT industry
Typical starting position is that of programmer
Computer Science is NOT programming.
Programming is only the primary tool.
Research Software
Research software is software created using new research or to prove or demonstrate new research.
As all graduates and academics in the department are supposed to be proficient programmers all non standard tools, programs and experiments are set up without assistance.
Software Engineering
Software engineering is the art and science of creating successful software - repeatably
Careers:
Allows entry into Software Development, typically on a large scale
Typical starting position is that of programmer or software engineer
Software Engineering is very new
It has become a standard IT degree only in the last 5 years
Software Engineering
The term “Software Engineering” was coined as the title of a NATO Science Committee sponsored conference in 1968. The conference aimed to find ways to combat the “software crisis”. A follow on conference in 1969 focuses on way to make software development more “Engineering like”.
Much work has been done since then, but all of it focuses on software developed in and for industry.
Teaching and Student Use
Since 1989 the computer science curriculum, as endorsed by the ACM Education Board has including “Software Methodology and Engineering” as one of its key requirements.
The ACM, Association for Computing Machinery, was the world’s first computer society. Accreditation by the ACM is an important requirement for any computer science course.
Teaching and Student Use
More recently a software engineering curriculum has been developed. Successful completion allows software engineering graduates to qualify as certified engineers.
In 1998 two studies were conducted comparing student use of software engineering and industry use. Both the study by Robillard and Robillard and the study by Humphrey showed most time spent on student projects was spent programming. Minimum effort was put into planning and designing work. Humphrey added that unless students were directed to use software engineering… they didn’t.
Teaching and Student Use
Software Engineering is taken seriously both by industry, and by teaching staff responsible for it.
Unfortunately we found many staff who did not teach software engineering did not know the methodologies and tools taught to undergraduates. Our research investigated if this was an accurate impression and what the implications were, given that academic staff and postgraduates do their own coding.
Staff and Postgraduate Student Use
Research confirmed the initial impression: a lack of software engineering usage by researchers.
We discovered postgraduates were not using software engineering or were trying and giving up on it.
Investigating these two situations lead us to ask why this was so. It was claimed that the nature of computer science research was not compatible with software engineering.
Finally we examined the nature of computer science research and developed a compatible software engineering approach.
Past work
• no prior work on the costs/benefits of software engineering for research software
• This trend was started by Royce (1970) when he suggested that small projects used only by the developer need only use a 2 step analysis / coding approach (rather than his waterfall SDLC)
• This view that research is too small to warrant software engineering is still prevalent.
Methodology
The research approach used will now be presented followed by some results, and finally the RAISER/RESET Software Development Life Cycle, our new approach to developing software in academia.
Approach
• Triangulation of:– statistical analysis of survey results– Interviews and E-mail discussion with experts– observations from case studies
These methods investigated the use, costs and benefits of using Software Engineering in
Computer Science Research
Survey Samples
Taught Software Engineering:
US 72%
AUS 43%
Training:
Computer Science: US 62% AUS 69%
Software Engineers: US 10% AUS 9%
Graphical Models
Have you used Graphical models when developing software?
US 68% (yes)
AUS 74% (yes)
Note the lower US response despite the higher number of US Software Engineering educators.
Flow Charts
This is perhaps best known, and one of the coldest design methods. It is mostly obsolete.
Both show low usage.
The higher level of occasional usage in the US is considered a factor of the sample, and should not be taken to represent the US more generally.
The Australian results are as one would expect, assuming most peop have moved on to newer methods.
Flow Chart Use in the US
0
1
2
3
4
5
6
7
8
9
Don’t knowthis
Know butdon’t use
Use rarely Usesometimes
Use often 3 Use Always
Frequency
n.o
. R
esp
on
den
ts
Flow Chart Use
Flow Chart Use in Australia
0
5
10
15
20
25
Don’t knowthis
Know butdon’t use
Use rarely Usesometimes
Use often 3 Use Always
Frequency
n.o
. R
esp
on
den
ts
Flow Chart Use
Class Diagrams
This is the most common design tool used in industry. It would be taught as part of any undergraduate computer science degree.
The high number of academics who do not know what it is, and who chose not to use it is cause for concern.
Class Diagram Use in the US
0
1
2
3
4
5
6
7
Don’t knowthis
Know butdon’t use
Use rarely Usesometimes
Use often 3 Use Always
Frequency
n.o
. R
esp
on
den
ts
Class Diagram Use
Class Diagram Use in Australia
0
1
2
3
4
5
6
7
8
9
10
Don’t knowthis
Know butdon’t use
Use rarely Usesometimes
Use often 3 Use Always
Frequency
n.o
. R
esp
on
den
ts
Class Diagram Use
Application of SDLCs in research
SDLC Use in the US
0
1
2
3
4
5
6
7
8
Waterfall Fountain Spiral RAD Other Unplanned
Frequency
n.o
. R
esp
on
den
tsSDLC Use in the Australia
0
2
4
6
8
10
12
14
Waterfall Fountain Spiral RAD Other Unplanned
Frequency
n.o
. R
esp
on
den
ts
SDLCs describe the systematic method used to develop software.
The Spiral (3rd from left) is the most common in Industry.
Note the high level of unplanned work. Again the US sample population has an impact.
Problems with a lack of Software Engineering
• A lack leads to a waste of new research students’ time
• Follow on research is harder to achieve (some valuable research may be shelved)
• Authenticity of results is harder to verify
• Shortens the useful life of the project
• Shorter projects have fewer benefits
Why Software Engineering is not used
Reason Percent that agree with this reason
Never thought about it 14%
Don't know about them 11%
Cost of learning them is too high 17%
Not appropriate for my work 83%
Cost of use is higher than pay off 46%
Organisational Policy against spending time on them
3%
Considerations for an SDLC to meets the needs of research
“A system built as part of a Ph.D. project is intended to prove
feasibility, and it would almost always be a mistake to spend the
time and effort during initial development to build it to product-quality standards”
(Brooks, 2002).
“The primary aim [of research] is to get a flaky prototype working sufficiently to get a few statistics
out. There is absolutely zero [incentive] for producing a robust,
flexible, extendable piece of software” (Allison, 2002).
“The major problem is that research projects tend to be
opportunistic rather than planned” (Waite, 2002).
“The implication is that any SE approach for research software
would have to be agile and evolutionary in nature”
(Pressman, 2002).
On User Documentation for the CDMS Case Study
“We're not sure how this will happen. We were sort of hoping it
would happen by magic or be delivered by a stork”
(Allison, 2002).
The RAISER / RESET idea
• Separate research activities from stabilization• Limit the negative impact during research phases• “Clean” up code so it is ready for the next
researcher to continue working on• While many researchers do not use software
engineering, those that do use it predict they will use more in the future, this is potentially as harmful as the current lack of application.
The SDLC Model
Questions?
NB: Future work will be undertaken in this area over the next three years, feedback is most welcome!
RAISER (for Research)
R eactive
A ssisted
I nformation
S cience
E nabled
R esearch
Minimum overhead, maximum benefit now
• High level design – before coding
• Use header blocks• Configuration Management• Paired Programming
– with other researchers
RESET(between Research)
R esearch
E nabled
S oftware
E ngineering
T echniques
Clean up and restructure for later
• Design and Code reviews
• Restructure for: – improved modularity
– ease of reuse
• Review API / User interface– Improve and document
• Create design documents
• Record current and future functionality
Implementation
The in-house software development lab
Top Related