KT1 Keynote
5/2/2013 8:30:00 AM
Asking the Right Questions? What
Journalism Can Teach Testers
Presented by:
Thomas McCoy
Australian Department of Families, Housing, Community Services and
Indigenous Affairs
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
Thomas McCoy
Australian software testing professional, teacher, and journalist Thomas McCoy has worked in the IT industry for more than two decades. Much of this time was spent as a software developer and IT manager with Australian government agencies. Wanting to make a greater contribution to software quality, Thomas re-oriented his career into the emerging discipline of software testing and has been an enthusiastic promoter of the profession ever since. At conferences in Australia, New Zealand, Canada, and the US, Thomas is a popular keynote speaker, who has received several best presentation awards. You can reach Thomas at [email protected].
STAR East 2013 2 May 2013
Thomas McCoy 1
What Journalism Can Teach TestersASKING THE RIGHT QUESTIONS
Thomas McCoyBJourn, BSc, DipEd, MInfTech
Australian Department of Families, Housing, Community Services and Indigenous Affairs
DISCLAIMER
STAR EAST 2013 2
The views expressed in this presentation are not necessarily those of the Department of Families, Housing, Community Services and Indigenous Affairs, the Australian Government, or even the presenter. ☺
Australia
STAR EAST 2013 3
I’m from Australia, which is around 9,500 miles away and now has a population of 23 million (just a few million more than the population of Florida).
Canberra
STAR EAST 2013 4
I’m based in Canberra, Australia’s capital city, which is in the southeast of Australia between Sydney and Melbourne. It was built in 1913 and designed by Chicago architect Walter Burley Griffin (who won an international design competition). Its population is around 370,000.
STAR East 2013 2 May 2013
Thomas McCoy 2
FaHCSIA
STAR EAST 2013 5
FaHCSIA, the government department I work for, is responsible for social security in Australia, with payments and support for groups including pensioners, the unemployed, people with disabilities, and our Indigenous Australians. It manages about one fifth of the federal budget.We have testing staff working within both the agile (embedded in development teams for the smaller systems) and waterfall (working as a large testing group for our major application) contexts.
Department of Families, Housing, Community Services and Indigenous Affairs
STAR EAST 2013 6
Background
Started in journalism and media, then moved into IT, first into programming and finally into software testing.Have become increasingly aware of the importance of communication, both in terms of project team communication and computer / end user communication.This session has been inspired by the parallels I’ve seen between journalism and software testing.
WHAT DOES SOFTWARE TESTING HAVE TO DO WITH JOURNALISM?
STAR EAST 2013 7
JOURNALISM SKILLS
STAR EAST 2013 8
• A questioning, curious, and inquisitive mind
• Communication skills• Writing ability• News sense• Listening skills• General knowledge• Empathy, patience and
understanding• Sense of public
responsibility
• Language skills• Interviewing skills• Ability to meet deadlines• Knowledge of computers
and technology• Analysis skills• Resourcefulness• Persistence• Truthfulness• Accuracy• Integrity
Notice big overlap with testing skills.
STAR East 2013 2 May 2013
Thomas McCoy 3
JOURNALISM PRINCIPLES
STAR EAST 2013 9
• Its first obligation is to the truth • Its practitioners must maintain independence from
those they cover• It must serve as an independent monitor of power• Its practitioners must be allowed to exercise their
personal conscience
These are principles in an “ideal world”, but again there is strong overlap with our testing career.
THE FOURTH ESTATE
STAR EAST 2013 10
Journalism has been called “the Fourth Estate”. Comes from British Parliament which has the monarch (1st estate), House of Lords, containing nobility and clergy (2nd estate), and House of Commons containing the other elected members (3rd estate).“In the reporters gallery yonder there sits a fourth estate more important by far than they all,” said 18th century British politician Edmund Burke.Fourth Estate rests on idea that it’s the media's function to act as a guardian of the public interest and as a watchdog on those in power.Are there parallels between this power structure and IT projects?
THE FOURTH ESTATE
STAR EAST 2013 11
Estate Journalism IT1st King Business areas?2nd House of
Lords (nobility)
Project Managers?
3rd House of Commons
Developers?
4th Journalists Testers?
TESTING: JOB OR VOCATION?
STAR EAST 2013 12
The enormous impact technology has on society, and our position in reviewing quality (too often at the end of the process), has led some to ask whether software testing is more of a vocation than a job. Vocations are sometimes differentiated from jobs because of their strong sense of social responsibility (e.g. teachers, nurses, clergy, journalists, etc.).
STAR East 2013 2 May 2013
Thomas McCoy 4
TESTING: JOB OR VOCATION?• Sacrifice• Burnout• Long hours• Commitment• Thanklessness?• Social conscience• Self-development• Fighting “the good fight”
STAR EAST 2013 13
Do you identify with any of these aspects in your role?
STAR EAST 2013 14
Impact of a recent technology issue.Apart from that, we’ve had people killed, fried by radiation machines, not be able to use the brakes on their car, etc.
STAR EAST 2013 15
Then there are the less publicised daily annoyances caused by deficient technology (which can’t be good for people’s health).
SHARING NEWS OF OUR SUCCESS
STAR EAST 2013 16
Because of the nature of our role it’s sometimes challenging to publicise our success without upsetting people.We might want to do something like this …
STAR East 2013 2 May 2013
Thomas McCoy 5
Testing News
STAR EAST 2013 17
… but this could cause offence, so we publicise in more benign ways, as in my newsletter story about our new test lab …
More palatable news
STAR EAST 2013 18
NEWS VALUES
STAR EAST 2013 19
Three useful news values are:• Impact: angle your message for maximum
impact in relation to your target audience• Proximity: if something is close to your
target audience or their aspirations it will have greater effect
• Timeliness: don’t delay messages unnecessarily
• For us, this is a common and difficult communication task
• Nobody likes to hear bad news
• Nobody ever says “thank you” for receiving bad news
DELIVERING BAD NEWS 1
STAR EAST 2013 20
STAR East 2013 2 May 2013
Thomas McCoy 6
DELIVERING BAD NEWS 2 • Avoid it being a surprise (perhaps it
can first be passed through informal communication channels so people can adjust to the idea)
• Pick a time when the recipient is relaxed (sometimes Friday afternoons work well)
• If possible, propose some alternatives when deliveringthe news
STAR EAST 2013 21
The original Larson dog cartoon …
STAR EAST 2013 22
Sometimes “selective listening” is a problem whenreceiving bad news, as illustrated by this re-tooling of a classic Larson cartoon.
… can be adapted to our situation
STAR EAST 2013 23
REHEARSING BAD NEWS DELIVERY WITH AUSTRALIAN NATIVEANIMAL PERSONAS
STAR EAST 2013 24
Sometimes, when having to deliver bad news, it can help to think about the “persona” you’ll adopt when delivering the message.In Australia, we have native animals with unique characteristics that can be used to rehearse the bad news delivery (we even have hand puppets). ☺Sometimes we might have to go in aggressively, while at other times it might be with an attitude of resignation (because we choose not to fight this particular battle), or even with a humorous approach.
STAR East 2013 2 May 2013
Thomas McCoy 7
TASMANIAN DEVIL
Aggressive and energetic
STAR EAST 2013 25
“Well what do you expect when we weren’t given the resources we needed!”
KANGAROO
Cheerful and easy going
STAR EAST 2013 26
“OK, we’ll see what can be done”
KOALA BEAR
Relaxed and nonchalant
STAR EAST 2013 27
“Yeah, right, whatever.”
KOOKABURRA
Always ready for a laugh
STAR EAST 2013 28
“Ha, ha, you’re joking, you know that was always going to be impossible.”
STAR East 2013 2 May 2013
Thomas McCoy 8
A KEY PART OF JOURNALISM:THE 5WS & 1H
STAR EAST 2013 29 STAR EAST 2013 30
“I keep six honest serving-men(They taught me all I knew);Their names are What and Why and When and How and Where and Who”Rudyard Kipling from “The Elephant’s Child”
Think about the 5 Ws & 1H in anything you write.For example, in a defect report, as shown on the next slide …
5WS & 1H
STAR EAST 2013 31
• Who: Fred Nurk• What: Discovered system crashes if
negative quantities entered• When: Last night• Where: Test lab• Why: Inadequate input field
checking?• How: Running automated test scripts
Defect report example. Note that you won’t necessarily need all of the 5Ws & 1H in everything you write so consider which ones to leave out.
WRITING EFFECTIVELY
STAR EAST 2013 32
STAR East 2013 2 May 2013
Thomas McCoy 9
STAR EAST 2013 33
“Always grab the reader by the throat in the first paragraph, sink your thumbs into his wind pipe in the second, and hold him against the wall until the last word.”
This might be a bit ambitious for the average test plan but the reason so few documents are read is because they’re so totally unengaging.
GEORGE ORWELL’S ADVICE
STAR EAST 2013 34
1. Never use a long word whena short one will do.
2. If it is possible to cut out aword, always cut it out.
3. Never use the passive voicewhen you can use the active.
4. Break any of these rules sooner than saying anything outright barbarous.
If you remember nothing else from this session, remember these rules. You will sometimes need to violate #3 and use passive voice to avoid blaming somebody.
Active Voice: Fred Nurk introduced a defect (blames Fred)Passive Voice: A defect was introduced (blames nobody)
THE INVERTED PYRAMID
STAR EAST 2013 35
Importance (signified by width)
Position
Top
Bottom
Content
People are so deluged with textual information that they may not read more than the first few lines.So use the inverted pyramid to present your information.
AN EFFECTIVE LAYOUT
STAR EAST 2013 36
• First paragraph: summarise the situation, covering the most important aspects of who, what, when, where, why and how (this may be the only paragraph the person reads)
• Next: provide more details• Finally: use a “call to action”
to have the issue resolved
When communicating about issues needing attention you may want to have a “call to action” at the end rather (than entirely following the inverted pyramid style of having less and less important information going down the page).
STAR East 2013 2 May 2013
Thomas McCoy 10
EXAMPLE
STAR EAST 2013 37
On the registration date screen February 29 is being accepted for all years, not just leap years.[Summary, covers “where” and “what”]The problem may be due to the new external date routine, which was recently implemented on this screen. (The other screens still work fine.)[More info]This could damage customer relations and create adverse publicity for the organisation and needs urgent attention.[Call to action]
WRITING 1
STAR EAST 2013 38
• Need a strong headline that will push relevant buttons in people. The “headline” might be, for example, the:
summary of your defect report; or thesubject line of your e-mail message
• The headline may be all they read• Think about the “angle” of your
message (what are you trying to achieve)?
The “buttons” that will work best depend on our target audience and this is where research and prior planning are important.
WRITING 2
STAR EAST 2013 39
• Start with most important point, which should answer some of 5Ws & 1H
• Secondary information later• Use Plain English (avoid jargon)• Use mix of long and short sentences• Keep paragraphs to 3 sentences on average
(large slabs of writing intimidate readers)
WRITING 3
STAR EAST 2013 40
• Clichés should be avoided like the plague ☺
• Check spelling and grammar• A reading level of Grade 8 is
recommended (can use Flesh-Kincaid readability checker in Word)
Newspapers are typically written at Grade 8 level.
STAR East 2013 2 May 2013
Thomas McCoy 11
USING THE READABILITY CHECKER
STAR EAST 2013 41
In Word 2010 do: File Tab, Options, Proofing…
… and make sure you tick the Show readability statistics checkbox. Also think about what you want to use for Writing Style (e.g. just Grammar or Grammar & Style) and check further options with the Settings …button.
STAR EAST 2013 42
Settings …These are some of the grammar checks the system can do under Grammar & Style but be aware that there are many it can’t do well (because writing is a human activity) so don’t rely on it too much and don’t follow it blindly (sometimes it gets confused).Passive sentences is worth using, though.
READABILITY STATISTICS
STAR EAST 2013 43
These are only a guide and have some quirky aspects.Flesh-Kincaid is based on average number of syllables per word and average number of words per sentence.This results in the kids’ classic “Green Eggs and Ham” achieving a grade level of minus 1.3, which would suggest it could be understood by kids before they start school and have learnt to read.We tried an interesting experiment …
STAR EAST 2013 44
Nuclear Physics: 15.5 grade level
We ran the Encyclopaedia Britannica article on Nuclear Physics through Flesh-Kincaid and it came back with a Grade level of 15.5(i.e. written at university level) …
STAR East 2013 2 May 2013
Thomas McCoy 12
STAR EAST 2013 45
Nuclear Physics & Green Eggs: 2.4 grade level
… and then we pasted the entire text of Green Eggs and Ham at the end of the article, and when we ran readability again it brought the Grade level down to 2.4!Does this mean a kid halfway through Grade 2 could now understand nuclear physics?
After this somebody helpfully suggested that we should paste Green Eggs and Ham at the end of all our unreadable documents (using a 1-point font and white text for minimum visibility) to bring our readability scores down, but that’s another story …
OTHER WAYS OF PRESENTING INFORMATION
STAR EAST 2013 46
Wordle Word Cloud of Seven Testing Principles
STAR EAST 2013 47
A word cloud is an innovative way of illustrating key concepts in a document. The more often a term occurs the larger it is.This is from the 7 principles of testing in the ISTQB Foundation Syllabus and was generated by Wordle(http://www.wordle.net/).
0500
100015002000
Jan Feb Mar Apr
Overtime Hours
True RepresentationSTAR EAST 2013 48
Graphics can be a useful way to present information (and they have enormous credibility).Here is a graph of a project where the overtime hours are doubling every month, from 200 to 400 to 800 to 1,600.But would reporting this trend cause unnecessary alarm?
STAR East 2013 2 May 2013
Thomas McCoy 13
1
10
100
1000
10000
Jan Feb Mar Apr
Overtime Hours
Logarithmic ScaleSTAR EAST 2013 49
Not if we switch to a logarithmic scale, which ensures that the world is still a happy place (and the slippery slope suddenly looks so much less slippery.)
Jan, 200 Feb, 400
Mar, 800Apr, 1600
Overtime Hours
Irrelevant Pie ChartSTAR EAST 2013 50
Or how about choosing an completely irrelevant chart type (Excel offers so many, and I’ve often seen them misused).
0%50%100%
JanFeb
Mar
Apr
Jan, 200Feb, 400Mar, 800
Apr, 1600
Overtime Hours
Meaningless 100% Stacked Horizontal Cone ChartSTAR EAST 2013 51
Or a completely meaningless chart (but you should probably remove the numbers from the cones).By the way, I’m not suggesting that we should deceive people. I’m just demonstrating how easily information can be manipulated even with so-called “scientific” approaches.
STAR EAST 2013 52
And many years ago (before everybody could do this on their iPhone) we created a rap-song video to expose the organisation to automation testing. It was done with a low-end camera and heaps of staff support.Some of the claims were (admittedly) exaggerated but this clip is still available on Sticky Minds (www.stickyminds.com/) just search for “Automation Rap”.Think about whether using video might be useful in your communication.
STAR East 2013 2 May 2013
Thomas McCoy 14
VOICE 1
STAR EAST 2013 53
1. PitchWe need to raise our pitch to instil a sense of urgency and panic (or lower it to restore calm)
The way we use our voice is critical to getting our message across effectively.And vocal variation can help keep people awake.There are 4 important aspects:
VOICE 2
STAR EAST 2013 54
2. PaceQuick pace can also convey urgency. Or can be used to gloss over boring details. Use slow pace for important information
3. PauseAn appropriate pause (count silently to 5) can help focus attention before a key word
4. ProjectionNeed to project voice to ensure it is heard
VOICE 3
STAR EAST 2013 55
• Invest some time in voice work• Use a recorder to play yourself back (and don’t be
shocked the first time you hear yourself – that happens to all of us)
• Volunteer for presentations (even in non-work environments such as sporting clubs, kids school, etc.)
• Think about joining a group like Toastmasters (http://www.toastmasters.org/) which is dedicated to public speaking
ACQUIRING INFORMATION
STAR EAST 2013 56
STAR East 2013 2 May 2013
Thomas McCoy 15
INVESTIGATIVE JOURNALISM
STAR EAST 2013 57
• Find several sources (don’t burn them)• Speak to as many relevant people as
possible• Verify information through reliable, factual
(preferably documented) evidence• Don’t take everything at face value
(because people may be running their own agendas)
SOURCES• Can increase our
understanding of “what’s really going on”
• Can serve as early warning of risks
• Respect “off the record” comments
STAR EAST 2013 58
You can independently verify off the record comments but you can’t credit them to the source, or even mention the source.
DEVELOPING SOURCES 1
Administrative
Business
Marketing
Technical
Finance
ManagementTentacled
Tester
STAR EAST 2013 59
To develop sources of the greatest value it can be useful to put your tentacles out as widely as possible (rather than remaining cloistered within IT).
DEVELOPING SOURCES 2
• Work social club• Sporting teams• Lunches• Attending presentations
and events• Joining the office choir
Departmental Choir
STAR EAST 2013 60
And there are many ways of reaching out beyond your immediate work environment. I can’t sing but still joined the office choir and was able to hear how the IT group appeared to people on the outside.
STAR East 2013 2 May 2013
Thomas McCoy 16
CRITERIA FOR EFFECTIVE SOURCES
• Past suitability• Reliability• Trustworthiness
• Authoritativeness• Articulateness• Discretion
STAR EAST 2013 61
INVESTIGATIVE TECHNIQUES
STAR EAST 2013 62
• Infiltration• Impersonation• Secret filming and/or recording
Sometimes it may be necessary to “infiltrate” meetings you haven’t been included in.Or to impersonate somebody (e.g. via a phone call) to obtain information. (You should not impersonate an existing person, but you might impersonate a generic role e.g. “I’m ringing from tech support”).Be very careful with secret filming or recording (more on this later).
Sheila
Chaz
Bluey
Kylie
Ernie
Tezz
Molly
Nicole
Sidney
Snowy Talisha
Bazza
Brian
Hugo
Kezza
Magda
MAPPING THE ORGANISATION 1
STAR EAST 2013 63
In understanding the operation of an organisation it can be useful to map the informal relationships onto the formal org chart as shown in the next slide.
Sheila
Chaz
Bluey
Kylie
Ernie
Tezz
Molly
Nicole
Sidney
Snowy Talisha
Bazza
Brian
Hugo
Kezza
Magda
is cousin of
disagrees with
got promoted over
plays rugby with
teaches trombone to son of
MAPPING THE ORGANISATION 2
RELATIONSHIPSSTAR EAST 2013 64Much more informative.
STAR East 2013 2 May 2013
Thomas McCoy 17
OVERT RECORDING: USABILITY TESTING
STAR EAST 2013 65
ALWAYS DO USABILITY TESTING
STAR EAST 2013 66
• It is cheap and can be run on a laptop• Testing even one user is better than no users• Have them perform some tasks and ask them
to “think out loud” as they complete them• Software like TechSmith’s Morae product will
capture the screen, the user and the audio in real time
• The “video evidence” can be compelling when shown to developers
STAR EAST 2013 67
We watched a usability clip where the “save” button had been removed from a new version of the system (which now did automatic saves).This traumatised a user of the old system who could no longer find a “save” button and did now know whether her work would be lost when she exited. She became “stuck” and would have rung the help desk for advice.
Fake SAVE button
STAR EAST 2013 68
My recommended, innovative, low-cost solution, of adding a fake “save” button to the application so users could still click it and reduce their anxiety, was rejected.
SAVE
STAR East 2013 2 May 2013
Thomas McCoy 18
COVERT RECORDING
STAR EAST 2013 69
COVERT RECORDING 1
STAR EAST 2013 70
• With the capability of today’s mobile devicesit is trivial to record audio and/or video covertly(you no longer need to be a secret agent)
• Many technical meetings generate so much information so quickly that recording is the only way of obtaining accurate notes
• Ideally you would ask participants if you can record (however, this makes some people so uncomfortable that sometimes not asking may be more humane) but check with local laws and company policies
COVERT RECORDING 2
STAR EAST 2013 71
• In the ACT (where I come from) we have a funny law where you’re allowed to covertly record any conversation that you were part of as long as at least one person agrees (and that can be you) ☺
• Note that covert recordings should never be used against somebody (once relevant notes have been made they should be destroyed)
• Here is an example where somebody inappropriately used covert recording to request a testing extension while the boss was going about his business (with the aim of later playing it back to the boss as proof of the extension): www.thomas.mccoy.name/DontDoThis.mp3
OUR TESTING RESPONSIBILITY(AND BAD ERROR MESSAGES)
STAR EAST 2013 72
STAR East 2013 2 May 2013
Thomas McCoy 19
Client Registration
Family Name:
Date of Birth:
Given Name:
Date of Death:
Gender:
System Date/Time: 22/4/2013 2:25PM
Nurk
Fred
M
01/01/2015
STAR EAST 2013 73
In testing a client registration application I entered a future Date of Birth to check for the error condition (note that Date of Death was left blank).As expected, an error message was displayed and I could have marked the test as “passed”.
STAR EAST 2013 74
However, I was horrified by the meaningless, technobabble “error message” that was clearly written by tech-heads with zero communication skills. As testers, when we see something like this, I feel we owe it to the end user to report it, even if it isn’t on the official test plan.
[1] Wrong field values or value types detected in Date of Birth.Please re-enter your field values. If you need additional assistance, please refer to the documentation.(SBL-UIF-00299)[2] The value '01/01/2015' for field 'Date of Birth' is required to be '<= Today() AND <= [Date of Death]'.Please enter a value that is '[Birth Date] <= 22/04/2013 AND [Birth Date] <= '.(SBL-DAT-00521)
STAR EAST 2013 75
Looking at this message reveals many common problems:1. Should not use terms like “wrong” and “detected”, they are negative2. Should not use “field values or value types”, this is technobabble3. The cryptic error code is meaningless4. Why are there three versions of the error message?
[1] Wrong field values or value types detected in Date of Birth.Please re-enter your field values. If you need additional assistance, please refer to the documentation.(SBL-UIF-00299)[2] The value '01/01/2015' for field 'Date of Birth' is required to be '<= Today() AND <= [Date of Death]'.Please enter a value that is '[Birth Date] <= 22/04/2013 AND [Birth Date] <= '.(SBL-DAT-00521)
STAR EAST 2013 76
5. Why is a function call i.e. Today() in an error message?6. Why are mathematical comparison operators and Boolean logic used?7. Why is Date of Birth written in 3 different ways: Date of Birth, ‘Date of
Birth’ and [Birth Date]?8. Why are square brackets used around variable names?9. Why do dates sometimes have single quotes and sometimes not?
STAR East 2013 2 May 2013
Thomas McCoy 20
[1] Wrong field values or value types detected in Date of Birth.Please re-enter your field values. If you need additional assistance, please refer to the documentation.(SBL-UIF-00299)[2] The value '01/01/2015' for field 'Date of Birth' is required to be '<= Today() AND <= [Date of Death]'.Please enter a value that is '[Birth Date] <= 22/04/2013 AND [Birth Date] <= '.(SBL-DAT-00521)
STAR EAST 2013 77
10. Why does the last comparison end with <= '. (this is because Date of Death was blank, so why are we comparing with blank)?
11. Why is there a second meaningless error code?The entire error message, of more than 50 words and meaningless codes and symbols, could be replaced with:
Date of Birth may not be in the future. Please re-enter.
FIREFOX ERROR MESSAGE
STAR EAST 2013 78
Firefox has moved towards some soft and cuddly error messages where the computer takes the blame and becomes humanised.This does not add value and quickly becomes tedious.Better to just write meaningful and useful messages in plain English.
ERROR MESSAGES 1• Good error messages should answer some
key 5W&1H questions:1. Describe Problem (clearly and succinctly)
WHAT2. Cause (in as much detail as necessary)
WHY3. Solution (steps to fix problem)
HOW
STAR EAST 2013 79
ERROR MESSAGES 2Should be:
understandablerelevantactionableuser-centredbriefclearspecificcourteousrare
STAR EAST 2013 80
STAR East 2013 2 May 2013
Thomas McCoy 21
PROMPTING ACTION
STAR EAST 2013 81
• If you are experiencing difficulty in conveying the importance of having an issue addressed, mocking up adverse media coverage can help
• On the next slide I have created a fictitious story about the error message and can then innocently ask “but what would happen if something like this came out in the paper”?
• Having it looking real, rather than being hypothetical, may be more persuasive
• “Stock prices plummet” might be another example
Cryptic error messagecauses user chaos
Help Desk phones were inundated as users struggled with a cryptic error message that would require a computer science degree to decipher.
The message, which included comparison operators, Boolean logic, function calls, field names and
variables, was described by one user as “complete gobbledegook”.
The error condition prompting the message was triggered when a user typed a date of birth for a client that was in the future.
While this is clearly invalid, the message was of little use in
THE TESTING TIMESwww.testingtimes.com.au May 2013
The cryptic error message
CHECKING ACCURACY (OURS)
STAR EAST 2013 83
MISTAKES
STAR EAST 2013 84
• Since we critique the work of others it is reasonable that we are held to high standards
• In the same was as newspapers sometimes issue corrections, we should promptly admit mistakes
• An excellent way of checking our written material is to hear it read out loud
• The Office 2010 products (Word, Outlook, Excel) have an inbuilt reader that can read any text highlighted by the user
STAR East 2013 2 May 2013
Thomas McCoy 22
THE OFFICE 2010 INBUILT READER
STAR EAST 2013 85
• To use the reader (called “Speak”) you need to add the Speak icon to your Quick Access Toolbar
• You can then select any text and click the icon to have it read out
• You can use it to read back e-mails before you press “Send” or to read out long, boring documents when your eyes are fading
• A video to show you how to do this is available at:www.thomas.mccoy.name/SpeakFeature.htm
QUESTIONING
STAR EAST 2013 86
QUESTIONING
STAR EAST 2013 87
• At the heart of journalism• Requires substantial research and
preparation• Journalist often knows answer in advance• Questions can be used to convey an
implicit message in a less threatening way than statements
EFFECTIVE QUESTIONING 1
STAR EAST 2013 88
• Do research, plan questions and strategy and write them down
• Be clear on the objectiveof every question
• Think about likely responsesand your follow-up question
• Make person comfortable with informal conversation to start with
STAR East 2013 2 May 2013
Thomas McCoy 23
EFFECTIVE QUESTIONING 2
STAR EAST 2013 89
• Ask mixture of open and closed questions
• Do not make it feel like an interrogation
• Adopt a non-threatening and inquisitive manner
• Smile and nod, endure silences,don’t interrupt, be polite,stay neutral
EFFECTIVE QUESTIONING
STAR EAST 2013 90
• Record answers (shows interest)• Repeat answer back for confirmation• Take special note of anything that comes “after the
interview”• Use leading questions, if necessary e.g. “When this
error message frustrates the user, who are they supposed to call”
CHOOSING THE MEDIUM
STAR EAST 2013 91
• Face to face• E-mail• Phone call• Post-it note• SMS • Social media
Your medium will depend on the message. For an informal message a Post-it note may be fine. For something more serious you may want to do face to face with e-mail follow-up.
CHOOSING THE CONTEXT
STAR EAST 2013 92
• One-on-one in private• One-on-one in open plan• Daily stand up• Team meeting• Project meeting• Executive meeting
Choose your context carefully and keep participants to an absolute minimum.
STAR East 2013 2 May 2013
Thomas McCoy 24
WHAT KIND OF QUESTIONS?
STAR EAST 2013 93
• This depends on who you are talking to, what you are hoping to achieve and what motivates them, for example:
Project Manager – not meeting deadlinesBusiness Analyst - incomplete or defective requirementsDeveloper - coding errors
EMBEDDED MESSAGE EXAMPLES
STAR EAST 2013 94
Project Manager• “How much overtime are we
able to accommodate under the current budget?”
• Embedded message: There is no way we can finish testing on time with the existing resources.
EMBEDDED MESSAGE EXAMPLES
STAR EAST 2013 95
Business Analyst• “I’m just wondering what
happens on the registration screen if the user has registered more than once?”
• Embedded message: These requirements have holes.
EMBEDDED MESSAGE EXAMPLES
STAR EAST 2013 96
Developer• “What happens if the
user puts in the year with just the last two digits?”
• Embedded message: There is a bug in the date routine.
STAR East 2013 2 May 2013
Thomas McCoy 25
DEVELOPERS AND TESTERSHOW CLOSE IS TOO CLOSE?
STAR EAST 2013 97
EMBEDDED JOURNALISM …
… OR IN-BEDDED?
STAR EAST 2013 98
STAR EAST 2013 99
“I found myself falling in love with my subject.I fell in love with ‘my’ Marines … the point wasn’t that I wasn’t reporting the truth, the point was that I was reporting the marine grunt truth … which had also become my truth”.US journalist Gordon Dillow quoted in Journalism Studies journal.
AGILE TEAMS: TOO CLOSE?
STAR EAST 2013 100
• Gordon Dillow was one of around 800 journalists embedded with soldiers in the Iraq war
• They depended entirely on the soldiers for food, shelter and safety
• It has been suggested, not just by Gordon, that their reporting lost some of its independence
• Is there a risk that our testing may lose independence (despite all the benefits that Agile allegedly offers)?
STAR East 2013 2 May 2013
Thomas McCoy 26
INDEPENDENT TEST TEAM
There are several ways in which a test team could be kept “independent”:• Physical?• Administrative?• Both?
STAR EAST 2013 101
• With our independent testing team we have found that it:• frees members from
relationship pressures• lessens “informal” fixes• builds a collegiate atmosphere• allows us to unashamedly
celebrate successes (which may be development failures)
Give me a hug,I found a bug!
INDEPENDENT TEST TEAM
STAR EAST 2013 102
It has its own space, which allows staff contributions and celebrations to be put on public display, creating positive energy.
INDEPENDENT TEST TEAM
STAR EAST 2013 103 STAR EAST 2013 104
And it allows us to shamelessly promote testing principles (and call out the numbers when the inevitable defect misses occur).
STAR East 2013 2 May 2013
Thomas McCoy 27
PERSONAL COURAGE
STAR EAST 2013 105
Like journalism, testing often requires personal courage, as shown in the following slides…
PROJECT MANAGER AFTER READING YOUR E-MAIL ON HIS IPHONE
STAR EAST 2013 106
PROJECT MANAGER DISCUSSINGIT WITH YOU
STAR EAST 2013 107
HOW YOU FEEL AFTER THE “DISCUSSION”
STAR EAST 2013 108
STAR East 2013 2 May 2013
Thomas McCoy 28
PROMOTING OURSELVES
STAR EAST 2013 109
• We created a 2-minute videoclip entitled Life Without Testing
• The clip was effective in focusing attention on the serious consequences of undetected bugs in the Medicare payment system
• You can watch it at:https://vimeo.com/65502104
FINAL THOUGHTS
STAR EAST 2013 110
• As with journalism, we have to keepre-inventing ourselves
• Our successes need to be communicated appropriately
• We have to expand our network of allies• And we must keep the end users at the
centre of what we do and advocate for them whenever possible
KEY POINTS
STAR EAST 2013 111
• Journalism and testing have similar skills and principles, particularly as they relate to an obligation to the truth and a sense of public responsibility.
• Testing is considered by some to be a vocation, rather than a job.
• Publicising our successes is a challenge.• Think about news values such as impact, proximity
and timeliness in your communication.
KEY POINTS
STAR EAST 2013 112
• Take the angle into account, use the 5Ws & 1H where possible, and remember the inverted pyramid.
• Take care with delivering bad news. Sometimes people only hear what they want to hear.
• Use personas to rehearse bad news delivery (if that works for you).
• Work on writing and speaking skills so your communication cuts through the noise.
STAR East 2013 2 May 2013
Thomas McCoy 29
KEY POINTS
STAR EAST 2013 113
• Pay special attention to the “headline” and first paragraph in your writing; people may not read beyond that.
• Use Flesh-Kincaid to check readability (but avoid using Green Eggs and Ham to fake it).
• Use the Speak feature in Word and Outlook to help you catch writing errors before you send them out (it also lets you rest your eyes when reading long documents).
• Remember George Orwell's writing advice.
KEY POINTS
STAR EAST 2013 114
• Consider using innovative communication methods, such as word clouds, video clips, and even mocking up a negative news story to illustrate the potential public impact of a defect.
• Remember that graphs can lie.• Practice recording yourself to improve voice
delivery.• Develop sources throughout the organisation and
consider mapping its relationships onto the formal structure chart.
KEY POINTS
STAR EAST 2013 115
• Capture information in “overload” situations by recording audio and/or video.
• Do a usability test whenever possible.• Review error messages for clarity and usefulness
as they appear during your testing.• Always correct mistakes and review your written
work.• Use questioning to gather information, but also to
convey implicit messages.
KEY POINTS
STAR EAST 2013 116
• When planning questioning, think about the medium, context, drivers (for the person you are questioning) and what you hope to achieve.
• Are embedded testers (e.g. agile) sometimes too close to developers?
• Personal courage is an essential aspect of journalism and testing. Be brave.
• Don’t lose your sense of humour (especially not if you can use it in a video to get your message through).
Top Related