Tales from the Trenches - OR –

Replicating things that have already been said at this meeting

Robert R. Hoffman, Ph.D.

© 2011 Robert R. Hoffman All rights reserved

“Data Coding, Analysis, Archiving, and Sharing for Open Collaboration: From OpenSHAPA to Open Data Sharing,”

© 2011 Robert R. Hoffman All rights reserved

Experimental Psychology (Cognitive, Psycholinguistics)

U of Cinti, U of Minn

Expertise Studies Cognitive Systems Engineering

Human Factors

Me

© 2011 Robert R. Hoffman All rights reserved

Klein et al 2002

IEEE Intelligent Systems

Mental simulation Problem detection

Coordination Re-planning

Expertise development

(2 seconds ?)

Cacciabue & Hollnagel 1995

Macrocognition as a paradigm in

Cognitive Systems Engineering

Gunnar Johansson 1980

© 2011 Robert R. Hoffman All rights reserved

© 2011 Robert R. Hoffman All rights reserved

Communities of Practice

“Behavior” Versus Activity

Activity Theory

Work Analysis

Sociotechnics

© 2011 Robert R. Hoffman All rights reserved

Ancient History

“How fast was the car going when it

(bumped, crashed) into the other car?”

Barbara Tversky, et al.

© 2011 Robert R. Hoffman All rights reserved

That was then, this is….

then. . . . . .

Myth of the Normal Curve

Examples: Sampling under a stopping rule Traffic delays (lots of brief ones, rare long ones) Achieving progressive criteria in pole vaulting Errors in motor coordination tasks

Patil, G. P. (1960). On the evaluation of the negative binomial distribution with examples. Technometrics, 2, 501-505.

Sichel, H. S. (1951). The estimation of parameters of a negative binomial distribution with special reference to psychological data. Psychometrika, 16, 107-127.

© 2011 Robert R. Hoffman All rights reserved

© 2011 Robert R. Hoffman All rights reserved

"Call For Data"

•  Usability/Learnability analysis

•  Performance at the very first trials

of learning any task; any DVs

•  Exact modeling of discrete

non-Gaussian distributions

Learning geometrical patterns Learning to use a cell phone by the elderly Learning to operate an automotive GPS (route-finding) Learning to recognize voices in auditory localization Learning to control an avatar in a virtual world Learning of the structure of biological categories Learning to fly a cockpit simulator

9 data sets on hand, 9 more pending

© 2011 Robert R. Hoffman All rights reserved

© 2011 Robert R. Hoffman All rights reserved

?

© 2011 Robert R. Hoffman All rights reserved

?

© 2011 Robert R. Hoffman All rights reserved

?

© 2011 Robert R. Hoffman All rights reserved

?

Retabbing

Fonts

Column widths

Etc.

Rule #1

Clean-up is always necessary

© 2011 Robert R. Hoffman All rights reserved

What does "No OT" mean?

What did you really do?

Did I fix the tab delineations correctly?

Rule #2

You always have to go back and talk to the researcher

Worse. . . .

People forget things about their own data, even short-term

© 2011 Robert R. Hoffman All rights reserved

How do you cope with the consequences of these Rules?

Do you impose constraints?

- OR -

Do you acknowledge that clean-up will always be necessary, and figure out ways to make it easier.

The Control Challenge

© 2011 Robert R. Hoffman All rights reserved

© Robert R. Hoffman All rights reserved

Requirements v. "Desirements"

("help" versus "impose")

Designing for kluges and work-arounds

Hoffman, R. R. & Elm, W. C. (2006, January/February). HCC implications for the procurement process. IEEE: Intelligent Systems, pp. 74-81.

Koopman, P. & Hoffman, R. R., (November/December 2003). Work-Arounds, Make-Work, and Kludges. IEEE: Intelligent Systems, pp. 70-75.

© 2011 Robert R. Hoffman All rights reserved

Challenge #1

Finding data by data constraint (e.g., data type,

meaning)

Challenge #2

Finding data by design constraint (e.g., between v. within, etc.)

The Search Challenges

© 2011 Robert R. Hoffman All rights reserved

Challenge #3

Frame problems (a priori categories v. search categories)

Challenge #4

Practicalities of representation - cryptograms, acronyms, abbreviations

The Representation Challenges

© 2011 Robert R. Hoffman All rights reserved

Challenge #5

Chiding resarchers re: design limitations, methodological questions

Challenge #6

Topical research data v. Statistics itself as an area of research

The Purpose Challenges

© 2011 Robert R. Hoffman All rights reserved

Usefulness

Usability

Understandability

Learnability

Having seen all the Tools & Systems. . .

© 2011 Robert R. Hoffman All rights reserved

These features are measurable

e.g., Data on Researcher time, effort, resources

“labor intensive” “efficient data mining”

© 2011 Robert R. Hoffman All rights reserved

www.ihmc.us/users/rhoffman/main www.ihmc.us

Thank you!

© 2011 Robert R. Hoffman All rights reserved