The Scientific MethodResearch Methods CPE 401 / 6002 / 6003

Professor Will Zimmerman CEng FIChemE BScEng, MScEng, PhD (all ChE), PhD minor (Applied Maths)Chair, Biochemical Dynamical Systems

Motivation

Salesman: You do research, right?

Professor: Yes.

Salesman: Then you don’t know what you are doing!

Professor (indignant): WHAT?

Salesman: If you knew what you were doing, it wouldn’t be research, right?

(He lost that sale!!!!)

Scientific research is conducted in a realm of UNCERTAINTY.We do not know the outcome of our tests, but we do know howWe should conduct them – according to the SCIENTIFIC METHOD

What is the Scientific Method?

Attitude: open mindedness Step 1: Identify the target Step 2: Scholarship – gather information – make

/collect observations Step 3: Hypothesis formation Step 4: Test predictions of hypothesis with proper

experimental methods. Step 5: Check and interpret your results Step 6: Either start again at any previous step or

report your results.

Attitude: Open mindedness A scientist must be receptive to new ideas, but then must be skeptical

and conservative in the approach to claims made with such new ideas. This is the fundamental duality of the scientific method: no progress in science is made by dogmatic acceptance of canonical ideas. If canonical ideas were capable of solving the open questions, they would have already. However, new ideas are not “scientific laws”. They could be fundamentally flawed (self-inconsistent) or unverifiable (taken on faith?)

Scientists should be critically minded, which means searching for self-inconsistencies in new ideas, agreement/disagreement with observations, and unbiased – theories that are apparently “ridiculous” might just reflect our own bias – apply the scientific method before dismissing hypotheses “out of hand.” It is human nature to be “conservative” and closed-minded and many “scientists” are biased.

Step 1: Identify the target

One of the least attractive features of modern research practice is that research projects are frequently “externally” defined by non-scientific (political or market forces) agendas. Even if your projects aims are externally defined, how you get there (objectives) is usually under the control of the investigators. The “playing field” is defined, but the route through that field is owned by the researcher. The key attribute of a mature researcher is “ownership” of a research agenda.

For doctorates, the central condition is that the doctoral thesis should make original contributions to science. Therefore the target for the research should be to pursue “unbroken ground”.

The constraint for a Master’s thesis or dissertation is the much less restrictive “training in the methods of research.” Nevertheless, it is a “misnomer” to say that a research project could be unoriginal (cf. salesman’s point). With a Master’s project, the ownership may never reside with the student. With a doctorate, it must.

Step 2: Scholarship – gather information

A key observation is that research is not limited to scholarship. In many cultures, the definition of a learned person is that they are a scholar. The Anglo-American (perhaps European Enlightenment legacy) is that research is just as much about creativity as knowledge. Identifying the target (creative step 1) is as important as placing its context in the hierarchy of science (what is known).

Nevertheless, step 2, which really should be done simultaneously with step 1, is to gather information about the target. In my opinion, the thing that separates Professors from ordinary researchers is the “vision thing” – the ability to select targets in light of what is know simultaneously and with relative ease. These two steps are “having a nose” for where one might find new science.

For new researchers, the gathering information step is finding out about the field surrounding the target, and placing it in context. Trivially, the gathering information step could determine that the target is not original at all.

Step 3: Hypothesis formation

This is the philosophical step and the biggest creative element. It is a concise statement of something that the researcher has found is unproven in the body of scientific knowledge about the target, if it is proven, will have profound consequences.

In general, science is distinguished from all other forms of scholarly activity by being hypothesis driven.

(Computational) Modelling is an activity that is automatically a hypothesis – the model is the hypothesis.

Sensor and technology development is itself a hypothesis – that such a sensor leads to a measurement inference or such a technology will be effective.

Some hypothesis examples from my own research (confidential):

Helical turbulence. There is a pairwise mechanism for the build up large scale structures in helical turbulence that leads to better mixing and control over molecular contacting patterns. Technological implication: low NOx, high efficiency of combustors.

Colloidal electricity generation. Variation of magnetic permeability due to inhomogeneous fluid mixtures oscillating in a magnetic field create AC electricity in a pick up coil. Oscillatory motion of partially charged colloidal particles in an electric field create a streaming potential that is an AC electric field.

Oscillation of the air-liquid interface in a forming bubble can pinch of much smaller bubbles and therefore enhance mass transfer (wastewater treatment).

Capacitance or acoustic “spectroscopy” applied to wavy air-liquid interfaces can inform about the surface tension and viscosity of the liquid by variation of the interface oscillation and/or the frequency of the applied field.

Microfluidic plasma generation creates more efficient formation of hydroxyl radicals that can be used, instead of ozone, to sterilize aqueous liquids.

Step 4: Testing hypothesis through experiment / observation.

When the hypothesis is originally formed, part of the scholarship (step 2) is to test the hypothesis against known observations / experiments. If this is easy, we usually lump it together with hypothesis formation (step 3). If it is difficult, we call this test against known observations “analysis” and consider it separately, on pair with new experiments.

If the hypothesis cannot be shown to disagree with known observations by analysis, then the researcher must create an experimental plan to test it. Running experiments that no one else has ever run is a trivial addition to knowledge. The originality of the contribution to science is why the experiments should be run – what are the predictions of the hypothesis that they test and how do they test it. The experimental planning and testing of the hypothesis are skill sets that are fundamentally philosophical and applications of logic. The tools, perhaps surprisingly, are primarily statistical. Biologists are particularly good at understanding that statistics underly the formation of hypotheses, the experimental plan (central role of replication, error, variability and reproducibility), and the determination of the consistency of the hypothesis predictions with the experimental results.

Step 5: Check and interpret results

This is the most frustrating part of training many researchers. It combines:

Critical and logical thinking Statistical inference and parameter estimation Knowledge of the subject matter – interpretation is based on

putting the results in the context of what is known about the field already. What did you expect to find based on the hypothesis to be tested.

Step 6: Iterate or report If the research programme was well crafted, the hypothesis will be

tested. If the hypothesis was an aim or objective of the research programme, the results should be reported.

Are scientists guilty of “cherry-picking”? Are only positive results reported?

Reports should be as honest and complete as possible. They should be accurate. It is not the job of referees / examiners to correct researchers mistakes.

Scientific review should be open-minded and unbiased. Reviewers, however, should put the onus of proof on the investigators – are the hurdles reported sufficient to justify the claims. Are the activities reported fully and accurately so that others are clear about the hypothesis, predictions based upon it, and the tests conducted for consistency.

Food for thought: no theory is ever proven. In fact theories are only ever disproven or shown to be consistent with experimental observations.

Is this really the way science is done? Or is it just how it is presented?

An alternative approach to science is what I call a “fishing expedition.”:

1. You think you smell where there might be fish.2. You collect as many fish as you can.3. After considering all the fish, you notice an interesting feature.4. You report that fish have this interesting feature.

I call this “descriptive science” or pattern recognition. If the research is conducted this way, it should still be reported

according to the scientific method. When this is done, a logical and orderly mind may find “holes” in the analysis which suggest a variations of the hypotheses and new tests which are used to validate the hypotheses. In this light, the fishing expedition is really part of hypothesis formation.

Essay AssignmentWrite a two page essay on the application of the scientific method to any of

the following “hot” topics in current scientific discourse:

1. Evolution vs. intelligent design.

2. Electricity generation by “zero point energy”.

3. Perpetual motion machine developed in Dublin, Ireland.

4. “Cherry picking results in scientific trials”

5. The Bible Code.

6. “So what the BLEEP do we know?”

7. Quantum mechanics: Does God play dice?

8. What is consciousness? Are our brains just computing machines? (Roger Penrose)

9. Autism and MMR (Andrew Wakefield) Stand rules on plagiarism (collusion and borrowing apply) – put it in your

own words and grammar, cite your sources, and argue a case based on your opinion about how the scientific methods has been or should be applied.

Summary

There is only one “scientific method”. You might argue that you are engineers or technologists, and therefore hold yourself to a lower standard of conduct. But “estimates” and “guesswork” are still covered in the scientific method.

Fishing expeditions / pattern recognition / descriptive science is just information gathering, not the scientific method. It can be an integral part of hypothesis formation, but it is open to the criticism that the results are biased by “cherry-picking.”