ACM 4063COMMUNICATION RESEARCH

Lecture 7

Content analysis (CA)

A research technique for making references by systematically and objectively identifying specified characteristics within a text

A method for studying mass mediated and public messages

The main goal of CA is to describe the characteristics of messages embedded in public and mediated text

Any systematic procedure devised to examine the content of recorded information (Walizer and Wienir (1978)

A research technique for making replicable and valid references from data to their context (Krippendorf, 1980)

A method of studying and analyzing communication in a systematic, objective and quantitative manner for a purpose of measuring variables (Kerlinger, (1986)

Sometimes regarded as true communication research as it involves directly with media, media contents and messages – texts, photos, pictures, scenes, advertisements, etc.

CA involves studying information or materials already existed, not bounded by structured questionnaire, and can handle massive data with the help of computers

Advantages of CA

It can be unobtrusive – no interaction of invasion of privacy

It can deal with large volumes of material – many years of records, newspapers TV programs, etc

It is systematic – structured procedures and inline with the requirement of scientific method

It can corroborate with other evaluation methods – to value add or support the other research findings

Limitations of CA

The materials must be complete Need coders to be trained CA alone cannot serve as a basis for making

statements about the effects on the content on an audience

The findings are limited to the framework of the categories and definitions used in the analysis

Lack of messages to be studied relevant to research

Frequently time consuming and expensive

Steps in CA:

- formulate the research question- define the population in question - select an appropriate sample from population- select and define unit of analysis- construct the categories of content to be analyzed- establish a quantification system (coding form/sheet) - train coders and conduct pilot study - code the content according to established definition- analyze the collected data- draw conclusions and search for indications

Formulate the research question

Based on existing theory, from review of literature or practical problems faced by the public

What is the objective? Normally to summarize or describe the content of written materials, the attitudes or perception of its writer, or its potential effects on its audience

Good research question would provide clear guide on conducting CA

Define the population in question What is the main source of data or media

that is going to be studied? Data availability? Any recorded materials

as long as the information is available to be reanalyzed for reliability checks- assess quality

For example newspapers or magazines for how many years? TV programs for how many seasons? Chat interaction for how many days?

Select an appropriate sample

It is impossible to study the daily newspapers for ten years or watch every TV programs for several years, thus sampling is conducted

To do sampling the population must be complete If it is an event, for example General Election, your

may decides the time before the election and after the election to study the news coverage – purposive sampling

If it is for a long time, then composite week for each month in the sample (Monday from one month, Tuesday from another month, etc until all 7 days are completed)

As other sampling methods, the sample must be representative

There must be adequate sample size to be credible

For composite or constructed week it is possible to do probability sampling

For newspaper, it is possible to study specific part of the paper, e.g. front page only, editorial column, letter to the editor, etc

For TV, it is possible to study during prime time only

Select and define unit of analysis The smallest element of CA and the most

important In written content, the unit of analysis might

be:- a single word- a sentence- a paragraph- an entire article or story- a symbol- a theme

In TV, video and film, the unit of analysis can be:

- character- act- scene- time (seconds)- the entire program or film

Specific rules and definitions are required for determining the unit of analysis to ensure closer agreement between coders

Construct the categories of CA

At the heart of CA is the category system used to classify media content

This is to provide structure for grouping recording unit and help in systematic analysis

There is standard requirements that categories should meet

Each category must be clearly explained so that the coders can do the work smoothly

Requirement for content categories:- categories should be exhaustive- categories should be mutually exclusive- categories should be independent- categories can be constructed through groupings, scales and matrices

The number of categories depends on the research questions and the research objectives

Categories can be used to measure three quantification levels – space, frequency and intensity

These methods allow researcher to compare relative strength of the respective issues and get indication of its value or importance

Establish a quantification system

Normally nominal, interval and ration level of measurement are used

At nominal level count the frequency of occurrence of the unit of analysis

At the interval level it is possible to develop scales for coders to use to rate certain attributes

At the ratio level, measurement in mass media research are generally applied to space (print media) and time (electronic media)

In print media, column-inch measurement are commonly used to analyze news, editorials, advertisements

In TV and radio, the number of commercial minutes, the time and types of programs etc

It is also taken into consideration where the news are printed (front page or inside), when the story is aired (prime time, or lead news) as part of the ‘meaning’ in CA

Scales provide for the rank ordering of information such as “support, uncommitted, oppose”

Matrices are useful formats when researcher seeks more information about issues than simply whether they are present or absent

Train coders and conduct pilot study

As the categories developed are specific to the CA, training must be given to coders

Coders must achieve an acceptable level of agreement called ‘inter-coder reliability’ before the actual data collection can be carried out.

The coders must understand and familiar with unit of analysis and the content categories developed

The coders must undergo pilot study to familiarized themselves with what they are supposed to do

Pre-testing must be conducted before actual data collection

Involves coding a small portion of the material to be analyzed or some other similar materials

Test and revise the coding categories and instructions

The outcomes of pre-test should make the categories clearly specified, coding instructions are adequate, and coders are suitable for the job

Inter-coder reliability should be calculated during pre-testing and coding should begin when high inter-coder reliability is achieved (80 –90 % agreement is acceptable)

A check of reliability tells researchers the extend to which a measuring procedure can produce the same results in repeated trials

In CA, this means determining the similarity with which two or more people categorize the same material – independent coder

Code the content accordingly The entire material for CA must be already

available and in good quality Coders are competence to conduct the CA Coders know which news of part of TV

programs to be taken as sample in the study or not

For written media, key words in the title, headline, or the first paragraph or photograph of the news became the basis for inclusion in the study or not

Commonly coding sheets are prepared for this

Define the recording units including procedures for identifying them

Descriptions of the variables and categories

Outline the cognitive procedures used in placing data in categories

Instructions for using and administering data sheet or coding sheet

The coders must count the total news available in the sample so that comparison can be made later

For TV, the presence of ‘variable’ or category in the unit analysis is recorded

Coders must record all scenes and count the scenes that content the element being studied

Coders may have to read the news or replay the recorded TV programs several times in conducting CA

Analyze the collected data

Use the data recorded in coding or data sheet

Summarize the coded data Discovering patterns and

relationships within the data Describing the results Relating the results to data obtained

from other methods or situations or from assessing the validity of the analysis

Use descriptive statistics such as frequencies, percentages, proportions to present the results or findings, compare results across categories, and note changes over times

Use specific example from data to support your claim

The report should provide clear idea what was done, why it was done, and why the results provide a sound basis for conclusions and recommendations

Draw conclusions

Based on the outcome of the analysis

State the conclusion as it answers the research questions and research objectives

If there is unexpected results give plausible explanation

Describe what is the main outcome of the study

Give suggestions for further study or application by the relevant authority

A CA reports should consist:- the study objectives, and justification of the choice of data, methods, and design- a description of the procedures including sampling, units of analysis, coding instructions, results of inter-coders reliability test, procedures for data handling and analysis, and efforts at validating the procedures, and- the findings and their significance contribution