8/7/2019 PT6_tcm53-161511

1/16

Focus group (FG)

8/7/2019 PT6_tcm53-161511

2/16

Table of ContentsFocus group....................................................................................................................................................1/14

1 Introduction......................................................................................................................................1/142 Methodology....................................................................................................................................2/14

2.1 Group design and dynamics...................................................................................................2/142.2 Moderating focus groups........................................................................................................3/142.3 Data input...............................................................................................................................4/142.4 Data output.............................................................................................................................4/14

3 Process.............................................................................................................................................4/144 Review.............................................................................................................................................5/14

4.1 Evaluation results..................... ..............................................................................................5/144.2 Operational aspects................................................................................................................5/144.3 Experiences............................................................................................................................6/144.4 Combinations.......................................................................................................................10/144.5 Strengths and weaknesses....................................................................................................10/144.6 Further work .....................................................................................................................12/14

4.7 References............................................................................................................................12/14

Focus group (FG)

i

8/7/2019 PT6_tcm53-161511

3/16

Focus group

sa Gerger Swartling (asa.swartling@sei.se)

1 Introduction

The focus group technique has become an important applied approach to integrate stakeholders' perspectivesand knowledge into Integrated Assessments (IA) and more recently and generally Sustainability Assessments(SA). Powell and Single (1996:499) define a focus group as: 'a group of individuals selected and assembledby researchers to discuss and comment on, from personal experience, the topic that is the subject of theresearch.' There are many variations of the basic method, but generally, a focus group is a method forcollecting qualitative research data through carefully planned group discussions with the purpose of obtainingperceptions of participants in a permissive and nonthreatening environment (Morgan 1988). The discussionsare guided by a skilled moderator, who works from a predetermined set of questions. The group membersinfluence each other by responding to comments made in the discussions. The results are analysed withquantitative and qualitative social science methods. If conducted and analysed properly, focus groups are

likely to stimulate learning, to increase awareness among participants and promote more democratic andeffective decisionmaking. In SA contexts, the technique is used to explore stakeholders' views andperceptions of sustainable development related matters with the purpose of reviewing existing policies andsupporting agenda setting and policy development (cf Kasemir et al 2003).

The focus group method is not a new phenomenon. Already in the 1940's, focus groups were used to explorepeople's reactions to wartime propaganda (Merton et al 1956). Subsequently, in the 1950s, focus groups beganto emerge as a method employed for market research (Goldman and McDonald 1987), and nowadays are stillan important tool for the systematic development of product marketing strategies. More recently, it hasbecome a technique of political campaigning and public policymaking (Morgan 1988). Thus in comparisonwith many other participatory techniques, focus groups are associated with a wider field of applications and a

longer history. Common for these variants of conventional focus groups are typically that they require a single(twohour long or so) session, among a reasonably homogeneous group of people who are unfamiliar witheach other. No preparations are made by participants, nor are they expected to develop consensus, solveproblems or take decisions on any matters. The method may be applied as standalone techniques, orcombined with e.g. questionnaires and various quantitative research methods. Also typical for theconventional focus group is that not much (scientific) information is put in the focus group. The aim of thefocus group is to get insight into the perceptions of the participants in the group, and the input of scientificknowledge may influence this perception.

In the 1990s, new focus group approaches were designed and applied in the emerging field of IA and morerecently in SA efforts. In this context, the technique is usually applied to explore stakeholder perspectives ofsustainable development related issues with the aim of reviewing existing polices and supporting policy

development. As a common approach to participatory IA, focus groups may be viewed of as a way todemocratise science matters and to empower ordinary citizens (van Asselt and Rotmans 2003) as well as ameans to improve the quality of IAs. (Jaeger et al 1999).

While conventional focus groups have been used as a means for assessing policy interventions, this hashappened without the provision of scientific knowledge to participants involved and without making theresults of the groups accessible to public and scientific dialogue (ibid). However, IA/SA related focus groupapproaches are informed stakeholder discussions, since scientific information is provided as stimulus to thediscussions (e.g. through computer models, graphic illustrations, scenarios etc.) to enable participants to reactto expert knowledge and, based on their own knowledge and views, develop their own opinions andpreferences of the topic under consideration. Thus the input from the tools trigger conversations among

participants that may have clearer sciencepolicy relevance than conventional group discussions. With regardto nonexpert involvement in IA/SA efforts, by combining focus groups with the use of e.g. novel scenariotechniques and models in a carefully designed procedure, focus groups with lay people are well equipped to

1/14

mailto:asa.swartling@sei.se

8/7/2019 PT6_tcm53-161511

4/16

make assessments with regard to local or regional environmental policy options, environmental risks arisingas a consequence of modernization (ibid) and future images (Kasemir et al 1999b). In general, stakeholderparticipation helps clarifying (albeit not reducing) uncertainty and conflicting issues and hence it can supportnegotiation processes on environmental policy better than any single 'optimal' account or scenario could.(Jaeger et al 1999). Moreover, focus groups possibly yield a wealth of empirical data, including minutes,audio and videotapes, texts produced by the groups and collages and questionnaire answers on the topic

(Jaeger et al 1998).

If designed properly, SA related focus groups are likely to provide relevant information and support tostakeholders engaged in sustainabilityrelated assessment processes. Moreover, they can also provide usefulfeedback to the scientific process by elucidating participants' expectations from, and perceptions of, the(potential) usefulness/user friendliness from notably models (cf. Dahinden et al 1999). Thus the focus groupprocess can contribute to a mutual learning process through the articulation and integration of a multiplicity ofperspectives, skills and competences.' (Rotmans and van Asselt 2002). Depending on the aim of the study, thisinformation may feed into model and scenario development exercises.

Compared to focus groups in the traditional sense, focus groups in participatory SA efforts require a moreambitious and policyoriented agenda and several successive meetings to allow participants to cover a widerange of aspects and the multidimensional scope that characterises theses types of assessment processes.This means that much more time and resources are required by organisers to plan and conduct focus groups inSA settings than in less complex types of investigations.

Eventually, the focus group analysis in SA studies aim at supporting agenda setting and policymaking andoccasionally this involves participants' own statements of policy options and recommendations for futurepolicymaking, e.g. in the form of 'Citizens Reports (Duerrenberger et al 1997, Kasemir et al 2003, Querol etal 1999).

2 Methodology

2.1 Group design and dynamics

Focus groups come in numerous shapes and sizes (see e.g the application of InDepth Groups in theULYSSES and Visions projects under 4.3). However, this variety is often considered to be a strength, becauseit provides researchers with many options and the technique can be applied in many different contexts. Focusgroups may involve different groups of stakeholders, e.g. representatives from interest groups, NGOs,policymaking bodies, or the general public. The stakeholders are addressed in their specific roles and relativeto their stakes in a particular environmental issue.

In the literature, homogeneity and heterogeneity among focus group participants is an issue for consideration.

In the SA context, focus groups are typically heterogeneous and sampled, aiming at sociodemographic andideological diversity among respondents, and thereby enhancing the probability of multiple perspectives andexperiences being represented in the assessment. However, the focus group method is neither bent oninvolving all relevant actors nor on getting a representative sample of the population, although SA focusgroup approaches put much more emphasis on the latter than the conventional ones. The method is aimed atdetecting patterns and trends in perspectives among social groups, which implies that it is entirely possible forthe organisers to involve certain social groups while leaving out others. If the focus group is run well and in a'fair' way (cf. Webler 1995), all participants have the opportunity to express themselves ('to contribute'), to askeach other questions ('to discuss') and thereby influence the collective outcomes ('to decide').

As the result of a single focus group may be biased due to specific people involved, moderation style etc.,even in small research studies a series of focus groups is appropriate to get reliable data for the analysis. Thisis particularly important in assessments on complex sustainability policy related matters, as a wide range ofperspectives is sought from a rather diverse sample of respondents. The issue under scrutiny may alsodetermine the selection criteria for recruitment of focus group participants and hence the group composition.

Focus group (FG)

2/14

8/7/2019 PT6_tcm53-161511

5/16

The number of participants involved may vary between half dozen and one dozen, but the typical group size isseven to ten participants. Too large groups may be more challenging to moderate, as they tend to requirestricter discussion rules to ensure that the participants speak one at a time and listen to each other, andconsequently, more rigorously structured discussions may inhibit the group members to speak freely andspontaneously with each other. On the other hand, too small groups are less likely to generate rich discussionswhere a wide range of perceptions and experiences are raised and reflected upon. As a basic rule of thumb,

more structured focus groups emphasize the research team's focus, whereas less structured focus groupsemphasize the group's interests (Morgan 1998).

Group compatibility does not always ensure the open and comfortable environment necessary for a smoothand fruitful focus group. Responses may be greatly affected by differences in background or opinions. Levelsof articulation and confidence may vary among the group, resulting in certain members being more vocal andless inhibited than others. This in turn may heighten the insecurities of shy or intimidated participants. Thus'Individuals may be stifled rather than stimulated by the group' (Frey and Fontana in Morgan 1993) althoughthe moderator should aim to prevent this problem.

2.2 Moderating focus groups

The invited participants meet for approximately 1.5 to 2.5 hours once or several times to discuss a theme thatis subject of the research. The role of the moderator is to guide the focus group discussion by askingpredetermined questions and to ensure there will be a good quality of focus group results that address theresearch objectives. The moderator is a welltrained professional who works from a predetermined set ofdiscussion topics. If necessary, (s)he intervenes in the process in order to stimulate the discussion. Eventhough the participants may wander off, the starting point remains determined by the organizers. In thiscontext, the moderator has an important task to ensure that the relevant topics remain the focus of attention inthe discussion without hindering participants from articulating their views and concerns. It is generallyregarded important that the participants are at ease with each other and speak freely, which may be moredifficult when the participants meet each other for the first time. On the other hand, it may be problematicwhen the participants already know each other as people may be more concerned about saying the 'right'things and about controlling their behaviour in front of acquaintances than in a group of strangers. In addition,some participants may dominate the conversation and hamper other group members to express their ownviews on the topic. Also in this context, the moderator has an important role to stimulate a friendly andpermissive atmosphere and a fruitful group interaction.

Effective moderating requires a complex array of skills. It is difficult to provide exact prescriptions for thestyle and strategy of moderators and there is a myriad of plausible ways to conduct a successful focus group.However, in order for the focus group discussion to become successful, some general personal qualities arerequired (Krueger 1998):

The moderator should be comfortable and familiar with group processes and dynamics, which

includes a mild, unobtrusive control over the group;

(S)he should possess a curiosity about the theme as well as the participants. Indifference, cynicismand apathy are quickly spotted by participants and inhibit the conservation.

(S)he ought to possess communication skills, both in writing and orally. Clear and precisecommunication avoids confusion among participants.

(S)he should have a friendly manner as well as a sense of humor as these are helpful bonding agents.(S)he should have a true interest in people, which grows out of a respect for each person.A good moderator is committed to multiple realities. Thus (s)he should be open to other methods,suggestions and approaches without bringing closure to the conversation.

The moderator should also possess listening skills and hence a slight dose of introversion is oftenhelpful. The participants should be at the centre of the stage and not the moderator.

Focus group (FG)

3/14

8/7/2019 PT6_tcm53-161511

6/16

2.3 Data input

Conventional focus groups usually rely on the information that is provided by participants, whereas SA focusgroups typically rely on some kind of scientific input that is presented during the meeting to support a processof social learning among participants. The scientific nature of the data input typically gives rise to argumentsabout their validity and are open to challenge and questioning by the participants. In brief terms, IA/SA focus

groups can be seen as discussions about arguments, not just statements of preferences (although these are alsoimportant ingredients in the focus group discussions).

The most commonly applied tool in an IA/SA focus group is a computer model, or scenarios generated frommodels, and this is often complemented with written handouts, oral presentations by invited experts and thelike. Apart from modeling and scenario tools, so far there appears to be few attempts to explore thecombination of various tools e.g. monetary (e.g. Cost benefit Analysis) and physical assessments andMultiCriteria Analysis (MCA) tools in focus group settings.

2.4 Data output

The focus group discussion is recorded in several ways: by field notes (from assistant moderator), by a taperecorder and sometimes also a video camera. Together with the notes and transcripts, flip charts, collages and,more rarely, written reports are outputs for analysis. The data may be analysed manually or with existingcomputer techniques and usually this is done by coding exercises.

3 Process

Each focus group session generates a rich amount of qualitative data in a few hours. However, to providevalid data, the focus group process requires very careful planning and timeline. In IA / SA projects, the

planning involves the preparation of the scientific input to ensure that it is presented to the stakeholders in apermissive, timely and userfriendly manner. This is particularly important with regard to focus groups withlay people. The steps taken by the organisers involve: Planning, recruiting, moderating, transcribing,analysing and writing up of the results (cf Morgan 1998, Krueger 1998, Duerrenberger et al 1997).

Planning includes developing a schedule for the various steps as well as an investigation of whatresources are necessary in each step; potentially involving outside experts, identifying participants onthe basis of casespecific selection criteria, inviting participants, developing interview questions,arranging meeting venue and associated logistics.

1.

Recruitment, undertaken by research assistants or a recruitment agency. This is usually done bytelephone. Recruiting can be a difficult and timeconsuming task. Thus an approach is needed that is

both effective (the right people accepts the invitation) and efficient (a sizable number say 'yes'quickly). (Krueger and King 1998). In this context, a respectful attitude of the recruiter, reflected inwords over the phone, in case of telephone recruitments, and in written words in (following up)invitation letters, will be helpful. Participants may be recruited by the researchers, volunteers or anoutside group or professional agency.

2.

Moderation, undertaken by a moderator and backed up by an assistant moderator, who does therecording/notetaking.

3.

Analysis, undertaken by the researchers. The procedure for the analysis is contingent on the aims ofthe research. Generally, larger projects require more detailed analysis (based on detailed orsemidetailed transcripts) while smaller (conventional) shortterm projects need less time (typicallybased on transcripts but sometimes only field notes).

4.

Reporting, usually done in writing and sometimes complemented with oral presentations. Thequickest types of analysis rely primarily on debriefing sessions and field notes after each focus group.IA/SA projects often require much more time also for the reporting procedure.

5.

Focus group (FG)

4/14

8/7/2019 PT6_tcm53-161511

7/16

It should be noted that in many IA/SA focus group projects, transcriptions of audio tapes recorded during thediscussions serve as an important output for analysis. In such contexts, 'transcription' come as a fourth step inthe research process. Such transcription is typically carried out by research assistants. For experts it may take2.5 hours per recorded hour and for less experienced transcribers, it may take some 5 or so hours per recordedhour. Good transcription equipment is required for this. However, in some settings, or in early project phases,field notes (if validated by multiple members of the research team) may be sufficient. Given the relatively

high costs of transcribing and full coding, it may be more effective to use carefully selected parts of thediscussions for partial transcription rather than make a full transcript as a standard procedure.

The entire process of smaller conventional focus group projects (about four meetings) in this vein may requiresome six weeks or so, but this only includes a few weeks of reporting. However, in IA / SA projects, severalmeetings per group, more groups in total, more complex forms of transcription based analysis and reportingare required (cf Duerrenberger et al 1997), at least three months in small IA projects and up to three years orso in larger (crossregional) projects in which FG research is embedded in a larger research context includingother aims and activities. Apart from the general scope of the project, as noted above the form of the reporthas substantial impact on the timelines and hence the focus group session itself (and the moderation of it) isonly the tip of the iceberg.

4 Review

4.1 Evaluation results

Focus groups are appropriate for providing insights into stakeholders' perceptions of what the problem areasare and why they are considered to be problems in relation to other issues. The tool can elicit respondents'views on how problems and issues can be addressed and in which contexts it is appropriate to do so.Comments made by the participants can tell us about how stakeholders perceive the characteristics of certainissues, and the relative importance and weight of problems and conflicting assumptions in comparison toother issues. Furthermore, it generates good insights into stakeholders' validations of policy proposals. From a

rational or strategic point of view, focus group participants do not necessarily arrive at the most adequateselection of a policy option. However, the results help guide the researchers and policymakers to determinewhich alternatives are preferred by stakeholder groups and the focus group project may have a considerableimpact on the selection of policy options. The focus group participants can advantageously be tasked withcomparing different proposals and assessing their applicability/suitability in given situations. However, thisprocess does not typically promote implementation of the selected policy option. The tool is useful forinvestigating stakeholders' views on how plans and programs being implemented work out in reality, and howto close gaps between expectations and actual performance of such policies.

4.2 Operational aspects

It is hard to give estimates about the time required to organise a focus group. Generalizations are difficult tomake, since the geographical scope and the complexity of the research topic require very diverse focus groupapproaches and different levels of extensiveness and detail of the analysis and reports. While small projects inthe traditional sense require at a minimum 6 weeks altogether, SA related focus group research typically needsmuch longer time, many more organisers, and more focus groups in various geographical locations. Ifinvestigators intend to carry out focus group research for assessments involving a smaller geographical andtopical scope and limited preparations/use of assessment tools, a set of six to eight focus groups can beorganised and analysed within four months. In contrast, focus groups in assessment projects that are large inscope (several case study regions in a comparative context), that focus on complex issues (e.g.multidimensional and interdisciplinary aspects of global environmental risks and policy options) and thatinvolve assessment tools that need to be tailored to/configured for the focus group context (notably computer

models) usually require 2448 personmonths. It should be noted that, while most EU wide IA / SAinitiatives using focus groups span over a threeyear period (FIRMA, ULYSSES, HarmoniCop, Visions)these projects typically run other research activities in parallel with the focus group process, e.g.

Focus group (FG)

5/14

8/7/2019 PT6_tcm53-161511

8/16

modelling/configuration of models, scenario development, development of methodological approaches,theoretical studies and policy dialogues.

The costs of the organisation of focus group research typically involve staffing (the work time of researchers,the moderator/s and assistants), focus group sessions (possibly room rental, honorarium and refreshments)equipment and supplies (telephone, mailing, printing/copying, miscellaneous). Based on the calculations made

in the US context in 1997 (Morgan 1998), the outofpocket expenses associated with smaller inhouse focusgroup projects (about four focus groups and project duration of six weeks) are roughly $ 1,000 (about 770Euro). Factoring in the staff time (excluding the hiring of professional moderators) the cost is closer to $ 5,000(approx. 3,850 Euro). The equivalent cost of larger projects conducting eight focus groups (excluding travelcosts, including staffing of professional moderator, recruitment agency/support staff as well as transcription oftapes) is roughly $ 40,000. However as noted above, these cost estimates are in the lower range of largerIA/SA projects, and can easily add up to well over 150,000 Euro per case study region. With regard toequipment a high quality tape recorder with microphone, transcriber, and possibly video camera are required.

In SA contexts, scientific input is required to enable focus group participants to assess these complex issues,potential policy options and actual policy measures. The high level of uncertainty of sustainability issues (noperfect technical tools available, lack of knowledge about global environmental risks and methodologicalapproaches) poses challenges to the availability of data applied as scientific input to stakeholder assessmentsin IA / SA settings, which need be taken into account during the planning phase.

The output of a focus group consists of qualitative data elicited through statements made by the participants.The discussion is usually taped and transformed into transcripts that are analysed through different analyticaltechniques.

The focus group technique is fairly straightforward and comprehensible and suits multiple cases. Nonetheless,the focus group approach requires casespecific and careful planning and execution. The unclear definitionand flexibility of this participatory method may complicate the study further. There are numerous possibleapproaches to the tool possible in SA settings and this may blur the transparency (for an illustrative example

of a focus group approach, see 'Experiences').

It is important to keep in mind that the focus group discussions are always context and momentdependent,and it is the opinion of that particular group, at that particular moment. The time before the results becomeinappropriate for further analysis depends on the context. Usually the data results can be used for several years(at least three), but as the social life and environment changes and as people change opinions and perceptionsin relation to new developments, the focus group results sooner or later become out of date. The time scale ofthe tool is not possible to quantify. While the present is normally the focus of the discussions in conventionalfocus group contexts, SA related focus groups are particularly appropriate for exploring longterm policyoptions with regard to potential future policies, risks, conflicts etc, although the exactness of 'future' or'longterm' is usually difficult to specify.

Focus group results are limited to the social segments of the population they are extracted from.Generalisation strongly depends on the number of focus groups and the number of sessions per groupperformed. The larger the number, the more one can say about the general trends and patterns of certain socialgroups.

4.3 Experiences

As this tool is a commonly applied qualitative participatory method, worldwide and over time, there aresimply too numerous focus group projects for anybody to keep track on. However, since the majority of thesefocus on other topics than sustainable development, and thus require much simpler approaches, they are also

rather inapplicable to the tool evaluation in the Sustainability ATest project. Consequently, the followingexemplified initiatives are limited to the use of focus groups in the context of IA/SA only. In many of theseprevious or currently ongoing projects, focus groups have been only one out of several participatory

Focus group (FG)

6/14

8/7/2019 PT6_tcm53-161511

9/16

techniques, such as the FIRMA, ULYSSES, VISIONS, Georgia Basin Futures and Geocognito projects. MostIA / SA projects involving focus group research have been carried out in Europe.

EU wide

The Urban Lifestyles, Sustainability and Integrated Environmental Assessment (ULYSSES) project (EU DG

XII, 19961999) was the first largescale, systematic attempt to explore focus groups in IA in acrossregional setting. On the basis of experiences with focus group research involving 600 citizens in sevenurban regions of Europe, the project designed the IAFocus Group methodology that allows lay participantsto interact with expert inputs (e.g. information on environmental change) on the themes climate change andenergy use. There were also some regional variations of focus group design, such as the InDepth Groups (theVenice region) and Citizen Panels (the Manchester region) (de Marchi et al 1998, Darier et al 1999a, 1999b).The particular IAFocus Group consisted of six to eight participants, which were recruited on the basis ofcriteria involving a mix of people with regard to place of residence, age, gender, occupation/education,income and attitudes towards the environment. Following the same agenda across regions and focus groups,the participants met five times for 2.5 hours. The first phase centred on participants' concerns aboutenvironmental problems and their spontaneous expressions of feelings about climate change and energy use(in part involving the production of collages). In the second phase, the groups were exposed to and interactedwith expert opinion (in particular some IA models) in discussions on the global context of climate change andregional options for energy use. The scientific input, particularly computer models, was normally presentedand discussed with the support of a second, model moderator. The third phase involved participants' making asynthesis of their views after completing the exercise, e.g. in the form of a written Citizens' Report. Thisreport included the group participants' assessments of global and regional climate change, proposals formeasures and who should take action to solve them; as well as foreseeable difficulties in achieving theproposed policy options (for further information, cf. Kasemir et al 2003, chapter 1). The project has oftenbeen considered successful in terms of its methodological advancement of the focus group technique in thecontext of IA, with sustainability policyoriented findings and lessons learnt for participation in sustainabilityscience (cf Kasemir et al 2003, part two). The method has been applied in e.g. the NOVAQUATIS andFIRMA projects (see below).

The integrated water assessment project Freshwater Integrated Resources Management with Agents (FIRMA,19992003) aimed at improving water resource planning by combining agentbased modelling and integratedassessment to describe physical, hydrological, social and economic aspects of water resource management inan integrated manner. This involved development of a generic model to be applied in five contrasting regionsin Europe. The outputs include the prototype models in the form of tools usable by managers, a methodologyfor developing and applying agentbased models, and educational materials for water resource managers andmodellers. The project involved focus group research with NGO representatives in the Thames region (UK),various stakeholder groups in the Orb Valley (France) and citizens in the Zurich region (Switzerland) focusingon regional water assessment issues in conjunction with models. The project developed amodelbuildingaslearning participatory process. The entire participatory study as reported by stakeholders

has increased their understanding of the complexity of the system and their willingness to interact with eachother in the future. The downside of this process is the cost, in term of time, money, resources and socialinteraction.

The IA project VISIONS (19982001) created a set of alternative scenarios for future sustainabledevelopment paths up to 2020 and 2050. Scenarios were used to test and apply stateoftheart software toolsin combination with focus group research along with Indepth Groups in the Venice region. In this region,four InDepth Groups were conducted to allow citizens to debate issues concerning sustainable visions forVenice and Europe, exploring not only the governance issues but also possible action and alternatives. Theparticipants were citizens of Venice representing two different generations. Hence, participants were recruitedon the basis of age, but also based on gender, education and occupation. No material was provided in advance.

In all groups, scenarios were presented using multimedia materials, slightly different according to age group;yet, more 'classical' versions of the scenarios were available if any of the participants would request it. Duringthe sessions, scenarios for Venice in 2050 were proposed by the citizens. The participatory process was not

Focus group (FG)

7/14

8/7/2019 PT6_tcm53-161511

10/16

framed in a policy context; therefore it was not required from the participants to assess policy matters. (Formore information about InDepth Groups, see e.g. de Marchi et al 1998 and Burgess et al 1988).

The HarmoniCop project (20022005) included the use of focus groups in two case studies (Spain andScotland) to generate practical information about participatory processes in river basin management inEurope, and to support the implementation of the public participation provisions of the European WFD. The

groups were composed by a selected number of 57 people intended to represent maximum diversity ofinterests, values and knowledges. Most of the participants were considered experts or else had a direct stake inthe management of the Muga resources, but also some few lay people participated. The focus groups hadmainly an explorative character and therefore, their purpose was not to create binding commitments betweenparticipants. It has been noted that the social learning aspect is one that is dependent on the possibility andability of relevant social actors and users to modify their own daytoday working assumptions with regard tothe value and functions of water both in nature and in society. The project found that it is undoubtedly adifficult task to assert to which extent such a social learning process has taken place.

The main objective of the ADVISOR project (20012004) was to provide an integrated project evaluationframework and methodology for the sustainable governance of Europe's river basins. This was achieved inpart by promoting public and stakeholder participation in integrated river basin governance in five Europeancase study regions. The project applied focus groups among a number of other participatory approaches toassess stakeholder perspectives of environmental policy options for the regions. The focus groups wereemployed in a context of research in two different cases: Case One involved two focus groups with citizensfrom two small villages affected by the Alqueva dam in Portugal. The local people were recruited by a localcollaborator upon criteria given by the research team (gender, age, education, any specific function in thevillages). The discussion was guided by a guiding questionnaire. Case Two involved one focus group withstakeholders in a multicriteria based discussion in the Malaga region (in Spain) on water options. The focusgroup approach was framed within multicriteria evaluation requirements. The group was used to debateissues concerning river basin governance, exploring not only the governance issues but also possible actionand alternatives. The focus group in Malaga gathered stakeholders from the tourism, agriculture and watersectors; local authorities and other relevant stakeholders a priori identified through an institutional analysis. In

terms of scientific input provided to the participants, in the two groups in the Alqueva dam region, only a tapefrom a TV programme was shown to trigger the discussion. In the latter case, multicriteria material wasprepared in advance on the basis of other forms of social research enquiry and then presented in the session tothe participants. The participants were involved in the preparation of the discussion through the socialresearch process (indepth interviews, mainly). The multicriteria tool used is a computer tool, its usagebeing prepared before application. The participatory process in the project was not framed in a policy context;therefore it was not required from the participants that they gave input into policy assessments. In this context,the project team had a socalled contract with participants, which was set from the beginning.

The River Dialogue project (20032004) aimed at identifying the best approaches to increasing publicparticipation and public empowerment in the implementation of the EU Water Framework Directive (WFD).

Seven to eleven focus groups were conducted (along with Citizens' Juries) respectively in three Europeanriver basins: the Motala Strm in Sweden, IJsselmeer in the Netherlands and the Emajgi River in Estonia tofacilitate the preparations and implementation of river basin management plans. The focus group approachwas designed in a more traditional way than in the case of most IA/SA studies, with less stringent andambitious discussion agenda, limited outside information, fewer meeting per group etc. The results indicatedinter alia that focus groups can, in different respects, contribute to involving the public to a higher degree inwater management, by increasing awareness and as a forum for participants to voice and refine their opinions.In addition, it appeared that focus groups should perhaps be regarded as a starting point, to help to find outhow people perceive a certain problem (problem finding), which can be combined with other methods thatcan facilitate the exploration of solution strategies to this problem (solution finding). In the case of the RiverDialogue project, the focus group approach was followed by the use of the citizens jury method. The

advantage of the focus group method, as indicated in the questionnaire participants filled out, was the opencharacter of the discussions but some considered the unclear connection to the policy process a disadvantage.

Focus group (FG)

8/14

8/7/2019 PT6_tcm53-161511

11/16

The UK

The UK (ESRC)funded Developing Public Involvement in Predictive Computer Modelling Project(20002002) (Yearley et al 2003) aimed at exploring how focus groups could be used to promote citizenparticipation in the environmental assessment of local air pollution in three UK cities: Sheffield, Bristol andYork. The focus groups, totalling about 27, were selfselecting (except for one 'control group' in York) so the

focus was their shared interest in the subject matter. A particular innovation introduced by the research teamrelated to the method of obtaining 'citizen maps'. The method allowed local citizens and stakeholders todiscuss problems and potential policy responses in an explicitly spatial manner, and to locate theirobservations and assertions physically on a map of their local area. The mapping exercises produced spatialrepresentations of local knowledge about air pollution which took into account local authority definitions ofair pollution, but which often stretched beyond these definitions towards a more holistic overview of theproblems of noise, odour and dust. The result is, in effect, a 'lay model' of local air quality. Initially the invitedgroup members drew and wrote on preprinted maps using coloured pens, using any categories orconventions they wished. During the exercise, the groups typically worked out their own approaches tocolouring and coding. At the end of the session, the researchers had a map to which the group had addedvisual and textual comments; these were subsequently digitised and then presented back either to the group orto individuals within the group to ensure that the digitisation was not unfair to the group's own representation.The key consequence of this procedure was that the focus group sessions gave rise to maps which weredirectly comparable to the outputs of the City Councils' of Sheffield, Bristol and York own computersimulations. This had two distinct advantages over earlier approaches. First, it meant that there was a form ofspatial feedback, which could be used as a form of citizens' quality assurance check on the officials' modelledoutputs. Second, it was possible to overlay the official and citizens' maps to see points of agreement anddisagreement. According to the research team, the focus group approach was largely successful because itbrought together lay perspectives involving the location, detail and specificity of community mapping(usually focusing on issues of transport related air pollution in urban areas) into the regional air quality policycontext.

Spain /Portugal

The IBERAQUA project (2002) aimed at developing a cooperative regime for the management of sharedwater basins in the Iberian Peninsula as well as providing knowledge and support for the implementation ofthe WFD and the LusoSpanish Convention (LSC). The study involved focus groups in three locations with atotal of 3550 citizens (510 in each focus group) with the aim of promoting and interactive process ofcolearning, and generating debates and social change through a public participation procedure. Twofollowup plenary meetings were arranged with the same citizens in which relevant information about thelegal instruments WFD and LSC was provided. Prior to the workshop the participants received and completeda questionnaire, which aimed at better framing the group discussions and obtaining secondary informationfrom actors unable to attend. The method was successful in terms of increasing communication amongstakeholders and bringing in a diversity of enriching knowledge and information into the water management

process. A drawback was that the participatory process was considered to be time consuming and resourceintensive.

Switzerland

The 'Integrated Climate Risk Assessment' (ICRA) subproject (19962000) of the Swiss Climate andEnvironment in Alpine Region II (CLEAR II) project (ongoing) aimed at developing an IA procedure foraddressing climate options for the Alpine Region. Seven focus groups using computer (TARGETS, IMAGEetc) models were run. The project codeveloped the IAfocus group methodology together with theULYSSES project.

The interdisciplinary NOVAQUATIS Project (http://www.novaquatis.eawag.ch) investigated consumerattitudes towards socalled NoMix toilets and the idea of anthropogenic nutrients using the IAfocus groupmethod (adopted from the ULYSSES project). It was explored how informed citizens judge risks and benefits

Focus group (FG)

9/14

http://www.novaquatis.eawag.ch/http://www.novaquatis.eawag.ch/

8/7/2019 PT6_tcm53-161511

12/16

of the technology of urine separation and the anthropogenic nutrient reuse in ecological farming. The mainissues that were addressed in this study included:Elicitation of consumer preferences regarding productattributes such as use, maintenance or design of the NoMix toilets in comparison to current technology;Assessment of citizens' perception of risks (chronic and failures) arising from pharmaceuticals in urine for theenvironment today and potential risks in agricultural products in the new system; and Assessment of citizens'perception of nutrient recycling and its contribution to sustainable development. They also discussed

arguments in favour or against closed nutrient circles and linking urban and rural areas of Switzerland.

Canada

In the fiveyear integrated project Georgia Basin Futures Project (19992004) focus groups have been runusing the integrated GBQUEST model in the form of an interactive game, to assess economic, social andenvironmental options for future development in the Canadian side of the Georgia basin.

4.4 Combinations

Conventional focus groups are often combined with other participatory methods, e.g. questionnaires,

telephone surveys, facetoface interviews, and sometimes other group dialogues e.g. citizen panels. In orderto provide meaningful stakeholder assessments of sustainability policy issues, combinations of stakeholdertechniques as well as other analytical tools have proven vital in many IA / SArelated projects:

Focus groups in this context are typically combined with Integrated assessment models (e.g. theapplication of TARGETS, IMAGE, PoleStar, CO2Lifestyle Calculator, GBQUEST and ICAM inthe ULYSSES, ICRA, Georgia Basin Futures, and VISIONS projects).

Focus groups have been combined with multicriteria tools in ADVISOR.Focus groups have been combined with Geographical Information System (GIS) in participatorymodelling (GISP) and community mapping exercises on air quality assessment in the 'DevelopingPublic Involvement in Predictive Computer Modelling' project.

Focus groups complemented the wider use of the Citizens jury approach in the River Dialogueproject.

The scenario technique Interactive backcasting, generated from the PoleStar model, have been used inULYSSES.

Focus groups have been combined with Sustainability Indicator based Assessment Tools (Indicatorsets for assessments) in the Swedish case study of ULYSSES.

Depending on the casespecific design and approach adopted, some other participatory methods can yieldsimilar qualitative data (citizens' panels, stakeholder dialogues). When deciding on which method to apply inSA contexts, one should carefully consider the relevance as well as the pros and cons of focus groups incomparison with other stakeholder analytical tools.

4.5 Strengths and weaknesses

Strengths

The focus group technique is a qualitative method that generates a rich understanding of stakeholderperceptions, experiences and beliefs.

It is an adaptable research method that suits a myriad of research topics and can be applied through agreat variety of approaches.

It can generate a diversity of perspectives and experiences that are shared and reflected upon.It is useful for exploring consumers' and users' attitudes toward products and marketing strategies.It is useful for discovering and identifying local problems, especially when the researchers have little

or minimal knowledge about the topic. Lay people possess local knowledge that outside experts maybe unaware of.

Focus group (FG)

10/14

http://ivm5.ivm.vu.nl/sat/chapdb.php?id=41http://ivm5.ivm.vu.nl/sat/chapdb.php?id=38http://ivm5.ivm.vu.nl/sat/chapdb.php?id=34http://ivm5.ivm.vu.nl/sat/chapdb.php?id=34http://ivm5.ivm.vu.nl/sat/chapdb.php?id=34http://ivm5.ivm.vu.nl/sat/chapdb.php?id=38http://ivm5.ivm.vu.nl/sat/chapdb.php?id=41

8/7/2019 PT6_tcm53-161511

13/16

The method is suitable for exploring stakeholder perspectives and assessments of complex issues,such as risk and uncertainty, water, air quality, climate change etc.

It is useful for identifying stakeholders' perspectives of goals, which in turn helps out in planning anddesigning projects that are likely to produce the desired outcome.

It is applicable during project implementation, as focus groups generate indepth and context specificqualitative information on e.g. how plans are working out, and how to close gaps between

expectations and actual performance.

The tool is good at generating assessments of successes and failures of projects and policies.It is useful in identifying and assessing community needs at local level in local plans and programs.More recent SA initiatives using focus groups have provided policy relevant insights into stakeholderassessments of sustainability policy issues.

In contrast with what is sometimes claimed, focus groups appear to be rather useful for investigatingsensitive topics.

The technique can serve an important role during the first stage in project development (e.g. prior tosurvey questionnaires), but is not limited to such a role.

It can easily and effectively be combined with other participatory research methods to strengthen thequality of results.

Weaknesses

To generate good and relevant results, the focus group process requires careful planning. Otherwise itmay result in useless data and thus a waste of time and resources.

It generates qualitative data that are rich and quite challenging to analyse. As in all qualitativeresearch methods, the robustness of the results depends on careful validation by multiple members ofthe research team, and ideally (in larger studies) validation across different research teams.

In complex assessment projects, the method becomes relatively expensive and timeconsumingprocess from beginning to end.

Focus group results may not yield policy relevant outcomes per se. Sustainability policy assessmentsrequire a more comprehensive research framework than even stateoftheart focus group techniques

can offer.

The method requires good skills and certain personal attributes of the moderator to generate a goodand spontaneous but focused discussion among participants. Unless there is a professional moderator,one risks getting data that are irrelevant to the purposes.

Due to the complexity and multidimensional scope of research topics, SA related focus groupresearch requires yet more systematic, careful and hence time consuming planning and analysis thanconventional ones.

The fairly elastic boundaries of focus group definitions often result in confusion about appropriate useand design of focus groups, and runs the risk of being misused for sale attempts, educationalseminars, therapy or consensus building exercises.

There is no exact template for interview guide to follow. It all depends on the specific project and

what the researchers want to get out of the focus group study. This runs the risk of results beinginterpreted by researchers in a biased way.

Quantification and strict generalisation to larger populations in not possible. Focus groups areindicative for the specific social group that they are representing. Therefore validity is somewhatlimited.

It is important to have appropriate equipment (notably highquality audio tape and microphone with360 degrees of recording of sound), otherwise there is a danger of getting unreliable data from thediscussions.

Researcher has less control in the group interview than in an individual interview.Focus group research is limited to qualitative research results. Generalisations and trend analysis mustnot signal a pseudoprecision by displaying numerical data and statistical detail that is not warrantedby the size of the sample. Instead, a categorical linguistic vocabulary referring to quantities (e.g. none,a few some, a majority or all) may be more appropriate to describe the proportion of participantsreferred to.

Focus group (FG)

11/14

8/7/2019 PT6_tcm53-161511

14/16

Not suitable for ranking exercises. There is often a reluctance to rank order of priority of problemsamong especially ordinary citizens.

4.6 Further work

Despite the wide spread and frequent use of this qualitative method, there is today some confusion on what afocus group really is and when it can be effectively applied. Part of this reason lies in the nature of the toolitself: the method is flexible enough to cover a myriad of topics, to be applied in different kinds of areas (e.g.science, public administration and marketing), as well as to allow for variations in group compositions,discussion guides, design of the process and output data. As argued above this is usually considered to be astrength insofar that the method can be tailored to the needs of the participatory assessment, but maynonetheless be deluding for practitioners who are not very familiar with the analyticalmethodologicalframework of the focus group method. Another source of the confusion is that, notwithstanding the growingrecognition of participatory IA in general, qualitative participatory IA approaches are still ratherunderdeveloped and this also applies to focus groups in such assessments. This is noticeable when comparingwith the relatively large proportion of time and resources put into model development vis a vis stakeholderanalytical approaches.

Moreover, further research is required to synthesize and document empirical findings of focus group researchin SA initiatives. For example, there is limited documented knowledge on the specific role and contribution offocus groups in overall participatory SA efforts and on how the method fits within the expanding toolbox ofparticipatory approaches for sustainability policy support. Examples of such initiatives could includeparticipatory evaluation frameworks and typologies of participatory tools in this context.

In addition, future work should also be devoted to the conduct of focus group studies to gather more empiricalevidence on the nature of, and opportunities for, focus group research in sustainable development relatedassessments and impacts on policymaking.

4.7 References

Burgess, J., M. Limb and C. Harrison (1988) 'Exploring Environmental Values Through the Medium of SmallGroups' 1. Theory and Practice. In Environmental Planning A 20:309326.

Dahinden, U., C. Querol, J. Jger, and M. Nilsson (1999) Using computer models in participatory integratedassessment Experiences gathered in the ULYSSES project and recommendations for further steps. InULYSSES Working paper 992. Darmstadt: Darmstadt University of Technology.

Darier, ., C. Gough, B. De Marchi, S.O. Funtowicz, R. GroveWhite, D. Kitchener, . Guimares Pereira,and B. Wynne (1999a). Between Democracy and Expertise? CitizensParticipation and EnvironmentalIntegrated Assessment in Venice (Italy) and St. Helens (UK). In Journal of Environmental Policy and

Planning:Vol, No 2: 103120.

Darier, ., S. Shackley, and B. Wynne (1999b) Towards a Folk Integrated Assessment of ClimateChange?. In International Journal of Environment and Pollution 11(3). Between Democracy and Expertise?Citizen Participation and Environmental Integrated Assessment in Venice (Italy) and St. Helens (UK).Journal of Environmental Policy and Planning.

De Marchi, B., S.O. Funtowicz, C. Gough, A. Guimares Pereira, and E. Rota (1998) The ULYSSES Voyage:The ULYSSES Project at the JRC. EUR 17760EN. Ispra: Joint Research Centre, the European Commission.

Duerrenberger, G., Behringer J., Dahinden, U., Gerger, ., Kasemir, B., Querol, C., Tabara, D., Schhle, R.,

Toth, F., van Asselt, M., Vassilarou, D. Willi, N., Jger, C. (1997) Focus Groups in Integrated Assessment: AManual for Participatory Research. In ULYSSES Working Paper WP972. Darmstadt: Darmstadt Universityof Technology.

Focus group (FG)

12/14

8/7/2019 PT6_tcm53-161511

15/16

Frey, J. H. and Fontana, A. (1993) 'The Group Interview in Social Research' in D. L. Morgan (ed.) SuccessfulFocus Groups: Advancing the State of the Art. Newbury Park, CA: Sage.

Goldman, A.E., and McDonald, S.S. (1987) The Group Depth Interview: Principles and Practice. EnglewoodCliffs, NJ: Prentice Hall.

Huitema, D. and M. van de Kerkhof (2006) Public participation in water management. The outcomes of anexperiment with two participatory methods under the Water Framework Directive in the Netherlands:Analysis and Prospects. In: Grover, V. (ed.). Water: Global common and global problems. Science Publishers.pp. 269296.

Jaeger, C.C., Schuele, R., Kasemir, B. (1998) Focus Groups in Integrated Assesment: A MicroCosmos forReflexive Modernization. In Darier, E., C.C. Jaeger, B. Kasemir, R. Schuele, S.Shackley, B. Wynne (1998)Contributions to Participatory Integrated Assessment. ULYSSES working paper 981. Darmstadt: DarmstadtUniversity of Technology.

Jaeger, C.C., B. Wynne, Gerger, ., Funtowicz, S., Giner, S., Giaoutzi, M. (1999) ULYSSES UrbanLifestyles, Sustainability and Integrated Environmental Assessment. Final Report prepared for DG XII of theEuropean Commission under the Environment and Climate Programme. Darmstadt: Darmstadt University ofTechnology.

Kangur, K. (ed.), Focus groups and citizens juries. River Dialogue Experiences in enhancing publicparticipation in water management. Peipse CTC. Tartu, Estonia.

Kasemir, B.; Gardner, M., Jger, J., Jaeger, C.C. (eds.) (2003) Public Participation in Sustainability Science.A Handbook. Cambridge: Cambridge University Press.

Kasemir, B. and C.C. Jaeger (1999a) 'Introducing ULYSSES: IAFocus Group procedures for publicparticipation'. In Jaeger, et al (1999) ULYSSES Urban Lifestyles, Sustainability and Integrated

Environmental Assessment Final Report, prepared for DG XII of the European Commission under theEnvironment and Climate Programme. Darmstadt: Darmstadt University of Technology.

Kasemir, B; Dahinden, U., Gerger Swartling, ., Schuele, R., Tabara, D., Jger, C. (1999b) 'Citizens Visionsof the Future: Collage processes in the first part of IAFocus Groups'. In ULYSSES Final Summary Report.Prepared for DG XII of the European Commission under the Environment and Climate Programme.Darmstadt University of Technology, Darmstadt, pp. 3959.

Krueger, R.A. (1994) Focus Groups: A Practical Guide for Applied Research. Thousand Oaks: Sage.

Krueger, R.A. (1998) Moderating Focus Groups, Focus Group Kit Vol 4. Thousand Oaks: Sage.

Krueger, R. A. & King, J. A. (1998) Involving Community Members in Focus Groups. Thousand Oaks, CA:Sage.

Merton, R.K., Fiske, M. and Kendall, P.L (1956) The Focused Interview: a Manual of Problems andProcedures. Glencoe, IL: Free Press.

Morgan, D. (1988) Focus Groups as Qualitative Research. Newbury Park, CA: Sage Publications.

Morgan, D. (ed.) (1993) Successful Focus Groups, Advancing the State of the Art. Thousand Oaks: SagePublications.

Morgan, D. (1998) Planning Focus Groups. The Focus Group Kit Vol 2. Thousand Oaks: Sage Publications.

Focus group (FG)

13/14

8/7/2019 PT6_tcm53-161511

16/16

Powell R.A. and Single H.M. (1996) 'Focus Groups'. In International Journal of Quality in Health Care, Vol.8, No. 5: 499504.

Querol, C., Gerger, ., Kasemir, B., Tabara, D. (1999) 'Citizens Conclusions on Climate Change andMitigation: Citizens report processes in the final part of IAFocus Groups'. In ULYSSES Final SummaryReport, prepared for DG XII of the European Commission under the Environment and Climate Programme.

Darmstadt: Darmstadt University of Technology, pp. 109131.

Rotmans, J and Van Asselt, M.B.A. (2002) Integrated Assessment: Current Practices and Challenges for theFuture. In Abaza and Barannzini (eds) Implementing Sustainable Development: Integrated Assessment andParticipatory DecisionMaking Processes. United Nations Environment Programme, Cheltenham: EdwardElgar Publishing Ltd: 78116.

Van Asselt, M.B.A. and Rotmans, J. (2003) 'From Projects to Program in Integrated Assessment Research'. InKasemir, B., Gardner, M., Jger, J., Jaeger, C.C. (eds.) (2003) Public Participation in Sustainability Science.A Handbook. Cambridge: Cambridge University Press.

Webler, T. (1995) Right Discourse in Citizen Participation: an Evaluative Yardstick. In Renn, O., Webler, T.and Wiedemann, P. Fairness and Competence in Citizen Participation. Evaluating Models for EnvironmentalDiscourse. Dordrecht: Kluwer Academic Publishers. pp. 3586.

Wilkinson, S. (1998). Focus group methodology: a review. In International Journal of Social ResearchMethodology. Vol. 1 (3). pp. 181203.

Yearley, S., Cinderby, S., Forrester, J. Bailey, B., and Rosen, P. (2003) 'Participatory Modelling and the LocalGovernance of the Politics of UK Air Pollution: a ThreeCity Case Study'. In Environmental Values, Vol 12,No. 2, 2003.

Focus group (FG)

14/14