CO-DESIGN AND DELIBERATIVE ENGAGEMENTWhat Works?Nicole Moore

Report No.3

02

DEM

OC

RAC

Y 20

25

CONTENTS

04 ABOUTDEMOCRACY2025–BRIDGINGTHE TRUSTDIVIDE

05 EXECUTIVESUMMARY

06 PREAMBLE

09 METHODOLOGY

11 STAGEONE:CASESTUDYREVIEWS

14 STAGETWO:THEORETICALSTUDYREVIEWS

17 STAGETHREE:VALIDATION

22 CONCEPTUALMODELFORASSESSINGTHE QUALITYOFENGAGEMENTWITHCITIZENS

27 CONCLUSION

28 REFERENCES

32 ABOUTTHEAUTHOR

03

DEM

OC

RAC

Y 20

25FIGURES, GRAPHS AND BOXES

06 Figure1:Definingspacesbetweendeliberativeengagementsandco-design

26 Figure2:Conceptualmodelforassessingthequalityofengagementswithcitizens

GRAPHS

07 Graph1: Co-designanddeliberativedemocracyliteraturefrom1980to2018

17 Graph2: Proportionofcasescitingrepresentationrelatedsub-variablesbyco-designordeliberative engagement

18 Graph3: Proportionofcasescitingnon-representationrelatedsub-variablesbyco-designordeliberative engagement

BOXES1 09 Box1: SystematicReviewFramework

16 Box2: Variablesandsub-variablesinfluencingpositiveoutcomesinco-designanddeliberative engagement

23 Box3: VariableandOutcomesMeasurementScale

04

DEM

OC

RAC

Y 20

25ABOUT DEMOCRACY 2025 – BRIDGING THE TRUST DIVIDE

AcrossAustraliatrustinourdemocracyisonthe

decline.Trustisthegluethatfacilitatescollective

actionformutualbenefit.Withouttrustwedon’t

havetheabilitytoaddresscomplex,long-term

challenges.Trustisalsocloselytiedtodemocratic

satisfaction.MoAD’s(MuseumofAustralian

Democracy)recentresearch,Trustand

DemocracyinAustralia,showsthatin2018

satisfactionindemocracyhasmorethanhalvedin

adecadeandtrustinkeyinstitutionsandsocial

leadersiseroding.By2025ifnothingisdoneand

currenttrendscontinue,fewerthan10percent

ofAustralianswilltrusttheirpoliticiansand

politicalinstitutions–resultinginineffective

andillegitimategovernment,anddecliningsocial

andeconomicwellbeing.

Thisproblemmustbeaddressedasamatterof

urgency.MoADistakingaction.Wearebringing

togethereverysectionofthecommunityand

ignitinganationalconversationonstrengthening

Australiandemocraticpractice.MoADandour

foundationpartner,theInstituteforGovernance

andPolicyAnalysisattheUniversityofCanberra

(UC-IGPA),haveembarkedonaboldnewinitiative,

Democracy2025,tobridgethetrustdivideand

re-engageAustralianswiththeirdemocracy.

MoADholdsauniqueposition,onthefrontlineof

democracy,civicagencyandchange,amuseum

notjustofobjectsbutofideas.Weempower

Australiansthroughexhibitions,schools’learning

programsandeventsthatbothstimulateand

inspire.Trustedbythepublic,government,public

serviceandbusinessalike,weadvancenational

conversationsaboutdemocracy,past,present

andfuture.

Democracy2025isdrivingaprocessofnational

reflectionandrenewalonhowwecanrebuildtrust

andstrengthendemocraticpracticeinAustralia.

Webelievethatthisambitiousgoaliscriticaltothe

healthofthenation.Nothinglesswilldo.

Daryl Karp Professor Mark Evans

Director, DirectorofDemocracy2025

MoAD UC-IGPA

05

DEM

OC

RAC

Y 20

25EXECUTIVE SUMMARY

ThisthirdDemocracy2025reportdocuments

findingsfromasystematicreviewintowhatworks

inpublicparticipation.

Uniquely,thereportintegratesexpertisefromtwo

distinctapproachestocitizenengagement,

co-designanddeliberativeengagement.Each

approachoffersdifferentyetcomplementary

insightsintothevariablesthatleadtoeffective

citizenengagement,providingusefulevidence

thatcaninformpublicsectorcapabilityinthisarea.

Thisreportcontributestothegrowingbodyof

evidenceoneffectivecitizenengagementby

developingaconceptualmodelforassessingthe

qualityandimpactsofrepresentationandnon-

representationrelatedvariables.

Theconceptualmodeldrawson33casestudies

and36theoreticalstudiestoidentifysixkey

variablesthatinteracttoinfluenceoutcomesin

co-designanddeliberativeengagement.

Theseinclude:

1. inclusiverepresentation

2. autonomyandequalityofallparticipants

3. pluralityofviewpointsandengagement

methods

4. qualityofprocessdesignandfacilitation

5. transmissionofcitizengenerated

recommendations,and

6. citizenparticipationasademocraticvalue.

Inaddition,theconceptualmodelidentifiesthree

outcomerelatedmeasuresforassessingthe

impactofeachofthesevariablesonthelegitimacy

ofpublicdecisions.

Theseinclude:

1. Participantsagreeingonthesolutionsor

recommendations,

2. Participantstrustinginthelegitimacyofthe

processtoinfluencedecisionmaking,and

3. Consequentialitydefinedasdecisionmakers

acceptingcitizengeneratedrecommendations.

Collectively,thevariablesandoutcomemeasures

formanevidencebasedconceptualmodelfor

assessingthequalityandimpactofcitizen

engagementprocesses,supportingpublicsector

capability,politicalaccountability,andultimately

thelegitimacyofpublicsectordecisions.

06

DEM

OC

RAC

Y 20

25PREAMBLE

Co-designanddeliberativeengagementsaretwo

citizenparticipationmethodsthatarerapidly

changingthewaygovernmentsaroundtheworld

aregrapplingwithcomplexpublicpolicyproblems.

Theseapproachesemergefromdistinctly

differenttraditions,theformermoredominantin

managementandpublic-sectorinnovation1and

thelatterbelongingtotheareaofpolitical

philosophy2.Furthermore,thepractice-based

focusofdeliberativeengagementhasdeveloped

onlyrecentlyfollowingasubstantialfocuson

theoreticalaspectsinthebroaderfieldof

deliberativedemocracy3.Thisisincontrasttothe

focusonapplicationandpracticetoolsthatlieat

thecoreofco-design.4

Giventhesedifferences,itisnotsurprisingthat

therehasbeennosignificantacademicattemptto

integratethesemethodsintoacohesive

frameworkforunderstandingboththetheoryand

practiceofcitizenengagement,makingthis

researchafirstinthisarea.

1 See:Bason,2010.2 See:Florida,2018.3 See:ElstubandMcLaverty,2014.4 See:EvansandTerrey,2016.

Theunderlyinghypothesisisthatleveragingthe

expertiseofbothapproachesmayleadtonew

insightsintothevariablesthatleadtoeffective

citizenengagement,providingusefulevidence

thatcaninformpublicsectorcapabilityinthisarea.

Bothapproacheshaverapidlyexpandedinthe

pastfourdecadesfromemergingconceptsto

entirefieldsofstudyandapplication.Toillustrate

thisgrowthinattention,Graph1(page7)provides

resultsofanadvancedGoogleScholarsearchof

literatureineachdisciplinebetween1980to2018

(accessedon11August2018).

Theseresultspaleincomparisontotheextensive

culminationofmaterialswrittenoneachsubjectin

variousotherformsofinvestigation,including

greyliterature,websites,andmedia.

GOVERNMENT PUBLIC SERVICE

POLITICS POLICY SERVICE/SYSTEMS

DELIBERATIVE ENGAGEMENTS CO-DESIGN

Figure1:Definingspacesbetweendeliberativeengagementsandco-design

07

DEM

OC

RAC

Y 20

25

What is a Deliberative Engagement?

Deliberativedemocracyasatheoreticaltradition

isnowinformingdemocraticengagements,most

commonly(thoughnotexclusively)intheformof

mini-publics.Democraticengagements,following

theprinciplesofdeliberativedemocracy,are

designedtoenableapluralityofvoices,toshinea

lightonthetensionsandconflictsbetween

differentactors,andtoenableasharedspacefor

forgingunderstandingoncollectiveproblems5.

What is Co-design?

Co-designprocessesinvolvecitizensinthe

iterativedevelopmentofinsights,prototyping,

evaluationandscalingofnewsolutions6.Design

thinkingenablesrepositioningofparadoxical

perspectivesintonewproblem-solvingframes7

withtheunderlyingbeliefthatengagementwith

citizensinthedevelopmentanddeliveryof

5 See:Baiocchi and Ganuza, 2017.6 See:EvansandTerrey,2016.7 See:Dorst,2011.

productsorserviceswillleadtobetter(andshared

responsibilityfor)outcomes8.

Legitimacy and Representation

Bothapproachesrecognisethatallcitizenshave

legitimatevoicesinpublicpolicyprocesses9.Tobe

legitimate;however,participationshouldenable

equitableopportunitiestocontributetodecision

making,regardlessofpersonalcircumstancesor

contexts.Inlargeandcomplexsocieties,itisnot

possibleforallaffectedpeopletobeinvolvedin

decisionmaking10,necessitatingsomeformof

representation.

8 See:ParkerandHeapy,2006.9 See:Holmes,2011;andErcanet.al,2018.10 See:Cohen,1997;Michelman,1997;andFlorida,2013.

0

5000

10,000

15,000

20,000

25,000

Deliberative Democracy Codesign

2010-182000-091990-991980-89

Graph1:Co-designanddeliberativedemocracyliteraturefrom1980to2018

08

DEM

OC

RAC

Y 20

25Powerfulgroupstendtodictatehowlesspowerful

groupsarerepresentedinpolicydebatesandhow

asocialgroupisrepresentedwilldeterminehow

legitimatetheirinterestsandperspectivesare

perceivedtobe11.Marginalisedgroupsand

individualcitizenshavealowercapacityfordriving

policynarratives,compellingarguments,and

voicingconcernscomparedtopowerful

stakeholdergroups,businessesandpoliticians12.

Sincerepresentationhasasignificantimpacton

howequallytheneedsofaffectedpeopleare

considered,itisimportanttounderstandhow

representationimpactsoutcomesincitizen

engagementprocesses.

11 See:HabibisandWalter,2015.12 See:Lowndes,2016.

Forthepurposeofthisreport,representation

referstothecollectionofvariablesthatdescribe

participantsincitizenengagementprocesses,

suchashowinclusivelyparticipantsrepresent

affectedpeople,howequallyparticipants

contributetodiscussionsanddecisions,andthe

diversityofviewpointstheyhold.

09

DEM

OC

RAC

Y 20

25METHODOLOGY

TheSystematicReviewFramework(hereafterthe

framework)presentedinBox1,wasdevelopedto

identifykeyquestionsanddefinitionsforthe

systematicreview,alongwithcriteriaforthe

selectionofrelevantliteratureandtheprocessto

beundertakentoidentifyandassessliterature.

Box 1: Systematic Review Framework

KEY QUESTIONS1. Whatvariablesinfluenceoutcomesinco-designanddeliberativeengagement?

2. Istherearelationshipbetweenthosevariablesandrepresentationinco-designanddeliberativeengagement?

Definitions for the purpose of this review (P=population | I=intervention | C=control group intervention | O=outcome)

P= Representatives Individuals acting on behalf of others in the context of public participation

I= Co-Design Citizen involvement in public participation processes where design thinking is used to drive innovation and creativity

C= Deliberative Engagements

Citizen involvement in public participation processes where deliberative reasoning is used to drive shared decision making

O= Legitimacy The genuine and effective involvement of citizens in decision making processes

Criteria for Literature Selection

1. Literaturemustdescribevariablesrelatedtorepresentationinco-designordeliberativeengagementtoaddressthekeyquestionsforthisreview

2. Literaturemayincludeboththeoreticalandcasestudiestocoverempiricalandnormativeperspectives

3. Literatureshouldbepeerreviewedwithsuitablecitationstoensurereliabilityoffindings

4. LiteraturemustbewrittenintheEnglishlanguagewithfulltextavailabletosupportindepthreview

Search Strategy

1. GoogleScholar,AnalysisandPolicyObservatory,WileyOnlineLibrary,JSTOR,WebofScience,SCOPUS,andAustralianPublicAffairsFullTextwillbereviewedtoidentifysuitableliterature

2. Abstractsreviewedinitiallytoconfirmsuitabilityagainsttheselectioncriteria

3. Selectliteraturereviewedindetailtoidentifyconceptualthemes

10

DEM

OC

RAC

Y 20

25Data analysis

Thedataanalysisinvolvedathree-stageprocess

toidentifyvariablesthatinfluenceoutcomesin

co-designanddeliberativeengagements.Both

representationandnon-representationrelated

variableswereassessedtodetermineif

representationmatterswhenengagingcitizens

Stage one: analysis of case studies

Areviewof13casestudieswasundertakento

identifyfrequentlyoccurringvariables.Noteswere

takenonvariablescitedineachcasetoenable

emergenceofthemeswhichwerethencodedinto

sub-variablesforcomparativeanalysis.

Inadditiontodescriptivevariablessuchas

location,duration,andformofrepresentation,any

benefitsornegativeimpactscitedinthecase

studieswerealsorecordedtoassessifparticular

variablesinfluencedoutcomes.

Stage two: analysis of theoretical studies

Buildingonthefindingsofstageone,areviewof

36theoreticalstudieswasundertakentoassessif

thevariablesidentifiedasinfluencingoutcomesin

stageone,werealsocommonlycitedasimportant

inco-designanddeliberativeengagementtheory.

Variablescitedbythemajorityofstudies

(50percentormore)wereconsideredofhigher

relevancethanvariablescitedbyfewerstudies.

Inaddition,stagetwoalsoidentifiedsub-variables

thatprovideadeeperunderstandingofhow

variablesinfluenceoutcomes.

Stage three: validation of review findings

Avalidationexercisewasundertakento

confirmtherelevanceandmeasurabilityofthe

sub-variablesidentifiedthroughstagesone

andtwo.

Thevalidationprocessfirstlyassessedthe

frequencyofeachsub-variablebeingreferredto,

eitherdirectlyorindirectly,in20additionalcase

studies(10co-designand10deliberative

engagementcases)toconfirmtheirrelevance

forpracticalapplication.

Secondly,thevalidationassessedtheapparent

achievementofeachsub-variableinorderto

confirmtheirmeasurability.

11

DEM

OC

RAC

Y 20

25STAGE ONE: CASE STUDY REVIEWS

Stageoneconsideredarangeofvariablesthat

mayhaveimpactedoncitizenengagement,such

asthenumberofparticipants,durationof

participation,selectionmethods,focusareas,and

locations,toidentifyifanyparticularvariables

appearedtoincreasethenumbersofpositive

benefitscitedbyeachcase.

Co-design cases

Arangeofbenefitswerecitedacrossthe

co-designcasesreviewed.Thehighestcited

benefitwastheconvergenceofdiverse

experiencesandexpertise13,followedbyincreased

understandingofissuesandsolutions14,and

increasedtrustandempathy15.

Ofthesixco-designcasesreviewed,fourcases

focusedonhumanservicespolicyissues16with

twocasesfocusingonissuesrelatedto

transport17.Whenfactoringinthenumberofcited

benefits,thehumanservicesfocusedcases

recordedahigherproportionofbenefitsat

75percentcitingthreetofourbenefits,compared

to50percentoftransportfocusedcasesciting

threetofourbenefits.

13 See:Lee,2007;EnserinkandMonnikof,2003;Brandtet.al,2010;Penuelet.al,2007;andBowanet.al,2013.

14 See:EnserinkandMonnikof,2003;Zimmermanet.al,2011;andBowanet.al,2013.

15 See:Lee,2007;Penuelet.al,2007;andBowanet.al,2013.16 See:Lee,2007;Penuelet.al,2007;Brandtet.al,2010;and,

Bowenet.al,2013.17 See:EnserinkandMonnikof,2003;Zimmermanet.al,2011.

Inaddition,75percentofcasesthattargeted

knownserviceusersasparticipants18and

75percentofcasesthatuseddesignworkshops

astheprimaryengagementmethod19bothcited

threetofourbenefits,higherthanopenselection

andprototypingdesignmethods.

Finally,bothcasesthatoccurredover12months

ormorecitedthreetofourbenefitswhichwasnot

achievedincaseswithashorterduration20.

Table1providesanoverviewofthenumberof

benefitsreportedbycaseswheretheabove

variableswerepresent.Thesmallsamplesizesfor

thesevariablesandinconsistencyinreportingmay

howeverimpactonthereliabilityoftheseresults

whichshouldbeinterpretedwithcaution.

18 See:Lee,2007;Penuelet.al,2007;Brandtet.al,2010;andBowenet.al,2013.

19 See:EnserinkandMonnikof,2003;Lee,2007;Brandtet.al,2010;andBowenet.al,2013.

20 See:Penuelet.al,2007;andBrandtet.al,2010.

12

DEM

OC

RAC

Y 20

25Table1: Numberofbenefitscitedinco-designcaseswheresub-variablewaspresent

1-2 benefits 3-4 benefits Total Cases

PotentiallyInfluentialVariables Number Percent Number Percent Number Percent

Focus

HumanServices 1 25% 3 75% 4 67%

Transport 1 50% 1 50% 2 33%

Selection Approach

TargetedSelection 1 25% 3 75% 4 67%

OpenSelection 1 100% 0 0% 1 17%

MixedSelection 0 0% 1 100% 1 17%

Co-design Approach

DesignWorkshops 1 25% 3 75% 4 67%

Prototyping 1 100% 0 0% 1 17%

Both 0 0% 1 100% 1 17%

Length of Participation

SeveralWorkshops 1 33% 2 67% 3 50%

Lessthan1Month 1 100% 0 0% 1 17%

12Months 0 0% 1 100% 1 17%

Over12Months 0 0% 1 100% 1 17%

Note: Benefits per centages refer to the proportion of cases where sub-variable is present (differs by variable). Total case per centages refer to the proportion of all cases (6).

Deliberative engagement cases

Aswiththeco-designcases,therewerearangeof

benefitscitedinthedeliberativeengagement

cases.Increasedknowledge,understandingand

empathy21wasthehighestcitedbenefit,followed

byenablingdiverserepresentation22.Participants

feelingrespectedandlistenedto23,increased

acceptanceintrustandlegitimacyofoutcomes24,

andenablingconvergenceandcompromise25were

alsocommonlycited.

21 See:McWhirteret.al,2014;Felicettiet.al,2015casestudiesaandb;Felicettiet.al,2012;CuratoandOng,2015;andNiemeyeret.al,2013.

22 See:McWhirteret.al,2014;andFelicettiet.al,2015casestudiesaandb.

23 See:McWhirteret.al,2014;Ercanet.al,2018casestudyb;andFelicettiet.al,2015casestudya.

24 See:McWhirteret.al,201;4andErcanet.al,2018casestudiesaandb.

25 See:McWhirteret.al,2014;Felicettiet.al,2015casestudya;andNiemeyeret.al,2013.

Whenconsideringbenefitscitedbyeachcase,

therewasnosignificantlyinfluentialfactors

presentinthedeliberativeengagementcases.

However,whileexpertswereinvolvedinsixofthe

sevencases,onecaseincludednoexpertsandstill

achievedfiveoftheninecitedbenefits26.

Thiscasewasopentodiverseviewpointsfromall

peoplewholivedinaparticularregionwhowere

invitedtodeliberateonissuesdirectlyconcerning

themratherthanapredeterminedtopicoffocus.

Itwasnotclearifeitherdiverseviewpointsor

enablingthefocustorelatespecificallyto

participant’slivedexperienceshadaninfluenceon

theseoutcomes.

26 See:Ercanet.al,2018.

13

DEM

OC

RAC

Y 20

25Furthermore,whilethreeofthefourAustralian

casescitedfivetoeightbenefitswhichwasnot

achievedinanyoftheotherlocations,thesmall

numberofcaseswouldrequirefurther

investigationintothepoliticaland/orsocial

contextsthatcouldhaveimpactedthisresult.

Table2providesanoverviewofpotentially

influentialvariablesbythenumberofbenefits

reportedinthecases.

Table2. Numberofbenefitscitedindeliberativeengagementcaseswheresub-variablewaspresent

1-4 benefits 5-8 benefits Total Cases

PotentiallyInfluentialVariables Number Percent Number Percent Number Percent

Representation

Descriptive 2 50% 2 50% 4 57%

Surrogate 1 100% 0 0% 1 14%

Geographic 1 50% 1 50% 2 29%

Location

Australia 1 25% 3 75% 4 57%

Italy 1 100% 0 0% 1 14%

Philippines 1 100% 0 0% 1 14%

UnitedStatesofAmerica 1 100% 0 0% 1 14%

Note: Benefits percentages refer to the proportion of cases where sub-variable is present (differs by variable). Total cases percentages refer to the proportion of all cases (7).

Key finding one: representation and non-representation related variables matter

Thecasestudyreviewidentifiedarangeof

variablesthatpotentiallyinfluenceoutcomesin

co-designanddeliberativeengagementcases,

severalofwhichrelatetoissuesofrepresentation.

Inparticular;theapproachtotheselectionof

representatives,ensuringdiverseviewpoints,and

involvementofthosewithlivedexperienceswere

identifiedaspotentiallyinfluencingoutcomes.

Therewerehowever,arangeofnon-representation

relatedvariablesthatalsoappearedtoinfluence

outcomes,including;thefocusoftheprocess,the

qualityandtypeofmethodsused,thedurationof

participation,andthepolitical/socialcontext.

Duetoinconsistenciesinreportingacrosscases,

theresultsofthisstageofanalysiswere

inconclusiveontheirown,providingabasisfor

furtherinvestigatingthenormativevalueplaced

oneachofthesevariablesinthetheoretical

literature.

14

DEM

OC

RAC

Y 20

25STAGE TWO: THEORETICAL STUDY REVIEWS

Stagetwosoughttodetermineifanyofthe

variablesidentifiedaspotentiallyinfluencing

outcomesinstageone,werealsocommonlycited

asimportantinthetheoreticalliterature.

Co-design theory

Inthe18co-designstudiesreviewed,fourofthe

variablesidentifiedinstageone,werehighlighted

inmorethan50percentofstudies.Fromthose

variables,fivesub-variableswerealsocitedin

50percentormoreofstudiescitingeachvariable.

Themostcommonlycitedvariableoverallwas

diverseviewpointswhichwascitedin78percent

ofstudies.Ofthosestudies,57percentplaced

particularvalueontheroleofbothserviceusers

andprofessionalsasequalparticipants27and

50percentstressedtheneedtoaddresspower

imbalancestoenableeffectiveco-design28.

Thenextmostcommonlycitedvariablewasthe

co-designmethodwhichwascitedin67percent

ofstudieshowever,itwasthequalityofprocess

designandfacilitationwhichwasmostcommonly

identifiedwithinthesestudiesratherthanspecific

co-designmethods29.

27 See:Binderet.al,2008;SzebekoandTan,2010;EvansandTerrey,2016;Trischleret.al.2018.

28 See:Binderet.al,2008;BuurandLarsen,2010;SzebekoandTan,2010;Holmes,2011;GreenbaumandLoi,2012;Harderet,al,2013;andTrischleret.al,2018.

29 See:BuurandLarsen,2010;SzebekoandTan,2010;Holmes,2011;Harderet.al,2013;Andersenet.al,2015;EvansandTerrey,2016;Trischleret.al,2018.

Inaddition,theco-designmethodwasfollowed

closelybypolitical/socialcontextwhichwascited

in56percentofstudies,withasignificantvalue

placedoncitizenparticipationbeingacceptedasa

democraticvalue30,highlightingtheimportantrole

thatco-designcancontributetothelegitimacyof

publicsectordecisions.

Finally,theimportanceofthefocusofthecase

wasreferredtoin56percentofstudieswhich

significantlyfavouredservicesandproductdesign

morebroadly31ratherthanhumanservices

contexts.

Table3outlinesthevariablescitedbythemajority

ofstudies,alongwiththeirmostcommonlycited

sub-variables.

30 See:Binderet.al,2008;BuurandLarsen,2010;SzebekoandTan,2010;Holmes,2011;GreenbaumandLoi,2012;EvansandTerrey,2016;andHuybrechtset.al,2017.

31 See:Leadbeater,2004;ParkerandHeapy,2006;Binderet.al,2008;SzebekoandTan,2010;GreenbaumandLoi,2012;Harderet.al,2013;Huybrechtset.al,2017;Bason,2018;andTrischleret.al,2018.

15

DEM

OC

RAC

Y 20

25Table3: Variablesandsub-variablesidentifiedmostfrequentlyinco-designtheoreticalstudies

Variable as percentage of total studies Sub-variables as percentage of studies citing related variable

1. DiverseViewpoints(78%) 1. Interdisciplinaryusersandprofessionalsasequalpartnersindesign(57%)

2. Equalisingpowerimbalances(50%)

2. Co-designMethods(67%) 3 Qualityprocessdesignandfacilitation(58%)

3. Political/Social Context(56%)

4. Citizenparticipationasanaccepteddemocraticvalue(70%)

4. Focus(56%) 5. Designofservicesandproducts(90%)

Note: Variable percentages refer to the proportion of all studies citing each variable. Sub-variable percentages refer to the proportion of those studies that also cited the sub-variable.

Deliberative engagement theory

Inthe18deliberativeengagementstudies

reviewed,threeofthevariablesidentifiedinstage

one,werehighlightedinmorethan50percentof

studies.However,onlyonesub-variablewascited

inmorethan50percentofstudiesreferringto

thosevariables.Subsequentlythehighestcited

sub-variableswereidentifiedfromeachofthe

majoritycitedvariables.

Thedeliberationmethodwasthehighestcited

variable,referredtoin83percentofstudies.

Ofthosestudies,thetransmissionofcitizen

generatedrecommendationstoformaldecision-

makingbodieswasthehighestcitedsub-variable,

referredtoin40percentofthosestudies32.

32 See:Bohman,1998;UhrandUhr,1998;Parkinson,2003;DryzekandNiemeyer,2008;Hendricks,2015;andDryzek,2016.

Diverseviewpointswasthesecondmost

commonlycitedvariablewhichwasreferredto

in56percentofstudies.Ofthosestudies,

80percentreferredtopluralityofbeliefs,

convictions,idealsandviewpoints33.

Finally,issuesrelatedtothepolitical/socialcontext

werecitedin50percentofstudies.Ofthose

studies,thehighestcitedsub-variableswerethe

recognitionthatlargeandcomplexsocieties

requireinclusiverepresentationduetothe

impossibilityofinvolvingallaffectedpeople34,

andtheimportanceofensuringparticipantsare

autonomousandcapableofformingaview

withoutexternalconstraints35.

Table4outlinesthevariablescitedbythemajority

ofstudies,alongwiththeirmostcommonlycited

sub-variables.

33 See:Cohen,1997;Bohman,1998;UhrandUhr,1998;SmithandWales,2000;DryzekandNiemeyer,2008;Dryzek,2016;Elstubet,al,2016;andBaiocchiandGanuza,2017.

34 See:Cohen,1997;Michelman,1997;andFlorida,2013.35 See:Cohen,1997;SmithandWales,2000;andFlorida,

2013.

16

DEM

OC

RAC

Y 20

25Table4: Variablesandsub-variablesidentifiedmostfrequentlyindeliberativeengagement theoreticalstudies

Variable as percentage of total studies Sub-variable as percentage of studies citing related variable

1. DeliberationMethods(83%) 1. Transmissionofcitizengeneratedrecommendationstoformal

decision-makingbodies(40%)

2. Diverseviewpoints(56%) 2. Pluralityofbeliefs,convictions,idealsandviewpoints(80%)

3. Political/Socialcontext

(50%)

3. Representationnecessarytocounterimpossibilityofinvolvingall

affectedpeopleinlargeandcomplexsocieties(33%)

4. Autonomyofparticipantstocounteradaptivepreferences(33%)

Note: Variable percentages refer to the proportion of all studies citing each variable. Sub-variable percentages refer to the proportion of those studies that also cited the sub-variable.

Key finding two: quality methods, context and diversity matter

Stagetwoidentifiedthreesharedvariablesthat

thestudiessuggestinfluenceoutcomesinboth

co-designanddeliberativeengagements:

1. qualitymethods

2. political/socialcontext,and

3. diversityofviewpoints.

Inaddition,eachvariablehasanumberof

sub-variablesthatappeartoincreasethechances

ofpositiveoutcomes,someofwhichrelateto

representation,andsomethatdon’t(as

demonstratedinBox2).

Whilethefocusofthecasewashighlycitedin

co-designtheory,thiswasnotthecasewith

deliberativeengagement.Hencethisvariablehas

beenexcludedforthepurposesofidentifying

sharedvariablesofinterest.

Box 2: Variables and sub-variables influencing positive outcomes in co-design and deliberative engagement

RepresentationRelatedVariables Non-RepresentationRelatedVariables

Quality Methods Political/Social Context Diversity of Viewpoints

Qualityofprocessdesign andfacilitation

Inclusive representation

Involvementofaffectedpeopleandprofessionals

Transmissionofcitizenengagementoutcomestoformaldecision-making

authorities

Autonomyofrepresentatives Equalityofparticipants

Citizenparticipationasdemocraticvalue

Pluralityofbeliefs,convictions,idealsandviewpoints

17

DEM

OC

RAC

Y 20

25STAGE THREE: VALIDATION

Validation involved identifying direct and indirect

references to each of the identified sub-variables,

and measuring their achievement in 20 additional

case studies, once more using secondary sources.

Relevance

The validation process showed that with the

exception of autonomy of representatives,

all of the identified sub-variables were referenced

in the majority of co-design and deliberative

engagement cases, suggesting a high level

of relevance.

Autonomy of representatives was also identified

in 70 per cent of the deliberative engagements,

suggesting that this variable may warrant further

investigation in co-design as well (see Graph 2).

In addition to the representation related

sub-variables, all three non-representation related

sub-variables were identified in the majority of

co-design and deliberative engagements,

suggesting high levels of relevance (see Graph 3).

Graph 2: Proportion of cases citing representation related sub-variables by co-design or deliberative engagement

18

DEM

OC

RAC

Y 20

25

Graph3:Proportionofcasescitingnon-representationrelatedsub-variablesbyco-designor deliberativeengagement

0

20

40

60

80

100

Deliberative engagementsCo-design

Transmission of citizen generated recommendations to formal

decision-making bodies

Quality process design and facilitation

Citizen engagement as an accepted democratic value

Measuring sub-variables in co-design cases

Simpledescriptorsbasedonthetheoretical

reviewwereusedtoassesstheachievement

ofeachofthesub-variables.Inaddition,each

casewasassessedaseitherachieving

consequentialoutcomesornot,whichwas

definedastheacceptanceofcitizengenerated

recommendationsbyformaldecision-making

bodies.

Eightofthe10co-designcaseswereassessed

asachievingqualityprocessdesignand

facilitation36whichwasnotsurprisinggiventhat

studiesweredrawnfromorganisationswith

significantexpertiseindesigningandfacilitating

co-designprocesses.

36 See:Hampsonet.al,2013;DesignCouncil,2016;Woodroffe,2018;ThinkPlace,2017(a);Thinkplace,2017(b);AucklandDesignLab,2015;AucklandDesignLab,2017;andIDEO,2015.

Whatwasmoresurprisingwasthattwoofthe

eightstudiesthatachievedqualityprocessdesign

andfacilitationdidnotachieveconsequential

outcomesduringthetimeframereported37.

Theothertwonon-representationrelatedsub-

variableswerealsoachievedinthemajorityof

cases,includingsevencasesthatachievedthe

transmissionofparticipantrecommendations

directlytoformaldecision-makingbodies38,and

sixcasesachievingcitizenparticipationasan

accepteddemocraticvalue39.

Intermsofrepresentationrelatedsub-variables,

theinvolvementofaffectedpeopleand

professionalswasachievedinthemajorityof

37 See:AucklandDesignLab,2015;andAucklandDesignLab,2017.

38 See:Hampsonet.al,2013;DesignCouncil,2016;Woodroffe,2018;ThinkPlace,2017(a);Thinkplace,2017(b);Budds,2016;andIDEO,2015.

39 See:DesignCouncil,2016;Woodroffe,2018;ACTGovernment,2018;ThinkPlace,2017(a)lThinkplace,2017(b)landBudds,2016.

19

DEM

OC

RAC

Y 20

25cases40,withpluralityofviewpointsachievedin

fourcases41,andequalityofparticipantsachieved

inthreecases42.

Inclusiverepresentationhoweverwasonly

achievedintwocases43andnocasesreferredto

theachievementofautonomyamong

representatives,makingthissub-variablenot

applicabletotheanalysis.

Table5outlinesthenumberofcasesassessed

asachievingeachsub-variableandtheproportion

ofthosecasesthatachievedconsequential

outcomes.

40 See:Hampsonet.al,2013;Woodroffe,2018;ThinkPlace,2017(a);Thinkplace,2017(b);AucklandDesignLab,2017;andIDEO,2015.

41 See:Hampson,et.al2013;DesignCouncil,2016;Woodroffe,2018;andThinkPlace,2017(a).

42 See:Hampson,et.al,2013;DesignCouncil,2016;andAucklandDesignLab,2017.

43 See:Hampsonet.al,2013;andBudds,2016.

Table5:Achievementofsub-variablesandconsequentialityinco-designcases

Sub-variables Consequentiality

Number Per cent Number Per cent

Political/Social Context

Citizenparticipationasdemocraticvalue 6 75% 6 100%

Inclusiverepresentation 2 40% 2 100%

Autonomyofrepresentatives NA* NA NA NA

Diversity of Viewpoints

Involvementofaffectedpeopleandprofessionals 6 60% 5 84%

Equalityofparticipants 3 60% 2 67%

Pluralityofbeliefs,convictions,idealsandviewpoints 4 68% 4 100%

Quality Approach

Qualityofprocessdesignandfacilitation 8 89% 5 63%

Transmissionofcitizenengagementoutcomestoformaldecision-makingauthorities 7 88% 7 100%

Note: Sub-variable percentages refer to the proportion of total valid cases (those referring to the sub-variable) achieving the sub-variable.

*No co-design cases referred to autonomy of participants, making this sub-variable not applicable.

Consequentiality percentages refer to the proportion of cases achieving the sub-variable that also achieved consequentiality, defined as acceptance of recommendations by decision making bodies.

Measuring sub-variables in deliberative engagements

Thesamemeasurementprocesswasfollowedto

assessvariableachievementandconsequentiality

inthedeliberativeengagementcases.

Ofthe10casesreviewed,qualityprocessdesign

andfacilitation44,andthetransmissionofcitizen

engagementoutcomestoformaldecision-making

bodieswerebothachievedinfivecaseseach45.

Inaddition,citizenparticipationasanaccepted

democraticvaluewasachievedinfourcases.46

44 See:Um,2013;Hargrave,2018;Weymouth,2016;Parry,2016;andHosmer,2017.

45 See:Fung,2009andFletcher,2018;Faubion,2012;Weymouth,2016;Hosmer,2017;andParry,2016.

46 See:Fung,2009andFletcher,2018;Faubion,2012;Weymouth,2016;andParry,2016.

20

DEM

OC

RAC

Y 20

25Intermsofrepresentationrelatedsub-variables,

inclusiverepresentation47andpluralityof

viewpoints48wereachievedinfivecaseseach.

Theinvolvementofaffectedpeopleand

professionals49andequalityofallparticipantswere

achievedinthreecaseseach50.Inaddition,

autonomyofrepresentativeswasachievedin

twocases51.

47 See:Um,2013;Hargrave,2018;Faubion,2012;Weymouth,2016;andParry2016.

48 See:Um,2013;Hargrave,2018;Weymouth,2016;Parry,2016;andShen,2012andFletcher,2016.

49 See:Fung,2009andFletcher,2018;Hargrave,2018;andParry,2016.

50 See:Fung,2009andFletcher,2018;Hargrave,2018;andWeymouth,2016.

51 See:Um,2013;andShen,2012andFletcher,2016.

Table6outlinesthenumberofcasestudies

assessedasachievingeachsub-variableandthe

proportionofthosecasesthatachieved

consequentialoutcomes.

Table6:Achievementofsub-variablesandconsequentialityindeliberativeengagementcases

Sub-variables Consequentiality

Number Per cent Number Per cent

Political/Social Context

Citizenparticipationasdemocraticvalue 4 40% 2 50%

Inclusiverepresentation 5 50% 2 40%

Autonomyofrepresentatives 2 29% 1 50%

Diversity of Viewpoints

Involvementofaffectedpeopleandprofessionals 3 30% 0 0%

Equalityofparticipants 3 50% 1 33%

Pluralityofbeliefs,convictions,idealsandviewpoints 5 71% 2 40%

Approach

Qualityofprocessdesignandfacilitation 5 56% 2 40%

Transmissionofcitizenengagementoutcomesto

formaldecision-makingauthorities5 50% 2 40%

Note: Sub-variable percentages refer to the proportion of total valid cases (those referring to the sub-variable).

Consequentiality percentages refer to the proportion of cases achieving the sub-variable that also achieved consequentiality, defined as acceptance of recommendations by decision making authorities.

21

DEM

OC

RAC

Y 20

25Validation Outcomes

Thevalidationofsub-variablerelevanceand

measurabilityhighlightedanumberoffindingsof

criticalimportancetothedevelopmentofthe

conceptualmodelforassessingthequalityand

impactsofcitizenengagementprocesses.

Inparticular,thevalidationprocessshowedthat

whilethesub-variablescouldbemeasured,

degreesofachievementwouldallowformore

accurateassessment.Forexample,affected

peopleandprofessionalsmaybeinvolved

throughouttheparticipationprocess,atparticular

pointsintheprocess,ornotatall.

Havingameasurementscalecapableof

accountingfordegreesofachievementwould

enhancethereliabilityofaconceptualmodelfor

assessingqualityandimpacts.

Inaddition,whilesecondarysourceswereusedto

testthesub-variables,theuseofprimarydatato

applytheconceptualmodelwouldenable

consistencyofdatacollectionandincreasethe

accuracyoftheresults.

Key finding three: interactions matter

Furthertothevalidationoutcomesalready

discussed,allvariableswerepresentbothin

consequentialandnon-consequentialcases,

suggestingthatwhiletheymayormaynotbe

necessary,individualvariablesareunlikelytobe

sufficientinandofthemselves.

Thisfindingsupportsasettheoryapproachto

analysingboththeimpactsandinteractions

betweenrepresentationandnon-representation

relatedvariablesincitizenengagementprocesses.

22

DEM

OC

RAC

Y 20

25CONCEPTUAL MODEL FOR ASSESSING THE QUALITY OF ENGAGEMENT WITH CITIZENS

Theaimoftheconceptualmodelistoprovidea

toolforpublicsectororganisationstoevaluatethe

qualityofengagementwithcitizenstoimprove

boththeoryandpracticeinthisspace.The

conceptualmodelalsosupportsstrengthened

accountabilityofpublicsectororganisationsby

ensuringcitizenengagementadequately

representsthecommunitiesofpeoplemost

affectedbypublicsectordecisions.

Theconceptualmodelincludesthree

representationrelatedvariables,whichhavebeen

developedbymergingthefollowingsub-variables:

inclusiverepresentationandinvolvementof

affectedpeopleandprofessionals,andautonomy

ofrepresentativesandequalityofallparticipants,

inordertoreducecomplexityandenablemore

holisticvariabledescriptors;andbyreframing

pluralityofviewpoints,beliefs,idealsand

convictionstoalsorecognisethatapluralityof

engagementmethodsarerequiredtoensure

diverseviewpointscanparticipateincitizen

engagementprocesses.

Representationrelatedvariablesthereforeinclude:

1. Inclusiverepresentationofaffectedpeople

andprofessionals;

2. Pluralityofviewpointsandengagement

methods;and

3. Autonomyandequalityofparticipants.

Theconceptualmodelalsoincludesthree

non-representationrelatedvariables:

1. Citizenengagementasanaccepted

democraticvalue;

2. Qualityprocessdesignandfacilitation;and

3. Transmissionofcitizengenerated

recommendationstoformaldecision-making

bodies.

Inadditiontotherepresentationandnon-

representationrelatedvariables,theconceptual

modelincludesthreesuggestedoutcomerelated

measurestoassesstheimpactofvariables

achievedthroughthecitizenengagementprocess.

Outcomesmeasuresinclude:

1. Citizensagreeingonthesolutionor

recommendationsarisingthroughthecitizen

engagementprocess;

2. Citizenstrustinginthelegitimacyofthe

citizenengagementprocesstoinfluence

decision-making;and

3. Consequentialitydefinedasdecision-making

bodiesacceptingcitizengenerated

recommendations.

Theseoutcomesmeasuresspeaktotheoverall

legitimacyoftheprocessinengagingcitizensin

decisionmakingandcanbesubstitutedforcase

specificoutcomeswheredeemedsuitable.The

measurementscalesforeachofthevariablesand

outcomesdiscussedaboveareoutlinedatBox3.

23

DEM

OC

RAC

Y 20

25Box 3: Variable and Outcomes Measurement Scale

Variables Not achieved Partially Achieved Fully Achieved

Representation Related Variables

Inclusiverepresentation(affectedpeopleandprofessionals)

Nostrategiesinplacetoensurerepresentativesareinclusivedemonstratedbylackofrepresentationofaffectedpeopleandrelevantprofessionalsatanystageinthedesignanddeliberationprocess

Somestrategiesinplacetoensurerepresentativesareinclusivedemonstratedbyrepresentationofaffectedpeopleandrelevantprofessionalsatvariousstagesthroughoutthedesignanddeliberationprocess

Effectivestrategiesinplacetoensurerepresentativesareinclusivedemonstratedbyrepresentationofaffectedpeopleandrelevantprofessionalsthroughoutallstagesofthedesignanddeliberationprocess

Autonomyandequalityofallparticipants

Noevidenceofpowersharingandfreedeliberationamongparticipantsdemonstratedthroughlowlevelsofcontributionbyallparticipantsandlowpreferencetransformation

Someevidenceofpowersharingandfreedeliberationamongparticipantsdemonstratedthroughmediumlevelsofcontributionbyallparticipantsandmediumpreferencetransformation

Evidenceofpowersharingandfreedeliberationamongparticipantsdemonstratedthroughhighlevelsofcontributionbyallparticipantsandhighpreferencetransformation

Pluralityofviewpoints(andengagementmethods)

Nostrategiesinplacetoengagediverseviewpointsdemonstratedthroughlowlevelsofdiversityinoriginalpositions

Somestrategiesinplacetoengagediverseviewpointsdemonstratedthroughmediumlevelsofdiversityinoriginalpositions

Effectivestrategiesinplacetoengagediverseviewpointsdemonstratedthroughhighlevelsofdiversityinoriginalpositions

Non-Representation Related Variables

Qualityofprocessdesignandfacilitation

Noexpertfacilitationandlackofmethodstosupporteffectiveengagementdemonstratedthroughlowparticipantsatisfactionwiththeprocess

Somefacilitationexpertiseandmethodstosupporteffectiveengagementdemonstratedthroughmediumparticipantsatisfactionwiththeprocess

Expertfacilitationandmethodstosupporteffectiveengagementdemonstratedthroughhighparticipantsatisfactionwiththeprocess

Transmissionofcitizenengagementoutcomestoformaldecision-makingbodies

Notransferofcitizengeneratedinputsorrecommendationstorelevantpoliticalactorsorpubicserviceorganisations

Indirecttransferofcitizengeneratedinput(viastakeholderdevelopedrecommendations)torelevantpoliticalactorsorpublicserviceorganisations

Directtransferofcitizengeneratedrecommendationstorelevantpoliticalactorsorpublicserviceorganisations

24

DEM

OC

RAC

Y 20

25Variables Not achieved Partially Achieved Fully Achieved

Citizenparticipationasanaccepteddemocraticvalue

Nopolitical/publicsectorsupportforcitizenparticipationintheengagementprocessdemonstratedbylackofauthorisationbypoliticalactorsorpublicserviceorganisations

Somepolitical/publicsectorsupportforcitizenparticipationintheengagementprocessdemonstratedthroughupfrontauthorisationbypoliticalactorsorpublicserviceorganisations

Fullpolitical/publicsectorsupportforcitizenparticipationintheengagementprocessdemonstratedthroughupfrontauthorisationbypoliticalactorsorpublicserviceorganisationsandin-principlecommitmenttotheacceptanceofcitizengeneratedrecommendations

Outcomes Measures

AgreedSolutionorRecommendations

Noevidenceofagreementdemonstratedbylowlevelsofparticipantsupportforthefinalsolutionorrecommendations.

Someevidenceofagreementdemonstratedbymediumlevelsofparticipantsupportforthefinalsolutionorrecommendations.

Evidenceofagreementdemonstratedbyhighlevelsofparticipantsupportforthefinalsolutionorrecommendations.

Legitimacy Noevidenceoflegitimacydemonstratedthroughlowlevelsoftrustheldbyparticipantsinthepoweroftheengagementprocesstoinfluencethedecisionsmadebygovernment.

Someevidenceoflegitimacydemonstratedthroughmediumlevelsoftrustheldbyparticipantsinthepoweroftheengagementprocesstoinfluencethedecisionsmadebygovernment.

Evidenceoflegitimacydemonstratedthroughhighlevelsoftrustheldbyparticipantsinthepoweroftheengagementprocesstoinfluencethedecisionsmadebygovernment.

Consequentiality Noevidenceofconsequentialitydemonstratedbynoorlimitedacceptanceofparticipantrecommendationsbydecisionmakingauthorities.

Someevidenceofconsequentialitydemonstratedbypartialacceptanceofparticipantrecommendationsbydecisionmakingauthorities.

Evidenceofconsequentialitydemonstratedbyfullacceptanceofparticipantrecommendationsbydecisionmakingauthorities.

25

DEM

OC

RAC

Y 20

25Theconceptualmodelalsorecommendsarange

ofmethodologiesformeasuringtheidentified

variablesandoutcomes,including:

1. Pre-engagement surveysinlivecasesto

determineparticipantcharacteristicsandthe

diversityofviewpointsrepresentedineach

process(note:thesemeasurescanbe

collectedthroughpost-engagementsurveys

forcompletedcases);

2. Post-engagement surveystoassess

participantexperiences,perceptionsand

agreementontherecommendedsolutions;

3. Targeted interviewswithasmallnumberof

participantsorkeystakeholdersinorderto

unpacksurveyfindingsusingreflective

questioning;

4. Interviews with process organiserstoassess

levelsofinvolvementandequalcontributionby

participants,andtoassesstheauthorisation,

transferofrecommendations,andacceptance

ofrecommendationsbydecisionmaking-

bodies;and

5. Observationsinlivecasestoassesstheequal

contributionofparticipants,qualityofprocess

designandfacilitation,andapparent

agreementwithrecommendedsolutions.

Figure2providesavisualrepresentationofthe

variablesandtheirrecommendedassessment

methods.

26

DEMOCRACY 2025

Figure2:Conceptualmodelforassessingthequalityofengagementswithcitizens

VARIABLES MEASURES ASSESSMENT METHODSInclusiveRepresentation

(affectedpeopleandprofessionals)

QualityofProcessDesign andFacilitation

AutonomyandEquality ofallParticipants

TransmissionofCitizenGeneratedRecommendations

PluralityofViewpoints (andengagementmethods)

CitizenParticipationasa DemocraticValue

Repr

esen

tati

on R

elat

ed

Varia

bles

Non

-Rep

rese

ntat

ion

Rela

ted

Va

riabl

esN

on-R

epre

sent

atio

n Re

late

d

Varia

bles

InvolvementThroughout EngagementProcess

PreferenceTransformation andEqualContribution

DiversityofOriginalPositions

ParticipationSatisfaction

TransferofRecommendations

Authorisation

ParticipantSupportforFinal SolutionorRecommendations

ParticipantPerceptionsofTrust

AcceptanceofRecommendations

Post Engagement Survey (+ pre-engagement survey questions for completed cases) and Observations (live cases)

Pre-engagement survey (Live cases only- collect through post engagement survey

for completed cases)

PreferenceTransformation (comparedtooriginalposition)

ParticipantSatisfaction,includingequalcontribution (Process)

ParticipantPerceptionsofTrust (Legitimacy)

ParticipantSupportforRecommendedSolutions (Agreement)

ParticipantCharacteristics OriginalPositionsofParticipants

Targeted Participants Interviews

Organiser Interviews

ReflectiveQuestioningtoUnpackSurveyFindings

IdentificationofAffectedPeoplesandProfessionals (Characteristics)andtheirinvolvementthroughout

theengagementprocess

AuthorisationofProcess,transferofrecommendationsandacceptanceofrecommendationsbydecisionmakingBodies

AgreedSolutionsorRecommendations

Legitimacy

Consequentiality

27

DEM

OC

RAC

Y 20

25CONCLUSION

Co-designanddeliberativeengagementboth

haveimportantandcomplementaryrolestoplay

inmoderndemocracies,andinparticularinthe

waypublicserviceagenciesengagecitizensin

designinganddeliberatingonissuesofcommon

good.Inaddition,theybothprovideimportant

insightstoguidethesuccessfulengagementof

citizensinpublicpolicyandservicedesign.

Thissystematicreviewlookedspecificallyatthe

variablesthatinfluenceco-designanddeliberative

engagementpracticetodeterminewhatrole

representationplaysinenhancingthelegitimacy

ofpublicdecisions.

Thesystematicreviewidentifiedthreekeyfindings

thatcollectivelyinformedthedevelopmentofthe

conceptualmodelforassessingthequalityof

engagementwithcitizensoutlinedinthisreport.

1. RepresentationandNon-Representation

RelatedVariablesMatter:

Representationrelatedvariables,suchas

inclusivity,autonomy,equalityandplurality,

influenceoutcomesinbothco-designand

deliberativeengagements;however,sotoodo

non-representationrelatedvariables,suchas

processquality,politicalauthorisation,and

connectionsbetweencitizenengagements

andformaldecision-makingbodies.

2. QualityMethods,ContextandDiversityMatter:

Thepolitical/socialcontextanddiversityof

peopleaffectedbythetopicoffocus,will

influencetheselectionofcitizenengagement

methods.Qualityprocessdesignand

facilitationplaysamoresignificantrolein

accommodatingdiversecontextsand

participants,thananyparticulardesignor

deliberationmethod.

3. InteractionsMatter:

Noonevariableislikelytobesufficientinandof

itself.Measuringboththeachievementand

interactionsbetweenvariablesislikelyto

enhanceunderstandingofthedynamicnature

ofcitizenengagementinanever-changing

socialcontext.

Aspublicserviceorganisationsincreasinglystrive

toenhancepublictrustandthelegitimacyof

decisionsmade,effectivecitizenengagementthat

adequatelyrepresentsthosemostaffectedby

thosedecisionsiscrucial.Itisnotsufficientthat

publicsectororganisationsseektheviewsof

thosemostvocalintheircommunities.

Representationmustbeinclusive,equal,and

diverse.Representativesmustbeautonomous

andsupportedbyqualityprocessesthatallow

themtobeactivecontributors.Andboth

governmentsandpublicsectororganisations

mustvaluetheinputofcitizensasdemocratic

agents,committingtotakingseriouslytheirviews

andrecommendations.

Theconceptualmodeloutlinedinthisreport,

providesaframeworkforpublicsector

organisationstomeasurethequalityoftheirown

citizenengagementprocessesinordertoidentify

areasforimprovement.

Themodelalsoprovidesaframeworkforholding

politicalsystemstoaccountoverthelegitimacyof

publicdecisionsthroughthegenuineandeffective

involvementofcitizensindecisionmaking

processes.

28

DEM

OC

RAC

Y 20

25REFERENCES

ACTGOVERNMENT2018.Family Safety Hub: Co-creating pathways to sustainable safety [Online].Available:http://www/communityservices.act.gov.au/_data/assets/pdf_fiile/0006/1198824?Family-Safety-Hub-Design-Report.pdf[accessed21September2018].

ALFRED,J.2009.Engaging Public Sector Clients: From Service-Delivery to Co-Production,UnitedKingdom,PalgraveMacmillan.

ANDERSON,L.B.,DANHOLT,P.,HALSKOV,K.,BRODERSENHANSON,N.&LAURITSEN,P.2015.ParticipationasaMatterofConcerninParticipatoryDesign.Co-design,11(3-4),pp250-261.

AUCKLANDDESIGNLAB2015.Exploring Security of Tenure through Co-design[Online]. Available:https://staticsquare.com/static/55ac5ee5e4b08d4c25220f4b/t/5b7602f82b6a281de0bad415/1534460707769/Rental+Tenure+-+Talkbook+15.9.2015.pdf[accessed21September2018].

AUCKLANDDESIGNLAB2017.Early Years Challenge: Supporting Parents to Give Tamariki a Great Start in Life[Online].Available:https://static1.squarespace.com/static/55ac5ee5e4b08d4c25220f4b/t/5b64ef6103ce647540da30c1/1533341548279/Early+Years+Challenge+report+FINAL+%28web%29.pdf[accessed21September2018].

BAIOCCHI,G.&GANUZA,E.2017. Popular Democracy: The Paradox of Participation,UnitedStatesofAmerica,StanfordUniversityPress.

BASON,C.2010.Leading Public Sector Innovation: Co-creating for a Better Society,UnitedKingdom,PolicyPress.

BINDER,T.,BRANDT,E.&GREGORY,J.2008.DesignParticipation(-s)-ACreativeCommonsforOngoingChange.CoDesign,4(2),pp79-83.

BOHMAN,J.1998.SurveyArticle:TheComingofAgeofDeliberativeDemocracy.The Journal of Political Philosophy,6(4),pp400-425.

BOWEN,S.,MCSEVENY,K.,LOCKLEY,E.,WOLSTENHOLM,D.,COBB,M.&DEARDEN,A.2013.Howwasitforyou?ExperiencesofparticipatorydesignintheUKhealthservice.CoDesign,9(4),pp230-246.

BRANDT,E.,BINDER,T.,MALMBORG,L.&SOKOLER,T.2010.CommunitiesofEverydayPracticeandSituatedElderlinessasanApproachtoCo-designforSeniorInteraction.22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction.Australia.

BUCHANAN,R.1992.WickedProblemsinDesignThinking.Design Issues,8(2),pp5-21.

BUDDS,D.2016.How One Florida City is Reinventing Itself with UX Design[Online].Available: https://www.fastcompany.com/3065107/how-one-florida-city-is-reinventing-itself-with-ux-design [accessed21September2018].

BURRAL,S.2015.Room for a View: Democracy as a Deliberative System.UnitedInvolve,Kingdom,pp1-68.

BUUR,J.,LARSEN,H.2010.TheQualityofConversationsinParticipatoryInnovations,Codesign, 6(3),pp121-138.

COHEN,J.1997.DeliberationandDemocraticLegitimacy.In:BOHMAN,J.&REGH,W.(eds.)Essays on Reason and Politics.UnitedStatesofAmerica:TheMITPress.

COLE,B.,PINFOLD,J.,HO,H.&ANDA,M.,2013.ExaminingthemethodologyofparticipatorydesigntocreateinnovativesanitationtechnologiesinruralMalawi.Delivering water, sanitation and hygiene services in an uncertain environment, 36th WEDC International Conference.Kenya.

COREY,Z.2009. Deliberative Democracy and the Emerging Jury System in Japan: A Natural Combination or Two Ships Passing in the Night. CornellUniversity.

29

DEM

OC

RAC

Y 20

25CURATO,N.,DRYZEK,J.S.,ERCAN,S.A.,HENDRIKS,C.M.&NIEMEYER,S.2017.TwelveKeyFindingsinDeliberativeDemocracyResearch.Daedalus- the Journal of the American Academy of Arts and Sciences,146(3),pp28-38.

CURATO,N.&ONG,J.C.2015.InclusionasDeliberativeAgency:TheSelectiveRepresentationofPoorWomeninDebatesandDocumentariesaboutReproductiveHealth.Television and New Media,16(6),pp576-594.

DESIGNCOUNCIL2016.Adur and Worthing: Using co-design to create a lasting legacy [Online].Available:https://www.designcouncil.org.uk/resources/case-study/adur-worthing-using-co-design-create-lasting-legacy[accessed21September2018].

DORST,K.2011.TheCoreof‘designthinking’anditsapplication.Design Studies,32(6),pp521-532.

DRYZEK,J.2016.DeliberativePolicyAnalysis.In:STOKER,G.&EVANS,M.(eds.)Evidence-Based Policymaking in the Social Sciences: Methods that Matter. UnitedKingdom,PolicyPress,pp286-301.

DRYZEK,J.&NIEMEYER,S.2008.DiscursiveRepresentation.American Political Science Review,102(4),pp481-493.

DRYZEK,J.S.&(WITH)NIEMEYER,S.J.2010.Foundations and Frontiers of Deliberative Governance, UnitedKingdom,OxfordUniversityPress.

ELSTUB,S.,ERCAN,S.&MENDONCA,R.2016.TheFourthGenerationofDeliberativeDemocracy. Critical Policy Studies, 10(2),pp139-151.

ELSTUB,S.&MCLAVERTY,P.2014.Deliberative Democracy: Issues and Cases, UnitedKingdom,EdinburghUniversityPress.

ENSERINK,B.&MONNIKOF,R.A.2003.InformationManagementforPublicParticipationinCo-design:EvaluationofaDutchExample.Journal of Environmental Planning and Management, 46(3),pp315-344.

ERCAN,S.A.,HENDRIKS,C.M.&DRYZEK,J.S.2018.PublicDeliberationinanEraofCommunicativePlenty.CrawfordSchool,AustralianNationalUniversity(ForthcominginPolicy and Politics).

EVANS,M.&TERREY,N.2016.Co-designwithCitizensandStakeholders.In:STOKER,G.&EVANS,M.(eds.)Evidence-based Policy Making in the Social Sciences: Methods that Matter. UnitedKingdom,PolicyPress.

FAUBION,S.2012.City of Canada Bay Council Citizens’ Panel. [Online].Available:https://participedia.net/en/cases/city-canada-bay-council-citizens-panel-0[accessed11June2018]

FELICETTI,A.,GASTIL,J.,HARTZ-KARP,J.&CARSON,L.2012.CollectiveIdentifyandVoiceattheAustralianCitizens’Parliament.Journal of Public Deliberation,8(1),pp1-27.

FELICETTI,A.,NIEMEYER,S.&CURATO,N.2015.ImprovingDeliberativeParticipation:ConnectingMini-PublicstoDeliberativeSystems.European Political Science Review,8(3),pp427-448.

FLORIDA,A.2013.ParticipatoryDemocracyversusDeliberativeDemocracy:ElementsforaPossibleTheoreticalGeneology,TwoHistories,SomeIntersections.7th ECPR General Conference Sciences Po.France.

FUNG,A.2009&FLETCHER,S.2018.2005BC Citizens’ Assembly on Electoral Reform[Online].Available:https://participedia.net/en/cases/2005-bc-citizens-assembly-electoral-reform[accessed 11June2018].

GREENBAUM,J.&LOI,D.2012.Participation,thecamelandtheelephantofdesign:anintroduction.Codesign,8(2-3),pp81-85.

HABIBIS,D.&WALTER,M.2015.Social Inequality in Australia: Discourses, Realities and Futures,Australia,OxfordUniversityPress.

30

DEM

OC

RAC

Y 20

25HAMPSON,M.,BAECK,P.&LANGFORD,K.2013.By Us, For Us: The Power of Co-design and Co-delivery [Online].Available:https://media.nesta.org.uk/documents/the_power_of_co-design_and_co-delivery.pdf[accessed21September2018]

HARDER,M.K.,BURFORD,G.&HOOVER,E.2013.WhatisParticipation?DesignLeadstoWaytoaCrossDisciplinaryFramework.Design Issues,29(4),pp41-57.

HARGRAVE,L.2018.Citizens Assembly on Brexit[Online].Available:https://participedia.et/en/cases/citizens-assembly-brexit[accessed11June2018].

HENDRIKS,C.M.2015.CouplingCitizensandElitesinDeliberativeSystems:theroleofinstitutionaldesign. European Journal of Political Research,55(1),pp43-60.

HOLMES,B.2011.Citizens’ engagement in policymaking and the design of public services. DepartmentofParliamentaryServices,Australia.

HOSMER,C.2017.Cormorant Island Economic Development Plan (Canada)[Online].Available: https://participedia.net/en.cases.cormorant-island-economic-development-plan-canada[accessed11June2018].

HUYBRECHTS,L.,BENESCH,H.&GEIB,J.2017.Co-designandthePublicRealm.CoDesign,13(3),pp145-147.

IDEO2015.Designing a school system from the ground up[Online].Available:https://www.ideo.com/case-study/designing-a-school-system-from-the-ground-up[accessed21September2018].

JENNSTÁL,J.2018.DeliberativeParticipationandPersonality:Theeffectoftraits,situationsandmotivation.European Political Science Review, pp1-24.

LEADBEATER,C.2004.Personalisation through Participation: a new script for public services, UnitedKingdom,Demos.

LEE,Y.2007.DesignParticipationTactics:thechallengesandnewrolesfordesignersintheco-designprocess.CoDesign,4(1),pp31-50.

LOWNDES,V.2016.NarrativeandStorytelling.In:STOKER,G.&EVANS,E.(eds.)Evidence-based Policymaking in the Social Sciences: Methods that Matter. UnitedKingdom,PolicyPress,pp138-161.

MCWHIRTER,R.E.,CRITCHLEY,C.R.,NICOLE,D.,CHALMERS,D.,WHITTON,T.,OTLOWSKI,M.,BURGESS,M.&DICKINSON,J.L.2014.CommunityEngagementforBigEpidemiology:DeliberativeDemocracyasaTool.Journal of Personalised Medicine,4,pp459-474.

MICHELMAN,F.I.1997.HowCanPeopleEverMaketheLaws:Acritiqueofdeliberativedemocracy.In:BOHMAN,J.&REGH,W.(eds.)Essays on Reason and Politics.UnitedStatesofAmerica,TheMITPress,pp145-171.

NABATCHI,T.&LEIGHNINGER,M.2015. Public Participation for 21st Century Democracy,UnitedStatesofAmerica,JohnWileyandSonsInc.

NIEMEYER,S.,ERCAN,S.A.&HARTZ-KARP,J.2013.UnderstandingDeliberativeCitizens:TheApplicationofQMethodologytoDeliberationonPolicyIssues.Operant Subjectivity: The International Journal on Q Methodology,36(2),pp114-134.

PARKER,S.&HEAPY,J.2006.The Journey to the Interface: How Public Service Design Can Connect Users to Reform,UnitedKingdom,Demos.

PARKINSON,J.2003.LegitimacyProblemsinDeliberativeDemocracy.Political Studies, 51,pp180-196.

PARRY,L.J.2016.DogsonInverlochBeachCommunityEngagement[Online].Available: https://participedia.net/en/cases/dogs-inverloch-beach-commuity-engagement[accessed 11June2018].

31

DEM

OC

RAC

Y 20

25PENUEL,W.R.,ROSCHELLE,J.&SHECHTMAN,N.2007.DesigningFormativeAssessmentSoftwarewithTeachers:AnAnalysisoftheCo-designProcess.Research and Practice in Technology Enhanced Learning,2(1),pp51-74.

RYAN,M.&SMITH,G.2014.DefiningMini-Publics.In:GRONLUND,K.,BACHTIGER,A.&SETALA,M.(eds.)Deliberative Mini-Publics.UnitedKingdom,ECPRPress.

SAAD-SULONEN,J.,ERIKSSON,E.,HALSKOV,K.,KARASTI,H.&VINES,J.2018.Unfoldingparticipationovertime:temporallensesinparticipatorydesign.CoDesign,1(14),pp4-16.

SAWER,M.,ABJORENSEN,N.&LARKIN,P.2009.Australia: The State of Democracy,Australia,TheFederationPress.

SHEN,S.2012andFLETCHER,S.2016.October 15 Movement[Online].https://participedia.net/en/cases/october-15-movement[accessed21October2018].

SMITH,G.&WALES,C.2000.Citizens’JuriesandDeliberativeDemocracy.Political Studies,48(1), pp51-65.

SZEBEKO,D.&TAN,L.2010.Co-designingforSociety.Australasian Medical Journal,3(9),pp580-590.

THINKPLACE2017a.Improving Literacy Outcomes for Children. United States of America[Online]. Available:https://www/thinkplaceglobal.com/work/improving-literacy-outcomes-children-1 [accessed21September2018].

THINKPLACE2017b.Those Most Affected Have a Say in Improving the National Disability Insurance Scheme [Online].Available:https://www.thinkplaceglobal.com/work/those-most-affected-have-say-improving-national-disability-insurance-scheme[accessed21September2018].

TRISCHLER,J.,PERVAN,S.J.,KELLY,S.J.&SCOTT,D.R.2018.TheValueofCo-design:TheEffectofCustomerInvolvementinServiceDesignTeams.Journal of Service Research,21(1),pp75-100.

UHR,J.&UHR,G.1998.Deliberative Democracy in Australia: The Changing Place of Parliament, UnitedKingdom,CambridgeUniversityPress.

UM,K.2013.Canada’s World Dialogue Series[Online].Available:https://participedia.net/en/cases/canadas-world-dialogue-series[accessed11June2018].

WEISS,H.A.2013.How Students Display Dialogue, Deliberation and Civic Mindedness: An Analysis of Democracy Plaza.UnitedStatesofAmerica,IndianaUniversity.

WEYMOUTH,R.2016.City of Greater Geraldton Deliberative Participatory Budget[Online].Available:https://participedia.net/en.cases/city-greater-geraldton-deliberative-participatory-budget [accessed11June2018].

WOODROFFE,L.2018.Brent Council: Keeping Vulnerable Young People Safely in the Community [Online].Available:https://www/designcouncil.org.uk/resources/case-study/brent-council-keeping-vulnerable-young-people-safely-community[accessed21September2018].

WRIGHT,S.&STREET,J.2007.Democracy,DeliberationandDesign:thecaseofonlinediscussionforums.New Media and Society,9(5),pp849-869.

ZIMMERMAN,J.,TOMASIC,A.,GARROD,C.,YOO,D.,HIRUNCHAROENVATE,C.,AZIZ,R.,RAVITHIRUVENGADAM,N.,HUANG,Y.&STEINFELD,A.2011.FieldTrialofTiramisu:CrowdSourcingBusArrivalTimestoSpurCo-design.SIGHI Conference on Human Factors in Computing Systems, pp1677-1686.

32

DEM

OC

RAC

Y 20

25ABOUT THE AUTHOR

NicoleMooreisanHonoraryFellowandPhD

CandidateattheInstituteforGovernanceand

PolicyAnalysis,UniversityofCanberra.Nicoleis

alsothe2019(inaugural)ACTLegislative

AssemblyLibraryFellowandhasheldnumerous

publicsectorrolesinalmostadecadeofservice.

In2013,NicolewasawardedaPublicService

ExcellenceAwardforthepromotion,co-design

andco-productionofserviceswithvulnerable

familiesintheACT.Nicole’sresearchisfocusedon

improvingpublicsectorcapabilityinengaging

citizensindecisionmaking,withaparticularfocus

onco-designanddeliberativeengagements.

Thisreportisthethirdoutputfromtheinitiative Democracy 2025—bridging the trust divide.

Forotherreportsinthisseriesvisitourwebsiteat:www.democracy2025.gov.au