IWMW 2004: Measuring the Impact of a CMS Implementation (A7)

IWMW BirminghamJuly 2004

Measuring the Impact of a CMS ImplementationIs the Web Guru dead?

Grant Malcolm,The University ofWestern Australia


About UWA● 16,000 students ● 3,500 staff● research intensive● “high tech, high touch”● member of Australian Group of Eight ● a “sandstone university”


Web @ UWA - pre CMS

● responsibility highly devolved ● infrastructure highly decentralised ● no web guidelines ● no SOE ● significant duplication of content/effort● poor ROI in web development ● Feb 2002 snapshot* courtesy of

http://www.archive.org/


Implementation – the sum of its parts● academic restructure ● faculty sub-branding ● web guidelines ● site management plans ● CMS


Web @ UWA - post CMS● still no SOE ● still devolved responsibility for content● infrastructure largely centralised● content reused and repurposed● limited guidelines supported by CMS ● better ROI in web development ● Two years later - Feb 2004 snapshot*


CMS Impact - High● consistent "look" ● consistent navigation styles ● improved accessibility● guideline & standards compliance ● easier to develop/maintain● “a University website”● shift focus of web development


Web @ UWA – pre CMS

Decentralised

●layout●branding●design●navigation●structure●accessibility●standards ●content●infrastructure

Centralised


Web @ UWA – post CMS

Shared●design●layout●navigation●accessibility

Centralised● branding● standards● infrastructure

Decentralised●structure●content


CMS Impact - Moderate● currency of content ● improved structure ● better information management ● better ROI ● fewer outlaws


Expectations for website (and CMS)● consistent● usable● accurate● alignment● available● compliant● current● affordable... suggestions?


Consistent● portion of site in CMS (37k pages or ~10%)*● lies, damned lies and Google page counts● intranet and applications?● organisational units in CMS (132 or 93%)● content weighting?● design, navigation and behaviour● flexible templates – the balancing act*


Number of Pages in CMS

Currently ~37,000 pages

0

5000

10000

15000

20000

25000

30000

35000

40000


Usable● navigation● structure● usability studies● client surveys and feedback


Accurate● sourcing from the source● repurposing and re-using content

– Static and/or unstructured e.g. pull or nest content, minimal document management*

– Dynamic and/or structured e.g. contact, event or handbook details*


Alignment● Supports policy and strategic initiatives● Integrates with enterprise architecture

– VLEs, LMSs and CMSs– Personnel and Financial Systems– Student information systems– Portals– Library Systems– SSO


Available● Client expectations ● SLA's


CMS Reliability & Stability 2003 - 2004

2003

2004

Excellent Good Average Fair

0

2

4

6

8

10

12

Excellent Good Average Fair Poor0

5

10

15

20

25

30

35

40

45


Compliant● validate against (which?) standards

– <table> not <div>– HTML not XHTML

● accessibility– currently minimum WCAG A rating

● site management plans– required for all official sites– currently checklist– monitor CMS (93%) vs non-CMS (76%)*


Site Compliance with Web Guidelines

Inside CMS Outside CMS0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Current● frictionless publishing “many hands”

– staff maintaining content in CMS (623 or ~20%)*– editor accesses (20-250 logins/working hour)*– development rates (0-150 pages/working hour)*– updating rates (20-150 pages/working hour)*– demand for support/training (11 requests/day)*– 30k pages/480 working days = 62 pages per day– CMS user surveys and feedback*

● expiry and review notification


Number of Staff Publishing in CMS

Currently 623 staff

0

100

200

300

400

500

600

700


Staff Using CMS by Role 2003 - 2004

AcademicAdministrativeManagerial ExecutiveTechnical

2003

2004


CMS Specific Training 2003 - 2004

No trainingLess than one hour1-3 hoursMore than 3 hours

2003

2004


CMS Ease of Use 2003 - 2004

Excellent Good Average Fair

0

2

4

6

8

10

12

14

16

18

20

1

19

54

Excellent Good Average Fair Poor0

5

10

15

20

25

30

35

40

45

50

10

45

15

12 12

2003

2004


Affordable● resourcing levels: staffing, hardware &

software infrastructure– Staff up to 3.5 FTE = A$150k– Six Intel boxes = A$50k– OpenSource software = A$0– Organisational unit costs not included

● two year implementation/rollout cost – A$200k/30k pages = A$6.67 per page


Overall Benefit of Using CMS 2003

Huge benefit Significant benefit Marginal benefit Neutral Marginal set back Significant set back0

5

10

15

20

25

30


Overall Benefit of Using CMS 2004

Huge benefit Significant benefit Marginal benefit Neutral Marginal set back Significant set back Huge setback0

5

10

15

20

25

30

35


Amount & Effectiveness of Effort - 2004

Decreased hugely Decreased significantly Decreased marginally About the same Increased marginally Increased significantly Increased hugely0

2

4

6

8

10

12

14

16

18

Amount of effort

Much more Marginally more About the same Marginally less Much less0

5

10

15

20

25

Effectiveness of Effort


Web @ UWA - what hasn't changed?● poor management predominates ● fragmented, siloed architecture ● disparity between sites depending on

resourcing levels ● web gurus?


Web Gurus?

Shared●design●layout●navigation●accessibility

Centralised● branding● standards● infrastructure

Decentralised●structure●content...from informationand navigation totransactional,strategic web use? ?

??

?

?


Questions/feedback?● Thank you

IWMW 2004: Measuring the Impact of a CMS Implementation (A7)

Education

Transcript of IWMW 2004: Measuring the Impact of a CMS Implementation (A7)