Commercial Competency and Computing StudentsUsing the Skills Framework for the Information Age in Higher Education

Tom McEwanInstitute for Informatics and Digital Innovation

Edinburgh Napier UniversityEdinburgh, UK

t.mcewan@napier.ac.uk

Abstract—Commercial acumen has emerged recently as a third desirable aspect of employability expected from computing graduates, in addition to technical capability and "soft skills" (or similar terms like transferable skills). Our experience has been that viewing commercial acumen (or even awareness) as simply one of the "soft skills", has failed to meet the needs of local employers, who seek innovation skills and entrepreneurship. A case study illustrates a structured approach to adding commercial awareness to the computing curriculum, and, more generally, tying the learning experience more closely to the achievement of standardized competency statements. Changes to future provision are discussed following workshop discussion of a draft of this case study.

This paper will be of interest to computing and engineering academics who seek to increase the commercial awareness of their students, and to those who seek to align their courses with commercial definitions of competency

Keywords—competency frameworks; commercial awareness in computing degrees; innovation

I. INTRODUCTION It has become increasingly desirable for computing

graduates to be able demonstrate commercial acumen. Governments and employers state this need in different ways, but our experience is that it is emerging as a third key factor in defining employability, beyond the existing desire for the optimal combination of domain-specific skills (programming, design, etc) and “soft skills” (time and project management, communication). In earlier work [1] we discussed the derivation of the current working set of generic graduate attributes and it was noticeable that only one of the eight components [2] referred to employability. Discussions with local employers suggest that these generic attributes are indeed desirable but more salient are areas such as customer-centeredness, innovation, entrepreneurship, business-awareness, market analysis and customer awareness. However, in an already overcrowded curriculum, how can this additional expectation be effectively integrated?

The growing use of national and global competency frameworks (such as SFIA: the Skills Framework for the Information Age [3]), both in term of job specification and skills gap analysis for individuals and organizations, and the increasing integration of these with the accreditation

requirements of professional bodies, provides a potential focus for a solution.

This paper builds on other previous work, first in engagement with the maintenance of competency frameworks [4] in arguing for more socially-responsible forms of innovation [5] and an earlier version of this work [6] presented in a workshop organized by an interest group of BCS, The Chartered Institute for IT.

After discussion of BCS activity in the relevant area, this paper discusses a case study which illustrates generic challenges in embedding SFIA competency statements in higher education learning outcomes, in particular where learners’ intended job roles are in a fast-emerging area and thus lack formal codification. The case study describes the process of gaining university quality acceptance for a masters-level module (Digital Markets) based on SFIA competency statements, and on the first running of the module. This provides a mapping of SFIA levels to higher taxonomies of learning, and illustrates how both students and course leaders construct degree courses. These experiences are contrasted with the position statements from other workshop participants (drawn from three continents), each of whom reported on related activity using SFIA. A parallel process, led by BCS Interaction Specialist Group, used this module to define both competency and courses for the relatively new skill of User Experience (UX).

All of this has fed back into subsequent delivery of the above module, this time to a small but more international cohort of students (from Asia, Middle East, Spain and the UK, drawn from a number of different MSc programs). Here the requirements become more complex, as the learners from different countries start with very different understanding of, and attitudes towards, commercial issues.

The module requires each learner to develop a concept for the digital marketplace, and to identify the nature of the opportunity, commercial viability, and "freedom-to-operate" (for example, whether patents exist in certain jurisdictions). The learners then peer-review the proposals, and indicate their preferences, before forming into two groups to develop the proposals for an eventual pitch to established entrepreneurs, who provide feedback as if this was a commercial situation. To conclude, learners provide a reflective commentary on the project experience as part of their assessment.

This paper discusses approaches to “level up” each learner to the same starting point for the group activity, reflects on the management of the groups and identifies patterns of learner activity and reactions, which can be used to improve subsequent delivery.

II. BACKGROUND: UX AND BCSThis paper views the domain of User Experience (UX) as

encapsulating much if not all of the commercial acumen agenda. UX, as a discipline, arguably has grown out of the Usability sub-specialism of Human-Computer Interaction (HCI), itself part of a longer tradition of Human Factors within Computing. UX also grows out of ergonomics, psychology, and marketing. It looks at people’s engagement with technology and services, and seeks to ensure that their experience of using the technology is not only problem-free, but also exceeds expectations, delights the user and thus becomes valued by that user. As more transactions are completed online, good UX is fundamental to fitness for purpose. There is a global shortage of these skills, and senior salaries are amongst the highest in any field of computing.

The global professional body, UXPA, has fifty chapters around the world, coordinates the annual World Usability Day in forty countries, and seeks to create a global professional community [7]. The author is a member of IEEE, BCS, ACM and UXPA, and informal contact with other members of UXPA suggests a considerable overlap in memberships across these and other professional bodies, including ergonomics, engineering, psychology, design and marketing. It is difficult to pigeonhole UX into the work of a specific sector skills council (the collective term used in the UK for bodies which define competency), although UX is an important part of the future of Computing, and thus a key opportunity area for BCS and other professional bodies for the field of computing.

A. BCS and HCI In 1984, BCS calved one of the world’s first specialist

groups in HCI, variously known through the years as BCS-HCI, British HCI Group and, now, Interaction Specialist Group (which the author chaired 2009-2011). After a special edition of BCS's Computer Bulletin, this group launched one of the first HCI academic journals [8], which, despite periodic changes in publisher, retains a good impact factor to this day (5-year impact factor: 1.455 [9]. The group's annual conference [10] and UsabilityNews.bcs.org web portal also have global impact.

The group worked with the UK government to produce Usability Now (1990), an initiative to embed usability in UK software. Many would argue that the industry is yet to fulfil these values. By the turn of the century, the group worked with the UK government's E-Envoy to ensure accessibility in public websites [11]. Members of the group executive committee and contributed to iterations of SFIA and the BCS's extended version SFIAplus [12], resulting in an increased number of usability roles being recognised from SFIA v3 onwards [4]. Most recently SFIA v5 [3] is the first version to include UX in its skill definitions, shifting the focus from “non-functional requirements” (particularly a misnomer, from a cognitive ergonomics point of view) towards a more holistic

understanding of the needs and the experience of the user, in definitions for skills such as:

User experience analysis (UNAN);

Ergonomic design (HCEV);

User experience evaluation (USEV);

Human factors integration (HFIN).

For example, the latter is defined as: "Achievement of optimum levels of product or service usability, by ensuring that project and enterprise activities take account of the user experience."

B. UX Competency WorkshopsOn behalf of the group, the author led two national

workshops, UXCF2010 in London, UK, and UXCF2011 in Newcastle, UK. UXCF2010 attracted around 40, mainly industry, participants and sought to understand roles and organisational processes in the fast-developing field of UX. While some of these roles are recognisably part of SFIA, others are not, nor are they adequately defined in national occupation standards (NOS) such as those by UK sector skills councils such as e-skills, Skillset or CCDesign.

UXCF2011 extended the materials produced in the first workshop in a smaller event involving three academics who design relevant BSc/MSc courses, and three senior UX professionals, supported by other interested parties. Most had contributed to and/or reviewed the recent SFIA revisions. We reviewed an example set of modules against typical UX roles to identify gaps, and to evaluate how well the new module discussed here, IMD11108 Digital Markets, fills any gaps. Another paper [6] describes the UXCF activity in more detail.

III. CASE STUDY: DIGITAL MARKETS

This section describes the first experiences of running a masters-level module, IMD11108 Digital Markets [13], which was written explicitly to use learning outcomes based on SFIA competency statements. This module lasts a semester, carries 10 ECTS (European Credit transfer System) credits, with 36 hours of classes and 164 hours of student centered activity including group-work. It addresses UX in the sense that it focuses on entrepreneurship and innovation from a Human-Centered Design (HCD) perspective [5], contrasting different approaches to innovation, in particular what the OECD term “non-R&D innovation” [14,15], which focuses on user-driven innovation, ie demand-led rather than "marketing-push".

A. Quality ApprovalThe first challenge was to gain university quality

acceptance for a module based on SFIA statements. University quality procedures, at least in the UK, reflect a complex set of stakeholders as previously discussed [1]. The module itself was not created for the sake of delivering competency – at best this would be seen as necessary but not sufficient. Meeting the emerging needs of employers, while providing an opportunity for critical evaluation of different approaches to applying innovation theories, more fully addresses all agendas. The module descriptor runs to several pages [13], but the key points discussed here are the Learning Outcomes (LOs). These are the

basis of our quality system: all assessments are moderated against the LOs, and all teaching events and directed study are designed to be necessary and sufficient for students to achieve the learning outcomes.

B. Matching Levels and BreadthThe first consideration of the module was in March 2009.

The first task was to contrast the language of the different skills levels within SFIA with the required vocabulary of module accreditation at levels 10 (Honours (final year) undergraduate) and 11 (Masters-level) of the Scottish Credit and Qualifications Framework (SCQF) [16]

Figure 1: "Acceptable" verbs to meet SCQF10 and SCQF11 learning outcomes (Edinburgh Napier University internal memo)

Our experience has been that the language in SCQF 10 is found between SFIA levels 4 and 5, and that of SCQF 11 at SFIA levels 5 or 6. Although the question has been asked at open forums, there appears to be no definitive mapping between SFIA levels and university learning taxonomies. A typical response is that higher level roles within SFIA require not only knowledge and understanding (as delivered in a degree), but practical experience (seen as not delivered in a degree). One might infer that universities educate at a level well above initial graduate jobs, and that employers are investing in the proven potential of these graduates to progress over 3-5 years to professional roles.

Our experience was that a single SFIA skill appears to be too narrow a focus for a 200 hour learning experience, and thus two related SFIA skills Emergent Technology Monitoring (EMRG) and Innovation (INOV) were mined to provide sufficient breadth for IMD11108. The language of learning outcomes (LOs) is traditionally more concise than that of competency. The relevant statements for SFIA levels 4- 6 in EMRG are (with key points highlighted):

"Maintains awareness of opportunities provided by new technology to address challenges or to enable new ways of working. Within own sphere of influence, works to further organisational goals, by the use of emerging technologies and products. Contributes to briefings and presentations about their relevance and potential value to the organisation". (EMRG4)

In any case level 4 statements appear to require less synthesis than expected at Masters or even Honours year level. Thus the following LOs were mapped onto EMRG5, although LO5 goes slightly beyond (as highlighted).

"Monitors the market to gain knowledge and understanding of currently emerging technologies.

LO4 Monitor technology markets to gain knowledge and understanding of currently emerging technologies

Identifies new and emerging hardware and software technologies and products based on own area of expertise, assesses their relevance and potential value to the organisation, contributes to briefings of staff and management".(EMRG5)

(EMRG5). LO5: Identify new and emerging hardware and software technologies and products, assess their relevance and potential organisational value and brief staff, management and investors (EMRG5).

"Co-ordinates the identification and assessment of new and emerging hardware, software and communication technologies, products, methods and techniques. Evaluates likely relevance of these for the organisation. Provides regular briefings to staff and management".(EMRG6)

Innovation (INOV) provided the basis for the other learning outcomes. Only two levels currently exist for this:

"Actively monitors for, and seeks, opportunities, new methods and trends in IT capabilities and products to the advancement of the organisation. Clearly articulates, and formally reports their benefits." (INOV5)

"Monitoring" and "reporting" alone, were judged insufficient to meet SCQF level 11, being more typical of the investigation that an Honours Project student might carry out (SCQF level 10). Thus INOV6 was the basis for the three other Learning Outcomes, although the taxonomy falls midway between levels 5 and 6

"Recognises potential strategic application of IT, and initiates investigation and development of innovative methods of exploiting IT assets, to the benefit of organisations and the community. Plays an active role in improving the interface between the business and IT." (INOV6)

LO1: Recommend potential strategic application of IT in the digital marketplace (INOV6).LO2: Work in a group to exploit IT assets in an innovative way, to the benefit of organisations and/or the community. (INOV6).LO3: Conceptualise ways to improve the interface between the business (or organisation) and IT. (INOV6).

The UX dimension is notably missing from both LOs, and other SFIA roles were considered including Human Factors Integration (HFIN). While these did not add value to the LOs, the "Overview" section of the HFIN skill is more relevant and thus provides a backdrop to the module:

Human Factors Integration – Overview

"Achievement of optimum levels of product or service usability, by ensuring that project and enterprise activities take account of the user experience."

The module was accepted at the second submission to the Faculty Quality process, subject to amendments. One such amendment was to clarify the variety of job roles to which the learning outcomes would lead. This information is fairly vague, even in SFIAplus, and the following was proposed to gain acceptance:

“This module helps prepare learners for the following roles in the Skills Framework for the Information Age (SFIA): INOV6,

EMRG5, HFIN5. These roles are found in digital media innovation and new product development teams in larger

organisations and relevant job titles include: Digital Media Developer/Designer within an Innovation Team or New

Product Development team, User Experience Developer/Designer/ Manager, Imagineer (a Disney Corp.

term), Human-centred Designer. This module will also

support a potential digital media entrepreneur to develop their concept.”

Once accepted, the module then also had to be made attractive to students – the first mandatory running for the integrated masters (MEng) programmes was not until 2011-12. In the meantime it was available as an option for other postgraduates, but there was insufficient demand until the Fall semester, 2011.

C. StructureThe module simulates a commercial innovation cycle. For

the mid-term assessment, individual students build up their own proposals for a viable product or service for the digital marketplace. After submission, they peer-review each other's work and select the best of these ideas to develop in groups of 3-4 to the point where they can pitch to local entrepreneurs, as if for around $250k in seed-corn investment (several such programs exist locally for university spin-out ideas). Their presentations are formally assessed as a group, and their final assignment is to reflect on the group and their own performance. The teaching is front-end loaded to prepare students for working in groups to accomplish the main objectives of the module. At the end of the module students must reflect individually on the experience, which is the opportunity to display their Masters-level capabilities.

Table 1: Time Management advice given to students

Week

Work Description

2-3 Heavy (20hrs per week)

You have 12 hours of classes and a great deal of directed study to do, meaning you will likely work more on this module than other modules.

4-9 Moderate (12hpw)

No classes - 5 short tutorial meetings: you will be focused on your individual coursework (due week 8) and starting to develop your group for the second coursework. Your only classes will be some individual supervision meetings and then short group meetings with the module leader to discuss group formation and objectives.

10-12

Heavy (20 hpw)

18 hours of classes: you will prepare in a group for, and give, a 15 minute Dragon’s Den type pitch for your concept

13-15

Light (10 hpw)

Completion of final 1500-word paper reflecting on the group experience

Table 1 explains the unusual contact time pattern – double the normal contact hours for the first few weeks, brief weekly group supervision meetings in the middle six weeks, and double the normal contact. The expectation was that students would be highly self-motivated and self-organizing, having already gained experience in managing their solo project in fourth year, and in participating in formal group projects in previous years.

D. Experience of Delivery For a variety of reasons, the cohort of undergraduate

integrated-Masters students who took the module lacked both the highest and lowest performing quartiles of the previous year. The "less average" students had elected to graduate with the more typical BSc (Hons) or BEng (Hons) at the end of their fourth year. Thus the first cohort for this module contained

"average students". We routinely contrast the performance of all students in any module, compared to their collective performance in their other modules. The resulting performance in 2011 was slightly above average for that cohort. This analysis could not be completed in 2012, as only a single student remained for the fifth year, as a favorable local job market absorbed the rest of the cohort.

In contrast, several one-year MSc students from overseas considered the module in 2011, but elected to take "more technical" subjects instead, expressing worry about the discursive, group and presentation-based assessment regime. In the 2012 delivery, proactive steps allayed such fears, and the resulting cohort, though small, was globally diverse: students from Saudi Arabia, Spain, Egypt, three regions of China, and one local student.

This 2011 instance involved only seven students, four from the MEng Software Engineering and three from the MSci Interactive Media Design. The overall response in student feedback was highly positive and the advice one student gave to future students was interesting:

"Try and create a new idea, as my idea wasn't new or a good business plan, I did not take it further after the module. But

with the support of the module, by the end of it, you have a lot of useful information if you decided to create a start up

company."Largely, the module met ambitions. The students did a very

good job of each researching a digital market idea and writing a short paper (3000 words) on it, submitting in week 8. Example topics included an on-demand BluRay burning kiosk (to cope with broadband rollout roadmap), an app for stag and hen parties ("bachelor(ette) parties"), and an app that used geodata and public licenses to detect and avoid "chuggers" – aggressive on-street charity collectors. Thereafter they adopted the last two of the above ideas, split amiably into two groups to further develop their preferred idea, until they were ready to pitch, in week 12, to an audience of six industrialists, entrepreneurial advisers and lecturers:

The CEO of a successful spin out company now in its second year of trading

An entrepreneur, business "Angel" and senior advisor to government

An entrepreneur and advisor for Proof of Concept (PoC) commercialization spin-out project funding from government agency Scottish Enterprise.

The director of the university's Moffat Centre, which advises around fifty students each year on how to start their own companies

An academic who had recently run a PoC

A professor with responsibility for coordinating faculty-wide research and commercialization

The panel rated each group's pitch in a number of dimensions (although final grading decisions were taken by the module leader). Both pitches, though trenchantly criticized by the expert audience (in a way that the students reported in the

post-presentation discussion session as “brutal” and “a wake-up call”) were also (in the formal feedback sheets) well-received by that audience – no-one rated any aspect less than "adequate", most were rated "good" and some were rated "very good". Since the panel had been asked to judge these on a commercial basis, this alone was a remarkable achievement, and one which suggests that the use of SFIA statements was effective in leading to competency, in particular LO5: ("Identify new and emerging hardware and software technologies and products, assess their relevance and potential organisational value and brief staff, management and investors").

1) 2012- second deliveryOnly minor changes were made to the timetabling, but

these, combined with a delayed start by several of the students, and acclimatization by those who had recently arrived from overseas (most of the students), meant that the first two weeks class activity did not result in the same level of familiarity with the nature of commercialization. Additionally some of the students come from cultural backgrounds with very different approaches to investment, legislation, regulation, consumer protection and so on. While the latter was all anticipated, trying to run a crash course in liberal Western economic and business values, at a time of acclimatization was unwise, resulting in an initial response of culture shock. The relatively poor performance in the mid-term assessment prompted a more hands-on approach during the middle period, something that is feasible with a small class, but would have been problematic with a more typical class size of 20-40.

It became clear that the students needed careful shaping into their groups for the second half of the module to work. The middle section of the module has a lot of activity that does not count for assessment but does build students' abilities to peer evaluate, as well as and providing a mechanism for the groups to coalesce around the most viable ideas.

The planned individual meetings in weeks 4-6 were augmented with two additional weekly meetings of four students at a time (Groups A and B) to develop their own and critique each other's proposals for the first assignment. Since all students had chosen quite distinct topics, there were no plagiarism risks involved in this. Some of the ideas were less well developed than the previous year and often highly derivative of examples provided (travel recommenders, study guides, tourism apps), but others were of a very high standard – a brokerage for statutory recycling of medium value electronic waste, a data scraping app that supplied "key information sets" (a new form of open data about UK universities that has proven costly to collate), an organic food delivery brokerage, personalized location-based M-Commerce services.

Within these groups a strong understanding each other's proposals emerged. This could have compromised the anonymous peer review of each other's papers in weeks 7-9 (which refers students to typical journal and conference reviewing guidelines).

The solution was to give three papers from group B to each student from group A to review, and vice versa. This then meant that each student received three anonymised reviews of

their business idea from members of the other group (with whom they otherwise had little close contact). Students then discussed these reviews within their groups. This now meant that all students had a detailed knowledge of all eight potential projects for the group stage. Students received a summary of the positive and negative points of each paper and then were invited to send their project idea preferences for the group stage. Most review criteria were on a scale of 1-5, and reviewers were invited to supply additional free text.

Reviewer ’s Expertise Please rate your own knowledge of the Concept proposed in the paper, how much you think you know generally about Business/Digital Markets and add any comments about your knowledge

Strengths of the Proposal and Potential improvements

Try to identify up to three good aspects of the proposal in this paper, and also three possible improvements

Commercial opportunity and potential barriers to success

Try to identify the profitability and scale of the commercial opportunity described , and the barriers that the proposer would have to overcome

Investment potential If you had £500,000, would you invest £100,000 in this idea?

The resulting preferences clearly prioritized two ideas, although, curiously, neither of the original authors voted for their own idea first! The final allocation into groups ensured that each student got their first or second choice, while maintaining a reasonable balance of ethnicity and gender, and transfer of members between groups A and B.

Subsequently, the resulting groups successfully presented to a similar audience to the previous year. Some niggles emerged within groups – a protocol had been advised for resolving group breakdowns, but it's more effective to avert the need for this with regular and detailed group supervision meetings. The grades for both the presentations and reflective statements were slightly better than the previous year.

These two experiences, coupled with group discussions at subsequent workshops, and comparison with teaching and learning approaches elsewhere in this subject area, has prompted a rethink of the delivery for next year. A more traditional contact pattern will be used: three hours a week of contact throughout the trimester, one of which will remain in small groups.

E. EvaluationThe experience of the 2011 delivery was presented at a

recent workshop [17], in a session with presenters who had attempted related integration of competency definitions with undergraduate teaching in Chile [18], Australia [19] and England [20]. This provided an opportunity to evaluate the intervention with peers attempting related innovations in their own countries.

While national or state governments in Chile and Australia emerged as more developed in mandating the mapping of courses to SFIA, a clear consensus emerged that basing curricula on competency definitions in SFIA was effective for educators, learners and employers. Other participants confirmed the mismatch between the advanced level of learning that universities require, and the likely lower level of initial graduate employment. Strategies are needed to help learners and graduates understand that, while initial roles may seem menial and to make little use of their advanced learning,

that their employers are investing in graduates' proven ability for future, more senior, roles. While graduates have always known that they need to gain practical experience as well as theoretical knowledge, it will be important to define that practical experience. Plainly some quite advanced practical experience can be gained during study, for example using the scaffolded PBL approaches we discussed at FIE2012 [1].

This module illustrates both the possibility and usefulness to build postgraduate teaching around SFIA competency statements. Both the university quality approval and quality enhancement processes proved to be no barrier, and the assessments produced clearly met expectations. The selection of level 5/6 for Masters-level, and 4/5 for Honours-level appear to be generalisable to other areas of computing, although basing a 20-credit module (nominally 200 hours of student work, including classes) on a single role level may be inadequate.

A subsequent workshop, organized by one of the constituent parts of the BCS Academy, identified many other entrepreneurship and innovation modules in computing-related degrees at a wide range of other UK universities, for example at Queens University, Belfast [21], University of Dundee [22] and the University of Northumbria [23] and around twenty others. Local employers appear in each area to remain unaware both of such modules (notable exceptions are Durham University, where Computing students carry out group projects for external clients, and the University of Kent, which has a student-led computing contracting company). More importantly employers are clearly unaware that the issues they highlight are very much being addressed by these type of modules.

As well as providing opportunities for students to gain knowledge and understanding well in advance of their first jobs in industry, university study can provide opportunities for developing practice competency to be ready for future opportunities.

IV. PARALLEL AND FUTURE WORK

The evolution of this module was done in parallel with the UXCF process described earlier – the author's attempts to build a coalition to call for updates to the most recent version of the skills framework, SFIA, to maintain currency with graduates' skills and emerging UX job specifications. Space only permits brief coverage here, but uxcf.org has been reserved for more detailed exploration of the work carried out, and for follow-up workshops.

UXCF2010 (BCS Covent Garden, Feb 2010) involved ten papers, half from industry, half from academia, and attracted a further 30 participants mainly from industry. Alternate sessions involved group discussions to identify how organizations and individuals developed competency in UX. As well as discussions, the groups created a number of affinity mappings or other diagrams intended to capture the skills needed in UX practice. By the end of the day much of the discussion was about organizational maturity, both in terms of the UX companies and also their clients. This is consistent with findings of other initiatives eg [24]. In other work we plan to analyze this in more depth.

In the first session, much of the dialogue centered on understanding to what extent definitions of job role or competency were shared across different organizations. In later discussion, groups focused on range of skills, knowledge and understanding required. For the academics present, there was much value in hearing about the different, often highly dynamic, approaches taken by employers to assemble project teams based on sets of skills. For the industrialists, there was growing appreciation of different levels of capability maturity within organizations to deliver UX consistently

UXCF2011 (University of Northumbria, July 2011) was intended to be more focused and specific to the design of degree programs. The individual learning outcomes of existing modules were mapped [6] against the table of professional competencies articulated the previous year. This enabled both employers and academics to agree potential gaps helped identity gaps, and confirmed where IMD11108 would contribute both to meeting existing requirements, but also to "energizing the future".

V. REFERENCES

[1] Davison, Brian and Tom McEwan, "Constructive scaffolding for accessible PBL". In FIE2012: 42nd Annual Frontiers in Education Conference, Seattle , 2012.

[2] Hounsell, D. 2011. Graduates for the 21st Century: Integrating the Enhancement Themes. The Quality Assurance Agency for Higher Education.

[3] "Skills Framework for the Information Age". 2012 SFIA Foundation. http://sfia.org.uk/ accessed 13 July 2013.

[4] McEwan, Tom "Playing the game: HCI careers in the competency era." In Proceedings of HCI Educators 2009, University of Abertay Dundee, UK, Swindon.

[5] McEwan, Tom. "Human-Centred University Commercialisation.," in People and Computers XXIII: Celebrating People and Technology, Swindon, 2009, pp. 362-369. http://ewic.bcs.org/content/ConMediaFile/13717 accessed 13 July 2013.

[6] T McEwan, "Digital Markets and UX – Case studies in SFIA in Higher Education," in Using SFIA in Education and Workplace Learning Conference, 13th June 2012, Milton Keynes, UK, 2012. http://www.bcs.org/upload/pdf/sfia-mcewan-130612.pdf accessed 13 July 2013.

[7] "About UXPA." 2013. UXPA.com http://www.upassoc.org/about_upa/index.html accessed 13 July 2013.

[8] Diaper, Dan April 1989."The discipline of HCI," Interacting with Computers, vol. 1, no. 1, pp. 3-5,

[9] "Interacting with Computers." 2013. http://iwc.oxfordjournals.org/ accessed 13 July 2013.

[10] Love, Steve. (2013, April) The 27th International British Computer Society Human Computer Interaction Conference: The Internet of things. http://hci2013.bcs.org/ accessed 13 July 2013.

[11] Office of the e-Envoy, "Quality framework for UK government website design: usability issues for government websites," UK Government, London, UK, 2003.

[12] BCS, The Chartered Institute for IT. (2011) SFIAPlus. http://www.bcs.org/sfiaplus accessed 13 July 2013.

[13] "IMD11108 Digital Markets." 2013 Edinburgh Napier University. http://www.modules.napier.ac.uk/Module.aspx?ID=IMD11108 accessed 13 July 2013.

[14] OECD. 2002. Frascati Manual 2002 Proposed Standard Practice for Surveys on Research and Experimental Development. OECD Publishing,.

[15] OECD. 2006. Oslo Manual: Guidelines for Collecting and Interpreting Innovation Data, 3rd Edition. OECD Publishing.

[16] The SCQF Partnership Executive Team. 2009. SCQF Handbook: User Guide. Glasgow, Scotland, UK: The Scottish Credit and Qualifications Framework Partnership.

[17] "Using SFIA in education and workplace learning." 2012. BCS Learning & Development SG. http://www.bcs.org/content/ConWebDoc/44160 accessed 13 July 2013.

[18] Quiroga, Karin. June 2012 "SFIA in the Redesign of IT Programs. PDF." . http://www.bcs.org/content/ConMediaFile/20355 accessed 13 July 2013.

[19] Jone,s Asheley. June 2012 "SFIA in the Australian ICT Profession. PDF". . http://www.bcs.org/content/ConMediaFile/20353 accessed 13 July 2013.

[20] Margaret Ross. (2012, June) SFIAplus in the Curriculum. PDF. [Online]. http://www.bcs.org/content/ConMediaFile/20358 accessed 13 July 2013.

[21] Barry McCollum. (2013, April) CSC4004 Timetable. Webpage. [Online]. http://www.cs.qub.ac.uk/~B.McCollum/csc4004/index_files/timetable.htm accessed 13 July 2013.

[22] Annalu Waller and David Farquhar. (2013, April) AC41007 Technology Innovation Management Mini Guide. Webpage. [Online]. http://www.computing.dundee.ac.uk/internal/miniguide.asp?AC41007 accessed 13 July 2013.

[23] Vijay Vij and Steve Ball, "Exploring the impact of entrepreneurship education on university nonbusiness undergraduates," Int. J. of Entrepreneurship and Small Business, vol. 9, no. 1, pp. 86-109, 2010.

[24] A. Marcus, R. Gunther, and R. Sieffert, "Validating a Standardized Usability/User-Experience Maturity Model: A Progress Report," in Human Centered Design , M. Kurosu, Ed. Berlin: Springer, 2009, pp. 104-109..