Collins 1

Regina Collins

Prof. Robert Myre

PTC 650

28 July 2009

Needs Analysis for

Data Manipulation for NJIT Coauthorship Database WBT

NJIT ADVANCE Project

Introduction

The NJIT coauthorship database was developed as part of the NJIT ADVANCE project, a

three-year Institutional Transformation Grant from the National Science Foundation. The

purpose of the NJIT ADVANCE project is to connect women researchers in science and

engineering to each other, to their male peers, and to female counterparts in industry and

government.

An important part of the original grant proposal was an examination of whether or not

female faculty are more isolated than their male peers. Dr. Nancy Steffen-Fluhr et al. determined

that coauthorship, an activity that is crucial to university and faculty advancement, would be an

effective tool in the assessment of female faculty isolation. Her team constructed a database of

articles authored/coauthored by NJIT faculty using faculty-supplied CVs as well as data

automatically extracted from such journal databases as Scopus and EbscoHost.

The data extracted from the database must be significantly manipulated before it can be

imported into a Social Network Analysis (SNA) application. The data manipulation is

performed primarily in Microsoft Excel. Currently, there are only two people trained to perform

this data manipulation. Clearly, training must be created to instruct other ADVANCE team

Collins 2

members in this critical task. Because the team members come from different departments

within the university, including some who are eLearning (distance learning) students, web-based

training (WBT) is the most appropriate medium through which to deliver this training.

Collecting Data Using Interviews With Justification

In order to create an effective WBT for the NJIT ADVANCE project, a needs analysis

was conducted to analyze the training needs and determine the following:

Best WBT method

Training goals

Audience and environment

To collect the necessary information, I interviewed several existing members of the NJIT

ADVANCE project team who are not currently trained in the data manipulation tasks. The

benefits of using interviews as opposed to observation or questionnaire for data collection are as

follows:

An interview enables me to ask in-depth questions about the learner’s knowledge,

skills, and attitude towards the subject. Because learners come from different

backgrounds, it is difficult to assess their competency through other means.

An interview provides the opportunity to prompt an interviewee for explanations

and adjust my questions based on the interviewee’s responses. The data gathered

through an interview is richer than data gathered through a predefined

questionnaire.

An interview allows me to understand the learner’s strengths and weaknesses,

things that may not become clear through other means such as observation. For

Collins 3

example, one learner may have more database knowledge while another learner is

a Microsoft Excel “guru”.

The Interviews, Their Analysis, and Results

The two people interviewed for this Needs Analysis document were two current NJIT

ADVANCE project team members who assist the PI but are not specifically trained to use the

database or perform data manipulation. Their responses will provide the basis for the design of

the WBT program to be developed. The following sections provide a sample of the interview

questions and the team members’ (TM) responses.

Question 1: What is your attitude towards the NJIT ADVANCE project?

TM1: I feel very strongly about the NJIT ADVANCE project. As a female engineering

student, I think studies focusing on advancing women in technology are extremely important.

TM2: The NJIT ADVANCE project is a great opportunity for me as an undergrad. I am

gaining valuable knowledge and will have hands-on experience to include on my resume when

looking for a job.

Question 2: What is the purpose of the coauthorship database?

TM1: The NJIT ADVANCE project is going to look at coauthorship at NJIT from a

Social Network Analysis point of view. When professors are trying to move up in rank, one of

the key factors looked at is their publication rate. So if female faculty are excluded from

coauthorship opportunities, they won’t advance as quickly and might therefore leave the

university environment. That’s why we’ve built the coauthorship database.

TM2: The ADVANCE team is using the coauthorship database to examine whether or

not female faculty at NJIT coauthor papers as frequently as their male peers.

Collins 4

Question 3: So what information does the coauthorship database give the team for Social

Network Analysis?

TM1: Based on all the faculty CVs and information pulled down from sites like Scopus,

we’ve built up a pretty accurate database of publications produced by NJIT faculty. The

database keeps track of which faculty members coauthor with other faculty members, as well as

with their grad students or outside collaborators. The ADVANCE team will look at the NJIT

coauthors and see if men coauthor more frequently than women, with a lot of different

permutations available through Social Network Analysis.

TM2: The coauthorship database keeps track of all publications produced by NJIT

faculty and it identifies faculty members who have coauthored together. The database also

contains the names of grad students and external authors, but our study isn’t really looking at

those people.

Question 4: The coauthorship database keeps track of attributes for each faculty member. Do

you know what those attributes are?

TM1: The attributes include the faculty member’s rank, their department, whether or not

they’re tenured, and their gender. I think that’s all of them.

TM2: I think the attributes include gender, obviously, because we are looking at whether

or not there’s a difference between male and female faculty. Other attributes are the department,

the rank of the faculty member, when they were hired, and whether or not they’re tenured.

Question 5: Do you feel these attributes are important to the coauthorship study?

TM1: Well, when we do the Social Network Analysis, we need to be able to look at the

faculty members by gender, and we also need to see their rank to know if they have gotten

promoted because of their publications. The department is there so we can see if any particular

Collins 5

department is better or worse towards women, and the tenure is there only because I think our

study is limited to only tenured or tenure-track faculty.

TM2: The gender is obviously the most important attribute. But we also need to know if

they’re tenured or tenure track because the ADVANCE study, at least right now, is limiting itself

to only those types of faculty members.

Question 6: When the coauthorship data is extracted from the database, it has to be

manipulated in Microsoft Excel. How familiar are you with Excel?

TM1: I’ve used Microsoft Excel for projects in my undergrad classes, so I would say that

I am somewhat familiar with it.

TM2: I’m not a Microsoft Excel expert, but I have used it and I feel comfortable with

most of the basic functions.

Question 7: Are you comfortable using formulas and functions in Excel?

TM1: I haven’t used all of the functions that Excel provides, but I can’t imagine that they

would be complicated to use. I am confident that I could figure them out.

TM2: I have used most of the string functions in Microsoft Excel, and I am confident

that I could figure out any functions I haven’t used.

Question 8: Based on what you currently know, do you think you would need to be trained to

perform the data manipulation for the NJIT ADVANCE project or could you figure it out

yourself?

TM1: Oh, I definitely need to be trained to do it. Just listening to the two people who

currently do the work makes it clear that, although the individual steps aren’t complicated, they

have to be performed properly or the SNA software won’t be able to use the data we generate. I

Collins 6

think there are a lot of little things that have to be done, and things that have to be checked, to

make sure the data is good and is properly formatted.

TM2: I don’t need to be trained to use Excel, but I need to learn the specific tasks

involved in the data manipulation. I know that the data has to be in a very specific format to be

imported in the SNA application, and I have no idea what that format is or how to achieve it.

Question 9: Would you be interested in learning the skills required to do the data

manipulation?

TM1: Yes, I would definitely be interested. I really support what the ADVANCE

project is trying to do and I want to be an active member of the team. So learning to work with

the database and do the data manipulation would definitely be something I would want to learn.

TM2: I would like to learn, but I would have to see how it fits into my schedule as I

already have a pretty full workload. If it didn’t take up too much of my time, I would do it.

Question 10: Have you ever had the opportunity to use Web-based Training?

TM1: I’ve used some tutorial lessons that were included in some of my software

packages, and for one of my classes we had a subscription to Lynda.com, which is a web-based

training website that offers training on a lot of the more popular software products like Adobe’s

DreamWeaver and PhotoShop.

Question 9a: Did you find the Lynda.com training helpful?

TM1: Well, not really. I was trying to learn to use DreamWeaver, and I was

starting a website from scratch, but the instructor would always open pre-existing pages

and show how to modify them. He never really showed how to start with a blank page

and build it up. He assumed we all had access to their sample files, but we didn’t. But

Collins 7

even if we did, he shouldn’t assume that we know how to create a blank page. I

eventually figured it out on my own.

TM2: I haven’t used web-based training myself but I have some friends who are distance

learners and they say that it’s pretty easy to use the software and follow the lessons.

Question 11: Would you be willing to learn and practice the skills on your own time via web-

based training?

TM1: I would actually prefer that because then I could do it when and where I want.

Sometimes I have time between classes and I sit in the student center, so I could work on it then,

or in the evenings when I have some down time.

TM2: I think web-based training would be better for me. Since my schedule is pretty

busy, it would be easier for me to learn and practice whenever I have a chance. It might take me

a little longer to get through it all, but I’d prefer to do it on my own schedule.

Question 12: Do you have the necessary hardware, connectivity, and software to complete

WBT from home?

TM1: Yes, being an NJIT student, I have a fairly powerful laptop, as well as a computer

at home. I have access to a VPN connection if necessary and my internet connectivity is good,

so I would not expect to have any problems completing the WBT at home.

TM2: I think all NJIT students are expected to have a pretty powerful laptop, so I could

definitely work on campus, and my internet connectivity at home is also pretty fast so I should be

able to work from home as well.

Collins 8

Question 13: Do you expect to have undisturbed time to complete the WBT?

TM1: I’m sure I’ll be able to steal 30 minutes here and there, but I doubt that I will have

extended periods of time where I won’t have to rush off to a class or answer the phone or do

homework or something else.

TM2: Being an undergrad, I don’t have a lot of undisturbed time. But I’m used to

working with interruptions. I would hope, though, that the training is broken down into sections

small enough that I can complete a section and practice and then run off and do something else.

Results and Analysis

The following table highlights the results of the interviews of the two NJIT ADVANCE

project team members.

Question Analysis of response

Question 1: What is your attitude towards

the NJIT ADVANCE project?

Determine the learner’s attitude towards the

project.

Result suggests that, because participation in the

ADVANCE project is voluntary, both team

members have a positive attitude towards the

project based on their responses and their

continued participation.

Question 2: What is the purpose of the

coauthorship database?

Spot gaps in knowledge about the project and

its purpose.

Result suggests that learners are familiar with the

purpose of the project.

Question 3: So what information does the

coauthorship database give the team for

Social Network Analysis?

Result suggests that learners understand the

value of the data collected in the coauthorship

database and why it is important to perform

Collins 9

Pinpoint existing knowledge about the

project and its importance.

Social network Analysis on the data.

Question 4: The coauthorship database

keeps track of attributes for each faculty

member. Do you know what those

attributes are?

Pinpoint existing knowledge about databases

and attributes.

Result suggests that learners understand what

attributes are and why they are included in the

study.

Question 5: Why are these attributes

important?

Spot gaps in knowledge about how attributes

affect the study.

Result suggests that learners understand why the

attributes are required for accurate Social

Network Analysis.

Question 6: When the coauthorship data is

extracted from the database, it has to be

manipulated in Microsoft Excel. How

familiar are you with Excel?

Detect prerequisite skills for WBT.

Result suggests that learners have at least some

familiarity with Microsoft Excel.

Question 7: Are you comfortable using

formulas and functions in Excel?

Detect prerequisite skills for WBT.

Result suggests that both learners feel

comfortable enough that, even if they have not

used a particular function, they could use it

without difficulty.

Question 8: Based on what you currently

know, do you think you would need to be

Result suggests that both learners agree that

training would be required for them to develop

Collins 10

trained to perform the data manipulation

for the NJIT ADVANCE project or could you

figure it out yourself?

Determines need for WBT.

the requisite skills to perform the data

manipulation process.

Question 9: Would you be interested in

learning the skills required to do the data

manipulation?

Determine attitude towards acquiring new

skills.

Result suggests that learners felt positively about

learning new skills for the ADVANCE project

because of their commitment to it.

Question 10: Have you ever had the

opportunity to use Web-based Training?

Determine attitude towards WBT.

Result suggests that the learners have differing

experiences with WBT. While neither has a

negative view, one learner had critical comments

about a particular training package, not the WBT

process itself.

Question 11: Would you be willing to learn

and practice the skills on your own time via

web-based training?

Determine attitude and commitment to

WBT.

Result suggests that learners are willing to

complete WBT from home on their own time,

and in fact, prefer the flexibility of that option

over a classroom setting or a scheduled time for

training.

Question 12: Do you have the necessary

hardware, connectivity, and software to

complete WBT from home?

Determine any hardware or software

Result suggests that, as NJIT students, learners

have access to the necessary hardware and

software to complete the WBT at a time and

location convenient for them.

Collins 11

constraints to take into account while

designing WBT.

Question 13: Do you expect to have

undisturbed time to complete the WBT?

Pinpoint student’s availability and

environment.

Result suggests that learners expect interruptions

and short periods of dedicated time. WBT

should be broken into manageable chunks with

frequent opportunities for practice.

Goal Statements for NJIT ADVANCE WBT Using Gagne’s Five Component Format

The following goal statements (formatted according to Gagne’s five component format),

describe the goals for the WBT being developed for the NJIT ADVANCE project.

Goal Statement 1

Given an introduction to the NJIT coauthorship database:

States the purpose of the coauthorship study and the data contained in the

database.

Discriminates between terms such as coauthors, isolates, and other key terms.

Situation The learner’s first stimulus in the course is the “Introduction

to the NJIT ADVANCE Coauthorship Study” section which

provides an overview of the coauthorship database and the

data it contains, as well as key terms used in the study.

Learned Capability

Verbs (LCV)

The learner can understand and explain the learned

capabilities required in completing the “Introduction to the

NJIT ADVANCE Coauthorship Study” section of the WBT.

These capabilities include the following:

Collins 12

Verbal skills – is able to read and comprehend the material in

the introduction and can state the purpose of the coauthorship

study and the data contained in the database.

Intellectual skills – is able to discriminate between coauthors,

isolates, attributes and other key terms.

Object The learner must become familiar with the NJIT ADVANCE

Coauthorship Study, the information contained in the

database, and the key terms used to describe the data.

Action Verbs The learner’s observable behavior will demonstrate

performance using action verbs such as read (the

introduction), understand (the coauthorship study and the

data contained in the database), comprehend (key

terminology), distinguish (between coauthors and isolates),

and complete (the section).

Tools, Constraints,

and Special

Conditions

The learner should gain her knowledge from the verbal

information in the Introduction section. The learner may not

be familiar with Social Network Analysis or the concept of

coauthorship and will have to understand these concepts.

Goal Statement 2

Given instruction via an audiovisual PowerPoint presentation with a sample Microsoft

Excel coauthorship file containing author names, contact IDs and number of joint publications:

Collins 13

Identifies the columns in the exported database file to be modified by selecting the

correct columns in Microsoft Excel.

Demonstrates an understanding of the rules governing the data modification process

by applying the correct formula or function to the appropriate column.

Adopts a rehearsal strategy to learn and recall the steps by practicing frequently.

Generates the proper data format by manipulating the appropriate columns based on

the learned rules.

Executes the proper steps in sequence to correctly modify the data.

Situation The learner’s stimulus in this section of the course is a

PowerPoint presentation with audio showing the steps

required to prepare the coauthorship file.

Learned Capability

Verbs (LCV)

The learner can understand the process of steps required to

properly prepare the coauthorship data for import into an SNA

application. These capabilities include the following:

Verbal skills – is able to state the proper format for coauthor

names and the required order of the data.

Intellectual skills – is able to identify the columns in the

exported database file and the modified file. Demonstrates

an understanding of the rules controlling the data modification

process, and generates the proper data format based on those

rules.

Cognitive Strategy – is able to adopt a rehearsal strategy to

help learn the steps and their sequence through frequent

Collins 14

practice.

Motor skills – is able to execute the steps in proper sequence.

Object The learner must learn the sequence of steps to prepare the

coauthorship data for import into the Social Network Analysis

software.

Action Verbs The learner’s observable behavior will demonstrate

performance using action verbs such as view (the PowerPoint

presentation), use (the steps described), recall (the steps

through rehearsal), practice (the steps to ensure accuracy),

receive (feedback on performance), revise (their learned steps

as necessary), and complete (the lesson once proficient in the

steps).

Tools, Constraints,

and Special

Conditions

The learner should gain her knowledge from the PowerPoint

presentation with audio. The information is learned

individually, at the learner’s own speed, and practice is

repeated until the steps become familiar to the learner.

Goal Statement 3

Given instruction via an audiovisual PowerPoint presentation with a sample Microsoft

Excel file containing faculty attributes:

Identifies the columns in the exported database file by selecting the columns to be

modified.

Collins 15

Demonstrates an understanding of the rules governing the data modification process

by applying the correct formula or function to the appropriate column.

Adopts a rehearsal strategy to learn and recall the steps by practicing frequently.

Generates the proper data format by manipulating the appropriate columns based on

the learned rules.

Executes the proper steps in sequence to correctly modify the data.

Situation The learner’s stimulus in this section of the course is a

PowerPoint presentation with audio showing the steps

required to prepare the attribute file for import into a Social

Network Analysis application. (Because the attribute file

requires extensive modification, the steps will be organized

into manageable modules in the actual WBT program.)

Learned Capability

Verbs (LCV)

The learner can understand the sequence of steps required to

properly prepare the columns in the attribute file for import

into the SNA application. These capabilities include the

following:

Verbal skills – is able to state the proper format for the

modified columns and the required order of the data.

Intellectual skills – is able to identify the various columns in

the exported database file and in the modified file.

Demonstrates an understanding of the rules controlling the

data modification process, and generates the proper data

Collins 16

format based on those rules.

Cognitive Strategy – is able to adopt a rehearsal strategy to

help learn the steps and their sequence through frequent

practice.

Motor skills – is able to execute the steps in proper sequence.

Object The learner must learn the sequence of the steps required to

prepare the attribute data for import into the Social Network

Analysis software.

Action Verbs The learner’s observable behavior will demonstrate

performance using action verbs such as view (the PowerPoint

presentation), use (the steps described), recall (the steps

through rehearsal), practice (the steps to ensure accuracy),

receive (feedback on performance), revise (their learned steps

as necessary), and complete (the lesson once proficient in the

steps).

Tools, Constraints,

and Special

Conditions

The learner should gain her knowledge from the PowerPoint

presentation with audio. The information is learned

individually, at the learner’s own speed, and practice is

repeated until all of the steps become familiar to the learner.

Goal Statement 4

Given two sample files, one for coauthorship data and one for faculty attributes:

Collins 17

States the objectives of the data manipulation by describing the desired format

of the modified files.

Identifies the various columns by applying the correct formulas to each.

Demonstrates the skills acquired by correctly manipulating the data.

Generates a correct file by applying the learned rules.

Executes the steps in the proper sequence by correctly completing the final

assessment.

Situation The learner’s stimuli in this section of the course are two

sample files, one with coauthorship data and one with

attribute data, in the format in which they are initially

exported from the coauthorship database.

Learned Capability

Verbs (LCV)

The learner can successfully complete all the tasks necessary

to modify the data in Microsoft Excel:

Verbal skills – is able to state the proper format for both

modified files.

Intellectual skills – is able to identify the columns that require

modification. Demonstrates newly acquired knowledge of

the rules controlling the data modification process, and

generates the proper data format based on those rules.

Motor skills – is able to execute the steps in proper sequence.

Object The learner must prove that she has learned the sequence of

steps to prepare both the coauthorship data and the attribute

data for import into the Social Network Analysis software.

Collins 18

Action Verbs The learner’s observable behavior will demonstrate

performance using action verbs such as answer (the

assessment questions), modify (the sample files), review

(their work to ensure accuracy), and submit (the final

assessment).

Tools, Constraints,

and Special

Conditions

The learner should work alone without the assistance of any

notes. Her assessment should be submitted in a timely

manner to the trainer for evaluation.

Audience Analysis

Because the NJIT ADVANCE project is funded by a grant given to NJIT from the

National Science Foundation (NSF), our prospective learners will meet the following criteria:

They will all be NJIT faculty or students. If they are students, they could be at the

undergraduate, graduate, or doctoral level.

Because they are all students at a technical university, they will have some level

of familiarity with computers, web-based applications, and commonly used

applications such as Microsoft Excel.

Similarly, they will have a positive attitude towards computers and should not be

intimidated by WBT.

Environment Analysis

Results of this study suggest the following about the learning conditions of our

prospective learners:

Collins 19

Our prospective learners are either students or faculty at a major university. This

means that interruptions during their training are likely to occur. A self-paced

WBT is clearly the best method for their environment.

Because the skills being taught in this WBT require extensive practice, learners

prefer an online environment where they can complete their practice from any

location at a time convenient for them.

Some of our prospective learners may be distance students (eLearning students)

of the university; WBT is particularly suitable for their needs, especially because

the software applications required are either commonly found on laptops and

home computers or can be freely downloaded.

Because all of our learners are affiliated with NJIT, we can be confident that their

computers have the disk space, bandwidth and throughput required to perform the

WBT successfully.

The WBT Method and its Justification

Based on my interviews and analyses, I have determined that a standalone W/CBT (Type

4) that includes audio, video, animation and judged interactions would be the best WBT method

for this project for the following reasons:

Standalone W/CBT allows the learner to advance at his or her own pace. Because

the learner is practicing a set of steps that must be fixed into motor skill memory,

the need for practice with judged interactions is critical. While some students

may pick up the skills more quickly, others may need to repeat the practices

numerous times in order for the process to become familiar.

Collins 20

Because the data manipulation process is highly structured, standalone W/CBT is

an appropriate choice for implementation. The steps being taught for data

manipulation are either right or wrong; there is no benefit to discussing the topic

or interacting with other students. A standalone W/CBT allows the learner to

focus on the skills being learned instead of what other students are doing.

Similarly, a learner who is moving at a slower pace may be frustrated by the

advancement of his or her classmates; a standalone W/CBT eliminates the

pressure of having to “compete” with other students.

As determined by the conducted interviews, our learners are students (or faculty

members) with hectic schedules and many demands on their time and attention.

A standalone W/CBT allows us to break the training into manageable modules,

each of which can be completed by the learners in 30 minutes or less. This allows

each student to work at a time and location convenient to them. Because practice

sessions are critical to the success of this training, standalone W/CBT can be

performed anywhere and at any time with little start-up time and little hardware;

all that the learner requires is a computer and the training software along with the

sample files provided.

The audiovisual capabilities of Type 4 W/CBT, as well as the ability to include

judged interactions, is critical to the effective delivery of the data manipulation

training. Learners must be able to hear the description of each step and view the

trainer performing the step so that they can see the initial format and the resulting

format after manipulation. The judged interactions will allow the learner to

Collins 21

practice and receive feedback regarding their accuracy in performing the data

manipulation steps.