2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose...
-
Upload
lindsey-ramsey -
Category
Documents
-
view
218 -
download
0
Transcript of 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose...
2008 FAEIS Annual Longitudinal Assessment
With a Comparison to the 2007 Survey Results
The purpose of the FAEIS annual evaluation is to develop longitudinal and trend analyses in order to assess progress and develop program components for the future.
Objectives
Describe selected demographic and other variables that have potential association with FAEIS use.
Determine the level of usefulness of FAEIS data resources and applications.
Determine the level of satisfaction with selected components of FAEIS.
Determine levels of adoption of FAEIS and factors that explain the levels of adoption.
Determine the relationship of selected demographic and usage variables (involvement, use, experience, primary job title) with perceived usefulness and satisfaction with FAEIS.
Methodology
2007- Panel of Experts - 7 FAEIS project staff, USDA representatives and chair of the FAEIS Panel
2007 - FAEIS Panel Review and Approval
2007 - Instrument Development 2008 - Population 1070 Users; 285
Respondents 2008 - 2 Follow Up Reminders
2008 Participants
Staff Faculty Department Head College Associate/Assistant Deans College Dean or VP University level administration “Government” added with recoding
Question 1 in 2008 My Involvement with FAEIS Includes: (select as many as apply)
(all possible combinations displayed; N=285)
Question 1 My Involvement with FAEIS Includes: (select as many as apply)
(all possible combinations displayed; N=285)
Involvement Type 2007Percent of Total
2008Percent of Total
Data Entry 54 53.3
User of FAEIS Products 8 9.1
Data Entry & Analytical 7 7.7
Analytical & User of FAEIS products
3 6.0
Analytical 6 5.6
Data entry & User of FAEIS products
5 5.6
None 3 2.8
Data entry, Analytical, & User of FAEIS products
4 2.5
No answer 2 2.5
Analytical, Product developer, & User of FAEIS products
0.003 2.1
Data entry & Product developer
3 1.1
Data entry, Analytical, Product developer, & User of FAEIS products
2 0.7
Data entry, Analytical & Product developer
2 0.4
Analytical & Product developer 0.003 0.4
Product developer 1 0.4
Product developer & User of FAEIS products
0.003 ---
Question 1Involvement with FAEIS of Survey Respondents(“none” and no answer responses eliminated)
2007 Percent of Total
(N=308)
2008 Percent of Total
(N=285)
Data Entry 81 75
User of FAEIS Products
21 27
Analytical 25 15
Product Developer
9 5
Analysis – in 2008 there were fewer data entry survey respondents, more users of FAEIS products, but fewer respondents indicated that they are analyzing and developing their own products, compared to 2007 survey respondents.
Question 2 How often have you used FAEIS in the past 12 months?
(N=285)
2008
Question 2 How often have you used FAEIS in the past 12 months?
2007 Percent of Total (N=308)
2008 Percent of Total
(N=285)
Not at all 26.3 27.4
1-2 times 47.4 44.9
3-4 times 16.2 16.5
More than 4 times
9.7 11.2
No answer 0.3 --
Analysis – slight increase in the more frequent use categories (3-4 times & >4 times) in 2008
Question 3 My total experience using FAEIS is?
(N=285)
2008
Question 3 My total experience using FAEIS is?
2007 Percent of Total (N=308)
2008 Percent of Total
(N=285)
More than 3 years
45 49.5
1 to 3 years 31 23.2
Less than 1 year
22 24.9
No answer 2 2.5
Analysis –greater number of “experienced” users (> 3 yrs) and greater number of “less experienced” users (<1 yr) in 2008 compared to 2007.
Question 4My professional position is best described as?
(N=285)
2008
Question 4My professional position is best described as?
2007 Percent of Total (N=308)
2008 Percent of Total (N=285)
Staff 49 46.7
Faculty 18 7.7
Department Head 6 19.3
College Associate/Assistant Dean
16 15.4
College Dean or VP 5 7.7
University level Administration
5 2.5
Government -- 0.7
No answer 1 --
Analysis – More administrators (non-staff/faculty) participated in the survey in 2008 (44.9%) vs 2007 (32%).
Question 5My level of adoption (use) of FAEIS is best
described as? (N=285)
2008
Question 5. “My level of adoption (use) of FAEIS is best described as:” (no answer – eliminated from analysis)
2007 Percent of Total (N=290)
2008 Percent of Total (N=285)
1=unaware 7.5 10.2
2=aware of it but do not use it 42.2 40.0
3=Use it but not sure of long term use
26.9 27.0
4=Use it and it is integral to my job 15.0 15.8
5=Integral to my job and expect long term use
6.1 6.7
6=So important as to never be without FAEIS
1.0 0.4
Mean Score 2.73 2.70
Question 5. “My level of adoption (use) of FAEIS is best described as:” (no answer – eliminated from analysis)
2007Mean
2008Mean
Data Entry Users Only
2.49(N=159)
2.30(N=152)
All Other Users 3.16(N=117)
3.24(N=125)
Total 2.78(N=276)
2.73(N=277)
1=unaware
2=aware of it but do not use it
3=Use it but not sure of long term use
4=Use it and it is integral to my job
5=Integral to my job and expect long term use
6=So important as to never be without FAEIS
Question 6Please rate the usefulness of the data resources in FAEIS (for
student enrollment, degrees awarded and placement, disciplines, gender, ethnicity, faculty salaries, etc.). (1 = not useful, 5 = very useful)
2008
Question 6Please rate the usefulness of the data resources in FAEIS
(for student enrollment, degrees awarded and placement, disciplines, gender, ethnicity, faculty salaries, etc.).
Score
2007 Percent of Total (N=192)
2008 Percent of Total (N=176)
1, Not useful 7 7
2 12 13
3 28 28
4 29 31
5, Very useful 24 21
Mean score 3.50 3.47
(“no answer” and “not used” responses eliminated)
Analysis – no significant changes between ‘07 & ‘08
Question 7 Please rate the usefulness of FAEIS at your institution for internal applications (institutional benchmarking, student and faculty recruitment, faculty hiring and
fundraising). (1 = not useful, 5 = very useful)
2008
Question 7 Please rate the usefulness of FAEIS at your institution for internal applications (institutional benchmarking, student and faculty recruitment,
faculty hiring and fundraising). (1 = not useful, 5 = very useful)
Score
2007 Percent of Total (N=160)
2008 Percent of Total (N=158)
1, Not useful 9 8
2 21 20
3 36 34
4 19 26
5, Very useful 15 12
Mean score 3.09 3.13
(“no answer” and “not used” responses eliminated)
Analysis – slight increase in higher ratings in 2008 (totals of 4,5 in ’07 = 34%, ’08 = 38%). Lower percentage of “not used” answer in 2008 (’07 = 43%; ’08 = 36%).
Question 8Describe your level of satisfaction with FAEIS components. (i.e., newsletter help desk,
data entry, report builder, instructions) 1=not satisfied; 5=very satisfied
2008
Question 8 Describe your level of satisfaction with FAEIS components. (i.e., newsletter help
desk, data entry, report builder, instructions) 1=not satisfied; 5=very satisfied
Score
2007 Percent of Total (N=222)
2008 Percent of Total (N=208)
1, Not satisfied 6 5
2 9 7
3 33 33
4 32 37
5, Very satisfied 20 18
Mean score 3.50 3.56
(“Unknown,” “no answer” and “not used” responses eliminated)
Analysis – slight increase in higher ratings in 2008 (totals of 4,5 in ’07 = 52%, ’08 = 55%). Similar percentage of “not used” answer (’07 = 23%; ’08 = 23.7%).
2008 Annual FAEIS Evaluation Did the type of involvement (Q1) have a significant relationship with ratings of: Frequency of use (Q2) Experience with FAEIS (Q3) Level of adoption (Q5) Usefulness of data (Q6) Usefulness of FAEIS (Q7) Satisfaction with FAEIS components (Q8)
Respondents listing “data entry” as their only level of involvement were compared to all other levels of involvement. “Not used” and “no answer” responses for Q6, Q7 and Q8 were removed from the analysis.
There were significant differences for “Frequency of Use (Q2),” “Level of adoption” (Q5) and “Usefulness of data” (Q6), and Usefulness of FAEIS to your institution (Q7) and “Satisfaction with FAEIS components” (Q8) between the two groups. There was not a significant difference between the means of the two groups for Q3. Respondents listing “data entry” as their only level of involvement had significantly lower means for Q2, Q5, Q6, Q7 and Q8 than the other group.
2008 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with: Adoption of FAEIS (Q5)
Yes, there is a positive correlation between “Frequency of use” (Q2) and “Adoption of FAEIS” (Q5). As the frequency of use increases, so does the rating of the adoption of the FAEIS.
F=45.0 (P<0.05)
Key:How often have you used FAEIS in the past 12 months?
1=Not at all2=1-2 times3=3-4 times4=more than 4 times
2008 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with: Usefulness of data (Q6)
Yes, there is a positive correlation between “Frequency of use” (Q2) and “Usefulness of data” (Q6). As the frequency of use increases, so does the rating of the usefulness of the data.
F=58.2 (P<0.05)
Key:How often have you used FAEIS in the past 12 months?
1=Not at all2=1-2 times3=3-4 times4=more than 4 times
2008 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with: Usefulness of FAEIS (Q7)
Yes, there is a positive correlation between “Frequency of use” (Q2) and “Usefulness of FAEIS” (Q7). As the frequency of use increases, so does the rating of the usefulness of FAEIS.
F=25.2; P<0.05
Key:How often have you used FAEIS in the past 12 months?
1=Not at all2=1-2 times3=3-4 times4=more than 4 times
2008 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with: Satisfaction with FAEIS components (Q8)
Yes, there is a positive correlation between “Frequency of use” (Q2) and “Satisfaction with FAEIS Components” (Q8). As the frequency of use increases, so does the rating of the satisfaction with FAEIS components.
F=26.1;P<0.05
Key:How often have you used FAEIS in the past 12 months?
1=Not at all2=1-2 times3=3-4 times4=more than 4 times
2008 Annual FAEIS Evaluation
Did the “years of experience with FAEIS” (Q3) have a significant relationship with ratings of: Level of adoption (Q5)
For Level of Adoption (Q5), the “Less than One Year of Experience with FAEIS” group was significantly lower than the other two “level of experience” groups. This seems to reason – the less experience with FAEIS, the less the level of adoption of FAEIS.
2008 Annual FAEIS Evaluation Did the “years of experience with FAEIS” (Q3) have a significant relationship with ratings of: Usefulness of data (Q6) Usefulness of FAEIS to your institution (Q7) Satisfaction with FAEIS components (Q8)
There was not a significant difference between the three groups for “Usefulness of data” (Q6), “Usefulness of FAEIS to their Institution” (Q7) and “Satisfaction of FAEIS Components” (Q8) .
2008 Annual FAEIS Evaluation Did the professional position (Q4) have a significant relationship with ratings of : Frequency of use (Q2) Years of experience with FAEIS (Q3) Level of adoption (Q5) Usefulness of data (Q6) Usefulness of FAEIS (Q7) Satisfaction with FAEIS components (Q8)
The only significant difference was with the “government” group for Frequency of use (Q2) and Level of adoption (Q5). This group had higher means than the other groups for both questions.
2008 Annual FAEIS Evaluation Did the level of adoption (Q5) have a significant relationship with ratings of: Usefulness of data (Q6) Usefulness of FAEIS (Q7) Satisfaction with FAEIS components (Q8)
Groups responding “unaware,” & “aware but do not use” to Q5 were compared to the other response groups for Q5 - “Use it, but not sure of long term use;” “Use it and it is integral to my job;” “Integral to my job and expect long term use;” & “So important as to never be without FAEIS.”
There were significant differences for all three of the questions above between the two groups. The means of the “unaware,” & “aware but do not use” groups were always lower for each of the questions when compared to the other group.
2008 Annual FAEIS Evaluation
Responses by non data entry users only 2007 Mean
2008 Mean
“My level of adoption (use) of FAEIS is best described as:” 3.16 3.24
SCALE:1=unaware2=aware of it but do not use it3=Use it but not sure of long term use4=Use it and it is integral to my job5=Integral to my job and expect long term use6=So important as to never be without FAEIS
2008 Annual FAEIS Evaluation
Responses by all users 2007 Mean
2008 Mean
Rate the usefulness of the data resources in FAEIS. Scale: 1=Not useful, 5=Very useful.
3.50 3.47
Rate the usefulness of FAEIS at your institution for internal applications. Scale: 1=Not useful, 5=Very useful.
3.09 3.13
Describe your level of satisfaction with FAEIS components. Scale: 1=Not satisfied, 5=Very satisfied.
3.50 3.56