Software/Tools For Running Software/Tools For Running Web ExperimentsWeb Experiments
ByBy
Robert SlaterRobert Slater
Texas A&M -Corpus ChristiTexas A&M -Corpus Christi
ConsiderationsConsiderations
SkillsSkills TechnologyTechnology Data PrivacyData Privacy FeaturesFeatures
SkillsSkills
What is your proficiency?What is your proficiency?– HTML, ASP, VB, JAVA, NONEHTML, ASP, VB, JAVA, NONE
Hire external partyHire external party– Do they understand what you are doing?Do they understand what you are doing?– How important is their friendship?How important is their friendship?
Learn new skills?Learn new skills?– learning curve.learning curve.
TechnologyTechnology
What type of computer systems will access What type of computer systems will access the software?the software?– Windows XP, ME, 2000, 98, 95, 3.1, DOS, MACWindows XP, ME, 2000, 98, 95, 3.1, DOS, MAC
Will the code you have work on the different Will the code you have work on the different operating systems or web browsers?operating systems or web browsers?
Do you have a web-server you can use to Do you have a web-server you can use to host your study?host your study?
Would a third party notify you in advance of Would a third party notify you in advance of server maintenance?server maintenance?
Data PrivacyData Privacy
Where will the data be stored and who will Where will the data be stored and who will have access to it? have access to it?
If you use a third party hosting site can you If you use a third party hosting site can you keep your data confidential?keep your data confidential?
Can you control backup copies of your Can you control backup copies of your data?data?
FeaturesFeatures
What kind of features do you need in your study? What kind of features do you need in your study? I needed:I needed:
– Randomization into treatment groupsRandomization into treatment groups– Piping with mathematical manipulationPiping with mathematical manipulation– Multiple Question formatsMultiple Question formats– Data ExportData Export– CustomizationCustomization– User sign in options (specific/anonymous)User sign in options (specific/anonymous)– Java script capability within survey pagesJava script capability within survey pages
My SolutionMy Solution
Adapted an open source survey software to meet Adapted an open source survey software to meet experimental needs (www.ClassApps.com)experimental needs (www.ClassApps.com)
ASP based open source softwareASP based open source software Multiple database support – Multiple database support –
– Supports data export to excelSupports data export to excel
Professional looking finished productProfessional looking finished product No programming experience neededNo programming experience needed Quick learning curveQuick learning curve
SelectSurveyASPSelectSurveyASP
SelectSurveyASPSelectSurveyASP
Finished ProductFinished Product
Survey OptionsSurvey Options
Export OptionsExport Options
Limitations and SolutionsLimitations and Solutions
Software not designed for multiple Software not designed for multiple treatments:treatments:– Create multiple questionnaires based on Create multiple questionnaires based on
treatment groups treatment groups Created one master survey and copied it to create Created one master survey and copied it to create
separate treatmentsseparate treatments
Software would not randomize subjects:Software would not randomize subjects:– Use separate web page and basic java script to Use separate web page and basic java script to
randomize subjects into treatment groupsrandomize subjects into treatment groups Software allowed hyperlinks to surveysSoftware allowed hyperlinks to surveys
Multiple TreatmentsMultiple Treatments
Limitations and Solutions (cont)Limitations and Solutions (cont)
Software allowed for piping but not Software allowed for piping but not mathematical manipulation as needed in my mathematical manipulation as needed in my study. study.
I needed to be able to:I needed to be able to:– Take users estimate of Earnings Per ShareTake users estimate of Earnings Per Share– Give them management’s estimate that was Give them management’s estimate that was
calculated as a percentage of user’s response.calculated as a percentage of user’s response.
PipingPiping
Take data input by participants and use in Take data input by participants and use in the study.the study.– Personalize questionsPersonalize questions– To develop rankings using data input by users.To develop rankings using data input by users.
Taking User InputTaking User Input
Using the InputUsing the Input
Capture Input for RankingCapture Input for Ranking
Ranking The Captured InputRanking The Captured Input
Piping SolutionPiping Solution
Program already allowed for piped Program already allowed for piped responses.responses.
Open source software allowed me to Open source software allowed me to manipulate the ASP source code to perform manipulate the ASP source code to perform mathematical function on piped response. mathematical function on piped response.
Users Input EPSUsers Input EPS
Design ScreenDesign Screen
Pipe InstructionsPipe Instructions
Actual Question TextActual Question Text
User View Piped ResponseUser View Piped Response
ConclusionsConclusions
Quick learning curve. Little expertise necessary.Quick learning curve. Little expertise necessary. Customizable software allowed for unique Customizable software allowed for unique
functionality.functionality. Locally housed database on web server allowed Locally housed database on web server allowed
for data privacy and further customized data for data privacy and further customized data gathering. (Timed responses)gathering. (Timed responses)
Customized programming problems led to Customized programming problems led to incompatibility with some operating systems. incompatibility with some operating systems. – Participants were dropped from the study before Participants were dropped from the study before
completion. completion.
Other Commercial SoftwareOther Commercial SoftwareZoomerang™ SuperSurvey Greenfield Online
Infopoll Perseus PollCat
Inquisite Cool Surveys Survey System
Apian Software Hosted Survey SurveyView
StatPac SurveyGold SurveySite
InstantSurvey EZSurvey Mercator
SurveyCrafter PollPro LiveSurveys
InSite ObjectPlanet SurveySaid
QuickSenseActive Websurvey SumQuest
CustomerSat StatSurvey SurveyMonkey
*(list of survey companies found on www.surveymonkey.com/Pricing.asp)
Recent Academic Studies on Web Recent Academic Studies on Web ExperimentsExperiments
Bryant, B., Hunton, J., and Stone, D., 2004, Bryant, B., Hunton, J., and Stone, D., 2004, “Web Experiments: Prospects and “Web Experiments: Prospects and Possibilities for Behavioral Accounting Possibilities for Behavioral Accounting Research,” Behavioral Research In Research,” Behavioral Research In Accounting, 16: 1-24Accounting, 16: 1-24
Birnbaum, M., 2004, “Human Research and Birnbaum, M., 2004, “Human Research and Data Collection via The Internet,” Annual Data Collection via The Internet,” Annual Review of Psychology, 55: 803-32Review of Psychology, 55: 803-32
Top Related