Customer Satisfaction Survey

Bridge Design-Rating 2013Administrative OverviewBridge Design-Rating User Group MeetingVirginia Beach, VA Bridge Rating Licensees FY2013

2 Bridge Design Licensees FY2013

3FY2012 Bridge Design-Rating RevenueFY2013 Bridge Design-Rating RevenueFY2012 ExpendituresFY2013 ExpendituresAASHTO Administration & OverheadStaff salaries, benefits, and overheadContracted Project ManagerProportional share of SCOJD, T&AA and indirect costsLegal ServicesTechnical and Applications Architecture Task ForceTechnical resource for SCOJD and product task forcesDevelop and maintain software standards and perform QA ReviewsAASHTO Administrative Overhead8Incorporates best practicesUsers share solutions and costsLicense fees cover overall expenses ensure software products are kept current with technology and functional requirementsEach product is self-supportingNon-profit operationManagement and oversight by agency (DOT) personnelAASHTO staff project management/assistance

Why Use AASHTOWare?The AASHTOWare Program offers member agencies a number of unique advantages over custom or commercially available solutions including:

The incorporation of best practices; both thru the design and development of software based on AASHTO specifications and the fact that task force direction comes about through the formation of consensus

Because the program is operated on not-for-profit basis, our sole focus is on satisfying the memberships needs and priorities there is no other motivation for operating the program

license fees are set to cover expenses and products are supported, maintained and enhanced to keep products current with functional requirements and technology as long as their is

9

AASHTOWare Program Management

Board of Directors & Executive Committee Meet Twice a Year Establishes and revises policies governing the Cooperative Computer Development Program as called for within and consistent with this Administrative Resolution and AASHTO's Governing Documents.

Approves the creation of projects and products and the revenue and expenditure budgets of the activities within this Program as part of its overall budget approval role. Approves the appointment by the AASHTO President of the members of the Special Committee on Joint Development.

Special Committee on Joint Development provides management oversight of AASHTOWare Program & Products. The SCOJD studies all policy issues and topics with respect to the AASHTOWare program in consultation with all active product and project task forces and brings recommendations before the AASHTO Board of Directors and Executive Committee for action, as appropriate.

They review and approve all procedural documents, manuals and other guidelines pertaining to the process and management of the AASHTOWare program

Include in their charge is the approval of appointments [and removal] of Product and Project Task Force members as necessary and the designation of chairpersons. They review project proposals and authorize solicitation of membership for participation and funding for AASHTOWare projects; and, establish advisory and user groups when appropriate on the recommendation of the product and/or project task forces

A full-time AASHTOWare Project Manager is retained to coordinate product/project activities, and fulfill contract administration and project management functions.

As authorized by the executive director, the project manager leads/participates in contract negotiations and revisions; oversees invoices, payments, and other financial transactions; coordinates software development activities with other AASHTO committees; promotes awareness of the product capabilities and gathers potential enhancements for the products; and assists the task force in related matters as needed.

The Task Force provides the day to day management of the product. They guide software support and development services performed by contractors, determine the functionality to be included in enhancements; they oversee the development and execution of the annual work plans, including the associated budget, for the product and work with SCOJD to improve the overall AASHTOWare program.

The task force typically are comprised of 7 to 9 representatives from the agencies and serve on a rotating basis; they represent expertise in functional areas as well as IT. Included with their many duties, the task forces are responsible for (1) long range product direction, (2) annual consultant contracts, (3) quality assurance and control, and (4) responding to questions from users. The task force also determines the appropriate software platforms needed to meet the overall needs of the participating agencies and review and approve product releases. A typical member is a long-term or high-level administrator or user and has been active in the User Group for several years.

The Task Forces are oftentimes assisted by Technical Advisory Groups which provide subject matter expertise in defining functionality and software requirements and fulfill a variety of other roles; the TAGs also oftentimes perform detailed reviews of beta code and make recommendations to the task force on approval of deliverables and product releases.

The Technical Advisory Group(s)s are appointed by the Task Force and assist the Task Force in working with contractor in development of new product/enhancement(s). Their composition and the number of members setting on the TAG are based upon need; the members are users familiar with the product/functions and are usually from agencies contributing funding. Their duties/business is generally conducted by e-mail, conference calls, and [sometimes] face-to-face meeting as required; typically, the TAG is chaired by a Task Force member.

User Groups are independent organizations with their own Constitution and/or Bylaws. The User Group represents member departments interest in the product, and provides advice and recommendations on the products (1) effectiveness, (2) deficiencies, and (3) any needed product enhancements.

The User Group also recommends product training and support needs, details and recommends prioritized maintenance, enhancement, and support needs. The outcomes of the User Group are submitted to the Task Force as recommendations and helps for the basis for the upcoming fiscal years work plan.

105% of RevenuesGoverned and controlled by AASHTO Executive CommitteeCovers risks associate with software developmentSeed money for new projectsLegal expenses associated with patenting and third-party testingNational Transportation Marketing Campaign (Federal Transportation Bill)AASHTOWare Rebranding Effort

AASHTOWare Capitalization11AASHTOWare Branding and TradeMark Guidelines

Brand IdentityAASHTOWare Branding and Trademark Guidelines have been established to ensure the strength of our brand is maintained

Internal Communication Task Force Meeting discussionTask Force / Licensee EmailsSharePoint workspace folders and filesInternal presentations at Task Force and User Group MeetingsUser Group websites, etc.Brand IdentityExternal Communication communication to groups outside the AASHTOWare community, including other AASHTO committees, AASHTO member agencies and the publicPresentationsAdvertisementsProduct BrochuresProduct NewslettersAASHTOWare Website, etc.Brand Identity - NamingFull Name (External)AASHTOWare Bridge Design & RatingTMAbbreviated Name (Internal only)BrDR

Strictly speaking, a trademark should always be used as an adjective, never as a noun or verb; however, if the product name is used repeatedly, the full name should be presented every time, but the name may be used as a nounAASHTOWare Service Units

A Brief Overview

Agencies can gain convenient access to services provided by the AASHTOWare contractor via service units.AASHTO serves as facilitator by accepting the commitment for contractor-provided services, invoicing and receiving payment from the agency and forwarding the order to the contractor for the appropriate number of service units. AASHTO makes payment for services rendered to the contractor following agency approval of the invoice. Service units remaining at the conclusion of a fiscal year are carried forward into the next fiscal year. AASHTOWare Service Units19Service units are intended to provide consultation and support to incorporate functional enhancements or to assist the licensee in the implementation of AASHTOWare products. AASHTOWare Service Units20Service Unit work by the contractor may include the following types of activities:Adding new agency-specific features to the system Developing custom reports Providing specialized training in the use of AASHTOWare productsUpdating prior releases of product databasesService Unit Example Activities21Supporting common software enhancements unfunded through product licensing fees that will become part of the code base and will be supported by Maintenance, Support and Enhancement (MSE) costsIncorporating analytical or specification engines into AASHTOWare productsFunding software development projects / solicitationsService Unit Example Activities22The example activities outlined previously may require more than one Service Unit each, depending on the specific agency requirements. Service Units may not be used to provide reimbursement for travel expenses by agency personnel.Service Units should not be used for work involving major new software development by member agencies. Service Units may be converted to provide additional enhancement funding under the guidance of the Task Force.

Use of Service UnitsService Units can be ordered in unit increments of $11,600 (this fee includes AASHTO administrative costs).Service Units must be paid upon receipt of the invoice. Each service unit provides $10,000 in routine contractor services.

Fee for Service UnitsService Units Use86.2the percentage of the Bridge Products Service Unit fee directly allocated to the software service provider 8.8the percentage of the Bridge Products Service Unit fee used to offset AASHTO internal administrative costsstaff salaries, benefits, and overheadcontracted project managerproportional share of SCOJD, T&AA and indirect costslegal services5.0the percentage of the Bridge Products Service Unit fee dedicated to support the Cooperative Development Capitalization Fund as required by governing policy approved by the Board of Directorscovers risks associate with software developmentprovides seed money for new projectsfunds expenses associated with patenting and third-party testingsupports product branding / marketing initiativesService Unit ProcessPartnership between requesting agency, Task Force and contractor.Task Force approval to ensure contractor resources are available.Analyze opportunities for collaboration between agencies and Task Force product work plans.

262013 Bridge Design-RatingCustomer Satisfaction Survey ResultsConducted June 25 July 26, 2013Survey ParticipationTwo survey instruments were publishedAASHTO Member Agencies (State Agencies, Counties, Cities)Consultants64 responses (48 in 2012)41 member agencies - state (33 in 2012) 2 member agencies - county 1 member agencies - city20 consultants (15 in 2012) 6 agency sponsored license13 special consultant option license 1 single workstation license

Software UsedBridge DesignBridge RatingBothMember Agency02223Consultant0736.46.36.2Member Agency3663Consultant1901Respondent RoleYesNoNot SureMember Agency3257Consultant1703Designated End User?Active User of the Software?YesNoMember Agency396Consultant182Operation (Speed, Reliability)Reports (Quality/Completeness)Program Features/CapabilitiesAnalysis ProvidedSoftware Use Comments Output / ReportsMore customizable reports ability to build a custom report for LRFR analysisAdditional output options and detailOutput in the format produced by the BRASS engine BRASS excelled at packaging the data into an easily accessible fileLRD Overall Summary Analysis Report page breaks to print all information for a particular member on the same pageInclusion of figures/definitions in reportsDefault report that includes table of contents to detailed information contained within

Software Use Comments (cont)Improve speed of analysis - time measured in days and hours instead of minutesSignificant resources are required to rate some bridges non-standard gauge trucks on continuous multi-span structure would not run (even with accuracy reduced to minimum)Develop a module in BrR to support permittingEnsure consistency of results between versions inconsistent rating factors are encountered from version to version

Software Use Comments (cont)Bridge Workspace is cumbersome to navigate and difficult to understand A large amount of meaningless, trivial and/or minor warnings are displayed during the save and analysis processesLimited graphics (typical section, girder profile, etc.) Recommend a complete redesign of the user interface, development of quality documentation and separation of the analysis program from the database.

Member Agency use of support from the contractor - 65%Extremely satisfiedModerately satisfiedNeither satisfied nor dissatisfiedModerately dissatisfiedExtremely dissatisfieda) quality of the support provided 31.3%46.9%12.5%9.4%0.0%b) contractor communication and follow-up31.3%43.8%18.8%6.3%0.0%c) effectiveness of contractor telephone & e-mail support31.0%44.8%20.7%3.4%0.0%d) knowledge of the contractor help desk staff38.7%48.4%12.9%0.0%0.0%e) overall quality of contractor problem resolution36.7%43.3%16.7%3.3%0.0%Contractor Support Comments Interactions with the contractor have generally been positiveGeneral help desk support has been satisfactoryDo not always receive an email when a problem is resolved or additional information is requestedInclude option for telephone support, in addition to email contactSome frustration with incidents being marked as enhancements sometimes incident status changes are not received

Contractor Support Comments (continued) Contractor is very good at resolving problems during beta testing; however, they are less responsive after the software is released to production[from a beta tester] In some cases there is little or no communication on reported incidents a more formal process is needed to ensure beta testers know when to follow up on issues and when to wait for resolutionDocumentation UsedDocumentation UsabilityDocumentation CompletenessDocumentation Comments The examples on the website are helpfulAdditional installation documentation is neededManual is not user friendlyInternal help documents have missing itemsHave not received a user manual since version 5.5.1 must reply solely on internal helpNeed detailed documentation on NSG & 3-D featuresWhy is access to documentation and help materials password protected?

Documentation Comments (cont) Example Bridges AASHTOWare BrDR Users Manual should reference the support website link for tutorials/trainingWebsite tutorials/training should be updated with each new version of the softwareUser Group training materials on the website should be posted separately rather than as one large ZIP fileInternal help contains important information that the user is unaware of unless they specifically review need an alert system

Member Agency contact with Bridge Task ForceResponsiveness of Bridge Task ForceTask Force Improvement SuggestionsBridge Task Force member contact information should be prominently displayed on the main web pageEstablish a centralized discussion site to help each other rather than taking time from contractor productionCirculate Bridge Task Force meeting summaries via a listserveDevelop a newsletterSet aside space in the annual work plan for agency service unit work with agency commitment established at the time the work plan is developed

Communication Between User Group and Bridge Task ForceTask Force / User Group Improvement SuggestionsProvide Bridge Task Force Meeting Summaries to the User Group members in a more timely manner

Bridge Task Force and User Group officers are doing a great job

General CommentsBetter Communication on long term plans for software underfunded enhancements would be good candidates for service unit pooled funds between agenciesLRFD analysis of trusses would be a great additionOnline webinars would be very useful to allow users to take training without travelThe FAQ on Virtis/Opis Technical Support web site does not work some input is repetitive, other input should be locked out when not required

General Comments (continued)Error ignoring positive moment calculations in continuous prestressed concrete girder bridgesBrR does not give good results for culverts with more than two cellsRuntime Error Tracking points to a subroutine, would like it to provide information that is meaningful to the end userCulvert rating module corner reinforcement location is backwardsMore effective technical supportYearly licensing process is too lengthy

Specific Questions AskedProblem with inputting a new bridge drag & drop a span in Superstructure Definition locks up when trying to save it unless he saves, gets out, and opens it again

Follow-up ActionsAASHTOWare Bridge Task Force Meetings (August 8 and November 5-6, 2013)Review the detailed results of the surveyDiscuss opportunities for improvementAssign action items to implement changesIncorporate changes into FY15 work plan as appropriateSpecial Committee on Joint Development (January 23-24, 2014)Bridge Design-Rating survey results presented/discussed

Thank YouQuestions?Comments?Sheet1City/County/TerritoryPhoenix, AZWashington, D.C.Puerto RicoMap KeyCountryConsultant Licensees = 460Non- LicenseeManitoba, CanadaAgency Licenses = 33Licensee

Sheet2

Sheet3

Sheet1Map KeyCountryConsultant Licensees = 20Non- LicenseeManitoba, CanadaAgency Licenses = 21Licensee

Sheet2

Sheet3

NameTitle

NameTitle

NameTitle

NameTitle

NameTitle

NameTitle

Team TitleNameTitle

Company NameCompany NameDepartment NameAASHTOBoard of Directors

Executive Committee

Special CommitteeonJoint Development

Technical and Applications Architecture Task Force

ProjectTask Forces

ProductTask Forces

TRTs and TAGs

TRTs, TAGsandUser Groups

Executive DirectorandStaff