Poor usability of a product can be very expensive, in terms of time, money, and credibility. Many product designers are unprepared to perform the tasks associated with usability assessments. Usability and user-centered design principles have had a difficult time becoming embedded in the product development life cycle. Project managers are somewhat receptive to the concept but have little data on how to plan for usability tasks. This overview and the documents contained in the package provide basic information on the process of usability testing and how it is applied within the Training and Documentation group of CIT.

The principles of User-Centered Design (UCD) as presented by Woodson, Wesley E. (Human Factors Design Handbook: Information and Guidelines for the Design of Systems, Facilities, Equipment, and Products for Human Use (New York: McGraw-Hill, 1981) provide the theoretical foundation for our process. Jeffrey Rubin’s Handbook of Usability Testing and Carol M. Barnum’s Usability Testing and Research provide the fieldbooks for Training & Documentation’s usability work.

Woodson described the goal of usability as “…the practice of designing products so that users can perform required use, operation, service, and supportive tasks with a minimum of stress and maximum efficiency."  Usability determines whether programs, products, or services are as easy to use as possible. It is significantly less expensive to correct usage flaws in the design stage than after deployment of the product or service.

Rubin and Barnum, as do others in the field, identify four types of usability tests: exploratory, assessment, validation and comparison. Training & Documentation typically focuses on assessment and validation testing.

1) Exploratory – Normally Marketing has a large role here as they usually identify the user profile and usage model. Training & Documentation expertise has evolved here to do the observation and testing of the model in the field.

2) Assessment – The product at this point is still in the design stage and Training & Documentation is running detailed task analysis by job function/skill to assess the usefulness of the design for the end user. Usually, the feedback and reported findings go to Marketing who is working with the developers. However, the report may go directly to the Project Manager.

3) Validation – Marketing has taken in all of the Training & Documentation findings along with their own findings at this time. Training & Documentation is involved at this stage to validate that the final designs are working as desired. This is the last chance to catch anything major before release date.

4) Comparison – This is usually Marketing driven, and may even be completely done by the Marketing group. The objective is to gauge the competition or how well a new version or new service or new feature stacks up to a specific target audience or a new target audience.

Training & Documentation is very active in the Usability areas of Assessment and Validation. During an Exploratory assessment, Training & Documentation functions as a support group for Marketing test and business objectives. In the Comparison assessment, Marketing is driving the plan and conducting the usability project.
 
It is important to note that Usability testing is not Quality Assurance testing. Although Usability tests may identify quality issues, the work plan for the Usability project is not designed to focus on Quality assurance testing objectives. 

Our starting point for a Usability testing project is the work breakdown structure, (Usability WBS.ver3).  With the following sample forms, we help refine our project plan:

Sample Task Scenarios
Sample Orientation Script
Sample Pre-test Questions
Sample Post-test Questions
Sample Background Questions
Sample Debriefing Questions

The above were used in the following actual projects:

NetID Distribution Test Plan1.3
NetID screen shots
NetID Distribution Report1.3

This report, ATA Findings and Recommendations Report, used an in-house review technique to conduct a limited usability test.

Although usability testing should be taken seriously to affect the final production release of a product or service, we should not take ourselves too seriously. In other words, usability testing as a discipline serves the User not the usability specialist.

We take a light-hearted approach to the serious subject of usability testing in the following 6 minute video. Our goal was to inform and entertain. Enjoy. 

Quicktime Version (79 MB)
Windows Media File (23 MB)
Audio only

Sebastian Carello
Assistant Director, CIT/CSM Training and Documentation Services
Computer Communications Center B06
Cornell University
Ithaca, NY 14853
607-255-9726
CIT values...
Respect...Community...Integrity...Innovation...Results