At the ITSS Help Desk we support the user authentication system called SJSUOne. In September 2004 we had around 600 people come to our help desk for help in using the interface of this system to activate their SJSUOne accounts. This was out of a base of 4,000 users. About 12% of users came to us for help with this web application. In September 2005 we are planning to use this same web application to create user accounts for a projected 12,000 users of the new campus wide wireless network, due to go live in 60 days. With the projected roll-out of this, it was decided we should do usability testing.
Eleven subjects were tested by our help desk in an unscientific usability test of SJSUOne. These subjects included staff and students (no faculty) and ranged in majors from electrical engineering to nursing. Five of the eleven test subjects were unable to complete the task of activating their SJSUOne account using the existing user interface without help. These five were rated as certain to have sought assistance from the help desk. A sixth frustrated test subject was able to do it, but was rated as likely to have called the help desk.
When concern about the system was expressed to one manager the manager said, "maybe it won't be all that bad and we will be pleasantly surprised."
Yes, perhaps that is true!