Usability Test

The Purpose of Document

The purpose of this experiment is to test the effectiveness and efficiency of the current ported OpenACS calendar and from the result of the test, to find out what improvement can be made to the calendar. In other words, this usability test is performed to improve the quality of the current calendar application.

About the OpenACS Calendar

The targeting web application for this usability test is the calendar from OpenACS, which can be accessed from http://ebt.ee.usyd.edu.au:9013. This is an online calendar, which can be personalized and can be easily accessed with any web browsers, that allows one to enter and keep track of anything that could reasonably be written on an ordinary paper calendar. It also provides different ways of viewing the calendar information; for example, day view, week view, month view and year view.

This package could be used with any application where tracking items in time is important.

There are a number of features, which any application must bring to the table to be reasonably considered a complete calendaring application. The OpenACS calendar package currently supports all but one of these features:

· Multiple different ways of viewing the calendar. At a minimum this must include day, week, month, and year views. The OpenACS calendar also includes two list views.

· A rich set of navigation features for easily selecting and changing both the views and the time spans viewed.

· The ability to easily add, edit and delete items.

· The ability to easily set up recurring items.

The only feature that is missing is the alarm and reminder function.

User Profile

The majority of the user population consists of those who use Internet frequently, as they are comfortable with the idea of using web-based application. The OpenACS calendar is designed to target primarily those who, to oneself, tracking items in time is important. This would include most business settings for meetings and appointments, educational settings, clubs, friends’ birthdays and other community organizations; and those who want to please and attract customers or Internet users by including a calendar as an extra functionality for their web site.

Users often use the OpenACS calendar to put down a small note for their schedule for the day, week, month or year, or to view or edit what they have already entered into the calendar. OpenACS calendar users often want to open up the calendar, use the least amount of time and in their current working environment, put down the events or activities to be done on a particular date and time, or to recall the upcoming events which they have pre-entered.

Thus using information obtained from the above analysis, the following description-user matrix is formed:

	Primary User	Secondary User
Main Occupation	Business people / Students	Website developers
Age Group	18 – 50	21 – 45
Aim	To enter and keep track of events or items in the personalized online calendar quickly and easily.	To add the calendar as an extra functionality on their website to attract customers.
Time spend on site	Short – Medium	Medium – Long
Computer/Internet skill	Medium – High	Medium – High
Main thing they look for	A user-friendly private online calendar.	Easy to install and ready to use.
Most common task	Add and view events.	Install the calendar software and make the calendar software available to use by customers.

Table 6 Description-user matrix.

Task Profile

Three tasks are chosen for the purpose of this usability test. The tasks are as follows:

1. Adding a new event:

Use the “add an event” function to add an event at a specific date and time (Table 7).

2. Retrieve an existing event:Edit and Delete an event:

Click on an event title to retrieve a pre-entered event (Table 8).

3. Edit an existing retrieved event:

Use the drop-down lists to adjust / edit event time and date. Modify event title and description using keyboard (Table 9).

4. Delete an existing retrieved event:

Click on the Delete button to delete an event (Table 10).

The task matrix of the three tasks are shown below:

Task Name	Add a New Event
Goal	Add an event on the calendar at a specific date and time.
Precondition	The user needs to realize where the Mini Calendar is.
Steps	Method 1 - Click on “add an event”, - Type in the title of the event, - Adjust the date from the drop down list (if appropriate), - Adjust the Start time and the End time (if appropriate), - Enter the description of the event, - Select calendar type (if appropriate) - Click on the “Submit Query” button. Method 2 - Click on a specific date on the Mini Calendar, - Click on “add an event” and follow method 1 above, OR - Click on a specific time on Today’s Schedule, - Type in the title of the event, - Adjust the date from the drop down list (if appropriate) - Adjust the Start time and the End time (if appropriate), - Enter the description of the event, - Select the calendar type (if appropriate), - Click on the “Submit Query” button.

Table 7 Task profile for Add an event. The table shows 2 different methods to add an event into the calendar.

Task Name	Open an existing event
Goal	The specific titled pre-entered calendar event is opened and the user can view the event.
Precondition	The user needs to realize where the Mini Calendar and the Today’s Schedule are. Calendar event/s had been added prior to this task.
Steps	Method 1 – day view - Click on Day on the Mini Calendar, - On the Mini Calendar, click on a specific date, - Click on the desired event title on the Today’s Schedule. - View your entry. Method 2 – week view - Click on Week on the Mini Calendar, - On the Mini Calendar, click on a specific week, - Click on the desired event title on the Today’s Schedule. - View your entry. Method 3 – month view - Click on Month on the Mini Calendar, - On the Mini Calendar, click on a specific month, - Click on the desired event title on the month-view calendar. - View your entry. Method 4 – list - Click on a specific date on the Mini Calendar, - On the Mini Calendar, click on List, - Click on the desired event title from the list. - View your entry.

Table 8 Task profile for Retrieve a pre-entered event. The table shows 4 different methods to retrieve a pre-entered entry from the calendar.

Task Definition	Edit a private calendar event
Goal	To change the information of an existing calendar event.
Precondition	The specific calendar event is present and has been opened (see Table 1.3).
Steps	- Have the calendar event opened, - Change the title (if necessary), - Change the date and time settings (if necessary), - Change the description (if necessary), - Press the Submit Query button.

(a)

Task Definition	Edit a public calendar event
Goal	To change the information of an existing calendar event.
Precondition	The specific calendar event is present and has been opened. Must have permission from administrator.
Steps	- Follow the steps in Table 1.4(a).

(b)

Table 9 Task profile for editing an opened pre-entered calendar event (a) for private calendar, (b) for public calendar.

Task Definition	Delete a private calendar event
Goal	To delete a pre-entered event from the calendar by its title.
Precondition	The specific titled calendar event is present and has been opened (see Table 1.3).
Steps	- Have the calendar event opened, - Press the Delete button.

(a)

Task Definition	Edit a public calendar event
Goal	To change the information of a pre-entered calendar event.
Precondition	The specific calendar event is present and has been opened. Must have permission from administrator.
Steps	- Follow the steps in Table 1.4(a).

(b)

Table 10 Task profile for deleting an opened pre-entered calendar event (a) for private calendar, (b) for public calendar.

Test and Evaluation Plan

The main goal of this usability test is to improve the way user can use the OpenACS calendar to access an event (this includes add, retrieve, edit and delete) in the shortest amount of time and complete their time management using the calendar without doubt and difficulties. This will eliminate the possibility that the user gives up the use of online calendar because the procedure is too complicated.

The critical, typical and potentially troublesome tasks to be chosen for evaluation will be the task of adding a new event, retrieving and editing an existing event, and deleting an existing event completing the calendar event management cycle.

They are defined as critical and typical tasks as they are the core functionality of a calendar, which are also designed to be the most frequently used functions. They are potentially troublesome tasks due to the fact that managing your daily, weekly or monthly schedule can become quite difficult when one have too many events happening on the same day, week or month, and user might possibly give up in the middle if they find it too complicated.

The main benchmarks selected for this evaluation are as follows:

1. Number of mouse clicks used in each step

2. Amount of time spent in each step

3. Number of pages visited in each step

The above benchmarks are quantified usability measures. These can reflect the efficiency and the difficulty or the ease of using the calendar application.

This test is designed to be carried out as soon as the implementation and testing cycle has completed, and requires one day evaluation and one day for analyzing the statistics of the outcome of the evaluation test, and produce a report containing the facts that is derived from the analysis. This report will then be sent to the development team to assist them to improve their design. A need for holding a meeting between the developing team and the evaluation team might be possible to discuss on the way the website can be improved.

To carry out this usability test, 3 users and 1 tester will be required. The duration of each test is approximately 30 minutes and the tester is required for recording the statistics of the users as they carry out the test and preparation of test equipment and materials.

As soon as the result of the evaluation test formed, a meeting should immediately be held by the evaluation team and the development team to make decisions as to which of the corrections should be prioritized. Depending on the result, the amount of functionality within a page can either be increased or decreased; the style of the interface can be changed.

The Usability Test Materials and Results

About the User:

Please collect the following information from the user:

User No:         o 1                 o 2                 o 3

Gender:           o Male                       o Female

Occupation:     _______________________

Age Group:     o 15-20     o 21-25     o 26-30     o 31-41     o 41-55     o 56+

How often do you use computer?

o   Daily      o   Weekly      o   Monthly      o    Every 3 months     o    Never used a computer

How often do you use the internet?

o   Daily      o   Weekly      o   Monthly      o    Every 3 months      o    Never used a computer

Have you used an electronic or online calendar before?

o   YES                       o    NO                       o   Never heard of it

Grouping all the above derived requirements and analytical outcomes, the following test mateial is produced:

Figure1 OpenACS calendar Usability Test material

The usability test (see Appendix C) for the OpenACS calendar was conducted on 6^th October 2002. Results from the usability test reflects the following need for improvements:

· Allow event adding by clicking on a particular date on the month-view calendar

· Add a function that will provide easier and faster date access. This can be three drop-down lists that allow users to select the day, month and year of the specific date and a GO button to retrieve the date

· Enable the view of mini calendars of each month on the year-view calendar

· Allow events on the same day to display in start time ordering in the calendar

· Provide instructions and help on calendar usage

· Highlight on the mini calendar the dates that has an event

· Provide distinction between events of different calendars in use

· Bullet points to separate out different events in the month-view calendar