UMass Lowell Dept. of Computer Science
COMP 4620 — GUI Programming II
Spring 2016 Semester, Section 201
Prof. Jesse M. Heines
Assignment No. 3
Project Milestone #3: Beta Version and Usability Test
Date Due:
Thursday, April 14, 2016
Contents
Top
What This Assignment Is About
Top
As your project development nears completion, you need to test your project with actual representatives of your target user population. You probably still have programming that you want or need to do, but it is time to step back a bit and see where you are and what you need to change before you finish up. You will do this by testing your user interface as you did for your alpha version, but more formally. Most importantly, this time you will invite people who are not members of our class to be your test subjects. I realize that it might be difficult or even impossible for some of you to have representatives of your actual target user population come to class, and we will discuss ways to address that situation in class.
A usability test involves “trying a product or service out on prospective
users to see what problems they have in learning and using it” (Jeff Johnson,
in GUI Bloopers: Don’ts and Do’s for Software Developers and Web
Designers). In this assignment, you will conduct a usability test of
your beta version to identify problems. You will
then write a brief report on what you found out, both positive and negative.
What You Are To Do
Top
Your goal for this assignment is to collect data so that you can write a report.
The beta version you create needs to be “ready for user testing.” All elements should be integrated so that they work together. All planned functionality
should either be implemented or removed. There should be no “dead”
menu items or buttons.
Documents Needed for the Usability Tests
- Instructions to be read to the test subject
- Tasks to be performed by the subject on a form that you will hand to the subject for him or her to follow during the usability test
- A form to be used by the evaluator (yourself) to record what happened during the test
- Questions you will ask of the subject after the test
Before the Tests
- If you have time, it is nice to add “hooks” to your software to
capture users’ actions
- even a simple time-stamped log is highly useful in reconstructing what
happened
- this step is optional for this assignment
- Test the software thoroughly yourself and with the other member(s) of your
project team
- be critical — help each other — work slowly and deliberately — look
at everything
- don’t be shy about making suggestions for improvement
- little changes can have big effects
- Identify potential users who are available to test your software
- schedule them at reasonable intervals
- don’t tell them about your software, let them discover things for
themselves
- just say that you’re asking them to test a program you wrote
for a class
- Write down the directions you will give these users when they come to take
the test
- even if you’re going to give the directions verbally, write them
dosn so that everyone gets the same directions
- Write down the things you will want users to do
- this list may be given to the testers when they come so that they
can refer to it during the test
- Make sure you have everything you need to take notes during the test
During the Tests
- Stick to your plan
- don’t deviate from your written directions
- don’t answer too many questions, let the user try to figure things
out
- don’t “improvise,” make every test the same
- Sit back and silently watch the subject use your program
- watch carefully — be as observant as possible
- look at facial expressions for signs of confusion
- Take copius notes during the test! — write down everything that you observe
- do not “filter” your notes at this point, just write down
everything
- you can analyze and filter the notes after the test is completed, but
you get only one chance to record what went on
- try to record how the user was reacting at specific, identified points
in the program
- Do not talk to the subject during the test!
- if the subject asks a question, reply with something like, “What
do you think?”
- if you must answer a question, try to give a simple “Yes”
or “No” answer
- do not interrupt the subject for any reason
- exception: when it’s time to end the test
- exception: if he or she is obviously stuck and cannot proceed
and is getting frustrated
- let the subject “struggle” for at least 60 seconds before
you interrupt, which can feel like an eternity
- If you have to “bail the subject out,” just fix the problem
and let him or her continue
- do not try to explain what went wrong, just get in, fix the problem, get
out, and let the subject proceed
- Be sure to thank the test participant
After the Tests — With the Subject
- Ask the subject to describe his or her experience before you start asking specific
quesitons of your own
- Ask all subjects the same questions in the same way
- as with the directions, the best way to ensure this is to write your
specific questions down beforehand
After the Tests — “Back at the Office”
- Organize your notes
- Review them with your fellow co-workers
- Prioritize the changes suggested by the tests
- Make those changes that you can
Final Step — Writing Your Report
- I am expecting a report of about five pages in length
- You are to report:
- the details of what happened during your tests
- the changes you plan to make in the final version of your software based on these results
- the changes your tests indicate should be made but that you simply don’t have the time to do
- your conclusions about the quality of your user interface
- your implressions of the usability test experience itself
- You are welcome to use the Short Usability Test Report Template on the Usability.gov website if you like
- Include in your submission all of your backup documents
- the instructions you read to the test subject
- that task list given to the subject
- your evaluator forms with the actual notes you took during the test
- the questions you asked of subjects after the test and their actual responses
Submitting Your Assignment for Grading
Top
First, be sure that you have addressed all the grading criteria listed below.
Second, convert all your documents to PDF format. Then post those documents to
one your team’s GitHub accounts and use the Assignment
Submission form to submit its URL. Be sure to select
91.462 Assignment No. 3: Project Milestone #3: Beta Version and Usability Test
from the assignment drop-down list on that form.
Important Note regarding the Assignment Submission program: If you do not receive an email within a few minutes confirming that your assignment has been submitted, something
has gone wrong. Try submitting again or contact me.
How You Will Be Graded
Top
This assignment will be graded on a 30-point system with points awarded as
follows.
| Criteria |
Possible
Points |
Usability Documents and Notes
- instructions to test subject
- task list to be given to subject
- evaluator form
- questions to be asked of subject
|
8 |
Report
- clearly describes what test subjects did and what happened during the test
- identifies user interface features with which test subjects had success and features with which they had problems
- suggests improvements that might be made to address those areas of the program with which test subjects had problems
|
10 |
Analysis
- discussion of the changes you plan to make in the final version of your software based on these results and why
- discussion of the changes your tests indicate should be made but that you simply don’t have the time to make
- your conclusions about the quality of your user interface and the usability testing experience in general
|
8 |
Presentation
- professional look
- logical organization and structure
- correct spelling and grammar
- adherence to paper layout and writing guidelines
|
4 |
