As I am creating the content of my instructions this week, it is timely that we are also talking about usability testing. Although common sense would tell us that we need to do some type of usablility testing, the TCT article and this week's Instructor Blog helped me understand what format I will use to evaluate my instructions' usability.
I think that it will be very reasonable to ask users to complete both the survey and the summary test. I feel that this will give me the opportunity to both ask the reader specific questions about the document, while also giving them the chance to point out areas I may have missed. It is important to give the reader direction in what they are looking for, to prevent someone telling you that everything is "fine" and leaving it there. However, I certainly wouldn't want to limit someone who had more ideas on how to improve my instructions.
My usablility test will be a one page document that will include up to six specific questions I have about the document for them to answer. So far, I have:
1) Do the brochure objectives clearly state the purpose and content of the pamphlet?
2) Do the comments and exptert tips provide usefull important information for people who quickly scan the document?
3) Is the overall layout of the document easy to follow and navigate quickly?
Following the survey portion of the test, I would then ask for the reader to summarize what they feel is the most important information from the document.
I am looking forward to putting the instructions together this week and I think that combining the survey and the summary usability tests will give me the most effective feedback. This will help me create the most effective documents for my targeted audiences.
4) Additional Comments?
I think that your application of the survey usability test is appropriate for this exercise. Up until now, I thought that it was in the best interest of instruction testing to follow through with the document markup and make improvements where suggested. But you bring to my attention a good point. Feedback is essential. But we can learn more if we narrow reader’s attention and ask for specific opinions.
But I think you may be limiting your feedback with a survey. Maybe consider leave room for additional comments that you may have not thought of when creating the survey. That will give you the opportunity to hear what readers thought during their usability test.
Great idea
It is a great idea to have a summary test along with your survey. I was thinking of doing something similar, but I didn’t know if we had to choose just one. Having just the feedback from the survey isn’t sufficient to cover all of the details from your instructions. Having an abundance of ideas that are supplied to your instructions will help with achieving the best user comprehension. Also the questions you have started with are good. A couple of other questions you may ask are grammar, punctuation, and other proof reading editing, as stated in the text.
Overall, I like what you have started and I am sure it will be effective.
Zebulon Rouse
Good ideas
It makes a lot of sense to ask the question you are reffering to. That will be the first question I ask, about punctuation and grammar. Hopefully when our evaluators read my usability test they will understand that I am trying to achieve the best overall feedback from my readers and the fact that I used two different types of usability tests won't matter.
I still feel this is the most effective way to recieve feedback in the format we are working with. I would love to be able to set my instructions in front of a sample of people and have them evaluate on the document where they were lost and where they were well instructed. However, within the context of what we are doing for our instructions project, a survey and summary test are the most effective ways to recieve comments.
Mike Sheridan
Survey Questions
With any kind of usability testing comes the caveat that you want to make it as unbiased as possible. This isn't easy, at all! So in your case I think you really need to look over your questions and ask yourself, 'are these leading questions?' Leading questions being, those that would lead someone to possibly answer in a specific way. Most of yours look pretty good, but you need to put yourself in your audiences shoes to possibly see how they may interpret these questions. The way one person may read it can really differ from the next person. If you take this into consideration I feel you will get more valuable feedback.