After reading through both chapter 12 and instructor bog 6 I have come up with what I think will be the best way to test my list of instructions. Just like the instructor said in the blog, it is hard to use some of these tests with an online course, but there is one test that I believe is a good one that can be used. I decided to use the survey style of test for testing my instructions. I think this is the best test to use for me for a few reasons. The first is the obvious, this is an online course and not everyone has access to the same equipment. Many of the tests listed in Chapter 12 require certain tools or equipment in order to use that style of testing. For instance, I would not be able to use the laboratory testing method because I do not have the room nor the cameras to conduction this type of testing method. Another reason why I chose the survey method is because I am planning on asking a few questions that will help me improve the context of my instructions.
Now, the overall makeup of my test. Like I said earlier in this blog I am planning on using a survey style test. What this entails is that I need to make up a list of questions for people to read and answer during or after using/reading my instructions. The First question that I want to ask is, "Was this instruction set clearly wrote and understandable?". Another question would be, "Are the graphics easily recognizable and helpful?". One last example of a question would be, "Was the vocabulary used easily understood?". Along with this list of questions and a few more I plan on including an area for suggestions on what else can be added or removed in order to increase the effectiveness of my project.
Good pics
Yeah I think that you are right about the laboratory test and not having enough room. After looking at your pictures, I think that a survey will work just fine. The survey will definitely help to see if the user understands every step and the important locations on the engine. When someone that knows engines writes an instruction manual on how to change oil the vocabulary there is definitely going to be a problem with vocabulary. In this case the survey will probably help your instruction manual our a lot. The only thing about making this instruction manual with one set of pictures is that even though in most American cars the oil is easy to change in foreign cars the items are sometimes hard to find.
Survey Test
I have to agree; I think a survey test is your best option for this assignment and the task you have chosen to create instructions for. I wonder, though, if people who don't already know how to do this (or people who might not have any experience with it at all) would be able to give you suggestions on what to add in your document. I suppose it can't hurt to ask in your survey, but I know I wouldn't be able to tell you anything about what to add when it comes to changing my oil. I think this will be a problem for a lot of us, though, especially with expert or advanced instructions.
Kristin
Yes/No
One thing I noticed about your sample questions is that all of them are yes/no questions. While you will have some of these, you need to think about the quality and usability of the answers you will get. "Yes" and "No" answers won't give you the "why" or "how." With yes/no answers you may discover that your instructions don't work, but you will not have learned why they didn't or how to improve them.
Adding to Yes/No
This is definitely a good point. It's not that hard to turn your yes/no questions into something more. Instead of "Was this instruction set clearly written and understandable?" you could ask "Which parts of the instructions did you find unclear or hard to understand?" Instead of "Are the graphics easily recognizable and helpful?" You could ask "How can I improve the graphics to make them more useful?"
If you've done a good job with these things, the tester will respond by saying "It's already pretty good." But if they need improvement then you've instructed the tester in a way that will get him or her thinking about the "why" and "how."
The problem with surveys is that you constrict feedback to what you think to ask. You have to combat that by taking full advantage of their strength, which is to direct the tester to give you information that you know will help you. Try to make your questions hard to answer with a lazy, easy "yes" or "no."
Survery testing
I believe you are right with picking a survey test. Because of the way this class is set up and the nature of the project, doing a survey test seems like one of the most beneficial ways of improving your instructions. One way you could improve your questions so they are not just yes or no would be to just add at the end for the user to explain his answer. Eg) Were the instructions clear and easy to use?? If not, what steps seemed confusing or too difficult?
Patrick Griffin
pgriffin@purdue.edu
survey
I have to agree that the survey test will be the most effective testing for your instructions. I believe that the survey testing could be the only beneficial ways to do our usability testing, since this is an online course. Kristin brings up a good point on how effective people can be to others document based on their experience. In general, it could be difficult for a lot of us to find people with experience in our set of instructions. However, this could allow feedback from the less experience users to critique your instructions from a different perspective.
Perception
It is important to note that often times a discrepancy between the subject's perception and reality of a situation. Just because the subject tells you that the instructions are good, does not mean they actually are. The subject might be embarrassed to admit they had difficulties (even with instructions), and lie. On the other hand, they may blame the instructions because they are not in a good mood. It is important to measure how the instructions actually helped them instead of just how helpful they thought the instructions were.