My documents are instruction sets for making creme brulee. The novice set is intended to be a step-by-step pictorial set of directions that will be as visually oriented as possible. The advanced user version will be more of a formal recipe informed with pictures, margin asides, and a brief introduction. That said, I intend to use the "read and locate" usability text with optional "summary test".
Novice users probably won't be interested in long explanations of things (like why a step is important) so it important that they understand what is happening in the pictures and be able to identify them easily and quickly. This is why I want to use the "read and locate" test; I think it is a little bit unreasonable to ask people in this class to actually make creme brulee (although they should!). However, there will need to be some sort of safety check as there is a step that involves the use of a blow torch (optional-but at least the use of a broiler). I might also ask something like "if you were to make this, could you do it based on the instructions given."
For the advanced instruction set, I think I would gain more information from a "summary test" but "read and locate" will still be used. It may be irrelevant to an "advanced user"-one that has possibly made creme brulee before-to ask if they understand the content since they likely do already, or at least they have some general understanding. Basically I want to get a feel for whether or not the reader gained something new from the document and if it was concise and/or efficient enough.
The biggest thing I want people to understand is that it is very possible-even easy-to make something as elegant as creme brulee and that it is so simple (the best things are simple) and unpretentious. The "read and locate" test should offer some insight into this. If they can locate and identify the images then the should be able to follow the directions well enough to produce a quality product.
You have mentioned that the
You have mentioned that the read and locate test might be best suited for your novice form of your instruction set, the one containing mostly pictorial instructions. But it seems to me that a survey or document markup test may be more sufficient for the novice “pictorial” set, due to the fact that as described it seems the read and locate test is best suited toward documents with much instruction and wording. I suppose that the read and locate test could also come into play with reading a small blurb and then locating the correct graphic that goes along with that blurb. This test may also help you to identify whether or not a user could easily lose their place going from graphic to instructions and vice versa.
Thanks!
Jason Flittner
www.JFlitt.com
Document Writeup
I agree with you about using a document writeup for the novice set. I believe the document writeup is a good novice usability test for most of the instruction sets on here. I think this test is especially important for any of the technology based sets when trying to connect text to visuals and then what your computer is showing.
I do think a summary test for the expert set would be appropriate. The "expert" will be able to point out what they think makes a great creme brulee. Also, if you're going to add extra information, such as special ingredients, unique preparations or different ways to cook it, then a summary test would work well. The expert user may be looking for ways to spice up their creme brulee and these extra little tips will be important to them.
Read and Locate Test
I agree with most of the comments on here suggesting that the read and locate test might not be the best way to test your "visually oriented" novice instruction set. This type of user using your instruction set isn't really looking for a particular piece of information from your instructions, but is really looking at the instructions in kitchen, while trying to make your recipe step by step. The type of feedback most helpful to improve your instruction set here would be the kind that points out where users have trouble understand a step, or where they run into problems.
Read-and-locate test
My understanding of the "read-and-locate" test is that it's used to see if people can navigate your instructions quickly, which I think would be most important for a FAQ or Troubleshooting type of instruction manual. I'm not sure how appropriate it would be for your process, and especially not for your beginner instructions. I think you could make an argument that it would be useful for experts who have done it before and have just forgotten a certain step and they want to find that step quickly. But, I think you might want to use a hypothetical document markup test for your novice instructions, as you suggest when you ask them, if you were to do this, would these be clear?
Kristin
Simple and Tasty
Separating the users like that is a good idea. It reminds me of Alton Brown from Good Eats. He makes the recipe but also gives the science behind it and why you add an ingredient at the beginning of the recipe rather than during the middle. Maybe throw in a few variations of the recipe for the advanced user as they may want to try a different style of crème brulee. As long as there is not a ton of ingredients listed (I know there is not) then the user will feel the recipe is simple. Also make sure the final product looks good as it will make the user want to make it!
different tests for different audiences
I agree with Jeff that it is a good idea to parse your audiences strongly. It helps keep your approach user-centered rather than systems-centered. It's a bit tricky to have different usability formats for different audience groups, but I do like that you're thinking of the groups as separate entities with different needs, values, and expectations. This is the important move: to view the audiences as related but discreet groups rather than points on a spectrum of experience.
Read and Locate
Like consensus of the posts so far, I do not feel that the read and locate test would be a good idea for the novice audience instructions. I could go into much detail but everyone else has done a good job of providing reasons for the other types of tests. Although it might not be a good decision to use this type of test for the novice audience, I think it might be a good choice for the expert audience. This is because if the audience is an expert, they probably already know the reasons for doing things in a particular order. They will most likely consult your instructions in the event they forget something, such as how much of an ingredient to use or what temperature to have the stove at. The expert audience will want to be able to quickly find the information that they need. The read and locate set of instructions will ensure that they can achieve this quickly.
Read and Locate
My opinion about the read and locate method for novice people differs from what other people are saying about this method. I believe that this method would be a good choice for novice users. This is a good choice because when working with novice users, they generally have no idea about what they are doing. With this in mind, if the directions aren't clearly written or the graphics aren't clear they might be easily confused about whats going on. If novice users use the test of reading and locating what needs to be done it will give you a better idea if the instruction set is going to work well with the novice audience.
Is this a poll? JK!
I'll have to go with you on this one Matt. Who ever said you can't read a picture!? After all a picture is worth 1000 words! I think your novice users should be able to essentially "read" a lot of information from your instructions. While you did mention that this may not be as appropriate for the expert set, I think you can pull it off because it seems the instruction set will have plenty of text anyways, so in this case they'll be analyzing the text a little more hopefully. Just my two cents! You seem to have thought about this pretty well, so keep on analyzing why you using this test and I think you'll come up with a good conclusion as to whether or not you should stick with it.
testing?
I like how in your usability testing you are conducting two different testing with the novice and expert instructions. This a great idea to gain a different perspective of the task with the selected usability testing, maybe to see if your missing a step or to give additional advice. The way you configure the usability test with the novice in the read and locate, will work in my opinion. As a novice you depend on the pictures to help support the text for reinsurance your are doing the steps correctly. However, I'm not sure the read and locate usability testing will fit the expert instruction.