This week’s readings were very helpful when addressing the issue of usability for our instructions project. I think the hardest part of this project will be translating the deep knowledge we have of our topics to something that can be easily followed by novice users. The readings give good insight on how to achieve this: it’s an iterative process. No one will get it right on the first or second try, but using the tools and testing methods discussed in Chapter 12 will be essential to creating a usable document.
My document will be instructions to install Windows on a Mac computer. Because of the nature of the task, I feel pretty limited on which usability tests I’ll be using. Not a whole lot of people have Mac computers, and even less have Mac computers with an extra copy of Windows lying around. The instructions also call for a dedicated external USB hard drive, which must be deleted and reformatted to complete this task. I’d have a hard time asking someone to buy a copy of Windows and a $100 hard drive for the purpose of testing a school project. The process is also time-consuming, typically taking 4-6 hours if you have all the needed materials in front of you. Based on these, I think multiple performance tests are out of the question. Perhaps I’ll have 1 or 2 people run through it on my computer with my materials over spring break, and then conducting an interview to locate any problem-spots in the instructions.
In addition, I plan on creating a questionnaire to give to a group of 8-10 potential users (of varying knowledge of the subject matter) after reading through my instructions. The questionnaire will help to determine if the language used is appropriate for both novice and expert users, if the design is clean and allows the user to find specific information easily, and, most importantly, if the instruction set makes sense as a whole. I will also try to identify what steps and information should be omitted for advanced users, and elaborated upon for novice users.
From reading your description
From reading your description it seems as if you are going to have each of your user’s actually work through your instruction set. Though, this doesn’t seem logical as you have mentioned for many people to actually be able to utilize your instructions during usability testing you could simply have someone act as if they are stepping through the process instead of attempting to have them actually complete the process. Understandably this may not be the most efficient form of usability testing; it may be the best and only option. I was also thinking it may be possible for you to create some type of Virtual Machine image that you could send to someone to attempt your instructions… Just an Idea!
Thanks!
Jason Flittner
www.JFlitt.com
Virtual Machines
JFlintt’s idea for using virtual machine is great. I don’t think most people know much about them or really what they are. As far as my knowledge is concerned, I think you can run an entire operating system completely independent of the main system, even pick a hard-drive size, which could save a lot of time when it comes to formatting. You may know a lot more about this which just makes me sound like an idiot. I definitely think that without having someone actually complete the task, there may still be some unseen problems with the instructions that won’t actually surface. You definitely seem to have a pretty good handle on things and ways to get quality and usable feedback.
Unforeseen Problems
I am a huge fan of virtual machines and use them in one form or another on almost a daily basis. You brought up a very good point though about unforeseen problems. I believe that this will be especially true for this set of instructions since a virtual machine will be created to emulate a system that will be running software that it wasn't particularly designed for. I would expect that there will be a few issues that arise on the virtual machine that did not arise while using a real machine. I think your best bet would be to try it out and see if there are any issues.
please use terminology to identify tests
You do have a tricky topic for usability testing in an online classroom. That said, it seems like you've put thought into how to handle this and come up with some good ideas. Do keep in mind that part of the assignment is having your classmates do usability tests that you design, so having your fellow students run through your instructions on your computer may not be an option. The questionnaire seems more feasible for class usability testing, but please use the terminology TCT establishes on page 332. (There is no "questionnaire" usability test, for example.) It helps us to communicate when we all share the same grammar.
Follow along
I think it would be neat if you could have some users try testing out your instructions on a computer you provide, however it also sounds like a difficult task. Jason’s idea about using a virtual machine setup would seem easier for those of us who don’t have a Mac computer sitting around to use. I think your best option here though would be to stick to other forms of testing. You may have to focus more on how the instructions prepared the user with knowledge should they try to attempt this in the future. I think by having your instructions tested through document markup they would be able to check wording and their ability to follow along without actually working on the computer. Best of luck with which ever you choose.
-Chris
Me too...
I think I am in a slightly similar situation as you are. I don't expect people to actually make creme brulee (admittedly much less costly to test than yours). Good idea to have someone else run through the instructions over break, I hadn't considered this until now-thought we were limited to our class-I'll probably do this too. What you could do for the class requirement for the instruction tests is do a 'read and find' test, as I plan to do, and gear that test towards the novice user instruction set. I'm sure there are a lot of steps involved with this process so being able to locate something in the document would be great especially if there are breaks taken in between steps (you mentioned that this would take several hours-I doubt anyone would sit and work non-stop for that amount of time). A summary test might work well for the expert version to extract information on which steps could be left out.
testing
I agree with Jason on the virtual machine idea for your usability testing on the instruction. It's a great idea to try to incorporate it in the testing, I just don't know how much knowledge the other fellow classmates/users know about it. Based on what you have told us you have a good grasp on how you are going to collected your feedback. I think a lot of us are finding out that our instructions are going to be conducted with a read and locate or survey type style usability testing. Unfortunately, due to constraint on our abilities(in the class) to to have proper equipment or time(i.e. 4-6 hrs to install windows) it will hinder some feedback results.