Quick notes on Drupal usability testing

At work we’re in the process of converting from our current CMS (proprietary system which I will not name) to Drupal. One of my big goals is making a really great experience for our several dozen (maybe 100+) editor-authors. I’ve been eagerly following work by Eileen Webb (her UX Burlington keynote is amazing) and Johanna Bates (forex, this great piece on authoring homepages) on the topic, and particularly on implementing with Drupal.

Yesterday we did usability testing with three of our editors. We got a nice mix, both in their overall tech-savviness and in which parts of the current system they use regularly. (We have two separate setups, one for the “old template” and one for the “new template”.) Justin was the person in the room with them — because I’m waaaaaay to close to it — and one of our student workers, another site editor, Susan, and I were observing. It was a fairly short test because of time constraints, only about a half an hour. We picked the most common tasks based in part on our experience and in part on a survey we did back in the spring.

• Explore the dashboard (Workbench, in our case) and look at the pages in the fake site they’d been assigned.
• Update some text based on a Word document. (Advanced: add bullet point and correct the spelling.) The text was in our “Primary Content” field.
• Add some links to some existing text; some of the linked content was in Drupal, some was not. The text was in our “Secondary Content” field.
• Upload an image and style it to match the main design.

The final task was going to be uploading and linking to a PDF, but there’s still too much wonkiness with that and Media CKEditor. But we did ask if there was anything else that they would do normally that we hadn’t asked them to do, and that did come up twice, so the follow-up to that was to ask them to explore how they might do that.

Here’s what we discovered really needed work:

• If you’re using the CAS module for authentication, logging out via the Admin menu doesn’t actually log out of CAS! After the first test, we discovered we had to close Firefox entirely and reopen.
• Nobody sees vertical tabs. I know people are really into them in Drupal editing screens, but for those secondary content links, it was way too hidden for the first two users. I switched it to horizontal tabs before the third user came in, and wow was that better. Our student worker noted that it was a much better match for how people’s eyes move across the screen. I’m inclined to agree.
• Styling terminology needs to be pretty verbose. A style labeled “Image” in the CKEditor Styles drop-down was a bit  vague.
• For our uses, alt text really needs to be required, and one of our testers skipped that but entered the title text, which is kind of pointless.

Other than that, it was really successful! They got the hang of the basics right away and expressed a lot of happiness about how “clean” and “simple” everything looked.

(Also: people who deal with lots of documents really want to be able to upload multiple documents. They can’t in our current system, but one of our testers uploads quite a bit of Excel, and other quite a bit of both PDF and Excel. They both found their way to the (broken, because libraries) multiple uploading functionality, but couldn’t use it. I’ll definitely be testing that in more detail.)

And the stuff that needed work: it’s all done already.

• The simplest way to handle the CAS logout issue is to set a Rule: when a user logs out, redirect them through /caslogout. I feel like this needs to be added to the CAS module documentation.
• Horizontal tabs FTW.
• Justin changed “Image” to “Image Right-Aligned” — we’ll see how that plays out next time around.
• Structure > File Types > Image > manage fields. Remove Title and make Alt Text required.

I know there’s still more complex stuff coming, and I have work to do wrangling Media CKEditor to handle documents the way I want them, but overall I’m really really pleased. These folks walked in without any training in using Drupal as editors, and walked out having accomplished the very most common tasks they need to do to be successful.

PS: making fake website content is just too much fun.