Tweet

Quantifying the User Experience

Written by: Robert Pothier

Article Overview: The user experience isn't esoteric at all; it's quantifiable, you just have to think about measuring in a new way. And by quantifying the user experience, ebusinesses can position themselves to be even more successful in terms of ROI.

Quantifying the User Experience

Let’s say you’re in the market for a new car. You jump into your old car, fire it up, and head down to the local auto mile—only in your town, it’s the local auto quarter mile, as there are only two auto dealers in town. You pass by the first dealership, and take note of its exterior: the building is dingy and old, the lot is poorly lit, the sinage is confusing . . . even the entryway to the lot is hard to find. But you really need a new car, and you’re determined to shop around until you find the best deal, so you decide to give this dealership a shot.

After driving around the block a few times, eventually—purely by luck—you find the entrance to the lot. You park your old car, get out, and search the lot for a new car you might like. That—the searching—is easier said than done, however, for almost right away, you realize the lot has been organized in no clearly discernable way: brand-new sedans are located right next to used minivans; diesel pick-up trucks with snowplows are right next to two-seat convertibles; used sports utility vehicles are next to brand-new hybrids; even motorcycles, snowmobiles, and scooters are scattered about for good measure. But you really need a new car, so you soldier on, and after searching the lot for a good hour or so, you find the car you want. Now it’s time to find a salesman.

Only you can’t find one; everyone seems to be on a break. After wandering around the showroom for a few minutes, you finally find a salesperson, and he seems willing to help—but he seems to be speaking a different language than you are. You ask him a question; he replies in pure gibberish and hands you a form. You’re not sure why you’re supposed to fill out the form, but some of the form elements look familiar, so you fill it out as best you can and hand it back to him. He disappears into an adjacent room, remains there for twenty minutes—while you wait at his desk—and then returns. He stares at you, silently, for a full five minutes. You ask him what’ wrong—he shrugs his shoulders and tells you that you must have filled out the form incorrectly. He gives you another copy of the form, you fill it out again, he disappears again into the adjacent room for twenty minutes, and then comes back, only to tell you that you filled out the wrong form. Then he asks you for $20,000—the price of the car you wanted.

At this point, of course, you’re already back in your old car, and you’ve laid down a patch in the lot in your haste to get to the other dealership.

Still shaky from your experience at the first dealership, you approach the next dealership cautiously, dreading another bad experience. But the next dealership is noticeably different from the first one: the building is tasteful and clean, the lot is well-lit, and the sinage—the branding—is clear and compelling. The entryway to the lot is easy to find. The lot is plainly organized; sedans, coupes, pick-up trucks, sport utility vehicles, hybrids, and minivans are all grouped together by model, trim level, and year. The lot is also easy navigate; the aisles are wide, the groupings make logical sense, the sinage is clear—so you can always find your way back to the showroom.

And find your way to the showroom you do, eventually, once you’ve found the car you want—the same make, model, trim level, and year as the car you liked at the first dealership. In the showroom of the second dealership, the salespeople are plentiful, attentive, easy to find, and easy to approach. You sit down at the desk of the first one you see. She anticipates your questions—she sets you at ease. She helps you fill out a form—clearly marked as a financing application form—and informs you she’ll need to submit it to her supervisor for approval. Within minutes, she lets you know that your request for financing has been approved, and that you can take delivery of your new car today, if you want.

So, from which dealership are you going to purchase your new car? Which dealership would you feel more comfortable with servicing your new car? Better still, which dealership would you go to first the next time you need to purchase a car? Which dealership would you recommend to your friends and family?

Still think user experience is unimportant?

In fact, with hundreds of millions of potential customers, and little physical differentiators between sites that perform the same basic ebusiness functionality, the user experience may be more important on the internet than it is in the brick-and-mortar business world. How come, therefore, companies spend millions and millions of dollars on websites and web-based applications without considering the user experience?

One reason, perhaps, is because the user experience is difficult to quantify. Not that the user experience is difficult to comprehend, of course—it’s just difficult to quantify because the elements that make up the user experience, stated broadly as branding, usability, functionality, and content, don’t lend themselves easily to objective measurement. A web designer, for example, may examine a website’s user experience and come up with one conclusion, while a web developer may examine the same website’s user experience and come up with a completely different conclusion.

That said, it’s possible to measure the user experience objectively by separating the overall analysis into separate mini-analyses that each focus on a specific portion of the user experience—branding, usability, functionality, and content. Each mini-analysis consists of a series of statements scored on a fixed scale totaling 100, to allow for a true percentage score at the end of the analysis. Each statement in the mini-analysis is really an overall goal or objective for the website or web-based application.

For example:

Branding consists of the design’s aesthetics—how well does the design project an organizational image and message? Statements in a branding mini-analysis could include:
- The website or web-based application provides an engaging, memorable experience.
- The website or web-based application’s visual impact is consistent with brand identity.
- The graphics, visuals, and multimedia elements of the website or web-based application add value to the brand experience.
- Functionality consists of the nuts and bolts of your website or web-based application. Do you deliver what you say you’ll deliver?

Statements in a functionality mini-analysis could include:
- Task progress (i.e., system working, form received successfully, etc.) is communicated clearly.
- Online self-help is provided for all functionality.
- Functionality maintains easily understandable security and privacy standards.
- Usability is concerned with your website or web-based application’s general ease of use, including accessibility for the impaired.

Is your functionality easy to use? Statements in a usability mini-analysis could include:
- The website or web-based application is designed to prevent errors, and helps users recover from errors via rich, detailed error - messages.
- The website or web-based application can be used by broad audience, including those with limited sight or hearing.
- The website or web-based application’s navigational controls are internally consistent.

Content is concerned not only with the actual content—words, pictures, video—on your website or web-based application, but its structure and information architecture as well. Does your content make sense? Statements in a content mini-analysis could include:
- Content is accurate and timely.
- Content is structured in a way that helps users complete tasks.
- Content is consistent with brand messaging.

Once the data for each mini-analysis is compiled, an overview of the total user experience analysis can be expressed by graphing each mini-analysis on a spider graph, with zero at the crossing point, and 100 at the extremity of each arm. Each mini-analysis—in this case, four—is plotted along each arm of the graph, and then each point is connected to its neighbors on the left and right by a line to create a diamond. The bigger the diamond, the better your overall user experience. The more skewed the diamond’s shape is to one arm of the graph, the better your user experience in that particular element. The smaller your diamond is along one arm of the graph, the worse your user experience is in that particular element.

The beauty of using this sort of methodology to analyze your user experience is that you’re not limited to four user experience elements to make up your mini-analyses—you can add or remove user experience elements as you like. For example, you could add an accessibility mini-analysis to measure your user experience as it pertains to meeting accessibility requirements, or you could add metrics mini-analysis to measure your user experience as it pertains to ROI, conversions, click through rates, etc.

Measuring the user experience is critical to online success. At the inception of an online project, the optimal user experience is defined through creating a set of personas, or profiles, of each type of user expected to make use of a website or web-based application. For each persona, all sorts of information is collected into a dossier describing one representative of the persona type: that person’s name, age, educational background, family information, job, technology experience, likes, dislikes, extracurricular activities, hopes, wants, desires—anything and everything, in other words. In a perfect world, the things your personas would want to do on a website or web-based application like yours should make up your website or web-based application’s feature set.

Remember that second car dealership? Someone designed that dealership’s building and sinage with the customer—you—in mind. That’s why you liked it. Someone organized the lot that way because they took the time—upfront—to understand the way their customers like to browse: that’s why you found, easily, what you wanted to find. Their attentive, helpful staff was trained to be helpful and attentive because by being so, and by making it easy for you to buy a car from them, they know that you are more apt to buy a car from them than their competition. And they know that you’re more apt to recommend their dealership to someone else. In other words, you could have purchased the same car from both dealerships, but you’re more apt to go with the second one for one reason: the user experience.

After all, would you really trust your purchase to the first dealership—the one that paid no heed whatsoever to the user experience? What does not caring about the user experience say about a company’s feelings about their customers? Would you trust them to handle your money for you? Would you trust them to service you after a sale? Would you have any confidence in them as an organization—after they paid so little attention to your needs?

That said, user experience is not a static thing, especially on the web. As technology grows, and makes our lives easier, we—as customers—demand bigger and better things from our online experiences. Successful ebusiness companies, therefore, use their personas and use cases to develop clear, objective, measurable metrics, and bake the ability to collect those metrics right into their websites or web-based applications. This allows successful ebusiness companies to collect, articulate, and report business feedback in an actionable fashion, so business people can make proactive, real-time improvements via incremental, innovative development. Planning up front—creating a user experience blueprint—helps successful ebusiness companies reduce overall costs in the long run by eliminating re-work, and by eliminating more costly, less useful features that might not get used in the marketplace.

After all, if your customers don’t want it, don’t need, it, don’t care for it, and don’t even want to think about it, why waste time and treasure building it? Is your bottom like really that unimportant to you?

Related Articles
  Lesson #5: Focus On the User
  Leveraging Usability to Profit in a Down Economy
  Guiding Search Traffic
  Site Design Usability and Navigation
  What is Visual Voicemail

Article Tags: auto dealers, auto mile, convertibles, diesel pick up trucks, entryway, gibberish, good measure, hybrids, local auto, minivans, new car, new sedans, quarter mile, salesperson, sinage, snowmobiles, speaking a different language, sports utility vehicles, stares, twenty minutes



Related Forum Posts
Re: Are you going to use Facebook Timeline? - Hi apu, I Recently used Facebook Timelines, actually it seems like a Facebook blog which makes profile page more attractive but i don't think it will help in internet marketing because it also slow the speed of Facebook Page. Share your Experience!
Picking a corporate lawyer - Hi Everyone, I was just curious to know what's the most important criteria a small business owner should look at when choosing a corporate lawyer? -Cost/hr? -Location? -Reputation/Referral? -Experience? -Personality? How did you pick your lawyer? And what's a fair price to pay per hour? Thanks
Re: New Guy Here - Automobile Industry - Hi Ray, Welcome to the forum. If you use a website to promote your business, please add it to your signature file (via the User Control Panel) so that we can check it out. Best wishes, David H
Re: Hello, I'm brand new - You need to go to your User Control Panel /Profile/Edit Avatar where you will be able to put a pic or avatar of your choice. Hope this helps, any problems let us know, take care, Carol
Re: Two Useful Books To Help You Focus On The CLIENT - Hi David, To add to your thread, I'd like to recommend Jonathan Tisch's "Chocolates On The Pillow Aren't Enough: Reinventing The Customer Experience". Tisch's book includes content on "Welcoming Customers", "The New Art of Customization", "The Challenges of Customer Diversity" and "Offering Something Extra to Your Customers" to name a few.


Share this article with your friends. Fund someone's dream.

Leave a comment below or share on the left and you'll help support entrepreneurs in Africa through our partnership with Kiva. Over $50,000 raised and counting - Please keep sharing! Learn more.

Tweet

Newsletter

Get advice & tips from famous business
owners, new articles by entrepreneur
experts, my latest website updates, &
special sneak peaks at what's to come!

Name:
Email:

Popular Articles

Quick Tips on Buying a Business

Remind Me...

Providing Feedback

Suggestions

Email us your ideas on how to make our
website more valuable! Thank you Sharon
from Toronto Salsa Lessons / Classes for
your suggestions to make the newsletter
look like the website and profile younger
entrepreneurs like Jennifer Lopez.