Cost-justifying usability : an update for an Internet age

0120958112......Page 1
Copyright page......Page 5
Table of contents......Page 6
THE BOOK......Page 24
ACKNOWLEDGEMENTS......Page 27
1.1 INTRODUCTION......Page 28
1.2 LESSONS FROM THE PAST: SEMINAL RESEARCH ON RETURN ON INVESTMENT IN HUMAN FACTORS IN THE 1980S AND 1990S......Page 30
1.3 WWW.SOWHAT......Page 32
1.3.2 If I Had a Hammer, I Wouldn’t Necessarily Be a Carpenter......Page 33
1.3.3 The Tug of “Internet Time”......Page 34
1.3.4 The Old “Get Something Out There” Approach Doesn’t Work Anymore......Page 35
1.3.5 The Dangers of Amateur Usability Engineering......Page 36
1.3.7 The World Is More Complex, so It Is Harder to Know What All the Possibilities Are......Page 39
1.4 ISN’T IT OBVIOUS?......Page 40
1.6 THE SOLUTION: APPROACH, TOOLS, AND COMMUNICATION......Page 41
REFERENCES......Page 42
2.1 INTRODUCTION: WHAT DO WE MEAN BY THE RETURN ON INVESTMENT OF USABILITY?......Page 44
2.2 HOW CAN WE “PROVE” THE RETURN ON INVESTMENT? SOME EXAMPLES AND STATISTICS......Page 45
2.2.2 Development: Reduce Costs......Page 46
2.2.3 Sales: Increase Revenue......Page 53
2.2.4 Use: Improve Effectiveness......Page 56
2.2.5 Other Return on Investment Factors......Page 60
2.3 CONCLUSION......Page 61
REFERENCES......Page 63
3.1 INTRODUCTION......Page 68
3.2 THE USABILITY ENGINEERING LIFECYCLE......Page 69
3.2.1 Phase One: Requirements Analysis......Page 72
3.2.2 Phase Two: Design/Testing/Development......Page 75
3.2.3 Phase Three: Installation......Page 80
3.3 GENERAL APPROACH......Page 82
3.4 SAMPLE COST-BENEFIT ANALYSES......Page 84
3.4.1 An Application for Internal Users......Page 86
3.4.2 A Commercial Application by a Vendor Company......Page 107
3.4.3 An E-Commerce Site......Page 113
3.4.4 A Product Information Site......Page 119
3.5 SUMMARY......Page 123
REFERENCES......Page 127
4.1 INTRODUCTION......Page 130
4.2 WHY MEASURE THE COST BENEFIT OF HUMAN FACTORS?......Page 132
4.3 USABILITY ENGINEERING IN WEB SITE AND  APPLICATION LIFECYCLES......Page 134
4.3.1 What Contributions Can Usability Engineering Demonstrate?......Page 135
4.5 HOW DO COST-BENEFIT ANALYSES RELATE TO BUSINESS CASES?......Page 143
4.6 ASSESSING COSTS AND BENEFITS OF USABILITY ENGINEERING......Page 145
4.7.1 Increased Sales or Revenues Resulting from Increased Completion Rates on a Web site......Page 147
4.7.3 Increased Sales or Revenues......Page 148
4.7.4 Increased User Productivity......Page 149
4.8 USABILITY ENGINEERING COSTS......Page 150
4.9.1 Usability Engineering Costs for Repeat Visits to a Web Site......Page 152
4.11 SIMPLE COST-BENEFIT ANALYSIS TECHNIQUES......Page 153
4.11.2 Payback Period Example......Page 154
4.12.1 Background......Page 155
4.12.3 Present Value of Cash Inflows Example......Page 157
4.12.4 Net Present Value......Page 158
4.12.5 Internal Rate of Return......Page 159
4.12.6 Further Interest-Based Selection Issues......Page 160
4.13 SUMMARY AND RECOMMENDATIONS......Page 164
4.14 A LOOK AHEAD......Page 165
REFERENCES......Page 166
5.1 INTRODUCTION......Page 170
5.1.1 Some Notes on Terminology......Page 171
5.1.3 Marketing 101......Page 172
5.2 THE MARKET......Page 173
5.2.2 External Factors......Page 174
5.3 PRODUCT......Page 176
5.4 PRICING......Page 178
5.4.1 Pricing Models......Page 179
5.5 PROMOTION......Page 180
5.5.1 Promotional Tactics......Page 181
5.5.2 The Internet......Page 182
5.5.3 Internal Promotion......Page 183
5.6 CONCLUSIONS......Page 184
REFERENCES......Page 185
Do You Have Any Advice for Someone Interested in Marketing Usability Services?......Page 186
What Do You Think Are the Biggest Challenges with Respect to Marketing Usability Services?......Page 188
What Do You Think Are the Biggest Opportunities with Respect to Marketing Usability Services?......Page 189
6.1 INTRODUCTION......Page 192
6.3 RESISTANCE TO USABILITY IN STARTUPS......Page 193
6.4 DIFFERENT VALUES DURING THE INTERNET BUBBLE......Page 194
6.5 TIME-TO-MARKET AND THE “FIRST-MOVER” ADVANTAGE......Page 196
6.6 FOCUS ON VALUATION—WHAT’S A STARTUP WORTH?......Page 197
6.7 SUCCESSFUL STARTUPS VALUE THE USER EXPERIENCE......Page 198
6.9 A TOOL FOR UNDERSTANDING AND  EVALUATING STARTUPS......Page 200
6.10 HOW DO STARTUPS DETERMINE VALUE?......Page 204
6.11 HOW MUCH SHOULD STARTUPS INVEST IN USABILITY?......Page 206
REFERENCES......Page 208
7.1 INTRODUCTION......Page 212
7.2 UNDERSTANDING YOUR COMPANY......Page 213
7.2.1 Customer-Focused Company......Page 214
7.2.4 Executive-Focused Company......Page 215
7.3 COST JUSTIFICATION IN VENDOR COMPANIES......Page 216
7.4 COST JUSTIFICATION FOR WEB APPLICATIONS......Page 218
7.5 COSTS......Page 219
7.5.1 Initial Costs of Building Usability Laboratories......Page 220
7.5.2 Sustaining Costs......Page 223
7.6 BENEFITS......Page 227
7.6.1 Benefits to the Vendor Company: Increased Revenues......Page 228
7.6.2 Benefits to the Vendor Company and Customers: Decreased Costs......Page 231
7.7 COST-BENEFIT EXAMPLE......Page 235
7.8.2 Perform Baseline and Ongoing Measurements......Page 237
7.9 SUMMARY......Page 238
REFERENCES......Page 239
8.1 INTRODUCTION......Page 242
8.2 CATEGORIES OF RETURN ON INVESTMENT......Page 243
8.3 INTERNAL RETURN ON INVESTMENT......Page 244
8.4 INTERNAL SOCIAL RETURN ON INVESTMENT......Page 246
8.5 IMPROVING INTERNAL SOCIAL RETURN ON INVESTMENT......Page 247
8.5.1 Making Connections......Page 248
8.5.2 Communicating with Product Teams......Page 251
8.5.3 Respecting the Rest of the Product Team......Page 252
8.5.4 Understanding How Others Are Judged......Page 254
8.5.5 Supporting Groups Outside Mainstream Product Development......Page 255
8.5.6 Enlisting Quality Assurance Teams as Partners in User Interface Quality......Page 256
8.6 REUSE AND INTERNAL RETURN ON INVESTMENT......Page 257
8.6.1 Methods for Raising Awareness of Reuse......Page 258
8.6.2 Within- and Between-Product Consistency Inspections......Page 259
8.6.3 User Interface Patterns as a Tool to Promote Reuse......Page 260
8.6.4 Group User Interface Inspections......Page 261
8.6.5 Graphic User Interface Rolls for Workflow Assessment and Comparison......Page 262
8.7 USER-CENTERED DESIGN INFRASTRUCTURE AND RETURN ON INVESTMENT......Page 263
8.7.1 Templates and Common Documents......Page 266
8.8 EXTERNAL RETURN ON INVESTMENT......Page 267
8.9 HISTORICAL, PREDICTIVE, AND SIMULTANEOUS MEASURES......Page 268
8.10 DIRECT AND INDIRECT MEASURES......Page 269
8.11 MEASURING IMPROVED USER EXPERIENCE......Page 270
8.11.1 First-Time Use......Page 271
8.11.2 The Learning Experience......Page 273
8.11.3 User Performance by Experienced Users......Page 274
8.11.4 Need for Customer Support and Service......Page 276
8.11.5 Customer Satisfaction and Attitudes......Page 278
8.12.1 User Experience Confounds Outside the User-Centered Design Team......Page 280
8.12.2 Addressing Confounds with Controlled Testing......Page 281
8.12.3 Confounds Unrelated to User Experience......Page 282
8.13 RETURN ON INVESTMENT FROM COMMUNICATING IMPROVED USER EXPERIENCE......Page 284
8.14 SUMMARY: WHY WE CATEGORIZE USABILITY RETURN ON INVESTMENT......Page 285
REFERENCES......Page 286
9.1 INTRODUCTION......Page 292
9.2 BENCHMARKING AWARENESS AND ISSUES OF IMPLEMENTATION......Page 293
9.3 A PROBLEM OF EXPERTISE......Page 294
9.4 THE OBVIOUS COST-BENEFIT FACTORS......Page 295
9.5 HOW USABILITY SCIENCE CAN BE UTILIZED IN A LARGE  CORPORATE SETTING......Page 296
9.5.1 Usability Science as a Tactical Asset......Page 297
9.5.2 Usability Science as a Strategic Asset......Page 299
9.6 THE ACTUAL DOWNSTREAM IMPACT......Page 300
9.7 CURRENT TRENDS IN THE APPLICATION OF USABILITY SCIENCE......Page 301
9.7.1 The Rate of Dispersion of Formal Usability Science into Large Corporate Projects......Page 302
9.7.3 Factors That Slow or Completely Inhibit Diffusion of Usability Science in Large Corporate Settings......Page 304
9.8 WHERE YOU WILL CURRENTLY NOT FIND USABILITY SCIENCE......Page 307
9.9 MOVING TO THE USE OF USABILITY SCIENCE AS A STRATEGIC ASSET......Page 308
9.10 CASE STUDIES: TACTICAL AND STRATEGIC......Page 309
9.10.1 Liability Claims......Page 310
9.10.2 Service and Maintenance Costs......Page 312
9.10.3 Customer Complaints......Page 314
9.10.4 Increased Complexity in New Products......Page 316
9.10.5 Market Competition......Page 318
9.11 SUMMARY......Page 321
REFERENCES......Page 322
10.1 INTRODUCTION......Page 324
10.2 RETURN ON INVESTMENT IN USABILITY OF WEB APPLICATIONS......Page 325
10.3 LESSONS LEARNED FROM ANALYSIS OF RETURN ON INVESTMENT IN USABILITY ON THE WEB......Page 327
10.4 THE ATTRIBUTES OF VALUE......Page 329
10.6 THE COST OF CHANGE......Page 332
10.7 REDUCING THE RISK OF FAILURE......Page 333
10.8 MEASURING VALUE......Page 334
10.9 HOW TO SELL RETURN ON INVESTMENT IN USABILITY......Page 337
10.10 SUMMARY AND FUTURE DIRECTIONS......Page 340
REFERENCES......Page 341
WEB SITES OF INTEREST......Page 342
11.1 INTRODUCTION......Page 344
11.1.1 Do We Know Our Users?......Page 345
11.1.2 Our Focus in this Chapter......Page 347
11.2.1 Making Strategic versus Tactical Business Cases......Page 348
11.2.2 The Strategic Importance of International Markets......Page 349
11.2.3 Linking Strategic Risk Management and International Design......Page 351
11.3 THE DREAM OF “SIMPLE” ANSWERS......Page 353
11.3.1 Translation, Localization, and Internationalization......Page 354
11.3.2 Cultural Design Guidelines......Page 356
11.4.1 Identifying International User Factors and Contextual Variables Specifically Relevant to Your Domain......Page 361
11.4.2 Developing Your Own List of Contextual Variables......Page 367
11.5 UNDERSTANDING COSTS OF INTERNATIONAL RESEARCH......Page 368
11.5.2 Simultaneous Translation......Page 369
11.5.4 Adapting the Recruiting and Scheduling Strategy......Page 370
11.5.6 Per Diem Expenses......Page 371
11.5.8 What Is the Bottom Line?......Page 372
11.6.2 Hiring Local Consultants......Page 374
11.6.3 Remote Evaluation......Page 375
11.7.2 Do Background Research......Page 376
11.7.5 Focus Stakeholders on Broader Benefits and Synergies......Page 377
11.7.7 Build Cumulative Learning......Page 378
Special Thanks to......Page 379
REFERENCES......Page 410
12.2 OVERVIEW OF UNIQUE ASPECTS OF USABILITY  ENGINEERING FOR INTERNATIONAL USER INTERFACES......Page 386
12.3 SAMPLE COST-BENEFIT ANALYSES OF USABILITY ENGINEERING ON INTERNATIONAL WEB DEVELOPMENT PROJECTS......Page 389
12.3.1 A Cross-Cultural E-Commerce Web Site......Page 390
12.3.2 A Cross-Cultural Product Information Web Site......Page 406
SUMMARY AND CONCLUSIONS......Page 409
13.1 INTRODUCTION......Page 412
13.2.1 Visual Impairments......Page 413
13.2.3 The Role of Guidelines and Browser Compatibility......Page 414
13.3 THE BENEFITS OF ACCESSIBILITY......Page 415
13.3.2 Accessibility Helps Everyone......Page 417
13.3.6 Niche Markets......Page 418
13.3.8 Legal Requirements......Page 419
13.3.11 When Is There No Benefit to Designing for Accessibility?......Page 421
13.4 AUDIENCE DIVERSITY AND MARKET SIZE......Page 422
13.4.1 Estimating Benefits......Page 423
13.4.2 Demographic Data on Individual Differences......Page 425
13.4.3 Uncertainties in Estimation......Page 426
13.5 THE COSTS OF ACCESSIBLE DESIGN......Page 428
13.5.1 Retroactive Process to Make Your Site More Accessible......Page 429
13.5.2 Estimating Costs......Page 430
13.6 ACKNOWLEDGEMENTS......Page 440
REFERENCES......Page 441
14: Ethnography for Software Development......Page 442
14.1 A BRIEF DESCRIPTION OF ETHNOGRAPHY......Page 445
14.2 DISTINCTION BETWEEN SITE VISITS AND ETHNOGRAPHY......Page 447
14.3 ETHNOGRAPHY AT DIFFERENT TIMES IN THE PRODUCT DEVELOPMENT CYCLE......Page 448
14.3.1 Early in the Development Cycle: Near-Term Product Planning and Future Planning......Page 449
14.3.2 Middle of the Product Development Cycle: Feature-Specific and Product-Specific Questions......Page 450
14.3.3 Final Phases of the Product Development Cycle......Page 451
14.4 SOME BEST PRACTICES FOR FIELD RESEARCH......Page 452
14.4.1 Protocols for Visits......Page 453
14.5 CASE STUDY 1: WINDOWS XP PROJECT: GUIDELINES FOR A SUCCESSFUL ETHNOGRAPHY—ANNE KIRAH......Page 455
14.5.1 Phase 0: The Planning Phase......Page 456
14.5.2 Phase 1: Recruitment of Participants......Page 459
14.5.3 Phase 2: Exploratory Studies......Page 462
14.5.4 Phase 3: Participatory Design Sessions......Page 463
14.5.6 Phase 5: Beta Testing in the Field......Page 464
14.6 CASE STUDY 2: TABLET PC BETA TESTING IN THE FIELD—EVAN FELDMAN......Page 466
14.6.1 Field Trial 1......Page 467
14.6.2 Field Trial 2......Page 468
14.6.4 Field Trial 4......Page 469
14.7.2 Safety and Security......Page 470
14.7.5 Development of Main Pillars......Page 471
REFERENCES......Page 472
15: Out of the Box: Approaches to Good Initial Interface Designs......Page 474
15.1 GETTING IT RIGHT OUT OF THE BOX......Page 475
15.2 CREATIVE DESIGN......Page 476
15.2.1 Preparation and Information......Page 477
15.2.2 Working Within the Rules......Page 478
15.2.3 Creativity as a Skill......Page 479
15.3 DESIGN BY ANALOGY......Page 480
15.4 RATIONAL DESIGN......Page 482
15.5 DESIGN BASED ON KNOWLEDGE OF PERCEPTUAL AND COGNITIVE PRINCIPLES......Page 483
15.6 CONCLUSION......Page 488
REFERENCES......Page 489
16.1 INTRODUCTION......Page 492
16.2 CASE STUDY......Page 496
16.2.1 Model Tasks......Page 499
16.2.2 Run Existing Applications Productivity Test and Use Results to Refine Modeling Technique......Page 506
16.2.3 Compare Modeling Results to Goals......Page 509
16.2.4 Address Unmet Goals......Page 510
16.2.5 Run Proposed Application Productivity Test......Page 512
16.3 CONCLUSIONS......Page 514
REFERENCES......Page 515
17.1 INTRODUCTION......Page 516
17.2 DEFINITION FOR RITE......Page 517
17.3 BUSINESS CASE FOR RITE......Page 518
17.3.1 RITE Method Results in More Issues Fixed......Page 520
17.3.2 RITE Method Results in More Issues Found......Page 521
17.3.3 RITE Method Results in Better Team Dynamics......Page 522
17.3.4 RITE Method Degrades Gracefully......Page 523
17.3.5 The Pitfalls of Using the RITE Method......Page 524
17.4 USING RITE EFFECTIVELY......Page 525
17.5 CASE STUDIES......Page 528
17.5.1 Case Study 1: Microsoft Age of Empires II Tutorial......Page 530
17.5.2 Case Study 2: Oracle Interactive Voice Response E-Mail......Page 533
17.5.3 Case Study 3: Microsoft Digital Image Library......Page 536
17.6 CONCLUSIONS......Page 541
REFERENCES......Page 543
18.1 INTRODUCTION......Page 546
18.2 TYPES OF USABILITY TESTS......Page 548
18.3 SUMMATIVE TESTING: METRICS......Page 550
18.3.1 Effectiveness, Dimensionless, and Embodied Metrics......Page 551
18.3.2 Efficiency and the Underlying Form of Data......Page 552
18.3.3 Satisfaction and the Reliability of Metrics......Page 556
18.3.4 Measuring Usability: In Summary......Page 561
18.4.1 The Statistical Argument, Hypotheses, Rejection, and Power......Page 562
18.4.2 Different Kinds of Hypotheses......Page 565
18.4.3 Do We Need Hypotheses?......Page 568
18.4.4 How Do We Compute Sample Size and Effect Size?......Page 570
18.4.5 When Do We Compute Sample Size and Effect Size?......Page 572
18.4.6 Statistics and Reality......Page 574
18.5 CONCLUSIONS......Page 576
REFERENCES......Page 578
19.1 ONLINE SURVEYS AND THEIR VALUE......Page 582
19.1.1 Cost and Return on Investment......Page 584
19.1.2 A Case Study for Online Usability Research: Staples Inc.......Page 587
19.2 MECHANICS OF ONLINE SURVEYS......Page 588
19.2.1 Study Design......Page 589
19.2.2 Recruiting......Page 590
19.2.3 Study Administration......Page 593
19.2.4 Analysis......Page 596
19.4 SURVEY VENDORS......Page 597
19.4.1 Requests for Proposals......Page 598
19.5 CONCLUSIONS......Page 599
REFERENCES......Page 600
20.2 ESTIMATING COSTS......Page 602
20.2.1 User-Centered Design Methods......Page 604
20.3 MAKING THE COST-BENEFIT CASE......Page 607
20.4 CASE STUDIES......Page 610
20.4.1 Israel Aircraft Industries......Page 611
20.4.2 Inland Revenue/EDS......Page 621
20.4.3 Comparison......Page 625
20.4.4 Taking up the Methods......Page 626
REFERENCES......Page 627
21.1.1 It Is Time for a Mindset Upgrade......Page 628
21.2.2 The Business Decision-Making Mindset......Page 629
21.2.3 Ignored and Undervalued?......Page 630
21.3.1 The Price of Admission......Page 632
21.3.2 Creating a Valid Assessment......Page 633
21.3.3 Sprint’s Program-Oriented Approach......Page 634
21.3.4 Identifying the Link Between Usability Data and Financial Results......Page 635
21.4.1 Brave New World......Page 636
21.4.2 You Are Aligned......Page 637
REFERENCES......Page 638
22.1 INTRODUCTION......Page 640
22.2.4 Kim Rachmeler, Amazon.com......Page 641
22.3 QUESTION #1: WHAT ARGUMENTS TO PAY FOR USABILITY SUPPORT HAVE YOU SAID “YES” TO?......Page 642
22.4 QUESTION #2: WHICH ARGUMENTS HAVE YOU SAID “NO” TO?......Page 644
22.5 QUESTION #3: WHAT WOULD YOU LIKE TO HEAR FROM YOUR WOULD-BE USABILITY SUPPORT (THE IN-HOUSE TEAM OR A CONSULTANT), TO HELP YOU MAKE YOUR DECISION TO “BUY” OR NOT?......Page 645
22.6.2 The Attribution Problem......Page 646
22.7 AND SO . . .......Page 647
Index......Page 650
About the Authors......Page 678