How We Reduced Mozilla Support Calls by 70% Using UX

User experience research using data mining and paper prototyping quickly led to tangible success for one of the busiest support sites in the world

One of the most common questions that clients or seminar participants ask us: “Is it worth doing a redesign to improve usability?” In other words, what is the return on investment (ROI) of a redesign?

In recent years, we ve seen a decline in usability ROI, most likely due to maximizing usability improvements: with more than 20 years of web design experience behind them, designers are learning very little. But redesigning for usability can still save you a significant amount of money.

In this article, we ll share the story of a Mozilla support site that was able to get a 233% improvement thanks to a usability redesign. Therefore, we can say that the redesigned site is about 3 times better than the original.

The cost of this redesign is 14 man-weeks or 560 man-hours.

Is it worth spending 14 weeks to get 3x better? It depends on the hourly rate of your staff and the value of your website. Therefore, this question is difficult to answer, everything is very individual. However, for Mozilla, which is receiving massive amounts of traffic, the improvement is definitely worth the effort, as it is for almost any large site or company that does massive online business.

So how did Mozilla do this? What was involved in this redesign?

Mozilla

Mozilla is a large, global open source software development organization with staff and volunteers. They create one of the most popular web browsers (Firefox), along with other useful products and services.

Pain points

Millions of people visit the Mozilla Support Site each year to get help with Firefox and other products or services. When users cannot get an answer from the information on the site, Mozilla is committed to helping by answering each person s questions in the customer support forum as quickly as possible.

As the Mozilla website grew organically, users found it difficult to find information and the support staff were struggling to keep up with the number of questions on the forums. In particular:

• On approximately 400 pages, problems finding specific information in the help documentation…
• Forum staff had a hard time answering questions as quickly as they wanted due to growing number of incoming questions by Firefox.
• Forum overload also made it difficult for employees to find time to write new help articles on topical issues. Articles could have helped, but the growing number of articles was also causing even more search problems.

Action plan

Mozilla decided to invest in discovery and iterative usability testing to improve the IA of its support site. The research questions were aimed at understanding: how people (users and employees) used the support system; what information is really important to users.

Key research questions

• How do users and employees interact with the support system?
• What are the most important issues for a website redesign?
• What information is most in demand?
• What words do people use when searching?
• What information is missing?
• What is the best way to organize and present information to meet user needs?

UX team

The UX team consisted of 3 people:
Susan Farrell, Senior User Experience Specialist, Nielsen Norman Group… Susan did the research, discovered and analyzed the data, and then made recommendations for design changes.

Crystal Beasley, Product Designer, Mozilla… Crystal led the project, coordinated with Mozilla stakeholders, and observed users during research sessions on paper prototypes.

Bram Pitojo, web application designer, Mozilla. Bram designed the prototypes and oversaw changes to the interactive design of the site. He also checked the old IA so that we could compare the results with the tests of the proposed new IA.

Steps

We used a variety of research methods to help us understand user needs and to redesign IA and the support site workflow:

• We searched and analyzed data to understand how users behave on the support site and what they are looking for there.
• In particular, we looked at various data sources to determine the main tasks of users, as well as the difficulties they encountered on this site. The table below summarizes the methods we used.

We conducted a paper prototype study with users in Portland, Oregon, USA. Bram designed prototypes in Jakarta, Indonesia.

Bram redesigned it and sent it to us as PDFs, which we sent to the office supply store to get it printed in wide format. Due to the time difference of 14 hours, most of the time the team could work around the clock, without stopping.

By revising the design between test days, we were able to promote the key pages in the design to 7 versions in just 2 weeks of testing… The prototype development time, including testing and finalization, took over 9 weeks.

While not every project can test 7 versions of a project in 2 weeks, this example is certainly proof that usability can be flexible (be it a lowercase A or an uppercase A) if the team is efficient enough and uses fast research methods.

Redesign results

Reducing the number of support questions by 70% means that forum staff are less busy and can respond much faster.

After testing the prototype and prior to implementing the new design, Mozilla implemented a temporary quick link navigation element on the homepage to see if the number of forum questions decreased by direct access to the most requested content. Quick links are often a workaround for poor navigation structures, so we do not recommend them, but in this case it was important to test these key research findings at scale before implementing a new navigation scheme.

As a result of the website redesign and content improvements, the Mozilla Support Help Documentation has been expanded to adequately address the hottest questions we have found. As articles on topical and specific issues are now more accessible, visitors ask fewer questions and their questions are now more specific.

Providing easy access to the most requested information has resulted in the volume of new reference questions on the forum immediately decreased by about 70% (from 7000 to 2000 questions per month), which allowed the forum staff to improve the quality of service. A month after the UX operations, Mozilla has revised the support documentation to make it more accessible, and ongoing data analysis allows you to change the hot topics as needed.

Lessons learned

Support sites consist of answers to frequently asked questions, this is their purpose. By periodically analyzing various types of user data, customer support can document the standard responses that users need, allowing support personnel to focus on new issues and questions that require unique responses.

Data is the key to getting the resources you need… Data analysis has proven that problems exist, and analytics have proven that solutions work. It was easy for stakeholders on the support site to gain resources and momentum to implement the recommended changes. This success has led to more UX support and new hires: Information Architect and Content Manager, who are now optimizing content, navigation and search.

Long-term collaboration has helped the UX team, support professionals and developers around the world share knowledge and develop a common vision. This combination of goals and objectives, working closely to solve problems, has resulted in a great website design that helps meet everyone s needs.

Obviously, Mozilla s investment in UX design for their support website paid off, and almost immediately showed a tangible improvement. We hope this example helps you justify the ROI of user and paper prototyping – for an iterative, user-centric design.

We would like to thank Mozilla for the opportunity to share this experience with the UX community.

Learn more about ROI in our ROI for Usability report or our UX measurement course.