Building a Digital Product Testing Team from Scratch
5 min readIf you don’t give time and attention to thorough digital product testing, unresolved development problems (bugs) can soon become enormous business problems. Lingering problems with your new site, app, or software can lead to a horribly unsuccessful launch. But how exactly do you go about getting the resources to do all this internal pre-launch development testing?
If you’re very lucky, you might have sufficient budget to hire a crack team of highly experienced professionals. But, since you’re reading this post, that’s probably not the case. That’s okay. You can use your existing team to wring every last bug out of your site or app before launching it into the world.
Test in three phases, bringing in the appropriate internal team members as you go. Here’s how to do just that:
Phase 1: Did We Build It Right?
First, check that your developers built the right thing. This seems rather obvious, but sometimes the wrong thing gets built. This may be due to vague requirements, misreading requirements, miscommunication, changes developers were never informed about, etc. It doesn't matter that the code is bug-free if the thing doesn't do what you need it to.
Who Should Test
Everyone who helped design the product should test it. That includes business analysts, UX experts, content strategists, designers, and that one user or stakeholder who had loud opinions in your kickoff meetings.
In this first phase of high-level testing, it’s vitally important to set expectations. Tell everyone involved that they should only report deviations from the original design. Beware of the phrase, "It would be better if…" Those simple words can lead to terrifying feature creep. Be vigilant, be brutal. Only make changes to the original plan if you encounter an issue that will prevent your users from being able to do their job or carry out a critical task.
Pro tip: You can minimize the likelihood of running into huge user experience problems during product testing by building and testing a prototype with users before beginning development.
Once everyone is sure that the code is fully on-track with the original plan, you can move on to phase two.
Phase 2: Did We Build it Good?
There will always be unforeseen weak spots in your site or app, especially wherever users enter their own information or interact directly with functionality. The goal of this phase is to find these weak spots before your users do.
Who Should Test
Somewhere in your company lurks a person who likes to break things. Often this person is a developer from another team who loves the challenge of breaking other people's code. While this may seem like an anti-social trait to be discouraged, this instinct makes for someone who will be very good at finding the development bugs in your product before launch.
Tell this person your application is verified defect-free and that the development team is exceedingly proud of it, but they’re welcome to look for problems if they want. Then wait for the bug reports to flood your inbox.
Other good testers include that annoying person who likes to point out the flaws in things and always spots problems, no matter how petty. You probably remember them from when they replied to your email last Friday to correct your grammar. You can also hit up the person who always seems to have things mysteriously breaking around them. Their email always has some weird problem or their Word documents break the printer. They will break anything. That’s good for product testing.
Treasure these people, because they will find bugs. They will do weird and peculiar things to your site or app that you cannot imagine. Their very presence will uncover issues that your developers have written off as impossible. Fix every one of their bugs thoroughly. Keep in mind that resolving some of these bugs will require talking to the original developers, UX experts, designers, writers, etc. who worked to build the site or app.
Phase 3: What Did We Miss?
At this point you should have found the most severe bugs. Now find the smaller (pettier) things that your site-breakers extraordinaire might have missed.
Who Should Test
In this phase, you need lots of eyes. Really anyone will do, as long as they didn't contribute to the building of the product. Be sure to include non-technical people. Technical people know too much about how sites and apps should work to be truly great product testers. Other than that, the more the merrier.
Give each person specific tasks and divide testing coverage appropriately. For example, one of your testers might look at the entire product on an Android phone while another uses Google Chrome on the desktop. Someone might check all links and another might look for design consistency.
Assign one person to process all the reported problems (bugs). This person should be familiar with the project, enough to know whether something is really a bug and, if it is, who should fix it. People will log bugs that aren't actually bugs. That's fine, your bug processor should just close them quietly. False positives are much better than people not wanting to report bugs because they might be a bother. Impress on all testers that pettiness is welcome here, respected even.
Review and prioritize all bugs. Prioritization is especially important if you’re low on testing hours. If some bugs just can’t get addressed before launch, make sure it’s the low-priority ones. Set about resolving the bugs, starting with the highest priority items. Once again, some fixes will require discussions with the original team members who built the site or app to determine the best solution.
Rinse and Repeat Phase 3: No, Really, What Did We Miss?
You could stop with just one round of Phase 3. And your project deadline or budget might necessitate it. But if you have the hours, use them to make your site or app as flawless as possible. Find the grammar police, the wannabe designers, the people who could spot a flaw in paradise and let them loose to report bugs. Revel in each petty issue they find, knowing that when they can’t find any more bugs, your users likely won’t be able to either.
Launch, Baby
Once all bugs are fixed to everyone’s satisfaction, launch your digital product. Rest easy knowing users won’t run into engagement-stopping, lead-killing, sales-squashing issues when they interact with your organization online. Congrats! Push this baby into the world feeling good about what you’ve done for your customers and for your business.