8 Tips for Using Online Surveys for User Testing

Surveys are one of the easiest and cheapest ways to collect valuable user experience feedback.

In fact, for every $1 invested in UX, it can increase your ROI by upwards of 100%.

However, there are some common mistakes that UX researchers and product managers make when writing these surveys and collecting user feedback.

In this post, we’re going to share some tips we've picked up whilst building Jotform to help you send better online surveys for your tests:

1. Know your testing goals

Before you write one question, it’s important to nail down what your desired goal is for this survey.

• Why are you sending this survey in the first place?
• What are you trying to accomplish with this user experience survey?
• Is there a specific feature you’re looking to get feedback on? Or is it more holistic feedback on the product?

Once you answer these questions, you’ll be better able to plan for and write the survey, and recruit the right users to take the survey. In essence, everything becomes easier.

2. Write the survey

Now, it’s time to write the survey. Instead of trying to reinvent the wheel and building a custom form and website, we recommend using a survey maker such as JotForm (we’re biased of course!), to build your survey.

While the survey questions and length will vary based on your specific goals, here are some general best practices to keep in mind:

• Be concise. The shorter the survey, the more likely people will complete it.
• Make sure your questions are specific and easy to understand.
• Include a mix of open-ended and closed-ended questions. Open-ended questions are great for identifying where you can improve and helping you see any blind spots in your thinking.
• Pay attention to formatting. The survey should be easy to read with no cluttered fields or small fonts, and it should be accessible on both laptops and mobile devices.

3. Review your questions for any unchecked biases

One of the most common mistakes that UX surveys get wrong is having multiple-choice or rating questions where all the choices are either skew positive or negative.

The wording you use for a question can prime users to think in a certain way.

For example, “How helpful is the X feature?” You might list a bunch of choices from “extremely helpful” to “somewhat helpful” to “not helpful.” This introduces bias into the equation, since you are priming people to automatically associate that feature with being helpful.

In addition, you can use tools, like Unbiasify and Gender Decoder to minimize bias.

4. Edit your survey

Once you’ve written a draft, it’s time to edit your survey. In our experience, editing your survey can take longer than writing it. This is because you want to make sure that the survey is cohesive and that every question ties back to your original goal.

Pro tip: As we alluded to earlier in this post, the shorter and more focused your survey is, the more likely people will fill it out.

5. Recruit enough testers

Once you have the survey ready to go, focus on getting enough survey responses so that you have a statistically significant sample.

The first step is making sure you’re recruiting the right people for your survey. This means putting together a plan to market your survey, allocating enough resources and/or budget, and potentially offering an incentive to get people to fill it out.

6. Make sure any incentives you offer don’t skew the results

Recruiting a large enough sample size that’s statistically significant is tough. This is why many companies bribe users to complete their survey by offering participants a $25 Amazon gift card or some other small perk.

While incentives can work extremely well, the types of people who take a user experience survey for the monetary reward can be very different from the people who actually use and get the most value out of your product. This is especially true if the reward is too generous or isn’t aligned with your target customer.

For example, if you run a SaaS tool that makes it easier for software engineers to deploy code faster and more accurately, and you want feedback from engineers, incentivizing them with a $25 Amazon gift card will likely flop since most engineers make six-figure salaries. A better incentive might be including them as beta testers or holding a one-on-one call with a product manager to give feedback.

7. Analyze your results

Once you have enough responses, categorize and report on your findings. Identify any trends you find, prioritize the feedback, and then share a report summarizing your findings with key stakeholders in your organization.

Pro Tip: When you spend a bunch of time on user testing, it can be tempting to create a mega report with all of your findings. However, no one wants to read a 100-page report. Instead, it is best to keep it compact. Here is an example of a great reporting framework you can use.

8. Act on the feedback

While analyzing your findings is important, it’s even more important to apply what you learned.

For example, create follow-on tasks and record them in your product management process — using tools like Asana, Trello, or Jira.

Then follow up with the people who took your survey and let them know what changes you made based on their feedback. This not only shows them that you appreciate their feedback, but it will make it easier for you to collect responses for future user experience surveys.

In summary, using online surveys can be an affordable way to get valuable feedback on your product or service.