How to guide your design with data
There is no silver bullet in analytics, and nobody can tell you what metrics are important without understanding the business context you operate in. Metrics such as site speed, device compatibility, session duration and pages per session can be valuable indicators of something being done right or wrong, but these alone do not allow you to make truly informed decisions.
If everything depends on your unique circumstances and there are no clear-cut answers, then how do you even begin to put data into practical use? That’s the exact question I’m hoping to answer in this blog post. Below I’ve shared four practical tips that have helped me apply data to design problems, and I’m hoping they will help you as well.
1. Utilize data as a team
The use of data should start from the very beginning of a project. Designers should form hypotheses on their design decisions when they feel unsure whether their design is the best one possible. They should then follow through with analytics to see how their decisions actually perform in the real world.
Designers are in a great position to form hypotheses: they are experts in asking thoughtful questions and applying their expertise about users to guide the analysis of a design experiment. Designers understand what matters from the users’ perspective, and they know how to get meaningful data to guide decisions. This is why you cannot separate data analysts and designers from each other; they need to work together in close collaboration to effectively share information.
2. Form a hypothesis, share it with your team, and follow through
A good hypothesis is essentially an understandable, testable, well-defined prediction of what you think the result of your test will be. It explains your belief about how your design change will affect your service or product. Working together with an entire team to form a hypothesis is important to ensure that your hypotheses are based on solid foundations.
A hypothesis statement can take a form such as:
If we change X, users will do Y because of Z. This will make metrics A and B change in ways C and D.
As you can see from my example, defining the actual metrics you think are affected is a crucial part of creating a good hypothesis. It helps in sharing your beliefs with your team members, making sure that everyone is on the same page about what a specific change should do.
3. Dig deeper to find real insights
Split your users into cohorts or segments – that way you can get a better understanding of your users’ behavior than you would if you only observed them as a single group. Consider the different needs and expectations of new and existing users. Choosing to sample one versus the other can give you very different data. This depends on whether you’re looking to optimize the experience for your existing users or trying to grow your business by acquiring new users. Also, consider your most extreme users: those who deviate from the norm and those who never behave as others do.
Using quantitative and qualitative data together is the best way to get to the truth, because sometimes numbers tell us very little without the stories of people behind them. While A/B testing is a valuable tool in revealing design flaws, it only reveals parts of the big picture. Qualitative and quantitative data should be used together to cover everything and to fully explain your visitors’ behavior and motives. Using multiple perspectives to reach an all-encompassing understanding is sometimes called data triangulation: the use of multiple methods of collecting data to arrive at a more thorough understanding of your product or service.
4. Give your users time
Humans often have a predisposition against change, so you have to give your users time to get accustomed to changes. It can be scary to see your metrics drop due to an update, but this can be expected in the short term.
The more tests you’ve run and the more research you’ve done, the easier it gets to spot when things are looking off due to change, and when it’s occurring because something is actually wrong. Whichever the case, learn from it and apply the knowledge when making further optimizations.
Data will never be perfect
You can never be 100% certain of the correctness of the decisions you make based on data. Assessing data and designing measurements is an inherently subjective activity. It all comes down to deciding what to measure, how to measure it, and how to assess the outcome. It is important to stay mindful of human bias in the process.
However, partial data usage is still better than no data usage. Without data, all the decisions you make are based on hunches and guesses. Even though data does not give a perfect description of the world, it certainly helps to understand it much better.
Interested in our way of doing data-guided design? Get in touch with us.
