Test everything: Why being wrong is right.


Full reading time 8 min.  Bold reading time 1 min. In our competitive world it’s sometimes difficult to admit to ourselves that we are not the experts of the products and services that we design: our customers are.

Sure, we know how these things work technically, we understand the maths behind them or all of the features that they have. What we don’t always know is how people use these things in the real world, how it makes them feel, and what they believe would make them better. That’s why we believe in testing everything!

Whether it’s an idea, a document or a website journey, using a well-designed testing approach gives us a far better understanding of what the customer - who should be at the centre of any business strategy - is thinking. Far too often we rely on our assumptions and experiences to dictate how we go about dealing with certain challenges that we face, instead of opening up a conversation with the people whose challenges we are actually trying to solve.

Additionally, there seems to be slight confusion as to the value of qualitative vs. quantitative data. With the absolute abundance of quantitative data – the stats and figures - of what people are doing, and considering the ease and cost-efficiency of getting that data, it’s tempting to devalue qualitative research. Qualitative research is invaluable in taking steps towards ‘quantifying the unquantifiable’. Just as valuable as data, qualitative research provides the insights, opinions, understandings and motivations of clients allowing us to more accurately and holistically ‘explain’ the numbers. The issue is that while numbers and graphs are really good at telling you what people are doing, they struggle to tell you why. And, at the end of the day, it’s understanding the ‘why’ behind someone’s behavior that will enable us to design a better product or service.

There are a whole range of different types of testing that one can do, and each of those ‘types’ have a multitude of different names. Furthermore, each is used to illicit different types of information with differing methodologies, but they all have one thing in common: they rely on having truly open and honest dialogues with people. This seemingly simple task is, in and of itself, a challenge.

Think about your friends’ response when you ask them how you look? Is it honest? Is it constructive or insightful?

To get useful, detailed and honest information and opinions out of people is a lot more challenging than just asking them a straight question. You have to engineer your conversation in such a way that you get the information that you are looking for without them being explicitly aware that they are giving it to you. That may sound manipulative, but it’s not at all. It’s about recognizing that people, particularly in unfamiliar circumstances or engaging with unfamiliar faces, as in a test environment, are going to try and tell us what they think we want to hear, and to not be confrontational. That’s why we need to design the environment and conversation so that they drop their defenses and tell us what they are really thinking.

We, at Gravity Ideas, have three core types of testing that we tailor based on our individual client’s challenges and what it is they are looking to achieve. These are assumptions testing, concept testing and experience testing. What those types are, how they differ from one another, and when we use each we will get to below.

You might be asking yourself but, wait, those names don’t mention people or users? That’s true they don’t, and that is because, fundamentally, we are not testing people or ‘users’, we are testing ideas, products and services. This is a very important distinction to make, and something that differentiates us in the field, as the idea of user-testing implicitly puts people ‘on guard’ counteracting our impartial, honest objectives from the outset. These individuals are our partners, not our project.

Before we do any testing we go through a process of setting objectives, ascertaining what the key information is that we want to get out of the sessions, as well as working out who the best people would be to recruit and talk to in order to ensure a diverse, honest and as accurate reflection of the given client base as possible. Below is a brief overview of the types of testing. We will delve into more detail on each in future posts.

Assumptions Testing These are either one-on-one or group discussions exploring a particular topic, idea, product or service and the various assumptions that you or anyone else in the business might have. These sessions are usually quite informally structured in order to get broad, open and honest answers from all of our participants without biasing their focus or feedback. We use the term ‘guided discussion’ as the idea is not to ask directed questions, but rather to get people to talk freely about the various topics without explicitly asking them. These sessions help us to understand what people’s sentiments and feelings are towards particular services, experiences or products.

Concept Testing This is the next stage of testing. In one of these structured sessions participants will be shown printed drafts of a document, form, website or product (a concept) and we gently guide a discussion around those. This allows us to identify any areas of confusion that they might have with the concept, any design flaws and where they are focusing their attention, for example.

Experience Testing This is the final step in the testing process, and is more commonly called ‘usability testing’. These sessions allow us to understand users’ experiences of a finished document, website, app, product or service. These are more focused and task-orientated sessions, and we spend the majority of the time observing rather than engaging. This is to stimulate more organic engagement as far as possible. The end result here is that we identify where people are getting stuck and possible ways to resolve these, ironing out any confusion and glitches before these products hit the market.

These are just three of the testing and business experiment services that we offer. We’ll expand on each of these, as well as our other services such as A/B testing, in-office experimentation and general business experiment design in future posts.

If you only remember one thing at the end of this piece, it’s not “Come to Gravity for testing”. It is: be brave. Be comfortable with being proven wrong. No, wait, be excited to be proven wrong. If you embrace letting your customers guide the development and improvement of your products and services, you will not only end up with a fundamentally better result, you will also save your company a huge amount in misguided time, money and effort.

Please get in touch via our website, Facebook or Twitter should you wish to find out a little more.