Today we will talk about relevant tools for testing product hypotheses. I’d like to indicate straight away that there is no silver bullet in this matter. Validating the viability of a product or feature is almost always a lottery, even with the most pragmatic data-driven approach backed by personal experience and faith :)

Due to the specifics of my experience, the bias of the article will go more towards digital (mobile, desktop and the Internet).

...

Validation on the idea stage 

A hypothesis is an assumption or idea, the implementation of which will bring us a certain value. What do we consider a validated hypothesis? Is it the fact that the consumer needs this feature or product? Or is it them willing to pay for it? Or maybe all together?

Context is important, and it includes a large number of components:

1. Your advertising budget and opportunities

2. Marketing channels

3. The target audience

4. Core-value of the product

5. The complexity of implementation (RICE/ ICE Scoring/ Lean Prioritization methodology

… there is a long list of variables that affect the result

30. Red/blue ocean

I think most readers are familiar with the concept of unit economics. But I’ll clarify just in case: unit economics is a method for determining the profitability of a business model using assessment of the profitability from the sale of a unit of goods. It plays an important role in the formation of a hypothesis, because we strive to obtain value by realizing a hypothesis. Unit economics will allow us to understand the economic feasibility of the hypothesis.

So the formula is:

Correct hypothesis = Unit Economy Works х Your product context

You have to take context into account when predicting the success of a hypothesis. Even if the context is not complete, you need to work on its clarification, make this process regular and timely notice if the product / feature has lost its relevance. Time to Market is a very important point, as the market won’t wait for you. You don’t need to stick to hypotheses, if it is difficult / long — let it go. Testing product / feature hypotheses should be done quickly, and so should be giving them up.

...

Search and selection of testing tools

After validating the hypothesis, you need to decide on the testing tools. You need to understand that the tools are different, because the product hypothesis and feature hypothesis have different contexts. In the context of a feature, we already have a finished product that customers can reach, and we want to understand the feasibility of its implementation. The product hypothesis is more complex — it tests the viability of the idea in your context.

Validation is key. It’s a check for necessity, while testing is a check for viability.

...

Product hypothesis testing tools

1. Fake Store Page (Splitmetrics/ Storemaven/ Tryitapp): testing tools for mobile products at the landing stage in the App Store / Play Market

Aim: Monitoring the competitiveness of a new product.

The main idea: We reproduce the layout of the original pages and it looks identical to the original one. We drive traffic and look at the indicators depending on what you want to check. If you have conceived a product in the “red” ocean, it makes sense to reproduce the main competitor’s page in any of the services, look at the latest UA creatives in the Facebook Ads Library and use them to buy your target audience, having received approximate data from this part of the competitor’s funnel. This will help you get a foundation for analysis and comparison.

In order to interpret the results on small volumes of traffic correctly, it is more correct to test one thing. For example, if you are testing creatives, the landing page is the same for all creatives. Conversely, if an effective creative is identified, then you can test different pages.

If you enter the so-called “red ocean”, but do not know the situation on the market and are not sure whether you can compete on it, you can make a copy of the competitors’ page and also drive traffic. This will help clarify CPI, CTR and other important metrics, and, accordingly, plan the budget and economic model of the product.

2. Pre-order in App Store and Pre-register in Play Market

Aim: ASO check. Collecting leads for soft launch.

The main idea: You start to quickly make a software product based on ready-made solutions + hacks. Make a minimal set of features to get into the app store.

This is a rather non-trivial way of testing product hypotheses for mobile. The only drawback is that the fee for such a test = development + paid developer account + traffic. The main goal of development here is to go through a visual review from the store, breaking the Apple / Google guidelines: you can achieve this through a dummy wrapper — no one except the reviewer will see this. If the hypothesis is not confirmed, withdraw it from sale.

The bonus is that you have a live page for several months in the respective app store. This gives you small organic traffic, and the ability to drive traffic to an existing page (albeit without optimizing campaigns), and collect pre-orders, which will turn into installs at the time of release. The downside is that analytics from Apple / Google delivers information with a delay, i.e. experimenting with UA quickly won’t work.

3. Zero-code (no-code/ low-code) tools

Aim: Test hypothesis very quickly. Getting the first clients, adjusting the strategy.

The main idea: We create a template software product in a visual interface. We release it to the public → track metrics → make a decision on further development or develop the product, in case we are not limited by zero-code tools.

Zero-code “movement” is dynamically growing, new tools and services are constantly appearing. By 2020, a wide range of tools is already available not only for testing product hypotheses, but also for creating MVPs that will help you understand whether it is worth spending resources on full-fledged development. In some cases, it allows you to create and develop complete products. The advantage of this approach is that a digital product can be created with little or no programming knowledge.

It is important to have information about most of the popular tools on the market, to know their features and capabilities. By combining zero-code solutions, product hypotheses can be tested very quickly and cheaply. Here are some examples of services to explore:

4. Fake product launch (Product Hunt/ Kickstarter/ Landing)

Aim: Testing the viability of a product hypothesis without development.

The main idea: Testing is carried out by presenting a non-existent product to your target audience (or investors): you can accompany this with public speeches, presentations, demonstration of a prototype. It is easier with digital products, where the main emphasis should be on the visual part and marketing: a high-quality promo video, logo, identity, landing page, etc. The ending of this story should be some kind of milestone, for example, getting “Product of the day” on Product Hunt or completing a fundraiser on Kickstarter.

...

Feature hypothesis testing tools

1.Splitmetrics/ Storemaven/ Tryitapp Applications

Same set, different approach.

Aim: Testing the reaction of our target audience to page changes in the App Store / Play Market.

The main idea: We add information about features to visuals and description in the store and test them against the original page. Engagement Rate, CTR and CPI will help determine how our target audience reacts to the planned feature, and whether it is worth planning its release.

2. Online customer discovery (Typeform/ Yandex Vision/ Google Forms)

Perhaps one of the key steps in the custdev methodology.

Aim: Correcting controversial / weak hypotheses, getting insights.

The main idea: We prepare questionnaires and, using the necessary tools, deliver them to the target audience. It is important to remember that this tool is only a part of the research that helps to gain insight, discard irrelevant topics, or vice versa — crystallize directions from a wide range of topics. Typeform, for example, allows you to buy and optimize campaigns through Facebook Pixel to help you optimize your UA campaigns. This tool is great for the content part of your product.

3. Fake feature, until you make feature

Aim: Checking the users’ need for the feature.

The main idea: If a feature is difficult / takes long to be implemented, it makes sense to make a transition to some functionality and measure the conversion without implementing this functionality. As a bonus — make a mini-funnel and ask to answer the question / questionnaire inside. Even without reading the answers, it will be possible to conclude: if many users have spent time answering questions and leaving an email, it is advisable to implement a feature.

4. Good old A/B

Aim: Checking the users’ need for the feature.

The main idea: Adding a feature to a product can have positive or negative results. It happens that you have been making features for a very long time and you end up getting iTunes :) The good old A / B test will help you with testing features.

A/B testing is a research method in which two variants are compared to one another. In our case, there is a working product and the hypothesis that users need the feature and adding it will positively affect the perception of the product and user experience. We divide users into 2 conditional segments: we add a new feature for one of them (we test the hypothesis about its implementation), and do not for the other, where we leave the product the same → we then analyze the metrics and draw conclusions whether to keep the new feature.

...

Short conclusion:

1. Tools: search, combine, invent your own ones

2. Do everything as quickly as possible

3. Test one thing at a time, don’t blur focus

4. Let the “dead-born” hypotheses go

Product to all!