1.2.4 The Lean Approach: Hypotheses and Preliminary Experiments


Today we’re going to talk about how to structure your hypotheses for the lean approach. As well as how to start conducting preliminary experiments to test these hypotheses. If you’ll recall, the lean methodology takes you through a series of steps that combines business hypothesis driven experimentation. Iterative product releases and validated learning. We talked about what an MVP is and today we’ll specifically talk about generating hypotheses and how to conduct early experiments. Crafting good hypotheses for your startup is difficult, one of the most important things to remember about your hypotheses, is that they should be testable. You need to properly validate or invalidate what you’re doing.
0:40
Make sure to also write down your hypotheses. It’s amazing how people do it, but the simple exercise of writing it down is significant. Try structuring hypotheses by starting out with this phrase I, or we, believe, and then try and finish that statement. Remember the statement has to be testable and it has to have the potential of failing.
1:00
Here’s an example of a hypothesis that you might have for start up TW, a content site for start up news in Taiwan. You might say something like, I believed that US based investors will read start up TW on a regular basis. Because they have a growing interest in the Taiwan’s startup ecosystem. The basic structure here is, I believed, that a target market will do this action or use the solution for a certain reason.
1:29
Again, we might say something like, I believe that US-based investors will read startup TW on a regular basis because they have a growing interest in the Taiwan startup ecosystem.
1:41
However, you might think to yourself what does read on a regular basis mean? Is that something you can validate or invalidate? Does regular basis mean daily, or weekly, or even monthly? Therefore, it’s important to be able to validate with a specific goal in mind. So instead a better hypothesis might be, I believe that US base investors will read at least one article per week on start up TW because they have a growing interest in the Taiwan start of ecosystem. That’s a more well structured reasonable and testable hypothesis.
2:16
Next, once you have your hypothesis you want to think about conducting preliminary experiments. Your preliminary experiments are about creatively and also relatively cheaply testing your hypothesis and your operating assumptions. This allows you to explore both the problem and the market as well as possible solutions. For example, let’s take experiment one. Here’s a preliminary experiment that you could run. Blog publicly about what you’re doing. This helps you get qualitative personal feedback. This can help you develop and make sense of quantitative information like measurable outcomes and metrics that you gather later. You can also include face-to-face interviews with customers to supplement this feedback. Another preliminary experiment you can run is asking open questions on online discussion tools such as Reddit. This is another form of qualitative feedback. Many consumers want to provide feedback but they just need to be asked. By going on a Q and A site or forms like Reddit, even one specific to your industry. You can start asking questions. You can begin with broad questions like, how to people solve such and such problem? And then, you can get more specific, for example. Asking, what accounting services are currently used by venture capital and private equity funds? This will give you information on both competitors and customers.
3:36
You can also create surveys and experiment with monetary and non-monetary incentives. Sending a questionnaire to your customer base, or potential customer base, is a great way to get feedback and discover needs. You can try using incentives, such as offering $50 off your product when it first hits the market. If people sign up, and they’re eager to use your product, even though it doesn’t even exist, it’s further validation of customer demand. You can also directly collect preorders. Crowd funding platforms like Indiegogo and Kickstarter have made it much easier to measure market demand for a product or service. Finally, you can run test ads. You can use things like Google Ad Words to create ads that take viewers to a page which solicits. Email sign-ups and possibly even pre-orders. You can test which ads are more effective and get information about what customers might be most interested in and what drives traffic most effectively. As we’ve seen hypothesis should be testable and falsifiable, you can creatively test your hypothesis through a number of preliminary experiments. You want to try a variety of different experiments to make sure you’re thoroughly validating or invalidating your hypotheses. Thanks for joining this lecture.

Jim Rohn