We all have exciting ideas to enhance a product. During my time at Jooble, one of the largest job search aggregators, I often received suggestions from colleagues like, “What if we add a CV review feature?” or “What if we introduce interview practice tools?” These ideas sound great—they could add value for job seekers and improve the platform. But as we often say in Product, "it depends." Not every idea will work, and guessing can only take us so far. Instead, it is worth validating hypotheses with data.

One effective method is the Fake Door test:

However, what if you have 20+ beautiful ideas to validate but need an entire sprint of development effort just to launch one Fake Door A/B test, followed by three days of data collection? That’s 17 days per idea, which means nearly a year to validate 20 ideas. Clearly, this isn't scalable.

Jooble has a strong data-informed culture with an ownership mindset. Everyone is encouraged to bring their ideas to improve the product and make it more valuable to job seekers. To support this, we envisioned a solution that allows anyone to validate an idea in just three days instead of 17, with zero development effort. Our goal was simple: empower everyone to test their ideas quickly and determine their potential value without wasting time or resources.

Discovery of the Solution

Before we go to the solution, let me walk you through the process we had in place for validating ideas and launching Fake Door tests. At the time, Fake Door tests were treated as Product Backlog items. This meant they had to be prioritized, refined, developed, and tested by the development team. If you had an idea to validate, you first needed to convince the product team to prioritize it for development, and only then could engineers and QA invest their time. The product team was often hesitant to take on new ideas because they inherently carried costs without clear evidence of their value. That’s why they asked for market research and competitor analysis to back ideas with data. This complexity discouraged people from proposing new ideas and going through the process altogether.

We asked ourselves: is there a way to launch a Fake Door test as easily as making a cup of coffee? We started a product discovery journey and uncovered some key insights that shaped the solution.

Assessing risks was another critical step in our discovery process. Risk assessment is essential because it helps identify potential challenges early, that can impact achieving our goal. Are there any limitations to enable Fake Door Test launch for everyone? Fortunately, we already had tools in place to track data for newly launched A/B tests automatically and display Test Results.

With this clarity, using the “How Might We” framework, we formulated our hypothesis: How might we enable anyone to validate their hypothesis themselves in less than 5 minutes by streamlining the launch process and leveraging automated tools? This challenge became the foundation for building a simpler, faster way to validate ideas.

The Solution: Building the Component

With a clear hypothesis in mind, we started researching tools that would help us build the solution efficiently. After evaluating various options, we chose Strapi and TeleportHQ as our no-code tools.

This combination formed the first iteration of Easy Widget.

However, we soon encountered an unexpected challenge: The user guide we created to help people launch Fake Door tests with Easy Widget was lengthy and complex. Users were reluctant to engage with it, fearing it was too complicated and that they might break something on the product website.

While engineers could now launch widgets in days instead of weeks, we still hadn’t achieved our goal of enabling everyone to launch tests independently.

We went back to the drawing board, reflecting on how we could make the process even simpler.

This led to Easy Widget 2.0, where we replaced the lengthy documentation with a short Loom video. The simplicity of this approach resonated with our business users—they could now launch Fake Door tests independently in just a few minutes without technical assistance.

The Results

With Easy Widget, the results were remarkable:

Sounds great, right? But is it really everything? There’s always a hidden cost. One critical concept in Product Management is Cost of Delay—the economic impact of postponing a product feature. This includes lost revenue, missed market opportunities, and increased competition. Time spent not validating ideas matters because it slows down innovation and delays potential revenue streams.

By enabling Easy Widget, we were able to validate over 10 hypotheses in just two months. Interestingly, we backlogged 80% of them because there was no interest from users in adopting those ideas. However, this process also uncovered a new market opportunity that allowed us to create new revenue streams and increase product revenue significantly.

Additionally, Easy Widget empowered marketing teams to launch promotional banners independently—without engineers—to drive traffic to other Jooble offerings. This allowed us to leverage the Jooble ecosystem effectively by tailoring solutions based on country-specific user behavior. By customizing promotions for different regions, we were able to boost engagement and further increase revenue across multiple markets.

How Other Teams Can Implement the Same Solution

For anyone interested in building a similar solution, here are some best practices based on our lessons learned. While you might choose different tools than Strapi and TeleportHQ—and that’s perfectly fine—remember that tools are just “means” to achieve the desired outcome. What truly matters are the core principles that remain unchanged and are essential for successful implementation.

Do the Homework

Once the problem is defined, dive deep into data and gather feedback from people to understand how the solution should look. Explore the challenges users face today and how these impact business results. Doing this homework will give you clarity on pain points and potential impact. Look for patterns—analyze the types of tests users are launching and identify similarities. This will help you design the simplest solution right from day one.

Define Success Criteria and Stick to Them

Clearly define what success looks like and stay committed to it. For us, the goal was to enable everyone to launch a Fake Door Test in just 5 minutes, and we iterated as much as needed until we achieved it. Having clear success criteria keeps your team focused and ensures you deliver meaningful outcomes.

Make It “Grandma Simple”

This might seem obvious, but it’s worth emphasizing: simplicity is subjective. Based on our backgrounds and skills, what feels “simple” can vary greatly. For a technical person familiar with coding, configuring widgets using Strapi and TeleportHQ might feel straightforward when following documentation. But what if we asked my grandma to do it? The goal should be to make the process so simple that even she could run a Fake Door Test without hesitation. Always aim for maximum simplicity—this is key for adoption.

Engage Target Users From Day One

Involve your target users in testing as early as possible—even if the solution is still running on a development machine. Early feedback is invaluable for identifying challenges users might face and making improvements before launch. This iterative approach ensures your solution addresses needs effectively.

Conclusion

In conclusion, I wish I could give you a silver bullet for optimizing A/B testing processes, but the truth is—it depends. Based on my experience, one approach that has consistently helped me while working on multiple initiatives to optimize A/B testing strategies is going back to the question: “What do we want to achieve?” and working backwards from there. This is a powerful approach because it starts with a clear vision of the end goal and maps out the steps needed to achieve it.

Start by mapping the end-to-end flow. Conduct a user journey mapping exercise to identify every step in the current process and how long each step takes. For example, in my case, I imagined myself as a business user who wanted to launch a Fake Door Test to validate an idea. I walked through the entire process step by step, as if I were the user. This exercise helped me pinpoint the main pain points, inefficiencies, and areas of waste in the process.

When I spoke with actual users, their feedback validated my assumptions: the main issue was that it took too long and required development resources. With this clarity, we were able to formulate a hypothesis and start exploring potential solutions.

By clearly defining the goal and performing user journey mapping, we identified inefficiencies and optimized the process effectively. This method not only helps uncover hidden bottlenecks but also ensures that any solution you implement directly addresses real user pain points and aligns with your ultimate objectives.