A product discovery case report: AMBOSS ❤️ Anki
Wanna know how to do product discovery? The first step I took was to Google it!
You’ll get 1.340.000 web results in 0.52 seconds. I did this a ton of times, and read anything that seemed even half decent. Fast forward a couple months. After reading, reading, reading, I felt confident that I was ready to rock and be the next Steve Jobs.
But instead I found myself in this conversation …
- Me: We’re not gonna build this, right, Thiago?
- Thiago: No. This is too costly to build, we’ll have tons of legal issues, and we’re not going to simply migrate this entire community.
- Me: Right. No one will use this.
There is a lot of talk about product discovery techniques. But there are too few real case reports. I found and enjoyed some exemplary and real-world ones, but overall there’s not a lot.
I want to share a real-world story of finding a product that people love at AMBOSS. It was my initiation into product discovery — and luckily a success.
The situation
Zoom back to 2018. Up until then, AMBOSS was mainly offering a multiple choice question bank for medical exam candidates in Germany, and had just started entering the US medical student market. AMBOSS was growing quickly. We faced very strong competition and trusted incumbents in the US, and needed to deliver innovation to win customers.
Our campus team was on the ground traveling to most US med schools, giving demos of the AMBOSS product, and listening to customers. There was one thing we heard over and over again from med students:
“If you also offer a solution to do flashcards, you’d have a killer product!”
Tobi and I were the two software PMs at that time. We knew that our biggest competitor didn’t offer anything to serve this need yet! But how should this product look? How would it differentiate from the existing flashcard offerings available in the US?
Tobi and I were new to product discovery, and absorbed all the books, blog posts, and podcasts we could. We were keen to apply some of these new techniques. And we were lucky — we discovered a solution that surprised everyone, including us. Even today it’s still growing and continues to delight our customers.
I’ll explain the different stages of this product’s discovery, and of how the discovery process increased the odds of its success.
A hackathon (Sep 2018)
Our new CTO, Elmar, brought up that we should regularly have hackathons.
One of the (multiple) winner projects of our first hackathon was the technical precursor of what later turned into our Flashcard solution: a Chrome Extension that would recognize all medical terms on a website and link out to the specific anchor on AMBOSS that deeply explains this term. We didn’t launch the Extension for another year and a half, but the idea was the initial spark for our Flashcard product.
Annual planning (Dec 2018)
We got to the end of the year and knew that we needed to deliver a solution involving flashcards. Structured discovery was new to AMBOSS, and we would usually just “spec it out”.
But we knew that we knew too little about the topic. We talked to one of our founders, Sievert, about the planning slides we’d just done.
They felt lofty, made up and with a ton of risky assumptions and false security. We asked Sievert for more time to explore Flashcards before committing to anything. Sievert understood, gave us more time, and so we moved on.
Internal stakeholder interviews (Jan 2019)
Tobi and I needed to better understand how medical students currently worked with Flashcards. We didn’t yet have access to many US students, but we could easily talk to our US employees, many of whom had studied medicine before. One of them was Christopher.
He had studied medicine and was a flashcard geek. He introduced us to the fact that most US students wouldn’t use any of the paid software options specifically built for medical studies. Instead they would use a software called Anki which on desktop devices is free and open source.
It would allow them to create flashcards and review them based on spaced repetition. There was an enormous community of Anki users that uses flashcard decks. The best decks by a few key deck creators would be shared among the crowd.
Tobi, who has a technical background, immediately thought about that merging our hackathon winner with Anki’s technology was totally an option. Christopher was surprised and immediately amazed by the idea. We were onto something!
Modeling different concepts (Mar 2019)
We felt we were onto a great opportunity! Wouldn’t it be smarter to create value for an already existing community instead of trying to force a new solution on anyone? We were amazed by our idea, but knew we shouldn’t fall in love with it too early. We should explore at least some other options. Then test what works best with users.
We followed Dan Olsen’s approach to defining concepts based on different user need patterns. We first collected all customer needs we knew from users, stakeholders, and competitor solutions. Then we assigned them to Kano-Model categories (delighter, must-have, performance benefit).
Then we constructed 4 distinct versions highlighting a different patterns of users needs. If we tested these concepts with users we could easily understand which needs are underserved and which aren’t. Here are the 4 concepts.
- Question bank on flashcard steroids: integrating our question bank product with flashcards
- Knowledge to learn-items transformer: allowing users to create flashcards from our content
- Anki deck viewer: our initial idea
- Motivational learning community: Making learning a social and motivating experience
In a next step I sat down and reviewed this matrix with our user researcher, Angelique, and our designer, Barik. Our goal was to quickly create very high level visual concept mocks for the four concepts.
Each concept was designed in a way that would visually outline the underlying user needs without being specific. Keeping it vague would keep people talking, ideating, and help us find out what they really need (not what they want!).
User research trip (Apr 2019)
We knew that we needed to get out of the building. We had to test these concepts with US med students on site. AMBOSS flew us out to our New York office in April 2019, where we talked to about 40 med students. It was one of the most eye-opening experiences I had in product management. After only five days, I’d learnt more about our users than I had in all the years before.
We’d show users this screen first, ask them to read it and tell us which concept they’d like to dive into first and why. We’d then dive into each concept and return to this page to get the 2nd, 3rd, 4th favorite concepts. This gave us a good orientation on desirability.
This is what one of the more detailed concepts would look like. Several parts of this visual would highlight certain needs from our needs-matrix. Notice that it feels like we’re showing actual features but we carefully prevented being specific to keep people generating ideas and voicing expectations.
Also notice that we prevented using the term Flashcard and instead used “Create a fact” to not anchor people. We’d make sure to keep people explaining to us how this would work, how they’d (not) benefit, what they’d wish for.
This is what the Anki concept looked like. This is pretty close to how the final product actually works!
After talking to the med students, it became very clear what we needed to do. The Anki concept was the winner!
Other, seemingly valuable ideas failed, like linking our question bank to flashcards (few people saw any value), or offering a social flashcard game (bad idea: US students are already under high peer pressure and don’t need games!). So we flew home knowing what to do.
Design workshop (May 2019)
User insights had given us a lot of confidence already. But I felt that there might still be a different solution we’d need to explore. Somehow, the Anki integration felt like a … small solution. This was just my product ego asking me to do more, as it turned out later.
To generate more ideas, and also to align stakeholders, Angelique and I conducted a two and a half day design workshop with the goal of diverging and converging once again on different options.
At the end of the workshop, we had a clear winner — an AMBOSS online Anki deck viewer. Users would upload their deck to AMBOSS and answer questions inside our product. We’d start moving that community to AMBOSS and would have full control over how much to integrate this with our existing education products. This was huge! Our product ego was satisfied.
Making a tough call, product intuition (June 2019)
The vast majority of product efforts fail not because of demand, but because they can’t come up with a solution good enough to get people to switch. (Marty Cagan, source)
Two days after the workshop, I sat down with our Head of UX at the time, Thiago. We discussed the workshop outcome. We had a winner and would now go on to spec out the solution, do a couple iterations and continue testing and refining this with users, right? RIGHT? Well, here’s our dialog …
- Me: We’re not gonna build this, right, Thiago?
- Thiago: No. This is too costly to build. We’ll have tons of legal issues, and we’re not going to simply migrate this entire community.
- Me: Right. No one will use this.
Product intuition kicked in and killed our product egos.
Proof of concept and a lot of luck (June 2019)
Despite our decision to move against the winner idea of our design workshop we were still amazed by the idea and the insights around the Anki opportunity. But we needed help and creativity in order to find a technical solution that was feasible and would delight users. This is when luck kicked in twice!
- We needed someone who deeply understood Anki’s software and the medical community around it. I looked for different people on Twitter and came across Aristotelis, who is deep into the development of Anki extensions. Ah, and he was a German medical student using AMBOSS every day, and a key opinion leader in flashcards and spaced repetition!
- We needed someone who deeply understood AMBOSS, had a lot of technical expertise, the urge to explore and be creative, and would know what could be made possible technically. Luckily, Yann, a senior software engineer at AMBOSS, joined the project and brought exactly that to the project.
Yann, Aristotelis and I had a call where we discussed three different solution options before converging on the idea to build an Anki extension that would recognize medical terms in flashcards on the fly, offer quick AMBOSS explanations for these terms and link out to AMBOSS articles. If we could do this, we’d deliver tons of value to users, and give them a good reason to pay for AMBOSS.
A couple of days later Aristotelis was on his way to Berlin. He and Yann developed a proof-of-concept in just three days!
Final demand test (August 2019)
This was the moment when my work on this project was done. The technical and product journey from here is worth at least 2 own blog posts! I’ll briefly summarize.
The project was graspable now, internally the value was very clear, and it was easy to find more people inside the company who were excited to move the project on.
We agreed it would be good to conduct a final demand test to be really sure that this straight forward solution would have a big enough effect. One of our product managers, Sam, and his team built a landing page showcasing the product and offering early access via mail. The sign up rates blew us away and people were raving about the solution, posted on Reddit and couldn’t believe it.
Launching the V1 (September 2019)
When launching the solution we literally got love notes by users and were approached by some key community members like the folks from AnKing. Find a couple of love notes in the Reddit posts here and here.
Anki & AMBOSS today (February 2022)
Today, we have a group of internal contributors that owns and drives the work on the Anki project. The group is driven by Yann and Aristotelis but other members joined, like Anna, a community coordinator who worked on this project and now is a product marketing manager. They’re all doing amazing work on our Anki integration and is causing frequent jaw dropping in our product reviews.
After the product’s initial success the group further iterated on the business model (e.g. switching to freemium), product value (e.g. offering images), internationalization (e.g. offering a DE version), partnerships (e.g. dedicated decks by key community members) and distribution (e.g. offering a mobile solution). To date, our Anki solution is still a strong entry channel to AMBOSS for new users and helps soon-to-be physicians worldwide to better retain medical knowledge.
Key product lessons
There are many lessons I remembered and acknowledged while writing this post. Here are my key learnings.
- You can sense product-market-fit. You’ll notice when you’ve created a lovable product. Eyes will open, people will reach out, FOMO will kick in, and people are going to put skin in the game by giving you their email addresses. If that doesn’t happen — something is wrong!
- Embrace your user’s ecosystem. We could have easily made the mistake to build our own Flashcard ecosystem, like other competitors had done. Instead, we embraced what was there. Even if the UI looks a little outdated and is not aligned with our brand. Turns out, it works for users and we found our way in!
- The power of complements. Anki is to AMBOSS what the Guide Michelin was for Michelin. A reason to use the core product even more. We’re now thinking more about complementing products as they expand user touch points and generate new value streams.
- Time. As you can see, our first-time discovery by the book took a long time! Usually, discovery should be smaller and faster. But for us, this was necessary to learn, and we’re grateful about the leadership buy-in we got!
- Good solutions can be small. Turn off your product ego. Building more is not equivalent to more value.
- Discovery is raising the odds of success. Discovery is not a linear process, and often luck plays a role. The process can stimulate luck. The fact that we did a hackathon and found the perfect Anki developers at the right time are completely unrelated. But viewed as a journey, winning Aristotelis and Yann for this project was only possible with the hackathon months earlier.
- Exploit your enabling technologies. The technology that allows us to recognize and link out to AMBOSS from thousands of medical terms is a core technology driving our product. Previously, we had never thought about exploiting it for other products but a hackathon helped changing this!
- Give users what they need, not what they want. Users asked for flashcards, and competitors gave them proprietary solutions with > 20k cards — and failed. We gave them what they needed — in depth knowledge for every term on every flashcard with one click inside a product they were already using.
- Get out of the building. Without talking to > 40 students we would have never had confidence in the smaller solution we prioritized at the end. Get solutions in front of users!
- Get great people. If we hadn’t found Aristotelis, maybe we wouldn’t have done this. He didn’t reply to my outreach several times. Turned out he was just busy doing customer support for his other Anki projects, and missed my messages! Never give up reaching out to great people, chances are they’re just busy doing other great things.
- Product intuition eats your plans for breakfast. Give intuition time to kick in. Talk to peers to get challenged on you plan. Often, after some time and discussions, the right solution will find its way to the top.
Thanks ❤️
Thanks to the many contributors for this initiative. What I described here was just a tiny part. The majority of the great work has happened after the launch of the V1 and I haven’t been involved in any of this.
Thanks to Yann for thoroughly reviewing this blog post and providing a lot of constructive feedback based on his experiences driving this project over a long period of time.