☠️ 30%+ of startups fail due to lack of product-market fit. In this eBook, you'll learn how to avoid it. Only $10 for a limited time.

❌ Failed startup
✅ Successful startup

ABBY: Here’s What Andy Has Learnt

ABBY was a documentation service for A/B tests. To make such a service successful, Andy would have needed to educate the users, and that was not possible.

Germany
Analytics
No Need
Big Competitors

Andy Goldschmidt

March 16, 2018

Sponsor Failory and get your business & product in front of +20,000 CEOs, startup founders, entrepreneurs, developers and marketers every month.

Do you want to grow your business? With GenM you can get free marketing from an apprentice as part of their training. The student will work 40 hours per month creating content, increasing SEO rankings, carrying out advertisement campaigns...

Want to take ownership over the growth of your business, but don't know where to start? Get 1-on-1 advice from vetted growth experts about your business.

No more changing developers every time you start a new project. With our full range of development services, no project is too big, too complex, too mobile or too software for us to complete. We’re the only web-dev partner you’ll ever need.

A 5-minute read that's informative, witty and free? That's Morning Brew — the daily email that delivers the latest news from Wall Street to Silicon Valley.

Hello Andy! What's your background, and what are you currently working on?

Hi! I’m a data scientist for four years now, and I’m currently working at AdTriba, a marketing attribution tool. I’m somewhere between a data scientist and a programmer, so I’m constantly trying to combine both skills and create something useful. Unfortunately, no project has come to fruition yet. My biggest side project so far was ABBY, a documentation and evaluation service for A/B tests.

 

What was ABBY about?

My first job was at Jimdo, a website builder company. I did lots of A/B tests there. Even though the concept of an A/B test is simple, it’s not that easy to do it right. And the more tests I evaluated, the more I realized that people neglected documentation of results. In my experience, the results were so valuable because they allowed me to have a knowledge of what worked and what didn’t.

For this reason, I built an internal tool to solve this problem and to evaluate tests automatically. My teammates liked this tool, as they stopped needing to evaluate their tests manually. Instead, they just needed to fire a SQL query, and the tool did the rest. Documentation of A/B test resulted in great benefits, too. It was so much easier to argue on what we did, how it worked and what the next steps were. Also, many co-workers found this documentation useful as a source of inspiration for what they could do next.

Seeing how successful this tool was at Jimdo, I decided (with the permission of my CEO) to turn it into a stand-alone service, called ABBY.io. It took me almost nine months to launch it. Someday after an exhausting coding session, I decided to pull the trigger in a “f*ck it, ship it” moment.

ABBY Website

How did you grow ABBY?

First of all, I posted ABBY to the usual social platforms. Twitter, Hacker News and relevant subreddits, but traffic was modest. Additionally, I had some free credits for Google AdWords, so I used them for a test campaign. The keywords for A/B testing are really expensive, so the campaign didn’t result in many clicks. But the traffic was more targeted than social traffic, so I was able to convert some visitors (ABBY was free-to-use, that helped a lot).

The biggest growth occurred when someone hunted ABBY on ProductHunt. That resulted in about 20k visitors and about 100 sign-ups. ProductHunt has a diverse audience, and ABBY was a niche site, so conversion rate was pretty low. But I was satisfied, as 100 sign-ups were more than I needed to validate the idea of the service.

Hire digital marketing interns. 40 hours for $49/month.

Learn More!

AD

From Wall Street to Silicon Valley...

Are you busy? Sure you are — that's why you need Morning Brew. It's a free daily email that gives a quick 5-minute rundown of the day's top business news. Join the 925,000+ people who start their day with Morning Brew!

👉 Learn More!

Reach +20,000 Startup Founders!

If you are looking to get your product in front of founders, CEOs, VPs, web and mobile developers, makers, consultants, marketers, bloggers, product managers, and many other thought leaders, then we can help you.

👉 Sponsorships

Which was the problem with ABBY? How did you realize?

I think the main problem was that most people didn’t realize the potential a documentation service for A/B tests has. To make such a service successful, I would have needed to educate the users, and that’s not something you can do before you offer any value to them. A service must provide value right after signing up for it. If it doesn’t, users will abandon it in a blink of an eye.

 

What were the mistakes you made?

I was unaware of how difficult it is to grow a B2B business, even more, if it is a niche product. Also, I was not well prepared for launch; there was no strategy. Promoting a product is hard. Doing so without a plan is virtually impossible.

                                                                                                                                                             

What were your biggest disadvantages?

I think the biggest problem was the competition and the tool landscape in general. You can run A/B tests in Google Analytics which is integrated into nearly every website. If you want to do more serious testing, Optimizely is the standard tool, and they offer end-to-end A/B testing; from design to implementation to evaluation of a test. You can also implement A/B tests completely on your own and use one of the countless free tools to evaluate them.

Your one-stop shop for everything dev-related.

Anything you need, our devs can build: complex e-commerce solutions, custom software or SaaS, beautiful WordPress websites… anything. We’ll fix, upgrade & customize your website, so you never have to worry about troubleshooting web stuff ever again.

Let’s get to work!

Grow your business. Get 1-on-1 calls with +100 mentors.

Learn More!

AD

If you had to start over, what would you do differently?

I would start smaller and gather feedback before building anything. A B2B service is incredibly hard to grow as a side-project or for solopreneurs in general.  “Build it, and they will come” is a big fallacy. Finding customers can be incredibly hard. And even more, if your customers are businesses, not individuals. 

Also, I tried to do too much with few resources. I only had a handful of hours per week to invest in the project. Combine that with web design, the backend part, promotion, etc. and it’s overwhelming. I considered outsourcing some of this stuff to freelancers. That way I could have focused more on my strengths and avoided getting frustrated.

 

What did you learn?

I learned a lot of things from this project. The most important one was that I need better and faster ways to validate ideas. I worked for months in my little echo chamber without any real outside feedback. It’s hard to stay motivated throughout this phase, and it’s heart-crushing once you hit the outside world and nobody cares about your product.

Also, you don’t just need a product, but a detailed plan on how to promote and grow your business. I didn’t know much about marketing before starting ABBY, and I think a lot of other wannapreneurs have the same issue, but it is necessary to achieve success with your startup.

 

What's your advice for someone who is just starting?

Start small and find people from your network (or online) who might be good customers. Tell them about your idea and ask them if they would be willing to pay for a service like that. It’s really important to validate your idea quickly and build as little as possible before you know there is a market. In my case, I spent months building a software no one used in the end. That is a depressing experience that you should avoid.

 

Which entrepreneur book would you recommend?

It’s not a business or marketing book, but I’m a huge fan of “Thinking, Fast and Slow” by Daniel Kahneman.

You can learn a lot about the way in which the brain works and apply this knowledge in the business world. Furthermore, you will be able to analyze every startup success story knowing you have just witnessed survivorship bias.

 

Where can we go to learn more?

I usually blog here, or you can follow me on Twitter: @datenheini.

✉️ Subscribe to our newsletter!

Invite us into your inbox and get immense learning and 80+ deals on tools to help you scale your startup (worth up to $50,000)!

Your subscription could not be saved. Please try again.
Your subscription has been successful.

👇 Other Interviews

Price2Spy: The Serbian price monitoring SaaS that makes $155k/month
By 
Misha Krunic
  •  
January 29, 2020
Serbia
SaaS
Over 150K
Pull Reminders: Bootstrapping a Side Project into GitHub Marketplace
By 
Abi Noda
  •  
February 21, 2019
United States
Web Application
Up to 50K
Hubstaff: Growing a Time Tracking Software to $316,000/Month
By 
Dave Nevogt
  •  
February 21, 2019
United States
SaaS
Over 150K
OneUp: How to bootstrap and be profitable in a VC-funded market
By 
Davis Baer
  •  
June 26, 2019
United States
SaaS
Up to 50K
ScrapingBee: From 2 failed startups to $3,000/month in 4 months
By 
Pierre de Wulf
  •  
January 2, 2020
France
SaaS
Up to 50K
ClaimCompass: Disrupting the Airline Business and Making $100k/month
By 
Alexander Sumin
  •  
August 6, 2019
Bulgaria
Web Application
Below 150K
Community Coders: Failing to build a profitable business while in University
By 
Kaito Cunningham
  •  
January 29, 2020
Canada
Education
People Management
Bad Marketing
Teamometer: $3k invested, 2 years gone and a HR SaaS no one needed
By 
Sergio Schüler
  •  
March 19, 2020
Brazil
SaaS
No Need
No MVP Validation
Eloquis: Losing $20,000 with a Bootstrapped SaaS
By 
Rohit Nallapeta
  •  
March 17, 2018
United States
SaaS
Unprepared Market
No Need
Onepagetrip: Monetizing a Startup Is Not That Easy
By 
Ana Santos
  •  
March 16, 2018
Australia
Marketplace
No MVP Validation
Monetizing Issues
Legaats: Why I Stopped Pursuing My 1st Startup Idea
By 
Deepak Chhugani
  •  
March 17, 2018
United States
Web Application
Monetizing Issues
No Passion
Ink: Burning $16,000 in an Online Tool for Designers
By 
Andrew Askins
  •  
April 28, 2018
United States
Web Application
Bad Marketing