How BuySellAds Went From 0 to 2+ Billion Ad Impressions Per Month
February 4, 2015 | Stephen Dionne
MaxCDN is proud to present #MaxScale, our new series that takes an in-depth look at how growing tech companies handle scale at the highest levels of bandwidth, storage, analysis and more. In this installment, we chat with BuySellAds founder Todd Garland.
BuySellAds is an advertising software and services company based in Boston, MA, founded by Todd Garland in 2008. BuySellAds helps web publishers sell ad inventory, at a rate of over two billion ad impressions per month. Similarly, they help advertisers find and purchase guaranteed ad inventory from their vetted and quality-controlled directory of publishers. In 2010, they were recognized as one of the fastest growing privately-held companies in the United States by the Inc. 500.
Todd Garland, Founder & CEO of BuySellAds
Founder Todd Garland began his career as a web developer. He was one of the early employees at HubSpot, an inbound marketing and sales platform. Todd started BuySellAds because he saw that there wasn't an easy way to sell ads to advertisers online. He explored ways to automate the ad-buying and selling process, which no other company had fully developed by the time he began BuySellAds.
Stephen Dionne: As not only the CEO, but also the founder of a tech company, you have a unique perspective on your organization. How directly involved are you in your company's IT operations?
Todd Garland: We're a company of eighteen people, so the highest level we have right now in terms of executives is the Director level. We don't have VPs or C-level management or anything like that. We have one individual who is running things in IT, and when necessary I'll step in and help make decisions. But I don't really throw down orders. I am more there for support. Really the entire engineering team is involved. I have my hands involved in the organization of the infrastructure, but largely the engineering folks are making the decisions and pushing the buttons. I still keep on top of everything we do on the technical side though, if for no other reason than because I enjoy it.
SD: As your company has grown from its start-up roots, what is the biggest challenge you've faced in scaling your IT operations? How did you face that challenge?
TG: We're selling and serving billions of ad impressions every month, which is no easy task if you're trying to manage that entire stack from end to end. So one of the early strategies that we employed for serving ads was to offload a lot of the decision-making onto the client side. Essentially we would use CDNs, like MaxCDN, to do all of the serving of those static assets.
When we initially launched BuySellAds, we were using Amazon Web Services (AWS). This was before Amazon had its CDN product, [Amazon CloudFront]. AWS was too cost-prohibitive for us to continue to use as we scaled. For our company, MaxCDN has been far above other companies in the space, like EdgeCast, [Amazon CloudFront], and other companies we've considered. One of the most important factors that we've looked at is automation. MaxCDN takes the brunt of the traffic that we need to support, and we stay focused on our production servers that manage the web app. Everything else is offloaded onto the CDN.
Outside of serving static assets, we no longer offload all of the ad serving onto the client side, and we're doing all of that decision-making server side. So we have a large server side component that we've had to scale as well. Currently, that is done at Digital Ocean.
There's been no one secret to overcoming the challenges involved. There is a lot of problem-solving as we go.
BuySellAds' office building, located just East of Tufts University in downtown Boston.
The tech stack that powers BuySellAds
BuySellAds is focused on working with the best content creators on the web and figuring out how to help them build sustainable businesses through advertising. Todd Garland is aware that many kinds of advertising are unpopular with Internet users and that ad-blockers are prolific. Still, he maintains that some of the best content creators on the web can support themselves with ads and become very profitable. The key is in the approach. To this end, BuySellAds focuses on high-quality ads in the form of display ads, sponsored content, and custom ad formats.
SD: Walk me through the technology stack that BuySellAds is built on. How did you choose each major piece?
TG: Believe it or not, we still only have one production server that handles all of the traffic for the web app. Even though we're dealing with thousands of customers, we've only needed to buy larger servers as we scale up. We do have a handful of auxiliary, worker machines that do any needed extra processing so as to not slow down the production server. Our entire app is built on a very lightweight, homegrown PHP framework.
Since all of our static assets are in the CDN, the other major portion, besides production, is our stats infrastructure. We have a box that sits next to our production server that does all of the processing, the roll-ups, the forecasting, etc. for the stats. The stats infrastructure is managed through a homegrown console that we've recently developed.
The actual ad servers that do all of the serving and collection of the stats are at Digital Ocean. We have approximately 12–20 stat servers in the cloud at Digital Ocean. We're kind of “living the cloud dream,” in the sense that when we need to bump things up to support higher volume, we can just turn some knobs and dials, change a few configuration settings, and we're done.
SD: Was this your infrastructure set-up right from the beginning? Did you consider other options?
TG: We didn't used to spit out servers quickly and easily with a cloud hosting provider. We actually used to roll our own stat servers. Our initial strategy was going to different hosting companies in different locations in the U.S. and in other countries, leasing a handful of servers at a time. Around July 2014, we started working on a new ad serving infrastructure, and that's when we decided to make the switch to the cloud. It was more cost-effective, and we had gotten tired of dealing with the slight differences between dedicated server providers. It just made more sense. What we like about having the Digital Ocean cloud infrastructure is that we can just trash a machine when we no longer need it. There's no overhead and no need to even think about the process, which is nice.
Solving BuySellAds' biggest tech challenges requires "smart people and brute force tenacity"
The ads that BuySellAds services are known as “automated guaranteed” (or "programmatic direct"). BuySellAds lets an advertiser buy futures on advertising, similar to how an advertiser might reserve a television or magazine ad. By buying in advance, the advertiser is guaranteed a particular space. This service allows companies to target ads to specific audiences when their marketing campaigns can be most effective. Advertisers pay a guaranteed rate to “own” these spaces for a certain period of time or for a certain number of ad impressions.
Postcards from BuySellAds' customers displayed on the office wall.
SD: Tell me a little about the tech providers with whom you have developed long-term relationships. What motivates you to continue these particular relationships?
TG: First and foremost, MaxCDN has always been really good to us and a reliable partner. The infrastructure hasn't caused us any heartache in seven years. One of the things I love about the web is that you can still have “hometown relationships” with other business owners, and they're not that hard to make. When we started working with the folks at MaxCDN, they were just starting out, and I got to know [MaxCDN founder] Chris [Ueland]. Actually, he took me out to dinner one time when he was in Boston. So the web can be a small world, and you can still get a kind of hometown loyalty from your business partners over the web. Of course, loyalty doesn't mean anything if the provider doesn't meet or exceed your expectations, like MaxCDN has for us.
Another relationship we've developed is with Amazon Web Services. We still use AWS for all of our data backups. We continue to use Amazon because, much like MaxCDN's technology: it's simple, it works, and it does what we expect it to do.
Next is DynDNS. I don't think I've ever needed to call tech support for DynDNS. And it's another company with which we have a kind of hometown business relationship. They treat us very well, and we've established a great relationship over the past seven years, too.
The last company that we've gotten close with is Digital Ocean. It's a newer company, but we believe in the business and what it's doing. We take the amount of money that Digital Ocean has been able to raise as a vote of confidence that it will be able to put new data centers in new locations and continue to grow with us.
SD: What has been your biggest technological challenge in scaling to handle billions of ad impressions every month? How did you address it?
TG: Since we're able to get all of the static stuff done through MaxCDN, that relieves one set of headaches. The other challenge is processing the logs and doing all of the querying against the logs. The storage engine we use is called Kyoto Cabinet, and we have a homegrown C++ system that does all of the processing and serving of those logs. It's an ongoing challenge. Since ad serving is so core to our business, it's something we work to improve every day. Our main approach has been simply hiring smart people and adopting a kind of brute force tenacity.
Supporting hundreds of thousands of transactions with a small, remote team
Since 2008, the bootstrapped start-up has continually scaled up its operations as its customer base and ad sales have grown. While BuySellAds is headquartered in Boston, its remote teams now extend across the United States and internationally into Canada, Romania and Germany. Most recently, BuySellAds has acquired an email ad start-up called LaunchBit.
With only eighteen employees, BuySellAds runs a tight, efficient ship. It leverages a talented and motivated staff with state-of-the-art, automated technology to keep costs low.
With a mostly remote team, BuySellAds maintains a modest-sized office headquarters in Boston, MA.
SD: I see fewer than two dozen faces on your company overview page. What particular organizational challenges appear when a small company takes on more customers?
TG: We've been able to keep our headcount so low because we've worked hard to automate as much of our business as is possible and as makes sense to do. This helps us to stay bootstrapped and to stay profitable. In terms of the core business that enables people to buy ads in a self-serve fashion, we largely don't need to talk to many of those customers unless they need help with the process.
Everything we've built in the ad space has been centered around the core concept of it being self-serve. But one of the challenges as we start to explore different ad formats is that we don't always start off being self-serve. So the new formats that we develop don't always scale very well. Once we start seeing success in one of the new, non-self-serve programs, we then have a lot of overhead involved in which our small team needs to do sales, ad trafficking, and customer support.
Every business faces pressure regardless of its funding status, but I think there is more pressure on companies our size that haven't raised any money. Especially when you're competing against companies that have raised tens of millions of dollars, it's necessary to stay on top of your balance sheet and to remain mindful of headcount and expenses. That's been in our DNA since the beginning.
SD: You celebrated your 100,000th order processed back in June of 2013, and I’m guessing you are likely closing in on 200,000 orders. What business or technological challenges emerge in regard to your billing systems as your company grows? How have you addressed them?
TG: About 80% of our business is fully automated on the payment side. The other 20% consists of the new stuff we've been working on that hasn't been integrated into the self-serve system yet.
The biggest challenge on the billing side is delivering payments to publishers. We're a global business, so we're paying folks in 79 different countries around the world. There's no standard API, so to speak, to send money to these people.
So we use several different methods. We use straight ACH for folks here in the U.S. We will physically write checks for some people. We'll send wire transfers to others, and we'll even pay people out in Bitcoin. The way that we take money in is pretty well automated, but we haven't been able to automate every way that we get money out. Wire transfers and international checks, in particular, are difficult to automate. I would love it if someone would create some kind of global ACH solution.
Largely when technical solutions aren't there, we put people against problems. Since our headcount is so low, fortunately there haven't been too many problems we haven't been able to tackle with technology.
Jen Vazmina and Garland in the collaborative office space.
SD: We're entering 2015. What milestone do you hope to hit by the end of the year? How do you plan to reach it?
TG: One thing we've been really good at since the beginning is sourcing unique inventory that advertisers can't buy anywhere else. We can help them discover inventory more efficiently than they ever could themselves. They just click a few buttons, and we serve it right up to them on a silver platter. A large focus for this year is taking that same approach for inventory that will sell on a performance basis. That is, we and the publishers who own the inventory don't actually make money unless the inventory is performing. Performance-based advertising isn't a new thing, but we like to think that because of our focus on quality and because of the experiments we've been doing with native/custom ad units, we can deliver a pretty valuable solution to advertisers.
The space in which we've been operating for seven years has now earned its own segment within the ad space. It's called “automated guaranteed.” Clearly our doing the performance-based advertising is at odds with that market, but we're trying to focus on getting quality inventory in front of advertisers so that they can have ad campaigns that perform well. So we're not as concerned with fitting in whatever confines the industry or the market says that we belong. We're going to build whatever we need to build and develop whatever products we need to develop in order to make publishers money and get advertisers customers.
Founder Todd Garland with BuySellAds' "Support Your Local Publisher" T-shirt. On the BuySellAds platform, publisher quality is worth more than traffic quantity - the company accepts just 5% of incoming publisher applications.
SD: What's the biggest lesson you've learned over the past year? How have you and the company grown from it?
TG: There are two big lessons I've learned. The first one is to be faster at making decisions. Not listening to your gut and not making decisions quickly can be very costly. Every time your company moves forward, it's because you've reached some kind of decision or reached some kind of milestone. And making decisions is just as important as reaching milestones. So don't be afraid to make the choices that you know you need to make.
Secondly, being CEO doesn't mean you get to do whatever you feel like doing. It means doing what's best for the business. Even though I might have a personal passion that inspires me to build some custom ad format or to take two months to build a new product – that's not my job anymore. I can't put aside my day-to-day role of leading the company to follow those passions. You can almost see the change overnight in your business when you focus on organizing and leading people rather than focusing on pet projects.
SD: What piece of advice would you give to an entrepreneur just starting out now that would best facilitate scaling in the future? That is, an ounce of prevention is worth a pound of cure – what can someone do to make eventual scaling as painless as possible right from the beginning?
TG: There are two key pieces of advice that I would give.
First, don't try to build everything yourself. If something isn't part of the core business that your customers are paying you to do, have somebody else deal with it. Offload it. Subscribe until your credit card is maxed out on thirty different third-party apps if that's what it takes. Trying to build everything on our own is a mistake we've made time and time again. We are a very tech-focused company, so we love building things from scratch. But in the end, your time is better spent on your core business.
Second, hire somebody who is incredibly detail-oriented as early as you can possibly afford it. It doesn't matter if you're already detail-oriented. You're supposed to be there to set the vision, to build product, and to talk to customers. You shouldn't be dealing with anything else. Don't distract yourself with tasks that are administrative, finance-related, having to do with company formation – any of those things. You'd be surprised how many folks who love being organized would leave a corporate job in three seconds to have the opportunity to work in a start-up role. But he or she just has to love being organized. Experience isn't everything.
Photography by Elyse Pono. All images released under Creative Commons license. Please attribute photos to "Photographer: Elyse Pono / Source: MaxCDN" and link to this post.