We recently hosted a webinar with our CEO and Co-founder Michael Driscoll and Hammond Guerin, Sr. Director of Analytics at Liftoff to share how the Metamarkets solution for interactive ad tech dashboards has been reimagined into a new solution called Rill.
Below is a transcript of the webinar where you will hear about:
- The latest dashboard improvements in Rill
- How Liftoff uses Rill to capitalize on ad revenue opportunities
- Sneak peek into what’s coming later this year
Introductions (02:00)
Mike (Rill): Before Rill I started a company called Metamarkets. Like Hammond, and one of the reasons I'm so happy to have him in on this call, I started my career as a data scientist. I've been in the trenches of data engineering for quite awhile, so super excited to have a fellow data scientist join us to talk about Rill. And with that, Hammond, I want to introduce you and let you say a bit about your journey to getting here today.
Hammond (Liftoff): I came into the data science world while I was doing my MBA. I was working with a couple of professors on what eventually became the data science concentration in the data science curriculum. We were doing some machine learning projects with a number of smaller companies and an opportunity came up to come to San Francisco and work with Vungle to build what we call then the “algorithm”. I did that with a couple of grad students and professors for six months, and eventually ended up getting invited to stay there to head the data science team that we built. I did that for several years, built out the data science team, which in Vungle terms basically means the ad serving algorithm team and worked on that until we grew and merged with Liftoff. As a part of the merger, my role has evolved. One reason why I'm working even more now with Rill than before is to focus on the analytics platform that we're using across the organization, trying to take learnings from both sets of legacy companies and socialize them around the new combined company. It's interesting that the Metamarkets journey for us started almost at the same time. When I started on Vungle back when we were doing a consulting project that was just prior to us starting to work with Metamarkets. So I have the before and after picture too.
Rill: the journey from Metamarkets (05:30)
Mike (Rill): The story of Metamarkets goes: we were acquired by Snap, the makers of Snapchat, in 2017. In 2020, Rill acquired the IP of Metamarkets from Snap, and we were very lucky and fortunate to do that. Then we very quickly began reaching out to many of the former Metamarkets customers and said,” hey, we're back at it and we're open for business”. At that time, Vungle was one of those customers that we reached out to, and they were interested in moving forward and becoming a customer of Rill. If we think about the core offering of Rill, what we really focus on is combining large scale data transformation, a very fast OLAP data engine, and ultimately delivering value through an interactive, fast, flexible dashboard. That is really the face of Rill and that was historically the face of the Metamarkets SaaS cloud offering. One thing we're known for at Rill, and we're known for at Metamarkets, is scale.
Liftoff: company background and analytics needs (07:40)
Hammond (Liftoff): Liftoff is a mobile advertising business, so that immediately means a very large scale of data. We're typically dealing with billions of rows being added on a daily basis, and needing to do analysis that covers a year of querying on that kind of data. The overall company was combined by plugging the demand side and supply side together. On the old Vungle side, we have an exchange where we sell our inventory to outside buyers like DSPs. Liftoff, the original company before we merged and became the combined Liftoff, was a DSP. One thing that's interesting from the data perspective is we're still in the process of bringing those datasets and systems together. We have a pretty well developed architecture on each side, and on each side you're handling similar kinds of scale.
Mike (Rill): To get a sense of the signal that Liftoff is looking at every day, would you say you are in the trillions in terms of what you're storing and managing with that data infrastructure? Is it, trillions of events over the course of a year? What sort of look back periods? When you're looking to make sense of data how often can you quickly do reporting? Last few hours, last few minutes, last few days?
Hammond (Liftoff): There's obviously different ranges. The analysis that we're running is typically running in billions, sometimes many billions. That's kind of the typical scale where we would get into other strategies like sampling and obviously aggregation, etc. When we're using Rill most of that analysis is what's really happening right now. So the critical thing is week over week or two weeks over two weeks prior, that's where the product shines and does a better job than the numerous other tools that we work with. It's really about the ability to drill into the trends that are happening right now, compared to a relatively recent period. But we do have rules set up right now to hold the past one year of data, and we use it regularly for kind of longer term analysis as well.
Is it just another tool in the stack? (10:40)
Mike (Rill): As a director of analytics, you know that there's a lot of tools out there to choose from. There’s the modern data stack industrial complex out there with hundreds of different tools. Even in the BI space, we've got Tableau, Looker, MetaBase and Superset to name a few. When you think back to that initial choice of considering working with what was then Metamarkets, why introduce yet another tool into the stack? When you saw this tool for the first time, what led you to thinking it might be worth adding to the arsenal?
Hammond (Liftoff): When we first added it to the arsenal, we didn't really have other tools. It was very early. We went from a period where we were trying to build the algorithms out and deploy these and test them, but we couldn't see results for two days, which in an ad network a lot of really bad things can happen in two days when you've changed the serving algorithm. So Metamarkets came and people on the call will be able to see how quickly you can interact with the data using this tool. To go from being blind for two plus days to having this all at your fingertips, that was kind of shocking. So first part is we didn't really have great tooling back then and it was one of the one of the first tools we integrated with.
The second part is now we do work with many of those tools that you mentioned. We have a huge user base using Tableau. We have a huge user base using Looker. We have a lot of people skewing more towards the engineering side who use Superset. We have a handful of homegrown systems that are also part of the landscape. But we also have Rill and I think that Rill is one of the few, if not only, tools where the support for it is really very bottom up. I was talking to our principal analyst last night and he reiterated the same thing to me “the analysts prefer to do everything in Rill if they could.” As you can see in the demo right now it's fast and responsive. I think that part of the thing that actually makes it that way though is that we don't try to do everything. So what you're seeing here, this time series analysis week over week comparison, these are the kinds of things where we do 90% of our analysis - let's find trends, let's see what's gone up, let's see what's going down, let's see who we need to reach out - and the tool basically does exactly that. So even though we have Looker with the exact same data loaded, people will go to this tool first whenever they can.
Mike (Rill): When you're bringing a new member of the team on at Liftoff, in your own words, how would you describe Rill, its key features, and what to look for?
Hammond (Liftoff): What's really typical for us is we might look at the last week of data, which is something that you can do in other systems relatively readily, although your connection and the way that it works under the hood is very fast and interactive. But where you start to get some of these features of the Explore product that become indispensable to people would be the comparison tool. That is a lot of what our account executives or customer facing analysts are using. Their goal is to help. They are trying to see if a campaign is doing well in these new apps, and let's try to push those or it's trending down or whatever. So this week over week trend analysis is critical. As you can see, you can visually look at the difference. You can see that this is looking like we're down 4% week over week, whether that's meaningful is obviously dependent on the data set. Again, you can build part of these visualizations in other tools, it's possible, but it takes a very long time. It also takes a learning curve. When people join, I don't think we have a Rill training for the organization. Yet we still have hundreds of users using it as their primary tool. As you're looking at time series analysis, we're also trying to drill down on things. It's really easy to filter because you can do the comparison feature, which allows you to show the impact of different dimensions over time. This is also extremely easy to use, and you can see that the data loads interactively. I think that's one other piece that I like to call out is that there's a really big difference between running a report where the data comes back in two minutes, which is still reasonably fast for some for these volumes, and having it come back in two seconds, because that’s the sort of speed where you're thinking and asking questions. I personally think it really changes how people interact with data. The point here is that we'll use this tool because it's so fast and interactive for all sorts of purposes. And I’ll say again what our analysts said to me last night “people go here first. If they can do it here, that’s what they do”. That's just how the working style has evolved over the team. The business team has been working with Rill, and then Metamarkets before for close to 10 years at this point.
What’s coming next with Rill (19:20)
Mike (Rill): While we were lucky enough to license all of the technology from Metamarkets into Rill, and we rebuilt that backend, what we're actually doing today is building a brand new application for Rill. Our product team is actively prioritizing the things that we want to put into this new front end. What Hammond has been showing us is the application that we had licensed from Snap, and there’s a new version of Rill that’s been built over the last few quarters. The biggest difference is the journey of getting data into Rill is something that we’ve been really focusing on. The not a lot of differences today between legacy Metamarkets and Rill, but the thing that's hidden is that we now have a path where through downloading an open source Rill Developer application, you can actually build dashboards like this auction sample dashboard yourself, and deploy them to the cloud in a matter of minutes. Bit by bit we’re bringing back many of the features that Hammond highlighted in terms of the ability to do a quick time comparison, look at the last seven days of data, zoom in to drill down, do searches, all the things that people know and love about the the legacy Metamarkets product, and we've rebuilt it with a brand new architecture and brand new code base. We're super excited to get that into the hands of our existing customers. In our experience, it was certainly a few weeks to get customers onboarded onto the old platform, just because of the amount of data engineering work that was required. One of the visions for Rill going forward is we want to make that data engineering task dramatically easier, radically faster. Could you share today how Liftoff moves data into the Rill product? What's that handoff look like?
Hammond (Liftoff): It’s actually an interesting point because the users and consumers of Rill tend to originate from the commercial side of the organization and they have a dependency on the R&D side of the organization to build these pipelines. It's probably an area where if the tooling was available to change some columns and alter the aggregation, that's something that we would take advantage of because currently that does introduce some delay, where we have to fit into the R&D roadmap, if we want to make any changes, etc. The impact of it is that we don't change it very often, and I think that we could use Rill more efficiently if we were able to do that better.
Q&A: for teams looking to figure out a way to make sense of huge log level data, what are some pointers on how to get started? (25:00)
Hammond (Liftoff): Being honest, that journey took a fairly long time. It’s important to understand the use case and what you're trying to achieve, because there's quite a difference between the tools that you would use to scan and pluck information from a huge batch of log files versus what we do with Rill, which is almost the other end of the spectrum in terms of usability and convenience. There's a lot to be said for trying to use that data efficiently. Also the cost obviously becomes a controlling factor with a large data set. Personally I think what has worked well for us is going with systems where we define data models in a way that's carefully built up over time, where we have a business facing data model that's that people can query. So you're not asking business users to interact with the engineering defined datasets, as opposed to more ad hoc analysis, where a tool like Tableau can be used very quickly, but maybe lends itself to a little bit more ad hoc analysis where something we do in Looker or in Rill, you can force yourself to put some structure to that data set that makes it sensible to the to the business users. It's been a 10 year journey of working on this, and we're not anywhere near the end of it.
Q&A: how do you measure value in a solution like Rill and continued investments? (27:45)
Hammond (Liftoff): At some point, we have to be prepared to include some qualitative assessment of the value. One of the strong pushes at Liftoff is to make sure that when we're working with customers that we're providing value in terms of here's your data, insight, and interpretation where perhaps you don't have the data to deliver on your own, or maybe the tools that we have can add value. We try very hard to come up with those kinds of pieces of information to create value, and in order to do that, we have to have the tools that enable people to work. So in Rill’s case, I've kind of alluded to this but it’s very ground up. The team is telling us “this is what I use and this is how I'm able to bring that to the table”. We have parts of the organization where we haven't set up these tools, and we can see the difference. I personally think there's a lot to be said for a qualitative assessment of what value is being created, how does it fit with your strategic priorities, how do you feel like it's an enabler for the more direct metrics that you're trying to deliver on? That's generally how we think about it. I don't think that we have tried to quantify the exact cost, or the exact value that's being delivered in terms of like a percent ROI. I think that would be very, very difficult.
Q&A: how should companies be thinking about being truly data driven? (29:40)
Hammond (Liftoff): There’s two sides to that. On one hand, you have to enable the team with the right tools and some level of trust to access the data. People have to be able to see the data in the first place. On the other hand, you have to build up the expectation that that’s what people are doing - they’re rolling up their sleeves, they’re asking questions and answering them, they’re coming up with insights. There’s sort of a push and pull. You build up the expectation that people operate that way culturally, but the tools have to support it. It doesn’t work without both sides.
Q&A: is there something unique to mobile data that Rill solves for you? (31:20)
Hammond (Liftoff): I don’t know if it’s mobile specific, but advertising data and the fact that it lends itself to time series analysis. Everything we do is essentially time series analysis. That’s where the product shines. I mentioned other tools constructing week over week, you can do it, but if I counted the clicks it would not be one. Time series analysis translates to many industries. We’re trying to look at a week of performance in time series, drill down and filter, see who moved up and who moved down, find subsets where campaigns are working - that’s the nature of the work with Rill. The types of questions we ask is what the product delivers. Rill does not intend to do everything; that makes it usable. That makes it so you don’t have to click through five other options to do that one analysis it's built for.
Q&A: where’s the first place you would advocate implementing Rill to get analytics leverage? (33:00)
Hammond (Liftoff): The teams that get the most immediate value is customer support. The folks that need to on a regular basis interact with customers and tell them what’s going on by drilling down very quickly. To say “these five things performed better than last week”, and then be able to say “and here’s why”. The users that are most on the front lines would be 80% of the use case, I’d be surprised if it wasn’t a larger majority.
Q&A: does Liftoff provide direct access to the dashboards, what’s the conversation like with the customer success team? (34:20)
Hammond (Liftoff): Someone’s day-to-day job is to use Rill to dissect performance, see what’s going on, and they put it into some form of communication for the customer. We have provided customers direct access to the dashboards, and one reason we are able to do this is because there’s no learning curve. We don’t need to ask the customer to learn a new tool in order to interact with the data, it tends to just be self-evident.
Rill Beta (36:00)
Mike (Rill): Soon we are going to announce Rill’s beta launch. Anybody can download it and our vision is that you can go from data lake, a S3 bucket or a Google storage bucket, to the very same dashboards that you saw Hammond and I working with in today’s webinar in 5 minutes without having to talk to any members of the Rill team.
Download and try it today!
