Coauthors: Ryan Tecco , Paul Bergeron , Prapti Sen
Advertisers want to know what happens after a LinkedIn member sees an ad from one of LinkedIn's marketing opportunities. Do they lose interest? Do they close the browser and come back a few days later to sign up? Do they actually convert? With Conversion Tracking, we can finally answer these questions. We can see how many members who view the advertiser's ads actually convert—that is, engage or perform relevant actions (like making an online purchase or signing up for a webinar), and thereby provide a measurable rate of return for advertising on LinkedIn.
Designing the system
Conversion Tracking Architecture (Image credit: Danhiel Baker) Overview of system components
Conversion tracking can be broken down into three main components:
Insight Tag Endpoint: When a potential customer hits the advertiser's site, the page makes a call out to the Insight Tag Endpoint, which hits a cached datastore derived from MySQL. Its purpose is to determine whether the page the user lands on is eligible for a conversion—which campaigns are associated to the Insight Tag, whether the campaigns are active, whether the URL fits the conversion action match rule, etc. Eligible conversion “fires” (events that trigger our attention) only occur if a user is found to be a LinkedIn member, and this will emit a Kafka event which is loaded into our offline HDFS (Hadoop file system) storage medium.
Data and reporting: Reading in the HDFS data, an offline process takes in each individual conversion fire to determine the potential impressions (ad views) and clicks for each conversion event. Conversion fires without associated impressions and clicks are dropped, and the remaining ones are aggregated into meaningful metrics broken down by conversion type, campaign type, etc. The results are stored in Pinot, an in-house online analytical processing (OLAP) data store, which provides an interface for very high performant data queries and reporting.
The user interface (API partners and UI): This system includes the conversion action definitions, reading the Insight Tag, the reporting dashboard, and general management of conversion tracking.
We chose to use a dual AWS-LinkedIn stack in our architecture. The AWS stack is part of the previous LinkedIn Lead Accelerator product offering, and already contains various integrations to support Insight Tag management. The LinkedIn stack allows us to use the LinkedIn domain cookie to extract tracking information, as well as included widely-adopted systems like Pinot and the Campaign Manager UI, to build conversion tracking on LinkedIn. More importantly, because of our members-first policy, it allows us to respect our members’ opt-out preferences. Data modeling
There are three primary data entities in Conversion Tracking:
Domain: We do not want arbitrary conversion fires to occur for website domains not registered to the account owner of the Insight Tag. Therefore, each Insight Tag needs to be registered to a valid domain in order to function.
Conversion Action: The conversion action dictates at what point on the advertiser's web page we should consider it to be a conversion fire. The match rule is entirely URL-based, and can contain match rules like "starts with" "business.linkedin.com" or a specific sub page.
We only need one-time placement of the Insight Tag on an advertiser's web page for conversion tracking to work.
It lets us configure URL match rules, conversion value, conversion type, and other properties without changing the Insight Tag. This is key because account managers can edit the parameters of the conversion action independent of the people managing the web pages.
The Insight Tag during Conversion Tracking setup Attribution logic
When we register a conversion fire event, we filter out fires that are not associated with a valid LinkedIn member. Then, only a subset of these conversion fires can actually be associated to an impression and possibly a click that the member has seen or performed. To determine the true conversions, we use Scalding to perform the offline attribution job in order to determine the total conversions a campaign recorded in various attribution windows (default is 30-day lookback).
The attribution job does the following:
Determine the last click, and if not found, the last impression for a single user prior to converting for each campaign type. If no impressions or clicks are found, the conversion fire is dropped.
For most conversion action types, we de-duplicate repeated conversions after a single ad is shown (impression) or clicked. Therefore, we only count the first conversion that occurred if there are multiple that are found, unless more impressions occurred after the first conversion.