• Skip to primary navigation
  • Skip to main content
  • Skip to footer

Hypus

  • Home
  • Services
  • Technology
  • Team
  • Case Studies
  • Contact
  • Home
  • Services
  • Technology
  • Team
  • Case Studies
  • Contact

Uncategorized

Telling your POAS (Profit on Ad Spend) from your ROAS for increased profitability

Overview

ROAS (Return on Ad Spend) and POAS (Profit on Ad Spend) 

All marketing professionals are familiar with the ROAS (Return on Ad Spend) metric, which measures the revenue generated by a specific ad spend. This metric is widely used to assess the success of various marketing channels, campaigns, product groups, and even individual product performance. Key Performance Indicators (KPIs) are often set based on ROAS, advertising platforms provide features and tools to optimize for target ROAS, and skilled marketers put tremendous effort into balancing ROAS values within and between campaigns to eliminate inefficiencies. 

However, despite its widespread use, we all know that revenue is a top-line metric, and what ultimately concerns businesses is the real return on ad spend – how much profit each and every advertising investment generates. Many retailers offer a range of products with vastly different levels of profitability due to various business variables and factors. Relying solely on revenue on ad spend as a KPI, target, or advertising metric is obviously flawed and deeply inefficient. 

The true return on ad spend, often referred to as POAS (Profit on Ad Spend), holds more significance for businesses. Understanding the actual profit generated from each & every advertising expenditure is crucial for effective decision-making. To achieve this, it is essential to incorporate variance in profitability into reporting and campaign configuration. 

However, advertising platforms and tools currently lack easy solutions to account for variance in profitability in their reporting and optimization features. As a result, many businesses resort to setting ROAS targets based on average profit margins or the break-even threshold of their lowest-profit-margin products. This approach is of course flawed and inefficient – but is there another way?

The Challenge 

Hypus has been working with Client X, a well-established footwear retailer, for several years. Hypus proposed to help Client X maximize the impact of their advertising budget by introducing a POAS-focused (Profit on Ad Spend) strategy. 

The plan was to delve deeper into the factors influencing profit margins, develop a framework to systematically categorize Client X’s inventory based on profit margins, and crucially to define & implement a differentiated strategy to configure the marketing campaigns to optimize for POAS. 

The goal was to substantially improve the overall profitability of Client X’s marketing campaigns without increasing the spend whatsoever – giving Client X the choice of whether to bank the extra profit or allow themselves to set lower ROAS targets for their marketing campaigns & generate substantially more revenue from the existing budget.

Our Solution 

Analyzing Business Factors and Inventory 

Our process began by working closely with Client X to firstly identify & subsequently quantify an exhaustive list of the factors that affect profit margins on a product level. After intensive discussions covering everything including VAT regulation,shipping & handling costs, returns percentages, COGs, partnerships, marketing costs and much more we were satisfied that we had comprehensively covered all important factors and variables. One very important discussion remained. How would we deal with pricing – more specifically – discounting? Should we incorporate discounted prices into our profit margin calculations or not? After careful consideration we decided that our calculations should focus on RRP rather than sale prices as doing otherwise would essentially reverse the intended effect of the sale price. (which is to reduce the profit margin to increase demand). 

Systemisation

With all involved satisfied that we had identified and quantified all factors that led to profitability variance, our attention swiftly turned to systemisation. Could we devise a methodology & system which would allow us to continuously categorize products by profit margins on a very granular level, as and when new products or information surfaced? Our analytics team got straight to work, designing & creating a user-friendly database that worked in real-time to automatically calculate the profit margins for each individual product using a series of predefined rules (incorporating all the factors that had been identified as affecting profitability variance) and actual transaction data..

Implementation

With a system in place for continuous monitoring of profitability variance we were now ready to enter the most crucial phase of the project – configuring the advertising campaigns to optimize for POAS. The challenge was of course to find the best & most efficient way to implement a differentiated strategy accounting for profitability variance, within advertising platforms that had no access to the profit margin data and no obvious way of using the features and tools available to optimize for POAS. After extensive thought we identified two possible implementation models and their pros & cons:

Option 1 – Bucketing Products into Profitability Groups

The first option involved grouping products into several buckets based on their profitability levels. Each bucket of similarly profitable products would be assigned the appropriate target ROAS to achieve a balanced POAS across the different buckets. 

Pros: 

  • Easier to implement with less technical effort. 
  • Easier to add rules/overrides. 
  • Great visibility

Option 2 – Adding a New Purchase Conversion Action

The second option involved adding a new purchase conversion action to the website, sending Google Ads the actual profit (Revenue minus all costs of goods sold) for each sale as “conversion data”. With this data, campaigns could then be optimized using the data from the new conversion action. In effect we would be creating a Profit and POAS metric within the advertising platform allowing us to optimize for POAS in the same way we would for ROAS.

Pros: 

  • Allows for a highly granular differentiation strategy, with accurate profit data 
  • Takes into account product purchased not the product being clicked on. (A small but significant percentage of clicks result in a purchase of an altogether different product than the one clicked on). 
  • Compatible with other existing & future campaign structures.
  • No data segmentation required

We presented Client X with the two implementation models and discussed the pros and cons of each. After careful consideration we reached the conclusion that we would indeed embark on the more ambitious of the two options, adding a new conversion action, which although a hugely technical & skillful operation would really allow for a fully differentiated profitability variance strategy that was both versatile and durable. 

Work began on building an application that could systematically calculate the profit value for all website transactions, using the same predefined rules we created in the earlier stage of the project. Once calculated the application would match the transactions with those recorded in Google Ads and send an adjusted value for those conversions so that Google Ads now had a new profit conversion value in addition to the standard conversion value. If successful, we would effectively be creating a new “profit” metric within the Google Ads platform that would automatically assign a profit value to every click that led to a conversion. 

After extensive work and testing, working hand in hand with Client X, we successfully developed a fully automated, highly stable system that uploaded profit values for every recorded conversion within Google Ads. 

Highly confident that our solution was exhaustive, stable and most importantly game changing, we were ready to launch it on the account.

Results

After allowing some time for Google Ads to accumulate profit data for the campaigns, we began our work to guide Google to optimize for profit on ad spend rather than ROAS.

Satisfied that Google was indeed optimizing to target POAS on an original cohort of experimental campaigns, we swiftly rolled out our solution across the entire account. Our 

attention quickly turned to balancing the POASs between the different campaigns so that every advertising spend was now being directed for the most profitable outcome. 

The results were a fully balanced account optimizing for POAS. Without increasing advertising budget whatsoever and while maintaining the same overall ROAS, the campaigns were now utilizing the advertising budget far more efficiently. Spend was now being shifted from less profitable products to more lucrative ones, significantly enhancing the overall profitability of marketing campaigns. Client X were left with the envious choice of either banking the extra profit, or lowering the overall ROAS target (now that the existing ROAS was more profitable than previously) allowing for a massive increase in revenue. 

Conclusions

This case study is yet another illustration of how the best digital marketers can learn to incorporate business variables and data previously unavailable to Google, into Google’s bidding solutions for fully optimized performance. 

The transition from a ROAS to a POAS strategy has proved pivotal for Client X. Hypus implemented a comprehensive approach, analyzing profit factors, categorizing products, and choosing a sophisticated conversion action model that would allow the campaigns to be optimized for best possible POAS. The results have been transformative; maintaining ROAS but significantly enhancing campaign profitability, and at no extra advertising costs.

Making Smart Bidding Smarter 

Overview

Over the last few years Google has rapidly integrated advanced machine learning features into their advertising platform, releasing a wide range of automated bidding features – most recently PMAX, a new campaign type that allows access to the full Google inventory via a single campaign. Broadly referred to as ‘Smart Bidding’, these tools have generally delivered widely on their promise to help advertisers get more for less. 

Given Google’s sole access to endless consumer data, its unique ability to incorporate contextual consumer signals (for example, other search activity & history) into the bidding algorithms & allied with some of the most powerful machine learning capabilities in the world – it comes of course as no surprise that many advertisers today feel their best (and only) option is to simply hand Google the controls, sit back and wait for the promised land. 

But, at Hypus, with more than a decade of experience crafting bespoke algorithmic bidding solutions for our clients (dating back well before Google released their smart bidding solutions), we know there is a better way. Retaining some control over the advertising levers allows outstanding advertisers to cleverly integrate business factors and variables into Google’s ‘Smart Bidding’ features. Providing Google with crucial, intricate business data that it cannot otherwise access can transform Google’s Smart Bidding features from an (excellent) off the shelf, one-size-fits-all tool into a powerful & bespoke bidding solution. 

To do so, however, requires hard work and a unique blend of scientific & technical skills combined with an intimate understanding of each individual business. But, with the potential to unlock tremendous growth for a business – it’s an effort well worth undertaking.

The Challenge

Client X, a long-standing client, has been a great beneficiary of Hypus’s bespoke algorithmic bidding solutions for over a decade. Working together with Client X, our solutions over the years have been built to identify & incorporate all the relevant variables that affect performance into the data analysis that underpins optimized bidding. The ascendancy of Google’s smart bidding features, with a more limited access to performance data but the promise of improved performance, presented us with both a great challenge and a huge opportunity. We urgently needed to understand if Google’s smart bidding tools were accounting for all the variables that we knew affected performance. If not, was there a way to incorporate these variables into Google’s smart bidding tools? Could we craft a bidding solution that encompassed all that was good about Google’s ‘Smart Bidding’ solutions while using our intimate knowledge of Client X’s business and the different business factors & variables to leverage those tools for even better results?

Our Solution

Setting Google a Test 

With deep experience across many verticals & industries, and an intimate knowledge of Client X’s datasets – we had a very good idea of where to begin. 

Client X is a well established footwear brand, carrying a wide selection of products with each product often coming in a large size range. Stock coverage can vary greatly with products in stock across the entire size range or sometimes only in a few or even one size. Backed by historical performance data – we knew that a product in stock across the full range of sizes likely performed significantly better than when it only had a few select sizes in stock. A click on a product shopping ad directing to a landing page with a full range of sizes available would be far likelier to lead to a conversion than one directing to a product with only a single size in stock. Those clicks, being far more valuable toClient X, were worth spending more on. Simple as it was, we believed that this was something that Google’s machine learning was not (and quite possibly unable) to take into account. Our next step was to carefully design a test to prove that Google’s smart bidding was indeed not factoring for differing size availability.

We developed a method to split the performance data of Brand X into different buckets, depending on the amount of sizes available at the time it was being advertised. This would show us unequivocally how the same products performed over time – with differing size availability – and whether or not Google was taking this into consideration.

Table 1

The results, as shown in Table 1 above, were very conclusive indeed. For Brand X, performance was far worse when there were products with fewer sizes available than where there were products with many sizes available. Products with only one size in stock achieved a ROAS of a little over 2, whereas products with more than 6 sizes in stock were achieving a ROAS of 6 and beyond. 

Crucially, Google’s ‘Smart Bidding’ solution was not factoring for this huge performance discrepancy. Were it to have been, we would have expected a shift in spend from the worse performing categories to the better performing categories, until over time the ROASs achieved from the different categories converged. The table above, showing performance over a substantial date-range, confirms that Google indeed wasn’t learning and even with time was still sending substantial budget (almost 50% of overall spend) to worse performing categories, budget that could have been better deployed elsewhere – a huge inefficiency.

Coaching Google to learn

Discovering this great inefficiency led us directly to our next challenge. Could we find a way to coach Google to factor for differing size availability? What we needed was a bidding solution that was capable of incorporating this hugely important factor – amount of sizes in stock – into the smart bidding campaigns so that Google’s machine learning could treat each size variance individually and use the specific data for each grouping to optimize the results. 

Wary of disrupting Google’s machine learning too much by making the campaign configuration too granular we set about designing a clever & intricate system to move products with wider or narrower size ranges into different shopping campaigns, automatically. With our new system designed and implemented we awaited the results eagerly. 

The results, shown below in Table 2, were immediate and hugely satisfying. We could now see that we were successfully coaching Google’s machine learning to treat the size variance groupings differently. Google was now optimizing based on the specific performance data for each individual size variance group, shifting budget from the worse performing categories to the better performing ones. The ROASs for the same brand shown in Table 1 were now far more similar across the different size variance groups, with a much larger percentage of the spend focused on the better performing categories – a far more efficient outcome than before.

Table 2

Rolling out the solution

Now fully confident in our solution, we were determined to apply it across Client X’s entire inventory. But to do so most effectively, we needed to understand how size-availability affected performance for each and every individual product/product group. We knew that different products/product groups would have different drop offs in performance with less size options available, but were very conscious that in some instances particular size/s being in stock gave a competitive edge to the Client X offering. We set about building a system that would constantly monitor & analyze the performance data for each product/ product group to gauge the impact of differing size options on a very granular level. With a system in place to continuously monitor the size-availability performance curves we were now ready to implement our solution across the accounts, quietly optimistic that the results could be very significant indeed. 

The results since, have been nothing short of staggering. Jamie B, Head of E-Commerce at Client X, has assessed that introducing the size-availability factor into the bidding solutions is by far the biggest contributor to the 30% YoY growth they’re now seeing in their Paid Channel. He added this about Hypus – “Hypus don’t cease to amaze. From ideation to implementation there is no one in the digital marketing space that I’ve ever encountered that comes close to their level of sophistication and nuance; the results now as always speak for themselves.”

Conclusions

Google’s ‘Smart Bidding’ tools have revolutionized the digital marketing space. Offering a user-friendly, black box solution for almost every business it has opened up great opportunities for even the less savvy marketers; leveling the playing field. To remain outstanding, the best digital marketers must learn to incorporate business variables and data unavailable to Google, into those very solutions turning them from off the shelf, black box tools into bespoke, fully optimized bidding solutions capable of understanding the intricacies of their business. Coupling the power of Google’s technology with a deep and intimate knowledge of a business can lead to some staggering results.

Making clicks pay with CRO (Conversion Rate Optimisation)

Overview – CRO

In today’s fiercely competitive digital landscape, with businesses investing significant effort and resources to diligently acquire each click for their websites, it has never been more imperative to deploy a complementary strategy that ensures those hard-earned clicks result in significant conversions. 

Conversion Rate Optimization (CRO) is a systematic approach to improving a website or digital platform by harnessing data-driven insights to refine the user experience, remove obstacles to conversion & increase the proportion of website visitors who evolve into valuable customers.

CRO & Hypus

With every hypothesis & iteration requiring extensive development & testing time, traditional CRO can often be an arduous & time consuming task. Poorly constructed testing frameworks and misinterpretation of the results can often lead to unvalidated and misleading conclusions and arguably a position of worse clarity than before. 

As a unique digital marketing agency with a revolutionary scientific approach rooted in a deep commercial understanding and aided by best-in class technical skills, Hypus’s approach to CRO is fundamentally different to others in the market. 

Hypus’s approach involves – where possible – using data driven insight to spot the leaks and opportunities to arrive at hypotheses that are almost 100% of the time correct before subsequently going on to test them. This cuts out many stages and iterations and saves valuable time and resources. Intuition-led improvement suggestions are also very valuable in any CRO project. For these, Hypus uses data -where possible – to support or counter the suggestion before putting them forward for testing.

By meticulously analysing user behaviour, testing different elements, and making data-backed adjustments, businesses can unlock the full potential of their digital presence. But, as with all data projects, the complexities of running a CRO initiative for reliable and validated results can be daunting. Navigating through the myriad of variables, potential issues, and noise in the data requires a well-structured methodology, a deep understanding of user behaviour, and great technical skill. 

The Challenge 

Client X is one of the fastest growing businesses in the UK. They offer a comprehensive range of luggage at hugely competitive prices with a fast & efficient shipping service. Hypus partnered with Client X over 5 years ago and have been hugely instrumental in helping them design and implement the optimal digital marketing strategy that has underpinned their rapid growth. 

With such a strong offering and optimal digital marketing setup in place, Hypus felt that the time was right to focus on improving website conversion rates. A conversion rate of little over 1% (despite being normal within the online luggage space) felt wholly underwhelming for such a heavyweight in the luggage space. 

When thinking about improving conversion rates, Hypus believes that it’s far more empowering and enlightening to think in terms of the 90%+ that are not converting and how this very large percentage can be reduced. Ascertaining what percentage of the 90%+ that didn’t convert is attributable to one of various obstacles to conversion, offers both an indication of the scope for improvement and a clear direction on where to begin searching for improvement. 

In Client X’s case, Hypus began by determining that at the very least 10% of total website users were in immediate need of a piece of luggage. With a comprehensive range & highly competitive prices, there was therefore very little reason to believe that anything other than website design was causing many valuable users to bounce. 

Hypus were convinced that with a very carefully designed data project, mindful of all the complexities involved, they could substantially & demonstrably improve the CR to add £m’s to topline revenue at no additional cost – every business’s dream.

The solution

Hypus – with a wealth of experience crafting & implementing data projects and solutions for clients – were well aware that a data project of this nature required meticulous strategic planning, perfect technical execution and a great awareness of the potential disruptions that could materialise in the dynamic world of digital marketing.

Forming a Hypothesis

The first stage of the project involved understanding which area of the website was the ideal place to begin focusing our work on. What we were looking for was for a feature/section of the website that:

1. was or could be a heavily-used part of the website – where improvements would be the most impactful

2. we could easily analyse user on-site behaviour allowing our suggested improvements to be predominantly data- driven rather than intuition led. Using a combination of Google Analytics (behaviour reports) and HotJar (digital experience insights and behaviour analytics) heatmaps, we identified the area of the website that best met the above criteria. We began using the associated data to form a hypothesis about which improvements would give the user the best possible experience and lead to improving conversion rates & revenue per user. With a detailed list of suggested improvements to the chosen website section – heavily backed by our team’s in depth analysis of user behaviour – we were ready to run some tests.

Test Design 

We scoured the market for the most reliable website A/B testing tools that could host multiple versions of a website, randomly serving each version on a predefined percentage of the occasions that users requested the website. It was important too, that the A/B testing tool had good editing functionality allowing to rapidly draft the suggested improvements. We tested a couple of different options and settled on a tool that easily integrated with Google Analytics, as well as meeting our other requirements. 

Initially, we decided to simply duplicate the original version of the website and run two identical versions of the website in an A/B test so that we could ascertain whether or not the A/B tool itself was reliable and whether or not running the tool’s code affected performance. After collecting enough data to confirm that indeed the A/B tool was satisfactory we set up an A/B test pitting our new improved website version against the original and awaited the results. 

Troubleshooting

Bots 

After an initial period with little disruption our test ran into its first issue. For a 3 day period we detected some very unusual session activity. On further examination it became apparent that it was most likely a bot attack that was skewing the data leading to much higher than usual bounce rate on one of the versions. A decision was made to disqualify the period from the test. 

Fruit bugs

With the test back on track a couple of weeks passed until we were faced with our next challenge. There was a sudden discernible drop in performance on one of the versions of the website only, which was difficult to explain. Our data specialists dug deeper into the data and discovered that the drop in performance was isolated to safari browser users only, leading us to believe that a recent swathe of IOS updates was perhaps impacting our A/B testing tool, affecting the validity of the test. We decided to conduct a series of statistical tests, computing p-values for multiple segments of the data, split into smaller data ranges (yet still with significant data) and into different browsers. What we discovered confirmed our suspicions, that there was indeed statistically significant probability that the most recent IOS launch was preventing Safari browser users from being shown a particular version of the website. The call was made to disqualify this data too from the test. 

Results & Analysis 

With the test back on track and with significant data already accumulated we were now ready to analyse the impact of our suggested improvements to the chosen website section. We began by first isolating the data for each website version to include only where the users had interacted with the part of the website that was being tested. With this done, the results were ready to be quantified. Our new improved version of the website showed an astonishing 17.5% uplift in revenue over the original version. Using the previous 12 months’ revenue as a baseline this would translate into well over £700K of extra revenue over the coming 12 month period alone, with the figure reaching close to £900k based on previous growth projections of almost 25% for Client X over the coming period – see table 1 below.

Conclusions

Data projects, even after hypotheses and theories have been empirically substantiated & identified patterns successfully modelled, more often than not leave questions when it comes to measurement & quantification – leaving some still unconvinced about exactly how to proceed. It’s not often that a piece of data analysis measuring the impact of highly specific changes on a vastly complex and dynamic interface provides such clarity as to be undeniable. 

The CRO project Hypus ran for Client X – not without its pitfalls and challenges that could have thrown it off course – provided Client X with exactly this type of clarity – offering an immediate pathway to £700k extra revenue within 1 year, at no extra cost. 

And with the scope of how impactful to topline revenue a well constructed CRO project could be now in clear view, Hypus’s attention quickly turned to other parts of the website for the next phase of CRO.

Introducing Incre-“mentality” to your marketing

Overview

In today’s dynamic business landscape, marketing strategies have become increasingly complex and are constantly evolving. It has become very difficult to confidently differentiate between strategies & changes that truly drive positive outcomes and those that yield coincidental results. Yet, marketing budgets are often limited, demanding
judicious allocation of resources.

Incrementality testing provides an evidence-based approach to identify changes & strategies that provide the best return on investment. Businesses can focus their efforts on those that demonstrably contribute to their goals, avoiding wastage on ineffective tactics. By continually conducting these tests, businesses can accumulate a wealth of insights about their marketing campaigns, the effectiveness of different strategies, and how these evolve over time. These insights contribute to the development of more informed and refined long-term marketing strategies and provide concrete evidence of the impact of changes, making it easier to communicate the rationale behind decisions to internal teams, clients, investors, and other stakeholders.

Creating a proper incrementality test is a task fraught with challenges and intricacies. The nuances lie not only in the technical implementation but also in the strategic thinking that underpins the entire process. From defining suitable control groups to accurately tracking user behaviour, every step demands a meticulous approach to ensure the reliability and validity of the test results. In the face of uncertainties like data bleed and external noise, this in depth case study showcases the strategic decisions necessary to arrive at a test design that maximises insights while minimising confounding factors.

The Challenge

Hypus partnered with Client X, a highly successful British premium watch brand in mid 2021. Tasked with overhauling their digital marketing efforts, Hypus quickly identified the paid social channel as an area for short & medium term focus – given Client X’s unique offering, compelling brand story and library of high quality creative assets.

Despite success on paid social channels having eluded Client X previously, Hypus were very confident that with skillful management Client X could create a portfolio of paid social campaigns that could both run and be scaled quickly at a profitable CAC (customer acquisition cost). With immediate results and a platform-reported CAC that was clearly profitable,

Hypus advised Client X to allocate substantially more budget to the campaigns in the knowledge that efficiency improvements could counterbalance any natural increases in the CAC brought about by scaling. Over a period of 18 months,
spend on paid social increased massively, all while the platform-reported CAC was being maintained at the profitable level – much to the delight of all involved.

Despite all stakeholders being very comfortable in the knowledge that the paid social campaigns were very profitable, there still remained a nagging uncertainty. With notable product launches, favourable press coverage and a diverse advertising strategy – including TV ads and professional sports team sponsorship- no one was under any illusions that the platform-reported CAC, relying on very favourable attribution models, was in any way an accurate reflection of the state of play. The time was now right to build an incrementality testing framework that would more accurately ascertain the true CAC of the UK paid social campaigns, crucial for the next phase of investment in the channel.

The Solution

With deep experience running incrementality tests, Hypus were acutely aware that creating an effective incrementality test goes far beyond the surface, involving a profound comprehension of both the technical landscape and the underlying business dynamics.

A/B Test Design

It was clear that the best design for the testing was an A/B model with the creation of two distinct groups – one exposed to the paid social marketing campaigns (the treatment group) and the other not exposed (the control group). This would allow for a reliable benchmark to be established by which the effectiveness of the paid social campaigns could be assessed.

A & B

Creating two UK population groups, one to be targeted (treatment group) and the other to be suppressed (control group) raised the all important question: what should be the segregating feature/s? We needed a segregation that was possible within the paid social advertising platform, allowing us to target only one of the groups. Additionally, we required the same segregation feature to be applicable within Client X’s backend data, meaning we could assess the relative performance of the two groups. It soon became apparent that only a geographical split of the UK population met both criteria, but that there were some challenges to overcome.

Bleed & Noise

Despite there being location targeting settings in the chosen advertising platform it remained unclear both how the platform defined a user’s location, and how well it could identify the user’s location. Additionally, with movement between locations, a person could well see an ad in a targeted location and then subsequently purchase an item and have it delivered to a non-targeted location or vice versa.. All this could potentially lead to a “bleed” between the groups, where someone targeted would be identified as belonging to the non-targeted group and vice versa – bringing the behaviour of the two groups closer together than they ought to be.
Another point to consider was the potential for “noise” – whereby a particular area within either group was disproportionately affected by a certain factor leading to unbalanced performance data. In the case of Client X, a good example of this would be the area within close proximity of the professional sports team that they sponsored; disproportionately affected by their exposure and feelings towards the team and its
sponsors. The best way to resolve the “noise” issue would be to randomly split the entire UK into small areas and assign them to group A or B, but while it would be useful in negating the noise problem it would certainly compound the “bleed” problem by creating many, many more borders between the two groups.
After much thought it was decided that the best option would be to establish a single border between the two groups – splitting the UK into two, geographically – that was likely to have the least amount of “bleed”. What we were looking for was to draw up a virtual border that ran through areas with low population density and that had few commuter routes. Once establishing the virtual border we would take steps to mitigate the “noise” by analysing different regions within each group and comparing their performance to understand what was “noise- affected”, and to what degree.

Drawing a line

Our next step was to investigate UK population density. Using a population density map map 1 we began work on drawing up potential virtual borders for the two groups. We narrowed down the options to line “a” and “b” in the illustration below.

Whilst line “a” is a cleaner line with likely less “bleed”, only 9% of Client X’s sales were North of the line. This would mean that the testing would have to run for a very long time to get meaningful results. Line “b”, with a sales split of 60% to the south and 40% to the North, presented an opportunity to run the test for a much shorter period for conclusive results. Being far enough North and West of London to avoid the commuter areas, whilst being just far enough from the West Midlands metropolitan areas, we felt satisfied that this was the best virtual border for our project. We would suppress the areas to the North in our advertising campaigns, whilst maintaining advertising to the South.

Matching area codes

We now required a simple, congruous method to implement the new segregation within both the advertising platform and Client X’s backend data. To do so, we decided to draw our virtual line using area code (letter/s that preface postcodes) borders trying our best to keep it as aligned as possible with our line “b”. We could then compile a list of area codes that we would target and a list of those that we would suppress. The illustration below shows how we managed to create Line “b” using area codes.

Negating any lingering effects

With the UK population now divided into two, geographically, we were ready to implement our incrementality test. We configured the advertising platform to target only area codes South of Line “Y” and ensured we could segment backend performance data by area code, meaning we could now compare the relevant performance of the targeted and non-targeted population groups.

In order to gain meaningful insight we decided that we would disqualify the first two months of results to negate the lingering effect of previous targeting to the Control group (pre- suppression), which would have brought many consumers into Client X’s funnel only for them to purchase after ad-targeting had already stopped

Interpreting the results

Despite being finally satisfied that the test had been designed and implemented for reliable & meaningful results, we still remained very conscious that the results would not tell us unequivocally what the CAC for the paid social campaigns was, due to “bleed” and “noise”. But what it would tell us was a broad range of the CAC and whether it was close to the platform-reported CAC or several times higher – crucial information to define the strategy around the next phase of investment in the channel.

Conclusions

In an era where data-driven decisions are paramount, the adoption of an “incre-mentality” is essential for informed decision-making in the ever-evolving marketing landscape. It enables companies to allocate resources effectively, maximise returns on investment, and ensure that marketing strategies are grounded in evidence-based data.
However, improperly executed incrementality tests can lead to reinforced misguided assumptions and contribute to misinformed decision-making. Meticulous planning and strategic thinking in designing reliable experiments is required to gain truly meaningful and valuable insight that can drive a business forward.

Sources

https://www.researchgate.net/publication/347530093_The_theoretical_potential_for_large-scale_underground_thermal_energy_storage_UTES_within_the_UK/citation/download

Footer

Home

services

case studies

team

contact

Hypus.co.uk

020 7126 7125

info@hypus.co.uk

  • LinkedIn

 *Scientific measurement
in marketing. What?

Scientific measurement in marketing may not require a lab or a white coat. But it certainly doesn’t just mean adding a token data scientist/s to your marketing team.

What it does demand is adopting an incre“mentality”, delving deeper into the real incremental value generated by every marketing channel and campaign.

It means leaving behind superficial observation and platform reported metrics, recognizing that these are illusions and delusions.

It requires the application of rigorous scientific principles, methodologies, and statistical techniques to measure the effectiveness of each and every marketing effort. 

It goes beyond crunching the numbers – it demands a deep commercial understanding and great technical skill to extract meaningful insight. 

It’s a transformative approach that brings clarity and precision to the world of data analysis and it’s key to leveraging marketing to maximise growth opportunities.

Scientific measurement is what we at Hypus do best –
decoding marketing data to reveal crystal-clear and profound insights, enabling our clients to make informed, strategic decisions and achieve measurable success.