11 Nov 2021 Search

When you measure include the measurer. — MC Hammer

Rebuilding web advertising without the third-party cookie is not really a privacy story. It’s a market design story. And it’s probably a mistake to think about third party cookies as some ideal system of which a fraction needs to be clawed back while keeping privacy. The limitations to measurement are complicated. (There is a lot of math needed to understand the reasons behind The Refrigerator Test and I’m still trying to figure it out.)

Some good background info.

Consumer Privacy Choice in Online Advertising: Who Opts Out and at What Cost to Industry? by Garrett Johnson, Scott Shriver, Shaoyin Du :: SSRN Though consumers express strong privacy concerns in surveys, we find that only 0.23% of American ad impressions arise from users who opted out of online behavioral advertising. We also find that opt-out user ads fetch 52.0% less revenue on the exchange than do comparable ads for users who allow behavioral targeting.

The Identity Fragmentation Bias by Tesary Lin and Sanjog Misra. This paper studies the identity fragmentation bias, referring to the estimation bias resulted from using fragmented data. Using a formal framework, we decompose the contributing factors of the estimation bias caused by data fragmentation and discuss the direction of bias. Contrary to conventional wisdom, this bias cannot be signed or bounded under standard assumptions.

Inferno: A guide to field experiments in online display advertising by Garrett A. Johnson. 19 Jul 2021 Online ad experiments suffer from an extreme statistical power problem, which limits how much can be learned from experiments. … Some of the coming changes to online advertising promise improvements for experimenters.

The Unfavorable Economics of Measuring the Returns to Advertising - 2015-lewis.pdf These initial findings show that when advertising at a level of intensity typical of digital advertising, [randomized controlled trials] require sample sizes in the single-digit millions of person-weeks to distinguish campaigns that have no effect on consumer behavior (100% ROI) from those that are profitable (ROI >0%). … Identifying highly successful campaigns from ones that merely broke even is not an optimization standard we typically apply in economics, yet our analysis shows that reliably distinguishing a 50% from 0% ROI is typically not possible with a $100,000 experiment involving millions of individuals.

Internet Activity Bias Causes Lumpy User Behavior by Jakob Nielsen This phenomenon is called activity bias: some days, people do a lot online; other days, they do very little. On very active days, people are more likely to do both Activity A and Activity B, no matter what A or B might be…. Crucially, even if there is no relationship between A and B, the very fact that you observe users doing A means that they are likely to be having one of their more active days and therefore are also more likely to do B.

Much Better Yellow Pages. Much Worse Television. by Bob Hoffman