Here’s the third installment of my series on Facebook ad demographic optimization.  In the previous posts, we defined the problem of Facebook advertisers misspecifying demographics .  In the second post, we examined the results of Facebook ad clusters derived from text mining . This instalment picks up where we left off by showing a method for determining the optimal demographics within a homogenious ad cluster.  Once the overall distribution of responses has been charted for each demographic, we explore a way to determine if a single ad shares the same response rate distributions.  Then and only then, can we determine if an ad has been misspecified.

Optimal Targeting

Once clusters of similar ads are identified, the relationship between the probability to click and user demographics can be explored.  Certain levels of the demographic predictors would exhibit a higher click probability than others; men are more likely than women to click on get six pack abs.  Candidate variables for a click model would include, but not be limited to: location, age, sex, education, relationship and interested in.  If probability to click by demographic can be modeled for each ad group, Facebook would be able to identify the target range for an ad group based on the number of clicks per day the advertiser wanted to receive.  If the advertiser’s budget were small, Facebook could serve the ad to the most targeted range.  If the budget were large, they would have to increase the range size serving ads to less and less targeted user groups.

Logistic regression using a selection method can be employed to determine the significance and contribution of each demographic to predict the probability of a click.  The interested in variable may need to be dropped because of insufficient frequency counts (small number of LGBT responders) at different levels of the other demographic predictors to prevent quasi-separation.  Other variables such as keywords and workplaces have too many response levels to be considered in the model without binning or clustering the data.  Additionally, location will need to be binned by region or clustered using census data to reduce the number of predictor levels.  Age will need to be plotted by the logit to determine if it should be entered as a continuous, quadratic or cubit predictor.

Below is the equation for the proposed logistic model:

Because a binary response variable exhibits non constant variance, coefficients in logistic regression are determined using maximum likelihood estimation as opposed to a least squares method.  An examination of the distribution of the predictor variables by the response coupled with variable coefficient scores and contrast statements can be used to understand how the probability of clicking changes at different levels of the predictor variable.  The results of contrasts will reveal what demographics to select to obtain an optimal click through rate for a specific ad cluster.

Ultimately, the optimal demographic range depends on the number of clicks an advertiser wants per day.  The larger the number of clicks desired, the larger and less targeted the optimal range would become.

Ad demographic Comparison

After the optimal and overall response range has been recorded for an ad cluster, Facebook could determine if a single ad’s demographics are over targeted with a comparison of distributions.  Before recommending that a test ad change its demographic range, one would first use statistical methods to compare the distribution of responders in the test ad to a similar set of responders from all ads in the same cluster.  If results of the tests indicated that the distribution of responders in each group came from the same population, it would be appropriate to assume that the test ad could expect to obtain similar response rates for a demographic range that its parent cluster achieved.  If the demographic range exhibiting the highest response rate was not captured by the test ad’s specification, the test ad would be deemed over targeted.  To ensure an apples-to-apples comparison, a sampling would control for all demographic characteristics between the test ad and its parent ad cluster.

Continuous Demographics

Assuming the ability to take a large sample, the distribution of responses for continuous variables could be compared using Kolmogorov-Smirnov test.  Age data was simulated for responders of a test credit score (CS) ad and for responders from all other CS ads using SAS 9.1 .  The distribution of response frequency by age is below:

Kolmogorov-Smirnov was used to see if the distribution of response was statistically different within the test range for the test ad and all ads.  An insignificant p-value (alpha=.05) indicated that the distributions of responders were not unequal.  This result suggests that the test ad could improve response rate by widening or shifting the specified age range to include the top of the response distribution for all CS ads.

Categorical Demographics

Response frequencies of categorical demographics can also be examined to determine if the distribution of responders for a test ad is the same as the distribution of all same cluster ads.  As with continuous variables, samples would need to be carefully taken to ensure a proper comparison.  Response data was generated for a test CS ad and for responders from all other CS ads.   Relationship status values were assigned based on two samples of one thousand values chosen at random from the binomial distribution.  Below is a graph of the frequency of responders by relationship status:

A Chi-Square statistic was computed to determine if the frequency of response counts were significantly different for the test CS ad vs. all CS ad values within the test range.  An insignificant p-value (alpha=.05) resulted in failing to reject the null hypothesis that the samples are drawn from the same population.  These results suggest that the test ad group is over targeted and could increase response rates by including married users.

In the next post, we’ll discuss the implications, extensions and pitfalls of the method laid out in the previous three posts.