This is the last in a 9-part series of “quick hit” blogs on the quickest way to uncover hidden revenue from leasing based on the presentation Bryan Pierce, Carol Enoch and Donald Davidoff gave at NAA’s 2024 Apartmentalize conference.

Up until now, this blog series has focused on a variety of different amenity “fails” that arise essentially from configuration issues, whether accidental or intentional. The key is to be able to look at the data in formats that make it easy to quickly identify and fix those errors.

Opportunity #9: A/B Testing

For our final amenity opportunity in this series, we turn to amenity pricing itself. How do we know whether the price we are adding on for a balcony, or a ceiling fan or a view, etc. is correct? Scientifically speaking, we should run “A/B tests” where we lease some units with the amenity in question and others without and compare the results. If the homes with the amenity lease faster than those without the amenity, then we can (and should) raise the amenity value; if the homes with the amenity lease slower than those without the amenity, then we should lower the amenity value lest we suffer from increase vacancy loss; and if they both lease at the same pace, then we’ve got the amenity priced “just right!”

So how do we do that? Fortunately, our regularly business operations give us a good approximation of A/B testing. We lease a variety of apartments with a variety of amenities, so we frequently are leasing some units with a particular amenity and others without. The timing isn’t exact, and there may be some “bundling” issues, but by and large it’s a pretty good A/B test approach.

The next step is to determine what metric best represents “how quickly a unit leases.” People often jump quickly to “days vacant,” as that’s a metric oft-measured and critical to leasing pace. However, any some portion of vacant days has more to do with turn times than amenity pricing, and what do we do with pre-leased notices? In our experience, a much better metric is “days on market” (DOM). This measures how long the unit was exposed to the market. If you’re not familiar with the term, DOM is notice date to application date for regular NTV scenarios and MO date to application date for skips/evictions or any scenario without a prior NTV.

Lastly, the determination of what is “faster, slower or the same” is not straightforward. Statistically speaking, this is not simply whether a cohort has an average DOM above or below another cohort. And what if the difference is tiny, say 51 days vs 50 days? Before you say those are the same, then at what point are they not the same? 52 vs 50? 53? Not until 60?

Again, fortunately, there’s a solution. The world of statistics long ago solve this problem. There are statistical tests that take the average, the number of observations and the standard deviation to assess the likelihood that one cohort’s average DOM is higher or lower than another’s. Because of the relatively low transaction density of our leasing world, this is particularly important.

It also means that there is no “yes/no” answer to the question of whether an amenity is priced correctly. Rather, these tests allow us to say, “There’s a 92% chance the price is too low,” or “There’s only a 32% chance the price is too high.” So we need to make a business decision as to how confident we need to be to act on a difference…spoiler alert, we typically use 80%.

Armed with all of this, we can calculate DOMs, run the statistical tests and make changes where appropriate. A few months later, with enough new leases on the new pricing, we can repeat the process and thus continually monitor amenity pricing. This is obviously much better done with software than manually.

One final note: beware of software that uses anything other than your own leasing data. First, until the present litigation around the use of non-public comp data is resolved, this could result in even further legal action. Even if resolved, new laws like the one in San Francisco make this approach illegal. Second (and maybe more importantly), it’s not scientifically sound. As we talked about in the first eight installments, there are many amenity configuration errors out there. You can be sure that your own configuration is clean before doing a pricing review, but you can’t know whether your competitor’s configurations are clean. Based on our 20+ years of experience, we’re pretty sure they are not!

There you have it. Nine different things to look for in your unit amenity configurations and pricing. Use these to dig for that hidden treasure!