Always Read the Study!

Let me be honest right up front. Today’s post is a thinly-veiled rant with a lesson shoehorned into it.

That said, I think it is a valid rant and an important lesson.

The Rant

Kathy Orton at the Washington Post ran a story the other day entitled “Homeowners are paying less, renters are paying more to live in Washington.” This article was sent to me, as I recently stated that in my area it is “much (much much) cheaper to rent than to buy.” Obviously I must have missed something if the venerated Washington Post disagrees with me.

The article opens like this:

“At the same time renting in the Washington area is becoming more expensive, homeownership is becoming cheaper. Low mortgage rates and moderating home prices have made owning a home more affordable. Rising rents have made leasing more costly.

A recent study by Apartment List found that the gap between what a renter pays and what a homeowner pays for housing in the D.C. region is growing wider by the year. The disparity between the two is among the greatest in the nation.”

Wow! So not only is renting more expensive than buying, but the gap between the two is growing and is among the widest gaps in the whole country?! Clearly I missed something. Hit me with some numbers, Ms. Orton. Show me what I missed. Maybe a chart?

Chart from Washington Post article
Chart from Washington Post article


This chart doesn’t at all say what the author says it does! Instead, it shows the movement of renter costs and owner costs since 2007. The cost of renting in the DC area has increased 12.5% since 2007, while the cost of owning has decreased by 15.9% since 2007.

This tells us nothing about the actual costs of buying or renting. Instead, it tells us how they compare to 2007.

Certainly an interesting choice to use 2007 as the benchmark when discussing the change in housing costs. I wonder if the decline in costs might have something to do with the fact that 2007 was the peak of a housing boom…

So the cost of owning has decreased since the peak of the housing boom. That is to be expected. What about the 12.5% increase in rental prices? That must be pretty outrageous right?

Well…not quite. The cumulative rate of inflation from 2007 to 2014 (the last date the study pulls cost data from) was 14.2%. This means that rental prices actually increased slower than the general inflation rate.

So I had a lot of suspicions and quarrels with the way the article was written, but I could not know for sure whether I was right or the Washington Post was right without some hard numbers. Luckily for me, the study that the article cited presented its data.

I opened the study, scrolled down to the “Renter and owner cost by metro, 2007-2014” section, and searched for Washington, DC.

In 2014, the median owner paid $2,202 per month. The median renter paid $1,525.

Not only was owning more expensive, but it was 48% more expensive!

The only reason that the chart that the Post presented looks the way it does is that in 2007 owning was 93% more expensive!

The gap between what a renter pays and what a homeowner pays for housing in the D.C. region is shrinking. This, despite the author’s statement that “the gap between what a renter pays and what a homeowner pays for housing in the D.C. region is growing wider by the year.”

The gap is shrinking, but it is still huge. It costs 48% more every month to buy than to rent in the D.C. metro area.

The Lessons

The first lesson to take away from this is to challenge everything.

This is not to say that you should assume that everyone is lying all the time or that you should give no weight to expertise or authority. A person coming from a position of authority or expertise should certainly get much more weight up front than some random person on the Internet.

However, even experts are wrong from time to time and people speaking from authority aren’t always completely objective in their analysis. Go ahead and give them the benefit of the doubt, but be willing to question if you spot holes in their logic or research.

This leads nicely into the second lesson, which is always try to poke holes in an argument. It may not be appropriate to verbally do this in a friendly conversation with someone (I am not advocating being that guy) but always think through where the holes in an argument may be. This is important whether you agree with the person or not and is an important tool for avoiding confirmation bias.

This is especially easy when the argument is made in writing. You can go back and reread a section or compare different parts of the article. You can stop and make sure that the pieces of the argument flow and make sense together. You can find a contrasting opinion and see which you think is better supported.

Finally, for any story that cites a study: READ THE STUDY! The goal of television news is to get viewers. The goal of radio shows is to get listeners. The goal of news websites is to get clicks. To do this, they will often oversell new studies or research findings.

This is not to denigrate journalism. Everyone’s gotta make money, and (most) news sources certainly aim to inform the public and perform the noble duties of the fourth estate.

But they also need clicks. And you’re more likely to click on a story entitled “Bacon, hot dogs and processed meats cause cancer, WHO says,” that declares that your risk of cancer jumps 18% than one that calmly points out that eating the equivalent of a hot dog every single day increases your lifetime risk of developing colon cancer from 5% to a bit under 6%.

Context matters. Seeing the cost of renting rise and the cost of ownership drop tells a very different story if you don’t know the actual numbers behind them. Click through to the study and see for yourself.

6 thoughts on “Always Read the Study!”

  1. Yeah, you can use statistics and studies to say anything you want if you’re willing to twist a few things. That’s definitely misleading.

Leave a Reply

Your email address will not be published. Required fields are marked *

CommentLuv badge