Yesterday's post sparked quite a reaction as people tweeted, blogged, and asked questions about research. I think it's a good thing when we ask about the accuracy and helpfulness of certain research. But a recurring question is, how do you tell good research from bad research?
A couple of years ago, this is the question I was asked to address while speaking at the Washington Post building in Washington, DC, about religion research to the Religion Newswriters Association, a gathering of secular reporters who cover religion. My focus was to address the question of how to discern good religious research from bad religious research-- and it sparked lots of discussion there as well. So pastors are not the only ones asking this question. It's a recurring theme and an important issue for us to address.
I've addressed this in depth in Christianity Today where I talked about "Curing Christians' Stats Abuse." The reality is, Christians have a bad reputation-- earned, I might add-- for using statistics wrongly. However, several people have asked, "How do I know if I'm using them rightly?"
Part of the answer is simply to take the time to move beyond the soundbite. Ironically, soundbites are often correct, but they can also be incomplete. I've made the mistake myself, but now hope to do (and help others do) better.
For example, in yesterday's post, I specifically pointed out the divorce statistic that Christians divorce at the same rate as non-Christians. Well, is this statistic true? Not in my understanding of what a Christian is. And my guess is, it's not true in what your understanding of what a Christian is either. Keeping in mind that 75-80 percent of Americans self-identify as Christians, if you simply ask someone "Are you a Christian?" and 80 percent say "yes," and "Have you been divorced?" and 50 percent say "yes," and then you announce that Christians divorce at the same rate as non-Christians, you have actually quoted (I believe) a misleading statistic (particularly if you are a pastor and people think you should know what a "Christian" actually is.
So if you are a pastor, and you stand up before your congregation and say that Christians divorce at the same rate as non-Christians, I think you're misleading your congregation. This is because most of us really don't believe that almost 80 percent of Americans are Christians, even though they identify as Christians on a census form.
This brings back an important discussion. I've talked about this before as the way people use the word "Christian." There are three categories that I have delineated earlier. I call them "demographic" Christians, "church attendance" Christians, and "faith conversion" Christians. Or, to be a bit more pithy, let's call them "census Christians," "church Christians," and "converted Christians."
It is a mistake of category to take census Christians (those who identify as Christians on a survey form), and use statistical quotes as if they are representative of born-again genuine followers of Jesus Christ (in other words, not all who identify as "Christian" fit the description of a genuine believer).
So, whose responsibility is it to make that distinction? Well, it is two-fold. First, it's the researcher's responsibility. For example, when we release data at LifeWay Research data, we'll often go to great length to say that certain groups of people are "churchgoers," "self-identified Christians," "born-again Christians," etc. It's important to define the category to which you are referring. For example, not all churchgoers are Christians. Not all people who self-identify as Christians are converted Christians. And so it's important to take the time as a researcher to make the distinction between the categories.
It's also essential for you, as a pastor or church leader, to take the extra time to see what the researcher is saying. For example, if you hear a statistic that says that Christians divorce at the same rate as non-Christians, ask the question "How is the researcher defining Christian?" If the researcher is using Christian as a demographic category, and you don't believe that everyone who calls himself or herself a Christian is a Christian, you shouldn't stand up before your congregation and say that Christians divorce at the same rate as non-Christians.
Perhaps a better way to say it would be that in a survey, people who identify as Christians, which is over 75 percent of the population, divorce at the same rate as the other 25 percent of the population-- which, to be perfectly honest, is not big news. So, part of the responsibility is with you as a pastor. For example, one of the sources of this research that says Christians divorce at the same rate as non-Christians actually made the distinction between the two, but pastors who quote the statistic that Christians divorce at the same rate as non-Christians are the ones who actually perpetuate the myth that genuine Christian faith does not impact divorce statistics.
So, with just a few extra minutes, a couple of Google searches, and a little bit more information, you can properly lay out and use statistics in your message. I do not think it's your responsibility to answer every question that every survey asks, but discernment (even a dose of healthy statistical cynicism) is key.
So, keep using statistics. I think they're important, I use them, we create them. But use them discerningly and with integrity, and use them in a way that advances the cause of Christ.