The data didn’t make sense.
The American Bible Society (ABS) and Barna Group researchers looked at the results from 1,000 cellphone interviews asking people about their engagement with Scripture. The numbers seemed to show more people were reading the Bible—a lot more. But nothing else had dramatically changed.
There were not more people praying, or going to church, or identifying religion as something important in their lives. There wasn’t a corresponding increase in people saying the Bible was the Word of God. It was just this one metric, breaking logic and defying trends.
“When you get a big surprise in the social sciences, that’s often not a good thing,” said John Farquhar Plake, lead researcher for the State of the Bible 2020. “We were seeing from the cellphone responses what we considered to be an unbelievable level of Bible engagement. You think, ‘That might be noise rather than signal.’”
Researchers found the cause of the “noise” when they compared the cellphone results with the results of their online survey: social desirability bias. According to studies of polling methods, people answer questions differently when they’re speaking to another human. It turns out that sometimes people overstate their Bible reading if they suspect the people on the other end of the call will think more highly of them if they engaged the Scriptures more. Sometimes, they overstate it a lot.
The ABS and Barna decided to do 3,000 more online surveys and then throw out the data from the phone poll. For the first time, the annual State of the Bible study was being produced using only online survey data.
Christian groups are not the only ones changing the way they measure ...1
Already a CT subscriber? Log in for full digital access.
Have something to add about this? See something we missed? Share your feedback here.
Subscribe to Christianity Today and get access to this article plus 65+ years of archives.
- Home delivery of CT magazine
- Complete access to articles on ChristianityToday.com
- Over 120 years of magazine archives plus full access to all of CT’s online archives
- Learn more
Did You Go to Church Last Week? Might Depend on Who’s Asking.
This slideshow is only available for subscribers.
Please log in or subscribe to view the slideshow.