Ignore any paper based on self-reported datacreated: ; modified:
- “60% of the “vegetarians” reported having eaten meat within the last 24 hours.” source (perma) (via JS Denain)
- “Administrative records matched to commonly-used surveys (ACS, SIPP, CPS) show that among actual food stamp recipients, 23-50% report not having received these benefits.” source (perma)
- “Almost all research examining the harmful effects of social media (SM) have used self-reports of SM. But, new research has found, such self-reports are hardly related to actual SM use. In fact, the average correlation between self and actual is about .20” source (perma)
- “dieters underreport consumption by hundreds of calories. And that’s 24-hour recall, which is supposed to be the most reliable.” source (perma)
- “People’s announcements about whether they would cheat or act honestly in an experiment bore very little relation to their actual behavior, casting a shadow on survey research.” source (perma)
- paper: “Those who said that they would return the entire amount actually did so more often than those who said that they would keep all the money, but the difference is small and not statistically significant at conventional levels.”
- “Program receipt in the CPS [Current Population Survey] is missed for over one-third of housing assistance recipients, 40 percent of food stamp recipients and 60 percent of TANF [Temporary Assistance for Needy Families] and General Assistance recipients.” source
- “Using the combined data rather than survey data alone, the poverty reducing effect of all programs together is nearly doubled while the effect of housing assistance is tripled. We also re-examine the coverage of the safety net, specifically the share of people without work or program receipt. Using the administrative measures of program receipt rather than the survey ones often reduces the share of single mothers falling through the safety net by one-half or more.“
- paper (perma)
- “For the weekly time spent online, the rank correlation between self-report and log data was r = .38 (p < .01), for the frequency per month measure, the relationship was even weaker (r = .29, p < .01).” source
- “State survey estimates of [alcohol] consumption accounted for a median of 22% to 32% of state sales data across years.” source (perma)
- People support both higher taxes and lower taxes simultaneously, depending on how the question is framed source (perma)
- “precisely 75 percent said the right level for top earners was 30 percent or below. The current rate for top earners is 35 percent. … Since 1998, Gallup has asked Americans whether they believe the government should “redistribute wealth by heavy taxes on the rich.” This year, 52 percent agreed, tying the all-time high set in 2013.”
- From Cards Against Humanity Pulse of the Nation:
- 57.9% say that “men and women are equal in their talents and abilities” and that “women in positions of power are better able to feel empathy than men”
- 56.4% say that “men and women are equal in their talents and abilities” and that “women are better at multi-tasking than men”
- 39.1% of Democrats say that “it is wrong to negatively stereotype people based on their place of birth” and that “southerners are more racist”
- 34.5% say that they “trust the scientific consensus” and that “GMOs are not safe to eat”
- 8.6% say that “humanity will not survive the next 100 years” and that “America will survive the next 100 years”
- 3.6 million US households report cash income below $2/person/day. If you then check this against admin data on income+transfers + include substantial assets, less than 10% of those households are actually in extreme poverty
Finally, let me share a bit from my personal experience: since I started my monthly email newsletter, a total of 34 people subscribed and then unsubscribed from it. Out of these 34 people, 6 (17%) told MailChimp that they “did not sign up for this newsletter”. Fun fact: MailChimp tracks every sign up and each one of these people filled out a sign up form on my site.
And of course, don’t miss the study that shows that “93% of teenage boys totally have sex all the time”.
Afterword: yes, the title is an exaggeration. As @dklnt notes on twitter, “self-reported race matches amazingly well with genetic data (>95% of the time)”. My point is that by default we should be way more skeptical of self-reported data than we are right now.