I thank everyone who commented on my post from last week, publicly or privately, whether you felt it described you as a reader or not. You reminded me that, as I try to get a better understanding of readers, I have a ‘drunk searching for keys under the streetlight’ issue.
My ‘original’ research is binge-reading Amazon reviews of books and sharing the analyses with my newsletter—speaking of that, I’m running a silly giveaway to build my email list, if you need an antique book from Victorian England with a broken spine and have a U.S. postal address, this is a great giveaway for you.
Amazon book reviews are a convenience sample—I only get information from people who post them. That is, the readers who have especially strong opinions. Lukewarm/casual readers are severely underrepresented. (But how do I know that? Actually, I don’t. I’m making an assumption. I don’t have concrete evidence to prove that lukewarm/casual readers are underrepresented in Amazon reviews).
One reason I focus on Amazon instead of GoodReads is Amazon is much less ‘social.’ Because GoodReads is ‘social media,’ judgement by others pressures users to write reviews which make them look good rather than the most honest description of how a book made them feel. That can happen on Amazon too, but since Amazon is less social, reviewers have less to gain by putting on a show for others. When I compare Amazon reviews and GoodReads reviews, Amazon reviews feel like they have less social signalling.
Every other means of understanding readers also has a street lamp problem. The What Should I Read Next? podcast relies on interviews with people who want public interviews—an even more slanted group than Amazon reviewers. I’ve considered watching more YouTube book reviews to get the nonverbal body language, but that a) requires much more time and b) people who post book reviews on YouTube are a much narrower group than people who write reviews on Amazon.
Jellybooks doesn’t have the problem of people saying things to improve their social status since they track reader behavior through electronic devices and individual data is not public. However, they only have data on people who consent to Jellybooks spying on their reading behavior, and only on electronic devices. Furthermore, Jellybooks primarily deals with not-yet-published and recently published books, so the readers who sign up are skewed towards people who like the idea of reading books before publication.
Book historians who study which books/pages are worn out from use and which ones are in pristine shape can only study the material they can get.
How do I handle the ‘searching under the street lamp’ problem? First, we can combine these methods for a more complete understanding. I focus on Amazon reviews because it requires no money (given that I already have internet access) or fancy technology. I don’t have to recruit hundreds of readers for studies and I don’t have to collect old books to study how worn out they are. To add more nuance, I read the findings of people who have more resources than I do.
Fortunately, searching under the street lamp is fine.
I’m not looking for a specific set of keys.
Since I don’t have a specific hypothesis to prove/disprove, I don’t need scientific rigor. Rather, I want to understand the environment. Searching under a streetlight isn’t as accurate as seeing the entire landscape, but it still tells me a lot about my surroundings. I might find money on the ground somebody else lost. Maybe a pretty dandelion grows in a sidewalk crack. Someone else’s id card might be down there, and if I return it, I’ll earn a pat on the back. All these discoveries would be cool.
Restraining my confirmation bias is the most important thing. The temptation to assume all readers are identical to me, or worse, to assume readers are just like my idealized projections, is strong. I need reality checks. Reading ridiculous quantities of Amazon reviews isn’t the most accurate way to research behavior, but it’s great for challenging my confirmation bias.
Your comments challenge my confirmation bias too. Thank you.
The Notes Which Do Not Fit 