In some ways the study is impressive. Using written transcripts of exchanges captured by body cameras, the authors had independent raters gauge how respectful, formal, etc., the police officers were. They also developed a computer algorithm that could give such ratings to a much bigger collection of transcripts — nearly 37,000 different utterances from about 1,000 police–civilian interactions. They statistically controlled for “the race of the officer, the severity of the infraction, the location of the stop, and the outcome of the stop.”
Both the humans’ ratings and the computers’ suggested a racial gap in respect. Interestingly, the race of the officers didn’t seem to matter, and interactions became more respectful and less formal as the conversations went on.
Notice something missing, though? They didn’t account for how the civilian treated the cop. They didn’t rule out the possibility that black civilians are more confrontational toward officers (or even sound that way without meaning to), and that officers are less respectful only in response to the way they’re treated.
Here’s how the authors address this problem:
It is certainly possible that some of these disparities are prompted by the language and behavior of the community members themselves, particularly as historical tensions in Oakland and preexisting beliefs about the legitimacy of the police may induce fear, anger, or stereotype threat. However, community member speech cannot be the sole cause of these disparities. Study 1 found racial disparities in police language even when annotators judged that language in the context of the community member’s utterances. We observe racial disparities in officer respect even in police utterances from the initial 5% of an interaction, suggesting that officers speak differently to community members of different races even before the driver has had the opportunity to say much at all.
It’s true that the human raters saw an utterance from the police officer paired with the driver’s utterance that preceded it. But they were not asked to correct for the driver’s attitude; they were just asked whether the cop was being respectful — and transcripts can’t make clear whether an officer is reacting to a driver’s tone or body language. As for the initial 5 percent of the interaction, well, a confrontational attitude doesn’t always take long to convey.
What’s especially frustrating is that the authors obviously have a very sophisticated way of measuring how respectful someone is being. (Though I’m not sure a cop should get dinged for using the word “suspended” to convey to someone that his license is showing up as, er, suspended, as happens in one of the authors’ provided examples of “disrespectful” language. Or for “disfluency,” a fancy word that just refers to a stutter or break in speech.) They could have applied the same techniques to the drivers and controlled for drivers’ level of respect. Their failure to do that makes their results hard to interpret.
Update: In the first paragraph I initially transposed “white” and “black.” Apologies for the error, which has been corrected.