Experts criticize methodology of report that called it "unlikely" Spokane Police exhibit racial bias

This time, SPD reached out to a Washington-based company called Police Strategies, which was paid $10,000 for the study. 

"That fundamental assumption drives pretty much everything else they're doing. And it's a problematic assumption," Byrnes says.

For example, take a look at the report's claim that all races were "equally likely" to be stopped by police. That's not actually true if you compare police stops to the population of minority groups in Spokane; using the population as a baseline would show disproportionality. But the Police Strategies report doesn't use population as the baseline. Instead, it compares police stops to reported crime numbers. When you compare those, the disparity disappears.


"That's basically taking an old narrative of minority criminality and wrapping it in a different veneer," Byrnes tells the Inlander.



Bob Scales, CEO of Police Strategies, admits there is "no perfect benchmark" to analyze racial disparities. Scales is a former King County deputy prosecutor and special assistant U.S. Attorney. His company conducts these sorts of studies for police departments across the country.

Yet while Scales is willing to say their way of analyzing police data may not be the definitive way to do it, he does believe that using reported crimes is a better benchmark than using population.

He uses an example like this: Population-based benchmarks may find that police use force five times more often against minorities than against White people. But if you compare uses of force to the people who are actually arrested — what he calls an "activity-based" benchmark — then the disparity may be much smaller or nonexistent, since minorities may also make up more arrests, proportionally.

Similarly, if he used population-based benchmarks in this report, he says he "would conclude that the department and its officers have a systemic bias against Black and Native American subjects that they come into contact with." But there would also be other disparities by age and sex, then, that couldn't be ignored.

"We would also have to conclude that officers are biased against males and those between the ages of 18 and 49," Scales says.

In arriving at the overall conclusion that it's "unlikely" that Spokane officers exhibit racial bias, he points to data on police actions taken once officers do stop each racial group. These are what he calls "discretionary" actions. And when it's up to officer discretion to issue a ticket or let a driver off without warning, they do so less often than he would expect.

But the analysis doesn't consider the stop itself a discretionary action, only what happens after. If it did consider stops a discretionary action — as Byrnes would — it would find that police are more than twice as likely to stop Black and Native American people. So what the data actually indicates is that Spokane officers are still more likely to stop Black people than other racial groups, but less likely to issue an infraction once they do.

You could argue, then, that officers are racially profiling people if they're stopping them without a valid reason. Scales, however, tries to swat down that argument by saying Blacks and Native Americans are equally likely to be stopped by police if compared to reported crimes. But again,  that's only using his methodology comparing stops to reported crimes, even though it's police who are choosing to make those stops.


"I have not conducted or seen any study using activity-based benchmarks that suggests a likelihood that systemic biased practices are occurring against any particular demographic group," Scales says.

But what about the disparities the company did find in this report? It showed, for example, that use of force occurred more often against Black and Native American people. And it showed that there were high disparities in which racial groups were searched by officers. There, Scales says that it's too small of a sample size to say that the disparities were due to systemic practices.

James, the WSU researcher, tells the Inlander she hasn't read through the entirety of the Police Strategies report. But her own research seemingly came to the opposite conclusion as this report. She's found that implicit bias exists among White officers, but also that they were less likely to shoot unarmed Black suspects in a laboratory setting. Similar findings have been found elsewhere. A study conducted by a Harvard professor found that racial disparities exist in non-lethal police uses of force, but not in fatal shootings by police.

She rejects any suggestion that research looking at discretionary police behavior elsewhere hasn't found evidence of officer bias.

"If they believe there isn't evidence of disparate treatment of minority groups by police, they're sorely mistaken," James says. "There's a great deal of evidence for that."

In 2019, the Inlander asked Spokane Police Chief Craig Meidl about racial disparities in policing. At the time, Meidl suggested that at least part of the reason minorities are overrepresented in policing data is because some demographics simply commit crimes at higher rates.

"When you're looking at violent crime in the city of Spokane, over 30 percent of our homicides in the last six years have been committed by African Americans. They make up 2.5 percent of the population," Meidl said at the time. "But 17 percent, over the last two years, of my aggravated assaults, as reported by the community, have been committed by African Americans. Over 10 percent of my rapes, 10 percent of my robberies."

The Police Strategies report, while not coming to definitive conclusions, would seem to support this point of view that disparities by population are caused by minorities committing crimes at higher rates.

Spokane City Council President Breean Beggs also takes issue with the methodology used. He says he's heard Meidl say that the reason they arrest more Black people is that Black people commit more crimes, an assertion that Beggs says is refuted by other studies. In Police Strategies, he says SPD found "a methodology to advance that narrative."

"My concern with the whole process is they should have had that conversation about what methodology with the community before they started," Beggs says.

But Byrnes calls that an "old dog whistle" narrative. Byrnes completed two previous reports for SPD — one in 2015 and one in 2017. He says he wasn't paid for it and worked together with SPD Capt. Brad Arleth to put them together.

Unlike the Police Strategies report, those reports considered officer-initiated contact as a discretionary action. While not measuring "bias" necessarily, Byrnes uncovered that disproportionality was "absolutely in place" when compared to population, he says.

"Even when controlling for some other factors, there was no getting way from it. There was no getting around it," he says.

Using a population-based benchmark is the only thing that makes sense in looking for disparities in police stops, he says, because anybody in the population can be subject to an officer-initiated contact. Comparing officer-initiated contacts to reported crimes, he says, would only make sense if you were only studying how police responded to calls for service.

It's a huge leap, Byrnes says, to apply it beyond that. Yet that assumption drives much of the Police Strategies report.

Byrnes worries it could be an effort to change the conversation on policing "away from social justice."

"We would all love to live in a community where there was no disproportionality," Byrnes says. "But when folks start saying, 'We looked at it this way and got this other result that people like better,' well, the result doesn't justify the methodology. I would just caution anybody to beware of being enamored with the result and therefore justifying the methodology for it."