Kroll Report Phase B from KROLL Consulting - 3-7-2022 meeting — original pdf

Backup

Evaluation of Austin Police Department: Use of Force / Public Interactions / Recruitment, Selection, and Promotions Prepared for City of Austin, Office of Police Oversight / City Manager’s Office January 21, 2022 Status Final Report Kroll Associates, Inc. 2000 Market Street, Suite 2700 Philadelphia, PA 19103 Kroll.com  Contents 1. EXECUTIVE SUMMARY ............................................................................................................... 1 2. BACKGROUND AND CONTEXT ................................................................................................ 11 3. REVIEW AND DATA ANALYSIS OF APD USE OF FORCE (January 2017 to December 2020) ........................................................................................ 16 3.1 APD Use of Force - Overview ........................................................................................... 18 APD Use of Force Policies ....................................................................................... 18 Measuring APD Use of Force ................................................................................... 21 APD Use of Force Descriptives ................................................................................ 23 Individuals’ Resistance ............................................................................................. 27 Individuals’ Impairment ............................................................................................. 30 Types of Force -- Severity ........................................................................................ 33 Individuals with Repeat Uses of Force ..................................................................... 34 Post-Use of Force ..................................................................................................... 36 Geographic Analyses ............................................................................................... 37 3.2 Racial/Ethnic Disparity Analyses ...................................................................................... 41 Disparity Ratio Findings............................................................................................ 45 Summary: Disproportionality Use of Force Benchmark Analysis by Sector ............ 53 Conclusion ................................................................................................................ 53 3.3 Predicting Use of Force .................................................................................................... 53 APD Uses of Force 2017-2020 ................................................................................ 54 APD Arrests with Use of Force 2017-2020 .............................................................. 55 Factors Influencing the Use of Force ....................................................................... 59 Limitations ................................................................................................................ 63 Descriptive Analyses ................................................................................................ 64 Multivariate Analyses ................................................................................................ 66 Summary .................................................................................................................. 71 4. REVIEW AND ANALYSIS OF APD USE OF FORCE (June to November 2019) .................... 73 4.1 Incident and Demographic Comparisons ......................................................................... 73 4.2 Lack of Reasonable Suspicion and APD Use of Force .................................................... 75 4.3 Additional Trends and Issues within the Problematic Cases ............................................ 77 4.4 Individual Examples of Problematic Use of Force Cases ................................................. 80 4.5 Conclusion ........................................................................................................................ 85  5. ANALYSIS OF APD TRAFFIC STOPS, CITATIONS, ARRESTS, AND SEARCHES (January – December 2020) .................................................................................................. 86 5.1 Background ....................................................................................................................... 86 5.2 Data Description and Limitations ...................................................................................... 86 Data Limitations ........................................................................................................ 88 5.3 Motor Vehicle Stops January 1 – December 31, 2020 ..................................................... 89 5.4 All Arrests January 1, 2017 – December 31, 2020 ......................................................... 105 5.5 Summary ......................................................................................................................... 114 6. RECRUITMENT, SELECTION, AND PROMOTIONS ............................................................... 117 6.1 Methodology ................................................................................................................... 117 6.2 APD Demographics and Diversity .................................................................................. 118 6.3 Recruitment ..................................................................................................................... 121 6.4 The APD Selection Process ........................................................................................... 133 6.5 The APD Promotion Process .......................................................................................... 142 6.6 Conclusion ...................................................................................................................... 150 7. RECOMMENDATIONS.............................................................................................................. 152 7.1 Data Collection Recommendations ................................................................................ 152 7.2 Use of Force Recommendations .................................................................................... 156 7.3 Organizational Recommendations.................................................................................. 158 7.4 Recruitment, Selection, and Promotion Recommendations ........................................... 161 7.5 Appendix to Section 7: Data Fields for Traffic Stop Data ............................................... 165 8. APPENDIX A ............................................................................................................................. 168 8.1 Kroll Project and Research Team ................................................................................... 168 9. APPENDIX B ............................................................................................................................. 170 9.1 References ...................................................................................................................... 170  1. EXECUTIVE SUMMARY On November 12, 2020, the City of Austin’s Office of Police Oversight (OPO), in consultation with the City Manager‘s Office (CMO), retained Kroll to review and evaluate the extent to which forms of racism, bigotry, and discrimination are present in the protocols, practices, and behaviors of the APD.1 The initial phase of Kroll’s review (“Phase A”), which was completed on April 23, 2021, involved assessing the APD Training Academy (Academy) on its ability and readiness to prepare cadets for policing in a multi-ethnic, diverse urban population consistent with best practices. (See Kroll Report Austin Police Department: Review and Assessment of Training Academy, April 23, 2021.)2 This second phase of Kroll’s evaluation (“Phase B”) addresses four distinct areas: (1) APD use-of-force incidents from January 1, 2017 to December 31, 2020 (48 months), including the role that race, ethnicity, gender, or similar person characteristics potentially impact APD’s application of force; (2) a qualitative review and analysis of approximately 1,321 APD use of force incidents from June to November 2019; (3) APD’s public interactions with civilians (e.g., traffic stops, arrests, citations, and searches) from January to December 2020 (12 months) and potential disparities involving race, ethnicity, gender, or other similar personal characteristics;3 and (4) APDs’ recruitment, selection, and promotion policies and practices and their potential impact on historically underrepresented groups such as women and people of color. Based on Kroll’s review and analysis over the past fourteen months, we provide a summary of our findings below, which are detailed more completely in the report that follows: Section 3: Review and Data Analysis of APD Use of Force (2017-2020) It is critical that readers of this report and APD are aware of the limitations of what aggregate police data can and cannot tell us; individual officer bias or lack thereof cannot be determined through statistical analyses. Aggregate police data alone cannot “prove” or “disprove” racially biased policing, and it should never be viewed—either by police or resident stakeholders—as a pass/fail test. Rather, as Fridell (2004) has explained, analyzing police data is “a diagnostic tool that can help pinpoint the decisions, geographic areas, and procedures that should get priority attention when the agency, in concert with concerned residents, identifies its next steps for addressing the problem or perception of racial profiling.” With that limitation in mind, we highlight below some of the key findings of Kroll’s four-year data analysis of APD use of force incidents. • Trends: Comparing 2020 to 2017, APD arrests declined by 51%, while use of force incidents increased by 58%. (This may be partially explained by a change in APD use of force reporting levels in 2019, which added some low-level use of force incidents that were not previously captured in APD reporting.) 1 Kroll’s retention is pursuant to contract MA 4400 PA210000018. 2 Kroll was subsequently appointed by the City to serve as Independent Evaluator of the Academy to ensure that APD effectively implements the short-term and long-term recommendations in Kroll’s Report that were approved by the City Manager and agreed to by APD leadership. Kroll submitted an Interim Report in its role as Independent Evaluator on October 14, 2021 (see Interim Report of Independent Evaluator: APD Training Academy, October 14, 2021). Kroll’s final report as Independent Evaluator will be issued in February 2022. 3 Due to APD data collection limitations, Kroll was unable to examine APD’s public interactions with civilians (e.g., stops, arrests, citations) from June to November 2019 for potential disparities involving race, ethnicity, gender, or other similar personal characteristics, as initially requested in the scope of work; however, as noted, Kroll conducted limited analyses from 2020 data involving traffic stops, arrests, citations and searches. 1  • Severity of Force: Although severity of force is not properly aggregated in APD data (see data collection recommendations in Section 7), what APD does measure indicates that the highest level of force used in an encounter is not severe and does not result in serious injuries. Thus, 44.7% of force incidents are Level 4 cases, the least severe of APD’s four-level classification, while 49.3% are Level 3 cases, the second least severe level of force. Moreover, force severity does not significantly vary across racial/ethnic groups. • Levels of Resistance: The levels of resistance individuals displayed toward officers during use of force events were consistent across racial/ethnic groups and stable across years, except for 2020. This is because 2020 had more unknown individuals who experienced force (likely in crowd control situations) and the unknown individuals in 2020 were twice as likely to display aggressive behavior or resistance compared to known individuals. • Perceived Impairment: The data shows that use of force on individuals who were perceived to be impaired—under the influence of drugs/alcohol or with mental health issues—occurred more often than use of force on individuals who were not impaired. o Black individuals who experienced use of force were less likely than others to have been impaired, but this comparison does not consider other factors that contribute to force. • Repeat Uses of Force: 30% of individuals who had force used against them were involved in more than one use of force event between 2017 and 2020. o Black individuals who experienced use of force were more likely to have experienced multiple use of force encounters. o Individuals who were perceived to be impaired were more likely than those with no noted impairment to have experienced multiple separate use of force encounters. • APD Sectors: The frequency of police use of force varies dramatically across APD sectors. o George Sector is the smallest in geographic size and least populated, yet the sector accounted for 23% of individuals who experienced force (nearly twice the percentage of any other sector) and a rate of use of force per 10,000 residents that is 21 times higher than the next highest sector. City-Wide Racial/Ethnic Disproportionality and Disparity Analyses of Use of Force Rates: Kroll examined five benchmarks (i.e., comparison groups) to interpret use of force rates by race/ethnicity: 1) residential population, 2) all arrests, 3) Part I violent offense arrests, 4) all criminal suspects (based on reported crimes), and 5) Part I violent offense suspects (based on reported crimes). • Population-based benchmarks use residential population data, such as census data, as the basis of the racial/ethnic composition of police interactions. As explained more fully in Section 3, however, population-based benchmarks do not accurately measure the complexity of the interactive nature of contacts with the police or the racial/ethnic differences in contacts with the police. As such, rates of disparity calculated using population data may exaggerate or understate disparity. • Other benchmarks use the racial/ethnic composition of arrestees and/or suspects as estimates of the racial composition of individuals “at-risk”4 of having force used against them. These benchmarks address some of the limitations inherent in population-based benchmarks. Both types 4 “At risk” in this context implies individuals engaged in situations involving police where uses of force might occur. No indication of risk related to any other factors or characteristics of individuals is implied. 2  of benchmarks are presented herein, but population-based benchmarks presented should be used only as a comparison to the arrest and suspect benchmarks. o Disparity Ratios: • Residential population benchmarks indicate significant racial/ethnic disparities in use of force for Blacks and Hispanics. Using Census data, Blacks were 6.7 times and Hispanics 1.5 times more likely than whites to experience force compared to their representation in the residential population. • Arrest and suspect benchmarks indicate less or no racial/ethnic disparity. • Using arrest data to establish the racial/ethnic composition of individuals “at-risk” for use of force—Black individuals were 1.2 times more likely and Hispanic individuals less likely to experience force compared to white individuals (based on the racial/ethnic population of arrestees). • Using Part 1 violence arrest data to establish the racial/ethnic composition of individuals “at- risk” for use of force—Black individuals and Hispanic individuals were less likely than white individuals to experience force (based on the racial/ethnic composition of Part 1 arrestees). • Using all suspect data to establish the racial/ethnic composition of individuals “at-risk” for use of force—Black individuals were equally likely and Hispanic individuals less likely than white individuals to have force used against them (based on the racial/ethnic composition of reported criminal suspects). • Using Part 1 violence suspect data to establish the racial/ethnic composition of individuals “at-risk” for use of force—Black individuals were less likely and Hispanic individuals were equally likely compared to white individuals to have force used against them (based on the racial/ethnic composition of criminal suspects for serious crimes). Sector Level Racial/Ethnic Disproportionality and Disparity Analyses of Use of Force Rates: An examination of racial/ethnic disparity in use of force across APD sectors also reveals mixed findings that are heavily dependent on the benchmark employed and the specific APD sector under examination. Nevertheless, the following consistent patterns emerge: • Disparity ratios created using residential census population as the benchmark consistently demonstrated major disparities in use of force for Black individuals (across all APD sectors) and for Hispanic individuals (in half of the APD sectors) compared to white individuals. • Disparity ratios created using arrestee population as the benchmark consistently demonstrated low or no disparities in use of force for Black and Hispanic individuals compared to white individuals. However, two APD sectors – George and Ida – showed that Hispanic individuals are 1.5 times more likely to have force used against them compared to white individuals when the arrestee population is used as the reference group. • Disparity ratios created using criminal suspects as the benchmark consistently demonstrated low or no disparities in use of force for Black and Hispanic individuals compared to white individuals across all but two APD sectors. In Edward sector, Hispanic individuals are 1.2 times more likely to have force used against them relative to white individuals. In George sector, Black and Hispanic individuals are 1.2 and 1.6 times more likely, respectively, to have force used against them compared to white individuals (relative to their representation as criminal suspects). 3  • Use of force disparity for Hispanic individuals has slightly more variability by benchmark type and police sector. Combined, the analyses suggest that the disparity ratios for Hispanics in George and Ida sectors (and to a lesser extent Edward) are higher than in other sectors. • Multivariate statistical analysis: Kroll examined 126,096 arrests from 2017-2020 to determine predictors of force during encounters with the public that result in arrest. The findings noted below for these analyses represent the independent effect of each variable on the likelihood of use of force, while holding constant all other measured variables: o The strongest predictors of force within arrests are legal and incident characteristics. Arrests were more likely to involve force when they were custodial (versus cite and release), occurred on a weekend (versus a weekday), or included weapons seizures from arrestees (versus no weapon). Overall, there existed a small amount of racial disparity in whether force was used in arrest situations. ▪ Black individuals were slightly more likely than whites to be involved in arrests that involved use of force. There was a small effect size for this difference in comparison to legal and incident characteristics, and the difference became even smaller once we accounted for community violent crime rates. ▪ Hispanic individuals were slightly less likely than whites to be involved in arrests that resulted in force, but this small effect was not present once community context was considered. o Arrests within areas with higher violent crime rates had a greater likelihood of use of force. o Race/Violent Crime Rate Interaction: Black individuals arrested in communities with low violent crime rates were slightly more likely to have force used against them when compared to white arrestees. In communities with higher violent crime rates, there were no racial/ethnic differences in the likelihood of use of force. o Previous studies have consistently shown that the strongest predictors statistically of whether officers use force include: (1) individuals’ resistance toward officers and (2) the severity of the arrest offense; however, resistance is not measured in APD’s arrest data and offense severity is not reliably measured or reported in APD’s data. Therefore, the two strongest predictors of the likelihood of use of force could not be included in our statistical models. Section 4: Qualitative Use of Force Analysis – June to November 2019 (6 Months) Kroll also reviewed and evaluated 1,321 incidents involving 2,960 uses of force by APD officers from June 1, 2019 to November 30, 2019. Kroll reviewed the incident reports and accompanying videos (body camera and dashboard) to determine whether APD’s use of force incidents over the six-month time frame were appropriate and/or reported accurately. Our findings are detailed in Section 4 and include the following: • Although the vast majority of use of force incidents were independently confirmed by Kroll to have been appropriate in the circumstances, Kroll identified 112 use of force incidents, approximately 8.5% of the cases reviewed, that contained issues of concern. In 82 of the cases reviewed, Kroll determined that the use of force was either inappropriate or caused by an unnecessary escalation of the encounter by APD officers. In the other 30 cases, Kroll found additional issues of concern. Importantly, in all of the cases highlighted, APD supervisors were notified of the use of force and approved any resulting arrests. • A total of 88 individuals had force applied against them in the 82 cases in which Kroll found the use of force to have been unnecessarily escalated or inappropriate. 4  Kroll’s review of the 8.5% of cases that were identified as problematic noted several areas of concern that demonstrated some troubling trends. We summarize these issues below: • Stop and Frisk without Reasonable Suspicion: Stop and Frisk is employed aggressively in Austin. In the incidents reviewed, most officers officially justify stop and frisks by claiming “my/officer safety” without articulating reasonable suspicion that criminal activity was afoot. However, officer safety alone does not justify a frisk and physical detention. In many of the problematic cases reviewed by Kroll, APD officers told citizens they were not under arrest and had done nothing wrong, while placing the subjects in handcuffs. In many instances, when subjects asked why or displayed uncertainty, resisting arrest/search/detention, in many cases unnecessarily or inappropriately (with no charge that led to the encounter). frequently charged with they were • Unnecessary Escalation: APD officers frequently “escalated” citizen encounters by not informing individuals why they were being stopped. In most of the problematic cases, officers did not inform the individual of the reason they were being detained. Officers routinely and aggressively placed individuals in handcuffs, which sometimes caused the individuals to resist, brace, or instinctively pull away because they did not comprehend why they needed to be restrained. • APD officers should instead de-escalate these situations by informing the individual of the reason that handcuffs are being used. Kroll found that resisting arrest is over-charged to “justify” officers’ actions. • APD officers frequently commanded individuals to get “on the ground” and used leg sweeps at the slightest resistance. The leg sweeps typically caused the individual to fall to the ground. For example, in one incident Kroll reviewed, four male officers swept the legs out from under a non- aggressive intoxicated female subject, causing facial injuries to the female. Moreover, when individuals tried to resist by hiding their arms and hands under their body, the encounters often escalated with officers using punches or other strikes (to the back or side of the head) to enforce compliance. • Resisting Detention/Search Charges: On many occasions in the problematic cases reviewed, APD officers did not have a valid criminal charge when they detained someone who failed to immediately comply, usually by asking “Why I am being stopped?” or by not immediately putting their hands behind their back. Within seconds, these individuals were typically arrested for Resisting Detention or Search and transported to jail without any other charge(s). • Supervisory Issues: Kroll reviewed dozens of cases where the force used was inappropriate due to an officer’s actively targeting with a firearm, failure to follow APD Taser policy, or preparing written reports that conflict with the body camera video. These issues need to be better addressed by supervisors reviewing and analyzing APD officer’s use of force and within the chain of command review process. o The APD use of force process appears to lack proper internal supervisory review and investigation, and thus frequently fails to address the appropriateness of the use of force used against members of the public. In all of the cases in which Kroll determined use of force was not appropriate, or that the situation was unnecessarily escalated by APD officers, APD supervisors had been notified of the use of force and, in all but one case, approved the resulting arrests. • Taser Usage: The Taser seems to be the intermediate weapon of choice for APD officers. Kroll noted several violations of policy that went unaddressed by supervisors, including the tasing of 5  individuals who were otherwise complying with officers, tasing individuals who were not involved in any criminal activity, and failing to provide Taser warnings, all of which went unaddressed by APD supervisors. • Pointing of Firearms: Kroll noted that the pointing of a firearm at an individual, or actively targeting with a firearm, are not considered a “use of force” incident under APD policy. Kroll noted many cases where the pointing of a firearm occurred apart from the use of force reported, and specifically identified nine pointing of firearms or actively targeting cases that were problematic. Because APD does not consistently capture the pointing of a firearm in its reporting structure, it is unknown how frequently this occurs. • Neck Restraints/Chokeholds: Kroll identified five cases in which neck restraints or chokeholds were used. In each of these cases, the use of lethal force would not have been authorized, and these holds can have lethal consequences. • Head Strikes: Kroll identified two cases where head strikes were used, including one with an APD issued Taser (used as an impact weapon). In these cases, the use of lethal force would not have been authorized. The use of head strikes can have lethal consequences. • Mental Health Related: Kroll identified 21 use of force cases of concern that involved a mental health component. Kroll determined that the use of force was inappropriate in 19 of these incidents. • Body Worn Cameras (BWC): Kroll identified 15 incidents in which APD officers failed to activate their body worn cameras or official reports conflicted with video footage. These policy failures and conflicts were not addressed by APD supervisors in the report review process. Additional areas of concern are further addressed in Section 4. Kroll’s recommendations pertaining to use of force are outlined in Section 7. Section 5: Analysis of Traffic Stops, Arrests, Citations, and Searches 2020 (Total = 68,330) Section 5 examined motor vehicle stops that occurred from January 1 to December 31, 2020, the outcomes of these stops (e.g., warnings, citations, arrests), and whether searches occurred during these encounters, as well as all arrests and searches within arrests, from 2017 to 2020. Although the APD motor vehicle stop and arrest data suffer from a number of limitations, below is a summary of the analyses conducted: • White (44.8%) and Hispanic (35.0%) individuals represent the largest percent of those stopped during motor vehicle stops; 14.9% of motor vehicle stops involved Black individuals, 4.1% were of Asian individuals, and the remaining 1.2% of stops were of other races. • Kroll cannot provide disproportionality analyses comparing the percentages of stops by race/ethnicity to an expected rate of stops (i.e., benchmark) because these types of comparisons are only appropriate for discretionary stops and APD’s stop outcome data does not distinguish officer-initiated from dispatched stops. Further, there are no readily available benchmark data to use, beyond the flawed use of residential population. • The majority of traffic stops were conducted based on moving traffic violations. • The most serious outcomes recorded for all traffic stops were verbal warning (11.2%), written warning (51.7%), citation (31.7%), and arrest (5.4%). • Although there were statistically significant racial and ethnic differences in reason for the stop and stop outcomes, bivariate analyses do not consider any other variables and there may be legally relevant behavior or other factors that explain these differences. 6  • Multivariate analyses of stop outcomes were also conducted to account for multiple factors’ impact on an outcome to understand the independent impact of each variable on outcome. o The most salient and powerful predictors of stop outcomes were legal and incident characteristics, including reason for the stop, whether contraband was seized, and the location of the stop. o Statistically significant racial/ethnic differences in APD stop outcomes remained, even controlling for legal and incident characteristics. ▪ Warnings: Black and Hispanic individuals, and individuals of other race/ethnicity, were significantly less likely to be issued a warning compared to whites. ▪ Citations: Hispanic individuals and those of other race/ethnicity were 1.5 and 1.3 times more likely to be issued citations than white individuals. Black individuals were no more or less likely to receive a citation compared to whites. ▪ Arrests: Black and Hispanic individuals were 1.7 and 1.5 times more likely to be arrested compared to white individuals, respectively, while individuals of other race/ethnicity were 2.4 times less likely to be arrested compared to whites. Searches during 2020 Traffic Stops • Searches were conducted in 7.6% (5,224) of the 68,330 traffic stops. • Most searches (61.6%) were for mandatory reasons (i.e., required by policy); 37% of searches were based on probable cause. o Black individuals and, to a lesser degree, Hispanic individuals were more likely than whites and individuals of other races/ethnicities to be searched based on probable cause. Whites were more likely than all other racial/ethnic groups to be searched incidental to arrest. • 23.9% of all searches resulted in seizures of contraband: 28.1% of discretionary searches and 21.3% of mandatory searches. o Black and Hispanic individuals were more likely than whites and individuals of other races/ethnicities to have contraband seized during both discretionary and mandatory searches. • A multivariate model predicting searches showed that legal reason for the stop was the strongest predictor, but even controlling for legal and situational characteristics, Black and Hispanic individuals were 2.1 and 1.8 times more likely than white individuals to be searched. These searches, however, were more likely to result in contraband seizures compared to searches of whites. Arrests 2017-2020 • From 2017 to 2020, there were 128,213 total arrests analyzed by the Kroll team. Arrests steadily declined by 51% 2017-2020; the decrease is fairly consistent across racial/ethnic groups. • 74.3% of individuals arrested were taken into physical custody and transported to jail. o Black individuals (72.5%) were slightly less likely than Hispanic (75.2%) and white (74.9%) individuals to be taken into custody when arrested. 7  • Arrest rates differed across APD sectors, but George Sector was a clear outlier. Although it is the smallest and least populated APD sector, it has an arrest rate that is five times the rate of the next closest sector. o Although the crime rate is relatively high in George Sector, the arrest rate far exceeds what might be expected, even based on this elevated level of crime, and by the trends between crime and arrests evident in other sectors. Comprehensive recommendations for improving the collection of motor vehicle stop and arrest data are included in Section 7. Section 6: Recruitment, Selection, and Promotions In Section 6, Kroll reviewed APD’s recruitment, selection and promotion policies and processes to determine how they potentially impact historically underrepresented groups such as women and people of color. Our findings are summarized below. Demographics and Diversity As of March 2021, APD personnel were approximately 66.7% white, 21.8% Hispanic, 7.5% Black, and 2.5% Asian/Pacific Islander. Based on recent residential census data, this suggests that whites are overrepresented and Hispanics and Asian/Pacific Islanders are underrepresented compared to the racial and ethnic makeup of Austin’s population, which is 48.3% white, 33.9% Hispanic, 7.8% Black or African American, and 7.7% Asian/Pacific Islander. Although women account for about half the Austin population, they account for 10.4% of all APD officers, which is slightly under the national average. Moreover, Black and Hispanic officers are underrepresented at some of the upper ranks. Recruitment Although APD’s recruiting efforts are comprehensive, creative, and reach a large group of diverse candidates, improvements are needed in its data collection and documentation efforts to help the department more consistently evaluate the effectiveness of its various recruiting strategies. Nevertheless, the department utilizes several industry-standard recruitment practices, and APD recruiters attend job and career fairs, host public information sessions, and set-up information tables at a variety of community events, with a focus on military-related sites, educational institutions, and other events that often are focused on historically underrepresented groups. From January 2016 to March 2020, a total of 13,351 potential applicants were recruited across all types of recruiting events; of these, only 1,298 (9.7%) applied. Hispanic and white recruits each represented approximately one-third of potential applicants, followed by Black recruits at 26%. In total, nearly two-thirds of those who expressed interest in applying to APD at a recruiting event were people of color. Of the 1,298 recruits who applied after attending a recruiting event, the percentages of white and Hispanic recruits were approximately 3-5% higher than in the overall recruitment pool, while the percent of Black recruits dropped by 7.4%. Approximately two-thirds of potential applicants were male, and one-third were female. Recruitment from colleges and universities was the most likely recruitment location across all racial/ethnic groups; however, Blacks were more likely than all other racial/ethnic groups to be recruited at colleges and universities, but the least likely to be recruited at job fairs. Kroll’s analysis found that, although APD is reaching a large and diverse audience when it recruits, there is variation across the types of events in producing applicants from the pool of recruits. Nearly half of all applicants were recruited at general career/job fairs or information sessions despite those types of events producing just 22% of the total number of recruits. Furthermore, irrespective of gender and race/ethnicity, 8  recruits who applied were most likely to do so if they had been recruited at general career and job fairs or information sessions. Kroll found that only ten percent of those who expressed interest in applying to APD at a recruiting event followed through and applied, although the Recruiting Unit has had difficulty accurately matching the information gathered from prospective applicants at recruiting events with the online applications that are later completed. Kroll also found that the recruitment pool demonstrates greater racial/ethnic and gender diversity than the current APD workforce. Hiring and Selection The APD hiring process, in terms of the selection components and their sequencing, is consistent with standard police department hiring practices in the United States, and APD frequently modifies its selection practices to increase retention of qualified and diverse applicants. Kroll analyzed race/ethnicity, age, and gender differences in the APD selection process to determine if any group was adversely impacted by APD’s selection procedures. Kroll’s analysis focused on the 6,601 total applicants for cadet classes 130-143, which included 5,890 applicants who were disqualified at some point during the process, and 711 applicants who ultimately became cadets at the Academy. Unfortunately, the structure of the disqualification data limited our ability to fully analyze attrition during the selection process. Kroll found that the current written (i.e., cognitive ability) test for applicants continued to show racial and ethnic disparities in scores, although the disparities were smaller than with the previous written test. Specifically, although the current test was marginally more difficult to pass, the differences between racial/ethnic groups who took the exam became smaller, with Black and Hispanic applicants performing better on the current test. Regardless of which written test was taken, white applicants were statistically less likely than applicants of other races and ethnicities to be disqualified from the selection process due to their written test results. In contrast, Black and Hispanic applicants were significantly more likely than applicants of other races and ethnicities to be disqualified due to the written test. Kroll found no significant racial/ethnic differences in physical ability test (PAT) failures. Although there previously were significant gender differences in PAT failures (i.e., women were more likely to fail the PAT), once the 2,000-meter rowing test was removed (and physical testing re-focused on 1.5-mile runs with a shortened 500-meter rowing test), the within-gender rate of male physical failures was slightly higher than female physical failures. Kroll also reviewed the background history statement and other disqualifiers for statistical differences between applicants with different demographic characteristics. Black applicants were disqualified statistically more often as compared to other population subgroups due to outstanding debt and credit history requirements. White applicants, when compared to other racial/ethnic groups, were statistically less likely to be disqualified due to their credit history or a problem with their driver’s license (e.g., suspension). On the other hand, white applicants were statistically more likely and Black applicants statistically less likely to be disqualified than others due to the polygraph, medical, or psychological exams, which occur near the end of the selection process. In addition, white applicants were statistically more likely to be disqualified due to drug usage, compared to Black and Hispanic applicants. Ultimately, after disqualifications and other drop-off factors, nearly two-thirds of candidates hired as APD officers are white (66.0%) compared to other racial/ethnic groups (Hispanic - 21.5%, Black - 7.7%, Asian/Pacific Islander - 4.5%). Given that the recruitment process is having success in creating a diverse applicant pool, this finding raises additional questions. 9  The Promotional Process Although APD has a comprehensive promotions process, Kroll’s statistical analyses of APD promotions from 2015 to 2020 indicate that, while there are no significant gender differences in promotion outcomes, Asian and white promotion candidates are significantly more likely to be promoted than Black and Hispanic candidates. In particular, the written promotional test scores across all ranks indicate that the promotional written test may have an adverse impact on candidates of color and between younger and older candidates. Kroll found that, at least in recent years, seniority bonus points have played a critical role in helping to close the promotional score gaps for Black and Hispanic officers, as it appears that Black and Hispanic candidates stay in grade longer than white candidates before promotion. Kroll also examined the racial/ethnic differences in assessment center scores for sergeants and found that Asian and white candidates have significantly higher average raw scores on the assessment centers than Black and Hispanic candidates. Thus, at least for the rank of sergeant, the assessment center scores have a statistically significant disparate impact on Black and Hispanic promotion candidates. Finally, promotion eligibility lists are constructed by rank, ordering candidates based on the highest total score for all promotion components. Among candidates who sought promotion, the percentage of those promoted within each racial/ethnic group was as follows: Asian/Pacific Islanders - 72.7%, white - 60.1%, Black - 56.4%, and Hispanic - 54.3%. All of these areas are examined in more detail in Section 6. In Section 7, Kroll provides a series of detailed recommendations to enhance APD’s recruitment, selection, and promotion processes in order to continue and sustain diversity within the department and throughout its ranks, and to further improve its overall recruitment, selection, and promotion efforts. 10  2. BACKGROUND AND CONTEXT On November 12, 2020, the City of Austin’s Office of Police Oversight (OPO), in consultation with the City Manager‘s Office (CMO), retained Kroll to review and evaluate the extent to which forms of racism, bigotry, and discrimination are present in the protocols, practices, and behaviors of the APD.5 The initial phase of Kroll’s review (“Phase A”), which was completed on April 23, 2021, involved assessing the APD Training Academy (Academy) on its ability and readiness to prepare cadets for policing in a multi-ethnic, diverse urban population consistent with best practices. See Kroll Report Austin Police Department: Review and Assessment of Training Academy, April 23, 2021.6 This second phase of Kroll’s evaluation of APD (“Phase B”) addresses four distinct areas: (1) APD use-of- force incidents from January 1, 2017 to December 31, 2020 (48 months), including the role that race, ethnicity, gender, or similar person characteristics potentially impact APD’s application of force; (2) APD’s public interactions with civilians (e.g., traffic stops, arrests, citations, and searches) from January to December 2020 (12 months) and potential disparities involving race, ethnicity, gender, or other similar personal characteristics;7 (3) a qualitative review and analysis of approximately 1,321 APD use of force incidents from June to November 2019; and (4) APDs’ recruitment, selection, and promotion policies and practices and their potential impact on historically underrepresented groups such as women and people of color This report is a continuation of the City of Austin’s efforts to reimagine policing following a series of national and local events that shined a spotlight on police-community relations throughout the United States. As with many other U.S. cities consciously re-examining policing following the tragic killing of George Floyd in the summer of 2020 and similar tragedies throughout the country that have resulted in protests and civil unrest, a series of alleged incidents and reports concerning the Austin Police Department over the past few years prompted the City, through Council resolutions and other calls for action, to more closely examine APD’s operations. As we noted in our April 2021 report assessing the Academy, additional background and context that colors Kroll’s work for this report include the following previously reported events, statistics, and reports: Reports of Racial Profiling and Bias. Several internal and external reports over the past decade have highlighted alleged racial disparities across various facets of APD’s law enforcement operations that have disproportionately impacted communities of color. For example, according to state-mandated racial profiling reports from 2010 to 2018,8 APD officers were more than twice as likely to search Black and Latino drivers than white drivers during traffic stops despite no greater likelihood of discovering contraband. 5 Kroll’s retention is pursuant to contract MA 4400 PA210000018. 6 Kroll was subsequently appointed by the City to serve as Independent Evaluator of the Academy to ensure that APD effectively implements the short-term and long-term recommendations in Kroll’s Report that were approved by the City Manager and agreed to by APD leadership. Kroll submitted an Interim Report in its role as Independent Evaluator on October 14, 2021 (see Interim Report of Independent Evaluator: APD Training Academy, October 14, 2021). Kroll’s final report as Independent Evaluator will be issued in February 2022. 7 Due to APD data collection limitations, Kroll was unable to examine APD’s public interactions with civilians (e.g., stops, arrests, citations) from June to November 2019 for potential disparities involving race, ethnicity, gender, or other similar personal characteristics, as initially requested in the scope of work; however, as noted, Kroll conducted limited analyses from 2020 data involving traffic stops, arrests, citations and searches. 8 https://www.austintexas.gov/page/racial-profiling-reports 11  The Center for Policing Equity (CPE) reported in 2016 that Black and Hispanic motorists made up a significantly higher share of discretionary stops than white motorists.9 According to APD discretionary arrest data from 2017 published on the City of Austin’s website, APD officers allegedly made discretionary arrests of Black citizens at more than twice the rate of white or Latino residents; Blacks and Latinos were disproportionately arrested for driving with an invalid license compared to other races and ethnic groups, and Black residents were more than seven times as likely to be arrested for low-level marijuana offenses than white residents.10 Moreover, in 2019, the department reportedly received eight formal complaints and 40 informal complaints of racial profiling, more than three times as many as in 2018.11 Alleged Racial Insensitivity by APD Officers. On October 30, 2019, the City received an anonymous complaint against former APD Assistant Chief Justin Newsom which alleged that, on several occasions over the past decade, Newsom used explicitly racist language in reference to former President Barack Obama, former City Council Member Ora Houston, and former APD Assistant Chief Frank Dixon, among other persons of color. This complaint also alleged that then APD Chief Brian Manley and other high-level police officials knew of Newsom's use of racially derogatory language but did not act on the information.12 While an independent investigation into these claims could not corroborate the specific alleged incidents ascribed to Newsom, it reported that “racist and sexist name-calling and use of derogatory terms associated with race and sex persists” within the department and “there is a very high level of fear of retaliation” for speaking out against officers, as well as “quiet resistance from those interviewed in the form of evasiveness, misdirection and deflection.”13 Moreover, in November 2019, a complaint received by the Office of Police Oversight alleged that a female Assistant Chief made disparaging remarks towards other officers, including one statement in which, referring to a Hispanic detective, the Assistant Chief allegedly said, “Maybe now more of your people [Hispanics] can get a job since college is no longer a requirement.”14 The complaint also alleged that officers rarely challenge their superiors on their misconduct “for fear of retaliation.”15 Demographic Analysis. An Austin American-Statesman review of APD demographic data in 2016 found that, on average since 2005, approximately 69% of APD officers have been white, 20% Hispanic, 9% Black and 2% Asian/Pacific Islander.16 This compared to 2015 U.S. census data, which showed that Austin’s 9 Phillip A. Goff et al., “The Science of Policing Equity: Measuring Fairness in the Austin Police Department” (New York, NY: Center for Policing Equity; Washington, D.C.: Urban Institute, 2016), https://policingequity.org/images/pdfs- doc/Austin_PDI_Report_2016_Release.pdf. 10 https://www.austintexas.gov/edims/document.cfm?id=300349 11 Bryce Newberry, “APD Racial Profiling Complaints More than Tripled from 2018 to 2019, Report Says,” KVUE, February 28, 2020, https://www.kvue.com/article/news/local/apd-racial-profiling-complaints-more-than-tripled-from- 2018-to-2019-report-says/269-eb531b4f-1970-4032-bdb3-2989772a798f. 12 Russell Falcon, “Anonymous Complaint to City of Austin Alleges Former APD Assistant Chief Used 'n Word' to Refer to African-Americans,” KXAN Austin, November 7, 2019, https://www.kxan.com/news/local/austin/austin-city- manager-approves-outside-investigation-into-complaints-against-former-apd-assistant-chief-department/. 13 Lisa Tatum, “Austin Police Department Independent Investigation: Fact Investigation” (The Tatum Law Practice, April 2020), http://www.austintexas.gov/edims/pio/document.cfm?id=338889. 14 https://joplin3-austin-gov-static.s3.amazonaws.com/production/media/documents/2019- 1308_NFC_Stephenson.pdf 15 Ibid. 16 Katie Hall, “For Last Decade, Police Had More Whites, Fewer Hispanics than City Has,” Austin American- Statesman, December 24, 2016, https://www.statesman.com/news/20161224/for-last-decade-police-had-more- whites-fewer-hispanics-than-city-has. 12  population at that time was approximately 48% white, 35% Hispanic, 7% Black, 7% Asian/Pacific Islander, and 3% other races and ethnicities. According to APD demographic data as of December 2020, the diversity among sworn personnel has not materially changed. Among all ranks, APD officers are 66.7% white, 21.8% Hispanic, 7.3% Black, and 3.0% Asian/Pacific Islander (compared to 2019 census data, which reflects that Austin’s population is 48.3% white, 33.9% Hispanic, 7.8% Black, 7.7% Asian/Pacific Islander, and 3.5% multi-racial).17 Moreover, the higher ranks of the Department are far less diverse, as white officers constitute 70% of sergeants, 79.7% of lieutenants and 84.2% of commanders, while other racial and ethnic groups make up significantly lower percentages of the higher ranks. Allegations of Excessive Uses of Force. The CPE’s 2016 report also found that APD was more likely to use force in African American and Latino communities, including severe force. These racial disparities persisted even after controlling for such factors as crime, education, homeownership, income, youth, unemployment, and poverty rates.18 There have been many additional allegations of excessive uses of force against APD officers in recent years. For instance, two officers were charged criminally and terminated in 2018 for allegedly using a Taser weapon on an unarmed man who was in a kneeling position. Body camera footage captured the incident, but the officers were later found not guilty.19 In addition, on April 24, 2020, an officer shot and killed Michael Ramos as he attempted to leave the scene of an encounter.20 The officer has since been indicted on murder charges.21 Finally, following the death of George Floyd in Minneapolis and the ensuing protests across the country, APD was involved in three incidents in which non-violent protesters were struck by less-lethal police fire (e.g., rubber or plastic bullets designed to be non-lethal): a man in his 20s who was in critical condition, a teenage boy who was hit in the forehead, and a pregnant woman who was hit in the abdomen.22 Finally, a report released in September 2019 by the Human Rights Clinic of the University of Texas School of Law alleged that “Austin police are violating international human rights standards during mental health calls”, as “Austin has the highest per capita rate of police shootings during mental health calls of the 15 largest US cities.”23 17 https://www.census.gov/quickfacts/fact/table/austincitytexas/LND110210 18 Kroll cautions, however, that the CPE report was based on comparisons to residential population data. Issues pertaining to different benchmarks are discussed in Section 3. 19 Russell Falcon, “APD Officers Fired after Previously Cleared of Excessive Force Charges,” KXAN Austin, March 18, 2019, https://www.kxan.com/news/local/austin/apd-officers-fired-after-previously-cleared-of-excessive-force- charges/. 20 Austin Sanders, “Activists Call for Firing of APD Leadership in Wake of Officer Involved Shooting,” The Austin Chronicle, April 27, 2020, https://www.austinchronicle.com/daily/news/2020-04-27/activists-call-for-firing-of-apd- leadership-in-wake-of-officer-involved-shooting/. 21 Tony Plohetski, “Austin Police Officer Christopher Taylor Charged with Murder in Michael Ramos Shooting,” Austin American-Statesman, March 10, 2021. https://www.statesman.com/story/news/politics/2021/03/10/michael-ramos- austin-police-department-officer-christopher-taylor-indicted-murder/6940079002/ 22 Kevin Clark, “Austin Police Chief: Force Used against Protesters 'within Policy',” KXAN Austin, June 1, 2020, https://www.kxan.com/investigations/austin-police-chief-force-used-against-protesters-within-policy/. 23 Robert Sims and Alyssa Goard, “Fatal Use of Force by APD during Mental Health Calls Violates Human Rights, Report Finds,” KXAN Austin, September 25, 2019, https://www.kxan.com/news/fatal-use-of-force-by-apd-during- mental-health-calls-violates-human-rights-report-finds/. 13  Resolution 66. On December 5, 2019, the Austin City Council adopted City Council Resolution No. 20191205-066 (“Resolution 66”), which directed the City Manager to initiate a comprehensive evaluation of the Austin Police Department. In particular, Resolution 66 called for an evaluation of: Use of Force / Public Interactions • • • • “All use-of-force incident reports from June 2019 to November 2019, analyzing them by location, any resulting charges, the outcome of each incident, and demographic information including race, ethnicity, and language spoken of all persons involved.” “Aggregate data on every recorded interaction from June 2019 to November 2019 with any member of the public, including type of interaction and its outcome (search, arrest with charges, citation with charges) and an evaluation on whether there are racial and/or ethnic or other disparities in searches, arrests, charges and citations.” Recruitment and Promotion • Recruiting and ways to improve ethnic and gender diversity in cadet classes and whether certain practices and procedures, “including the use of disqualification codes for cadets, have a disparate effect or impact based on race, ethnicity, national origin, sex, religion, sexual orientation, or gender identity.” “Creating measurable benchmarks for enhancing diversity at all levels of the department.” “The basis for the weighting of assessment center panel scores in the professional ranking and promotion of APD officers, including the degree to which the use of assessment centers counters potential bias or disparate impact in the promotion process, if at all, and best practices for the transparency and use of assessment centers in ranking, and the appropriate weights for promotion consideration at each officer level.” Kroll’s Scope of Work. This report summarizes Kroll’s findings and recommendations concerning the above areas of focus, where the availability of reliable data allowed. Use of Force / Public Interactions Starting with Section 3, Kroll has attempted to provide a contextualized understanding of how, when, and against whom the APD uses force. From these findings, APD leadership can identify appropriate policies, training, employee screening and monitoring, and other concrete ways to improve officer decision-making and reduce the use of force during encounters with the public. Specifically, this report examines factors that contribute to the use of force by APD officers; whether race, ethnicity, gender, or similar person characteristics impacts the decision to use force in a particular case; whether disparities exist in use of force rates across different racial and ethnic groups; and whether the rates of force experienced by persons of different races and ethnicities align with the risk factors for having force used by police. Kroll further examines how APD can improve its use of force data collection processes to facilitate more comprehensive analyses in the future, and we recommend ways to reduce disparities in APD use-of-force incidents. The report further examines use of force counts and relevant demographics, use of force by geographical context, use of force rates (with different denominators including population counts, total arrests made by APD, and total suspects by criminal events reported to APD), force severity, individuals involved in one use of force incident compared to those individuals with multiple use of force incidents, and within-arrest analyses to determine the factors that delineate arrests without force compared to those that result in force. 14  Section 4 provides a comprehensive qualitative analysis and review of approximately 1,321 specific use- of-force incidents from June to November 2019. This section highlights a number of specific instances in which the use of force does not appear to have been justified, despite APD supervisory review and justification as set forth in APD’s documentation. We also provide analysis of trends and patterns within that six-month data set. Stops, Arrests, and Searches Section 5 documents the patterns and trends observed for APD motor vehicle stops during a one-year period (January 1 to December 31, 2020). Although benchmark analyses could not be conducted due to the limitations of these data and the lack of a valid benchmark, Kroll included additional multivariate analyses to examine the racial/ethnic disparities in the outcomes of motor vehicle stops – including verbal warnings, written warnings, citations, and arrests. Section 5 also examines all arrests during a four-year period (January 1, 2017 to December 31, 2020). These analyses include all arrests and are not limited to an examination of arrests that occur as the result of a traffic stop (see above) or arrests that involve the use of force (the focus of Section 3). Specifically, Section 5 details the patterns and trends associated with APD arrests over the course of four years, including any resulting racial/ethnic disparities. Recruitment, Selection, and Promotions Section 6 details Kroll’s review and analysis of APD’s recruitment, selection, and promotion policies and processes to determine how they potentially impact racial, ethnic, and gender diversity. In this section, Kroll also examines career development procedures to determine if there are sufficient career development opportunities to help officers achieve their career goals. In Section 7, Kroll makes recommendations on how APD’s policies and processes can be improved to increase diversity at all ranks within APD. A Note on Methodology and Data Collection The methodology employed by Kroll to analyze the various sources of data is explained in each section of the report. Although a portion of Kroll’s contract, consistent with City Council Resolution 66, had called for a detailed analysis of aggregate data on every recorded interaction from June 2019 to November 2019 between APD and . . . any member of the public, including type of interaction and its outcome (search, arrest with charges, citation with charges) and an evaluation on whether there are racial and/or ethnic or other disparities in searches, arrests, charges and citations. However, APD’s data collection was flawed and did not permit us to provide an accurate and complete analysis of all APD public interactions with civilians encompassing stop, arrest, and citation data for the specified time period. As detailed within our report, the APD data documenting the frequency and circumstances surrounding APD motor vehicle stops prior to January 1, 2020 are not valid. As a result, we provide findings from analyses of traffic stop data from January 1 to December 31, 2020, rather than the 6- month period in 2019 originally specified in the contract. These analyses are limited based on the availability and validity of specific data points. The problems and limitations of all APD data used in this report (e.g., use of force, stops, arrests, citations, warnings) are documented both prior to presenting findings within each report section, and also as part of the recommendations in Section 7. As noted, for the use of force review, Kroll expanded the analysis from the six months requested to a four- year period covering January 1, 2017 to December 31, 2020, which allowed for a more comprehensive and meaningful review of the use-of-force data. 15  3. REVIEW AND DATA ANALYSIS OF APD USE OF FORCE (January 2017 to December 2020) Police agencies across the country are beginning the difficult work of examining their use of force policies and practices; identifying any racial, ethnic, and gender disparities in these practices; exploring the possible contributors to these disparities; and developing and executing evidence-based practices to reduce these disparities. With this study of the use of force by the Austin Police Department (APD), the Kroll research team provides additional information for a more contextualized understanding of how, when, and against whom the APD uses force. From these findings, APD can identify appropriate policies, training, employee screening and monitoring, and other concrete ways to improve officer decision-making and reduce the use of force during encounters with the public. Broadly defined, police use of force is an action taken by the police that threatens, attempts, or employs physical force to compel compliance from an unwilling subject (Garner et al.,1995; Henriquez, 1999). Most studies find that the use of force is a rare occurrence, with approximately 1-5% of police encounters resulting in force (Davis et al., 2018; Garner, Hickman, Malega, & Maxwell, 2018). The prevalence of police use of force, however, depends upon how it is measured. Unfortunately, most use of force studies do not clearly define the concept of force and vary in its measurement; similarly, reporting requirements differ across police agencies, making comparisons across agencies challenging (Garner, Maxwell, & Heraux, 2002, 2018; Hickman et al., 2008; Terrill et al., 2018).24 These differences in how force is measured are critical to understand because the characteristics that predict police use of force frequently vary by how it is measured (Garner et al., 2002). The prevalence of force also depends on whether the sample includes all encounters or only those resulting in arrests, with a higher rate of force and more serious force for those arrested (Davis et al., 2018; Garner et al., 1995; Hickman et al., 2008). All these issues are taken into consideration in Kroll’s analyses presented within this report. Specifically, Kroll has attempted to address the following research questions in its analysis of APD use of force: 1. What factors, including a combination of factors, contribute to the use of force by APD officers? We specifically attempt to unravel the role of race, ethnicity, gender, or similar person characteristics in APD’s decisions to use force. 2. Are there disparities that exist in use of force rates that are differential across different racial and ethnic groups? 3. Does the rate of use of force experienced by persons of different races or ethnicities align with those groups’ representation among persons at risk for having force used against them by police? 4. How can APD improve its use of force data collection processes to facilitate more comprehensive analyses in the future? 5. What recommendations may assist APD to reduce disparities in use of force incidents? 24 For a comprehensive review summarizing how police use of force has been conceptualized and measured, as well as the methodological limitations of previous research, see Hollis (2018). For a review of the strengths and weaknesses of various use of force data sources, see Garner, Maxwell, & Heraux (2002). 16  The current report examines use of force reported by APD from January 1, 2017, through December 31, 2020 (48 months). The analyses presented examine: • Use of force counts and relevant demographics • Use of force by geographical context • Use of force rates (with different denominators including population counts, total arrests made by APD, and total suspects by criminal events reported to the APD) • Force severity • Individuals involved in one use of force incident compared to those individuals with multiple use of force incidents • Within-arrest analyses to determine the factors that delineate arrests without force compared to those that result in force The analyses for this report are comprehensive and rely on the prior scholarly and empirical literature to ensure that contextual factors explaining uses of force are included. Beyond examining patterns in uses of force, our report also highlights suggested changes to the data structure maintained by APD to provide within-agency oversight on key issues as they arise in the future. It is important to note that, although this report assesses APD’s use of force patterns and trends, statistical analyses used to measure disproportionality and disparity cannot be reliably used to determine the reasons for these differences, including whether individual officers, or the agency as a whole, engages in racial discrimination or bias. For this report, we define disproportionality as a difference in outcomes for a single racial/ethnic group (e.g., use of force against Black individuals) when compared to that group’s representation in a selected comparison population (e.g., Black residential population). In contrast, disparity refers to differences in outcomes (e.g., use of force) across racial/ethnic groups. These two concepts require different analytical strategies. The analytical strategy used for this use of force report involved several steps. First, we conducted a systematic data audit of internal affairs reports related to use of force to identify missing or inconsistent data (and fields) across the measures of interest. Second, we examined the variables descriptively through percentages and counts, and present a series of benchmark analyses to estimate racial/ethnic disproportionality. Finally, we considered racial/ethnic disparities in use of force by examining differences in the types of force used, locations of force, circumstances of the events where force was used, the location of the civilian whom force was used against, and officer(s) characteristics who used force. Section 3.1 describes the individuals who had force used against them by APD officers from 2017-2020 and the characteristics of that force. Section 3.2 examines racial/ethnic disparities in use of force by comparing rates of force at the department and sector level to several different “expected” rates of force, known as benchmarks. Disparity ratios compare Black and Hispanic individuals’ likelihood of experiencing force compared to whites. Section 3.3 examines all arrest encounters from 2017-2020 to identify the factors that predict whether the use of force occurred during these encounters. 17  3.1 APD Use of Force - Overview APD Use of Force Policies Use of force is governed by APD General Order 200 (Response to Resistance), which delineates when and how force can be used by APD officers.25 APD officers are trained on the Dynamic Response to Resistance Model (DRRM), which is a decision-making model defined in APD training materials as follows. The DRRM: Supports the progressive and reasonable escalation and de-escalation of officer-applied force in proportional response to the actions and level of resistance offered by a subject. The level of response is based upon the situation encountered at the scene and the actions of the subject in response to the officer’s commands. This DRRM training model is based on the tenet of fluidity and ongoing assessment within an encounter; it focuses on the idea that officer actions are taken in response to subject resistance. The goal of the DRRM is to bring every encounter to a compliant resolution, with APD officers seeking to “use only that amount of objectively reasonable force which appears necessary under the circumstances to successfully accomplish the legitimate law enforcement purpose in accordance with this order” (General Order 200.3). The response-to- resistance General Orders have undergone several changes in recent years; we provide a brief summary of some of the most noteworthy revisions below. In 2017, APD updated and revised General Orders 200 and 211 (APD, 2019) by: • Adding a definition of de-escalation and providing a description of de-escalation tactics and guidance for using them. • Expanding the review of response to resistance incidents to include the entire chain of command through the Commander of the involved officer. • Instituting the practice of a secondary review of Level 3 incidents by a commander not in the involved officer’s chain of command. In November 2018, the current four-level classification system for severity (described in Table 1 below) was upgraded from a three-level system.26 Most recently, at the direction of the City Manager, the Office of Police Oversight partnered with APD to begin rewriting several of the APD’s General Orders across a variety of policy topics including use of force. This is an ongoing process that has included an OPO assessment of six use-of- force related General Orders based on “8 Can’t Wait” recommendations as well as best practices identified by the Police Executive Research Forum (PERF) and the International Association of Chiefs of Police (IACP); they also conducted a survey of 1,400 community members who provided feedback on current policies and OPO’s 25 Other relevant APD General Orders include General Order 202 (Firearm Discharge Situations), General Order 204 (Leg Restraint Device), General Order 206 (Control Devices and Techniques), General Order 208 (TASER Device Guidelines), General Order 211 (Response to Resistance Inquiry, Reporting and Review), and General Order 212 (Force Review Board). 26 At the time of this change, Level 4 was added to the classification; no changes were made to Level 1; and any Taser application became a Level 3, whereas prior to this use of Taser that caused incapacitation was Level 2, but if it did not it was level 3. Later, in May 2020, deployment of a police canine that did not result in injury was also added to Level 3. Previously, APD had only tracked actual bites to skin and/or injury to subject as Level 2. 18  recommended changes (OPO, 2021). This feedback and OPO’s final recommendations were published in October 2021 and included suggested revisions to the following policy areas: 1. Restrict shooting at moving vehicles27 2. Exhaust all alternatives before using deadly force 3. De-escalation28 4. Duty to intervene in cases of improper or excessive use of force29 5. Ban chokeholds and strangleholds30 6. Warn before shooting (OPO, 2021)31 As of this report writing, APD and the City Manager are reviewing OPO’s final recommendations; final revision of the policies is pending and will be based on legal review, policy development, input from City Council, and other city stakeholders. APD General Order 200 requires that any use of force be documented promptly and accurately in accordance with General Order 211 (Response to Resistance Inquiry, Reporting, and Review). By policy, upon using force, the involved officers must notify their supervisor as soon as practicable following the incident. For purposes of inquiry, reporting, and review, APD General Order 211 categorizes use of force into four types: Level 1, Level 2, Level 3, and Level 4, which are based on the response to resistance employed during the incident. These levels are established for inquiry, reporting, and review purposes only. Level 1 is the highest severity, while Level 4 is the lowest. Table 1 provides a summary of the types of reportable uses of force that are categorized within each of these levels. For all use of force incidents, the primary reporting employee completes the Response to Resistance incident report using title code 8400; supervisors review these reports and change the title code to the appropriate level of force (e.g., 8403 for a Level 3 incident). Supplements to the incident report are completed by all other employees who were involved, witnessed, or assisted at the scene of a use of force incident. Supervisors are also required to complete a Response to Resistance Supplement (written supplement to primary employee's incident report) for all Level 1 incidents. Supervisory response to the scene also varies by level according to General Order 211 (Sections 211.6-211.9). It is required for all Level 1 and 2 incidents, and when certain criteria are met for Level 3 incidents (e.g., use of OC spray or other chemical agent, Taser application, use of an impact weapon, any incident resulting in injury or continued complaint of pain, any deployment of a police canine for the purpose of biting a subject whose 27 APD General Order 202, Firearm Discharge Situations, Section 202.1.3, Moving Vehicles, established this mandate on July 20, 2017. 28 APD General Order 200, Response to Resistance, Sections 200.2 and 200.2.1, De-Escalation of Potential Force Encounters, established this mandate on December 13, 2017. 29 APD General Order 200, Response to Resistance, Section 200.1.3, Duty to Intercede, established this mandate on July 20, 2017. 30 This proposed revision is also a specific Kroll APD Response to Resistance (Use of Force) policy recommendation. 31 APD General Order 202, Firearm Discharge Situations, Section 202.1.1, Policy, established this mandate on July 20, 2017. 19  location is known to the handler which results in no injury to the subject). It is not required for Level 4 incidents. Supervisors are responsible for chain of command notification. For all reportable use of force incidents, supervisors must conduct a thorough review of all documents, including the response to resistance incident reports and any supplements to ensure completeness, accuracy, and quality. Level 1 incidents require supervisors to administer the public safety questions and record with body worn camera, but supervisors do not immediately interview involved personnel to ensure due process if a potential criminal investigation ensues. Levels 2 and 3 require verbal reviews of the incident with involved personnel, subjects, and witnesses; statements by APD employees are to be recorded with the body worn camera recording system. Supervisors are required to identify any training issues or potential policy violations as part of completing the Incident Review Packet. Once supervisory review is complete, it is forwarded up to the next level in the chain-of-command. Level of Force Level 1 Force Incidents and In- Custody Deaths Table 1: APD Use of Force Severity Levels Force Description • Any force resulting in death. • Any force that resulted in a substantial risk of death. • Any intentional firearm discharge at a person, vehicle, or structure regardless of injury. • Any intentional firearm discharge at an animal that results in injury to another person. • Any unintentional firearms discharge resulting in another person's injury or death. • Any force that resulted in serious bodily injury requiring admittance to the hospital, beyond emergency room treatment and release (e.g., serious disfigurement, disability, or protracted loss or impairment of the functioning of any body part or organ). • Use of any impact weapon, including kinetic energy projectiles, and improvised weapons, • • that strikes the head of a subject. In-Custody Deaths: For inquiry, reporting, and review purposes, all in-custody deaths occurring prior to or within 24 hours after booking shall be treated as Level 1 incidents and require concurrent inquiries conducted by SIU and IA, regardless of whether force was used on the subject. The utilization of the Precision Immobilization Technique32 when serious bodily injury or death occurs. Level 2 Force Incidents • Any strike to the head by an employee with any weaponless technique. • Use of any impact weapons, including kinetic energy projectiles (other than a Taser), and improvised weapons, to strike a subject and contact is made, regardless of injury. (A strike to the head is a Level 1). • Any deployment of a police canine resulting in a bite to a subject's skin, or which results in • any injury to a subject. The utilization of the Precision Immobilization Technique unless serious bodily injury or death occurs. 32 Precision Immobilization Technique is commonly referred to as the PIT maneuver in policing and by APD in use of force reports. 20  Level 3 Force Incidents • Use of Oleoresin Capsicum (OC/Pepper Spray) or other chemical agent on a subject. • Any Taser application. • Use of any impact weapon, including kinetic energy projectiles or any other similar object, in an attempt to strike a subject but no contact is made. • Use of a baton for a non-striking purpose (e.g., prying limbs, moving, or controlling a subject). incident. • Any force resulting in injury or a continued complaint of pain, but not rising to a Level 1 or 2 • Any weaponless technique (e.g., hand/palm/elbow strike; kicks or leg sweeps; take-downs) that causes an impact to the body with or without a complaint of injury or pain. (A weaponless strike to the head is a Level 2). • Any deployment of a police canine for the purpose of biting a subject whose location is known to the handler which results in no injury to the subject. Level 4 Force Incidents • A level of force utilizing empty hand control techniques that does not result in injury or continued complaint of pain and does not rise to a Level 3 response to resistance. Examples include, but are not limited to: o Restricting a subject’s movement by strength or body weight (to include resisted escorting or handcuffing of a subject who is actively resisting arrest beyond the initial or reflexive stiffening or pulling away of a person’s arm(s) that officers commonly encounter during handcuffing). o Using leverage or strength to bring a subject’s arms or legs together for the purposes of controlling, handcuffing, or hobbling the subject (to include resisted control, handcuffing, hobbling when the subject is actively resisting arrest beyond the initial or reflexive stiffening or pulling away of the subject’s arm(s) or leg(s) that officers commonly encounter during efforts to control, handcuff, or hobble a subject). o Pressure point control tactics Source: APD General Order 211.2.1-211.2.4 (Response to Resistance Inquiry, Reporting and Review) Measuring APD Use of Force Our evaluation relies on APD’s official use of force data. Use of force incident counts are captured from APD internal affairs reports. Each use of force incident has an internal affairs identifier. Counting uses of force from these incidents is not a straightforward endeavor and requires a thorough coding and measurement audit, which was completed by the Kroll team. Based on this audit, several recommended changes to the collection and analysis of these data are provided at the end of this report. It is important to note that use of force counts can vary dramatically based on the unit of analysis at which they are measured. For example, as depicted in Figure 1 below, a single use of force incident or encounter may involve one or more individuals receiving one or more police actions by one or more officers. And because a single use of force incident may include multiple types of force, used against multiple individuals by multiple officers, there are a variety of ways force could be counted, such as (1) the number of incidents involving any use of force, (2) the number of individuals who had force used against them in a single encounter, (3) the number of different types of force (or officer actions) used, or (4) the number of officers using force. Each of these measures would result in different use of force counts. For most of the analyses that follow, we measure the use of force as the number of individuals that had force used against them during a single encounter (i.e., any use of force that was person-event specific).33 If an 33 If a different unit of analysis is used to examine use of force incidents, it is noted in the text and accompanying tables. 21  individual had force used against him/her during more than one encounter with the police during the study period, multiple uses of force are included in the data analyses. Measured in this way, use of force (the number of individuals having force used against them) likely includes multiple police actions given the escalating nature of force (i.e., an officer may initiate with a low level of force and increase in severity if resistance increases), and multiple officers that could use force against a single individual. Thus, within internal affairs reports for incidents in which officers used force against two or more individuals, each individual who had force used against them (in a given incident) was measured as a use of force event. Likewise, if a single individual had force used against him or her that involved multiple officers, Kroll counted one use of force per individual (per event) regardless of how many officers used force on the individual. It is also important to note that not all use of force incidents include information about the specific individual that force was used against. APD requires officers to complete use of force incident reports even where the identity of the person the force was used against was unknown or otherwise undocumented (e.g., use of force in crowd control during disturbances or demonstrations). While uses of force against unknown individuals are included in some analyses below documenting the amount of force used by APD, they are excluded in other analyses that examine the specific characteristics of the individuals who had force used against them. Figure 1: Hypothetical Example of Use of Force Measures, by Unit of Analysis Source: “Examining the impact of Integrating Communications, Assessment, and Tactics (ICAT) de-escalation training for the Louisville Metro Police Department: Initial Findings,” by Engel et al., 2020, pg. 13. Finally, based on the way APD collects use of force data, only the highest level or severity of use of force is available for analytical purposes. Previous studies have consistently shown that police force is often used progressively and calibrated as a response to individuals’ resistance (Alpert & Dunham, 1999; Garner et al., 1995; Hine, Porter, Westera, & Alpert, 2016; Wolf, Mesloh, Henych, & Thompson, 2009). This progressive use 22  l s a u d v d n i i I f o # 900 800 700 600 500 400 300 200 100 0 of force, however, cannot be measured with the APD data that was provided to Kroll. Only the most severe type of force used against an individual is systematically and reliably captured. As a result, the counts of specific types of force are undercounted within the APD data. This is particularly true for lower levels (or less severe) types of force, which previous research shows comprises a majority of all use of force incidents (Bayley & Garofalo, 1989; Garner et al., 2018; Stroshine & Brandl, 2019). APD Use of Force Descriptives Given these caveats in the use of force measurements, the remainder of this section provides a detailed description of the force used by APD during the period under examination (January 1, 2017 to December 31, 2020).34 Figure 2 graphs, by month, the number of individuals who had force used against them during unique encounters with police (including both known and unknown individuals). Figure 2: Individuals Who Had Force Used Against Them by Month (Jan 1, 2017 – Dec 31, 2020) 7 1 - n a J 7 1 - b e F 7 1 - r a M 7 1 - r p A 7 1 - y a M 7 1 - n u J 7 1 - l u J 7 1 - g u A 7 1 - p e S 7 1 - t c O 7 1 - v o N 7 1 - c e D 8 1 - n a J 8 1 - b e F 8 1 - r a M 8 1 - r p A 8 1 - y a M 8 1 - n u J 8 1 - l u J 8 1 - g u A 8 1 - p e S 8 1 - t c O 8 1 - v o N 8 1 - c e D 9 1 - n a J 9 1 - b e F 9 1 - r a M 9 1 - r p A 9 1 - y a M 9 1 - n u J 9 1 - l u J 9 1 - g u A 9 1 - p e S 9 1 - t c O 9 1 - v o N 9 1 - c e D 0 2 - n a J 0 2 - b e F 0 2 - r a M 0 2 - r p A 0 2 - y a M 0 2 - n u J 0 2 - l u J 0 2 - g u A 0 2 - p e S 0 2 - t c O 0 2 - v o N 0 2 - c e D During this 48-month study period, APD recorded 10,026 individuals who had force used against them during unique encounters. Of these, 985 individuals (9.8%) had missing identification or demographic information (e.g., name, race, ethnicity, gender). The largest reason given for missing this information was “unidentified person” (82.4% of missing cases), followed by “record no longer exists” (9.4%), and “unknown” person (8.2%). Given that the reasons for “unidentified” persons vary, further details are needed (but unavailable) to understand the circumstances of the use of force event. Based on discussions with APD staff, it appears that “unidentified” persons could be juveniles who are not identified in the records system (e.g., when a record no longer exists, it is possible that it has been expunged). It is also possible that the force was used as a response 34 APD Officers reported 46 individuals had force used against them in which the force occurred outside of APD Police Districts. It is unknown by the research team why this discrepancy appears in the data. These cases are excluded from later geographic analyses. 23  to resistance in crowd control, and the individual is unknown. As noted in the data enhancement recommendations in Section 7 of this report, it would be instructive to gather this information moving forward. In summary, approximately 10% of individuals who had force used against them are unidentified in APD data or unknown by APD officers, and the reason this information is unavailable is not systematically captured. This missing information is especially salient for examining use of force patterns in 2020. As shown in Figure 2 above, there is a striking increase in reported uses of force in May 2020, likely related to APD’s response to protests and civil unrest. When considering yearly trends, the number of individuals who had force used against them steadily increased from 2017 to 2020 (see Figure 3). In 2017, force was used against a total of 1,962 individuals (4.7% of whom were unknown). Slightly more (2,091 individuals) had force used against them in 2018 (4.0% of individuals were unknown). Continuing this trend, in 2019 force was used against 2,864 individuals (3.1% of whom were unknown). Thus, from 2017 to 2019, the number of individuals who had force used against them by APD increased 46.0% (+773 individuals). This upward trend in police use of force continued in 2020 (increasing 30.1% from 2019, and 58.4% across the four years). For several reasons (e.g., response to the COVID-19 pandemic, significant protests and civil unrest related to police use of force, and the economic downturn), APD use of force patterns in 2020 may be considered an anomaly. One significant difference in 2020 is the percentage of unknown individuals involved in the use of force. Figure 3 below shows that the percentage of unknown individuals who had force used against them increased from 3.1% of force incidents in 2019 to 30.1% of force incidents in 2020. Again, this pattern is likely related to APD’s response to protests and civil unrest in 2020. Given that the primary purpose of this report is to identify and better understand any racially or ethnically disparate patterns in APD use of force, only use of force incidents where the individual’s race/ethnicity is known (90.2% of the total number of individuals who had force used against them during the 48-month study period) are included in most of the statistical analyses that follow. For analyses that consider what APD sector the force occurred in, the sample size decreases to 8,995 individuals due to missing data.35 35 68 use of force incidents were unable to be mapped in the APD sectors; 30 did not have x/y coordinates on the state plane, and 38 had coordinates outside of each sector parameter. 24  Figure 3: Number of Known and Unknown Individuals Who Experienced Use of Force by Year l s a u d v d n i i I f o # 3500 3000 2500 2000 1500 1000 500 0 1962 1868 94 2017 2864 2776 88 2019 2091 2008 83 2018 3109 2389 720 2020 Unknown Individuals Known Individuals Although it is important to understand the use of force patterns and practices in response to protests and civil unrest (when the race/ethnicity of the individuals may not be routinely captured), such is beyond the scope of this report, which examines trends in racial/ethnic differences in use of force. Kroll recommends that the City of Austin conduct a separate examination of APD’s response to protests and use of force during these types of events. For the remainder of analyses that follow in this report, the sample examined is limited to those who had force used against them with known demographic characteristics. The significant increase in the number of individuals who had force used against them is in direct contrast to the trends for APD arrests. (This may be partially explained by a change in APD use of force reporting levels in 2019, which added some low-level use of force incidents that were not previously captured in APD reporting.) As shown in Figure 4, the total number of arrests (for any criminal offense) declined by 50.6% from 2017 to 2020. While the number of arrests in 2020 were likely impacted by the response to the pandemic, the overall downward trend in the number of arrests was established prior to March 2020. When considering the yearly reduction from 2017 to 2019, total arrests declined by 28.1%. As a result, the significant increase in use of force documented above cannot be explained by more police involvement in encounters that result in arrests. 25 Figure 4: Total Number of APD Arrests (Jan 1, 2017 – Dec 31, 2020) h t n o M y b s t s e r r A f o # 4500 4000 3500 3000 2500 2000 1500 1000 500 0 7 1 - n a J 7 1 - r a M 7 1 - y a M 7 1 - l u J 7 1 - p e S 7 1 - v o N 8 1 - n a J 8 1 - r a M 8 1 - y a M 8 1 - l u J 8 1 - p e S 8 1 - v o N 9 1 - n a J 9 1 - r a M 9 1 - y a M 9 1 - l u J 9 1 - p e S 9 1 - v o N 0 2 - n a J 0 2 - r a M 0 2 - y a M 0 2 - l u J 0 2 - p e S 0 2 - v o N Table 2 shows the gender and race/ethnicity of the individuals who had force used against them in the four- year period from 2017 to 2020. Of the 9,041 known individuals who had force used against them, roughly 74% are male and 26% female. Approximately 31.4% of these individuals are Black, 33.6% Hispanic, 33.1% white, and 1.8% are collectively categorized as “other” (which includes 0.9% Asian/Pacific Islander, 0.2% American Indian, 0.3% Middle Eastern, and 0.4% unknown). When the data is combined, Black, Hispanic, and white individuals accounted for 98.1% of all use of force events, and therefore the remainder of the statistical analyses in this report examine only these three racial/ethnic categories. Table 2: APD Use of Force: Race/Ethnicity and Gender of Individuals (Jan 1, 2017 – Dec 31, 2020) All Individuals (n=9.041) Gender Female Male Race/Ethnicity Black Hispanic White N (%) 2,362 (26.1%) 6,679 (73.9%) 2,836 (31.4%) 3,037 (33.6%) 2,994 (33.1%) 174 (1.8%) Other (Asian/Pacific Islander, American Indian, Middle Eastern, Unknown) 26  Individuals’ Resistance The conduct, resistance, and emotional state of individuals during use of force situations are also captured in use of force reports completed by officers. For individuals’ conduct/resistance, APD trains officers to consider the following: • In any force encounter, and regardless of the tool/weapon/tactic chosen by the officer, the officer’s decision to use the tool/weapon/tactic is dependent upon the suspect’s level of resistance, the threat reasonably posed by the suspect, and the totality of the circumstances. • All responses to resistance must be objectively reasonable. In other words, another reasonable officer, faced with the same circumstances, could believe that the officer’s response to resistance was reasonable (APD Academy Lesson Plan: Force Options, 2021). In mid-2017, APD altered its collection of subjects’ actions on use of force incident reports. The new reporting includes six categories: (1) No Resistance; (2) Passive Resistance; (3) Defensive Resistance; (4) Preparatory Resistance, (5) Aggressive Resistance, and (6) Deadly Resistance. These terms are further defined in APD training materials, provided in Table 3. Table 3: APD Types of Resistance Description Type of Resistance No Resistance (Compliant) # 1 2 a. A subject who does not resist and follows all commands is compliant. Only a law enforcement officer's presence and verbal commands are required when dealing with these subjects; no coercive physical contact is necessary. Passive Resistance a. A passively resistant subject fails to follow commands and, although not threatening, may be verbally non-compliant, questioning or disagreeing. The subject’s behavior, actions and cues are neutral in nature and non-assaultive. b. An example would be a subject going limp during an arrest. Policy does not allow for the use of a Taser when the only resistance offered is passive resistance. 3 Defensive Resistance a. Defensive resistance is voluntary physical movement and / or muscular tension resistance by a subject that attempts to prevent the officer’s control. b. This is the most common type of resistance encountered by officers. c. The situational context (totality of the circumstances) must be considered when determining if a subject is defensively resisting or merely passively resisting. Likewise, the “totality of the circumstances” must be considered when choosing which level of force is necessary to control the situation and prevent unnecessary harm. d. Examples of defensive resistance by a subject may include pulling away from the officer’s grasp, locking arms under their body, resisting handcuffing/frisk, fleeing from an officer, or evading arrest by concealment. 4 Preparatory Resistance a. Although, on the surface, a suspect may offer or exhibit behaviors associated with “passive resistance,” or “defensive resistance,” the officer may also recognize the subject is preparing to offer greater resistance or launch an attack through behavioral cues (verbal, non-verbal and/or physical). Depending on the circumstances, certain 27  forms of passive resistance, such as verbal non-compliance and/or verbal threats, may be indicators of Preparatory Resistance. The officer must be prepared to adjust tactics for such a change or attack. b. Preparatory Resistance may be exhibited or observed during any of the [other] resistance categories. In response, the appropriate degree of force would depend on the articulable and specific threat actually perceived by the officer. 5 Aggressive Resistance a. Aggressive resistance is an offensive action by the subject who attempts to push, throw, strike, tackle, or physically harm the officer or another person. b. If the officer or members of the public are threatened by the subject’s actions, the officer must respond with appropriate force to stop the attack and defend himself/herself or others. 6 Deadly Resistance a. Deadly Resistance is an offensive action by the subject that could seriously injure or kill the officer or another person if immediate action is not taken to stop the threat. As noted by APD, the old classification system for the level of resistance cannot be readily translated into the new classification system. Therefore, Kroll’s analyses below that examine the level of resistance shown during use of force events is only reported for a three-year period (2018 – 2020). For 76% of the cases, there was a single measure of resistance reported. In the remaining 24%, either a single officer reported more than one level of resistance by the individual, or multiple officers reported a single measure of resistance that was different from one another. Therefore, when multiple levels of resistance are reported, we collapse the data into the highest level of resistance shown for each individual. Table 4 documents the highest level of resistance reported by officers for the individuals who had force used against them; as shown, over half of the individuals who experienced force were defensively resistant (57.8%). The second most common level of resistance was aggressive resistance (26.1%), while less than one percent of individuals (0.6%) who experienced force displayed deadly resistance toward officers. It initially appears that the level of resistance shown to officers during force events was relatively stable across the three-year period. On a 6-point scale of resistance severity (where 1 = not resistant, 2 = passive resistance, 3 = defensive resistance, 4 = preparatory resistance, 5 = aggressive resistance, and 6 = deadly resistance), the average level of resistance each year does not significantly differ (2018 average = 3.61; 2019 average = 3.54, 2020 average = 3.61).36 36 APD's training material describing resistance levels lists five types of resistance in order of severity and then lists preparatory resistance last, indicating that it "may be exhibited or observed during any of the five preceding categories." In creating our resistance severity scale, however, we did not place it last (because it is not most severe), and found it did not have a logical place within the scale. Since the definition of preparatory resistance is "the officer may also recognize the subject is preparing to offer greater resistance or launch an attack through behavioral cues," we labeled preparatory resistance as more serious than defensive resistance based on the "launch an attack" verbiage. However, we also calculated the maximum resistance variable on a five-point scale that excluded preparatory resistance. When doing so, we found that the averages decreased negligibly, but the substantive findings remained the same. That is, there were no significant differences in average resistance level by year, gender, or race/ethnicity. The percent of events with defensive resistance as most serious increased by approximately the percent that is currently listed as preparatory resistance. 28  Table 4: Resistance Level of Known Individuals during Use of Force Events, January 1, 2018 – December 31, 2020 (n=7,173) 1 = Not resistant 2 = Passive Resistance 3 = Defensive Resistance 4 = Preparatory Resistance 5 = Aggressive Resistance 6 = Deadly Resistance Average Resistance Level Total Individuals (n=7,173) 2018 (n=2,008) 2019 (n=2,776) 2020 (n=2,389) 58 (0.8%) 5 (0.2%) 35 (1.3%) 18 (0.8%) 310 (4.3%) 96 (4.8%) 137 (4.9%) 77 (3.2%) 4,148 (57.8%) 1,162 (57.9%) 1,615 (58.2%) 1,371 (57.4%) 737 (10.3%) 187 (9.3%) 279 (10.1%) 271 (11.3%) 1,871 (26.1%) 538 (26.8%) 696 (25.1%) 637 (26.7%) 46 (0.6%) 20 (1.0%) 13 (0.5%) 13 (0.5%) 3.58 3.61 3.54 3.61 As previously noted, 2020 had unique patterns related to APD’s use force, with a much larger percentage of cases having unknown suspects—30.1% compared to 3.1% in 2019—likely representing more use of force in 2020 during crowd control situations, especially in the month of May. It is therefore informative to examine the individuals who had force used against them in 2020 who were unknown to police (n=720) and compare their levels of resistance to force events where the suspect was known (n=2,389). The results show that, for unknown individuals, the average resistance score (on a scale of 1 to 6, with 6 as the highest level of resistance) is 4.1, compared to only 3.6 for individuals with known identities. Even more illustrative (see Figure 5 below), more than half of the individuals with unknown identities (55.2%) showed resistance at the two highest levels – aggressive resistance and deadly resistance – compared to only 27.2% of known individuals involved in police force situations. This suggests that officers in 2020 experienced notably higher levels of resistance during use of force events compared to previous years (when both unknown and known individuals are considered). Figure 5: Individuals Displaying Aggressive or Deadly Resistance in Police Use of Force Situations 55.2% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 27.2% Unknown Individuals Known Individuals 29  Finally, we turn to the level of resistance by individuals’ race/ethnicity to determine if differences across demographic groups exist during use of force incidents, which are displayed in Table 5. The findings suggest that there are few racial/ethnic differences in displays of resistance during use of force events. This finding is consistent with other research that has similarly found no significant differences in resistance by race/ethnicity when resistance is captured as several resistant behaviors or on a scale of aggression (Engel, 2003; Whichard & Felson, 2016). Although the sample size is small, we also note that, where there are differences, it is white individuals who are more likely to show deadly resistance toward officers (1.2% of all white individuals who had force used against them) compared to other racial/ethnic groups (0.4% of Black individuals, and 0.7% of Hispanic individuals). Table 5: Levels of Resistance during Use of Force Events, by Gender and Race/Ethnicity (January 1, 2018 - December 31, 2020) 1 = Not resistant 2 = Passive Resistance 3 = Defensive Resistance 4 = Preparatory Resistance 5 = Aggressive Resistance 6 = Deadly Resistance Average Resistance Level Total individuals (n= 7,173) Gender Male (n=5,183) Female (n=1,990) Race/Ethnicity Black (n=2,217) Hispanic (n=2,441) White (n=2,367) Other (n= 148) 58 (0.8%) 42 (0.8%) 16 (0.8%) 13 (0.6%) 23 (0.9%) 18 (0.8%) 4 (2.7%) 310 (0.4%) 223 (4.3%) 87 (4.4%) 94 (4.2%) 111 (4.5%) 98 (4.1%) 7 (4.7%) 4,148 (57.8%) 2,968 (57.3%) 1,180 (59.3%) 1,304 (58.5% 1,402 (57.4%) 1,357 (57.3%) 85 (57.4%) 737 (10.3%) 616 (11.9%) 121 (6.1%) 235 (10.6%) 252 (10.3%) 233 (9.8%) 17 (11.5%) 1,871 (26.1%) 1,294 (25.0%) 577 (29.0%) 560 (25.3%) 637 (26.1%) 642 (27.1%) 32 (21.6%) 46 (0.6%) 38 (0.7%) 8 (0.4%) 9 (0.4%) 16 (0.7%) 18 (0.8%) 3 (2.0%) 3.58 3.58 3.59 3.57 3.58 3.61 3.51 Table 5 also shows the levels of resistance by gender. The findings suggest that there are few differences between males and females in displays of resistance during use of force events. The only notable difference is that a slightly higher percentage of males than females displayed preparatory resistance. Individuals’ Impairment In some cases, police use of force involves individuals who are impaired in some manner, including through drug/alcohol use or intoxication, emotional distress, mental health issues, behavioral disorders, and intellectual or developmental disabilities. APD systematically collects information on two types of individuals’ impairment that may impact the use of force; specifically, APD officers document if they perceive that the individual is: (1) 30  under the influence of drugs or alcohol, or (2) emotional disturbed/mentally unstable. Officers may select both types of impairment. If none are selected, this is represented by the “No impairment listed” column. As documented in Table 6 below, females who had force used against them were slightly more likely to be impaired compared to males. Within impairment categories, males were more likely to be impaired by alcohol or drugs, while females were more likely than males to be reported as experiencing behavioral health-related issues or both types of impairment. Table 6 also demonstrates that Black individuals were most likely to have force used against them when they were not impaired (29.8%), while white individuals were least likely to have force used against them when they were not impaired (14.3%); that is, white individuals who had force used against them were more likely to be impaired by drugs/alcohol, behavioral health-related issues, or both. Within impairment categories, Hispanic individuals were most likely to be impaired by alcohol or drugs, while white individuals were most likely to be reported as experiencing behavioral health-related issues or both. Table 6: Impairment of Individuals during Use of Force Events, by Gender and Race/Ethnicity, January 1, 2017 – December 31, 2020; (n=9,041) No Impairment Listed Alcohol/Drug EDP/Mentally Both Unstable All Individuals (n=9,041) 2045 (22.6%) 3574 (39.5%) 1619 (17.9%) 1803 (19.9%) Gender Female (n=2,362) Male (n=6,679) Race/Ethnicity Black (n=2,836) Hispanic (n=3,037) White (n=2,994) Other (Asian/Pacific Islander, American Indian, Middle Eastern, Unknown) (n=174) 456 (19.3%) 1,589 (23.7%) 845 (29.8%) 729 (24.0%) 427 (14.3%) 44 (25.3%) 710 (30.0%) 2,864 (42.9%) 898 (31.7%) 1,442 (47.5%) 1,173 (39.2%) 61 (35.1%) 687 (29.1%) 932 (14.0%) 558 (19.7%) 413 (13.6%) 613 (20.5%) 35 (20.1%) 509 (21.5%) 1,294 (19.4%) 535 (18.9%) 453 (14.9%) 781 (26.1%) 34 (19.5%) It is instructive to further consider the categories above when the “both” category is added to “under the influence” and “EDP/mentally unstable.” In this analysis, the categories are no longer mutually exclusive. As shown in Figure 6, the majority of individuals who had force used against them were under the influence of drugs/alcohol (59.5%). In addition, 37.8% of individuals who had force used against them were considered by officers to be emotionally disturbed or mentally unstable. In contrast, only 22.6% of individuals involved in police use of force were not considered impaired in some manner by officers. 31  Figure 6: Impairment of Individuals during Use of Force Events (n=9,041) 59.5% 37.8% l s a u d v d n i i I f o % 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 22.6% No Impairment Listed Drugs/Alcohol EDP/Mentally Unstable Note: Under the influence of drugs/alcohol and EDP/Mentally Unstable are not mutually exclusive. Next, we consider the racial/ethnic differences in individuals’ impairment who had force used against them. Figure 7 shows the racial/ethnic patterns in impairment. Most striking is that roughly 30% of Black individuals who had force used against them were not listed as impaired by officers, compared to 24% of Hispanic and only 14% of white individuals. A larger percentage of Hispanic (62.4%) and white (65.3%) individuals were under the influence when they had force used against them, compared to 50.5% of Black individuals. Finally, white individuals who had force used against them were the most likely to be considered mentally unstable by officers (46.6%), compared to smaller percentages of Black (38.5%) and Hispanic (28.5%) individuals. Figure 7: Impairment Type by Individuals’ Race/Ethnicity during Use of Force Events (n=9,041) 50.5 38.5 29.8 62.4 65.3 46.6 24.0 28.5 14.3 70.0 60.0 50.0 40.0 30.0 20.0 10.0 0.0 Black Hispanic White % No Impairment Listed % Alcohol/Drugs % EDP/Mentally Unstable 32  Types of Force -- Severity As noted previously, it is only possible to analyze APD data using the most severe level of force used, rather than progressive force that would include all types of force used during a single encounter with an individual. For the remainder of Kroll’s analyses, we provide descriptive statistics for the most severe uses of force (as recorded by APD). Note that the highest use of force level is coded by APD as Level 1, while the lowest level of force is coded as Level 4. Furthermore, as described above, APD changed its level classification categories in 2018, adding a fourth category and revising the level classification for a few types of force. As a result, comparisons can only be made for the data collected in 2019 and 2020 (n=5,965). The frequency of use of force by severity level is documented in Figure 8 below. As shown, of the 5,965 known individuals who had force used against them in 2019 and 2020, only 0.5% experienced the most severe types of force (i.e., Level 1 severity, which includes firearm discharge, all death in custody, any force resulting in hospitalization, any strikes to the head). Conversely, the most common level of force severity—capturing almost half of all uses of force (49.3%)—is Level 3 (e.g., OC spray, Taser, canine bite, weaponless techniques that cause an impact to the body). Notably, 44.7% of all individuals who had force used against them experienced the lowest levels of force (i.e., Level 4, which includes empty hand control techniques that do not result in injury or continued complaint of pain). Over half of the known individuals (55.9%) had no complaint of injury or pain after force was used, while another 18% had a complaint of injury or pain but neither was observed by the reporting officer. This is consistent with other police use of force studies that for decades have shown that most police use of force incidents are minor in severity and do not involve severe injuries (Alpert & Dunham, 1999; Garner et al., 2018; Henriquez, 1999; Hickman, Strote, Scales, Parkin & Collins, 2021; Stroshine & Brandl, 2019). Figure 8: Severity Level of Use of Force (Jan 1, 2019 - Dec 31, 2020) 49.3% 44.7% 5.3% 0.5% Level 1 (n=26) Level 2 (n=275) Level 3 (n=2,547) Level 4 (n=2,311) Note: Severity level is defined in Table 1; Level 1 is the most severe category, while Level 4 is the least severe category. Table 7 below displays the use of force severity for the two-year period with available data (January 1, 2019 – December 31, 2020). As with the level of resistance analyses previously shown, the level of force severity does 33 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0%  not significantly differ across racial groups. Recall that the higher the average severity score, the less severe the use of force. Although the level of force is slightly lower for white individuals, this difference is not statistically significant. The level of force severity does vary significantly by gender. Females are about 1.6 times more likely than males to experience the least severe level of force (level 4), while males are significantly more likely to experience the other three levels of force severity. Males also have a lower average severity score than females (3.3, compared to 3.6), which again, is equivalent to more severe use of force. Table 7: Levels of Use of Force Severity by Gender and Race/Ethnicity (Jan 1, 2019 - Dec 31, 2020) Total individuals (n= 5,159) Level 1 26 (0.5%) Level 2 Level 3 2,547 (49.4%) 275 (5.3%) Gender Male (n=3,650)) Female (n=1,509) Race/Ethnicity Black (n=1,567) Hispanic (n=1,793) White (n=1,690) Other (n= 109) 23 (0.6%) 3 (0.2%) 10 (0.6%) 5 (0.3%) 7 (0.4%) 4 (3.7%) 253 (6.9%) 22 (1.5%) 88 (5.6%) 102 (5.7%) 78 (4.6%) 7 (6.4%) 2001 (54.8%) 546 (36.1%) 772 (49.3%) 910 (50.8%) 813 (48.1%) 52 (47.7%) Average Severity Score 3.38 3.29 3.60 3.37 3.37 3.41 3.28 Level 4 2,311 (44.8%) 1,373 (37.6%) 938 (62.0%) 697 (44.5%) 776 (43.3%) 792 (46.9%) 46 (42.2%) Individuals with Repeat Uses of Force Of the use of force situations with known individuals, approximately 70% (n=6,343) involved an individual who only had force used against him/her a single time during the 48-month study period (see Table 8 below). Comparatively, roughly 30% (n=2,698) involved “repeat” individuals. Repeat individuals are defined as those who had force used against them two or more times (during unique events) within the 48-month study period. For repeat individuals with multiple use of force events, the average was 3.2 events per individual, ranging from 2 to 12 use of force events against a single individual. Repeats by Race/Ethnicity and Gender To further examine the phenomena of repeat individuals experiencing APD use of force, the racial/ethnic composition of these individuals was considered and is shown in tabular format in Table 8, and graphically displayed in Figure 9. When single versus repeat individuals in use of force situations are considered, the pattern for Black individuals deviates from the other racial/ethnic groups. Specifically, Black individuals who experienced use of force were more likely to have experienced multiple separate use of force encounters with police during this four-year period. That is, 36.2% of all Black individuals who had force used against them experienced multiple use of force encounters, compared to white (27%) or Hispanic (27%) individuals. There 34  was little variation by gender, with males slightly more likely to be involved in multiple use of force encounters with police (30.4%, compared to 28.2% for females). Table 8: Individual Race/Ethnicity for Single Events vs. Two or More Use of Force Events (2017-2020) Single Event Multiple Event All Individuals (n=9,041) Gender Female (n=2,362) Male (n=6,679) Race/Ethnicity Black (n=2,836) Hispanic (n=2,994) White (n=3,037) Other (Asian/Pacific Islander, American Indian, Middle Eastern, Unknown) (n=174) 6,343 (70.2%) 1,697 (71.8%) 4,646 (69.6%) 1,809 (63.8%) 2,181 (72.8%) 2,206 (72.6%) 147 (84.5%) 2,698 (29.8%) 665 (28.2%) 2,033 (30.4%) 1,027 (36.2%) 813 (27.2%) 831 (27.4%) 27 (15.5%) Figure 9: Single vs. Multiple Use of Force Events, by Individuals’ Race/Ethnicity (Jan 1, 2017 – Dec 31, 2020) 72.8% 72.6% 63.8% 36.2% 27.2% 27.4% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Single Event Multiple Events Black Hispanic White Repeats by Impairment It is also possible that individuals involved in multiple use of force incidents are more likely to be impaired by drugs/alcohol or behavioral health-related issues. As shown in Table 9, roughly 26% of all uses of force against an individual who had force used in a single event had no impairment listed, compared to about 16% for uses of force against individuals with repeat use of force events. Although there is a high comorbidity between drug/alcohol impairment and mental/emotional stability, the percentages reported represent any officer 35  indication that the individual was impaired by drugs/alcohol or behavioral health-related issues; that is, the two impairment categories are not mutually exclusive. Across all racial/ethnic groups, a higher percentage of individuals involved in repeat use of force events were impaired by drugs/alcohol or behavioral health-related issues compared to single events. Over 90% of white individuals involved in repeat use of force events had some type of impairment, compared to approximately 80% of Black and Hispanic individuals. Across all racial/ethnic groups, most of the increase in impairment between single and multiple events was due to increased behavioral health-related impairment. Blacks were the only racial/ethnic group whose percent of drug/alcohol impairment notably increased between single and multiple events. Table 9: Individuals’ Impairment by Race/Ethnicity for Single Events vs. Multiple Use of Force Events (Jan 1, 2017 – Dec 31, 2020) All Individuals (n = 6,343) Black (n=1,809) Hispanic (n=2,203) White (n=2181) Other (n=147) All Individuals (n=2,698) Black (n=1,027) Hispanic (n=831) White (n=813) Other (n=27) Single Event No Impairment Listed 1,625 (25.6%) Drugs/Alcohol 3,726 (58.7%) EDP/Mentally Unstable 1,960 (30.9%) 649 (35.9%) 576 (26.1%) 359 (16.5%) 41 (27.9%) 846 (46.8%) 1,381 (62.6%) 1,418 (65.0%) 81 (55.1%) 548 (30.3%) 500 (22.7%) 860 (39.4%) 52 (35.4%) Multiple Event No Impairment Listed 420 (15.6%) Drugs/Alcohol 1,651 (61.2%) EDP/Mentally Unstable 1,462 (54.2%) 196 (19.1%) 153 (18.4%) 68 (8.4%) 3 (11.1%) 587 (57.2%) 514 (61.9%) 536 (65.9%) 14 (51.8%) 545 (53.1%) 366 (44.0%) 534 (65.7%) 17 (63.0%) Post-Use of Force A final consideration of APD use of force events is what happens to the individual after force is used. Of the 9,041 known individuals who had force used against them, most post-force transportation (64.2%) involved movement to a jail facility post-arrest. Approximately 73% (n=6,621) of the 9,041 known individuals who had force used against them were arrested; in most cases, the arrested individual was taken into police custody (over 86%), but other options are available and routinely used by APD, including non-custodial arrests (i.e., cite and summons) or arrest with transport to an alternative location than jail (e.g., mental facility, sobering center). For all individuals who experienced force, the second most frequent location for transportation was the hospital (12.4%). Approximately 5% of individuals were transported to mental facilities, and in 1.1% of cases individuals were transported to a detoxification center. In a very small number of incidents (< .1%, n=8), individuals were 36  1,800.0 1,600.0 1,400.0 1,200.0 1,000.0 800.0 600.0 400.0 200.0 0.0 transported to the morgue after lethal force was used.37 For 12% of the individuals who had force used against them, the officer indicated that no transportation was made, while in 4.9% of cases there is missing information in the data field. It is recommended that APD standardize a data field in the use of force incident report that captures all individuals’ post-force dispositions, even if they are not transported anywhere by APD. Geographic Analyses Studies of police use of force have consistently demonstrated that the frequency of force varies dramatically across geographic locations. As noted earlier, due to missing data on x-y coordinates and out-of-sector events, the valid number of cases for sector analyses is 8,987. Figure 10 displays the use of force rates per 10,000 residents based on 2020 census population estimates. It shows that George Sector (which includes the downtown entertainment district) is a clear outlier, with the highest use of force rate per 10,000 residents across all sectors; 23.0% of individuals who had force used against them by APD were in this sector, which is the smallest in geographic size and has the lowest number of residents. The use of force rate in George is 11 times higher than the next closest sector, which is Ida. Charlie and Henry Sectors have similar rates of force among their similar residential populations (approximately 70,000-76,000). Figure 10: Use of Force per 10,000 Residents* 1711.9 32.9 42.7 133.0 51.1 99.3 58.0 138.1 155.6 *The airport has no population so APT sector is excluded from this graph. To better understand the geographic distribution of use of force within the City of Austin, a series of maps are presented in Figures 11-20 below that display the number of individuals who had force used against them during a 48-month period by APD Sector. 37 Note that a data collection error was identified regarding reported accuracy of transportation location. In 376 events (roughly 3.7% of all use of force cases), the data demonstrated discrepancies for the capture of where an individual was transported when more than one officer completed a use of force report for the same event/individual. That is, given the same use of force event, officers’ individual reports did not match for this item. To minimize this incongruence, it is imperative that APD supervisors review across multiple use of force reports for consistency and reliability. 37  Figure 11: Use of Force Incident Counts by APD Sector (Adam) Figure 13: Use of Force Incident Counts by APD Sector (Baker) Figure 12: Use of Force Incident Counts by APD Sector (Apt) Figure 14: Use of Force Incident Counts by APD Sector (Charlie) 38  Figure 15: Use of Force Incident Counts by APD Sector (David) Figure 17: Use of Force Incident Counts by APD Sector (Frank) Figure 16: Use of Force Incident Counts by APD Sector (Edward) Figure 18: Use of Force Incident Counts by APD Sector (George) 39  Figure 19: Use of Force Incident Counts by APD Sector (Henry) Figure 20: Use of Force Incident Counts by APD Sector (Ida) There could be many reasons for this geographic disparity in police use of force, the most obvious being differences in police deployment patterns, calls for service, and reported crimes that increase the relative risk that individuals residing in certain neighborhoods will come into contact with police, and ultimately have force used against them. The first step in any analysis is to describe the distribution of the data and identify any specific patterns or trends that should be considered further. To better understand the geographic distribution of use of force, a closer examination by APD Sectors will be conducted in the analyses in Section 3.3. 40  3.2 Racial/Ethnic Disparity Analyses A key focus of this study is assessing the extent to which APD uses force disproportionately across individuals’ race or ethnicity. One analytical approach is to directly compare the counts of force used against different racial/ethnic groups as a proportional analysis. However, a simple frequency analysis conducted in this manner does not account for possible differences across racial/ethnic groups in the risk of force being used against them by the police. Therefore, to interpret police use of force rates, the percent of racial/ethnic groups who experience force must be compared against an external source of data that provides an “expected” rate of force (Engel, Calnon, & Bernard, 2002; Fridell, 2004). This comparison is known as a benchmark. The most common benchmark used to compare group rates of force is the groups’ representation in population statistics (e.g., census data). The comparison group data, however, is supposed to represent similarly situated people at risk of experiencing force, assuming no bias exists (Engel & Calnon, 2004; PERF, 2021; Tillyer, Engel & Cherkauskas, 2010). For the past two decades, leading scholars and policing experts have detailed the numerous limitations of census-derived benchmarks to represent this risk (Alpert, Smith, & Dunham, 2004; Engel & Calnon, 2004; Smith, Tillyer, Engel, & Cherkauskas, 2019). Essentially, the difficulty with census-based comparisons is two-fold: First, the risk of force being used against members of any racial or ethnic group is unlikely to be calibrated directly with that group’s representation in the residential population because different racial/ethnic groups vary in both their frequency and nature of contacts with the police, along with their known or suspected involvement in criminal activity (Cesario, Johnson, & Terrill, 2019; Geller et al., 2020; Fridell, 2004; Fryer, 2019; Shjarback & Nix, 2020; Worrall, Bishopp, & Terrill, 2020). Second, aggregate-level census data do not measure the complexity and interactive nature of individuals’ interactions with the police, or the legal and extralegal characteristics that research has shown puts individuals at risk of experiencing force, particularly individuals’ legally relevant behaviors like resistance, presence of a weapon, and criminal behavior (Engel, Sobol, & Worden, 2000; Garner et al., 2002; Klahm & Tillyer, 2010; Morgan, Logan, & Olma, 2020). In short, aggregate level comparisons of coercive police outcomes, like use of force, to census population figures by racial/ethnic group are not methodologically valid as a measure of police bias and should not be relied upon to make this determination (Alpert et al., 2004; Cesario et al., 2019; Fridell, 2004; Ridgeway, 2007; Smith et al., 2019; Tregle, Nix, & Alpert, 2019). Residential population benchmarks can provide a gross estimate of the racial/ethnic disproportionality in policing outcomes within a given jurisdiction, but they cannot provide reliable information regarding the reasons for these differences, and therefore should not be used to estimate the prevalence of police bias. In this examination of APD’s use of force, we use several different benchmarks to compare against the rates of force experienced by racial/ethnic groups in their encounters with police. Our purpose in doing so is two-fold: (1) to establish an introductory examination of differences in APD use of force across racial/ethnic groups, and (2) to demonstrate the volatility and limitations of examining racial/ethnic disparities using these methods. We begin with what is likely the most flawed benchmark (i.e., residential population counts) and highlight the 41  resulting disparities in use of force. Residential population rates are based on the 2020 U. S. Census data.38 We next compare the results to other analyses using different benchmarks (described below) that many scholars and policing experts consider more valid when analyzing police use of force. Our second benchmark data is based on the population of individuals arrested by APD for criminal offenses (i.e., arrestee population). Of the benchmark data examined by Kroll, the arrestee population is likely the strongest proxy measure for estimating those who are most at risk for having force used against them (Davis et al., 2018; Garner et al., 2018; Hickman, Piquero, & Garner, 2008). An inherent problem, however, with using arrestees as a benchmark for use of force comparisons is that officers may be biased in who they arrest. This bias would go undetected because the analysis begins with arrests (based on the inherent assumption that this measure is unbiased). If officers have disproportionately over-arrested racial and ethnic minorities (due to overt or implicit bias), the use of arrest as a benchmark will underestimate the actual disparity between arrestees and rates of use of force (Cesario et al., 2019; Geller et al., 2020; Knox et al., 2020; Knox & Mummolo, 2020). Furthermore, some uses of force occur in situations that do not result in arrest, including one-third of APD’s use of force between 2017 and 2020, which suggests that this benchmark has important limitations as a proxy measure for the population at risk of having force used against them (assuming no police bias) (Fryer, 2020; Shjarback & Nix, 2020; Tregle et al., 2019). The final benchmark data used by the Kroll team is criminal suspects data, which includes persons as described by victims to the police when reporting a crime. These data were obtained from the APD records management system for the four-year period examined. The use of this benchmark data reduces the potential police bias associated with official arrest data because the police collect, but do not generate, information about the race of the suspect (Ridgeway & MacDonald, 2010; Tregle et al., 2019). Instead, community members are typically the source of information for most criminal suspects, through reported crimes and calls for service. Furthermore, the information provided by citizens when reporting crimes and disorder aligns better with the population at risk of police contact and subsequent use of force compared to residential population (Ridgeway & MacDonald, 2010). It is also possible that the public introduces biases of their own which may under- or over-report certain activities that are related to the likelihood of use of force (Klinger & Bridges, 1997; Ridgeway & MacDonald, 2010). Despite these possible shortcomings, it is likely that the criminal suspect data creates the most valid benchmark comparisons for use of force analyses (Ridgeway & MacDonald, 2010; Smith, Rojek, Tillyer, & Lloyd, 2017; Tregle et al., 2019). Note, however, that although some benchmarks are clearly better than others, none are without limitations. We address this issue by providing results from additional analyses (beyond benchmark comparisons) in Section 3.3 below. First, however, we present the benchmark results using these three data sources: (1) residential population, (2) arrestee, and (3) reported criminal suspects. These three data sources determine the weights for the denominator population for two different geographic units: (1) the city, and (2) within each APD sector. In other words, the number of individuals who had force used 38 The 2020 U.S. Census data was available for the racial and ethnic demographic breakdown after November 2021. Our initial analyses were based on the 2010 U.S. Census data (and the results were virtually identical to 2020) because while the population in Austin grew between 2020 and 2010, the rate of growth per race/ethnicity was extremely comparable. For APD police beat data (used in later analyses) we were forced to use the 2010 U.S. Census as not all measures needed for measures of disadvantage and residential instability were consistently available at the time of the report. Given the consistency between the citywide disparity analyses when relying on either 2010 or 2020 census data, we are confident the results are parsimonious regardless of time of census due to this ratio stability between the decades. 42  against them (by race/ethnicity) first across the city, and then in different police sectors are compared to the corresponding residential population, arrestee population, or criminal suspect population (by race/ethnicity). Three different databases were used to create five separate benchmarks – (1) residential population, (2) all arrestees, (3) Part 1 arrestees, (4) all criminal suspects, and (5) Part 1 criminal suspects. We produce a range of proxy measures for “at risk with contact” with the APD, setting up an opportunity to compare results across benchmarks. The analytical steps followed to create these estimates of racial/ethnic disparities are described in detail below. Step 1: Creation of Disproportionality Indices The first step is to create a Disproportionality Index (DI) for each racial/ethnic group by dividing the observed percent of uses of force by race/ethnicity by the “expected” percent of force by race/ethnicity given no police bias. Using force counts as the numerator and a benchmark (proxy for expected force) as the denominator, a “disproportionality index” is created. Disproportionality indices estimate the differences between the “actual” and “expected” rates of police outcomes for different demographic groups and are calculated as follows (Fridell, 2004): Disproportionality Index = Proportion of racial/ethnic groups observed uses of force Proportion of racial/ethnic groups expected uses of force (using benchmark proxy) Benchmark comparisons focus on comparing the numerator (proportion of use of force) to the group’s denominator (i.e., the group’s representation in the benchmark). Indices greater than 1.0 show the group has force used against them more than would be expected based on their percentage in the benchmark population; conversely, a DI of less than 1.0 indicates that a group has force used against them less often than would be expected compared to a benchmark. The larger the size of the DI, the greater the disproportion between use of force and the groups’ representation in the various benchmarks. There are several issues to note when creating and interpreting disproportionality indices. First, there is an obvious connection between the perceived validity of disproportionality indices and the type of benchmark used to make the comparison. A benchmark with a higher degree of validity will produce disproportionality indices with more validity. As described above, not all benchmarks are of equal validity. Therefore, disproportionality indices based on residential census data, for example, must be interpreted with caution and an acknowledgment of its limitations. Second, the stability of the disproportionality indices is based in part on the size of the denominator. This is especially a concern when census figures are used to estimate the expected rate of uses of force by a specific racial/ethnic group. For example, as will be shown below, in nearly half of the sectors in Austin, the residential population of Blacks and/or Hispanics is less than 30 percent. Thus, any moderate number of uses of force against Black or Hispanic individuals in these areas would significantly increase the disproportionality indices because the denominator is small to moderate in size. In other words, in areas with smaller denominators (benchmarks), the numerator has a larger influence on the resulting disproportionality index. Third, there is no accepted standard or threshold for the interpretation of the size of disproportionality indices. It is therefore difficult to determine if a given disproportionality index is “too big” or “too small.” Likewise, there 43  is no accepted “rule of thumb” used by researchers regarding the appropriate size of disproportionality indices (Farrell, McDevitt, Bailey, Andresen, & Pierce, 2004; Fridell, 2004; Geller et al., 2020). Consequently, one of the shortcomings of the disproportionality index is the difficulty in interpreting the level of disproportionality based on the method described above. Some of these shortcomings are minimized by creating a disparity ratio (described below). Step 2: Disparity Ratio Given the concerns regarding the interpretability of disproportionality indices, a better method of reporting the results is to calculate a disparity ratio. To calculate this value, the disproportionality index must be available for both the minority group population and the majority group population. Once those values are determined, the disparity ratio is calculated as follows: Disparity Ratio = Minority Group’s Disproportionality Index Majority Group’s Disproportionality Index The resulting value is the disparity ratio, which is interpreted as the likelihood of having force used against a person within that racial/ethnic group compared to the majority group. For example, if the disparity ratio is 3.0, this indicates that the group of interest (minority group) is three times more likely to have force used against them in comparison to the majority group (white, non-Hispanic). Disproportionality indices and disparity ratios are different measures. While the disproportionality index is strictly calculated using one racial group, (i.e., the percentage of Black individuals who have force used against them is divided by the percentage of Black individuals in the benchmark), the disparity ratio compares the difference between the disproportionality index of the minority group against the majority group (Fridell, 2004; Smith et al., 2019; Tillyer, Engel & Wooldredge, 2008). While the disparity ratio is often preferred to the disproportionality index (for reasons of interpretability), the validity of the benchmark remains an unresolved issue. In addition, the statistical instability of the denominator (noted above) is also not rectified, meaning that benchmarks with small percentages of minority populations may be unduly affected by small changes in the amount of force. Finally, there is no agreed-upon value that unequivocally provides a threshold for a determination of disparity. Notwithstanding these concerns, the disparity ratio is a superior measure to the disproportionality index due to its clearer interpretative value and comparison across racial/ethnic groups. Disparity ratios are generally interpreted as the percent difference relative to the majority group. For example, a disparity ratio of 1.5 is typically interpreted as 150% higher, or 1.5 times more likely for the minority group, compared to the majority group. In the analyses reported below, both disproportionality indices and disparity ratios are calculated. Using the use of force event counts as a numerator and a series of benchmarks as denominators—1) census residential population, 2) arrestee population (including all arrests, and only Part I arrests), and 3) criminal suspects (all criminal suspects and Part I criminal suspects39)—we present citywide disparity ratios in the use of force by race and ethnicity (Black and Hispanic) relative to white, non-Hispanics. We did not create disparity ratios for 39 According to the FBI Uniform Crime Reporting system, the list of Part 1 violent crimes is: murder and non-negligent manslaughter, forcible rape, robbery, and aggravated assault. 44  other racial/ethnic groups due to limited statistical power (97.4% of all uses of force in Austin involve Black, white, or Hispanic individuals).40 Disparity Ratio Findings When using census population as the denominator, the white disproportionality index is 0.68, while the Black disproportionality index is 4.6, and the Hispanic disproportionality index is 1.0. As shown in Table 10, the percent of white individuals who had force used against them was 32.4%; for Black and Hispanic individuals, the percentages were 31.4% and 33.6%, respectively. The Black disparity ratio was 6.7 (6.7 = 4.6/0.68). This means Black individuals had force used against them at a ratio that was six times greater than the white use of force compared to the white population ratio. This finding suggests that use of force for Black individuals compared to white individuals is highly disparate relative to their respective representation in the residential population. Also, for the entire city, the Hispanic disparity ratio was nearly 1.5 times higher compared to the white (non-Hispanic) use of force population ratio. In summary, when the residential population is used as a benchmark comparison to estimate risk for police use of force (i.e., a proxy measure for the expected use of force assuming no police bias), Blacks and Hispanics are substantially overrepresented in use of force reported in the City of Austin. This finding of significant racial/ethnic disparity in use of force compared to residential population has been demonstrated in previous reports examining APD data and remains in our current examination across four years of use of force data. As noted earlier, however, the residential population benchmark is fraught with several unsupported assumptions and limitations that do not withstand empirical scrutiny, and therefore these findings should not be interpreted as evidence of police bias. Comparing rates of force experienced by racial/ethnic groups to their representation in the residential census population is problematic because such comparisons fail to adequately consider legitimate differences across racial/ethnic groups in their risks of experiencing force used by the police. Next, we present analyses assessing use of force by race/ethnicity while accounting for the race/ethnicity of the APD arrestee population from 2017 to 2020.41 When using race/ethnicity of arrestees as the benchmark or denominator, compared to the actual use of force counts, we are assessing how frequently individuals of different racial/ethnic groups have force used against them relative to their representation in the arrestee population. The disproportionality index for whites when using arrest as the benchmark was 0.95 (slightly under a 1-to-1 ratio of uses of force relative to arrests). For Black individuals, the disproportionality index was 1.14 (slightly over a 1-to-1 ratio). For Hispanic individuals, the disproportionality index was 0.95 (identical to the white index). As shown below in Table 10 and Figure 21, the disparity ratio for Blacks (Black index relative to white index) drops from 6.7 (using residential census population), to 1.2 (using arrestee population). This illustrates that using a different benchmark (arrests) produces a dramatically different interpretation of racial/ethnic disparity in police use of force. When the benchmark is changed to only Part 1 (most serious) arrests, the disparities are even further reduced, with whites actually more likely to have forced used against 40 Disparity ratios are often unstable with small denominators, which led our team to focus solely on white, Black, and Hispanic individuals in the disparity ratio analysis reported below. 41 Each arrestee is person-event specific. There were 128,213 total arrests in Austin from 2017 to 2020. Each arrestee could be charged with multiple charges. We count an arrest as a person-event specific (i.e., each person-event is one arrest, regardless of how many officers are included on the arrest report or how many charges are levied against the arrestee). 45  them than Blacks and Hispanics based on comparisons to the race/ethnicity of those arrested for the most serious types of offenses. The third and final data set used by Kroll to create benchmarks is criminal suspects. Here, the race/ethnicity recorded for individuals reported as criminal suspects (by the public reporting criminal events) is used as a proxy measure to estimate individuals at risk of having force used against them. As noted previously, criminal suspect data are less likely to be influenced by potential police bias (compared to arrest data, where officers have discretion to make arrests). This measure, however, captures only information about reported crimes when the suspect is known (or seen) by the reporting party. As a result, the measure does not accurately capture the race/ethnicity of all who commit crime (e.g., burglary and other property crime suspects are often unknown to the reporting victims). This proxy measure, therefore, likely better reflects the race/ethnicity of perpetrators of violent crimes (situations where an offender is more likely to be known or at least seen by the victim). Using all crime suspects as the benchmark, use of force rates against Black and Hispanic individuals are consistent with, or less than, use of force rates for white individuals. For example, the disparity ratios based on all suspect and Part I suspect benchmarks for Hispanic individuals are 0.91 and 1.04, respectively. The disparity ratio for Blacks based on all suspects is 1.03; further, when the suspect-based benchmark is restricted to only Part I criminal suspects, Black individuals are 1.8 times less likely than white individuals to have force used against them (disparity ratio = 0.55). Figure 21 visually displays the disparity ratios for Black and Hispanic individuals based on each of the five benchmarks reported in Table 10. The red line indicates no racial/ethnic disparities detected (disparity ratio = 1.0). Bars that are above the 1.0 threshold show that Black and Hispanic individuals have more force used against them than expected (compared to the respective benchmark), while bars that fall under the red line demonstrate less force than would be expected. As shown in the graph, the disparity ratios are the highest when based on the residential census data for both Black and Hispanic individuals, but particularly for Black individuals. As illustrated, the remaining disparity ratios are all close to 1, indicating that there is little or no disparity between white individuals and Black or Hispanic individuals’ likelihood of having force used against them when different benchmarks are used. For several of the benchmarks, the disparity ratios are less than 1, indicating that white individuals are actually more likely to have force used against them compared to Black and Hispanic individuals given their representation in the arrestee and criminal suspect populations. These findings are consistent with other scholarly research that has compared use of force benchmarks to one another and show that comparisons based on population statistics nearly always show large racial/ethnic disparities in use of force, while benchmarks that better capture actual risk of experiencing force show reduced or eliminated racial/ethnic disparities (Cesario et al., 2019; Fryer, 2016; Geller et al., 2020; Ross, Kalinowski, & Barone, 2020; Smith et al., 2019; Tregle et al., 2019). 46  Use of Force (N = 9,041) % Use of Force % Population % All Arrests (N = 128,213) % All UCR Part I Arrests % All Suspects Crime Incidents % All Suspects UCR Part I Incidents 6.70 o i t a R y t i r a p s D i 7.00 6.50 6.00 5.50 5.00 4.50 4.00 3.50 3.00 2.50 2.00 1.50 1.00 0.50 0.00 Table 10: Citywide Disparity Ratios by Race with Various Benchmarks as Denominators (Total Arrests, Part I Arrests, All Crime Suspects, UCR Part I Crime Suspects) Citywide Race/Ethnicity Use of Force Distribution Percent Race/Ethnicity Disproportionality Indices Disparity Ratios White Black Hispanic White Black Hispanic Black Hispanic 2,994 2,836 3,037 32.4 31.4 33.6 -- -- -- -- 47.0 6.8 32.5 0.69 4.62 34.2 27.6 35.5 0.95 1.14 -- -- 1.03 0.95 -- -- 6.70 1.20 -- -- 1.50 0.83 25.5 34.7 38.0 1.27 0.91 0.88 0.71 0.97 33.0 29.8 34.7 0.98 1.05 0.97 1.03 0.91 21.4 38.0 38.9 1.51 0.83 0.86 0.55 1.04 Figure 21: Citywide Disparity Ratios by Race with Various Benchmarks as Denominators % of Population % of All Arrests % of All UCR Part I Arrests % of All Suspects Crime Incidents % of All Suspects UCR Part I Incidents 1.20 1.03 0.71 0.55 1.50 0.97 0.91 1.04 0.83 Black Hispanic 47  In summary, these citywide benchmark analyses demonstrate the following patterns: • Use of force against Black and Hispanic individuals is disproportionality higher (6.7 times higher for Blacks, and 1.5 times higher for Hispanics) compared to white individuals when residential census population is used as the benchmark of reference. That is, Black individuals in Austin are 6.7 times more likely to have forced used against them compared to white individuals, and Hispanics are likewise 1.5 times more likely than white individuals to have force used against them than would be expected based on their relative representation in the residential population. o As noted, however, population-based benchmarks do not measure the complexity of the interactive nature of contacts with the police, and thus rates of disparity calculated using population data may exaggerate or understate disparities. • Use of force against Black individuals show slight disparity (DR = 1.2) when using the race/ethnicity of individuals arrested (all arrests) as the benchmark. That is, there appears to be some remaining disparity in force against Black individuals compared to white individuals relative to their racial composition within the arrestee population. • Police use of force is not disparate for any racial/ethnic group when the benchmark is limited to arrestees for Part I offenses (serious crimes). • One of the better proxy measures for individuals at risk of having force used against them is the criminal suspect data, and more specifically, individuals suspected of committing Part I serious crimes (Smith et al., 2021). While not perfect, the benchmarks created using criminal suspect data do not suffer from the same known limitations of using residential population and arrestee data as proxy measures for the population at risk for police use of force. Using these benchmarks, APD use of force is not found to be disparate for Black or Hispanic individuals relative to white individuals. Disparity Ratios by Police Sectors The analyses below examine use of force trends by location, and more specifically within the ten APD sectors. Table 11 shows the counts of individuals who had force used against them by sector during the four-year study period, along with the percent by racial/ethnic groups. It is noteworthy that certain sectors had higher percentages of force by race/ethnicity than did others (e.g., Adam, Baker, and David sectors have the highest percentages of use of force against white individuals; Charlie, Edward, and George sectors have the highest percentages of use of force against Black individuals, and Frank and Henry sectors had the highest percentages of use of force against Hispanic individuals). Given the small number of individuals who had force used against them in Apt sector (n=27), this sector is excluded from further analyses due to low statistical power. In addition, 68 use of force incidents were mapped outside the APD sectors. For the benchmark analyses at the sector level that follow, we examine all arrestees and all criminal suspects because limiting the comparisons to only Part I arrestees and criminal suspects limited the statistical power based on the reduced number of cases. Therefore, at the sector level, the results using only three benchmarks are reported. 48  Table 11: Citywide Use of Force Analyses: 2017-2020 (N = Number of Unique Individual-Event Encounters) Location Total City % White N N White % Black N Black % Hispanic N Hispanic % Other N Other % Citywide 9,063 100% 2,999 32.8% 2,835 31.4% 3,027 33.8% 202 2.0% Sectors 9,022 100% 2,995 32.3% 2,819 31.3% 3,010 33.5% 202 2.2% Adam 553 6.1% 286 51.7% 135 24.4% 114 20.6% 18 3.3% Apt 27 <.3% 7 37.0% 9 33.3% 7 37.0% 1 0.6% Baker 581 6.5% 307 52.9% 133 22.9% 128 22.0% 13 2.2% Charlie 1,013 11.2% 209 20.6% 468 46.2% 318 31.4% 18 1.8% David 794 8.8% 413 52.0% 124 15.6% 239 30.1% 18 2.3% Edward 1,131 12.6% 251 22.2% 424 37.5% 441 38.9% 15 1.4% Frank 848 9.4% 278 32.8% 138 16.2% 428 50.4% 4 0.6% George 2,069 23.0% 710 34.3% 728 35.2% 566 27.4% 65 3.1% Henry 979 10.8% 249 25.4% 252 25.7% 466 47.6% 12 1.3% Ida 1,000 11.1% 252 25.2% 417 41.7% 310 31.0% 21 2.1% Benchmark 1: Residential Population When measuring disparity ratios based on residential population across the nine remaining APD sectors, each APD sector shows racial/ethnic disparities in use of force for Black individuals compared to white individuals. These are displayed in Table 12. For example, within the Baker sector, Black individuals comprise only 2.7% of the residential population but account for nearly 23% of all uses of force (disparity ratio = 10.6 when compared to the disproportionality index for white individuals). This is roughly interpreted as Black individuals in Baker sector being 10.6 times more likely than white individuals to have force used against them, when compared to their representation in the residential population. A similarly high disparity ratio was observed in George (DR = 12.5). A similar disparate pattern in use of force (although less extreme) emerges for Hispanic individuals in half of the APD sectors. However, as noted previously, disparity ratios based on disproportionality indices using residential Census population as benchmarks fail to account for the likelihood of contact with police that would lead to a greater risk of use of force. It is important to consider these analyses in comparison to other potentially more valid benchmarks to see if the observed patterns remain. 49  Table 12: Citywide and Sector-Specific Use of Force Disparity Ratios (Benchmark = Residential Population) % UoF Black % UoF Hispanic % Hispanic Pop % UoF White % Pop Black Disparity Ratio Black % White Pop Disparity Ratio Hispanic Citywide 47.0% 32.4% 6.8% 31.4% 32.5% 33.6% 6.7 1.5 Sectors Adam 51.6% 51.7% 5.2% 24.4% 18.0 20.6% 4.6 Baker 66.4% 52.9% 2.7% 22.9% 14.7% 22.0% 10.6 Charlie 34.2% 20.6% 17.5 46.2% 39.9% 31.4% David 65.3% 52.0% 2.7% 15.6% 24.7% 30.1% Edward 22.2% 22.2% 11.5% 37.5% 50.6% 38.9% Frank 36.9% 32.8% 6.0% 16.2% 50.4% 50.4% George 71.0% 34.3% 5.8% 35.2% 12.2% 27.4% 12.5 Henry 32.8% 25.4% 9.1% 25.7% 49.6% 47.6% Ida 44.9% 39.1% 8.8% 41.7% 37.8% 31.0% 1.4 1.8 1.3 1.6 0.9 1.1 4.7 1.2 1.4 4.3 6.9 3.7 3.0 3.5 8.4 Benchmark 2: Arrestees The next examination of disparity ratios uses all arrestees (by race/ethnicity and sector) as the benchmark. Using this comparison, as shown in Table 13 below, the disproportionality index for white individuals is virtually a 1 to 1 ratio, ranging from 0.8 to 1.0 across sectors. When estimating disparity ratios, three sectors have slightly elevated disparity in use of force for Black individuals compared to white individuals: Adam, Edward, and Henry sectors. In summary, the disparity ratios for Black individuals compared to white individuals (when using arrestee population as the benchmark) show relative uniformity across APD sectors, with low or no levels of racial disparities detected. Recall, however, that the arrestee benchmark does not account for possible police bias in the initial arrest decision. The use of force disparity ratios for Hispanic individuals are more variant across APD sectors when measured relative to Hispanic arrestee populations. First, eight of the ten sectors have a disparity ratio of less than 1.0 (demonstrating that use of force involving Hispanic individuals compared to their representation in the arrestee population is less than that for white individuals). Second, the remaining two sectors (George and Ida) have disparity ratios that equal 1.5, which is roughly 150% higher compared to the white individuals. In summary, the disparity for Hispanic individuals in use of force is heavily dependent on APD police sectors. Although the vast majority of APD sectors show no disparity in use of force for Hispanic individuals (compared to their 50  representation in the arrestee population and relative to white individuals), two sectors are noteworthy given their relatively high disparity ratios for Hispanic individuals involved in use of force events. Table 13: Citywide and Sector-Specific Use of Force Disparity Ratios (Benchmark = All Arrestees) % White Arrests % UoF White % Black Arrests % UoF Black % Hispanic Arrests % UoF Hispanic Disparity Ratio Black Disparity Ratio Hispanic Citywide 128,213 34.2% 32.4% 27.6% 31.4% 35.5% 33.6% 1.1 0.9 Sectors Adam 10,319 52.2% 51.7% 20.4% 24.4% 23.9% 20.6% 1.2 0.8 Baker 10,196 51.8% 52.9% 20.6% 22.9% 24.7% 22.0% 1.1 0.9 Charlie 15,457 20.0% 20.6% 44.2% 46.2% 34.9% 31.4% 1.0 0.8 David 14,236 49.3% 52.0% 13.0% 15.6% 35.9% 30.1% 1.1 0.8 Edward 17,951 23.6% 22.2% 31.4% 37.5% 43.2% 38.9% 1.2 0.9 Frank 13,870 30.7% 32.8% 14.3% 16.2% 53.7% 50.4% 1.1 0.8 George 14,691 40.6% 71.0% 35.9% 35.2% 21.7% 27.4% 1.1 1.5 Henry 14,391 26.1% 25.4% 21.8% 25.7% 51.0% 47.6% 1.2 0.9 Ida 15,505 26.3% 39.1% 39.5% 41.7% 21.7% 31.0% 1.1 1.5 Total Arrests (City %) (100%) (8.1%) (8.0%) (12.2%) (11.2%) (14.1%) (10.9%) (11.5%) (11.3%) (12.2%) Benchmark 3: Criminal Suspects the residential population benchmark As noted previously, while likely overestimates racial/ethnic disproportionality, the arrestee benchmark possibly underestimates it. Examining disparity ratios in use of force across racial/ethnic groups while using the criminal suspect population as a benchmark likely produces findings with the strongest validity. The results of the disparity ratio analysis based on all criminal suspects are described below and displayed in Table 14. Across APD sectors, the disproportionality index for white individuals (used in the disparity ratio calculations) is highly variant. For example, in Adam sector, the disproportionality index for white individuals is 1.6 because 51  white individuals make up nearly 53% of those that had force used against them but only comprise 40.2% of criminal suspects. Using suspects as the benchmark for the creation of disproportionality indices, the disparity ratios demonstrate that use of force against Black individuals was greater than or equal to 1.0 in five of the nine sectors, with the highest disparity ratio observed in the George sector (1.2). This demonstrates that minimal racial/ethnic disparities were detected in use of force for Black individuals relative to white individuals using the criminal suspects as a proxy measure for those at risk for use of force. For Hispanic individuals, the same pattern emerges, with disparity ratios equal to or less than 1.0 in seven out of the nine APD sectors analyzed. The disparity in use of force reported for Hispanic individuals relative to white individuals using criminal suspects as the benchmark is elevated in George sector, where Hispanic individuals are 1.6 times more likely to have force used against them. Table 14: Citywide and Sector-Specific Disparity Ratios (Benchmark = Criminal Offense Suspects) Total Suspects (% City) % White Suspects % UoF White % Black Suspects % UoF Black % Hispanic Suspects % UoF Hispanic Disparity Ratio Black Disparity Ratio Hispanic Citywide 350,577 33.0% 32.4% 29.8% 31.4% 34.7% 33.6% 1.0 Sectors Adam 36,608 32.8% 51.7% 22.0% 24.4% 33.2% 20.6% 0.7 0.9 0.4 Baker 32,968 40.2 52.9% 19.8% 22.9% 18.1% 22.0% 0.9 0.9 Charlie 40,068 17.6% 20.6% 39.8% 46.2% 29.2% 31.4% 1.0 0.9 David 40,453 39.8 52.0% 13.6% 15.6% 27.3% 30.1% 0.9 0.8 Edward 51,912 20.5 22.2% 34.7% 37.5% 29.4% 38.9% 1.0 1.2 Frank 41,815 28.2% 32.8% 14.5% 16.2% 42.9% 50.4% 0.9 1.0 George 25,097 37.0% 71.0% 33.0% 35.2% 18.8% 27.4% 1.2 1.6 Henry 36,292 23.3% 25.4% 21.4% 25.7% 42.7% 47.6% 1.1 1.0 Ida 39,028 23.3% 39.1% 34.4% 41.7% 27.5% 31.0% 1.1 1.0 (10.6%) (11.2%) (8.0%) (10.4%) (8.3%) (7.4%) (5.8%) (6.4%) (8.7%) 52  Summary: Disproportionality Use of Force Benchmark Analysis by Sector An examination of racial/ethnic disparity in use of force across APD sectors reveals mixed findings that are heavily dependent on the benchmark employed and the specific APD sector under examination. Nevertheless, the following consistent patterns emerge: • Disparity ratios created using residential census population as the benchmark consistently demonstrated major disparities in use of force for Black individuals (across all APD sectors) and for Hispanic individuals (in half of the APD sectors) compared to white individuals. • Disparity ratios created using arrestee population as the benchmark consistently demonstrated low or no disparities in use of force for Black and Hispanic individuals compared to white individuals. Only two APD sectors – George and Ida – showed that Hispanic individuals are 1.5 times more likely to have force used against them compared to white individuals when the arrestee population is used as the reference group. • Disparity ratios created using criminal suspects as the benchmark consistently demonstrated low or no disparities in use of force for Black and Hispanic individuals compared to white individuals across all but two APD sectors. In Edward sector, Hispanic individuals are 1.2 times more likely to have force used against them relative to white individuals, and in George sector, Black and Hispanic individuals are 1.2 and 1.6 times more likely, respectively, to have force used against them compared to white individuals (relative to their representation as criminal suspects). • Use of force disparity for Hispanic individuals has slightly more variability by benchmark type and police sector. Combined, the analyses suggest that the disparity ratios for Hispanics in George and Ida sectors (and to a lesser extent Edward) are higher than in other sectors, even where more stable benchmarks (arrests and criminal suspects) are used as the point of reference. Conclusion Although we can be more confident in the arrest and suspect benchmark comparisons that better measure risk of experiencing force compared to residential census population, disparity ratios still cannot account for other situational factors shown by decades of police research to influence police use of force, including individuals’ resistance, presence of a weapon or other threats, presence of bystanders, and individual impairment (Alpert & Dunham, 2004; Garner et al., 2002; Terrill, 2003; Terrill & Mastrofski, 2002; Tregle et al., 2019). Nevertheless, bivariate analyses (an assessment of the relationship between two variables) are an important first step in assessing and understanding patterns in racial/ethnic disparities in police use of force, as well as informing more sophisticated multivariate statistical analyses that simultaneously consider the influence of legal and extralegal factors. These additional analyses are the focus of Section 3.3 below. 3.3 Predicting Use of Force It is important to consider when and how force is used by comparing situations where force is used to similar situations where force is not used. In this manner, we can continue to explore if individuals’ race/ethnicity alone has an impact on whether officers use force, or if individuals’ race/ethnicity interact with other factors that lead to a greater or lesser likelihood that police use force during police interactions with the public. 53  We know that police interactions with individuals that result in arrest are more likely to involve the use of force compared to all other types of police contacts (Davis et al., 2018; Garner et al., 2018; Hickman et al., 2008). By examining this specific population of individuals most at risk of experiencing force (people who are arrested), we can better understand the factors that predict use of force (Fryer, 2019; Worrall et al., 2020). Therefore, in this section, we shift from describing force (Section 3.1) and examining racial/ethnic disparities in use of force (Section 3.2), to examining all arrest encounters to identify the factors that predict whether the use of force occurred during these types of encounters with the public. In this section, we first describe the total number of arrests and the percent of arrests resulting in the use of force. We compare uses of force that result in an arrest to the uses of force that do not result in arrest (instead ending with an alternative, non-arrest outcome). We then describe the key indicator variables used in the statistical analyses to delineate arrests that involve force from arrests that do not involve force. We conclude by describing the findings of three multivariate models, which simultaneously assess the impact of legal, situational, individual, and community characteristics on whether force occurred. APD Uses of Force 2017-2020 Between 2017 and 2020, 73.2% of the 9,041 known individuals who had force used against them resulted in an arrest for that encounter. Of those arrested, 86.3% were taken into custodial custody, while 4.7% were taken to the hospital, 4.0% were not transported (i.e., released / non-custodial arrest), and approximately 1% were taken to a detox center or mental facility. Of the 2,420 individuals who were not arrested, 33.7% were not transported anywhere, 33.5% were taken to a hospital, 19.6% were taken to a mental facility, 1.8% were taken to a detox center, and 0.3% were transported to the morgue.42 Figure 22 graphically displays demographic comparisons across these two groups (those who had force used against them and were arrested, compared to those who had force used against them but were not arrested). Some racial/ethnic differences are shown. A higher percentage of Blacks and Hispanics who experienced force were arrested than not arrested, while a larger percentage of whites was not arrested than arrested. In addition, there is a larger gender difference between the two groups; males comprised 77.5% of those who were arrested, but only 64% of those who were not arrested. There were slight differences in the average resistance level and force severity level as well that are not graphically displayed. Specifically, those who were not arrested displayed a slightly lower maximum resistance level at 3.52, compared to an average maximum resistance of 3.61 for those arrested. The average force severity level for those arrested was 3.15 compared to 3.45 for those not arrested. Recall that 1 is the most severe level of force, so a lower average indicates more severe force; those arrested experienced a more severe average level of force than those not arrested. 42 As noted in Section 1, there is approximately 5% missing data on the transport data field; a larger proportion of missing data exists for those who were not arrested: 7.2% missing for those not arrested compared to 4.0% for those who were arrested. Another possible inconsistency in the data was detected: 3.8% of those not arrested were transported to jail. 54  Figure 22: Demographic Comparison of Those Arrested and Not Arrested Who Experienced Force 77.5% 64.0% 32.6% 28.0% 34.4% 31.3% 37.8% 31.4% 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Black Hispanic White Male Arrested Not arrested APD Arrests with Use of Force 2017-2020 As noted in Section 3.1, the total number of arrests (for any criminal offense) declined by 50.6% from 2017 to 2020, while the number of individuals who had force used against them steadily increased for a total increase of 58.4% during the same time.43 Similarly, Figure 23 shows that the overall percentage of arrests that resulted in use of force has also steadily increased, more than doubling from 3.6% of arrests in 2017 to 7.8% of arrests in 2020. The average percent of arrests that involved use of force for this four-year period was 5.1%. In short, yearly trends with fewer arrests and more uses of force resulted in a more than doubling of the percentage of arrest encounters that result in the use of force. Despite this increase, the overwhelming majority of arrests by APD are accomplished without the use of force. The remainder of this section focuses on understanding what characteristics influence why force was used in this small percent of arrests. As such, we focus specifically on 86.3% of use of force incidents that resulted in an arrest. 43 According to APD, this may be partially explained by an agreement between APD and the U.S. Department of Justice in 2018 that impacted the reporting of use of force incidents. 55 9.0% 8.0% 7.0% 6.0% 5.0% 4.0% 3.0% 2.0% 1.0% 0.0% 12.0% 10.0% 8.0% 6.0% 4.0% 2.0% 0.0% Figure 23: Percent of Arrests that Resulted in Use of Force by Year (2017-2020) 7.8% 6.6% 3.6% 4.3% 2017 (n=42,098) 2018 (n=35,036) 2019 (n=30,280) 2020 (n=20,799) The following descriptive analyses reveal several troubling patterns regarding the use of force by APD. First, as shown in Figure 24, the percentage of arrests that resulted in the use of force increased across this four- year period for every racial/ethnic category. For example, between 2017 and 2020, the percentage of arrests that resulted in the use of force nearly doubled for Black and white individuals (90.2% and 92.9% increases, respectively), more than doubled for Hispanic individuals (104.9% increase) and increased dramatically (225%) for those categorized as “other” race/ethnicity. Figure 24: Percent of Arrests that Resulted in Use of Force by Year by Race/Ethnicity (2017-2020) 2017 2018 2019 2020 Black Hispanic White Other To further consider these patterns, we examine the data by APD Sector. As shown in Table 15 below and Figure 25, for this four-year period, the overall percentage of arrests that result in the use of force varies slightly across most sectors. George Sector, however, appears as an extreme outlier. Compared to other sectors with 56 similar numbers of arrests, the percentage of those arrests that result in the use of force within George Sector is more than double, and in some cases close to three times as large. Table 15: Arrests that Result in Force by Sector and Race/Ethnicity (2017-2020) Overall Black Hispanic White Other Sector (number of arrests) ADAM (n=10,319) APT (n=569) BAKER (n=10,191) CHARLIE (n=15,351) DAVID (n=14,196) EDWARD (n=17,929) FRANK (n=13,870) GEORGE (n=14,691) HENRY (n=14,391) IDA (n=15,488) 3.4% 5.3% 3.5% 3.9% 1.9% 3.7% 2.4% 2.2% 5.0% 3.7% 3.8% 4.9% 4.1% 4.6% 2.1% 5.0% 6.0% 5.0% 6.2% 2.6% 3.9% 5.7% 4.1% 4.9% 2.0% 4.5% 6.6% 4.7% 5.0% 2.2% 4.2% 5.8% 5.0% 5.0% 1.9% 11.3% 12.1% 15.8% 11.3% 6.9% 4.9% 6.3% 5.6% 5.0% 2.2% 4.7% 6.0% 5.6% 4.9% 2.1% 57 12.0% 10.0% 8.0% 6.0% 2.0% 0.0% Figure 25: Overall Arrests that Result in Force by Sector (2017-2020) 11.3% 4.0% 3.4% 3.7% 3.8% 3.9% 5.0% 4.5% 4.2% 4.9% 4.7% ADAM (n=10,319) APT (n=569) BAKER (n=10,191) CHARLIE (n=15,351) DAVID (n=14,196) EDWARD (n=17,929) FRANK (n=13,870) GEORGE (n=14,691) HENRY (n=14,391) IDA (n=15,488) These findings are further displayed by individuals’ race/ethnicity. As shown in Figure 26 below, when considering the three largest race/ethnicity groups, George is the only sector where Hispanic arrestees are more likely than Black and white arrestees to have force used against them. In addition, five sectors (Adam, Edwards, George, Henry and Ida) have percentages of Black arrestees that have force used against them that are 1% (or more) higher than the percentage of white arrestees who have force used against them. 58 18.00% 16.00% 14.00% 12.00% 10.00% 8.00% 6.00% 4.00% 2.00% 0.00% Figure 26: Sector Arrests that Result in Force by Race/Ethnicity (2017-2020) ADAM (n=10,319) APT (n=569) BAKER (n=10,191) CHARLIE (n=15,351) DAVID (n=14,196) EDWARD (n=17,929) FRANK (n=13,870) GEORGE (n=14,691) HENRY (n=14,391) IDA (n=15,488) Black Hispanic White Other Factors Influencing the Use of Force There are many factors that can influence whether force is used during an arrest encounter. Members of the public often focus on one factor – the individuals’ race/ethnicity. But those in policing recognize that many other factors increase the likelihood of the use of force during arrest situations. It is possible some of these factors contribute to the differences in rates of force across APD Sectors as demonstrated above. To determine if an individuals’ race/ethnicity has an impact, these other factors need to be considered. One way to consider these factors is to statistically control for them in prediction models. In essence, we examine all of the factors (that are captured on the APD arrest forms) and consider whether or not, after taking these factors into account, race/ethnicity has an effect on the likelihood that force is used. We use a statistical modeling technique called regression analysis to explore the data in this way. Regression modeling allows us to create multivariate models to predict the outcome likelihood (i.e., multiple variables are examined in a statistical model that is used to predict the likelihood that force is used during encounters, after accounting for all the measured factors that might influence officer decision-making). The multivariate models in this section demonstrate which factors (of those that are included in APD arrest data) predict whether force was used during an arrest, while holding the other factors constant. The variables that are included in these models are based on academic research that has shown their influence on police use of force and therefore should be statistically controlled to examine the variable of interest (e.g., arrestee’s race/ethnicity). We briefly summarize the scholarly literature for the impact of legal variables, situational or incident characteristics, individual characteristics, and community characteristics (measured at the police district level) on the likelihood of use of force during arrest encounters. We also describe how the variables in 59  each of these categories are measured in the current models. We then describe the limitations of APD’s arrest data and our statistical analyses. Legal variables Decades of police research have consistently shown that legally relevant variables are the strongest predictors of officers’ use of force and the severity of force they employ. Specifically, incidents that involve more serious resistance or threats toward the officer, the presence of a weapon, evidence of criminal behavior, or a foot pursuit are more likely to result in the use of force; these variables also explain the severity of force used (Engel et al., 2000; Engel & Swartz, 2014; Garner et al., 2002; Klahm & Tillyer, 2010; Kramer & Remster, 2018; Lee et al., 2014, Morgan et al., 2020; Stroshine & Brandl, 2019; Terrill & Mastrofski, 2002). The APD arrest data are somewhat limited in terms of legal characteristics that can be assessed for their potential impact on use of force; the most notable variable we cannot assess is the resistance shown to officers. Although use of force data includes information about resistance shown, APD does not collect whether resistance is shown in all arrest situations (only those that result in force). As a result, individuals’ resistance cannot be used in the statistical models as a factor to determine if force is used during an arrest. This is a major limitation, as we know from prior research and officer experience that non-compliance and resistance (verbal and physical) are the strongest factors that impact whether officers use force during encounters. With the data available, we can measure the following legally relevant characteristics of arrest events: • Type of arrest: 1 = custodial, 0 = non-custodial, where custodial indicates the person arrested was transported to jail and booked, while non-custodial arrest indicates the person was cited for a criminal offense but released with a court date. • Search occurred: 1 = search, 0 = no search44 • Weapon seized during a search: 1 = yes, 0 = no45 • Other (non-weapon)46,contraband seized during a search: 1 = yes, 0 = no47 Situational or incident characteristics Several studies have examined the impact of situational variables on the use of force, including whether the contact was proactive, whether there were other officers or bystanders present, and the location of the contact (Bolger, 2015; Engel et al., 2000; Lawton, 2007; Paoline & Terrill, 2007; Rydberg & Terrill, 2010; Terrill & 44 Roughly 49% of the non-searches were calibrated with noncustodial arrests (i.e., arrest and release arrests). However, in just over 50% of the non-searches the arrestee was taken into custody, but no indicators of a search (including subsequent to arrest) were made by the APD officer. There were not any false negatives in these counts (i.e., none of the ‘search found’ measures showed anything documented by the arresting officer(s)). 45 Within the searched field, APD only collects the ‘highest ordered item’ found. Weapon is considered the highest ordered measure on the search-recovery inventory. For this measure, we are gauging the impact of weapon recovery against the following two items: non-searches, and searches which yielded ‘nothing’ or ‘other’ (the lowest rank-ordered indicator since other gauges ‘something, but not something illegal or inked with the arrest). 46 For this measure, we are specifically measuring any contraband other than a weapon (i.e., alcohol, cash, drugs). 47 We checked for collinearity in searches, weapons recovered, and other contraband beyond a weapon. This check is necessary to determine if the variables can be included in the regression models simultaneously. None of the correlation coefficients was greater than 0.36 (correlation coefficients range from 0 = no correlation to 1 = perfectly correlated), suggesting this was not a problem. This is likely because ‘nothing illegal was recovered’ in over 80% of events. 60  Mastrofski, 2002). The findings from this research are mixed in comparison to the consistency of the effects of legal variables, but nevertheless are important to consider. Again, our analysis of these types of variables is limited by what is available in the APD data. We can measure the following situational characteristics: • Reason for the contact: traffic = 1, all other reasons = 0 • Street Type: 1 = city street, 0 = state or US highway48 • Nighttime (7 pm – 7 am): 1 = night, 0 = day; • Weekend: 1 = weekend, 0 = weekday; reference category49 Individual characteristics • Season: three dichotomous variables for Spring, Fall, and Winter, with summer as the excluded Studies of use of force typically examine the impact of individual demographic characteristics. Gender is a consistent predictor of use of force; though its effect size varies by study and how force is measured, males are generally more likely than females to have forced used against them and to experience more severe types of force given similar situations (Gau et al., 2010; Kaminski et al., 2004; Stroshine & Brandl, 2019). Individuals’ age is not as strong a predictor of force as gender but is often negatively related to use of force; that is, older individuals are less likely than younger individuals to have force used against them (Hickman et al., 2008; Terrill & Mastrofski, 2002). The impact of individuals’ race/ethnicity on whether police use force and the severity of force is more complex. Research findings are mixed. Some studies indicate weak or non-significant relationships between race/ethnicity and force (whether lethal or non-lethal) once other legal, situational, and community characteristics are controlled (Brandl & Stroshine, 2017; Engel & Swartz, 2014; Hollis & Jennings, 2018; Jennings et al., 2019; Nix et al., 2017; Smith et al., 2017; Worrall et al., 2020). Other studies, however, have found that Blacks and Hispanics are more likely than whites to have any force used against them or to experience more severe force (Alpert et al., 2004; Fridell & Lim, 2016; Fryer, 2019; Kramer & Remster, 2018; Terrill & Mastrofski, 2002; Terrill & Paoline, 2017). Others have hypothesized that racial/ethnic disparities in the use of force might be more common at lower levels of force; less severe force is often under-studied but there is some empirical support for this argument (Crawford & Burns, 2008; Kaminski et al., 2004; Lawton, 2007, Morrow et al., 2017). Unfortunately, our data do not permit an examination of different severity levels of force in order to test this within APD arrests. 48 This city street/highway designation is not based on GIS mapping by APD analysts but rather a selection criterion within the arrest reports calibrated with the incident location. 49 When there are multiple categories for a single concept (e.g., race/ethnicity = white, Black, Hispanic, and other), each category is used to create individual variables (e.g., white = yes or no, Black = yes or no, Hispanic = yes of no, and other = yes or no) and all but one of these variables are included in the regression model. The excluded variable represents the category that is used to compare to others. For example, in the case of race/ethnicity, Black, Hispanic, and other are all included in the model (3 variables) and the excluded category is white. Therefore, the effects of race/ethnicity variables reported in the models are in comparison to whites. The odds ratio represents the likelihood of a Black arrestee having force used against them compared to a white arrestee. The other dichotomous variables in the models (where there are only two categories for a concept, e.g., gender is measured as male or female) were simply compared against their opposite (e.g., male drivers compared to female drivers). 61  In our models, arrestee demographic characteristics are measured as follows: • Age: in years • Gender: 1 = male, 0 = female Community Characteristics • Race/ethnicity: three dichotomous variables for Black, Hispanic, and other50; white is the excluded reference category that each of these variables are compared to in the analyses. Many scholarly studies examining the use of force have noted that characteristics of the environment in which police encounters occur often predict the decision to use force (Lee et al., 2014; Smith, 1986, Terrill & Reisig, 2003). Research has examined the impact of economic disadvantage, racial heterogeneity, and violent crime rate, among others on police use of force. The evidence, however, is mixed and scholars have suggested that differences in how force is measured, or at what level of geographic aggregation it is measured, may contribute to these inconsistent findings (Lautenschlager & Omori, 2019; Lee et al., 2014). Furthermore, other studies have shown that people of color were more likely to have force used against them based on the neighborhood in which their encounters occurred (Lee, 2016; Terrill & Reisig, 2003). Therefore, these types of variables are important to consider to better understand their independent impact, as well as their influence on racial/ethnic differences in use of force. Taking into consideration this previous research, we analyze the APD arrest data with a series of Hierarchical Generalized Linear Models (described below). At Level 1, we consider the situational and individual factors described above. At Level 2, we include community characteristics. These variables are primarily based on a series of census measures, which were culled and aggregated to 75 APD Districts where arrests occurred.51 Specifically, to provide a more detailed contextual analysis, we include the following variables as measures of structural conditions of the various APD police beats.52 • Percent in poverty, percent unemployed, and percent Female headed households, combined into a single factor variable: disadvantage.53 • Percent aged 15-24 (higher risk group via age demographics) 54 50 Other includes Asian, Hawaiian/Pacific Islander, American Indian/Alaska Native, Middle Eastern, Unknown. 51 Roughly 1,787 arrests could not be linked to any police district due to an issue with incomplete x/y coordinates and incomplete address information. While these were included at the sector level (since this measure was captured as part of the crime report), the specific district of location for these was unclear. Furthermore, there were 84 APD beats as the starting population of neighborhoods. We excluded nine beats from the hierarchical analysis because seven of the nine were Airport districts (where census measures were not applicable, despite having crime events that could be counted to these beats). Two beats (C8-UT3 and B7UT) had no arrests within them during this period of study. 52 Data were culled from census block groups to the APD beat boundaries. Where there was an uneven boundary matching, the proportion of the census block group housed within the APD beat (and its corresponding raw count of people per category) were aggregated to the APD beat boundary. 53 The measure was obtained using principal components analysis, no rotation, and the factor loading was roughly 68.8% (intra-variable correlation with combined loading from these three distinct census measures). 54 As a series of sensitivity tests within communities we also controlled for the racial/ethnic makeup of communities to assess the impact of differential racial and ethnic group majorities on use of force likelihood within arrests and found no statistically significant effects. 62  • Percent renter occupied units to gather (as a proxy for residential instability) • Part I UCR violent crime rate55 Limitations A multivariate statistical model is one that measures the individual and independent impact on the outcome of each variable in the model, holding all other variables in the model constant. Simply put, the statistical models estimate the likelihood that force is used in similar arrest situations – and the models isolate what factors in these similar situations predict the use of force. For example, if you want to know if Blacks are more likely than whites to have force used against them during arrest situations, you must make sure you are simultaneously considering other factors (e.g., other characteristics of the person, situation, and neighborhood) that may also impact if force is used. This is how and why multivariate statistical models are used. Multivariate analysis is a strong analytical strategy, but it is not without its weaknesses. The most important concern with a multivariate model is that it can only statistically control for those variables that are measured. This is called “specification error” or the error in a statistical model due to the inability to specify all factors that might have an influence over the outcome. Therefore, although researchers can be more confident in multivariate results, they still must be interpreted with this limitation in mind. For example, with the APD data, we cannot control for the level of resistance shown to officers during an arrest situation to determine if that has an impact on the likelihood of use of force. This is particularly important if the level of resistance shown to officers (or any other unmeasured variable) varies across racial/ethnic groups. As noted above, the APD arrest data do not include measures of a number of notable predictors of use of force. In addition to individual resistance or threatening behavior (to self, officer, or public), there are not consistent measures of whether individuals attempted to flee and/or there was a foot pursuit, individuals’ impairment by drugs/alcohol, individuals’ behavioral health issues, presence of bystanders, and presence of multiple officers. The lack of a measure of individual resistance is a major weakness of these data as resistance is, across years of studies, the single strongest and most consistent predictor of the use and severity of force; controlling for resistance often diminishes or eliminates racial/ethnic differences in use of force (Bolger, 2015; Engel & Swartz, 2014; Fridell & Lim, 2016; Garner et al., 2002; Gau et al, 2010; Morgan et al., 2020; Rossler & Terrill, 2017; Stroshine & Brandl, 2019). Furthermore, the overwhelming majority of individuals who had force used against them by APD officers were perceived to be impaired by drugs/alcohol, behavioral health issues or both. Although research findings are mixed regarding the impact of types of impairment on the use of force, Morabito and colleagues (2017) found that civilians with comorbid issues were more likely to experience police use of force than individuals with a single impairment. Therefore, the lack of a measure of this factor in the arrest data is a limitation. In addition, although information related to the criminal offense/charge was available, Kroll could not create a reliable offense/charge severity scale for use in the statistical models with what was provided. Specifically, APD only collects information related to the highest charge (rather than the number and type of all charges), charge 55 The crime rate for each APD beat was calculated by taking the total counts for each beat from 2017 to 2020 and dividing by four years to smooth the average risk of violent crime incidents per beat. The average crime count was divided by the population of the APD beats and multiplied by 1,000 residents (since roughly 54% of all APD beats studied here had fewer than 10,000 residents based on the 2010 census measures). As a sensitivity test, we also created a total crime rate average and the results mirror those presented herein. The point estimates were greater in magnitude for violent crimes than total crimes when predicting use of force. 63  severity is based on the highest possible rather than actual categorization for the offense, and the provided offense severity table that charges are linked to did not include all charges. Finally, although APD provided Kroll with officer demographic characteristics, this data could not be included in the statistical models. This is because a single officer prepares the arrest report, and this may or may not be the officer who made the decision to arrest or the officer who used force. Therefore, we cannot determine the variation across officers’ race, gender, experience, assignment, or training as it relates to their likelihood of use of force. In sum, our inability to fully consider these legal, situational, and individual variables based on the available data fields in APD arrest data limits our ability to explain APD decisions to use force. These limitations should be considered when interpreting the findings. It may be that these excluded variables vary across racial/ethnic groups, and that their inclusion in the statistical models would increase or lessen the predicted impact of individuals’ race/ethnicity on officers’ use of force. What the analyses below will provide, however, is additional evidence regarding the possible impact of individuals’ race/ethnicity on use of force given an arrest situation. Multivariate analyses are a stronger indicator of the patterns and trends related to police use of force compared to benchmark analyses and provide another piece of the puzzle to identify and understand reported racial/ethnic disparities in force. Descriptive Analyses From January 1, 2017 to December 31, 2020, there were 128,213 arrests within APD’s ten police sectors.56 Among these arrests, 6,621 use of force events (individual-incident specific) occurred. Our analyses, however, are based on 126,072 arrests for which we have complete data.57 Table 16 shows descriptive statistics for the dependent and independent variables in the multivariate models that follow. The dependent variable of interest is whether arrest resulted in use of force, which occurred in 5.1% of the arrests between 2017 and 2020. Roughly 74.3% of all arrests were custodial in nature. In 6.4% of arrests, a weapon was seized during a search, while 12.3% of the arrests involved the seizure of some other type of contraband (e.g., drugs, alcohol, suspected criminal proceeds, etc.). In 19% of the arrests, the reason for the police contact was a traffic stop. Additionally, roughly 84.8% of arrest locations were considered city streets (while the remaining 15.2% occurred on state and US Highways). The total arrests declined year-after-year with the largest percent (33%) of the four years of arrests occurring in 2017, while just 16% of all arrests during the study time-period occurred in 2020. The distribution of arrests was steady across the seasonal periods (between 23% to 27% of the total per season). The majority (73%) of arrests took place on weekdays (Mon-Thurs) while weekends (Friday-Sunday) accounted for roughly 27% of all arrests. Over half (55%) of arrests took place at night (7pm-7am). Most arrestees were male (76%); the 56 Arrests are individual-incident-date specific; that is, each individual on each date of arrest represents a single arrest, regardless of how many charges were filed against the individual on a given incident/date of arrest. 57 In Model 1, the total n declines by 365 arrests (and 9 uses of force, for a new n of 6,612 uses of force) because of the missing data on year of birth (and thus the calculation of the age at offense) among arrestees. In Models 2 and 3, we lose another 1,776 cases (and 65 uses of force, for a new n of 6,547 uses of force) because arrests either did not map to any police beat (roughly 4% of the missing cases) or they matched to locations within Austin where census data did not readily cull to the units of analysis (roughly 60% were in Adam Sector A1 UT, another 30% in the Airport areas, and the remaining 5% evenly distributed among the remaining unmatched locations). Importantly for the analysis, the range of the proportion of use of forces per arrests was between 5.2% to 5.5% across the entire distribution. 64  Dependent Variable Use of Force Independent Variables Legal Characteristics Custodial arrest Weapon(s) seized Other contraband seized Incident Characteristics Traffic stop City street 2017 2018 2019 2020 Winter Spring Summer Fall Weekend Nighttime Arrestee Characteristics Male Age White Black Hispanic Other racial/ethnic makeup of arrestees was 34% white, 28% Black, 37% Hispanic, and less than 1% were classified as a different/race ethnicity than Black, white, or Hispanic. The average age of arrestees was 33.4 years old. The average measure of disadvantage was slightly higher (0.07) than the average disadvantage measure for the entire City of Austin (0.00). This is unsurprising because we are using arrest data, and arrests occurred in just slightly more disadvantaged communities than the average Austin neighborhood. Additionally, the average violent crime rate was also above the average (i.e., 8.7 violent crimes per 10,000 residents) due to the influence of the extremely high violent communities (209 violent crimes per 10,000 residents) in the sample. Thus, we should not interpret these findings to be generalizable to all Austin neighborhoods (though virtually all neighborhoods in Austin are represented in this study) but rather a tilt toward more distressed and violent communities within the city. Table 16: Descriptive Statistics for Population of Arrests (n = 126,072)* 2017-2020 % (Mean) 5.5% 74.3% 6.4% 12.3% 28.2% 84.7% 33.1% 27.2% 24.2% 16.4% 25.3% 27.5% 25.1% 23.1% 27.0% 55.4% 75.9% (33.4) 33.9% 27.7% 36.6% 1.6% SD .22 .44 .24 .33 .45 .35 .47 .44 .42 .36 .43 .44 .43 .42 .44 .49 .43 10.07 .47 .44 .48 .07 Min 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 13 0 0 0 0 Max 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 95 1 1 1 1 65  Community Characteristics (for 2-Level Models) Disadvantage Residential Mobility Population Ages 15-24 Average Violent Crime Rate (per 1,000) (.07) (.22) (.13) (8.70) 0.96 .07 .10 23.92 -1.43 .08 .05 .00 2.53 .47 .96 209.58 *The final n across the models varies from 127,848 to 126,072 (and the means and standard deviations are identical for each of these measures (up to the hundredths of a decimal). Multivariate Analyses As noted previously, multivariate analysis is a statistical method that simultaneously accounts for multiple factors’ impact on an outcome to understand the individual impact of each variable on outcome. Therefore, the findings for these analyses represent the independent effect of each variable on the likelihood of use of force, while holding constant all other measured variables. The appropriate modeling technique for a dichotomous outcome (e.g., whether force is used) is logistic regression, as the outcome is binary. To understand the impact that neighborhood level variables might have on the use of force, we extend the analyses to include Hierarchical Growth Linear Modeling (HGLM), which is a special type of multivariate modeling required for data reflecting more than one level of aggregation. Regular multivariate analyses are based on one level of data and reflect a one-to-one ratio between variables at that level. That is, variables in most data are independent of other variables. Our data, however, do not conform to this rule because we are dealing with arrests and uses of force that occur within and across neighborhoods. Thus, the shared characteristics between events within neighborhoods are not independent of one another; arrests and use of force within the same neighborhoods share those neighborhood characteristics. This was a common problem in educational research when trying to assess the achievements of children in school independent of school structures (i.e., kids from the same classrooms share the same teacher characteristics; and kids from the same schools share the same school characteristics). HLM analysis was specifically designed to handle this child/teacher/school problem and has been extended over the past thirty years to tackle similar issues in criminal justice (i.e., the impact of neighborhood characteristics on crime). HGLM analysis partitions all level-1 measures and level-2 measures independently (so, using our school example, we can assess the impact of children’s attributes on their achievement independent of their shared teacher and shared school characteristics when comparing many kids from many different schools). This 66  partitioning of levels avoids a violation of statistical independence given that many arrests occurred within the same police districts.58 Interpreting the Models There are two components of the statistical models presented that are of note. The analyses produce information about the strength of an observed relationship in two related values: 1) the coefficient, or predicted log-odds, and 2) the odds ratio for each independent variable in the model. The coefficient represents an additive expression of a particular variable. If the coefficient is accompanied by a negative sign, this indicates the direction of the relationship is negative, i.e., the influence of the variables means the outcome is less likely. If the coefficient has no sign (i.e., is a positive number), this indicates that the influence of that variable is positive, and the outcome is more likely. For example, a positive number for the “city street” variable’s coefficient would indicate that arrests on city streets were more likely to result in force than arrests on state or US highways, while a negative sign would indicate they are less likely. In logistic regression and HGLM, the results are presented as “odds ratios” which represent the strength of association between two events.59 Similar to the relative risk ratios reported in Section 2 (i.e., the disparity ratios), odds ratios are interpreted as a change in the likelihood of force occurring because of a specific variable present in the encounter. Odds ratios are > 1.0 when the coefficient is a positive number and are less than 1.0 when the sign on the coefficient is negative. The amount of influence of a particular variable, or the strength of its relationship with the dependent variable (represented by the size of the odds ratio) is one of the most important considerations; depending on the effect size, even if the relationship between variables is statistically significant, it may not be substantively important. A rule of thumb for interpreting odds ratios is that 1.5 or lower is small; 1.5 to 2.5 is medium; 2.6 or greater is large (Chen, Cohen, & Chen, 2010). Finally, sometimes group differences may exist, but they are not statistically significant. This means that we cannot be confident that the difference is not due to chance. A threshold of statistical significance is identified with a p-value. The social sciences traditionally rely upon a confidence level of 95% (indicating statistical significance – that the finding is 5% or less due to random chance and/or sampling error). Within Table 17, when a variable’s coefficient and resulting odds ratio reach statistical significance it will be denoted with an 58 Using data at two or more levels of aggregation introduces a statistical dilemma where regression residuals for the level 1 cases (arrests) within the same level 2 units (APD police districts) may be correlated (i.e., are more similar than level 1 cases taken from independent districts). This violates the assumption of independence that underlies most ordinary regression techniques. The implications of violating this assumption are substantial, as dependence can lead to inefficient estimates and biased test statistics, making the analyses appear to have more power than they do (Raudenbush & Bryk, 2002). Hierarchical Linear Modeling (HLM) is a modeling procedure that can overcome this statistical dilemma (Raudenbush & Bryk, 2002). HLM includes an extra error term, Ui, which reflects the extra variation common to all level 1 cases within the level 2 unit, so the level 1 error term (Rij) can be independent. That is, HLM explicitly models the dependence of the residuals through this error term. For binary outcome variables like the ones utilized here, hierarchical models cannot use the standard level 1 model which assumes a linear model and normally distributed errors at level 1, once the additional error term is included (Raudenbush & Bryk, 2002). To account for these characteristics of this type of dependent variable, we employ a nonlinear form of hierarchical modeling that uses a binomial sampling model with a Bernoulli distribution, as opposed to a normal sampling model, and a logit link instead of an identity link (Guo & Zhao, 2000; Raudenbush & Bryk, 2002). 59 Technically, this odds ratio is a form of log-odds, but the interpretation of this value is not intuitively straightforward; therefore, this type of coefficient is usually exponentiated to allow for interpretation in terms of odds (Liao, 1994). The odds ratio represents this antilog transformation of the coefficient into the multiplicative odds of the outcome variable based on the predictor variable, all being equal. 67  asterisk (*). In the following description, we only discuss odds ratios for estimates that reach the threshold of statistical significance. Results Table 17 displays the multivariate regression results predicting use of force during APD arrests in three models: 1) the base model of level 1 only variables (i.e., just arrest characteristics), 2) the bi-level model with level 1 and 2 variables (including community characteristics), and 3) the bi-level model with a cross-level interaction term included. We describe the results of each model below. First, Model 1 (where no community characteristics are accounted for) demonstrates that the largest effect size (odds ratios) were consistently observed for the legal dimensions of the arrest. Specifically, custodial arrests were more likely to involve force than non-custodial arrests. When a person is taken into custody (versus those who are cited and released), force is 4.4 times more likely to be used. This is likely due to several factors (severity of offense, circumstances of the encounter, receptivity of the arrestee, etc.). Unfortunately, as we noted above, these factors cannot be reliably measured with the APD arrest data. Another legally relevant variable that predicts the likelihood of the use of force is whether a search was conducted by an arresting police officer (odds ratio = 1.65); that is, arrests that involved a search were 1.65 times more likely to result in a use of force compared to non-searches. Additionally, if a weapon was seized during a search, the odds of force within arrests are 1.4 times more likely, suggesting that the presence of a weapon is likely calibrated with arrestee resistance and perceived officer threat. Beyond legal characteristics, there are many incident characteristics that influenced whether use of force occurred during arrests. Traffic stops were negatively associated with force, meaning that motor vehicle stops (when compared to non-vehicle stops) resulting in arrest were less likely to involve force (odds ratio = 0.28, or 72% less likely to involve force). Arrests that occurred on city streets (compared to state and US Highways) were more likely to involve force (odds ratio = 1.11). Uses of force were higher in 2018, 2019, and 2020 when compared to 2017 (suggesting an increase in the probability of force within arrests over time). The seasonal periods in Austin (fall, winter, and spring) had higher uses of force across arrests than did the summer period (the reference category). Weekend arrests also had a higher probability of force (odds ratio = 1.25) than weekday arrests. Nighttime arrests (versus daytime arrests) had no bearing on use of force. When examining the likelihood of force without accounting for community characteristics, and net of legal and incident characteristics of arrests, we find older arrestees were less likely to be involved in arrests where force was used (odds ratio = 0.97). There were no significant gender differences in the likelihood of force within arrests. This is an unusual finding in comparison to previous research, but we note that our analyses in Section 3.1 revealed that males and females overall displayed very similar levels of resistance during use of force events. Furthermore, among the 9,041 individuals who had force used against them overall, a significantly larger percentage of males were taken into custody than females, while females were more likely to be taken to the hospital or a mental facility. This indicates that females who are arrested and taken into custody after force might not vary from male arrestees as much as females who were transported elsewhere. In terms of race/ethnicity, two findings emerged in Model 1. First, Blacks who were arrested had a higher likelihood of force being used against them during arrests (odds ratio = 1.18) compared to white arrestees (the reference category). Additionally, Hispanics arrestees were less likely to have force used against them in arrests than were white arrestees (odds ratio = 0.92). The effect sizes for race/ethnicity, however, were small for Black arrestees and extremely small for Hispanic arrestees. 68  Model 2 is a two-level hierarchical regression model that adds community characteristics measured at the police beat level to the legal, incident, and arrestee characteristics measured at Level 1. The most salient community-level predictor of use of force across arrests was the average Part I violent crime rate of the beat where the arrest occurred. When arrests took place in communities with higher violent crime rates, they were more likely to result in the use of force. Residential instability (i.e., percent renters) also had a positive association with use of force (i.e., arrests that took place in communities with a higher likelihood of residential turnover were more likely to involve force). Neither concentrated disadvantage nor the percentage of the population aged 15 to 24 significantly predicted use of force across arrests. This means that the likelihood of force across arrests was the same between wealthier and less wealthy communities and that use of force was the same regardless of the percent of the population in the higher risk age population.60 The magnitude and direction of level-1 predictors in the multi-level analysis were consistent with the Model 1 findings. The clear importance of legal variables (custodial arrest, weapons seized during the arrest) and incident characteristics (traffic stops, time of day, and seasonal measures) to predict force across arrests were robust when accounting for varying community contexts. The influence of the race/ethnicity of arrestee, however, became statistically insignificant for Hispanic arrestees, suggesting that white and Hispanic arrestees were equally likely to be involved in use of force incidents when accounting for legal, incident, and community effects. The coefficient related to Black arrestees retained statistical significance in these findings, but the effect size was cut by more than half between Model 1 (odds ratio = 1.17) and Model 2 (odds ratio = 1.07). This finding shows that Black arrestees were still at slightly higher risk for use of force than white arrestees, net of other important incident and community level factors, but that the effect size of the difference was miniscule. Table 17: Multivariate Regressions Predicting Use of Force within APD Arrests Logistic Regression (Model 1) and HGLM (Models 2&3) Model 1 (N = 127,848) Model 2 (N = 126,072) Model 3 (N = 126,072) B (SE) B (SE) B (SE) Independent Variables Intercept Legal Characteristics Custodial arrest Weapon(s) seized Other contraband seized Incident Characteristics Traffic stop City street 2018 2019 2020 Winter Spring Fall -4.14* (.08) 1.65 (.05) .36 (.04) -.02 (.04) -1.24 (.04) .34 (.04) .15 (.03) .52 (.03) .47 (.03) .07 (.03) .05 (.03) .09 (.03) Odds Ratio .-- 5.28* 1.43* -- 0.28* 1.43* 1.16* 1.69* 1.61* 1.07* 1.07* 1.10* -4.35* (.11) 1.68 (.05) .38 (.04) -.05 (.04) -1.15 (.04) .14 (.04) .19 (.03) .57 (.03) .55 (.04) .04 (.03) .02 (.03) .08 (.03) Odds Ratio .-- 5.36* 1.46* -- 0.31* 1.15* 1.21* 1.76* 1.73* -- -- 1.09* Odds Ratio .-- 5.31* 1.44* 0.90* 0.31* 1.12* 1.20* 1.74* 1.71* -- -- 1.09* -4.38* (.11) 1.67 (.05) .37 (.04) -.06 (.04) -1.15 (.04) .13 (.04) .18 (.03) .55 (.03) .54 (.03) .04 (.03) .02 (.03) .08 (.03) 60 A key issue that we focused on addressing was whether the measure of disadvantage predicted force/non-force across arrests absent other community variables. At no point did disadvantage predict force in any models here (with or without other covariates included at Level-2). 69  Weekend Nighttime Arrestee Characteristics Male Age Black Hispanic Other Community Characteristics Economic Disadvantage Residential Mobility Population Ages 15-24 Average Violent Crime Rate (per 1,000) Cross-Level Interactions Violent Crime Rate*Black .27 (.02) -.01 (.01) .04 (.03) -.02 (.00) .16 (.03) -.08 (.03) -.17 (.11) -- -- -- -- -- 1.32* -- -- 0.97* 1.18* 0.92* -- -- -- -- -- -- .23 (.02) -.00 (.00) .02 (.03) -.02 (.00) .07 (.03) -.04 (.03) -.15 (.11) .05 (.03) .01 (.00) -.67 (.41) .06 (.01) 1.25* -- -- 0.97* 1.07* -- -- -- 1.01* -- 1.07* .22 (.02) -.00 (.00) .02 (.03) -.02 (.00) .07 (.03) -.04 (.03) -.14 (.11) .03 (.03) .01 (.00) -.66 (.41) .07 (.01) 1.25* -- -- 0.97* 1.07* -- -- -- 1.01* -- 1.07* -- -- -.001* (.000) -- *p < .05; only statistically significant odds ratios are presented (for parsimony) Figure 27 below shows the predicted probability of use of force difference between Black arrestees and white arrestees, net of all other measures in the models. The independent probability for white arrestees of use of force events net of other measures is 3.5%, while for Black arrestees the probability of use of force is 3.7%. Figure 27: Predicted Probability of Force Across Arrests Between Black and White Arrestees, Net of All Multivariate Variables 5.0% 4.5% 4.0% 3.5% 3.0% 2.5% 2.0% 3.7% 3.5% Black Arrestee White Arrestee Model 3 includes the same variables as Model 2 but adds the examination of a cross level interaction between community violent crime rates (a robust level 2 predictor of use of force across arrests) and Black arrestees (compared to white arrestees), net of all other measures. The model demonstrated a statistically significant cross level interaction between these measures. We plotted the varying values within the interaction term in Figure 28, which can best be summarized as follows: Black arrestees arrested in low violent crime communities were more likely than white arrestees in these communities to have force used against them (3.5% versus 70  2.9%). However, once the violent crime rate was average or higher, the likelihood of force encounters within arrests was virtually identical between Black (3.6%) and white arrestees (3.4%). Thus, the findings show that the slight difference in the likelihood of Black and white arrestees in having force used against them is driven heavily by arrests that occurred in low violent crime communities.61 Other research has found similar results. For example, Fridell and Lim (2016) noted that Black individuals in areas with low violent crime rates were more likely to experience the most severe type of less lethal force compared to less severe force, but in areas with high violent crime rates there were no racial differences in force severity used (see also Correll et al., 2011). Figure 28: Predicted Probability of Force Across Arrests Between Black and white Arrestees in the Bottom 25% of Violent Crime Communities, Net of All Multivariate Variables 5.0% 4.0% 3.0% 2.0% 1.0% 3.5% 2.9% Black Arrestees in Low Violent Crime Communities White Arrestees in Low Violent Crime Communities Summary The purpose of this section (3.3) was to understand the different types of factors that predict whether an arrest involves the use of force. We did so by estimating logistic regression and HGLM multilevel models that simultaneously considered the impact of legal factors, incident characteristics, arrestees’ demographic characteristics, and the neighborhood contexts where arrests (and uses of force within arrests) occurred. As described previously, the quality and content of the APD arrest data limited the precision of our statistical models, and by extension, the findings of those models. Nevertheless, our analyses show four important trends: 1) The most robust, salient, and powerful predictors of the use of force within arrests are legal and incident characteristics. Specifically, custodial arrests, weapons seized from arrestees, and weekend arrests were those most likely to involve force. 61 As a series of sensitivity tests, we conducted multiple cross level interactions between residential instability (given its importance in Models 2 and 3) and race of arrestees (in particular Black arrestees when compared to white arrestees), and no model yielded any statistically significant findings. This suggests the impact of residential instability is a level-2 only effect and does not interact with demographic level-1 factors. Additionally, we replaced crime rates with the residential racial/ethnic makeup (i.e., percent white, percent Hispanic, and percent Black, independently) in a series of models to assess whether Black arrestees arrested in white neighborhoods were more likely to have force used on them, and/or vice versa. None of the models yielded any significant results, suggesting the importance of community violent crime rate does not intersect in any meaningful way with the communities’ racial/ethnic makeup. 71  2) Even controlling for legal and incident characteristics, there were still some statistically significant demographic differences in the likelihood of force, although these differences were inconsistent and smaller in magnitude than other factors: a. b. In the model without community characteristics, Black arrestees were slightly more likely to have force used against them than white arrestees, and Hispanic arrestees were slightly less likely to have force used against them than white arrestees. In both cases, the effect sizes were small. In the models where community characteristics are also considered, the violent crime rate and residential instability within the beat were significant predictors of use of force in arrest encounters. When the community characteristics are considered, the likelihood of force for Hispanics did not significantly differ from whites. For Blacks, the increased likelihood of force remained statistically significant, but the effect size diminished by more than half, moving from small (when not accounting for community effects) to minuscule (in more fully specified models). Therefore, Blacks remain slightly more likely than whites to have force used against them during arrests. 3) Violent crime rates within different communities were a strong predictor of use of force within arrests. Additionally, concentrated disadvantage had no bearing on use of force (i.e., force was equally likely in arrest scenarios within wealthy vs. less wealthy communities). 4) Blacks arrested in communities with low violent crime rates were slightly more likely to have force used against them when compared to white arrestees. Though statistically significant, this effect size was small. In communities with higher violent crime rates, there were no racial/ethnic differences in the likelihood of use of force. These findings underscore the importance of examining police use of force with multilevel statistical models, when possible, to explore the overall impact of community characteristics and any potential cross-level interactions between individual or situational characteristics and neighborhood measures. Data Limitations The statistical models presented and described herein are as complete as possible with the limitations of the data APD provided. Given the inability to measure known predictors of use of force, further information and analysis is necessary to provide a more complete examination of what independent impact, if any, individuals’ race/ethnicity has on police decisions to use force. To better understand officer decision-making during arrest encounters, APD would benefit greatly from gathering more detailed information within their arrest reports. Given the clear importance for predicting use of force of the legal and situational factors we were able to obtain, collecting and analyzing additional legal, situational, and arrestee characteristics would help to unravel the complex relationships between race/ethnicity and these other factors. Of particular importance to include: systematic collection of arrestee resistance or compliance, the presence of a weapon regardless of whether a search occurs, and individual impairment (by drugs, alcohol, or mental health issues). Furthermore, the current data collection for charges and offense severity is problematic and needs to be revised. Section 7 provides specific recommendations for how APD can improve their overall arrest and use of force data collection. 72  4. REVIEW AND ANALYSIS OF APD USE OF FORCE (June to November 2019) Kroll’s aggregate use of force analysis in Section 3 provided a detailed and comprehensive analysis of APD use-of-force incidents that were documented from January 2017 to December 2020. This aggregate analysis expanded upon Kroll’s contracted scope of work, which had initially called for examining use of force incidents and trends from June to November 2019. The extension from a six-month period to a four-year period of the base analyses allows for the examination of patterns and trends over time. Nevertheless, much can be gained from conducting an in-depth analysis of a smaller time frame. Using a mixed methods approach – where quantitative statistical analyses are conducted on four years of data, and qualitative analyses are conducted on six months of the same data – this report provides both trend analyses and detailed individual use of force case analyses. 4.1 Incident and Demographic Comparisons It is important to note that the six-month time period selected for the individual case analysis – June to November 2019 – did not vary significantly in its incident or geographic characteristics from the larger four-year study period (January 2017 to December 2020). As shown in Table 1 below, T-test comparisons of means analyses (comparing the four-year and six-month samples) show few statistically significant differences in the characteristics of use of force incidents between the two time periods. This means that, for purposes of our analyses, there was little that was different or unique about the smaller number of cases selected for in-depth examination. The only two exceptions are (1) severity of force and (2) gender of the individual. Specifically, in the six-month period identified for qualitative review, the cases were marginally less severe forms of force, and more likely to involve women. These two statistically significant differences, however, had extremely small effect sizes. All other measures (geographic contexts, racial/ethnic composition of individuals who had force used against them, and the incident characteristics of the uses of force) show no significant differences for this sample of cases. Additional analyses examined the differences in the two samples across APD Sectors. Again, the six-month sample used for more in-depth analyses reported within this section are statistically similar to the full four years of data across APD Sectors, with only two exceptions. In both Edward and Frank Sectors, a smaller number of males were included in the six-month sample compared to the full four-year data. These marginal differences should result in no noticeable impact on the qualitative analyses presented within this section. Table 1: Incident and Demographic Comparisons for Use of Force Analyses, June-November 2019 vs. Jan 2017 – Dec 2020 (T Test Results) Measure Severity Level of Force Multiple Uses of Force Weekend 6-MONTH Sample June – November 2019 Mean 3.36* .29 0.36 N Included 1421 1376 1437 4-YEAR Sample January 2017 – December 2020 N Included 8602 7665 8706 Mean 3.19 .30 0.40 73  Individual Demographics White Individual Black Individual Hispanic Individual Male Individual Age of Individual Officer Demographics White Only Officer(s) Black Only Officer(s) Hispanic Only Officer(s) Other Race Only Officer(s) Mixed Race Officer(s) * p < .05; ** p < .01; N Included = Data was available for these measures (note: some racial, ethnic, or age measures are sometimes missing) .33 .31 .33 .74 31.13 2.01 1.51 1.59 1.49 1.81 .32 .31 .35 .71* 31.15 1.69 1.20 1.25 1.17 1.53 1376 1376 1376 1376 1375 1437 1437 1437 1437 1437 7679 7679 7679 7679 7650 8706 8706 8706 8706 8706 Identification of Problematic Incidents As documented within this section, Kroll qualitatively reviewed and evaluated all APD incident reports and accompanying videos (body camera and dashboard) involving use of force from June 1, 2019 through November 30, 2019. The purpose of this review was to provide additional information – beyond aggregate quantitative analyses – for individual cases. Specifically, Kroll’s review of these cases was designed to determine whether APD’s use of force incidents over this six-month time frame were appropriate and reported accurately or warranted further supervisory action. Kroll reviewed each incident by examining the APD incident reports containing the officers’ description of events, any resulting charges, and whether supervisory review or approval was indicated. Kroll also examined relevant body camera or dashboard camera video footage to independently assess whether the force used was necessary under the circumstances. During the six-month period under examination, Kroll reviewed 1,321 use of force incidents.62 Within these 1,321 incidents, the vast majority (91.5%) were independently confirmed by Kroll to have been appropriate uses of force. However, the Kroll team identified 112 problematic incidents (8.5%), involving 141 individuals who had force used against them that raised a variety of concerns.63 An incident was labelled as “problematic” if the incident had a use of force of any individual that was either inappropriate or caused by an unnecessary escalation of the encounter by APD officers, or if there was some other type of APD policy violation. We examined any differences in the 141 individuals that had force used against them in the 112 incidents labeled as problematic, as compared to individuals involved in the remaining 1,209 incidents (91.5%) that were not flagged as problematic during our review. The results of these comparisons are presented in Table 2 below. Again, using independent T-test sample comparisons, the results demonstrate that individuals involved in 62 Kroll’s review consisted of the following number of use of force incidents at each Level: Level 1 (7), Level 2 (78), Level 3 (671), and Level 4 (565). Kroll determined 203 incidents provided in the RMS (Records Management System) data provided by APD did not involve the use of force (many were Department motor vehicle collisions) and were thus not included as part of Kroll’s review. 63 As noted in Section 3, multiple individuals may have force used against them in a single incident. While the statistical analyses conducted and reported in Section 3 are based on the number of individuals who had force used against them, the qualitative analyses presented in this Section are based on our review of specific cases (or incidents) that often involve multiple individuals. 74  problematic cases were no more or less likely to be of a particular race/ethnicity, age, or gender. The problematic incidents identified, however, were significantly more likely to involve the use of more severe types of force, more likely to have a single use of force during the incident, and more likely to occur during the weekend. Table 2: Incident and Demographic Comparisons for Use of Force Analyses, Problematic vs. Non- Problematic Cases within 6-month Sample June-November 2019 (T Test Results) Measure Severity Level of Force Multiple Uses of Force Weekend Individual Demographics White Individual Black Individual Hispanic Individual Male Individual Age of Individual Individuals Involved in Problematic Incidents Mean 3.04* 0.21* 0.45* 0.26 0.29 0.41 0.75 31.57 N Included 140 122 141 122 122 122 122 122 Individuals Involved in Non-problematic Incidents Mean 3.39 0.30 0.35 0.32 0.31 0.34 0.71 31.11 N Included 1281 1254 1296 1254 1254 1254 1254 1253 * p < .05; ** p < .01; N Included = Data was available for these measures (note: some racial, ethnic, or age measures are sometimes missing) Importantly, in all of the incidents identified as problematic by Kroll based upon inappropriate uses of force, the documentation shows that APD supervisors were notified of the use of force and approved any resulting arrests. Kroll’s review of the 112 problematic cases noted several areas of concern that demonstrated some troubling trends and patterns that repeatedly appeared within these incidents. We summarize these issues below: 4.2 Lack of Reasonable Suspicion and APD Use of Force Within the 8.5% (n=112) of incidents Kroll determined were problematic, APD officers escalated the use of force unnecessarily in 82 incidents (involving 88 subjects). The racial/ethnic breakdown of the 88 individuals is as follows: Black – 28.4%; White – 21.6%; Hispanic – 47.7%; Asian/Other – 2.3%. Over 30% of the incidents involved females. Kroll also identified that, in 18 of these incidents “reasonable suspicion” was not properly articulated in accordance with APD General Order 306,64 and the resulting use of force was not necessary or appropriate. Since December 2017, APD Response to Resistance policy65 has required officers to de-escalate potential force encounters. During Kroll’s review of the problematic use of force incidents that occurred during the six-month period examined, we found a trend of APD officers failing to de-escalate encounters contrary to the provisions of APD policy. Enforcing APD policy is the responsibility of the Department’s chain of command. APD commanders and supervisors are responsible for setting a tone and example of acceptable conduct for subordinate personnel. If a commander/supervisor has knowledge of misconduct on the part of subordinate personnel and fails to act, 64 APD General Order 306, Search, Section 306.2, (f), Protective frisk based on reasonable suspicion. 65 APD General Order 202, Response to Resistance, Section 200.2, effective December 13, 2017. 75  that commander/supervisor must be accountable and held to a higher standard of behavior. When commanders/supervisors fail to hold personnel under their command accountable for policy violations, it is inevitable that systemic issues will arise from unacceptable behavior. Overall, Kroll found that officers’ actions were appropriate in the vast majority of cases, and we saw many examples of fine, proactive police work, successful de-escalation tactics, and officers making legal arrests – both in response to dispatched calls and as a result of self-initiated activity. In a number of the problematic cases, however, Kroll noticed APD officers employing aggressive stop-and-frisk activity. Within these cases, officers were quick to go “hands on” (use their hands) to physically restrain the individuals they decided to detain. These officers frequently presumed – often, with no basis – that whomever they were contacting was armed. In many situations, the individuals turned out to be unarmed. In many of the problematic reports Kroll reviewed, officers gave no information to support their presumptions that the individuals they detained may have been armed, other than citing their own training and law enforcement experience, the alleged placement (near the waist area) of an individual’s hand, and/or the type of offense that an individual had committed. In several of the problematic reports identified, the officers justified their physical restraint of a citizen by simply reporting that drug dealers, car thieves, or other broad category of individuals were generally known or believed to possess weapons. Within the problematic reports identified, most officers stated that the stop and frisk of a subject was conducted for officer safety or the subject’s safety, without articulating any Reasonable Suspicion66 that criminal activity was happening or about to happen. Kroll notes that, the Fourth Amendment to the United States Constitution requires that, before stopping an individual, the police must have a reasonable suspicion that a crime has been, is being, or is about to be committed by that individual. If the police reasonably suspect that the individual is armed and dangerous, the police may frisk the individual, meaning that the police may conduct a limited pat- down search of the individual's outer clothing. Reasonable suspicion must be based on “specific and articulable facts” and not merely on an officer’s hunch. In many of the problematic cases we reviewed, however, officers told individuals they were not under arrest and had done nothing wrong (i.e., criminal activity was not afoot), and then immediately placed them in handcuffs. Moreover, whenever individuals asked why they were being detained or displayed uncertainty or confusion over the officer’s actions, in many cases they were arrested and charged criminally with resisting or evading arrest/search/detention/transportation. These unnecessary and unjustified charges resulted in several individuals being automatically entered into the criminal justice system with no underlying charge that led to the encounter. We also observed in these cases that APD officers frequently handcuffed individuals citing only officer safety as a justification. In the problem cases reviewed and highlighted by Kroll, it did not seem to matter if the subject was an 11-year-old female, a 70-year-old female suffering from dementia, a 67-year-old female suffering from 66 In 1968, the United States Supreme Court held that the Fourth Amendment to the Constitution prohibition on unreasonable searches and seizures is not violated when a police officer stops a suspect on the street and frisks him without probable cause to arrest, if the police officer has reasonable suspicion that the person has committed, is committing, or is about to commit a crime, and has a reasonable belief that the person “may be armed and presently dangerous.” The reasonable suspicion must be based on “specific and articulable facts” and not merely upon an officer’s hunch. Terry v. Ohio, 392 U.S. 1 (1968). 76  a traumatic brain injury, or a victim of a crime. Officer safety was also used as the stated justification for searching a person’s home on disturbance calls. Kroll found that APD officers in these cases “escalated” situations by not informing the individuals encountered of why they were being stopped or detained. Officers aggressively placed individuals in handcuffs and, when the individuals resisted the handcuffs because they did not comprehend why they needed to be in handcuffs, they were forced to the ground and arrested. The aggressive and frequent use of handcuffs, and the failure of officers to de-escalate the encounters by informing the individual of the reason that handcuffs were being utilized, resulted in many unnecessary force situations and arrests. The charge of resisting arrest/detention appears to be overused to justify officers’ actions.67 Kroll also found in these cases that APD officers often commanded subjects to get on “on the ground” usually by a leg sweep at the slightest resistance. The leg sweep causes the subject to fall to the ground. For example, in one incident reviewed, four male officers swept the legs from a non-aggressive intoxicated female subject, which caused the female to fall to the ground and experience facial injuries. This was just one example of an incident in which escalation was unnecessary and resulted in a preventable injury. Moreover, when using this technique, once on the ground, APD officers frequently had a hard time controlling the individual, who was understandably confused and upset by what was happening. In some instances, Kroll observed that officers did not give individuals adequate time to comply with commands. When the individuals responded to weaponless pressure point control tactics, the officers would then tell the individuals to “stop resisting” – even when the facts and video reviewed by Kroll did not suggest resistance. 4.3 Additional Trends and Issues within the Problematic Cases Additional issues were identified during Kroll’s assessment of the 112 problematic cases. Several of these concerns were present across multiple problematic cases identified, representing potential trends that should be corrected. • Resisting Detention/Search Charges: In many of the problematic cases flagged during Kroll’s review, APD officers did not have a criminal charge when detaining someone who either wanted to leave or who resisted in some way—usually by simply asking why they were being stopped or by not immediately placing their hands behind their back. Typically, within seconds, the officer arrested the individuals for Resisting Detention or Search and transported them to jail without any other underlying charge(s). 67 APD General Order 211, Response to Resistance Inquiry, Reporting, and Review, recognizes officers will commonly encounter reflexive stiffening or pulling away by subjects when using empty hand control techniques during escorting or handcuffing that is different than a subject that is actively resisting. Section 211.2.4, Level 4 Force Incidents, effective December 18, 2018, provides the following two examples of force utilizing empty hand control techniques: 1. Restricting a subject’s movement by strength or body weight (to include resisted escorting or handcuffing of a subject who is actively resisting arrest beyond the initial or reflexive stiffening or pulling away of a person’s arm(s) that officers commonly encounter during handcuffing). 2. Using leverage or strength to bring a subject’s arms or legs together for the purposes of controlling, handcuffing, or hobbling the subject (to include resisted control, handcuffing, hobbling when the subject is actively resisting arrest beyond the initial or reflexive stiffening or pulling away of the subject’s arm(s) or leg(s) that officers commonly encounter during efforts to control, handcuff, or hobble a subject.) 77  • Citizen Complaints/Internal Investigations: Kroll identified three cases that should have been referred to the Special Investigations Unit or Internal Affairs to address potential misconduct in accordance with APD policy.68 • Supervisory Issues: Kroll’s review found there existed insufficient supervisory oversight and review of problematic cases involving use of force. This was true whether the force used involved actively targeting with a firearm, failure to follow APD Taser policy, or officer incident reports that conflicted with the video footage contained in the body worn or dashboard cameras. Use of force cases need to be thoroughly reviewed by the supervisory chain of command and problem areas immediately addressed with the involved officers. o The APD use of force process appears to involve routine reporting and superficial review. We found little evidence that supervisors sufficiently addressed the appropriateness of the use of force used or whether the tactics employed were justified or whether the reported facts conflicted with the video footage. In each of the cases where Kroll determined use of force was not appropriate or that the encounter was unnecessarily escalated by APD officers, the supervisors routinely approved the resulting arrests.69 • Taser Usage: Based on our review, it appears that the Taser or ECW (Electronic Control Weapons) is the intermediate weapon of choice for APD officers. Kroll’s review noted very limited use of Oleoresin Capsicum (OC) spray or mace. We identified eleven cases in which there appeared to be a violation of APD Taser policy70. This included tasing subjects who were otherwise complying with officers, tasing subjects who were not involved in any criminal activity, and failing to provide Taser warnings, all of which went unaddressed by APD supervisors. • Pointing of Firearms: Kroll noted that the pointing of a firearm at an individual, or actively targeting with a firearm, is not considered a reportable “use of force” under APD policy. In our review, we noted many cases where the pointing of a firearm occurred apart from the use of force reported. Kroll thus sought to gain a greater understanding of the circumstances involved in these APD incidents and determine whether the reported pointing of firearms was necessary and appropriate. Of the 1,321 cases reviewed, Kroll specifically identified nine pointing of firearms or actively targeting cases that were problematic. For example, in one case, two officers in a foot pursuit pointed their firearms at a subject while running and then re-holstering, and in two separate incidents, an officer was in a back- up officers’ line of fire while making an arrest. In the six remaining incidents highlighted, the pointing of firearms occurred where the use of lethal force would not have been justified under the circumstances 68 APD General Order Response to Resistance Inquiry, Reporting, and Review, Section 211.3 Violations of Law or General Orders, effective April 25, 2019. 69 APD informed Kroll that, starting in 2021, APD created the Force Review Unit, which is ultimately responsible for use-of- force incident reviews within APD. However, this does not absolve responding first-line supervisors from taking corrective action when appropriate following use of force incidents. 70 APD General Order 208, Taser Device Guidelines. 78  and not in accordance with APD policy.71 Additionally, APD does not consistently capture the pointing of a firearm/actively targeting in its reporting structure, so it is unknown how frequently this occurs in other cases, including encounters that do not result in any documented uses of force. o Recent research shows that police agencies with policies that require documentation of a pointing of firearm have significantly lower rates of officer-involved shootings; additionally, they find that this type of policy was not associated with increased injury or death rates among officers (Jennings & Rubado, 2017; Shjarback, White, & Bishopp, 2021). In addition, the Police Executive Research Forum's 30 Guiding Principles on Use of Force specifically recommends that police agencies document the pointing of firearms or Tasers at individuals (PERF, 2016). • Neck Restraints/Chokeholds: Kroll identified five cases in which neck restraints or chokeholds were used. In each of these cases, the use of lethal force would not have been authorized, and these holds can have lethal consequences. • Head Strikes: Kroll identified two cases where head strikes were used, including one with an APD issued Taser (impact weapon). In these cases, the use of lethal force would not have been authorized by or comported with APD policy.72 The use of head strikes can have lethal consequences. • Mental Health Related: Kroll identified 21 use of force cases of concern that involved a mental health component (18.8% of problematic cases). Of these 21 problematic incidents involving an individual with mental health issues, in 19 incidents, Kroll determined that the use of force was inappropriate. That is, of all use of force incidents reviewed during the six-month time period, Kroll determined that 1.4% of these cases involved an inappropriate use of force against an individual displaying some type of mental health issue or concern. • Body Worn Cameras (BWC): Kroll identified 15 incidents in which APD officers failed to activate their body worn cameras or official reports conflicted with video footage. These policy73 failures and conflicts were not addressed by APD chain of command in the use of force report review process. • Field Training Officers (FTO): Kroll identified three incidents involving FTO’s that inappropriately escalated use of force encounters with individuals while the FTO was in the process of mentoring/instructing a trainee. • Level 1 Cases: Kroll reviewed six of the seven Level 1 cases during this assessment. These incidents are the most serious under APD policy. Level 1 incidents involve, for example, any force that could result in serious bodily injury or death. Of the six Level 1 cases reviewed, Kroll found that the use of force was appropriate in five cases under the circumstances, and inappropriate in one case. The seventh case involved a suicidal subject armed with a large butcher knife. Lethal force was deployed 71 APD General Order 202, Firearm Discharge Situations, Section 202.1.4 Display of Firearms. Firearms may be readied for use in situations where it is anticipated they may be required. Firearms shall not be displayed or pointed in a threatening or intimidating fashion unless it is objectively reasonable to believe there is a substantial risk that the situation may escalate to the point where deadly force would be permitted. Firearms shall be secured or re-holstered as soon as reasonably practicable when it is determined that deadly force is no longer necessary. 72 APD General Order 206, Control Devices and Techniques. 73 APD General Order 303, Body Worn Camera Systems. 79  by two APD officers killing the subject. On August 6, 2021, the Travis County District Attorney notified APD that a Grand Jury had returned murder indictments against both officers. As the criminal case is currently pending, APD’s administrative review is delayed, and APD does not intend to reach a conclusion during the criminal prosecution to protect the integrity of the investigation. Accordingly, Kroll takes no position on the appropriateness of these criminal charges or the delayed administrative review. 4.4 Individual Examples of Problematic Use of Force Cases In our review of the use of force incidents, including a review of police reports and body camera and dashboard video footage of the cases that occurred from June to November 2019, Kroll identified several troubling incidents that provide clear examples of poor supervisory oversight and lack of accountability by the Department. We provide a brief discussion below of eleven cases reviewed, which are but a sample of the 112 cases identified that were of concern. As noted below, each of these incidents required more scrutiny than APD supervisors applied. Incident No. 1 This incident involved a trespass radio call. A homeless 21-year-old Hispanic male fell asleep in a restaurant. The first officer arrived and approached the subject (accompanied by the restaurant manager), whereupon he verbally notified the man that he needed to leave and would be arrested for trespass if he returned. The officer then asked the man to come outside. When the officer asked the man for his name and identification, the man provided his name and said he was homeless and did not have any identification. The officers warned the man that if he returned within the next year, he would be arrested. The officer instructed him to get his stuff and leave. The man complied with the officers’ request and began to leave. As the man began to pick up his bag, one of the officers accused the subject of lying (it was later determined that he had not lied), at which point the officer and assisting officer pushed the subject to the ground and forced his hands behind his back. Although there was no evidence of resistance, the officers unjustifiably escalated the encounter by repeatedly punching the subject in the chest and head, causing injuries and requiring an EMS response. Subsequently, as shown on the body camera footage, an uninvolved civilian witness was interviewed and said that the officers’ actions were inappropriate, that the officers were not assaulted, and that the allegation against the homeless man (resisting arrest) was a lie as the man never hit either officer. Kroll noted that an officer (non-supervisor) who subsequently took statements from the involved officers used leading questions in an attempt to establish that the force was justified. The body camera footage clearly shows that the subject was not resisting when the officers became physical with him and used inappropriate force. The subject was a homeless man who had fallen asleep in the restaurant. When asked to leave, he was cooperative, committed no crime, and posed no threat to the officers or to others. Kroll finds that the officers’ use of force was not appropriate and was the result of unnecessary escalation by the officers. The supervisor who was notified and responded to the scene approved the arrest. In the incident reports, important facts were left out about the encounter, including the factual statements provided by the independent witness. As a result, the subject was unjustifiably arrested on a felony charge, Assault on a Peace Officer, and jailed. 80  Kroll finds that an appropriate review of the police reports by the supervisor conducting the use of force inquiry and review by the APD chain of command, as required by APD policy,74 should have identified this potential misconduct and referred this use of force report to the Special Investigations Unit and Internal Affairs Division (also required by APD policy) for appropriate follow-up investigation. Incident No. 2 In this incident, an officer used force on a 21-year-old Black male who was riding his bicycle during the early morning hours in an area where a high-risk search warrant had been executed. APD SWAT was on location when the individual approached the area while coasting at a slow rate of speed on his bicycle. When the individual failed to heed an officer’s direction to turn around and go the other way, the officer approached and intentionally stopped him by grabbing the left side of the bicycle’s handlebar, causing him to crash into the side of a police vehicle. The individual received medical treatment and was eventually released. A review of the incident showed that the young man had committed no criminal offense and was not riding or peddling his bicycle directly towards any officers. His hands were visibly on the handlebars and he was clearly unarmed while slowly coasting down the street. The other side of the police cars contained an open street passage that would have avoided the officers. In the incident report, the officers failed to describe articulable facts warranting the use of force. Contrary to the written report, body camera footage showed that the subject was not peddling the bicycle at a high rate of speed; the factual descriptions contained in the incident report were contradicted by the video footage. Additionally, the incident report noted under the use-of-force details that the subject “Resisted”; however, under “Resistance Type” the report indicates “Not Resistant.” These conflicts in the report were not addressed by APD supervision. Moreover, the officer inappropriately escalated the incident and created an unnecessary risk of injury. Incident No. 3 In this incident, a male complainant contacted APD out of concern for his girlfriend who allegedly said she wanted to kill herself (Mental Health/CIT related incident). The complainant informed the responding officers that his girlfriend, a 57-year-old white female, had a “fake” gun in the residence. An officer called the female subject on the phone, and she came outside as requested. She was visibly unarmed and dressed only in a tee shirt. The officers demanded to see her waistband. The female pulled up her shirt exposing her naked upper body, front and back, and bikini under bottoms, which demonstrated that she was clearly unarmed. When she turned in an attempt to re-enter the residence, an officer shot her in the left buttocks with a bean bag, causing visible injuries. Five APD officers were on the scene, including a supervisor. Kroll’s review found that this incident was an overreaction by APD, which responded to the scene as if it were a barricaded gunman situation. The officers wore body armor and Kevlar helmets, while one officer pointed a patrol rifle at the female subject and another 74 APD General Order 211, Response to Resistance Inquiry, Reporting, and Review. 81  officer aimed a less lethal shotgun (capable of shooting rubber or plastic bullets).75 The use of deadly force would not have been justified in this situation. Video footage showed the officers safely positioned across the street behind a marked patrol unit. Although the subject naturally questioned an officer why she was shot, the shooting was not sufficiently explained or articulated in the incident reports. From the moment the female exited her residence, she was visibly upset by the number of officers on the scene and afraid for her safety. Kroll finds that this use of force was not appropriate or necessary under the circumstances Nevertheless, a supervisor approved the use of force report and the case was “Cleared Administratively”. Incident No. 4 In this incident, a 22-year-old Black male fled a motor vehicle accident scene on foot (he was subsequently charged with Evading on Foot, Crash with Serious Bodily Injury) and eventually received medical treatment at a local hospital. Officers pursued the man and found him sitting on top of a shed roof. The roof was approximately 10 feet high, and it was dark outside. Officers attempted to speak with the man, whose legs were hanging over the side as he talked to the officers. The officers demanded that the man come down from the shed, at which point he asked the officers to please put their lights down so he could see and come down from the shed in compliance with the officers’ request. The man complained of breathing issues and physical injury from the crash and appeared to pose no threat of escape. Nevertheless, before the individual had an opportunity to voluntarily comply, an officer forcibly pulled the man’s right leg, which caused him to fall ten feet to the ground, thus risking further injury. Kroll finds that the officers should have deescalated the situation and used better tactics or alternatives. Instead, one officer threatened to tase the subject, although APD Taser policy76 prohibits taser use against passively resisting subjects and individuals whose activity may result in collateral injury (falls from heights); the officer then physically pulled the man from the shed’s roof, rather than seeking assistance from EMS or by providing a ladder. Video footage shows that the man was unarmed and posed no immediate threat, and he was allowed only 26 seconds to comply. The officer who pulled the subject down from the shed roof unnecessarily escalated this situation. Two supervisors were notified of this incident and approved of the officer’s actions. Incident No. 5 In this incident, a 34-year-old white male was arrested for Public Intoxication after he had called 911 and said he had a personality disorder and requested police assistance. The man was of small build, non-violent, and appeared in distress when the officers arrived (Mental Health/CIT related incident). The reporting officer documented the incident as a “Check Welfare” call. A review of the incident shows the arresting officer (three officers were present at the incident) achieving a takedown by lifting the man off the ground and slamming him down hard. Although the officer wrote in his report that the subject "lunged" at him, thereby requiring a forcible takedown, body camera footage did not support this description. The video footage did not show the man "lunging" at the officer but simply trying to evade being body slammed. Kroll finds that the actions of the officer on a subject in distress was excessive. The incident was "Cleared by Arrest” and approved by a supervisor. 75 APD General Order 206, Control Devices and Techniques, Section 206.5, describes a less lethal shotgun as a shotgun intended to shoot Kinetic Energy Projectiles (e.g., rubber or plastic bullets) that are not intended nor likely to cause death or serious bodily injury. 76 APD General Order 208, Taser Device Guidelines. 82  Incident No. 6 In this incident, a 27-year-old white male was arrested for Public Intoxication. The arresting officer struck the subject in the lower back multiple times while the intoxicated man lay face down on the ground and did not pose a threat. Two other officers were present and could have assisted in handcuffing the individual. Kroll’s review found that the striking officer appeared angry, and the application of physical force unnecessary. Although Kroll does not challenge the appropriateness of the criminal charge, force should only be used when necessary to complete a lawful objective. In Kroll’s review of the body camera footage, the officer appeared to be out of control and overly aggressive in his actions. At one point, another officer was observed grabbing and trying to restrain the officer’s right arm that was doing the striking. The strikes in the subject’s back were excessive and not appropriate. Despite these facts, a supervisor was notified and approved the arrest. Incident No. 7 In this incident, an 11-year-old Hispanic female, while standing outside of the apartment complex of her home, began to run when she saw an officer, telling the officer that she did not want to go home. The officer pursued the child on foot and, when he caught up to her, he forcibly grabbed her arm and caused her to fall to the ground. The child was placed in handcuffs and frisked by a male officer. The officer asked if she had any guns, knives, or weapons on her, and then handcuffed and searched her. Kroll finds that the use of force, handcuffing, and search of this juvenile was not appropriate. No reasonable suspicion existed to detain her for being a missing person/runaway. Although officers wrote in the incident report that the juvenile was frisked for their safety, this did not meet the constitutional standard of reasonable suspicion. The juvenile was simply walking away from and looking at the officer, who "determined that the child was showing conduct indicating a need for supervision and that a detention was needed to ensure her safety." This detention was not reasonable and caused unnecessary injury to an 11-year-old child. Nevertheless, the juvenile was arrested for Evading on Foot, which was approved by a supervisor. Incident No. 8 In this incident, a 48-year-old white female got into an argument with her boyfriend and called 911 reporting a disturbance. Three APD officers arrived and knocked on the door. The female was heard by officers yelling at the male. The female said from behind the door that she could not let the officers in because she was naked, although she cracked the door partially open as she talked to the officers, who could see that she was unclothed. The officers nevertheless forced their way into the residence and then forced the visibly naked and unarmed female to the ground, handcuffed her, and arrested her, claiming that she resisted. An officer documented in his report that the woman’s demeanor was aggressive and that the officer’s safety and hers were threatened as he attempted to place her in handcuffs. Kroll finds that the detention was not based on reasonable suspicion and that the use of force was inappropriate. The female called for assistance and ended up being placed in handcuffs when no crime was committed, no weapons were visibly present, and there was no threat to the officers. Although the incident report stated that the female was acting aggressively, Kroll disagrees and finds that the officers unnecessarily escalated this incident. The female was arrested for Interference with Public Duties and transported to jail. A review of the case showed that the female was experiencing a mental health related incident, possibly from consumption of PCP; however, the officers did not call EMS for an evaluation/assessment or attempt other more appropriate action. A supervisor was notified of the use of force and approved the arrest. 83  Incident No. 9 In this incident, a 34-year-old Black male was observed by APD officers crossing the street outside of a crosswalk (a Transportation Code violation, Section 552.005, Crossing at a Point Other than a Crosswalk). The body camera footage shows that officers entered the store/gas station where the man was standing in the counter area. The officers asked the man to go outside and talk, and then physically grabbed the man, who seemed confused by the officers’ actions. The incident quickly evolved into the officers physically handling the individuals before informing him that he was being detained, thereby unnecessarily escalating the encounter (Kroll notes that it was later discovered that the man had active warrants, but this became known only after he was in custody and not during the pedestrian encounter). The officer charged the man with Evading on Foot and Resisting Search and transported him to jail. A supervisor was notified of this incident and approved the officers’ actions. Kroll finds that the use of force was inappropriate, and that the incident was unnecessarily escalated by APD officers. The physical detention was not based on reasonable suspicion. The officers almost immediately forced the man to the ground and handcuffed him while telling him he was not under arrest. Although one officer reported that the man was detained in handcuffs for the safety of officers and the individual, this justification was not supported by the facts displayed in the body camera footage. Incident No. 10 In this incident, a 37-year-old Black male was observed by officers crossing the street at an intersection while the pedestrian control device light was red (jaywalking). When an officer instructed the man to stop, he continued walking from the officers but was quickly boxed in by four officers. One officer unnecessarily escalated the encounter by throwing the man to the ground and stunning him multiple times with the department issued Taser. Officers gave confusing commands for the man to get on the ground, even after he was already on ground. The body camera footage shows the officers going hands on almost immediately, thereby causing the encounter to escalate. Moreover, the man was charged with Evading Arrest or Detention and Resisting Arrest, Search, or Transportation for a minor incident that began with a simple pedestrian control light violation. A supervisor was notified and approved the arrest. Incident No. 11 In this incident, officers observed a moving traffic violation and possible registration violation, both of which were minor traffic violations. The unarmed subject, a 19-year-old Hispanic male, exited his vehicle after parking at an apartment complex. Two APD officers approached him and one officer pointed a firearm, actively targeting the subject. The officers then forced the individual to the ground and handcuffed him. Kroll finds that the officers unnecessarily escalated the encounter and applied unnecessary force. The individual was issued two traffic citations for Improper Lane Change and not having a driver's license and subsequently released. An officer documented in his report that, based on his five years’ experience, he and the other officers had reason to fear for their safety for a list of generalized reasons. However, the officer provided no objective facts regarding this incident as to why a lethal seizure was warranted. The officer described the area where the encounter occurred as a “high crime area” but provided no facts that justified the pointing of a firearm and the physical use of force in the circumstances. The officer's stated justification was implausible. A review of the body camera footage indisputably shows that the man posed no immediate threat, the crime was a minor traffic offense (Field Release Citation and Summons issued), and he was not resisting or evading. There was simply no objective reason to justify forcing the man to the ground. 84  Additionally, both of the officers’ reports related that one officer holstered his firearm as they moved in to detain the driver in handcuffs, stating that they were handcuffing him for the safety of the officer and the individual. As previously noted, however, officer safety does not equate to reasonable suspicion that criminal activity is afoot. This person was inappropriately subjected to a lethal seizure with a firearm, was taken to the ground, handcuffed, frisked, and suffered an injury to his forehead. He was ultimately issued two citations and released. 4.5 Conclusion Kroll’s review of the 1,321 use of force incidents from June to November 2019 highlighted that, while the vast majority (over 90%) of use of force incidents were justified consistent with the facts described in the incident reports, there remains a need for improvement. In at least 8.5% of the reviewed cases, officers unnecessarily escalated encounters and/or applied inappropriate or unnecessary uses of force. Moreover, meaningful supervisory review was extremely limited or non-existent within these flagged incidents. Specifically, within the 112 problematic incidents highlighted by Kroll that warranted further scrutiny, none resulted in a referral to the Special Investigations Unit or Internal Affairs. Supervisors in each case were notified and approved any resulting arrests.77 In these cases, APD officers appeared too quick to force individuals to the ground or to go “hands on” with subjects, and to stop and frisk subjects without any reasonable suspicion of criminal activity. It is precisely these seemingly unnecessary and inappropriate actions that contribute to issues of public mistrust and a breakdown of community relations. Reducing the likelihood of these types of problematic encounters – and holding officers accountable when they do occur – will require more oversight and better supervisory practices. In Section 7, Kroll has provided a series of recommendations to help address some of the concerns identified here. 77 Kroll notes that, in one of the 112 cases highlighted, the arresting officer subsequently determined that he did not have probable cause to arrest the individual after the officer had used force while physically taking the individual into custody. The supervisor on scene agreed and the individual was released. 85  5. ANALYSIS OF APD TRAFFIC STOPS, CITATIONS, ARRESTS, AND SEARCHES (January – December 2020) The initial scope of work requested an analysis of stops, citations, charges, arrests, and searches for a six- month period from June to November 2019 for the purpose of understanding whether racial and ethnic disparities exist in these outcomes. Unfortunately, Kroll was unable to examine most of these interactions for the specified period due to limitations with APD’s data collection, which are described in detail below. Based on extensive conversations with APD’s Chief Data Officer and Information and Technology Supervisor, the Kroll team determined that any traffic stop data collected prior to January 1, 2020 was inaccurate and would result in findings that could not meet basic social scientific standards for validity, even for the most basic descriptive statistics. It was determined that data collected after January 1, 2020—which had undergone extensive cleaning efforts by APD staff—was the only opportunity available to examine patterns and trends in APD motor vehicle stops. Therefore, in this section, we examine motor vehicle stops from January 1 to December 31, 2020, the outcomes of these stops (e.g., warnings, citations, arrests), and whether searches occurred during these encounters. We also extend our previous analyses presented in Section 3 of all arrests from 2017–2020, to provide an overview of searches that occurred in conjunction with those arrests; these analyses are beyond the examination of arrests and searches only during traffic stops (this Section) or only during use of force events (Section 3). 5.1 Background In 2001, Texas, like many other states at the time, enacted a racial profiling statute (Texas Law on Racial Profiling, Senate Bill 1074) that required police agencies to adopt a detailed written policy on racial profiling, collect data related to traffic stops (resulting in warnings, citations and arrests), submit an annual report of the collected information to the agency’s governing body that includes a comparative analysis of stops, stop outcomes, and searches by race/ethnicity to the population of the agency’s reporting area, and ensure officers complete the required training on racial profiling established by the Texas Commission on Law Enforcement (TCOLE). These requirements are now included in the Texas Code of Criminal Procedure 2.131-2.134. The Sandra Bland Act (2017) also extended the requirement for the annual report to be submitted to TCOLE as well as the agency’s governing body. APD’s current policy, General Order 328 Racial or Bias-Based Profiling, includes all these requirements. Required information includes the location of the stop, the person’s race/ethnicity, the person’s gender, whether the officer knew the race/ethnicity prior to the stop, the initial reason for the stop, the outcome of the stop, whether a search was conducted and the reason for that search, whether any contraband was discovered during a search and a description of that contraband, whether an arrest was made and its legal basis if so, and whether force that resulted in bodily injury was used. Beginning with stops conducted on January 1, 2022, each of these additional data fields, other than location of the stop, must be broken down by race/ethnicity for the TCOLE report. 5.2 Data Description and Limitations It is important to note at the onset that APD does not have a comprehensive motor vehicle stop database. In contrast to most police agencies that collect traffic stop data – and well documented best practices in data collection (see Fridell, 2004; Ramirez, McDevitt, & Farrell, 2000; McDevitt, Farrell, & Wolff, 2008; Pryor, Goff, 86  Heydari, & Friedman, 2020) – APD does not have one form that is completed for every traffic stop, regardless of disposition of the stop. Rather, to meet TCOLE stop reporting requirements described above, APD compiles information by combining data from three separate databases that are not mutually exclusive: (1) street checks (i.e., verbal or written warnings), (2) citations, and (3) arrests. APD recently discovered errors in their TCOLE motor vehicle stop data collection methodology, whereby stops were being counted multiple times if a single stop resulted in multiple dispositions (e.g., if a driver during a single stop was issued a citation and arrested, it was recorded as two traffic stops, rather than one stop with multiple outcomes). While the duplication between the citation and arrest database was clearly identified, the possibility also remains that there is duplication between the warning and citation databases, and the warning and arrest databases, although APD officials have suggested this type of duplication is less likely. Unfortunately, there is no automated method available for identifying and removing duplicates on APD data because there is not a common linking variable (unique identifier) across the datasets. Therefore, the duplicate entries must be manually removed from the data, based on the combination of the stop date, time, person name, etc. Further, APD staff readily acknowledges that this likely data collection problem had a greater likelihood of over-counting searches because a high percentage of arrests result in searches. APD decided to identify and correct the duplicate entries using the labor-intensive manual correction process for the data collected in 2020; after cleaning and data reliability checks, they resubmitted the 2020 stop data to TCOLE and uploaded data to the APD open data portal. All data collected prior to January 1, 2020, however, have not been corrected. Although Kroll’s scope of work called for an analysis of police-citizen contacts during a six-month period of 2019, based on guidance from APD and knowledge that the data prior to 2020 is not valid, the Kroll team instead analyzed a full year of 2020 motor vehicle stop data. Further, we used only the variables within this database that have been verified by APD staff and are available in the open data portal. The findings reported in this section should not be directly compared to previous reports on APD traffic stops conducted by others78 because these reports were produced using data that we now know to be invalid. At a minimum, previously issued reports based on these data overestimate the number of stops and searches; the degree to which this was a problem is unknown for 2017-2019. The duplication of stops in previous years’ data could have had a disproportionate impact on racial/ethnic minorities if they were more likely than whites to receive a citation and arrest; whether this is the case is also unknown for 2017-2019. We were able to explore 78 Goff, P.A., Obermark, D., LaVigne, N., Yahner, J., & Geller, A. (2016). Measuring Fairness in the Austin Police Department. Available at: https://policingequity.org/images/pdfs-doc/Austin_PDI_Report_2016_Release.pdf Office of Police Oversight, Office of Innovation, and Equity Office. (2020, January). Joint Report: Analysis of APD Racial Profiling Data, 2015-2018. Available at: https://joplin3-austin-gov- static.s3.amazonaws.com/production/media/documents/Final_-1.28.2020_Racial_Profiling_Report.pdf Office of Police Oversight, the Equity Office, and Office of Innovation. (2020, November). 2019 Joint Report: Analysis of APD Racial Profiling Data. Available at: https://joplin3-austin-gov- static.s3.amazonaws.com/production/media/documents/2019_Joint_Report_Analysis_of_APD_Racial_Profiling_Data_FI NAL.pdf Joyce James Consulting. (2020, November). Racial Inequities and Institutional Racism: A Report Submitted to The City of Austin Equity Office and The Austin Police Department. Available at: https://www.austintexas.gov/edims/document.cfm?id=352711 87  this possibility with the corrected 2020 data. APD identified 346 of 68,330 stops (0.5%) that resulted in both a citation and arrest, for which they removed the duplicate entries. All the duplicate cases included a search, and Hispanics and Blacks composed a larger percentage of the duplicate cases than whites and individuals of other races/ethnicities. Specifically, Hispanics and Blacks made up 45.7% and 30.9%, respectively, of the cases where there was duplication, compared to 22.3% for whites and 1.2% for individuals of other races/ethnicities. We cannot determine whether similar search rates and racial/ethnic disparities in duplicate cases occurred prior to 2020 because APD made the decision not to try to identify duplicates for 2019 and earlier. It is, however, a distinct possibility that the known issue of duplication of stops disproportionately impacted racial/ethnic minorities and over-counted searches. Data Limitations The APD motor vehicle stop and arrest data suffer from a number of additional limitations that impact the extent of the analyses Kroll can provide. First, the data fields that are collected across the warning, citation, and arrest databases are not uniform. When the three databases are combined to create the motor vehicle stop database, there are two specific limitations for the types of analysis that can be conducted. First, patterns and trends in stops and stop outcomes cannot be examined at lower organizational levels (i.e., by APD Sector) because this information is only collected in the warning and arrest databases and is not available for citations. Second, geographic analyses are not possible because the stop location information collected is also inadequate in the citation data. The warning and arrest databases include x/y coordinates, zip code, and street addresses; however, the citation database only includes street addresses. The street addresses do not link to ARC GIS street maps (e.g., missing directional indicators) for 48% of the stops that resulted in citations. Another difference in the citation data compared to the warning and arrest databases is that there is no way to ascertain whether demographic information (i.e., individuals’ race/ethnicity, gender) reported from the citation data are for the vehicle’s driver or passenger; as a result, when we discuss demographic characteristics, we refer to individuals warned, cited, or arrested, rather than “drivers.” APD’s data collection is tailored to the TCOLE reporting requirements, which entail collecting information for all motor vehicle stops, regardless of how they are initiated. Likely because of this, APD’s stop data cannot distinguish officer-initiated from dispatched stops because it does not include a data field that provides this information – and it cannot be reliably linked to an external data source that could provide this information. Therefore, we cannot provide disproportionality analyses based on these data because the comparison of stops to an expected rate of stops (i.e., benchmark) is only appropriate for discretionary stops. For example, when officers are dispatched to a call that results in a motor vehicle stop, they have little discretion regarding whether to initiate that stop; likewise, sobriety checkpoints, or other specialized use of traffic stops may not be within officers’ discretionary decision-making. Further, as previously noted in Section 3, the selection of a valid benchmark that estimates the population at risk for traffic stops is especially difficult. Although census data is often relied upon because it is readily available, for the last two decades policing scholars have argued against its use as a valid benchmark for traffic stops. Population statistics do not measure differences in who is at risk for traffic stops based on driving location, driving frequency, and driving quality or offending behavior (Engel & Calnon, 2004; Fridell, 2004; Ridgeway & MacDonald, 2010). For these reasons, residential population is not a valid comparison group for traffic stops. No conclusions about racial/ethnic disparity should be made based on these flawed comparisons. There is no valid benchmark readily available against which to compare the APD traffic stop data. 88  As noted in Section 3, there are also issues with reliability and missing data on the arrest charges and offense severity information in the arrest data. To summarize, APD only collects information regarding the highest charge, rather than the total number and type of all charges. Furthermore, charge severity that is easily linked to the arrest charges is based on Texas legal statutes and includes many felony or misdemeanor possibilities (e.g., Felony 1st degree, Felony 2nd degree, Felony 3rd degree, etc.) based on a single charge which are contingent on various factors including prior criminal history and circumstances of the event. These circumstances are not readily available in our data and thus do not lend themselves to the creation of a reliable indicator. Finally, the offense severity table to which charges are linked does not include all charges utilized by APD. As a result, we cannot examine trends or potential disparities in arrest charges or the ‘seriousness of the offense’ (using possible sanctions per offense as an indicator) between 2017 and 2020 due to data coding and reliability limitations. Finally, analyses of aggregate level data cannot be used to determine whether or not APD officers have individually or collectively engaged in “racial profiling.” The legality of prior (or future) individual traffic stops cannot be assessed with these aggregate data. Furthermore, the majority of the analyses presented are descriptive or bivariate in nature. That is, they summarize patterns and trends, or assess the relationship between two variables (e.g., race/ethnicity and stop outcome), but do not consider other factors that likely influence the outcome. Multivariate analysis, which simultaneously considers the impact of multiple variables on an outcome, is a stronger statistical technique in which we would typically have more confidence. The number of relevant data fields that are available for inclusion in these models, however, are limited. Multivariate analysis can only control for the impact of measured variables, and even this more advanced statistical technique is inadequate to truly assess the independent impact of race/ethnicity on APD stop outcomes. Despite these considerable limitations and the accompanying caveats, the information provided in this section is designed to give feedback to APD administrators regarding the status of the data collection process, along with exploring emerging trends and patterns in the data that may be utilized for continual improvement and training purposes. 5.3 Motor Vehicle Stops January 1 – December 31, 2020 This section describes the findings based on the data compiled from APD warning, citation, and arrest databases as described above, which includes 68,330 motor vehicle stops from January 1, 2020 to December 31, 2020. We present the initial reason for the stop and stop outcomes and examine gender and racial/ethnic differences. Thereafter, we detail the 5,224 searches conducted during motor vehicle stops in 2020 and consider the rate of contraband discovery resulting in a seizure. As noted above, the descriptive information available for motor vehicle stops across all three datasets is limited. The only location information consistently collected across the datasets is the street type where the stop was conducted. More than half of stops occurred on city streets (58.9%) compared to 41.1% that took place on state or US highways. Figure 1 below provides the temporal breakdown of traffic stops by month for January to December 2020. The first three months of 2020 account for nearly 40% of all stops conducted within the year (prior to the COVID-19 related shutdowns). Another spike in the number of stops occurred in May, possibly related to changes in COVID stay-at-home protocols, and APD’s response to protests and civil unrest. The number of stops in the last five months of 2020 were consistent, but at a notably lower frequency than early 2020. 89  Figure 1: APD Motor Vehicle Stops January 1 – December 31, 2020 10,020 9,696 7,990 7,261 12,000 10,000 8,000 6,000 4,000 2,000 0 5,444 4,864 4,382 4,414 4,335 4,091 3,440 2,393 Jan Feb Mar Apr May June July Aug Sep Oct Nov Dec Of the 68,330 motor vehicle stops conducted by APD during 2020, the following percentages represent the racial/ethnic and gender breakdown of the individuals involved: • White (44.8%) • Hispanic (35.0%) • Black (14.9%) • Asian (4.1%) • Middle Eastern (1.1%) • American Indian / Alaska Native (0.04%) • Hawaiian/Pacific Islander (0.02%) Due to the small percentage of individuals in racial/ethnic groups other than Black, Hispanic, and white, they are hereafter grouped together in an “Other” race/ethnicity category for analysis purposes. Figure 2 (below) shows the modified racial/ethnic representation of those involved in APD traffic stops in 2020. Considering individuals’ gender, almost two-thirds of those stopped were male (64.1%), while 35.9% were female. 90  Figure 2: Race/Ethnicity of Individuals Involved in APD Motor Vehicle Stops January 1 – December 31, 2020 Other, 5.3% Black, 14.9% White, 44.8% Hispanic, 35.0% Figure 3 below displays the reported initial reason for the stop as one of four broad categories used by APD: 1) moving traffic violation, 2) violation of law other than traffic, 3) vehicle traffic violation, and 4) pre-existing knowledge (i.e., warrant). These categories are also mutually exclusive; that is, only one initial reason for the stop is recorded in the available data. This is another data limitation because in practice, multiple reasons for a traffic stop are possible, but not captured. As shown, the most prevalent initial reason for a traffic stop is for moving violations (74.8% of all stops). Figure 3: Reasons for APD Motor Vehicle Stops 74.8% 17.3% 7.7% Moving Traffic Violation Vehicle Traffic Violation Violation of Law Other Than Traffic 0.2% Pre-existing knowledge (i.e. warrant) Figure 4 below displays the racial/ethnic differences in reason for stop. As shown, whites and individuals of other races/ethnicities were more likely than Blacks and Hispanics to be stopped for moving traffic violations. Blacks 91 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0%  90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% and Hispanics were more likely than whites and individuals of other races/ethnicities to be stopped for vehicle traffic violations and violations of law other than traffic. Due to the small percentage of motor vehicle stops that are based on pre-existing knowledge (0.2%), this information is not presented graphically. Black individuals were more likely than Hispanics, whites, and individuals of other races/ethnicities to be stopped based on pre- existing knowledge (0.5% of Blacks, compared to 0.2% of Hispanics, 0.1% of whites, and 0.1% of others). Figure 4: Racial/Ethnic Differences in Reason for Stop 85.2% 80.3% 71.6% 67.6% 9.4% 11.3% 5.0% 3.0% 18.5% 21.0% 14.7% 11.7% Moving Traffic Violation Vehicle Traffic Violation Violation of Law Other Than Traffic Black (n=10,156) Hispanic (n=23,933) White (30,638) Other (n=3,603) When considering differences in the reason for the stop by individuals’ gender, very few differences emerge. These minor variations are displayed in Figure 5 below, where females are slightly more likely than males to be stopped for moving traffic violations, while males are slightly more likely to be stopped for the other three reasons. Figure 5: Gender Differences in Reason for Stop 76.4% 73.9% 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 7.3% 8.0% 16.2% 17.9% Moving Traffic Violation Vehicle Traffic Violation Violation of Law Other Than Traffic Female (n=24,559) Male (n=43,771) 0.1% 0.2% Pre-existing knowledge (i.e. warrant) 92  Stop Outcomes Figure 6 below shows the percentage of stops that resulted in one of four possible outcomes (verbal warning, written warning, citation, or arrest). Note that these categories are mutually exclusive. In the process of eliminating “duplicate” traffic stops, APD staff created a single stop database that only captures the most serious outcome resulting from a traffic stop. Therefore, it cannot be determined what percentage of stops result in any warning or citation – rather, it can only be reported what percentage of stops had a verbal warning, written warning, or citation as the most serious outcome recorded. In addition, it cannot be determined if multiple warnings or citations were issued to a single individual during a stop. Based on these limitations, we can only report that in 11.2% of the traffic stops recorded, the most serious outcome was a verbal warning; likewise, in 51.7% of stops, the most serious outcome was a written warning. Citations were the most serious outcome in 31.7% of traffic stops, and arrests occurred in 5.4% of all traffic stops. Figure 6: Outcomes for APD Motor Vehicle Stops Arrest, 5.4% Verbal Warning, 11.2% Citation, 31.7% Written Warning, 51.7% Figure 7 and Figure 8 also illustrate the variation in post-stop outcomes by individuals’ race/ethnicity. Blacks were more likely than all other racial/ethnic groups to receive verbal warnings, while whites and individuals of other races/ethnicities were more likely than Blacks and Hispanics to receive written warnings. Hispanic individuals were more likely than individuals from all other racial/ethnic groups to receive citations. Finally, Blacks and Hispanics were more likely than whites and individuals of other races/ethnicities to be arrested. These differences are statistically significant based at a 0.001 level chi-square analysis. That is, the differences noted are likely due to chance no more than 0.1% of the time. Based solely on the statistical significance, these results suggest that a difference exists in the likelihood of receiving warnings, citations, or being arrested, depending on the race/ethnicity of the individual. It is important to recognize, however, that chi-square analyses do not consider other variables when determining statistical significance. In other words, this analysis does not consider any other factors that may be associated with the likelihood of receiving one outcome over another (e.g., seriousness of the offense; time, location, and reason for the stop, age, gender, etc.) that may also vary 93  70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 45.0% 40.0% 35.0% 30.0% 25.0% 20.0% 15.0% 10.0% 5.0% 0.0% by race/ethnicity. Additional multivariate analyses are presented below to take some of the factors into consideration. Figure 7: Racial/Ethnic Differences in Verbal and Written Warnings During APD Motor Vehicle Stops 59.4% 59.5% 45.7% 43.2% Verbal Warning Written Warning Black (n=10,156) Hispanic (n=23,933) White (n=30,638) Other (n=3,603) Figure 8: Racial/Ethnic Differences in Citations and Arrests during APD Motor Vehicle Stops 16.1% 10.4% 10.4% 10.0% 39.3% 30.6% 29.0% 26.3% 7.5% 7.1% 3.9% 1.6% Citation Arrest Black (n=10,156) Hispanic (n=23,933) White (n=30,638) Other (n=3,603) Gender differences for 2020 motor vehicle stop outcomes are displayed in Figure 9 below. As shown, males were significantly more likely than females to receive a verbal warning, but significantly less likely than females to receive a written warning. Males were also significantly more likely than females to be cited and arrested. 94  Figure 9: Outcomes of APD Motor Vehicle Stops by Gender 55.6% 49.5% 30.9% 32.1% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 12.2% 9.5% 6.3% 4.0% Verbal Warning Written Warning Citation Arrest Female (n=24,559) Male (n=43,771) It is important to remember that these are bivariate differences based on gender and race/ethnicity; they do not statistically control for any other variables, including legally relevant behavior, that might explain these differences. Therefore, the information provided in these graphs cannot be used to assess whether differences in outcomes across racial/ethnic and gender groups are due to officer bias. To explore the possibility that other factors might account for racial/ethnic and gender differences in post-stop outcomes, more advanced statistical analyses that control for other legally relevant variables are presented below. Multivariate Regression Models To explore the possibility that other factors might account for racial/ethnic and gender differences in post-stop outcomes, more advanced statistical analyses that control for other legally relevant variables are presented below. A multivariate statistical model is one that takes many different factors into account when attempting to explain a particular behavior. That is, a multivariate model examines variables simultaneously; by statistically controlling for other variables that may be associated with the outcomes of interest, we can better understand the independent impact of individuals’ race/ethnicity. The multivariate analyses to follow examine the impact of demographic characteristics, legal factors, and situational characteristics on traffic stop outcomes (e.g., verbal warnings, written warnings, citations, arrests, and searches). Specifically, we examine the independent impact of race/ethnicity, gender, reason for the stop, whether a search recovered contraband, street type, and time of year on the likelihood of receiving a warning, citation or arrest during a motor vehicle stop. Analyses presented later in the section also examine the likelihood of being searched during a traffic stop. The descriptive statistics for these variables are presented in Table 1. For the analyses that follow, three outcomes are examined for all traffic stops recorded by APD in 2020: 1) warning (created by combining the verbal and written warning categories), 2) citations, and 3) arrests. As noted above and displayed in Table 1 below, combined verbal and written warnings were the most severe sanction for just under two-thirds (62.9%) of all motor vehicle stops recorded by APD in 2020. Citations were the most serious outcome for 31.7% of 95  individuals stopped, and the least common outcome was arrest, occurring in only 5.4% of all motor vehicle stops. For the multivariate models below, we also consider additional independent variables that capture some legal characteristics, incident characteristics, and demographic characteristics of the individuals stopped by the police that may be related to the sanction received. In addition to the reason for the stop (detailed above), we also include whether contraband was seized (e.g., weapons, drugs, cash, or any other type of contraband) from the individual stopped. The seizure of contraband, however, is a rare event (occurring in only 1.8% of all traffic stops) and likely a strong predictor of arrest. Several incident characteristics were also included as independent variables. Roughly 58.9% of traffic stops occurred on an Austin city street (while the remaining 41.1% occurred on state or US highways).79 Given the unique characteristics of 2020, including the response to the COVID-19 pandemic and widespread protests and civil unrest following the killing of George Floyd, the statistical models control for the time of year, measured as quarters (where quarter 1 = January, February, March; Quarter 2 = April, May, June; Quarter 3 = July, August, September; and Quarter 4 = October, November, December). As shown in Table 1, nearly 40% of traffic stops were conducted in Quarter 1 (pre-pandemic), followed by Quarter 2, (21.2%) and Quarter 4 (20.8%), while Quarter 3 only accounted for 18.6% of stops. In the analyses that follow, Quarter 1 is the excluded category; all other quarters are compared to this pre-pandemic timeframe. As noted previously, just under two-thirds of the stops involved males, and the most frequently stopped racial/ethnic group is white (44.8%), followed by Hispanics (35.0%), Blacks (14.9%), and other (5.3%). Table 1: Descriptive Statistics for 2020 Motor Vehicle Stops (n=68,330) Dependent Variables Verbal Warning Written Warning Any Warning (verbal and written) Citation Arrest Search Independent Variables Legal Characteristics Reason for stop: Moving Traffic Violation Reason for stop: Pre-existing knowledge Reason for stop: Vehicle traffic violation Reason for stop: Violation of law other than traffic Contraband seizure 96 Mean 0.112 0.517 0.629 0.317 0.054 0.077 0.750 0.002 0.825 0.173 0.018 SD .316 .500 0.483 0.465 0.227 0.266 0.434 0.042 0.380 0.378 0.134 79 This city street/highway designation is not based on GIS mapping by APD analysts but rather a selection criterion within the arrest reports calibrated with the incident location.  Incident Characteristics City street Quarter 1 (January, February, March) Quarter 2 (April, May, June) Quarter 3 (July, August, September) Quarter 4 (October, November, December) Demographic Characteristics Male White Black Hispanic Other 0.589 0.395 0.212 0.186 0.208 0.641 0.448 0.149 0.350 0.0530 0.492 0.489 0.409 0.389 0.406 0.480 0.497 0.356 0.477 0.223 In Table 2 below, the results of a series of logistic regressions are presented. Model 1 estimates the likelihood that individuals were issued a warning as the most severe outcome of a motor vehicle stop using binary logistic regression estimation, while Model 2 estimates the likelihood of receiving a citation as the most serious outcome, and Model 3 estimates the likelihood of being arrested. Similar to other sections in this report, the appropriate modeling technique for this binary outcome is logistic regression, as the outcome is dichotomous; that is, the outcomes of interest (warnings, citations, and arrests) are equal to one, and all other types of outcomes equal zero. The results reported below include the odds ratios for key relationships. Similar to the relative risk ratios reported in Section 3 for the disproportionality ratios, the odds ratios are interpreted as a change in the likelihood of receiving a particular outcome during a traffic stop resulting from the presence of a specific variable in the encounter (e.g., individuals’ race/ethnicity). The Nagelkerke R Square is a model fit statistic used specifically for binary logistic regression estimation (i.e., the multivariate analysis used in this section); its results range between 0 and 1 and, the larger the number, the better the models fit the data within the analysis. Finally, for each model, there were four reasons for a stop to occur (moving traffic violation, preexisting knowledge, vehicle traffic violation, and a violation of law other than traffic). In the analyses that follow, a violation of law other than traffic is the excluded category – meaning that the other reasons for a stop are compared against a violation of law other than traffic in all models. Model 1: Warnings Reviewing the results presented below in Table 2 for Model 1 (Warnings), individuals stopped for a moving traffic violation were significantly more likely to have a warning as the most serious outcome compared to those stopped for a violation of law other than traffic. If contraband was seized during the stop, individuals were significantly less likely to receive a warning as the most serious outcome. Events that occurred on Austin city streets (compared to highways) were more likely to result in a warning. In the time periods following the pandemic (Quarters 2-4), individuals were significantly more likely to receive a warning as the most serious outcome, compared to those individuals stopped pre-pandemic. After controlling for these legal and incident characteristics, as well as the pre-COVID time period, the results show that Black individuals had a significantly lower likelihood (b = -0.139, p < .001, OR = 0.870) of receiving a warning relative to white individuals (the reference category). Hispanic individuals also had a significantly lower likelihood of being warned compared to white individuals (-0.488, p < .001, OR = 0.614), as did individuals of other race/ethnicity (-0.167, p < .001, OR = 0.847). The primary takeaway from Model 1 is that net of the 97  legal characteristics and seasonal trends that could be measured with these data, Black and Hispanic individuals, and individuals of other race/ethnicity, were statistically significantly less likely to be issued a warning as the most severe outcome during a traffic stop, compared to whites. Model 2: Citations Model 2 provides the estimate for citations as the most serious outcome resulting from a motor vehicle stop. The results show that compared to violations of law other than traffic, individuals stopped for moving traffic violations or pre-existing knowledge were significantly less likely to receive a citation as the most serious outcome. If contraband was seized during the encounter, the result of a citation as the most serious outcome was 4.9 times less likely (b = -1.581, p < .001, OR = 0.206). Events that occurred on Austin city streets were also 1.3 times less likely to result in citations (b = -0.259, p < .001, OR = 0.772). Opposite of the findings for warnings, in the time periods following the pandemic (Quarters 2-4), individuals were significantly less likely to receive a citation as the most serious outcome, compared to those individuals stopped pre-pandemic. These results also show that net of legal characteristics of the incident, and seasonal influences, Hispanic individuals were 1.5 times more likely than white individuals to be cited (b = 0.408, p < .001, OR = 1.504); likewise, individuals of other race/ethnicity were 1.3 times more likely to be issued a citation compared to whites (b = 0.277, p < .001, OR = 1.319). For citations, Black individuals were no more or less significantly likely to receive a citation compared to whites. In short, after controlling for the limited situational and legal factors captured in the APD data, racial / ethnic differences for Hispanics and those of other race/ethnicity were still detected for issuing citations during traffic stops. Model 3: Arrests Model 3 predicts the likelihood that an individual is arrested during a motor vehicle stop. Here the results show that compared to stops for violations of law other than traffic, individuals stopped for moving traffic violations were significantly less likely to be arrested, while those stopped for pre-existing knowledge (which are often warrant-based) were significantly more likely to be arrested. As expected, the strongest predictor of an arrest – the seizure of contraband – showed that individuals who had contraband seized were nearly 52 times more likely to be arrested (b = 3.949, p = .000, OR = 51.885). Events that take place on city streets, compared to state or US highways, were roughly two times more likely to result in an arrest (b = 0.673, p < .001, OR = 1.96). Net of these incident and situational characteristics, Black individuals remained significantly more likely to be arrested during motor vehicle stops (b = 0.510, p <.001, OR = 1.666) compared to white individuals. Likewise, Hispanic individuals were significantly more likely to be arrested (b = 0.428, p < .001, OR = 1.534) compared to whites, and males were significantly more likely (b = 0.337, p < .001, OR = 1.40) compared to females. In contrast, individuals of other race/ethnicity were significantly less likely to be arrested compared to whites (b = -0.863, p <.001, OR = 0.422). In summary, the main findings from Model 3 show that after controlling for some of the known predictors of arrest (i.e., contraband seizures, the reason for the stop, and gender) – along with pre/post COVID influences – Blacks and Hispanics were still 1.7 and 1.5 times more likely to be arrested compared to whites, respectively. Alternatively, individuals of other race/ethnicity were 2.4 times less likely to be arrested compared to whites. 98  Table 2: Multivariate Regressions Predicting Outcomes within Motor Vehicle Stops (n=68,330) Model 1: Warnings B (SE) -.635** (.032) Model 2: Citations B (SE) .224** (.032) Independent Variables B (SE) -2.730** (.065) Model 3: Arrests Odds Ratio Odds Ratio Odds Ratio Intercept 1.251 .065 .530 Legal Characteristics Reason for stop: Moving Traffic Violation Reason for stop: Pre-existing knowledge Reason for stop: Vehicle traffic violation Contraband seizure Incident Characteristics Individuals’ Characteristics City street Quarter 2 Quarter 3 Quarter 4 Black Hispanic Other Male 1.799** (.023) .305 -20.498 -2.438** (.090) .121** (.019) .242** (.025) .413** (.027) .451** (.026) -.139** (.028) -.488** (.021) -.167** (.042) -.085** (.019) -1.391** (.023) -1.421** (.257) .249 .241 -1.639** (.039) 1.159** (.223) .194 3.186 21.229 -- -20.314 -- .087 -1.581** (.102) .206 3.949** (.078) 51.885 6.044 -- -- 1.129 1.273 1.512 1.570 -.259** (.020) -.208** (.025) -.350** (.027) -.459** (.027) .632 -.076 .673** (.045) -.211** (.052) -.343 (.055) .510** (.054) .428** (.043) -.863** (.143) .337** (.074) 1.960 .810 .710 -- 1.666 1.534 .422 1.400 .870 .014 .614 .847 .408** (.021) .277** (.042) .918 .009 -- .772 .813 .705 -- 1.504 1.319 Nagelkerke R Square .324 .286 As noted in the data limitations section, the multivariate models presented above do not include a number of important explanatory variables – including whether the stop was officer-initiated or dispatched, the severity of the offense, whether a third-party requested police action, an individuals’ age, and community characteristics including the location of the stop, neighborhood crime rate, SES, and racial composition. While these findings show that individuals’ race/ethnicity maintains an independent effect on the likelihood of most traffic stop outcomes, we cannot rule out the possibility that the statistical model may be misspecified due to the limited explanatory factors that are reliably measured using APD data (this issue is discussed further in the summary section). The overall strength of the models – as measured with the “Nagelkerke R Square” statistic – shows that the estimation models are of moderate strength across all outcome models (ranging from 0.286 to 0.353). This suggests that the variables included in the analysis present a consistent association to the various outcomes; however, the precision of these variables is likely overestimated because some of the most .353 99  important factors that explain officer decision making are not included, nor are there sufficient proxies for these measures. We also cannot ascertain temporal sequencing with the measures included in these models. For example, a contraband seizure is the strongest predictor of arrest, but searches (and subsequent seizures) are often a product of the arrest decision (i.e., the officer decides to arrest and then searches the individual to ensure safe transport). Thus, we highlight the need for caution when interpreting these results. The measures that could more closely measure factors predicting officer decision making were not readily available in the APD publicly available data for 2020, and the validity and completeness in the non-publicly available data preclude their inclusion in the analyses. Despite these known limitations, the statistical models suggest that, after controlling for at least some of the known predictors of motor vehicle stop outcomes and possible changes in police behavior related to the timing of the pandemic, Black, Hispanic and individuals of other race/ethnicity are statistically significantly less likely to receive warnings and more likely to be cited compared to white individuals stopped for similar reasons. Likewise, Black and Hispanic individuals are statistically significantly more likely to be arrested, compared to white individuals stopped for similar reasons, while individuals of other race/ethnicity are significantly less likely to be arrested during motor vehicle stops compared to whites. Searches In addition to examining the most serious outcomes individuals received during motor vehicle stops during 2020, Kroll also examined whether a search was conducted (regardless of the other outcomes of warnings, citations, and arrests). Of the 68,330 traffic stops recorded during this 12-month period, 5,224 (7.6%) resulted in a search of the driver, vehicle, or passenger.80 Although searches of individuals are statistically infrequent, they are nonetheless a highly visible form of coercive police action that merit further scrutiny. As shown in Figure 10, initial racial/ethnic disparities in searches are evident; Blacks and Hispanics were two times more likely to be searched than whites and individuals of other races/ethnicities. Specifically, of all Black individuals stopped, 11.3% are searched, compared to 10.0% of Hispanics stopped, 5.1% of whites, and 2.3% of individuals of other race/ethnicity. Figure 10: Racial/Ethnic Differences in Searches 11.3% 10.0% 14.0% 12.0% 10.0% 8.0% 6.0% 4.0% 2.0% 0.0% 5.1% 2.8% Black (n=1,143) White (n=1,575) Other (n=101) Hispanic (n=2,405) 80 Due to the limitations of the APD data, the subject of the search – driver, passenger, or vehicle – cannot be clearly determined. 100  Multivariate Regression Model As noted previously, it is important to consider the other factors that predict the likelihood of a search using multivariate regression modeling. The predictor variables in the statistical model for searches are the same as those examined for other traffic stop outcomes, except for contraband seizures – which result from a search, rather than predicting the likelihood of a search. The results of the logistic regression model used to predict the likelihood of conducting a search during a traffic stop are reported in Table 3 below. Compared to the previous models predicting warnings, citations, and arrests, the strength of the model predicting searches is considerably weaker (Nagelkerke R Square = 0.148, compared to previous R Squares values of 0.353, 0.324, and 0.287 for warnings, citations, and arrests, respectively). In short, predicting the likelihood of a search being conducted during a traffic stop using the limited predictor variables available in the APD data suggests that other (unmeasured) factors are likely stronger predictors. Despite these limitations, as shown in Table 3 below, the reason for the initial stop is a significant predictor of whether a search is conducted. Compared to stops for violations of law other than traffic, individuals stopped for moving traffic violations and vehicle traffic violations were significantly less likely to be searched, while those stopped for pre-existing knowledge (which are often warrant-based) were significantly more likely to be searched. Being stopped on a city street (compared to a state or US highway) more than doubled the likelihood of being searched (b=0.762, p<.001, OR = 2.142). Compared to the first quarter of 2020 (pre-pandemic), stops conducted during the next six months (Quarter 2 = April, May and June; Quarter 3 = July, August, and September) were significantly less likely to result in a search. Net of these situational and legal characteristics – and after considering the temporal impact of the pandemic – Black individuals were 2.1 times more likely than white individuals to be searched (b = 0.744, p < .001, OR = 2.10). Similarly, Hispanic individuals were 1.8 times more likely than white individuals to be searched (b = 0.588, p < .001, OR = 1.80). Male individuals were also 1.7 times more likely to be searched than females (b = 0.559, p < .001, OR = 1.749). In contrast, individuals of other race/ethnicity were 1.9 times less likely to be searched (b = -0.619, p < .001, OR = 0.538) compared to whites. Thus, in a pattern similar to arrests, after controlling for some situational and legal characteristics, Black, Hispanic, and male individuals were significantly more likely to be searched during motor vehicle stops compared to whites, while individuals of other race/ethnicity were significantly less likely to be searched. Table 3: Multivariate Regressions Predicting Search within Motor Vehicle Stops (n=68,330) Independent Variables Intercept Legal Characteristics Reason for Stop: Moving Traffic Violation Reason for Stop: Pre-existing Knowledge Reason for Stop: Vehicle Traffic Violation 101 Dependent Variable: Search B (SE) -2.656** (.054) -1.438** (.032) 2.044** (.207) -1.752** (.070) Odds Ratio .070 .237 7.722 .173  Incident Characteristics City street Quarter 4 Individuals’ Characteristics Black Quarter 2 Quarter 3 Hispanic Other Male .762** (.036) -.128** (.041) -.274** (.044) .032 .744** (.042) .588** (.035) -.619** (.106) .559** (.034) 2.142 .880 .760 -- 2.104 1.801 .538 1.749 Nagelkerke R Square .147 As noted previously, the multivariate models presented are missing several important explanatory variables. Although these findings suggest that there are racial/ethnic disparities in stop outcomes, the models are possibly misspecified due to the limited number of explanatory factors that are reliably measured by APD data. Based on the limitations of any one statistical method or data source, it is important to consider multiple data sources and conduct various types of statistical analyses. While any single analysis or data source when explored on its own will present an incomplete picture, multiple data sources and analyses can identify consistency in patterns and trends. Sometimes, the findings across data sources and methods will produce what appears to be conflicting information—explaining human decision-making is not a straightforward science. But when pieced together, these various sources can provide a clearer picture of the existence of, and reasons for, racial/ethnic disparities in policing outcomes. Therefore, the analyses presented below consider some additional context around searches and how the seizure of contraband during these searches can be explored to better identify and understand racial/ethnic disparities. Reasons for Search When analyzing searches, it is important to determine the level of officer discretion involved; in many instances, searches are mandated by police policy or for officer safety purposes. Mandatory searches include those conducted incident to an arrest, or as part of a vehicle inventory procedure. When examining APD searches in 2020, the majority (61.6%) were for mandatory reasons; specifically, 54% were conducted incidental to arrest and 7.6% were based on an inventory of a towed vehicle. Discretionary searches refer to those that are not required by policy. APD discretionary searches represent 46.1% of all searches conducted. The bulk of discretionary searches are based on probable cause (37% of all searches), while a very small percentage of searches are based on plain view (1.0%) or consent (0.5%). As with previous limitations of the stop data noted above, the information available regarding searches is also incomplete. While in practice, a search could be conducted for multiple reasons, only one reason is provided in these data (and it is unknown how the single reason is selected). The mutually exclusive categories for search reason are displayed in Figure 11 below. 102  Figure 11: Reason for Searches 0.5% 1.0% 37.0% 7.6% 54.0% Consent Contraband/Evidence in Plain View Incidental to Arrest Inventory of Towed Vehicle Probable Cause When we consider the three largest categories of reason for a search (probable cause, incident to arrest, and inventory of towed vehicle), statistically significant racial/ethnic differences are evident. Black individuals, and to a lesser degree Hispanics, were more likely than whites and individuals of other races/ethnicities to be searched based on probable cause. Whites, on the other hand, were more likely than all other racial/ethnic groups to be searched incidental to arrest. Finally, drivers of other races/ethnicities were more likely to be searched based on the inventory of a towed vehicle. Two reasons for search are excluded from Figure 12 due to their infrequency (plain view and consent). Figure 12: Racial/Ethnic Differences in Search Reasons 62.1% 54.1% 49.5% 43.2% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 47.9% 36.5% 33.7% 29.9% Incidental to Arrest Inventory of Towed Vehicle Probable Cause Black (n=1,143) Hispanic (n=2,405) White (n=1,575) Other (n=101) 16.8% 7.1% 7.8% 6.9% 103  Contraband Seizure Rates The discovery of contraband during searches—sometimes referred to as the search “success rate,” or “hit rate”—is important to consider when examining potential bias by police officers. Some researchers use the “outcome test” to identify racial and ethnic disparities by examining differential outcomes in search success rates (Knowles, Persico, & Todd, 2001; Ayres, 2001). When applied to police searches, the outcome test is essentially a comparison of the successfulness of those searches, or a statistical comparison of the percentage of searches that result in seizures across racial/ethnic groups. It is hypothesized that if drivers are searched strictly based on legal factors and suspicions unrelated to race, one would expect similar percentages of searches resulting in seizures across racial groups. As with other analytical techniques however, limitations exist which limit the conclusions that can be drawn from the outcome test (Engel, 2008; Engel & Tillyer, 2008). The outcome test is only appropriate for an analysis of motor vehicle stops that result in a discretionary search; mandatory and consent searches violate the assumptions of the outcome test because the decision to search is not based solely on the officers’ discretion (Engel, 2008; Fridell, 2004). Despite this limitation and the fact that the likelihood of discovering contraband during mandatory searches is likely not based on officer skill or training, these search success rates by race/ethnicity are provided for APD’s informational purposes only. No conclusions about racial bias should be drawn from these mandatory search success rate comparisons. Furthermore, any racial/ethnic disparities in hit rates discovered using this method do not necessarily imply officer bias. Combining search reasons based on the presumed level of officer discretion associated with each reason, we create two categories of searches: discretionary and mandatory. Discretionary searches include those based on probable cause, plain view, and consent, which comprised 37.9% of all searches during motor vehicle stops in 2020. Mandatory searches, as noted above, include searches incidental to arrest and based on inventory of towed vehicles; these represented 61.6% of searches during motor vehicle stops in 2020. For the purposes of this analysis, the small number of consent searches (n=24, 0.5%) are excluded. Almost one quarter (23.9%) of all searches resulted in seizures of contraband. When separating discretionary from mandatory searches, discretionary searches have a higher rate of contraband seizure (28.2%) compared to mandatory searches (21.3%). It is important to consider the differences in the discovery of contraband across racial/ethnic groups who are searched. Figure 13 below displays the search success rates for discretionary and mandatory searches by race/ethnicity.81 81 The contraband seizure rates for individuals of “other race/ethnicity” are excluded from this graph due to their small sample size. 104  Figure 13: Discovery of Contraband During Discretionary and Mandatory Searches, by Race/Ethnicity 31.6% 28.3% 24.4% 21.7% 23.2% 18.6% 35.0% 30.0% 25.0% 20.0% 15.0% 10.0% 5.0% 0.0% Discretionary Search Success Rate (n=1,982) Mandatory Search Success Rate (n=3,218) Black Hispanic White As graphically displayed above, Black and Hispanic individuals were more likely than whites to have contraband seized during both discretionary and mandatory searches occurring during APD motor vehicle stops. That is, although Black and Hispanic individuals were searched at a higher rate compared to white individuals, contraband was more likely to be discovered during those searches. This is an especially important analysis for discretionary searches, where we are trying to better understand officer decision making. Some may interpret this finding as evidence that the APD is not biased in their decisions to search. However, we again caution readers that individual officer bias (and in this case, lack of bias) cannot be determined through these statistical analyses. The findings do demonstrate, however, that “unproductive” discretionary searches are not being conducted at a higher rate against minorities; rather, discretionary searches are more likely to result in no contraband seizures for whites who are searched during motor vehicle stops. 5.4 All Arrests January 1, 2017 – December 31, 2020 From January 1, 2017 to December 31, 2020 there were a total of 128,213 arrests. Section 3 examined arrests in the context of whether they resulted in the use of force. Arrests were also examined above as an outcome during traffic stops. In this section, we provide an overview of all arrests, irrespective of whether force was used, or whether they occurred during a traffic stop. As we noted previously in Section 3, arrests steadily declined by 51% between 2017 and 2020. Figure 14 displays the change in the number of arrests by racial/ethnic groups; as shown, the decreases were consistent across racial/ethnic groups.82 The overall percent change in the total number of arrests was a reduction of 49.7% for Blacks, 48.4% for Hispanics, and 53.4% for whites. The percent change in arrests between 2019 and 2020, however, was identical across all three groups (-31%). The difference between 2017 and 2018 drives the overall racial/ethnic differences in percent change in arrests for the full four-year period, as the number of arrests for whites declined 22% between 2017 and 2018, with reductions of 17% and 11% for Blacks and Hispanics, respectively. 82 Arrestees of other races/ethnicities are not included in Figure 14 due to their small numbers of arrests overall (n=971). A similar decline in overall arrests was evident for the “other” category, from 358 in 2017 to 161 in 2020 (percent change - 55.0%). 105 Figure 14: Number of Arrests by Race/Ethnicity from 2017-2020 16,000 14,000 12,000 10,000 8,000 6,000 4,000 2,000 0 100.0% 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 2017 (n=42,098) 2018 (n=35,036) 2019 (n=30,280) 2020 (n=20,799) Black Hispanic White Figure 15 below shows that each racial/ethnic group's percent of all arrests is consistent across years. That is, while the overall number of arrests declined, the racial/ethnic composition of these arrests remained constant across the four-year period. This is important to note, as sometimes police agencies have differential impact for the use of alternatives to arrest across racial/ethnic groups. This does not appear to be the case in Austin, where all racial/ethnic groups are experiencing fewer arrests. Figure 15: Percent of Arrests by Race/Ethnicity 2017-2020 0.9% 35.5% 0.8% 33.4% 0.6% 33.5% 0.8% 33.5% 35.2% 37.6% 36.9% 36.8% 27.5% 27.3% 28.0% 28.0% 2017 (n=42,098) 2018 (n=35,036) 2019 (n=30,280) 2020 (n=20,799) Black Hispanic White Other What is clear, however, is that there is great variation in the use of arrests across APD Sectors. Figure 16 shows the total number of arrests by APD sector from 2017-2020. As shown, the largest number of arrests 106  occurred in Edward Sector (n=17,929), followed by several sectors in the 14,000 to 15,500 arrest range (Charlie, George, Henry, and David). The smallest number of arrests occurred in the Apt Sector. Figure 16: Arrests by APD Sector 2017-2020 17,929 15,351 14,196 13,870 14,691 14,391 15,488 20,000 18,000 16,000 14,000 12,000 10,000 8,000 6,000 4,000 2,000 0 10,319 10,191 569 ADAM APT BAKER CHARLIE DAVID EDWARD FRANK GEORGE HENRY IDA One of the most likely reasons for this variation in arrests is the size and community characteristics of the sectors. As shown in Figure 17 below, however, there are significant differences in the geographic size and total population of each of the APD sectors.83 Adam, Baker, David, Edward, and Frank Sectors all have population estimates that exceed 100,000. The population in Charlie, Henry, and Ida Sectors ranges from approximately 64,000 to 76,000, while George Sector’s total population is estimated to be approximately 12,000 people. Figure 17: 2020 Population Estimates by APD Sector 180,000 167,893 136,062 155,240 146,134 113,937 76,142 70,884 64,284 160,000 140,000 120,000 100,000 80,000 60,000 40,000 20,000 0 12,086 83 Apt Sector encompasses the airport, which has no population, so it is excluded from Figures 17-19. 107  To better account for these factors, Figure 18 provides a geographic representation of the APD Sectors’ arrest rates per 10,000 residents based on 2020 census population estimates. The arrest rates are shown in the map legend next to each sector name in parentheses. As shown, George Sector is a clear outlier; this is the geographically smallest (2.15 square miles) and least populated APD sector (1.3% of population), but it has an arrest rate that is five times the rate of the next closest sector (Ida). Figure 18: Arrest Rates per 10,000 Residents by APD Sector The high rate of arrest in George Sector (along with the high frequency in use of force, and racial/ethnic disparities within that use of force – see Section 3) cannot be explained by the reported crime from this area. Figure 19 below displays comparisons of five items of interest across nine sectors: 1) Percentage of total residential population, 2020 census 2) Percentage of reported criminal offenses, 2017-2020 3) Percentage of reported Part I violent crime, 2017-2020 4) Percentage of APD arrests, 2017-2020 5) Percentage of APD uses of force, 2017-2020 108 l a t o T y t i C f o t n e c r e P 25.0% 20.0% 15.0% 10.0% 5.0% 0.0% As displayed, some sectors have a similar smaller share of the crime and police activity than their share of the residential population (e.g., Adam, Baker, David, and Frank). Other sectors have higher reported levels of crime than their share of the residential population, but police activity that is relatively comparable to their share of crime (e.g., Charlie, Henry, and Ida). However, two sectors have patterns that are substantially different from the other sectors: Edward and George. These two sectors are highlighted in Figure 20 on the next page. Figure 19: APD Sector Comparisons ADAM BAKER CHARLIE DAVID EDWARD FRANK GEORGE HENRY APD Sector IDA % Population % Offenses %Violent Crime % Arrests % Force As shown below, Edward Sector has the highest percentage of reported criminal offenses across the nine sectors (14.9%) and the highest percentage of Part 1 violent crime (19.1%). Nevertheless, the percentages of arrest and use of force in Edward Sector is noticeably lower than the crime rate alone would predict. In comparison, George Sector has the lowest percentage of the overall residential population across these nine sectors (1.3%), but one of the largest percentages of arrests (11.6%) and the highest percentage of use of force (23.1%). Note that the crime rate (per 10,000 population) is relatively high in George Sector (given the small residential population); however, the arrest rate and use of force rate far exceeds what might be expected, even based on this elevated level of crime. 109  Figure 20: APD Sector Comparisons: Edward and George 23.1% 19.1% 14.9% 12.1% 14.2% 11.6% 12.6% 7.2% 5.9% 1.3% 25.0% 20.0% 15.0% 10.0% 5.0% 0.0% % Population % Offenses % Violent Crime % Arrests % Force Edward George It is unknown to the Kroll team what might be driving these differences in George Sector. However, a Patrol Utilization Study produced by the Police Executive Research Forum (PERF) in July 2012 for the APD provided the following description: George sector, which includes the city’s entertainment district, has a higher proportion of proactive activities than the other sectors. Traffic Stop, Subject Stop, On Site Incident, Special Assignment and Directed Patrol are five of the top six activities. The other dispatch activity in the top six is Flagged Down. Because much of the patrol work done in the George sector is foot patrol, George officers are more accessible and therefore are easier to flag down (2012: 32). The study further described George Sector as having “both the highest average number of hours consumed by self-initiated activities and the highest average percentage of patrol officer time consumed by self-initiated activities, at 36.3%” and that “the walking beats in George help facilitate these officer-initiated public contacts. This proactive approach helps George sector officers achieve their objective of crowd control during special events and order maintenance in the entertainment district” (2012: 39). A more recent analysis of calls for service trends by APD Sector (January 2017 to June 2020), conducted by the City of Austin’s Office of Innovation,84 similarly shows that George Sector is one of only two sectors (Apt – covering the airport – is the other) with a higher percentage of officer-initiated activity (54.4%) than dispatched calls for service (45.6%). All other sectors were reported to have between 27% and 39% officer-initiated calls. 84 https://public.tableau.com/app/profile/city.of.austin.office.of.innovation/viz/APDCallsforService-911DispatchedvsOfficer- initiatedJan2017-June2020/OfficerInitiatedTrends3 110  It is possible that the nature of the interactions between police and the public in the entertainment district are different because of the possible intoxication levels of patrons in that area. Of the individuals who experienced force, George has the highest percentage of individuals who were suspected to be under the influence of alcohol and/or drugs (52.8%), compared to the department-wide average of 40.6% of individuals. Again, as shown in Figure 21, the use of force encounters reported in George Sector appear to be different than the other sectors. Figure 21: Percent Use of Force Involving Individual Suspected to Be Under the Influence of Drugs/Alcohol by APD Sector 52.8% 40.5% 38.7% 39.9% 36.9% 34.1% 36.7% 36.6% 32.5% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% ADAM BAKER CHARLIE DAVID EDWARD FRANK GEORGE HENRY IDA While this information regarding the nature of police encounters with the public, deployment patterns, and patrol strategies may be different within George Sector (as articulated in the PERF report), the elevated rates of arrests and uses of force within this sector when compared to other areas of the city should be further examined by APD executives. In addition, the benchmark analyses documented in Section 3 consistently demonstrated that George Sector had higher rates of racial/ethnic disparities in uses of force compared to other sectors, regardless of the benchmark selected for comparison. Therefore, the proactive practices used by officers in this area should be further examined. In addition, APD should continue to focus on use of force training that emphasizes de-escalation skills for interactions with persons acting erratically or in crisis. Custodial vs. Non-Custodial Arrests Across the four-year period, the average percent of arrests that involve taking an arrestee into physical custody (i.e., a custodial arrest) was 74.3%. Figure 22 shows that the percent of custodial arrests steadily increased from 2017 to 2020: 68.0% in 2017, 71.4% in 2018, 75.1% in 2019, and 90.7% in 2020. The large increase in the percent of custodial arrests between 2019 and 2020 was not, however, related to an increase in the raw numbers of custodial arrests, which actually decreased. The percent of total custodial arrests (i.e., where individuals were taken into physical custody) increased between 2019 and 2020 because non-custodial arrests declined more dramatically (-74.5%) than custodial arrests (-17.0%). 111 100.0% 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Figure 22: Percent of Custodial and Non-Custodial Arrests 2017-2020 32.0% 28.6% 24.9% 68.0% 71.4% 75.1% 9.3% 90.7% 2017 (n=42,098) 2018 (n=35,036) 2019 (n=30,280) 2020 (n=20,799) Custodial Non-Custodial Figure 23 shows that there were small but statistically significant differences in the likelihood of the type of arrests that individuals experienced. Blacks (72.5%) were slightly less likely than Hispanics (75.2%) and whites (74.9%) to be taken into custody when arrested, and slightly more likely to be cited and released with a court summons. Figure 23: Racial/Ethnic Differences in Custodial and Non-Custodial Arrests 2017-2020 72.5% 75.2% 74.9% 27.5% 24.8% 25.1% Custodial Non-Custodial Black Hispanic White 112  Contraband Seizures during Arrest Incidents A search was conducted in 79.5% of all arrest events during this four-year period.85 Blacks, Hispanics, and white arrestees were searched at similar rates (80% for Blacks, 79.2% for both Hispanics and whites). The reason for the overwhelming majority of these searches was incidental to arrest (82.6%), but 13.2% were based on probable cause and 3.2% were frisks for safety. It is important to reiterate that a limitation of these data includes that the temporal ordering of searches and arrest is difficult to determine. A search conducted during an arrest incident can be: 1) the reason for the arrest (i.e., search results in contraband seizure, and therefore individual is arrested) or 2) the result of an arrest (i.e., individual is searched prior to taking into custody, and contraband is found at that time). While we presume that searches incident to arrest are “mandatory” searches, and probable cause and frisks for safety are “discretionary” searches, this is an assumption made by the Kroll team. When a search is conducted, APD documents if any contraband was seized; however, contraband is reported based on the “highest ordered item” found, which is weapon. The other categories are alcohol, drugs, cash, stolen property, and other. Because these other categories of contraband would be undercounted if they occurred in conjunction with a weapon seizure (that is, only the weapon would be counted), we report the percent of searches during arrests that resulted in any contraband seizure and weapon seizures specifically. The overall seizure rates during arrests for any contraband was 30.3%, while 8.1% of searches resulted in the seizure of a weapon specifically. There were some racial/ethnic differences noted for these contraband seizure rates, which are displayed in Figure 24 below. The overall seizure rate was highest for Black arrestees (34.5%); Hispanics and whites had a similar overall search success rate (29.1% and 28.5%, respectively), while arrestees of other races/ethnicities had the lowest overall search success rate (25.9%). For weapons seizures only, Blacks and Hispanics had the highest seizure rate at 8.5%, while whites and arrestees of other races/ethnicities had similar rates at 7.3% and 7.4%, respectively. 85 APD reported that 96,691 of 128,213 arrests (75.4%) resulted in a search. Among the 31,522 arrests where no search was indicated, there are 5,056 cases where they completed the data fields for reason for search (n=4,357), contraband (n=27), or both (n=672). Therefore, we consider these additional arrests as having involved a search as well, giving us a new n of 101,747 searches. In 156 cases where there was no search, no search reason, and “nothing” was selected for contraband, we did not include that as a search. We examined the descriptive statistics for the original measure of search and our corrected measure and note the following findings. The increase in percent of searches was similar across racial/ethnic groups (4.8% for Hispanics and whites, 5.4% for Blacks, and 6.1% for individuals of other races/ethnicities). The overall reasons for search varied very little (no more than 0.2%) between the two measures. Contraband found decreased by 1.4% with our corrected measure of search. The difference between the two measures was an increase of 266 cases in 2017, 209 cases in 2018, and 239 cases in 2019. The difference in 2020, however, was 4,342 cases. Recommendations to minimize data inconsistencies such as this are included in Section 7. 113  Figure 24: Racial/Ethnic Differences in Contraband Seizure Rates for Arrestees 34.50% 29.10% 28.50% 25.90% 40.00% 35.00% 30.00% 25.00% 20.00% 15.00% 10.00% 5.00% 0.00% 8.50% 8.50% 7.30% 7.40% Any contraband seizure Weapon Seizure Black Hispanic White Other A limitation of this analysis is that the seizure rates combine both mandatory and discretionary searches; therefore, these analyses are not designed to understand officer decision making for discretionary searches (a typical use of the “outcome test”). Rather these analyses demonstrate that given an arrest situation, Black individuals are significantly more likely than all other racial/ethnic groups to be in possession of contraband (e.g., alcohol, drugs, criminal proceeds, stolen property, etc.), and Black and Hispanic arrestees are slightly more likely to have a firearm seized during arrest events, compared to whites and individuals of other race/ethnicity. Unfortunately, due to additional issues with reliability and missing data, we cannot examine trends or potential racial/ethnic disparities in arrest charges or offense severity between 2017 and 2020 that might better explain these findings. 5.5 Summary This section examined motor vehicle stops that occurred from January 1 to December 31, 2020, the outcomes of these stops (e.g., warnings, citations, arrests), and whether searches occurred during these encounters, as well as all arrests, and searches within arrests, from 2017 to 2020. The APD motor vehicle stop and arrest data suffer from a number of limitations, which are detailed in subsection 5.2, that impacted the extent of the analyses Kroll could provide. Below is a summary of the analyses we were able to perform: • White (44.8%) and Hispanic (35.0%) individuals represent the largest percent of those stopped during motor vehicle stops; 14.9% of motor vehicle stops involved Black individuals, 4.1% were of Asian individuals, and the remaining 1.2% of stops were of other races. • Kroll cannot provide disproportionality analyses comparing the percentages of stops by race/ethnicity to an expected rate of stops (i.e., benchmark) because these types of comparisons are only appropriate for discretionary stops and APD’s stop data does not distinguish officer-initiated from dispatched stops. 114  • The majority of traffic stops were conducted based on moving traffic violations. • The most serious outcomes recorded for all traffic stops were verbal warning (11.2%), written warning (51.7%), citation (31.7%), and arrest (5.4%). • Although there were statistically significant racial and ethnic differences in reason for stop and stop outcomes, bivariate analyses do not take any other variables into account and there may be legally relevant behavior or other factors that explain these differences. • Multivariate analyses of stop outcomes were also conducted to account for multiple factors’ impact on an outcome to understand the independent impact of each variable on outcome. o The most robust, salient, and powerful predictors of stop outcomes were legal and incident characteristics, including reason for the stop, whether contraband was seized, and the location of the stop. o Statistically significant racial/ethnic differences in stop outcomes remained, even controlling for legal and incident characteristics, but were small to moderate in effect size in comparison. ▪ Warnings: Black and Hispanic individuals, and individuals of other race/ethnicity, were significantly less likely to be issued a warning compared to whites. ▪ Citations: Hispanic individuals and those of other race/ethnicity were 1.5 and 1.3 times more likely to be issued citations than white individuals. Black individuals were no more or less likely to receive a citation compared to whites. ▪ Arrests: Black and Hispanic individuals were 1.7 and 1.5 times more likely to be arrested compared to white individuals, respectively, while individuals of other race/ethnicity were 2.4 times less likely to be arrested compared to whites. • Searches during 2020 Traffic Stops o Searches were conducted in 7.6% (n=5,224) of the 68,330 traffic stops. o Blacks and Hispanics were about two times more likely to be searched than whites. o Most searches (61.6%) were for mandatory reasons; 37% of searches were for probable cause. ▪ Blacks, and to a lesser degree Hispanics, were more likely than whites and individuals of other races/ethnicities to be searched based on probable cause. Whites were more likely than all other racial/ethnic groups to be searched incidental to arrest. o 23.9% of all searches resulted in seizures of contraband: 28.1% of discretionary searches and 21.3% of mandatory searches. ▪ Blacks and Hispanics were more likely than whites and individuals of other races/ethnicities to have contraband seized during both discretionary and mandatory searches. o A multivariate model predicting searches showed that legal reason for the stop was the strongest predictor, but even controlling for legal and situational characteristics Black and Hispanic individuals were 2.1 and 1.8 times more likely than white individuals to be searched. 115 • From 2017 to 2020, there were 128,213 total arrests. Arrests steadily declined by 51% 2017-2020; the decrease is fairly consistent across racial/ethnic groups. • Average percent of custodial arrests was 74.3%. o Blacks (72.5%) were slightly less likely than Hispanics (75.2%) and whites (74.9%) to be taken into custody when arrested. • Arrest rates differed across APD sectors, but George Sector was a clear outlier. It is the smallest and least populated APD sector, but it has an arrest rate that is five times the rate of the next closest sector. o Although the crime rate is relatively high in George Sector, the arrest rate far exceeds what might be expected, even based on this elevated level of crime, and by the trends between crime and arrests evident in other sectors. Comprehensive recommendations for improving the collection of motor vehicle stop and arrest data are included in Section 7. 116  6. RECRUITMENT, SELECTION, AND PROMOTIONS President Obama’s Task Force on 21st Century Policing identified diversity within the ranks of law enforcement agencies as a primary avenue for police departments to build trust and improve relationships with the communities they serve (Task Force, 2015). A more diverse police agency helps with increased cultural competency, as diverse candidates bring distinctive skills and abilities to the job; enhanced community relations, as diverse candidates can help improve often strained police-community relations; and an improved organization, as diverse candidates enhance the internal dynamics of a law enforcement agency by “breaking down the police subculture, and by weakening both the occupational solidarity and the social insularity of the police” (Sklansky, 2006). Although overall police diversity has increased in recent decades, the degree of improvement varies considerably by department (Sklansky, 2006). Nationally, the barriers to increasing diversity in law enforcement include strained police-community relations, lack of trust, perceptions of occupational culture, lack of awareness of opportunities, and the use of eligibility and screening criteria and tests that disproportionately impact women and people of color (DOJ/EEOC, 2016, Donohue, 2019). It is within this context that Kroll reviewed APD’s recruitment, selection and promotion policies and processes to determine how they potentially impact historically underrepresented groups such as women and people of color. Kroll also examined career development procedures to determine if there are sufficient career development opportunities to help internal APD candidates achieve their career goals. The processes examined and our findings are summarized below. In Section 7, Kroll recommends measures to improve APD’s policies and processes and increase diversity within all ranks of APD. 6.1 Methodology Kroll reviewed hundreds of documents that are relevant to APD’s recruitment, selection, and promotion processes. Kroll also interviewed civilian and sworn human resources staff to clarify the provided information and to obtain a more in-depth understanding of each process. Kroll focused this analysis on data related to APD’s recruitment, selection, and promotion processes from 2015 to the present since these datasets are the most complete. Kroll notes, however, that as APD frequently changes its recruitment, selection, and promotion processes, any recent changes could potentially impact our findings and recommendations. The statistical analyses used in this section of the report include basic descriptive statistics, bivariate analyses, and multivariate analyses. Descriptive statistics like frequencies summarize quantitative data. Bivariate analyses assess the relationship between two variables (e.g., race and promotion), but do not consider any other factors that might influence recruitment, selection, and promotion outcomes. A multivariate statistical model is one that simultaneously takes many different factors into account when attempting to explain an outcome. In other words, the individual and independent impact of one variable (e.g., race) on the outcome can be measured while holding all other variables in the model constant and controlling for their influence. Although multivariate analysis is a stronger analytical strategy than bivariate analysis, the key weakness of multivariate statistical analysis is that it can only statistically control for those variables that are measured. This is called “specification error” or the error in a statistical model due to the inability to specify all factors that might have an influence over the outcome. Therefore, while researchers can be more confident in multivariate results, the results must be interpreted with this limitation in mind. Finally, sometimes group differences may exist, but they are not statistically significant. This means that we cannot be confident that the difference is not due to chance. 117  Statistical significance is reported as a p-value at 0.05, 0.01, or 0.001 levels, which means that we are 95%, 99% or 99.9% confident, respectively, that observed results are not due to chance. 6.2 APD Demographics and Diversity The primary goal of diversity in law enforcement is for an agency’s officers to represent all community members that they serve. As stated by the DOJ/EEOC Advancing Diversity in Law Enforcement report: Law enforcement agencies fulfill a fundamental role in our society, and in many communities, individual police officers are often the public face of local government. It therefore is critical that our nation’s law enforcement agencies broadly reflect the diversity of the communities they serve (DOJ/EEOC, 2016, p.7). Figure 1 compares the overall racial/ethnic percentages of APD sworn personnel as of March 2021 with the 2019 U.S. Census Bureau population statistics for the City of Austin. According to census data, the racial and ethnic makeup of Austin’s population is approximately 48.3% white, 33.9% Hispanic or Latino, 7.8% Black or African American, 7.7% Asian/Pacific Islander, 3.5% multi-racial, and less than 1% American Indian or Alaska Native. As of March 2021, APD personnel are approximately 66.7% white, 21.8% Hispanic or Latino, 7.5% Black, and 2.5% Asian/Pacific Islander, with less than 1% reporting as American Indian, Alaska Native, or multi- racial. Figure 1: Race/Ethnicity Comparison of APD Personnel and City of Austin Population Statistics 70.0% 66.7% 48.3% 80.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 33.9% 21.8% 7.5% 7.8% 7.7% 2.5% 4.2% 0.7% White Hispanic Asian Other Black or African American Austin Police Department Austin Census Population 118  Figures 2 to 4 present APD sworn personnel demographics by rank (broken down by age), race/ethnicity, and gender.86 As shown in Figure 2, APD personnel in the 20 to 29-year-old range make up close to 20% of the police officer rank and none of the higher ranks. The majority of supervisory personnel and command staff fall in the 40 to 49-year-old age range. As shown in Figure 3, Black officers are underrepresented at the ranks of lieutenant and commander. Hispanic officers are underrepresented at ranks of sergeant, lieutenant, and commander. Asian police officers are also underrepresented at some ranks. Finally, women account for about half the Austin population, but as shown in Figure 4, as of March 2021, are underrepresented at all ranks, accounting for 10.4% of all APD officers and 11% to 12% of corporals, sergeants, and lieutenants. There is a particularly small percentage of females (5.3%) at the rank of commander. Of the six Assistant Police Chiefs, two (33.3%) are women. Figure 2: APD Sworn Personnel Age by Rank, March 2021 44.1% 80.0% 70.0% 60.0% 50.0% 40.0% 20.0% 10.0% 0.0% 55.9% 60.5% 55.1% 57.9% 66.7% 42.1% 36.2% 33.3% 30.0% 26.7% 19.4% 24.2% 20.0% 28.5% 9.5% 0.5% 8.7% 7.2% 2.3% 1.4% Police Officer (n=1,110) Police Corporal/Detective (n=401) Police Sergeant (n=172) Police Lieutenant (n=69) Police Commander (n=19) Assistant Police Chief (n=6) 20-29 30-39 40-49 50-59 60-69 86 These data change regularly due to hiring, promotions and attrition; however, for analysis purposes it was necessary to select a single time-period on which to report. For example, since March 2021 there has been one promotion of a Black officer to Police Commander. Even with that promotion the underrepresentation of Blacks displayed in Figure 3 remains. Furthermore, for ease of presentation, the percentages of those who chose not to disclose their race/ethnicity are not included in Figure 3 (less than 1% of officers, corporals/detectives, lieutenants, and assistant Chiefs, 5% of commanders and 1.7% of sergeants). 119 90.0% 80.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Figure 3: APD Sworn Personnel Race/Ethnicity by Rank, March 2021 79.7% 84.2% 70.0% 64.6% 68.1% 70.3% 24.1% 20.7% 7.3% 3.0% 0.5% 8.0% 1.5% 1.0% Police Officer (n=1,110) Police Corporal/Detective (n=401) 15.1% 9.3% 11.6% 5.8% 1.7% 1.7% 1.4% 1.4% 5.3% 5.3% Police Sergeant (n=172) Police Lieutenant (n=69) Police Commander (n=19) Assistant Police Chief (n=6) White Hispanic or Latino Black or African American Asian Other Figure 4: APD Sworn Personnel Gender by Rank, March 2021 100.0% 89.6% 87.8% 88.4% 88.4% 94.7% 10.4% 12.2% 11.6% 11.6% 5.3% Police Officer (n=1,110) Police Corporal/Detective (n=401) Police Sergeant (n=172) Police Lieutenant (n=69) Police Commander (n=19) Assistant Police Chief (n=6) Scholarly research on this topic confirms that people of color and women tend to be underrepresented in the upper ranks of many police departments (Hyland & Davis, 2019; Shjarback & Todak, 2019; Sklansky, 2006). It is a cause for concern because Commanders hold influential positions in APD over the four divisions of Patrol, Investigations, Administration, and the Training Academy. Without diversity in those ranks, it may be more difficult for subordinate officers who are female or people of color to develop a career Male Female 120 50.0% 33.3% 16.7% 66.7% 33.3%  path to higher leadership, and it is difficult for the community to trust that APD leadership represents their needs. Simply comparing APD’s diversity numbers to Austin’s population data, however, may not be representative of how many historically underrepresented individuals actually want to be police officers. This possibility is explored in the Recruitment section. 6.3 Recruitment According to members of APD’s recruitment and selection staff, historically the department did not have a consistent recruiting program. Currently, however, the Recruiting Unit is headed by one lieutenant, who reports directly to the Commander of Training and Recruitment. The unit is staffed by two sergeants (one in charge of recruiting and the other in charge of intake and background investigations), six temporary civilian background investigators, three permanent civilian background investigators, two administrative assistants, one civilian recruitment coordinator, and 15 officers, eight of whom are background investigators and seven of whom are a team of diverse recruiters that includes women, people of color, and military veterans.87 The 2021 Recruiting Unit Standard Operating Procedures (SOP) requires that recruiting staff “continually assess and evaluate current processes and procedures to increase effectiveness.” As addressed in our recommendations in Section 7, although some data collection and storage methodology have been clearly defined by the APD, some data has not been effectively analyzed due to many factors such as the large size of the data pool or because the priority of some data may be outweighed by others. The Recruiting Unit SOP also requires the development of partnerships with the community and religious leaders as a duty in the recruiters’ job description. At this time, each APD recruiter has a network of community members with which they engage routinely, partly to recruit and partly to stay connected to the community. These interactions occur throughout the year. The recruiting sergeant archives the activity of each recruiting or community event attended by the recruiters. The effectiveness of community events, however, are difficult to assess as no “sign in” documentation is maintained, such as would occur at a recruiting event. According to the 2021 Recruiting Unit SOP, the three overlapping areas of responsibility for the Unit include: • Recruiting—the proactive recruitment of prospective applicants and answering questions related to the hiring process. • Intake—initial processing of all incoming applications to determine basic eligibility and schedule testing of applicants. • Background investigations—conducting a thorough background investigation of each applicant that passes Phase I testing of the selection process. The department has employed or now employs several industry-standard recruitment practices, including internships, participation in job fairs, youth programs, and digital, radio, and print advertising. APD’s use of each of these recruiting efforts is described in further detail below. 87 For budgeting purposes, the Recruiting Unit is authorized eight full-time recruiters, but due to APD staffing shortages, it is currently unable to fill the final recruiter position. 121  APD Internship Program The APD internship program, facilitated by the City of Austin Youth and Family Services, began in 2017.88 It is offered to college or university students interested in possibly becoming a law enforcement officer. The selection process for this program has undergone regular changes and was inconsistent across the historical documentation reviewed by Kroll. This is most likely because the APD staff member who started this program has retired. In some years, the minimum requirements were extensive (e.g., application, criminal history check, employment history, references, interviews, essays), and in other years, less so. In previous years, the internship was directly marketed to Austin Community College, Concordia University, Huston Tillotson University, St. Edwards University, the University of Texas, and Texas State University. As of this report, APD has an MOU in place with each of the above universities excluding Austin Community College. APD is working to add Sam Houston State and Prairie View A&M (HBCU). In 2018, APD took over the internship program, with the Austin Police Association outlining the provisions of this program in their Agreement with the City. Table 1 below shows the number of internship applicants and hires from 2017-2021. In the past, students in this program were paid for their weekly hours in addition to receiving college course credits. However, only nine of the 159 applicants for internships were hired. One reason for this low number of hires in comparison to the number of applicants was that, historically, APD’s budget permitted only four interns per year. Second, according to APD staff members, the internship program was not directly targeted to students who wanted a career in law enforcement. Instead, APD hired college students who needed income and credits and the program did not directly address whether those students were likely to become eventual hires. As one staff member said, “It was a great resume builder for the students, but they did not have an interest in law enforcement.” Table 1: APD Internship Program 2017-2021 Applicants Hired 2017 2019 Fall Semester 2020 Spring Semester 2020 Fall Semester 2021 Spring Semester Total 14 31 37 19 58 159 1 4 4 0 0 9 The lieutenant in charge of recruitment, who took over the internship program in 2018, acknowledged that few interns had applied prior to 2018. In 2022, APD plans to reinstitute the program and administer it internally. The new program director said he would like to use the program to help attract diverse applicants to APD. Prior to implementation, the Recruiting Unit will assess the number of interns that APD can successfully manage to determine if they can increase the number that they hire. The internship is unpaid, but the students receive three college credits. Currently, APD’s internship program is intended for college seniors who are interested in a career in law enforcement; acceptance into the internship is treated as a 88 No additional data was available on APD’s use of an internship program prior to 2017. 122  commitment to apply for the first Academy class for which they are eligible following the completion of the internship. The collective bargaining agreement allows up to 38% of any given cadet class to consist of interns. During the internship, students are exposed to all divisions in APD, which exposes them to the full breadth and depth of law enforcement. Figure 5 below reports the race/ethnicity and gender of the 159 internship applicants and hires from 2017- 2021. As shown, the nine applicants hired included six females (two white, three Hispanic, one Black) and three males (all white). In addition, nearly half of the applicants were women and 57% were people of color. Although a small sample size, this is an encouraging sign that the program can attract diversity. However, as stated earlier, the internship program was not directly targeted to students who want a career in law enforcement, although four interns applied to become police officers with APD, and one was hired. With the anticipated changes, the program has a better chance to reach historically underrepresented groups, which is often an initial barrier to recruitment (DOJ/EEOC, 2016). Figure 5: APD Internship Program Applicants and Hires by Race/Gender 33 28 29 35 40 35 30 25 20 15 10 5 0 10 10 2 3 3 0 1 0 3 0 3 0 0 0 1 0 White Female White Male Hispanic Female Hispanic Male Black Female Black Male Asian Female Asian Male Alaska Nat / Aleut Female Alaska Nat / Aleut Male # Applicants # Hired Military Recruiting, Job/Career Fairs, and Information Sessions According to the 2021 Recruiting Unit SOP, for each recruiting event, recruiters are required to document the following information in an activity report and submit it to their supervisor within three days of the event: location, date, duration, cost, target population, number of contacts and their names, as well as their self- reported race, gender, and age-range. The prospective applicant information is gathered via a Google form on APD iPads. Although APD later attempts to add information in their master spreadsheet on whether a prospective applicant applied, in practice this is difficult. As reported by APD Recruiting staff: When the prospects log their information into APD’s tablets, they often misspell their names, use nicknames, include typos in their email addresses, or intentionally log erroneous information to avoid potentially being spammed. When an applicant applies to APD, they use a different software program to create a profile and apply. Matching their official information with the information provided during recruiting events is challenging. Due to the erroneous sign-in information, the totals attained from matching sign-ins to applications underestimates how many applicants applied after meeting us at a recruiting event. 123  APD recruiters attend job and career fairs, host information sessions open to the public, and set-up information tables at a variety of community events, with a focus on military-related sites, educational institutions, and other events that often are focused on historically underrepresented groups. Military Recruiting. APD extensively recruits potential APD applicants from the different military branches and installations. Military-related recruiting occurs in several different forums, including at military installations’ career fairs, Soldier for Life/out processing centers, and education centers, all of which are designed to assist individuals transitioning from military service to the civilian sector. Military dependents are included in the pool of potential job seekers. Several of the current APD recruiters have military experience and can connect with and understand the process for military members exiting their respective branches. APD recruitment occurs at military installations throughout the United States, including Fort Sill, Oklahoma; Fort Polk, Louisiana; Fort Carson, Colorado; Camp Pendleton, California; Fort Campbell, Kentucky; Fort Gordon, Georgia; and several military installations in Texas, including Fort Hood, Fort Bliss, Fort Sam Houston, and Camp Mabry. Since 2015, Fort Hood job fairs have consistently produced a high number of attendees. University Job Fairs. Several university job fairs also produced high numbers of attendees, including both in-state (Texas State, the University of Texas at El Paso, University of Houston, University of North Texas, Sam Houston State, Texas A & M) and out-of-state locations (John Jay and Monroe College in New York, Northern Colorado, and Southern Mississippi). In 2015-2017, the job fairs that focused on women’s groups produced more attendees than in recent years and this holds true for job fairs at religious organizations. Information Sessions. In addition to community job and career fairs, APD periodically hosts free information sessions (e.g., APD Informational Sessions held in Texas, New York, and Puerto Rico, and APD Women’s Information Session in Austin) open to the general public, where recruiters share information about the city, the department, and the hiring process, followed by an opportunity for questions and answers. APD recruiters also complete class talks at various schools (e.g., high schools and colleges) where they speak with students in a classroom environment about the requirements for a job with APD and the hiring process. From January 2016 to March 2020,89 the Recruiting Unit staffed 556 events at 126 different locations, including 60 at colleges/universities, 40 at career fairs, and 26 at military installations. During this time, a total of 13,351 potential applicants were recruited across all types of recruiting events; of these, only 1,298 (9.7%) actually applied. As noted above, however, the Recruiting Unit has had difficulty accurately matching information gathered from prospective applicants at recruiting events with the online applications that are later completed; they believe their tracking spreadsheet is undercounting actual applicants. Figure 6 below displays the percent of all recruits and those who applied by the type of event at which they were recruited. Nearly half (49%) of all recruits were contacted at colleges and universities, followed by military-related recruiting events (28.6%) and general career/job fairs and information sessions (22%). Interestingly, 46.1% of applicants were recruited at general career/job fairs or information sessions, while just 28.2% of applicants had been recruited at colleges and universities. A similar percentage of applicants came from military-related recruiting events compared to the percent of all recruits. 89 In 2020, due to the COVID-19 pandemic, APD suspended in-person recruiting beginning in March 2020. They resumed in-person recruiting events in late 2021; Kroll does not have those data at this time. 124  Figure 6: Type of Recruiting Events for APD Recruits January 2016-March 2020 49.0% 46.1% 28.2% 22.0% 28.6% 25.7% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Colleges & Universities Career & Job Fairs, Info Sessions Recruits who Applied Overall Recruits Military-Related Figure 7 below displays the race/ethnicity and gender of all those recruited and those recruits who applied from January 2016 to March 2020. As shown, Hispanics and whites each represented approximately a third of the potential applicants, followed by Blacks at 26%. In total, two-thirds of those who expressed interest in applying to APD at a recruiting event were people of color. Of the 1,298 recruits who applied, the percentages of white and Hispanic recruits were approximately 3-5% higher than in the overall recruitment pool, while the percent of Black recruits dropped by 7.4%. Approximately two-thirds of potential applicants were male, and one-third were female. Of the 1,298 recruits who applied, however, the percentage of males increased by about 10%, while the percentage of females decreased by about 10%. As with the applicants to the internship program, the recruitment pool was appropriately diverse. Although the percent of recruits who applied was less than 10%, the resulting applicants were also diverse, if slightly less so. Overall, these data suggest a positive indication of interest in employment with APD from individuals in historically underrepresented groups. Figure 7: Demographic Characteristics of APD Recruits January 2016-March 2020 80% 70% 60% 50% 40% 30% 20% 10% 0% 36.2% 33.2% 38.6% 33.4% 26.1% 18.7% White Hispanic or Latino Black or African American 77.7% 67.1% 32.9% 22.1% 4.0% 3.4% 3.3% 2.9% Asian Other Male Female Overall Recruits Recruits who Applied 125  Figure 8 shows the recruitment locations for all sign-ins by gender, while Figure 9 shows the recruitment locations by gender of only those who applied. As shown in Figure 8, females are more likely than males to be recruited from colleges and universities, while males are more likely than females to be recruited as part of military-related recruiting events. Males and females are equally likely to be recruited from recruiting job fairs. Interestingly, as shown in Figure 9, of the recruited females who applied, they were most likely to apply if they were recruited from job fairs (61.7% of female applicants). Despite being more likely to be recruited from colleges and universities, females were slightly less likely than males recruited from colleges and universities to apply (26.1% compared to 28.8%). Males were also most likely to apply if they were recruited from job fairs (41.6% of male applicants). Figure 8: Comparison of Recruitment Locations for All Sign-Ins by Gender 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 63.2% 42.0% 21.8% 22.8% 35.3% 15.0% University/College Career & Job Fairs, Info Sessions Military-Related Female (n=4,360) Male (n=8,909) Figure 9: Comparison of Recruitment Locations for Applicants by Gender 61.7% 41.6% 26.1% 28.8% 29.5% 12.2% University/College Career & Job Fairs, Info Sessions Military-Related Female (n=287) Male (n=1,009) Figure 10 below shows the recruitment locations for all sign-ins by race/ethnicity, while Figure 11 shows the recruitment locations by race/ethnicity of only those who applied. As shown in Figure 10, recruitment from colleges and universities is the most likely recruitment location across all racial/ethnic groups, but 126  60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Blacks are more likely than all other racial/ethnic groups to be recruited at colleges and universities. They are, however, least likely to be recruited at general job fairs. Interestingly, as shown in Figure 11, Blacks were more likely to apply if they were recruited from job fairs (41.2%) or military-related events (36.6%) in comparison to college/university events (22.2%). Hispanics were more likely than all other racial/ethnic groups to be recruited at general job fairs, but they were also highly recruited from colleges and universities. They were the least likely of all racial/ethnic groups to be recruited from military-related recruiting events. Hispanic and white applicants were also most likely to apply if they were recruited from job fairs (49.3% and 45.1%, respectively). Figure 10: Comparison of Recruitment Locations for All Sign-Ins by Race/Ethnicity 52.6% 49.4% 47.3% 47.7% 41.4% 27.7% 21.7% 21.3% 18.5% 15.6% 31.0% 33.8% 31.8% 37.3% 23.0% University/College Career & Job Fairs, Info Sessions Military-Related White (n=4,375) Black or African American (n=3,434) Hispanic or Latino (n=4,393) Asian (n=524) Figure 11: Comparison of Recruitment Locations for Applicants by Race/Ethnicity 54.5% 49.3% 45.1% 41.2% 32.1% 28.1% 28.9% 22.2% 18.2% 36.8% 36.6% 34.2% 26.8% 27.3% 18.6% University/College Career & Job Fairs, Info Sessions Military-Related White (n=470) Hispanic (n=501) Black or African American (n=243) Asian (n=44) Other (n=38) APD Youth Programs In 2008, APD began participating in the Explorer program, a national not-for-profit program funded by donations that has served police departments throughout the United States for over 70 years. The Explorer program is a hands-on, co-ed program for young adults between the ages of 14 and 21 who are interested in pursuing a career in law enforcement. There is no cost for participants, but they must pass a background check before being accepted into the program. As part of the APD Explorer program, candidates attend a 127  four-week long orientation with the Central Texas Explorer Academy. The program is designed to build confidence, character, good citizenship, and physical fitness in its participants while exposing them to the participating law enforcement agency. The mission, vision, and goals of the Explorer program are strongly supported by several prominent national law enforcement-related organizations, including the International Association of Chiefs of Police, National Association of Black Law Enforcement Executives, Hispanic American Police Command Officers Association, National Association of Women Law Enforcement Executives, and International Association of Women Police. Each of these groups is committed to marketing the recruitment of Explorers, which is a potential strategy for diversifying the law enforcement workforce with qualified officers (DOJ/EEOC, 2016). In 2017, APD created the Explorer program/Police Activities League (PAL) SOP. The Explorer/PAL Unit was intended to be staffed with one sergeant, five senior police officers (two Explorer and three PAL), volunteer officers and civilian employee advisors, and APD-approved volunteers with a valid ID/City of Austin Volunteer Badge. Currently, however, this function falls under APD Recruiting and is currently staffed with one sergeant and one corporal that are not part of the Recruiting Unit staff. According to APD staff, the program falls under Recruiting because it is considered one of several community intervention programs, which are handled by Recruiting. PAL focuses on after-school programs, direct engagement and programming in the Boys and Girls Club of Austin and the City’s recreation centers. It is focused on at risk youth 8-18 years old and offers positive interactions with the police through sports and other activities. While PAL may attract youth to law enforcement as a career, the focus of the program is directing youth into constructive and productive activities, not necessarily to recruit them. Currently, Kroll has no data on how or whether this program is impacting APD recruitment. Marketing and Advertising Table 2 below summarizes the APD Recruitment Division’s online marketing, print advertising, and other advertising utilized to attract recruits. APD maintains a website solely dedicated to recruiting that includes a special focus on women and veterans. Information provided includes a description of the steps in the hiring process, preparing for the training academy, job opportunities within APD, Frequently Asked Questions, and professional profiles and contact information for the Recruiting Team. These various strategies cover industry-standard methods (DOJ/EEOC, 2016). Print Advertisements Signs/Billboards Table 2: Marketing and Advertising 2014-2021 • AustinFit Magazine (2017) • Austin Relocation Guide (2020) Magazine and website • Austin Times (2017) • Austin Woman Magazine (2016-2017) • Equal Opportunity Employment Journal (2017) • Fort Hood Sentinel (2017-2019) • Homeland Magazine (2017) • Daily Herald Veteran’s Day Issue (2018) and website (2018-2019) • NAACP Annual Freedom Fund Banquet Advertisement (2014-2019) • NOKOA The Observer Weekly Paper (2017) • National Women of Achievement - Austin Chapter Program Ad (2018) • US Veterans Magazine (2015) • Airport (2018-2019) • Round Rock Express (2015-2017) • Local Billboards at strategic locations across Texas (2020-2021) • Wallscape at APD Headquarters (2021) • Join APD advertising vehicle • Bumper stickers promoting recruiting website on all patrol vehicles 128  Radio/TV/Theater • NCM Local/Regional Movie Theaters (2019-2020 15 sec ads in 8 theaters) • Zoom Media (Gym TV) • Local interviews with various news/radio outlets • Organic social media posts on Facebook, Instagram and Twitter • Paid Facebook advertisements for hiring and information sessions Social Media Internet throughout the U.S. (2017-present) • Glassdoor, TikTok, SnapChat, LinkedIn • KVUE website (2018 & 2019) • Officer.com (2019) • PoliceOne.com (2017-2019) • Saludos.com (2017) • Univision (2017-2019) • apdrecruiting.org website Effectiveness of Different Recruiting and Advertising Strategies APD asks each applicant at intake how they were recruited or how they became interested in the Austin Police Department. That information is cataloged in APD’s applicant management software and reviewed periodically to assess the effectiveness of marketing campaigns and recruiting events. Applicants’ specific answers are categorized as one of the following seven types of advertising or recruiting sources: internet, personal reference, job fair, self, TV/radio, sign/print ad, and social media.90 Figure 12 below graphically displays the percent of different answers provided by the 5,992 applicants from 2015 to 2021. The top three answers for how applicants were recruited or became interested in APD were: Internet (39.1%), • • Personal reference (19.1%) • Social media (19.0%) Figure 12: Percent of Type of Advertising or Recruiting Sources Reported by APD Applicants Overall from 2015 to 2021 39.1% 45.0% 40.0% 35.0% 30.0% 25.0% 20.0% 15.0% 10.0% 5.0% 0.0% 19.1% 19.0% 11.4% 4.0% 1.2% 6.2% Internet Personal Reference Job Fair Self TV/Radio Sign/Print Ad Social Media 90 Kroll notes that APD does not analyze the effectiveness of providing this information, although the information is brought to recruiting events and shared online. 129 50.0% 45.0% 40.0% 35.0% 30.0% 25.0% 20.0% 15.0% 10.0% 5.0% 0.0% Figure 13 graphically displays the percent of each recruiting/advertising source by year from 2015 to 2021. As shown, the internet remains the most frequently cited source by applicants across all years. The influence of social media and sign/print advertising have increased over time, while the percentage of applicants reporting they were recruited or became interested in APD by personal references, job fairs, self, and TV/radio have decreased between 2015 and 2021. Figure 13: Percent of Type of Advertising or Recruiting Sources Reported by APD Applicants by Year, 2015-2021 Internet Personal Reference Job Fair Self TV/Radio Sign/Print Ad Social Media 2015 2016 2017 2018 2019 2020 2021 Table 3 below displays the specific answers provided within each of the seven categories and Table 4 provides the most frequently mentioned answers for the three most commonly reported advertising or recruiting sources. As shown in Table 4, the most common source of internet related recruiting or advertising was APD’s website (58.5%). APD employees were the most common source of personal references (72.6%). Finally, the most commonly reported source of social media recruiting, or advertising was Facebook (50.2%), followed distantly by Instagram, Twitter, and YouTube. Table 3: Specific Advertising or Recruiting Sources by Category as Reported by APD Applicants 2015-2021 Specific Sources within Each Category Advertising or Recruiting Source Internet APD website (apdrecruiting.org), Austin American Statesmen, Austin City Jobs, BlueLine.com, CareerBuilder, CLEAT.org, Google search, GovernmentJobs.com, Indeed.com, MilitaryExits.com, Monster.com, Officer.com, PoliceJobs.com, PoliceLink.net, PoliceOne.com, TCOLE website, USAJobs.com, WorkAustin.com Personal Reference APD Employee, Friend/Family Member/Co-worker Job Fair Self Events at various universities, military installations, city-wide job fair events in Austin and other cities (e.g., Chicago, New York) Austin local resident, Intern, Police Explorers, prior applicant, Ride-Along, walk-in/phone call 130  TV/Radio Sign/Print Ad FoxNews, KXAN, KVUE, Telemundo, Univision, Various local radio stations Ad on APD police vehicle, billboards in various locations, Capital Metro Bus, newspaper, poster at Fort Hood Social Media Email blast, Facebook, Instagram, LinkedIn, Reddit, Twitter, YouTube, Table 4: Percent of Specific Advertising or Recruiting Sources Reported by APD Applicants 2015-2021 1st 2nd 3rd 4th APD Website (58.5%) Google (11.2%) Work Austin (10.1%) Police One (6.2%) APD Employee (72.6%) Facebook (50.2%) Friends / Family or Co-Worker (27.4%) N/A N/A Instagram (19.8%) Twitter (16.8%) YouTube (7.5%) Advertising or Recruiting Source Internet (n=2,340) Personal Reference (n=1,145) Social Media (n=1,140) Recruitment Observations Although APD’s recruitment program is fairly comprehensive, there is room for improvement. The functions of APD’s Recruiting Unit are three-fold: recruiting, intake, and background investigations. In carrying out its functions, the Recruiting Unit encourages creativity. This is important because recruiting qualified diverse applicants can be challenging due to competing job markets. APD does a good job of reaching applicants who are not in Austin. APD’s recruiting strategies are wide- reaching and attract a large and diverse audience. However, as discussed further below, APD needs to better track recruitment data. During Kroll’s interviews with past APD cadets, it was apparent that some cadets had difficulty discerning whether the recruiter and the background investigator were one and the same person during their interactions. Others reported that they were not assigned to a recruiter, or that their recruiter was either unavailable, unable to provide the assistance the recruits were seeking, or did not feel the need to request assistance.91 Kroll also learned during our interviews with former and recent cadets that some cadets were told that the recruiter could not always provide the information requested so as not to give one recruit an advantage over another. This was confusing to cadets and sometimes left them without needed assistance. Typically, an applicant will meet one or more recruiters at a recruiting event prior to applying, but then may not interact with them much after that event. Although Kroll recommended that APD recruiters seek to develop stronger relationships with applicants during the applicant process, APD indicated that the ratio of recruiters to applicants makes this practically impossible. APD acknowledged, however, that following the first set of selection tests (written and physical ability tests), which eliminates about half the applicants, it may be possible for recruiters to develop stronger relationships with the applicants at that time. While doing so will require some logistical coordination (e.g., assigning patrol officers to assist with emails, Instagram, 91 It is possible some of these interviews do not reflect the current practices of the Recruiting Unit. 131  other means of communication), APD agreed that it would allow recruiters to assist applicants in completing the entire selection process (i.e., submitting required documents, attending scheduled events, etc.). Kroll notes that APD had suspended its participation in the Explorer program as of this report. This is unfortunate, as Explorer is a nationally renowned law enforcement recruitment program that can help increase minority and female participation in law enforcement (EEOC, 2016). APD indicated that it intends to bring back the program when the budget permits. Kroll’s analysis of internship applicants and APD’s general recruitment efforts at colleges, universities, and military installations, as well as general job fairs and open information sessions, reveals that APD is reaching a large and diverse audience when it recruits, and that people of color and women are interested in becoming police officers. There is, however, variation across the types of events in producing applicants from the pool of recruits; nearly half of all applicants were recruited at general career and job fairs or information sessions despite those types of events producing just 22% of the total number of recruits. Furthermore, irrespective of gender and race/ethnicity, recruits who applied were most likely to do so if they had been recruited at general career and job fairs or information sessions. Kroll found that APD analyzes the success of each recruiting activity by printing a recruiting report (these data are maintained online so reports can be produced at will) that indicates the number of people who viewed social media posts (i.e., YouTube videos, Facebook and Instagram posts), the number of people who attended recruiting events by state and city (nationally), and cities within Texas. These same data are then analyzed to determine the top recruiting events by rank-order according to the count of people who attended. In addition, at the test site for Phase I, a survey is administered to applicants to determine how they were recruited. Although the survey results provide some indication of how applicants were recruited, only ten percent of those who expressed interest in applying to APD at a recruiting event applied. It is unclear why there is such a high attrition rate at this point in the process. As noted above, the Recruiting Unit has had difficulty accurately matching the information gathered from prospective applicants at recruiting events with the online applications that are later completed, and they believe the tracking spreadsheets are undercounting actual applicants. This could be contributing to the low applicant percentage. It is also possible that when recruits become more informed about the hiring process, they simply change their minds or discover they would be ineligible. Furthermore, Kroll found that the recruitment pool demonstrates greater racial/ethnic and gender diversity than the current APD workforce. As noted in our recommendations, creating a Realistic Job Preview (RJP)92 might assist in enhancing the fit and qualification of applicants. A Realistic Job Preview is a recruitment and hiring tool that highlights the positive, negative, typical, and unique aspects of a job in the interest of transparency and increasing the likelihood of a good employer-employee match (Dahm, 2006). The U.S. Office of Personnel Management (OPM) indicates that assessing fit is still in its infancy. However, they provide some options for attempting to better assess fit: Although job-fit can be measured with interviews or other instruments, job-fit instruments are typically administered to applicants in the form of self-report questionnaires or surveys. Technological advancements of the Internet have made it easier to administer job-fit measures on-line, or as a possible feature to an agency or company's website. An example item from a job-fit measure is: "I prefer a work 92 The U.S. Office of Personnel Management: https://www.opm.gov/policy-data-oversight/assessment-and- selection/other-assessment-methods/realistic-job-previews/ 132  environment which doesn't demand constant adaptation" (1 = strongly disagree, 5 = strongly agree).93 As a cautionary note, the fit information gathered must be carefully reviewed for legitimacy since the questions are subject to response bias; that is, applicants may tell the potential employer what they want to hear. Although APD uses “informative videos” and provides additional online data to inform applicants about the department and selection process, a Realistic Job Preview may better aid recruits and potential applicants in self-evaluating whether they are truly a good “fit” for the job; in the long term, it can assist the department in reducing attrition. The next section examines how well diversity in recruitment translates to a diverse applicant pool, and ultimately, diversity in hiring. 6.4 The APD Selection Process The APD selection process is based on the eligibility and testing processes described in Article 14 of the Collective Bargaining Agreement between the City of Austin and the Austin Police Association (most recent effective date: November 2018). The APD Recruiting Website currently provides comprehensive information about the hiring process, including how to complete the application, document submission, common disqualifiers, and required testing. The website also provides a candidate orientation guide and free practice exam questions for applicants. Since 2015, an applicant seeking to become an APD police officer must, at a minimum: 1) be a U.S. citizen (by birth or naturalization), 2) be between the ages of 20.5 and 45 years at time of application, 3) possess a High School Diploma or G.E.D., 4) possess a valid driver’s license, and 5) read, write, and speak English (added in 2020). In 2017, the paper application process was replaced with an online application process. The newest online process was designed by iCIMS, a recruiting software company. Current applicants must also supply an Experian© credit report dated within 30 days of the online application. All applications undergo an initial Intake Review process, which checks for completeness, required documentation, basic eligibility, and potential disqualifiers. Applicants who meet the minimum qualifications are scheduled for Phase I testing (described below). If the application review reveals that an applicant fails to meet any of the five minimum criteria, they are automatically eliminated from the process. Those who are disqualified by the initial intake review are notified and those who are only temporarily disqualified are notified of an eligibility date to reapply. An applicant may appeal a disqualification factor, and if that appeal is upheld, they are eligible to reapply for the next testing cycle. An applicant may reapply to APD a maximum of three times. The seriousness of the disqualification factor determines when an applicant may reapply. A less serious disqualification factor (e.g., missing application information) expedites the reapplication process, whereas a more serious disqualification (e.g., falsifying information) extends the reapplication process or may exclude the applicant permanently. 93 https://www.opm.gov/policy-data-oversight/assessment-and-selection/other-assessment-methods/job-fit-measures/ 133  Since 2015, the APD selection process has changed or adjusted some of its components. Through interviews with APD and City human resources staff, Kroll finds that the purpose for these changes was to (1) increase applicant retention by revising cumbersome or difficult requirements that eliminated a considerable proportion of otherwise potentially qualified applicants; (2) increase diversity of the applicant pool by revising requirements that may have unintentionally disqualified applicants from historically underrepresented groups; and (3) increase the efficiency of the selection process for applicants and APD recruitment and selection personnel. Table 5 summarizes the current steps in the APD selection process that follow the initial application and intake review. The purpose of these steps and what they entail are summarized below. We then provide statistical analyses regarding demographic differences in the various steps of the selection process. Table 5: APD Selection Process • • • • • • Cognitive Ability (Written) Test that includes a reading comprehension component Psychological Questionnaires Physical Ability Test Background History Statement Document submission Communication with assigned background investigator as needed Psychological Interview • • Oral Board Interview • • Medical Exam and Drug Testing Polygraph Examination Phase I Testing Background Investigation Phase II Testing Phase I Testing Applicants who successfully pass the Intake Review process move on to Phase I testing, which includes the written test, physical ability test, and psychological questionnaires. Frequent changes have been made to the written test and the physical ability test (PAT) since 2015. The primary change in the written test occurred in 2018 when the Nelson-Denny Reading Test (NDRT) was replaced by the National Police Select Test (NPST).94 The NDRT is a reading comprehension and vocabulary test, while the NPST consists of reading comprehension as well as four additional sections—mathematical reasoning, writing ability, situational judgment and human relations, and reasoning / analyzing—designed and validated to assess skills and abilities necessary for job performance as a police officer. The PAT changes have been more frequent and are summarized in Table 6. 94 The NPST was developed by an outside vendor Fire & Police Selection, Inc. 134  Table 6: Physical Ability Test (PAT) Changes, 2015-2021 Physical Requirements of PAT Year 2015 Push-Ups Sit- Ups Vertical Jump 1.5 Mile Run 300 Meter Run 2016 2015 test plus Row Test 2017-2018 Speed 2000m Row Test (60th Percentile for their age/sex/weight) December 2018-2021 APD-developed Fitness Test (timed): 1.5 Mile Run 500m Row Average score of 70% for age/sex needed to pass; Average score 60-69% eligible for retest within 30 days An APD Commander, who has since retired, developed APD’s physical ability test. It was based on the Texas DPS fitness assessment and combines scoring for a 1.5-mile run and 500-meter row test. Kroll could not obtain APD data demonstrating the validity of this test. Finally, applicants complete two psychological questionnaires: the Minnesota Multiphasic Personality Inventory 2 (MMPI2) and the Inwald Personality Inventory (IPI).95 These are designed to assess different personality and psychological factors that are relevant to law enforcement. A psychologist reviews the results and asks relevant follow-up questions if the applicant continues to the psychological interview step of Phase II. Background Investigation Applicants who successfully pass Phase I testing move on to the Background Investigation Process. Applicants are required to complete the Background History Statement (BHS), which has varied in length and content over the years, but which currently asks for information about an individual’s last 10 years of work and volunteer history, residential addresses, and personal relationships; a lifetime history of vehicle accidents, citations, and warnings; contact information for all immediate family members; and five personal references. These requirements are consistent with TCOLE Rule 211.1(a)(8). Applicants are also required to submit copies of the following documents: • Driver’s License, Social Security Card, official birth certificate, and naturalization documents, if applicable. • High school diploma, transcript(s), or GED; college transcript(s), if applicable. • Official marriage license and all official divorce decrees and addendums, if applicable. • Last two earnings statements/paystubs. If unemployed, the applicant’s last filed tax return. • All DD-214s (Certificate of Release or Discharge from Active Duty), if applicable. • Past law enforcement personnel files and any internal affairs investigations involving the applicant if he or she has prior law enforcement experience. 95 The MMPI2® was developed by Yossef S. Ben-Porath and Auke Teilegen. The IPI® was developed by PSI. 135  Failure to submit a complete set of documents is an automatic disqualifier. The background investigator reviews the BHS and supporting documentation for completeness and potential causes for disqualification based on personal character, finances, false statements, or unsuitability. Prior to 2018, the BHS and its accompanying documentation were required in their entirety at the time of the application. After 2018, the order of submission changed to move the BHS to after Phase I testing. Kroll agrees that the new sequence of submissions makes sense for many reasons. First, it is time-consuming and challenging for applicants to obtain all the information requested for the BHS. Applicants interviewed by Kroll reported that it took them anywhere from two to three months to complete this process due to the extensive nature of the information requested, which may have deterred potential applicants from completing the process. Second, with the later submission of these documents, it is being requested only from individuals that have successfully completed Phase I testing. As a result, the applicant pool should be significantly smaller, allowing APD to establish a business relationship with each applicant by assigning them to a specific recruiter, who can personally advise applicants on the BHS and document submission process. This should enhance the recruiting process by helping to retain qualified applicants who may be struggling with the submission process. Phase II Testing Phase II testing includes the psychological exam, oral board interview, polygraph examination, and medical exam/drug screen. The psychological exam is conducted by an APD clinical psychologist who reviews each applicant’s results on the previously taken personality inventory tests (MMPI-2® and IPI®), and then conducts a one-on-one interview with the applicant. The purpose of this evaluation is to determine whether the applicant meets the mental and emotional standards necessary to successfully serve as an APD Academy cadet and eventually an APD officer. The oral board interview is designed to assess applicants’ knowledge, skills, and abilities considered essential to effective employment as an APD officer. According to the 2020 Recruiting Section Operations Manual, the interview board consists of three APD personnel: one officer, the background investigator, and a chairperson who holds the rank of lieutenant or sergeant. The polygraph exam is designed to verify the truthfulness of the BHS and the corresponding documents submitted and serves as another method to assess whether applicants meet APD’s Personal Character, Finance, Honesty and Suitability requirements. Finally, the medical exam involves a general fitness assessment and standard physical. It is conducted by APD Wellness staff. The primary purpose of this exam is to ensure that the applicant can safely perform as a police cadet and officer. The medical exam is typically not a major disqualifier unless a person has poor vision, hearing or some other relevant medical condition that prohibits them from becoming a police officer. Applicants also submit to drug screening at this point in the process. Demographic Differences in the Selection Process Kroll analyzed race/ethnicity, age, and gender differences in the APD selection process to determine if any group is adversely impacted by APD’s selection procedures. Kroll’s analysis focused on the 6,601 total applicants for cadet classes 130-143, which included 5,890 applicants who were disqualified at some point during the process, and 711 applicants who became cadets at APD’s Academy. Unfortunately, the structure of the disqualification data limited our ability to thoroughly analyze attrition during the selection process. Specifically, although the reasons for disqualification are provided, they are recorded in note format and are not quantitatively captured into systematic categories. Furthermore, there is no record of the point during the process or the actual date at which the applicant was disqualified. For example, past drug usage could be discovered and result in disqualification during the background investigation or the polygraph exam. Therefore, we are only able to report demographic differences in the 136  reasons for disqualification, rather than when those disqualifications occurred within the application process.96 This data limitation does not apply to applicants who failed the written or physical tests, since these disqualifications always occur after the intake application review. Furthermore, the number of applicants for whom we have data varies for each of the disqualification reasons. Written Test Differences In 2018, APD transitioned from the NDRT to the NPST for applicant written testing. Therefore, Kroll conducted two sets of analyses to examine whether there were racial/ethnic differences in the NDRT and NPST pass/fail rates and standardized test scores. Using data from the 2,125 applicants for whom we have written scores, we conducted a relative comparison of those who took either the NDRT (n = 1,412, years 2014-2017) or the NPST test (n = 713, years 2018-2020).97 The two tests had different overall distributions and passing scores; for NDRT, a passing score was 102 or above and for NPST a passing score was 70 or above. To obtain relative score comparisons, we calculated standardized test scores (applicant score – average(test)/standard deviation(test)).98 These z-scores allowed us to make relative distribution comparisons by race/ethnicity on each test by converting the scores to the same standardized scale. We then grouped these standardized scores by race / ethnicity. The tables for written test comparisons present only racial/ethnic differences because there were no significant gender differences across any of the written test results. Comparing the two tests, a higher percentage of applicants passed the NDRT (87.8%) than the NPST (80.3%). Table 7 and Figure 14 below shows the passing rates by race/ethnicity for the two types of written tests employed by APD in 2014-2020. The NDRT had the highest overall percentage of passes by race/ethnicity for all groups (except “other”, which had a small sample size of 30 total). Over 94% of white applicants passed the NDRT, compared to 77% of Hispanics and 73.5% of Blacks. Comparatively, the NPST test had lower overall passing rates for most racial/ethnic groups included in this analysis. Nearly 87% of white applicants passed the NPST, compared to 74.8% of Hispanics and 70.8% of Blacks. Kroll notes, however, that in addition to changing the written test in 2018, the sequence of steps in the selection process also changed. As described in the Background Investigation section, the BHS moved from the initial application phase to after the Phase I testing phase. As a result of this change, a larger percentage of applicants likely took the NPST written test because the BHS disqualifications had not yet occurred. 96 It is for this reason that we can only report that there are 2,868 applicants who appear in the disqualification data, but not the applicant testing data; we speculate that these are applicants who were disqualified due to the initial intake review and who did not proceed to Phase I testing. Given the changes in the timing of the BHS submission, it would make sense that a large number of applicants were disqualified prior to Phase I testing when the BHS was required to be submitted with the initial application. 97 It is important to note that there were 111 applicants from 2014-2016 who did not have a written score on the NDRT test but were in the “pre-academy pt retest” group. There were no substantive racial/ethnic differences among this group of ‘missing test data’ cases relative to those who took the test. 98 NDRT average = 125.27, with a SD = 19.75. NPST average = 76.58, with a SD = 8.99. 137  Table 7: NDRT vs NPST Written Test Results (Percent Who Pass by Race/Ethnicity) Asian Black Hispanic Other White NDRT NPST % Passed (n size) % Passed (n size) 89.5% (n = 51) 72.0% (n = 25) 73.5% (n = 117) 70.8% (n = 72) 77.1% (n = 367) 74.8% (n = 258) 80.8% (n = 21) 88.9% (n = 9) 94.4% (n = 839) 86.8% (n = 349) *American Indians (Total n = 8) were excluded from this analysis due to limited sample size. Figure 14: NDRT vs NPST Written Test Results (Percent Who Pass by Race/Ethnicity) 100.0% 89.5% 94.4% 88.9% 86.8% 80.8% 77.1% 73.5% 72.0% 70.8% 74.8% 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% NDRT NPST Asian Black Hispanic Other White Table 8 shows the standardized test scores for the two written tests. It is important to note that the overall score distributions are higher for each race/ethnic group for the NPST relative to the NDRT. The standard deviations are also smaller, showing the distributions as more evenly distributed. Specifically, Black candidates’ z-scores increased from the 25th percentile on the NDRT test to 36th percentile on the NPST. Hispanic candidates’ scores also increased, although not as much as Black candidates, from the 34th percentile on the NDRT to the 38th percentile on the NPST. White candidates scored roughly the same in terms of percentile distributions (61st vs 62nd percentile for NDRT and NPST, respectively). Those in the ‘other’ racial and ethnic categories also increased from the 41st percentile on the NDRT to the 54th percentile on the NPST. Asian/Pacific Islanders are the only racial/ethnic group that did worse on the NPST, although marginally so, at the 40th percentile for the NPST compared to the 42nd percentile for the NDRT. Table 8: NDRT vs NPST Written Test Results (Standardized Scores by Race/Ethnicity) NDRT NPST Mean Z-Score (SD) Percentile Mean Z-Score (SD) Percentile Asian -.1842 (1.09) 42nd -.2469 (1.15) 40th Hispanic -.4131 (1.03) 34th -.2883 (1.03) 38th Other -.1987 (1.12) 41st .1222 (0.84) 54th White .2897 (0.82) 61st .3068 (0.88) 62nd *American Indians (Total n = 8) were excluded from this analysis due to limited sample size. Black -.6641 (1.17) 25th -.3645 (0.93) 36th 138  In sum, the NPST was marginally more difficult to pass for most racial or ethnic groups, but the variance or spread of the scores became much more centralized. That is, the differences between racial/ethnic groups who took the exam became smaller; Black and Hispanic applicants, in particular, performed better on the NPST.99 Regardless of which written test was taken, however, white applicants were statistically less likely than applicants of other races and ethnicities to be disqualified from the selection process due to their written test results, while Black and Hispanic applicants were significantly more likely than applicants of other races and ethnicities to be disqualified due to the written test. Physical Ability Test Differences The PAT is a pass/fail timed test. There were no significant racial/ethnic differences in PAT failures. There were, however, significant gender differences in PAT failures. These differences varied by year due to the changing requirements of the PAT documented earlier in Table 6.100 Additionally, female applicants in 2017 and 2018 seemed to be adversely impacted by the 2,000-meter rowing test. The physical test failure rate during these years was significantly higher for females than males.101 Once the 2,000-meter rowing test was removed (and physical testing re-focused on 1.5-mile runs with a shortened 500-meter rowing test), the within-gender rate of male physical failures was slightly higher than female physical failures.102 These findings provide some quantitative support that APD consistently adjusts their physical testing requirements when such tests adversely impact candidates. Other Disqualification Differences Kroll also reviewed the BHS and other disqualifiers for statistical differences between applicants with different demographic characteristics. An important caveat to the findings noted below is that it is unknown whether applicants of color would fail or disqualify at rates similar to or significantly differently from white applicants for particular disqualification reasons that occur later in the selection process. This is because people of color are less likely than whites to progress as far into the selection process based on the differences in the written tests. Black applicants were disqualified statistically more often as compared to other population subgroups due to outstanding debt and credit histories. Kroll does not know if this reflected actual money owed by applicants or if applicants did not provide the requested data. White applicants, when compared to other racial/ethnic groups, were statistically less likely to be disqualified due to their credit history or a problem with their driver’s license (e.g., suspension). On the other hand, white applicants were statistically more likely and Black applicants statistically less likely to be disqualified than others due to the polygraph/medical exam/psychological exam, which occurs near the end of the selection process. Also occurring near the end of the process is the Oral Board interview. The Oral Board failure rate is less than 3% for all groups. APD, however, does not keep sufficient records on applicants’ performance during the Oral Boards to allow a 99 Another indication of the compressed variability was the reduction in the One-Way ANOVA F-statistic for the NDRT (F = 30.03) and the NPST (F = 14.75), suggesting a reduction in the between-to-within group variability by nearly half between the tests. 100 For example, when we examined 2014 data, 25 out of the 35 females failed the physical test (7 passed, and 3 others failed for other reasons). We do not have specific and consistent indicators for the test scores for these candidates, or what the test requirements were, but once this year was removed, the overall distribution of male/female failed physical tests became more normalized (in years 2015-2020 overall). 101 For years 2017-2018, 14% of all females failed the physical test compared to 6.3% of males. This is a female failure rate that was 2.22 times that of the male rate. For years 2015, 2016, 2019, and 2020 combined (thus excluding the 2000m rowing test), roughly 21.8% of females failed the physical tests compared to 17.9% of males who failed the physical test (a female failure rate that was closer to 1.2 times greater than the male rate). 102 The male physical test rate failure from 2018 through 2020 was 33.7% for males, compared to 21.7% for females. 139  more precise examination of whether differences in scores exist across race/ethnicity and gender. A recent DOJ/EEOC publication indicated that the St. Paul Police Department found applicants of color outperformed whites during in-person interviews. Past drug use is not an automatic disqualifier; it depends on the drug used, as well as when and how often it was used. White applicants were statistically more likely than others to be disqualified due to their drug usage. Due to the structure of APD’s disqualification data, it is unknown whether this is drug use that was discovered during the background history statement/investigation, polygraph, or actual failure of the drug screening. Nevertheless, this finding is highly statistically significant and different than other groups; 30% of disqualified white applicants were disqualified due to drug usage, compared to 16.2% and 21.8% of Black and Hispanic applicants, respectively. At the completion of each selection process, those who are hired become cadets. They must then enter and successfully complete the Training Academy to become a full-time police officer. In order to examine whether racial and ethnic differences exist in who ultimately gets hired, we conducted a Chi-Square statistical comparison of whites and Blacks, whites and Hispanics, and whites and Asians/Pacific Islanders.103 A Chi Square is used to statistically compare observed results with expected results. This analysis indicates that there is a statistically significant relationship between race/ethnicity and hiring decisions. Ultimately, after disqualifications and other drop-off factors, nearly two-thirds of candidates hired as APD officers are white (66.0%) compared to other racial/ethnic groups (Hispanic - 21.5%, Black - 7.7%, Asian/Pacific Islander - 4.5%). It is important to reiterate, however, that this does not examine the influence of any factors other than race/ethnicity. Given that the recruitment process is having success in creating a diverse applicant pool, this finding raises additional questions. Selection Process Analysis According to the Uniform Guidelines established by the Equal Employment Opportunity Commission, the tests and processes associated with the evaluation and selection of new police officers must be valid; that is, they must be demonstrably related to successful job performance in the Academy and as a police officer (EEOC, 1978; 1978; APA, 2018). The process of selecting applicants who are the best qualified for the job, who represent the population they serve and who possess the best personality traits of a police public servant, is labor intensive and can have high financial costs associated with effectively conducting evaluations of a large population of recruits. Recent national surveys show that virtually all sampled police departments use an application process, background investigations including checks of criminal history and driving record, interviews, and medical exams; additionally, most use a psychological assessment, drug testing, physical fitness/agility test, a credit history check, and written aptitude test (Cochrane, Tett, & Vandecreek, 2003; DOJ/EEOC, 2016; Reaves, 2012). The APD hiring process, in terms of the selection components and their sequencing, is consistent with standard police department hiring practices in the U.S. Furthermore, as described above, APD frequently modifies its selection practices to increase retention of qualified and diverse applicants and maximize efficiency in the process (e.g., use of online application and document submission, change in timing of submission of background history documents, etc.). Based upon Kroll’s experience and research in the field, one reason police departments lose qualified applicants is because the hiring process is too long or too complicated, so the fact that APD frequently addresses these issues is important (DOJ/EEOC, 2016). 103 Given the small number of American Indian and Alaska Native applicants, there is insufficient statistical power to make valid comparisons to whites, but we can conclude that they are underrepresented in APD as compared to the Austin population. 140  There are, however, some components that should be further evaluated for valid and diverse outcomes; these are described in the following section. Selection Observations APD is not currently documenting why applicants separate during the selection process; however, this data would be helpful in analyzing issues that may relate to the selection process or the applicant that might be actionable by APD. The latest written (i.e., cognitive ability) test for applicants continues to show racial and ethnic disparities in scores, although the disparities are smaller than with the previous written test. It is possible that disparities in written scores would persist across any type of entry-level written test (DOJ/EEOC, 2016); however, there are other options available that might help in reducing group differences. Our analyses of physical ability testing failures indicate that there are no significant racial/ethnic differences in this outcome and, once the 2,000-meter rowing test was eliminated, the statistically significant gender differences that were present during its use were no longer evident. There were previously, however, some racial/ethnic and gender differences in injury rates and the timing of injuries that occurred in the Academy, which Kroll noted in its April 2021 Assessment Report of APD’s Training Academy. The current APD physical ability test was based on the Texas DPS fitness assessment but APD could not find data demonstrating the validity of its test. Further, it is not known whether a run and a row at the designated standards are predictive of Academy physical requirements. A run measures aerobic capacity (Cantwell, 1985), while the row requires anaerobic and aerobic ability and the use of most of the major muscle groups in the body (Jones, 2011). The research on physical ability testing for police officers indicates that the components of the test “should assess specific qualities (e.g. strength endurance, aerobic capacity, change of direction ability) and avoid redundancy” (Cesario et al., 2018, p.1064); however, as the run and the row both measure aerobic capacity, there may be some unnecessary redundancy in those tests. Kroll reviewed a document called “Oral Interview Boards” that is a document from the Recruiting Division dated 2017. It is not an SOP or General Order but appears to be a guide for interviewers. In this guide, it states: The Oral Interview Board Members will evaluate each candidate as objectively as possible on eight job-related dimensions. These dimensions are Preparation for Police Career, Cultural Diversity, Handling a Physically Demanding Job/Physical Fitness, Overall Maturity/Life Skills, Employment/Work History, Integrity, Communication Skills, and Situational Scenarios. All eight areas are evaluated on the basis of the candidate’s behavior during the oral interview and on information from the candidate’s history. The interviewer examines the candidate’s application as well as answers to oral board questions for evidence of these qualities. Following the interview, each Board member rates the candidate anonymously with no discussion or comments by other interviewers. The Board members then openly discuss their findings. Ratings may be changed after the discussion. Each oral board member will fill out a ratings sheet for each candidate which will be turned over to recruiting personnel once the oral board has completed evaluation of the candidate. The guide refers to “standardized questions,” for which the Recruiting Division is supposed to be responsible. The guide further states that “Oral Board members should not create their own questions or scenarios unless they have a specific concern with an applicant. Any questions that will deviate from the standardized questions must be approved by the Recruiting Unit Lieutenant and documented on the back of the rating sheet.” 141  Kroll requested oral board interview records related to applicants’ performance, interview questions, scoring rubric, and board members’ training, but none were provided. Accordingly, Kroll could not evaluate applicants’ performance and whether there were any racial/ethnic or gender disparities in oral board interview scores. We also could not assess the standardization and relevance of the questions asked as compared to what they intend to measure, the characteristics and training of the panel members who compose the board, or whether the scoring rubric and criteria may have a disproportionate impact on certain applicants. For APD to attain value from the interview process, the questions should be designed to elicit the competency information intended and focus on information about each applicant that has not been fully solicited or evaluated up to that point in the selection process. For example, applicants’ verbal and interpersonal skills have not been formally evaluated before the interview process so the oral board should be designed to evaluate those two competencies. The oral board can also assist in assessing applicants’ biases, their problem solving and reasoning skills, leadership skills, decision making skills, organization, and planning ability. Finally, in the last two decades, test preparation has become a best practice. For example, a physical ability test necessitates preparation and applicants must know precisely how to prepare. The APD’s recruiting website offers a training video on the row and indicates the standards by age and gender for the 1.5-mile run, which gives applicants the opportunity to prepare. Proper preparation for the physical ability test reduces injury and increases the opportunity to pass the test (Stober and Pekrum, 2004). Additionally, for written tests, vendors should offer practice questions and preparation instructions to ensure applicants are aware of the test process prior to taking it. NPST offers this option, as does the Oral Board. Test preparation reduces the test takers anxiety and helps to ensure they can perform at their best (Stober and Pekrum, 2004). Our review of APD’s testing practices found that APD offers some online preparation for applicants. 6.5 The APD Promotion Process According to an analysis of the 2016 Law Enforcement Management and Administrative Statistics (LEMAS) survey of a nationally representative sample of local police departments, only 20% of first line supervisors are Black or Hispanic, compared to 25% of officers; while just 10% of first line supervisors are female, compared to 12% of officers (Hyland & Davis, 2019). Diversity in upper ranks of law enforcement is even more challenging (Shjarback & Todak, 2019). This section describes APD’s promotion process and policy, presents data analysis of recent promotion tests to determine whether disparities based on race/ethnicity or gender exist for any of the promotional components, assesses APD’s career development program, and provides recommendations based on Kroll’s review of each of these areas. Promotion Process Description The City of Austin’s personnel rules, and by extension, APD’s hiring and promotional systems, are dictated by the Municipal Civil Service Rules of the City of Austin (Rules), including Rule 4: Hiring, Promotions, and Lateral Transfers (2014). The City’s Civil Service Rules state: All Positions in the Classified Municipal Civil Service shall be considered Competitive Positions unless designated otherwise by the Commission or as specified in these Rules. Competitive Processes may differ depending on the level and type of Position but shall be designed in a manner intended to identify the best Candidate for the Position. 142  The Rules also dictate the requirements for posting of promotional vacancies, the application process, other allowable criteria, and a mandatory probationary period after promotion.104 Kroll has examined civil services personnel rules in other jurisdictions and there is nothing in these Rules that is unusual or that conflicts with best practices. The promotion process is also governed by APD General Order 918: Promotion, Transfer, and Vacancy Guidelines for Sworn Employees, and the eligibility and testing processes described in Article 13 of the Collective Bargaining Agreement between the City of Austin and the Austin Police Association (most recent effective date: November 2018). For competitive promotions, the commanders in the division or unit where the vacancy exists are responsible for setting the minimum qualifications, training, and experience needed for the position, and for ensuring that the promotional process is “job-related and evaluated fairly.” APD General Order 918 describes administrative procedures for filling vacancies in detail. APD is thorough in ensuring that all those who are qualified for the vacancy are notified in a timely manner. Job postings for a vacancy and test are detailed and include the test date, time and location, test study (source) material, the weight of each of the source materials, and APD attempts to give promotional candidates at least six months to study prior to the test date. Although the promotional procedures and components are negotiated between APD and the union, the City hires a contractor who specializes in test validation and administration; they are also required to give an orientation to candidates. The exams reviewed by Kroll were developed and validated by I/O Solutions, Inc. Our review of the contractor’s validation reports explaining their processes to validate the exams found that they are consistent with the Uniform Guidelines on Employee Selection Procedures. There are four promotional exams: 1) Corporal/Detective, 2) Sergeant, 3) Lieutenant, and 4) Commander. All four ranks utilize a written multiple-choice test of job knowledge on which the applicant must achieve a score of 70/100 or higher to be considered for promotion. Candidates who do not pass the written exam do not proceed in the promotion eligibility process. The written test is the only component of the Corporals’ exam. The remaining ranks also utilize an assessment center, described below, for those who pass the written test. In addition, for all ranks, extra points are awarded on the below criteria to those who pass the written exam: • Education: 0.5 points for 60 college credit hours, 1.0 point for a bachelor’s degree, 2.0 points • Seniority: One point per year of service, prorated for partial years, maximum of 15 points for for a master’s degree. Corporals only. • Time in rank: One point per year in rank, prorated for partial years, maximum of seven for Sergeants, five for Lieutenants and Commanders. • Master Peace Officers certification: One point, for Corporals only. The assessment center is a type of test that directly measures competencies such as oral communication, interpersonal skills, reasoning, application of knowledge, and management/leadership skills. Research demonstrates that an assessment center is a method of testing that tends to have much lower disparate impact against minority applicants, especially when compared to other types of measures such as written tests. According to Article 13 of the Collective Bargaining Agreement, when an assessment center is included in the promotional process, the written examination and assessment scores (both scored out of 100) are weighted. For the rank of Sergeant, the written exam score is multiplied by 0.7, while the 104 The Rules also provide for Direct Appointments in certain circumstances (e.g., specialty assignments) which do not require a competitive process and provide the method to do so. A lateral appointment in APD, for example, would qualify under Direct Appointments. 143  assessment center score is multiplied by 0.3. For the ranks of Lieutenant and Commander, the written exam score and assessment center score are equally weighted (i.e., both multiplied by 0.5). The contractor hired to administer promotions is responsible for videotaping candidates as they perform their exercises in the assessment center; they also select and train the assessors who evaluate the assessment center. While the Agreement does not state it specifically, the contractor generally administers the entire assessment center process, and Kroll considers this to be a best practice because it helps ensure the consistency of the process. Furthermore, once the assessment center process has concluded for each rank, the Agreement requires a formal critique of the process so that it can be improved in the future. Kroll considers this a best practice, as it demonstrates APD’s commitment to improving the testing process and ensuring that it is perceived by candidates as fair. The final promotional score—based on the combination of the components described above—is calculated and candidates are rank-ordered on an eligibility list. Each eligibility list expires in two years after it is posted. Analysis of Promotion Processes 2015-2020 Kroll examined a total of 11 promotional exams offered by APD from January 2015 to December 2020, where the tests occurred roughly two years apart. The next test within each rank occurred after the prior promotion eligibility list expired on a set deadline. There were 988 test takers between January 2015 and December 2020; of those, 3.8% applied for Commander (n=38), 10.7% applied for Lieutenant (n=106), 19.3% applied for Sergeant (n=191), and 66.1% applied for Corporal (n=653). The racial/ethnic makeup of the examinees was: 67.9% white; 20.9% Hispanic; 8.6% Black; 2.2% Asian/Pacific Islander. The average age of examinees was 38.8 years old, with a range from 24.0 to 57.7 years old. 105 Across all ranks, the overall percent of candidates promoted was 59.6%; the percent promoted varied by rank: 57.1% Corporal, 71.2% Sergeant, 59.4% Lieutenant, and 44.7% Commander. Kroll statistically analyzed these 11 promotional tests for disparate impact, also known as adverse impact. Adverse impact occurs when a selection instrument, neutral on its face, has a negative and disproportionate impact on minority applicants as compared to the majority.106 The Uniform Guidelines on Employee Selection Procedures defines adverse impact as a statistically and practically significant difference in passing rates between two population subgroups, including race, age, and gender. Kroll focused on these three population subgroups because our interviews with APD staff and current career personnel informed us, anecdotally, that these three demographic characteristics may be related to adverse impact in the promotional processes in APD. At each rank, over 99% of applicants who take the written test passed (score of 70% or higher). A high passing rate like this means that the written test does not successfully differentiate between low and high performers. Given the high pass ratings, our research focused on the impact of three primary independent variables—officer age, race/ethnicity, and gender—on three major outcomes: • Final written exam score (numeric scale) • Final promotional score (numeric scale) • Promotion (dichotomous measure: occurred/did not occur) 105 The pooled average age at test varied across each test/rank. For example, the average age for the Commander test was 46.7 years old. 106 Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, Department of Justice. (1978). Adoption of Four Agencies of Uniform Guidelines on Employee Selection Procedures, 43 Federal Register 38, 290-38, 315 (August 25, 1978). Society for Industrial and Organizational Psychology, Inc. (2003). Principles for the Validation and Use of Personnel Selection Procedures (3rd ed.), College Park, MD: Author. 144  Bivariate and Multivariate Analyses Table 9 below shows the demographic breakdown for the three promotional outcomes of interest. There are no statistically significant gender differences in any of the promotional components; that is, males and females scored similarly on all outcomes of interest.107 Age-based analyses, not shown in Table 9, revealed that older candidates scored significantly lower on written tests (Pearson Correlation = -.133, p < .05) and final promotional scores (Pearson’s Correlation = -.240, p < .05). However, there was no correlation between candidate age and whether a candidate was promoted. For written test scores, there are statistically significant differences across racial/ethnic groups. Asian/Pacific Islanders and whites scored the highest on written exams, while Blacks and Hispanics scored on average 2-3 points lower. This was consistent across all ranks; the results remained statistically significant, and the effect sizes were similar. Interestingly, although there are slight differences in final promotional scores between the racial/ethnic groups, none of these are statistically significant differences. Kroll also found that seniority bonus points have played a critical role in closing the promotional score gaps for Blacks and Hispanics. It appears that Black and Hispanic candidates stay in grade longer than white candidates before promotion. Although our data cannot speak to why this may be the case, the resulting boost in scores that occurs for these groups due to seniority diminishes the observed group differences in written test scores to the point of no statistically significant impact for final promotional scores. Other statistically significant differences in bonus points were evident by candidate race/ethnicity and gender.108 Specifically, Black candidates had the highest average number of education points—1.1 points, compared to 0.8 for both white and Hispanic candidates. Female candidates also received a slightly higher amount of education points compared to males (1.0 vs. 0.8). As described above, assessment centers are used for promotion tests for the ranks of sergeant and above. Due to limited statistical power for the upper ranks, we can only reliably examine the racial/ethnic differences in assessment center scores for sergeants. Asian and white candidates have significantly higher average raw scores on the assessment centers than Black and Hispanic candidates. The difference equates to approximately 5.0 raw score points between white candidates and Black candidates, and 5.4 points between white candidates and Hispanic candidates. Asian candidates score approximately 2.2 raw score points higher than white candidates. At least for sergeants, the Assessment Center scores have a statistically significant disparate impact on Black and Hispanic promotion candidates. Kroll cannot say whether these racial/ethnic differences in assessment center scores would hold at the upper ranks, but if they do, their impact would be greater because the assessment center score is given equal weighting to the written exam at those ranks. Finally, promotion eligibility lists are constructed by rank ordering candidates based on the highest total score. Among candidates who sought promotion, the percentage of those promoted within each racial/ethnic group was as follows: Asian/Pacific Islanders - 72.7%, white - 60.1%, Black - 56.4%, and Hispanic - 54.3%. Chi-Square and ANOVA analyses revealed that there were three distinct groups on promotion: Asian candidates, white candidates, and Black/Hispanic candidates. That is, Black and Hispanic 107 We examined whether gender significantly influenced promotion within ranks. For Corporals and Sergeants, it does not. For Lieutenants and Commanders combined, the influence of gender on promotion approaches statistical significance at the p < .05 level. This difference, however, is in favor of females; 78% of females get promoted on these tests vs 53% of males). 108 These analyses examined only candidates for corporal and sergeant due to insufficient statistical power at higher ranks. 145  candidates were not significantly different from each other, but collectively they were significantly different than all other groups. Table 9: Average Score Differences by Gender and Race on Promotional Components109 Written Score Final Promotional Score Percent Promoted Male Female White Black Hispanic Asian 82.9 83.1 83.5 81.6 81.4 84.7 89.2 89.7 89.6 88.0 88.6 90.3 59.7% 59.2% 60.1% 56.4% 54.3% 72.7% The impact of candidate demographic characteristics was also examined using multivariate analyses. As described in the methodology, this is a statistical technique that allows simultaneous estimation of various predictor variables on an outcome. The type of multivariate regression techniques employed depend on the nature of the outcome of interest. Both written scores and final promotional scores are examined using ordinary least squares (OLS) regression because they are normally distributed outcomes. The appropriate modeling technique for the dichotomous promotional outcome is binary logistic regression because the outcome has only two possible values (promotion yes or no). The results reported reflect odds ratios (e.g., how many times more likely the outcome is) for key relationships among the predictors and the outcome. The independent variables include candidate age (natural age scale), gender (male=1, female=0), and race/ethnicity (Blacks and Hispanics = 1, all other groups = 0). The reason for combining Black and Hispanic candidates into a single measure is three-fold. First, mean difference analyses (T-Tests) showed that Black and Hispanic candidates had extremely similar distributions for written exam scores, and that those distributions were significantly divergent from white and Asian candidate distributions. Second, Blacks and Hispanics are frequently combined into racial/ethnic minorities in demographic social science research related to opportunities, structure, and organizational components related to hiring and career advancement. Third, combining Black and Hispanic candidates into a single independent variable enhanced statistical power due to Blacks comprising less than 9% of the distribution of testing candidates. Table 10 below presents regression results for the analysis of written test scores. Black and Hispanic candidates scored significantly lower than all other candidates (b = -2.051, SE = .502, p<.01). The difference equated to roughly two points on the written exam. Additionally, older candidates scored significantly lower than younger candidates (b = -.146, SE = .036, p<.01); predicted value analyses showed that every five years in age corresponded with about one point lower on the written test. There were no statistically significant gender differences. 109 We also examined whether racial/ethnic or gender differences existed for actual promotion to individual ranks and found none. 146  Table 10: Specific Test Score Differences on Promotional Written Tests Coefficient Standard Error Standardized p-value Intercept Black/Hispanic Male Age Unstandardized Beta 89.98 -2.051 -.535 -.146 1.63 .502 .749 .036 Beta -- -.130 -.023 -.126 <.001 <.001 .475 <.001 Table 11 presents regression results for the analysis of final promotional scores. Black and Hispanic candidates scored marginally lower (significant at p < .10) on a two-tailed distribution than all other candidates (b = -1.045, SE = 1.96, p = .086). The difference equated to roughly one-point difference on the final promotional score. Additionally, older candidates scored significantly lower than younger candidates (b = -.336, SE = .044, p < .01). Males and females did not differ on final promotional scores in any substantive or significant manner. Table 11: Score Differences on Final Promotional Score Coefficient Standard Error Standardized p-value Intercept Black/Hispanic Male Age Unstandardized Beta 103.7 -1.04 -.504 -.336 1.96 .605 .901 .044 Beta -- -.054 -.017 -.237 <.001 .086 .576 <.001 Table 12 below displays the results of the logistic regression analysis of whether promotion occurred. Black and Hispanic candidates were less likely to be promoted than all other candidates (marginally significant— p < .10 on a two-tailed distribution) (b = -.275, SE = .143, p = .055). Neither the gender or age variables’ coefficients approached statistical significance, meaning the likelihood of promotion was the same for males and females as well as older candidates. Table 12: Differences in Promotion Rates by Gender and Race Coefficient Intercept Black/Hispanic Male Age Unstandardized Beta 0.549 -.275 -.046 -.001 Standard Error Odds Ratio p-value .469 .143 .215 .010 (Exp(B)) -- 0.76 0.95 .999 .242 .055 .830 .934 Career Development Career development programs and training can impact diversity within a department by providing information to all candidates about how to properly prepare for upcoming vacancies or to motivate them to excel in their current rank if they choose not to compete for promotion (Ginac, 2019). APD General Order 942 defines APD’s Training and Career Development for civilian and career staff as follows: The Department seeks to provide ongoing training and encourages all personnel to participate in advanced training and formal education on a continual basis. Training is provided within the confines of funding, requirements of a given assignment, staffing 147  levels and legal mandates. Whenever possible, the Department will use courses certified by the Texas Commission on Law Enforcement (TCOLE). (APD General Order 942.2) The SOP indicates that career development training is offered in many forms to include live, web-based and shift briefings. Kroll’s interviews with APD staff indicated that the current career development program is unstructured, but job skills training for different assignments is available (e.g., homicide investigation, drug recognition expert). Kroll interviewed an APD Lieutenant who oversees internal promotions who indicated that APD formed a “Lateral Transfer Committee” that is currently working on career mapping plans for each of the specialized positions within APD. This should assist officers in preparing themselves for positions they are interested in when vacancies become available. Kroll believes this is a sound method to allow all officers an opportunity to obtain specialty position and lateral assignments. According to General Order 942, APD is expected to provide or facilitate external training to newly promoted staff either immediately before or within a year after promotion. No description of the curriculum for this training is specified. There are certain required classes for sergeants and lieutenants (e.g., National Incident Command System—NCIS), but the schedule for that training was not provided to Kroll. Lieutenants and commanders must attend an APD approved long-term management school within three years of achieving that rank. This three-year time-period is specified to allow for the possibility that a program may not be immediately available for all newly promoted lieutenants and commanders. APD informed Kroll that newly promoted staff attend training at the FBI National Academy, Southern Police Institute, and Texas Leadership Command College (A&M). Lieutenants and commanders must also attend, and successfully complete, additional APD-sponsored courses as determined by Executive Staff and the Training Academy. There are other methods to enhance career development for all officers in APD. For example, mentoring programs often improve employee retention, maximize mentees’ performance and job satisfaction, communicate important institutional knowledge, and share best practices and lessons learned (DOJ/EEOC, 2016; Robinson & Reio, 2012; Suboch, Harrington, & House, 2017). They can be challenging to implement because suitable and committed mentors must be identified and trained. Ideally, mentors should also be diverse in terms of race, gender, assignments, and backgrounds so that officers learn about career pathing from individuals with different career histories, challenges, and successes. As described in Kroll’s October 14, 2021 Interim Report of Independent Evaluator: APD Training Academy (pp. 74-75), APD developed a mentorship program, including mentor training, for female academy cadets in 2018. The program was opened to any interested cadet at the start of the 144th cadet class. The Texas Peace Officers Association (TPOA) also has a mentorship program, with approximately ten mentors working with cadets of color. In addition to these programs focused on cadets, Kroll understands that the City of Austin has contracted with Joyce James Consulting LLC to develop a mentorship program for APD sworn officers of color. Another process for developing officers is through “shadowing,” which is similar to mentoring but slightly different in that officers observe other officers in the specific job they wish to obtain in order to learn how to perform the tasks required. In the best-case scenario, officers are assigned to observe higher ranking officers who are considered highly competent and who have a desire to share knowledge, skills, and abilities by explaining tasks and answering relevant questions about task performance. Promotion and Career Development Observations Kroll’s statistical analyses of APD promotions from 2015 to 2020 indicate that, while there are no significant gender differences in promotion outcomes, Asian and white promotion candidates are significantly more likely to be promoted than Black and Hispanic candidates. The written promotional test scores across all ranks indicate that the test may have an adverse impact on candidates of color and between younger and older candidates. 148  The use of a written test for promotions is standard practice; however, research has shown that written tests such as these can lead to disparate impact because they typically result in statistically significant test score differences between whites and Asians as compared to Blacks or Hispanics, especially when administered without any other test components (e.g., Hough, Oswald, & Ployhart, 2001; Hunter & Hunter, 1984). Test scores are a major component of the promotional test score, so this score difference might place Black and Hispanic candidates lower on the promotion eligibility list than white and Asian candidates. Kroll notes that the Collective Bargaining Agreement requires an appeals process for the written exam, which allows candidates to challenge questions that may have unanticipated errors (e.g., information in the study material conflict). As appeals are reviewed formally by an unbiased committee, it appears to be an objective and fair process. Kroll found that seniority helped close the score gap between white candidates and candidates of color during promotions from 2015 to 2020. Promotion candidates of color benefit from higher seniority scores more than white candidates, indicating that they stay in grade longer than white candidates before promotion. Our data do not allow for an examination of why this may be occurring. Research indicates officers from diverse backgrounds often have multiple reasons for opting out of or delaying the promotion process, including the lack of interest in administrative duties, satisfaction with their current position, feeling unprepared or wanting to gain more experience, issues related to tokenism (e.g., the perception that candidates were selected based on diverse characteristics), and impact on family lives (Archbold, Hassell, & Stichman, 2010; Todak, Leban, & Hixon, 2021). It could also be that, because Black and Hispanic promotion candidates score lower on the promotion test and are lower on the eligibility list, they do not get promoted on their first attempt before the eligibility list’s time-period expires. It may also be that candidates of color are not encouraged to take the test as quickly as white applicants. Kroll recommends that APD examine these possibilities to better understand the reason some racial/ethnic minorities may delay seeking promotion. Kroll cautions that seniority points do not provide a dependable long-term solution to minimize group differences, as seniority points may not consistently impact diversity in a positive manner. Rather, to reduce disparate impact and changes in the test components, the competencies measured and method of measurement may decrease group differences (e.g., Goldstein, Yusko & Nicolopoulos, 2001). The weighting of the written test (70%) and the assessment center (30%) for the rank of Sergeant is somewhat unusual and not in accordance with best practices since assessment centers measure a broad range of knowledge, skills, and abilities and usually result in less adverse impact between groups (e.g., Goldstein, Yusko, Braverman, Smith & Chung, 1998; Goldstein, Yusko & Nicolopoulos, 2001; Hunter & Hunter, 1984; Hunter 1986; Hough, Oswald & Ployhart, 2001; Roth, Bevier & Bobko, 2001; Roth, Bevier, Bobko, Switzer III & Tyler, 2001). Furthermore, although written tests measure job knowledge, they do so indirectly. In written tests, candidates are not demonstrating their job knowledge, but are instead recalling information they memorized from the study material. We cannot tell from this instrument if they can apply that knowledge in the real world. For this reason, and for its known adverse impact, the written test should be assigned a lower weight for sergeants. For the ranks of lieutenant and commander, the written test receives a weight of 50% of the total test score and the assessment center receives a weight of 50% of the total test score. This is more in line with best practices and the research that indicates higher weights applied to the assessment center can minimize group differences. Kroll’s analysis of assessment center scores for sergeants, however, were significantly higher for white and Asian candidates, as compared to Black and Hispanic candidates. This conflicts with standard research findings that assessment centers tend to have low or no adverse impact. The Collective Bargaining Agreement permits the Police Chief to replace the assessment center with a technical skills evaluation. This component “will consist of one or more written scenario(s) to which the 149  candidate shall submit a written response” and will be scored later by trained assessors. This option is unusual, and Kroll does not consider it a valuable alternative. As mentioned previously, written tests tend to have higher disparate impact than other types of tests. The Agreement requires all promotional candidates to pass the written test with a score of 70% before taking the next component. The resulting combined test score is a compensatory score, which means that poor performance on one component is offset by higher performance on the other. Compensatory approaches tend to have less disparate impact than using passing scores on the written test. The current test contractor for APD wrote a white paper on this topic110 that addresses the disparate impact of written tests but also discusses how compensatory models can help to diminish adverse impact and possibly even increase validity. However, that is not how APD’s promotional system is scored. Finally, promotional candidates must also achieve a 70% on the total test score (combined score). Kroll considers this a best practice (assuming a compensatory approach to combining test scores) because often police departments promote most candidates on an eligibility list, and it would not be in the Department’s best interest to promote candidates who did not pass the total test. Kroll suggests that candidate orientation can be improved if the contractor adds a test preparatory session. There is a tremendous amount of research dedicated to this subject that shows that test preparation can increase candidates’ scores and decrease score differences between racial/ethnic groups because it teaches candidates how to study and memorize source material used for the test (Moore, Sanchez & Ofelia San Pedro, 2018). Although the lack of a formal career mapping program is not unusual, understanding officers’ interests and long-term goals early in their careers can assist APD in supporting individual officers’ aspirations through training opportunities, mentorship, and support from supervisors. This helps with individuals’ preparation, job satisfaction, retention, and can help diversify the higher ranks and specialty positions (DOJ/EEOC, 2016; Haddad, Giglio, Keller & Lim, 2012). It is particularly important that the ranks of Lieutenant and Commander reflect the diversity of Austin to mentor future diverse leaders within the department, attract diverse recruits and applicants (DOJ/EEOC, 2016), and demonstrate that APD’s leadership is representative of all residents of the city of Austin. Although APD indicated that the Lateral Transfer Committee is working on this issue as it relates to specialized positions, Kroll has not received any additional information on this to date. 6.6 Conclusion Kroll’s review of APD’s recruitment, selection, and promotions processes found many things that APD is doing well, with needed room for improvement in other areas. Demographics and Diversity. White officers are overrepresented and women, Hispanic and Asian/Pacific Islander officers are underrepresented compared to the racial and ethnic makeup of Austin’s population. Black and Hispanic officers are underrepresented at the ranks of Lieutenant and Commander, while Hispanic officers are also underrepresented at the rank of Sergeant. Recruiting. Although APD’s recruiting efforts are comprehensive, creative, and reach a large group of diverse candidates, improvements are needed in APD’s data collection and documentation to help the department more consistently evaluate the effectiveness of its various recruiting strategies. Kroll found that the department utilizes several industry-standard recruitment practices and, while APD is reaching a large and diverse audience when it recruits, there is significant variation across the types of events in producing 110 https://iosolutions.com/compensatory-system-testing/ 150  applicants from the pool of recruits, with certain types of recruiting events producing better results than others among different racial/ethnic groups and women. Overall, only ten percent of those who expressed interest in applying to APD at a recruiting event actually followed through and applied. Moreover, the actual percentages of APD personnel that are women and people of color do not mirror the level of diversity in the recruitment pool, which suggests that some diverse recruits are either falling through the cracks or are not necessarily a good match for the selection requirements of APD. Hiring and Selection. APD’s hiring and selection process is consistent with standard police hiring practices throughout the United States and APD frequently modifies its selection practices to increase retention of qualified and diverse applicants. Kroll found, however, that the latest written (i.e., cognitive ability) test for applicants continues to show racial and ethnic disparities in scores, although the disparities are smaller than with the previous written test. Regardless of which written test was taken, Black and Hispanic applicants were significantly more likely than applicants of other races and ethnicities to be disqualified. While Kroll found no significant racial/ethnic differences in physical ability test failures, there were racial/ethnic disparities in some of the background history statement criteria. Overall, whites are statistically more likely to be hired by APD (66.0%) than any other racial/ethnic groups when compared to those who apply (Hispanics 21.5%, Blacks 7.7%, Asians 4.5%). As these numbers contrast with the Recruiting Unit’s success in creating a diverse applicant pool, it raises the question as to why some of the diverse recruits are either not applying or not matching APD selection criteria. Promotions. Although APD has a comprehensive promotions process, Kroll’s statistical analyses of APD promotions from 2015 to 2020 indicate that, while there are no significant gender differences in promotion outcomes, the written promotional test scores across all ranks indicate that the promotional written test may have an adverse impact on candidates of color and between younger and older candidates. Although this is partially offset by seniority bonus points, which have partially closed the promotional score gaps for Blacks and Hispanics, Kroll also found that the assessment center scores have a statistically significant disparate impact on Black and Hispanic promotion candidates at the rank of sergeant. In Section 7, Kroll provides a series of recommendations to enhance APD’s recruitment, selection, and promotion processes in order to continue and sustain diversity within the Department and throughout its ranks, and to further improve its overall recruitment, selection, and promotion efforts. 151  7. RECOMMENDATIONS Based on Kroll’s evaluation and analyses as outlined in this report, we make the following additional observations and recommendations concerning the areas discussed below. 7.1 Data Collection Recommendations The evidence-based policing movement of the 1990s changed the way police departments fundamentally thought about and collected data. At its core, evidence-based policing encourages the use of research to guide practice and evaluate practitioners (Sherman, 2013). Seeking to progress beyond the use of anecdotal or experiential evidence alone, evidence-based policing suggests that police decision-making on “what works” to address specific problems should be guided by objective facts produced from scientific research. In evidence-based policing, targeting, testing, and tracking (what is referred to as the “triple t”) have become central in many police departments across the country (Sherman, 2013). This approach has allowed police departments to use data to systematically review processes and procedures for efficiency and effectiveness, which enables proactive efforts to address crime, deployment patterns, and calls for service, as well as citizen concerns related to racial/ethnic disparities in traffic stops, stop outcomes, and use of force. Recent big data or data movement trends in digital data management focus on how efficiently data can be merged and converted with other data systems and sources to achieve these types of goals. If an agency seeks to be data-driven, it must have quality data on which to base its decision-making. The current research project focused on several APD datasets: street checks, citations, arrests, use of force, and police administrative data. While query building between data sets is a natural endeavor in data processing systems, the process of building data that allowed for Kroll’s analyses in this report required several data merges and revealed significant limitations in APD’s current data collection and storage protocols. APD would benefit tremendously from a wholesale systems approach to use their data for both legal/case processing as well as ways to improve efficiency/effectiveness practices. While APD currently collects the legal and required information for arrests, uses of force, and motor vehicle stops for court processing and legal tracking, its data remains deficient. Based on Kroll team members’ experiences working with police agencies over the last three decades, we believe APD’s electronic data management systems are woefully behind other professional police agencies of its size and resources (i.e., these systems are administrative in nature, are not well kept or maintained, and often do not merge or link together readily or consistently to improve the efficiency and effectiveness of policing practices). These antiquated collection processes require APD analysts to backtrack in order to fix problems with the data systems and provide basic information regarding police use of stops, citations, arrests, and uses of force, rather than spending time providing APD officials more extensive feedback on what analyses of the data show to guide more effective training, policies, and strategies. This section includes specific recommendations for improving arrest and use of force data, as well as creating a new comprehensive traffic stop data collection process. Overall, Kroll recommends the following with regard to data collection and management: • APD should maintain linking identifiers across the various data sources. Based on our experience with other agencies, the call-for-service identifier should be on every data source so that links are manageable and can be conducted by third parties outside of APD. The call-for-service identifier has been used to link in missing information about the suspect, officer, and circumstances of the events which may have been captured thoroughly in one form (e.g., the arrest form) but may have been ‘missing’ in a different report (e.g., the use of force report). 152  • APD would benefit considerably from collecting and maintaining a codebook for each of the measures it electronically collects in each data source. The codebook values and value labels should mirror the options available to police when inputting to view common responses relative to available options for officer input. This approach would allow for external parties to provide even further and more detailed feedback in the future on data collection and data processing. Arrest Data Our analyses of arrest encounters were limited in their precision based on the data fields available and the quality of data. Our multivariate analyses reported in Section 3.3 show that use of force is strongly predicted by the legal and situational factors we were able to examine. APD would benefit greatly from gathering additional detail within its arrest reports, not necessarily for the purpose of building a case against an arrestee for court/trial purposes, but for better understanding police decision-making during arrest encounters. The below series of recommendations is for APD’s consideration to improve overall arrest data collection and management, and by extension, the quality of understanding of arrests that result in the use of force, with the end goals of reducing racial/ethnic disparities and improving policies and training to make police encounters with the public safer for all. • Although information related to the criminal offense/charge was available, Kroll could not create a reliable offense/charge severity scale for use in the statistical models. Specifically, APD only collects information related to the highest charge; charge severity is based on the highest possible charge for an offense, rather than its actual categorization for the offense within the incident, and the offense severity table APD provided for linking charges did not include all charges. Kroll recommends that the number and type of all charges be included in the arrest reports and data. • Similarly, APD provided Kroll with officer demographic characteristics, but we were unable to include this information in the statistical models predicting force. This is because a single officer prepares the arrest report, and this may or may not be the officer who made the decision to arrest or the officer who used force. Therefore, we cannot determine the variation across officers’ race, gender, experience, assignment, or training as it relates to their likelihood of using force. Kroll recommends that the characteristics of all officers involved in an arrest encounter be included in the arrest data. • The arrest data does not include any data fields that allow for the systematic collection of several important correlates of use of force, except those where force was used. The lack of a measure of individual resistance, which is the single most consistent predictor of force, is a major limitation of the APD arrest data. Furthermore, nearly 80% of individuals who had force used against them were perceived to be impaired by drugs/alcohol, behavioral health issues or both. Kroll recommends that APD begin documenting in arrest reports the following: arrestees’ resistance/compliance, suspected mental health issues, and suspected alcohol and/or drug use. These variables should be measured in the same way that they appear in the use of force data. APD should also consider including in the arrest data other data fields, such as whether there was a foot pursuit, bystanders, multiple officers, or a weapon present. Including these additional explanatory variables in the arrest data would enhance the statistical models predicting use of force and would potentially offer additional avenues for training and policy improvements. • Finally, individuals arrested by APD should be given a unique numeric identifier (often referred to as a jacket number) so statistical analyses of arrests could control for arrest/charge history by selecting all prior arrests per person, which is information available to the police when they conduct 153  a background check on the person. Decisions made therefrom can be influenced by prior police contacts, arrests, and criminal charges. Use of Force Data Use of force incident counts are captured from APD internal affairs reports. Counting uses of force from these incidents is not a straightforward endeavor and requires a thorough coding and measurement audit, which was completed by the Kroll team. Based on this audit, several recommended changes to the collection of these data are provided. • APD currently only collects data on the highest level of force used. As a result, less severe types of force are undercounted. This is problematic from both the analytic and agency perspectives. Force type is often a progression that increases or decreases in severity in response to the individual’s compliance or resistance, but this interactive nature of force cannot be examined by APD’s current data. Examining the ‘highest’ force creates bias in interpretation when compared to the levels that were required for the use of force encounter. Kroll recommends that APD follow the approaches taken by many agencies to collect all force actions and individual resistance actions (already collected), and to do so in sequential order. This method of capturing data will better reflect APD’s training and General Orders, will permit a more nuanced understanding of how use of force encounters evolve, and ultimately produce the type of information that can be used to refine training and policy to reduce the need to use force. • APD General Order 200.2 requires that officers use de-escalation techniques “when safe and reasonable under the totality of circumstances…to reduce the likelihood for force and increase the likelihood of voluntary compliance.” General Order 200 goes on to detail the assessment process in which officers are expected to engage during encounters with the public and describes multiple de-escalation techniques. Kroll recommends that APD officers document their use of de-escalation tactics during use of force encounters in order to better understand the interactive nature of use of force encounters and to be able to assess how often APD officers are employing de-escalation tactics as trained and required by policy. • The measure of “weaponless” force included in APD’s use of force data is too vague a category for any meaningful interpretation. It accounts for approximately 80% of use of force encounters but includes a range of force severity from low level firm grips or blocking to intermediate or serious impact strikes to the body or head. While it is helpful that the ‘level’ of force is recorded (Levels 1- 4 per APD policy), the exact actions that officers take are not readily collected in APD’s current data. Kroll recommends that this weaponless category be disaggregated. Many police agencies electronically capture the types of weaponless force used such as ‘come along’ techniques or blocking (hands to the side) (as these are often the lowest level force) to hitting/striking (higher force) to gain compliance. Other miscellaneous use of force data recommendations are noted below: • Nearly 10% of the individuals who had force used against them had missing identification or demographic information. The largest reason given for this missing information was “unidentified person” (82.4%), followed by “record no longer exists” (9.4%), and “unknown” person (8.2%). o Kroll recommends that more detailed information be gathered, when possible, regarding the reason that individual identifying information is missing to better understand the circumstances of the use of force event (e.g., juvenile, unknown individual in a crowd control setting, record expunged, etc.). 154  • Nearly 40% of individuals who had force used against them were suspected to be emotionally disturbed or mentally unstable. Moreover, the City of Austin is home to a state mental health facility where APD officers regularly transport individuals who have force used against them. o Kroll recommends that the use of force report capture whether the responding officer was an APD CIT-certified officer, whether an APD CIT-certified or other mental health/crisis responder was requested, and if yes, whether they responded. • For 12% of the individuals who had force used against them, the officer indicated that no transportation was made, while in 4.9% of cases there is missing information in the data field. o Kroll recommends that APD standardize a data field in the use of force incident report that captures all individuals’ post-force dispositions, including if they are released and not transported anywhere by APD. Traffic Stop Data • In order to meet best practices in traffic stop data collection (see Fridell, 2004; McDevitt et al., 2008; Pryor et al., 2020), Kroll recommends that APD modify its traffic stop data collection process by developing a comprehensive, stand-alone data collection system for the sole purpose of uniformly documenting motor vehicle traffic stops regardless of stop disposition. Kroll specifically recommends that APD tailor this traffic stop form to collect the TCOLE required information and to allow the addition of information that is not required by the State of Texas. • Currently, APD motor vehicle stops are collected in three separate (though not necessarily distinct) data systems: 1) street checks (which includes verbal warnings, written warnings, and field observations), 2) citations, and 3) arrest data. This approach has led to a known error in the combined database (described in detail in Section 5) with duplication of stops that result in more than one outcome, for which there is no automated identification and correction method available. It is more efficient and accurate to have a ‘stops’ database. Furthermore, although the TCOLE- required information is included across each of these datasets, it is not collected uniformly. For example, stop location is included in the warning and arrest databases as x/y coordinates, zip code, and street addresses, while the citation database only includes street addresses, which were missing directional indicators approximately half of the time in 2020. These datasets also differ in data fields beyond those that are required by TCOLE. For example, data regarding the APD Sector in which traffic stops occur (not a required field) are only collected in the warning and arrest databases, but not available for citations. The lack of uniformity across datasets hinders the analysis of these data. • APD’s stop data collection processes are tailored to meeting state statute and TCOLE requirements, but TCOLE’s reporting categories are not as meaningful for understanding stops and predicting stop outcomes as a specifically designed stop data collection system would be. By focusing on having an overall count of stops, the traffic stop data has been oversimplified to represent only a single reason for stop, outcome of stop, reason for search, and type of contraband seized. Important legally relevant information is thus lost in the process of creating mutually exclusive categories based on an ordered scale for reporting. For example, a stop that occurs for multiple reasons may be more likely to result in a more severe outcome. Kroll recognizes that APD cannot change the TCOLE requirements, but they are not prohibited from collecting more comprehensive information and adjusting it based on TCOLE standards for reporting purposes only. For example, in 2017 the state law’s definition for race/ethnicity was limited to only the following categories: Alaska Native or American Indian; Asian or Pacific Islander; Black; white; and 155  Hispanic or Latino; APD, however, includes Middle Eastern and Other in their race/ethnicity data field. As documented in APD’s internal 2018 Racial Profiling Report, they excluded stops for the latter two race/ethnicity categories in their report to TCOLE because they are not included in TCOLE’s reportable racial/ethnic categories but maintained the more detailed data for internal analysis purposes (APD, 2019). • The limited information that is currently collected and inconsistent across databases does not represent best practices in traffic stop data collection (Pryor et al., 2020). The information that should be collected includes the information that is currently required by state statute and TCOLE, but also includes more detailed information. APD should continue to collect the information that is required by state statute and TCOLE, but Kroll recommends that the department consider the inclusion of more detailed additional data fields (see Appendix, Section 7.5). It is recommended that APD create a small committee or task force (preferably one that includes community members) charged with reconsidering the information that is currently collected and making recommendations regarding proposed changes to enhance the data collection effort. This committee should also consider how much of the collected data should be made publicly available as part of APD’s open data portal to promote transparency to the community of Austin. • The selection of a traffic stop data system should necessarily consider current data management systems, budgetary constraints, and the department’s technological capacity. Ideally, traffic stop data should be collected electronically either via a mobile data computer or a handheld mobile device similar to the Brazos hand-held devices that are currently used for citations and street checks; this is less labor-intensive and more accurate than paper-based traffic stop forms (Fridell, 2004; Pryor et al., 2020). Data validation checks can be built into the data collection software. For example, the system can be set up to warn officers of missing data or a possibly invalid entry. The system can also be set up to eliminate conflicting data entry. For example, search reason and contraband seized would only be available to complete if the response to the search data field was yes. • Electronic data collection also can minimize data entry by auto-populating data fields if it can be linked to other existing electronic reports. For example, citations include a field for vehicle state of registration. If the data collection systems for citations and stops can be electronically linked, this information could be auto populated for stops that result in citation but might have to be manually entered for stops that result in warnings. Dropdown menus should be used whenever possible versus manual entry of text or numeric information as the latter increase the potential for data entry errors. 7.2 Use of Force Recommendations APD is currently engaged in a systematic, collaborative review and revision of several General Orders, including use of force, with the Office of Police Oversight at the direction of the City Manager. This is an ongoing process that has included an OPO assessment of six use-of-force related General Orders based on “8 Can’t Wait” recommendations, as well as best practices identified by the Police Executive Research Forum (PERF) and the International Association of Chiefs of Police (IACP); the city also conducted a survey of 1,400 community members who provided feedback on current policies and OPO’s recommended changes (OPO, 2021). This feedback and OPO’s final recommendations were published in October 2021 and include several suggested revisions to use of force-related policies. As of this report writing, APD and the City Manager are reviewing OPO’s final recommendations; final revision of policies is pending and will be based on legal review, policy development, input from City 156  Council, and other city stakeholders. As a result of this ongoing effort, we have attempted not to duplicate efforts in our below recommendations pertaining to use of force policy, training, and supervision: APD Policy • Definitions of subject resistance should be incorporated into General Order 200. The definitions of subject resistance were provided to our team via APD training materials, which provide examples of the types of resistance and indicate the types of force that are prohibited for certain types of resistance, but these levels of resistance are not currently defined in APD’s General Orders. • APD should revise the Response to Resistance/Use of Force policy to prohibit Neck Restraints and Chokeholds unless lethal force is authorized. A neck restraint or chokehold is considered lethal force. Neck Restraints or Chokeholds are defined as: o Carotid restraint hold: a hold that inhibits blood flow to the brain by compression of blood o Arm-bar control hold: a hold that inhibits breathing by compression of the airway of the vessels in the neck. neck. o Lateral vascular neck constraint. o A hold with a knee or other object to the back of a prone subject’s neck. • APD Search & Seizure 306, Section 306.9 should include the definition of “reasonable suspicion” and language mandating that officers specifically and clearly articulate in their official Department reports the facts upon which reasonable suspicion and probable cause are based, as a justification for any protective frisk. • APD should revise its General Order 211, Response to Resistance Inquiry, Reporting and Review to include a provision for consistent reporting/capture and evaluation by supervisors/commanders of the appropriateness of the pointing of firearms and/or actively targeting subjects. This would be an affirmative step to address lethal seizures that may be inappropriate, and to alleviate the risk for unfortunate and unjustified shootings. o In addition, pointing or actively targeting a subject with a firearm should be considered a reportable use of force incident (simply displaying or holding a firearm, as with low ready positions, without aiming it at a person would not apply). • APD should also revise General Order 211, Response to Resistance Inquiry, Reporting and Review to include a provision in Section 211.2.4, subsections 1 and 2, that makes clear that a subject’s verbal statements, bracing, or tensing alone do not constitute active resistance. • APD should add language to General Order 319, Arrests, Section 319.1.1 Arrests Requiring Supervisor Approval, to require that the responding/approving supervisor review the arrest documentation to determine whether specific and articulable facts support probable cause for the arrest or reasonable suspicion for the stop and resulting detention and/or frisk. The policy should also require that available witnesses be identified. Training • Individuals who are under the influence of drugs/alcohol or have mental health issues represent nearly 80% of those who had force used against them by APD during our four-year study period. Kroll recommends that APD assess its use of force, mental health, and crisis related training to determine where enhancements can be made. Specifically, APD should continue to focus on use of force training that emphasizes de-escalation skills for interactions with persons acting erratically 157  or in crisis. Recent research shows that regular de-escalation refresher training minimizes the possibility of training decay (Engel et al., 2020). • The APD Training Academy should provide ongoing in-service training that is co-facilitated by outside content experts for all APD first line and mid-level supervisors and commanders regarding the constitutional provisions of stops, seizures, and searches, and how to properly analyze the use of force. Supervision • APD supervisors must be held accountable for assessing the appropriateness of any underlying charges and evaluating incidents before authorizing any Resisting Detention, Search, Arrest or Transportation criminal charges against any individual.111 • Regardless of the type of data that is collected and the method of collection, proper training and accountability for the data collection processes are critical. It is important to ensure that officers are completing all forms and reports accurately and entirely using regular data quality reviews (Fridell, 2004). The presence of unknown and missing data requires different corrective measures by APD. For example, a variable with higher rates of unknown values might require clarification, retraining, or the revision of the data fields to include new categories. In contrast, high rates of missing data should involve more accountability and oversight measures to reduce mistakes or identify any patterns in purposeful omission. • Inconsistencies across reports within the same incident are dependent upon supervisory review. The use of force data reviewed by Kroll were based on compiled reports from all involved officers and, for some data fields, there was disagreement between separate reports from two or more officers. For example, in approximately 4% of use of force events, officers filled out different locations in the post-force transportation field. To minimize this incongruence, it is imperative that APD supervisors reviewing multiple use of force reports review for consistency and reliability. 7.3 Organizational Recommendations It is critical that readers of this report and APD are aware of the limitations of what aggregate police data can and cannot tell us; individual officer bias or lack thereof cannot be determined through statistical analyses. Fridell (2004) addressed the concern over how to interpret findings from data analyses as it pertained to traffic stop studies, but the implications also extend to analysis of other police outcomes. She noted, “because the data will never ‘prove’ or ‘disprove’ racially biased policing, we contend that vehicle stop data collection and analysis should never be viewed—either by police or resident stakeholders—as a ‘pass/fail test.’” Rather, she argued that “it should be viewed as a diagnostic tool to help pinpoint the decisions, geographic areas, and procedures that should get priority attention when the agency, in concert with concerned residents, identifies its next steps for addressing the problem or perception of racial profiling.” With this view of the data analyses, Kroll makes the following recommendations to APD administrators. • Examine the trend of increasing force despite decreasing arrests. APD leadership should closely examine the consistent increase in use of force (58.4% from 2017-2020) while arrests steadily declined by 51% over the same time period. Although the decrease is fairly consistent 111 Although APD has informed Kroll that, starting in 2021, the APD Force Review Unit conducts use of force reviews within APD, this does not absolve APD supervisors from their immediate and pro-active responsibilities. 158  across racial/ethnic groups, this is a trend that needs to be closely examined. Our analysis did not find a similar increase in average resistance toward officers during use of force encounters. It is possible that as APD increases its use of alternatives to arrest for less serious crimes, that individuals that are arrested are involved in more serious or violent crimes. Kroll was unable to examine severity of charges with the available APD data. As noted above, an emphasis on de- escalation and crisis response training may reduce use of force. It should, however, be part of an overall organizational culture (evident in supervisory oversight, policy, as well as training) that prioritizes crisis resolution through de-escalation and resorts to force only when alternatives are not viable. • Examine racial/ethnic disparities and continue to monitor them over time. Collecting, regularly analyzing, and making public enforcement data are important accountability measures for the police. Determining if and where racial/ethnic disparities in policing outcomes exist is critical for police agencies and communities. Kroll’s examination of racial/ethnic disparities in enforcement outcomes showed there were significant racial/ethnic disparities in force for Blacks and Hispanics when comparing to population statistics, but disparity ratios created based on risk-based comparison groups (e.g., arrestee or criminal suspect populations) consistently demonstrated much reduced disparities in use of force for Black and Hispanic individuals compared to white individuals. Force severity and resistance toward police officers did not significantly differ by race/ethnicity. Multivariate analyses predicting use of force, citations, arrests, and searches showed that legally relevant variables were the strongest predictors of these outcomes, but small to moderate statistically significant racial/ethnic differences remained. • Once improvements to APD data are made, Kroll recommends that regular analysis of these data should evaluate whether observed disparities change over time. In doing so, police executives can and should also test the impact of changes in policies, practices, and training over time. It is especially important that APD’s future examinations of racial/ethnic disparities be based on alternative benchmarks that better approximate risk of police contact and enforcement than population-based comparisons do. Although one of the OPO Joint Reports examining traffic stops indicated the use of census statistics for comparison purposes was widely accepted, the opposite is true; two decades of police research have detailed the significant limitations of population statistics as a benchmark for traffic stops and use of force. All benchmarks are flawed, but some are recognized in the scholarly literature as being more appropriate than others (Engel & Calnon, 2004; Fridell, 2004). Some of the more reliable benchmarks like observations of roadway usage and driving behavior are cost-prohibitive and time-consuming (Alpert et al., 2004; Engel, Calnon, Liu, & Johnson, 2004; Lange, Blackman, & Johnson, 2001; Tillyer et al., 2010). Others, however, rely on data that already exist and are easier to attain, including accident data, photo enforcement, veil of darkness analyses, and propensity scores (Alpert et al., 2004; Groggery & Ridgeway, 2006; Lovrich, Gaffney, Mosher, Pickerill, & Smith, 2003; McConnell & Scheidegger, 2001; Montgomery County Department of Police, 2002; Ridgeway & MacDonald, 2010; Ross et al., 2020; Smith et al., 2021; Stacey & Bonner, 2021; Tillyer et al., 2010; Vito, Griffin, Vito, & Higgins, 2020; Worden, McLean, & Wheeler, 2012). Use of force rates should continue to be compared against arrest and suspect based benchmarks as we have used in this report. • Treat statistical findings as a diagnostic tool. Although the traffic stop data could not be explored at lower organizational units within APD, the use of force and arrest data did allow for analyses at the sector level, which allow APD to further examine areas that may provide evidence of potential problems. Specifically, Kroll recommends that APD closely examine the enforcement practices used by officers in George Sector, a clear outlier in comparison to the other nine sectors. 159  George Sector is the geographically smallest and least populated APD sector, but it has an arrest rate that is five times the rate of the next closest sector (Ida), and the highest percentage of arrests resulting in force across APD sectors. George Sector also had higher rates of racial/ethnic disparities in uses of force compared to other sectors, regardless of the benchmark selected for comparison. Although the crime rate is relatively high in George Sector, the arrest rate and use of force rate far exceeds what might be expected, even based on this elevated level of crime, and by the trends between crime, arrests, and force evident in other sectors. • Adopt a holistic approach. For police agencies to properly respond to racial and ethnic disparities, it is not enough to analyze quantitative data alone; it is critical to engage police administrators, officers, and community stakeholders to better understand the context within which these encounters occurred.112 This approach to data collection and analysis focuses on understanding the reasons behind disparities, increasing trust between the police and the public they serve, and collaboratively developing strategies to respond to observed disparities. The City of Austin and APD are already well positioned to engage in this type of collaborative response to racial/ethnic disparities based on the existing Office of Police Oversight. • Explore other data sources to better understand the potential factors contributing to racial/ethnic disparities. Specifically, analysis of additional data sources may be informative in identifying possible sources of racial/ethnic disparities, including staffing levels, calls for service volume, targeted enforcement efforts, crime rates, commercial areas (e.g., retail, tourism, entertainment, etc.), locations of accidents, and body camera footage review (Ross et al., 2020). Community requests for changes in enforcement activity and political influences may also explain changes in police behavior. It is not unusual for increases in law enforcement activities to increase racial/ethnic disparities in outcomes. Reducing racial/ethnic disparities has proven to be an incredibly challenging and sometimes elusive goal for police executives. Yet increases in racial/ethnic disparities can happen rapidly and are often linked to changes in policies or practices that may be race-neutral on their face but sometimes lead to unintended consequences. It is important to engage in a dialogue regarding enforcement activities and whether they meet the goals of both the police agency and the community. • Finally, there should be a shared acknowledgment among all interested parties (e.g., the APD, the City of Austin, the community) of the limitations of the data, methodologies, and statistical techniques in evaluating police enforcement outcomes. As stated above, it is unknown if the racial and ethnic disparities observed are due to actual racial bias by police, or other unmeasured factors (Engel, 2008). Interpretation of the results and conclusions drawn from them need to be nuanced and in support of the goal of reducing racial/ethnic disparities in coercive outcomes. Results from these types of studies have real-life consequences for various stakeholders and impact policymaking, judicial decisions, legislation, and public opinion. The problem with assuming all racial/ethnic disparities are due to police bias is that the solutions that are subsequently offered focus on police bias, but if there are other explanations for the disparities, corrective measures 112 A prominent example of this process in action is the Connecticut Model, which conceptualizes traffic stop data collection and analysis as the beginning, rather than the end, of a data-driven conversation regarding disparities between researchers, police, and the public (Ross et al., 2020). They describe the four phases of the model as follows: Phase 1: Collection of traffic stop data; Phase 2: Empirical analysis of data by researchers to identify disparities that warrant further scrutiny; Phase 3: Researchers and police administrators collaboratively explore possible factors that may have contributed to the disparities (e.g., collisions, calls for service, crime rate, tourism, etc.); and Phase 4: Police administrators, researchers, and other community stakeholders collectively identify policy, training, and other responses that can potentially reduce reported disparities. 160  rooted in the assumption of police bias will likely not actually reduce the racial/ethnic disparities identified. Further, these potentially unsupported accusations of officer bias serve to erode community trust in police and worsen police-citizen interactions. Perhaps the racial/ethnic disparities in police outcomes that are reported across the country are due to individual officer bias, but they may be the result of systemic societal issues that are beyond the capacity of local police to control. 7.4 Recruitment, Selection, and Promotion Recommendations Based on Kroll’s findings and observations as detailed in Section 6, we make the following recommendations concerning APD’s polices and processes concerning recruitment, selection, and promotions: Recruitment • As illustrated by APD’s current personnel demographics, further efforts are needed to increase recruitment and hiring for individuals who represent the diversity of Austin, particularly for Hispanics and females. Moreover, although Black representation in APD is generally equivalent to their proportion of Austin residents, given the disparate impact that law enforcement activity has historically had on the Black community, Kroll suggests that APD should aspire to achieving a higher-than-proportionate representation of Blacks in APD as compared to the general population. • In 2015-2017, the job fairs that specifically focused on women’s groups and religious organizations produced more attendees than in recent years. Kroll recommends that APD further examine this. For example, how do past marketing strategies for those job fairs compare to now? • The Recruiting Unit should continue to develop community partnerships to enhance APD recruitment efforts and should formally document and track these interactions (e.g., who and what method) to ensure that they are regularly meeting with all key stakeholders in the community and tracking partnerships for further development. • The Recruiting Unit should improve and increase follow-up communications with recruits by phone and email. To the extent practical, the Recruiting Unit should collect and analyze information from those who choose not to apply regarding the reasons for their decisions. Some reasons will likely not be actionable by the Recruiting Unit, but others may be. Hosting a group application workshop might also increase the application rate (DOJ/EEOC, 2016). • APD should begin to collect data on how informational campaigns and recruiters distribute information to recipients on police officer benefits, salaries, educational and promotional opportunities, and to what extent certain data or information attracted them to APD. Collecting this data can help APD develop new and better ways to promote police officers’ benefits and attract more applicants. • The Explorer Program should be reinstated and enhanced to reach a diverse group of youth who are attracted to careers in law enforcement. o Kroll agrees with APD’s plans to reinstitute and expand the internship program, as historically this program has attracted a diverse pool of applicants, provides an opportunity for APD and the intern to mutually evaluate the fit, and helps prepare interns for the selection process. APD should also continue expanding partnerships with local schools, colleges and universities, community organizations, faith-based organizations, and social 161  service agencies, particularly those with a focus on historically underrepresented populations (DOJ/EEOC, 2016; Morison, 2017). • APD should explore methods of formally linking the recruiting data to applicant and academy performance data.113 This is an important area for improvement because it is possible there is variation in success in the selection process and academy performance by recruitment type. • As mentioned in Section 6, there will always be times when recruits are not a good fit for the APD. While this is to be expected, given the current state of law enforcement, negative media coverage, and community demands being placed on the profession, APD might consider utilizing a Realistic Job Preview in addition to the videos and online media it currently uses. A Realistic Job Preview would be designed to show recruits what being an APD officer realistically entails. Designed properly, it would (1) be visual, verbal, and tactile; (2) demonstrate mundane tasks like report writing and portray non-emergency and emergency community service; (3) show police engaging with challenging and cooperative community members alike; and (4) show how diversity in the workforce and community is valued. In short, it would attempt to capture the full spectrum of the job so that it is described accurately and realistically. The U.S. Office of Personnel Management offers a quick reference guide for creating an effective Realistic Job Preview.114 • Although Kroll finds that the APD reapplication method is consistent with modern approaches, APD should continue to evaluate the disqualification factors and the speed at which applicants may reapply to ensure that no potentially qualified applicant is unnecessarily excluded from applying. • APD should collect in a standardized way why applicants separate during the hiring process and maintain these data for each cadet class. Data fields that should be included are: name of applicant; unique numeric identifier assigned; race, age, and gender of applicant; date of application; date application intake review completed; all test dates and scores (or pass/fail outcomes); date of disqualification; where in the selection process disqualification occurred (e.g., a drop down menu with list of all process steps);115 the reason disqualification occurred (e.g., a drop down menu with standardized categories116); and whether the candidate was accepted into the Academy. Notes or remarks can still be utilized but these should be used to provide context for the systematically collected information rather than in lieu of it. o This information should be compiled and analyzed in a written review for each cadet class selection process. Examining this information regularly will allow APD to continuously improve the recruitment and selection process with each cadet application cycle. Having documentation of the sequencing of disqualifications may also provide actionable items for the Recruiting Unit. For example, there are some disqualifying factors utilized by APD that 113 It is possible that APD has the information to link these datasets, but this was not provided to Kroll. 114 https://www.opm.gov/policy-data-oversight/assessment-and-selection/other-assessment- methods/realistic_job_preview.pdf 115 We recommend including the initial application intake review, as well as each Phase I and II test (separating the drug screen from the medical exam), and background investigation. 116 Based on Kroll’s review of the disqualification data, we recommend the following categories at a minimum: does not meet minimum eligibility, failure to provide required documentation or incomplete documentation, failed required test (the specific test will be captured in the process step drop down menu), criminal history, credit history / financial collection issues, driving record / driver license suspension, past DUI, other than honorable military discharge, past drug use (use notes for type of drug, when used, frequency, etc.), making false statements / deception, medical disqualifier, failed to attend required testing/interviews; stopped responding to contact, voluntary withdrawal (use notes for further explanation if available). 162  are having an adverse impact on certain racial and ethnic minority groups. Kroll recommends reviewing these data in a more in-depth manner to determine root causes. For example, concerning the finding that Black applicants were disqualified more often than other applicants due to their credit, is this because many of these applicants did not submit the required documentation, or was it due to a negative credit history? • APD recruiting staff should better assist and advise applicants on the document submission requirements. In general, recruiters should increase interaction with applicants throughout the hiring process, although Kroll understands this is an admittedly challenging suggestion given the large number of APD recruits. However, there may be email and text options for sharing information that helps answer questions for large groups of recruits. Selection Process • Kroll found racial disparities in APD’s entry level cognitive test, the NPST. Accordingly, APD should continue to monitor these disparities as more applicants complete the NPST test. If disparities persist or increase, APD should research the use of alternative applicant exams. o In the near term, APD should examine whether NPST scores predict Academy success. Currently, applicant data and Academy data cannot be linked to evaluate this empirical question; Kroll recommends addressing this situation by assigning applicants a numeric identifier that stays with them throughout the selection process and the Academy. • APD should retain an independent consultant to conduct a formal validation study of its physical fitness requirements at (1) pre-hire, (2) during the Academy, and (3) in the job of police officer to ensure that the pre-hire physical ability test measures the correct level of physical fitness while minimizing disparate impact against females and persons of color. Pending the results, it may be necessary to revise or replace the current physical ability test. According to the U.S. Office of Personnel Management: Many factors must be taken into consideration when using physical ability tests. First, employment selection based on physical abilities can be litigious. Legal challenges have arisen over the years because physical ability tests, especially those involving strength and endurance, tend to screen out a disproportionate number of women and some ethnic minorities. Therefore, it is crucial to have validity evidence justifying the job-relatedness of physical ability measures.117 • Regardless of which physical ability test APD ultimately administers, APD should find a method to emphasize to applicants the necessity of preparing for this test. Better preparation can help avoid injury and is a best practice for minimizing gender differences and reducing test anxiety. • APD should make several improvements to the record keeping and process of the oral board, including by (1) documenting oral board scores (not just pass/fail) in the previously recommended disqualification database; (2) maintaining copies of the oral board questions, rating criteria, anonymous scoring sheets, and panel members’ names and titles, (3) validating the oral board’s questions, (4) providing all panel members with appropriate training to ensure they are evaluating 117 https://www.opm.gov/policy-data-oversight/assessment-and-selection/other-assessment-methods/physical-ability- tests/ 163  applicants in an accurate and unbiased manner and documenting this training for all participants, and (5) adding a community member on the oral board interview panel. Promotional Process • For career development and promotional improvements, APD leadership and command staff should affirmatively support the development of formal mentorship and/or shadowing programs by helping to identify strong candidates to serve as mentors and encouraging participation among APD ranks, with particular emphasis on officers from historically underrepresented groups. • As noted in Section 6, it is uncommon for assessment centers, when used as part of promotional processes, to adversely impact minority candidates, particularly when they measure knowledge, skills and abilities verbally rather than in writing. APD should continue to carefully analyze the promotional score data of sergeants, lieutenants, and commanders, and consider whether other assessment centers might (1) produce lower disparate impact, (2) be a better measure of the critical competencies, and (3) assist in increasing the rate of promotion for Black and Hispanic candidates to leadership positions. o If the technical skills test component option is ever utilized for promotion, the process should consist of the same types of written scenarios as described in the General Order (that measure critical competencies needed for the job) but with verbal responses instead of written. This process would save time and have a higher probability of lowering disparate impact (Goldstein et al., 2001). • APD should reconsider the promotional test components, the weighting of each, and the passing requirements of the written test to allow entry into the assessment center, which together, could reduce group differences in test scores. • A better alternative to the current scoring system for promotions is a compensatory model; that is, all candidates who take the written test should also complete the assessment center, with the two scores weighted and combined. As noted, if the assessment center minimized group differences as the research literature suggests they should, a higher weight for the (new) assessment center, which is also a more direct measure of crucial competencies than a written test, would likely result in lower disparate impact than the current system. • APD personnel should continue to be afforded training opportunities when it is practical and financially feasible, particularly in support of advancing a specialized interest or leadership development. Kroll also recommends that APD formalize training plans through a career mapping program for each officer to the extent possible. The recommendations outlined in this report are intended to be a springboard from which APD can continue to make a good department even better, one more attuned to the pulse of the Austin community and the demands of policing a diverse, vibrant, and progressive city in 2022. Ultimately, the hard work of establishing a long-term foundation for a resiliency-based, guardian-centric, community-oriented police department has only begun. 164  7.5 Appendix to Section 7: Data Fields for Traffic Stop Data Kroll recommends that APD consider the inclusion of more detailed additional data fields in its collection of traffic stop data. The following comprehensive list of possible data fields is based on Kroll team members’ experiences with other police agencies and recommendations from a recent guidebook (Pryor et al. 2020)118: • Unique stop identifier: This would allow for the possibility that a single stop involves enforcement with the driver and/or passenger as well as provide accurate counts of both stops and the individuals in contact with the police. • Officer information: unique identifier to link to other databases that include: o Officer demographics: race, age, gender, education. o Job-related information: assigned unit (APD Sector and district), years of experience, and rank. • Driver or passenger information: o Data field denoting whether driver or passenger received enforcement action. o In addition to the required demographic information (gender, race/ethnicity), APD should also collect individuals’ age at time of stop119 and zip code of residency. Many agencies also collect information regarding: ▪ Impairment (e.g., alcohol, drugs, intellectual or developmental disability, or behavioral/mental health) ▪ Language barrier with the officer / limited English proficiency ▪ Behavior (e.g., civil, disrespectful, non-compliant, verbally resistant, physically resistant) ▪ Whether the individual has a criminal history, and if yes, what type. • Vehicle information:120 o State of vehicle registration, o Vehicle condition (measured as good, fair, or poor), o Number of passengers. • Stop information: o Stop date and time o Roadway type o Location: x/y coordinates or street address that includes directional indicators o Duration of stop in minutes 118 We also note that the two Joint Reports by the OPO / Equity Office published in 2020 previously made several similar recommendations regarding improving APD’s motor vehicle stop data collection. 119 It is possible that date of birth could be auto populated into the traffic stop data form from other reports (e.g., warning, citation, arrest), but there may be privacy concerns with releasing date of birth in the publicly available data. It is recommended that this information be transformed into age at time of stop prior to public release. If auto-population is not an option, simply capturing year of birth on the traffic stop data collection form would be sufficient. 120 Vehicle information is not currently available in the public open data portal dataset, although state of registration is included in the citation database. 165  o Some agencies have begun to denote whether stops were “intelligence-led,” which indicates that they were based on shared analytical information among law enforcement partners regarding hot spots, repeat offenders, and threats to public safety.121 • Reason for the stop: o Data field capturing whether stop was proactive (i.e., initiated by officer) or reactive (i.e., dispatched, flagged down by someone, etc.) o TCOLE requires the reason for stop to be categorized as a moving traffic violation, vehicle traffic violation, violation of law other than traffic, or pre-existing knowledge. ▪ These categories are mutually exclusive; that is, only one initial reason for the stop is provided in the available data. This is another data limitation, because in practice, multiple reasons for a traffic stop are possible. ▪ Additionally, these categories lack specification and would provide a more accurate picture of the legally relevant offending behavior if they were more detailed. While it would be unwieldy to include every specific type of violation that could prompt a traffic stop, further subcategories based on the most common types of violations within each of the first three TCOLE categories should be explored. For example, moving violations should be separated into speeding and other moving violations. If speeding, the amount over the posted speed limit should be captured. Vehicle traffic violations should capture whether the stop was based on an equipment, inspection or registration violation. • Enforcement outcomes of each stop: o Number of verbal and written warnings o Number of citations o Whether individual arrested o The violations or offenses associated with warnings, citations, and arrests should be included or able to be linked from an external data source. • Search related information (if applicable): all applicable categories should be selected by officers and available in the data instead of categorizing search reasons and types of contraband into mutually exclusive categories based on a hierarchy. o Search target: Person, vehicle, both. o Reason for search should include: Incident to arrest, inventory,122 officer safety (Terry frisk), plain view contraband, probable cause, consent, other o Whether consent to search was requested; if requested, whether consent granted. o Contraband seized: None, alcohol, cash (criminal proceeds), drugs (consider whether to include paraphernalia and whether personal use or sale/transportation amounts), stolen property, weapons, and other 121 As described by Pryor and colleagues (2020, p.16): “Requiring officers to record this information allows an agency to better assess the stops being made and orients officers to think critically about the reasons they are making a stop in the first place—which may make them less likely to act on any implicit biases.” 122 According to APD General Order 306.7, an “inventory is a legitimate law enforcement activity that is not a search and that sometimes, incidentally, results in the discovery of evidence.” Because it is possible that contraband could be seized as a result of vehicle inventory, it is advisable to include it as a reason for search. 166  o Consider whether to include a separate field for “no seizure, other criminal activity detected” to account for situations where there was evidence of illegal behavior but not anything that can be seized (e.g., drug residue, false logbooks in commercial vehicles, evidence of drug trafficking). • Pre-Stop Indicators of Suspicion: o Defined as those activities prescribed by the state’s criminal code. o Categories that other agencies have developed and adopted based on focus group interviews with officers: ▪ None (default category) ▪ Body language (Rigid posture, staring straight ahead, etc.) ▪ Driving behavior (Coasting, frequent lane changes, etc.) ▪ Passenger behavior (Overreaction to patrol car, furtive movements, etc.) ▪ Vehicle characteristics (Type, condition, modifications) ▪ Other (Included to allow for the changing nature of indicators of suspicion) Note that Kroll does not necessarily recommend that APD add all the fields above to a newly designed APD traffic stop form; rather, this list can serve as a guide for what is being collected and used by other agencies. The more relevant data elements that are included, the better researchers can understand statistical findings and make meaningful recommendations to police administrators regarding policy and training. However, any data collection effort must balance operational efficiency and analytical needs. 167  8. APPENDIX A 8.1 Kroll Project and Research Team John R. “Rick” Brown is a former Lieutenant Colonel and Deputy Commissioner for Administration and Professional Responsibility of the Pennsylvania State Police (PSP). During his 29-year tenure, Brown oversaw the PSP’s reform and accountability efforts in the areas of misconduct, sexual harassment, use of force, and early intervention/risk management initiatives. He developed the PSP’s Equal Employment Opportunity Office’s statewide liaison program and had oversight of citizen complaints that alleged discrimination or disparate treatment. Brown also oversaw the PSP’s five-year Police-Citizen Contact Project, which utilized applied research techniques to assess the extent to which PSP officers engaged in racial or biased-based policing. Brown subsequently oversaw the implementation of proactive training and operational strategies to monitor and prevent racial profiling. In 2010, following a distinguished career in law enforcement, Brown created Transparency Matters, LLC, a certified Minority-owned Business Enterprise (“MBE”) that focuses on building transparent policing policies and process change that provides organizational efficiencies, accountability, diversity, community education, training, and monitoring. Daniel Linskey, former Superintendent-in-Chief of the Boston Police Department, is a nationally renowned expert in urban policing, training, and police-community relations. As Head of the BPD from 2009 to 2014, Chief Linskey developed and oversaw the BPD’s social media, recruitment, community outreach, and engagement strategies, which have been cited as among the best practices in police management in the United States. Chief Linskey also changed the management and response of BPD to large-scale public disorder events. He oversaw the peaceful and successful management of the Boston Occupy movement and earned an international reputation at planning and overseeing major special events, and as the Incident Commander during the Boston Marathon Bombing Attack. Chief Linskey was part of a team sent to St. Louis County to assist the Department of Justice with an assessment of the St. Louis County Police Department and collaborative reform project following the events in Ferguson, Missouri in 2014. He is a frequent commentator on proper police tactics for national news organizations. Dr. Robin Engel, a Professor of Criminal Justice at the University of Cincinnati (“UC”) and Director of the UC Center for Police Research and Policy, is a nationally recognized expert on biased-based policing and one of the top-ranked female academics in the country. Dr. Engel has studied and written extensively on biased-based policing and worked with police departments throughout the United States to help them improve and reform. She has published over 60 peer-reviewed articles and book chapters and conducted research on such topics as biased-based policing, police-community relations, police use of force, police use of discretion / decision making, police legitimacy, violence reduction initiatives, reform efforts, and problem-oriented policing. Dr. Engel has conducted statistical analyses examining racial/ethnic disparities in policing outcomes for over a dozen jurisdictions. Dr. Jennifer Calnon Cherkauskas is a senior research associate at the University of Cincinnati Center for Police Research and Policy. She holds a doctorate in Crime, Law, and Justice from The Pennsylvania State University. Dr. Cherkauskas currently works with police agencies across the country as part of multiple research projects that are examining police use of force, traffic stops disparities, and violence reduction. She spent three years as the project manager and liaison to the external monitor for the University of Cincinnati Police Division’s voluntary reform agenda. Over the last twenty years, she has served as project manager for research projects with the Pennsylvania State Police and the Arizona Department of Public Safety and supported projects with the Ohio State Highway Patrol, the Nebraska State Patrol, and the Tulsa Police Department. She has published articles in Justice Quarterly, Journal of Crime and Justice, Police Quarterly, and Policing. 168  Dr. Nicholas Corsaro is a consultant for Kroll and serves as Associate Professor of Criminal Justice at the University of Cincinnati. He holds a PhD in Criminal Justice from Michigan State University. He has published over 30 articles on police interventions, strategies, and organizational processes. His research focuses on the role of the police in crime prevention with a particular emphasis on the use of strategies, tactics, and organizational policies. He has served as a principal investigator for a number of projects across various urban police agencies and has worked to develop rigorous evidence regarding the most viable, effective, and efficient practices that police have used to address serious crime problems. His research has been published in Crime and Delinquency, Criminology and Public Policy, Journal of Criminal Justice, Journal of Experimental Criminology, Journal of Quantitative Criminology, Justice Quarterly, as well as evaluation and public health journal outlets. J. Larry Mayes is serving as a Project Advisor on community engagement strategies and civilian input into police internal investigations. Mayes has worked with government officials and community leaders for more than two decades. From 2004 to 2010, Mayes served as the joint Cabinet Chief of Human Services government/community-based initiatives to reduce crime and stabilize communities. Currently he serves as Vice President of Programs for Catholic Charities in Boston, where he leads the organization’s statewide programs on adult education, immigration resettlement and legal services, childcare, and new poverty strategies. In 2014, Mayes was appointed to the Community Ombudsman Oversight Panel, which reviews the BPD’s Internal Affairs citizen complaint cases. the City of Boston, where he led for Dr. Cassi L. Fields is an expert in the design, development, validation, and administration of large-scale public safety human capital projects, with special expertise in selection and promotional systems in organizations with a history of alleged employment discrimination. Dr. Fields has pioneered many of the nation’s most successful human capital selection initiatives, promotion and training programs, and routinely teaches and writes about implicit bias. Since receiving her Ph.D. in 1989, Dr. Fields has dedicated her career to helping public safety agencies remove roadblocks for people of diverse backgrounds compete for promotions. She has developed hundreds of assessment centers for police, fire, and sheriff departments throughout the United States. A.J. Bingham is serving as an advisor in multi-cultural community outreach in Austin, facilitating local community input into police-citizen interactions, and providing insight into Austin’s business, civic, and non-profit communities. Bingham is Founder and CEO of The Bingham Group, LLC, a City of Austin-certified MBE, and full-service consulting firm that represents and advises clients on legislative and regulatory matters throughout Texas. An Austin native, Bingham is active in the community and serves on the boards of the Young Men's Business League of Austin, the Austin Trail of Lights Foundation, and the Long Center for the Performing Arts. A lawyer by background, Bingham received his B.A. in Political Science from Wake Forest University, and law degree from Washburn University School of Law. Mark Ehlers is the Engagement Leader of Kroll’s work with the City of Austin. Ehlers has 35 years of combined legal and investigative experience in the public and private sectors, specializing in government and university investigations, discrimination and harassment in the workplace, and alleged ethical violations. He is currently a managing director in Kroll’s Philadelphia office. Prior to joining Kroll, Ehlers served for 18 years as an Assistant United States Attorney, first in the District of Columbia, where he served in the Homicide and Sex Offense Units, and later in the Eastern District of Pennsylvania, where he served on the Organized Crime Strike Force. Since joining Kroll, Ehlers has conducted numerous internal investigations and best practice reviews for a diverse array of public and private sector clients, including the University of Cincinnati Police Department (review and investigation of a UCPD officer’s fatal shooting of an unarmed motorist) and North Carolina State Highway Patrol (review of hiring and selection practices, training and supervision following public reports of police misconduct), among others. Ehlers received a B.A., magna cum laude, from Wittenberg University, and a J.D., with honors, from George Washington University. 169  9. APPENDIX B 9.1 References Alpert, G., & Dunham, R. (1999). The force factor: Measuring and assessing police use of force and suspect resistance. Use of force by the police: Overview of national and local data (October-NCJ 176330; Use of Force by Police: Overview of National and Local Data, pp. 45–60). U.S. Department of Justice. Alpert, G. P., Smith, M. R., & Dunham, R. G. (2004). Toward a better benchmark: Assessing the utility of not-at-fault traffic crash data in racial profiling research. Justice Research and Policy, 6, 43–69. APA (2018). Principles for the Validation and Use of Personnel Selection Procedures. Society for Industrial/Organizational Psychology, 5th edition. APD. (2019). Response to Resistance Report: 2017. Austin, TX: APD Research & Planning Unit. APD. (2019). Annual Racial Profiling Report: 2018. Available at: https://www.austintexas.gov/sites/default/files/files/Police/2018_racial_profiling_report.pdf Archbold, C. A., Hassell, K. D., & Stichman, A. J. (2010). Comparing promotion aspirations among female and male police officers. International Journal of Police Science & Management, 12(2), 287-303. Bayley, D. H., & Garofalo, J. (1989). The management of violence by police patrol officers. Criminology; an Interdisciplinary Journal, 27(1), 1–26. https://doi.org/10.1111/j.1745-9125.1989.tb00861 Cantwell, J. D. (1985). Cardiovascular aspects of running. Clinics in sports medicine, 4(4), 627- 640.https://pubmed.ncbi.nlm.nih.gov/3902253/. Cesario, J., Johnson, D. J., & Terrill, W. (2019). Is there evidence of racial disparity in police use of deadly force? Analyses of officer-involved fatal shootings in 2015–2016. Social psychological and personality science, 10(5), 586-595. Cesario, K.A. Dulla, J.M., Moreno, M.R., Bloodgood, A.M., Dawes J.J., & Lockie, R.G. (2018). Relationships between Assessments in a Physical Ability Test for Law Enforcement: Is There Redundancy in Certain Assessments? International Journal of Exercise Science, 11(4), 1063-1073. Chen, H., Cohen, P., & Chen, S. (2010). How big is a big odds ratio? Interpreting the magnitudes of odds ratios in epidemiological studies. Communications in Statistics—simulation and Computation, 39(4), 860-864. Cochrane, R. E., Tett, R. P., & Vandecreek, L. (2003). Psychological testing and the selection of police officers: A national survey. Criminal Justice and Behavior, 30(5), 511-537. Dahm, J. (2006). Practical approaches to employee retention. Law and Order, 54(12), 68-71. Davis, E., Whyde, A., & Langton, L. (2018). Contacts between police and the public, 2015 (NCJ 251145). Washington, DC: U. S. Department of Justice. Department of Justice / Equal Employment Opportunity Commission. (2016). Advancing Diversity in Law Enforcement. Washington, D.C.: U.S. Department of Justice. Donohue, R. H. (2019). Shades of blue: A review of the hiring, recruitment, and selection of female and minority police officers. The Social Science Journal (Fort Collins). Engel, R. S. (2003). Explaining suspects' resistance and disrespect toward police. Journal of Criminal Justice, 31(5), 475-492. Engel, R. S., & Calnon, J. M. (2004). Comparing benchmark methodologies for police-citizen contacts: Traffic stop data collection for the Pennsylvania State Police. Police Quarterly, 7(1), 97-125. Engel, R. S., Calnon, J. M., & Bernard, T. J. (2002). Theory and Racial Profiling: Shortcomings and Future Directions in Research. Justice Quarterly, 19(2), 249–273. 170  Engel, R.S., Calnon, J.M., Liu, L. and Johnson, R.R. (2004), Project on Police-citizen Contacts, Year 1 Final Report, University of Cincinnati, Cincinnati, OH. Engel, R. S., Sobol, J. J., & Worden, R. E. (2000). Further exploration of the demeanor hypothesis: The interaction effects of suspects’ characteristics and demeanor on police behavior. Justice Quarterly, 17(2), 235–258. Engel, R.S., Corsaro, N., Isaza, G.T., & McManus, H.D. (2020). Examining the impact of Integrating Communications, Assessment, and Tactics (ICAT) de-escalation training for the Louisville Metro Police Department: Initial Findings. IACP/UC Center for Police Research and Policy. Equal Employment Opportunity Commission. (1978). Uniform guidelines on employee selection procedures. 43 Federal Register, 166. https://www.uniformguidelines.com/uniform- guidelines.html Farrell, A., McDevitt, J., Bailey, L., Andresen, C., & Pierce, E. (2004). Massachusetts Racial and Gender Profiling Study, Final Report. Submitted to the Massachusetts Department of Public Safety. Fridell, L. (2004). By the Numbers: A Guide for Analyzing Race Data from Vehicle Stops. Washington, D.C.: Police Executive Research Forum. Fryer, R. G. (2016). An Empirical Analysis of Racial Differences in Police Use of Force. National Bureau of Economic Research. https://doi.org/10.3386/w22399 Fryer, R. G. (2019). An empirical analysis of racial differences in police use of force. Journal of Political Economy, 127(3), 1210-1261. Fryer, R. G. (2020). An empirical analysis of racial differences in police use of force: A response. The Journal of Political Economy, 128(10), 4003-4008. Garner, J. H., Hickman, M. J., Malega, R. W., Maxwell, C. D. (2018). Progress toward national estimates of police use of force. PLoS ONE, 13(2), e0192932. Garner, J. H., Maxwell, C. D., & Heraux, C. G. (2002). Characteristics associated with the prevalence and severity of force used by the police. Justice Quarterly, 19(4), 705–746. https://doi.org/10.1080/07418820200095401 Garner, J. H., Schade, T., Hepburn, J., & Buchanan, J. (1995). Measuring the continuum of force used by and against the police. Criminal Justice Review, 20(2), 146–168. https://doi.org/10.1177/073401689502000204 Geller, A., Goff, P. A., Lloyd, T., Haviland, A., Obermark, D., & Glaser, J. (2020). Measuring racial disparities in police use of force: methods matter. Journal of Quantitative Criminology. https://doi.org/10.1007/s10940-020-09471-9 Goldstein, H.W., Yusko, K.P., & Nicolopoulos, V. (2001). Exploring black-white subgroup differences of managerial KSAs. Personnel Psychology, 54(4), 783-807. Grogger, J., & Ridgeway, G. (2006). Testing for racial profiling in traffic stops from behind a veil of darkness. Journal of the American Statistical Association, 101, 878-887. Haddad, A., Giglio, K., Keller, K. M., & Lim, N. (2012). Increasing Organizational Diversity in 21st-Century Policing: Lessons from the US Military. Santa Monica, CA: RAND Corporation. Henriquez, M. (1999). IACP national database project on police use of force (October-NCJ 176330; Use of Force by Police: Overview of National and Local Data, pp. 19–24). U.S. Department of Justice. Hickman, M. J., Piquero, A. R. & Garner, J. H. (2008). Toward a national estimate of police use of nonlethal force. Criminology & Public Policy, 7(4), 563-604. Hickman, M. J., Strote, J. N., Scales, R. M., Parkin, W. S., & Collins, P. A. (2021). Police use of force and injury: Multilevel predictors of physical harm to subjects and officers. Police Quarterly, 24(3), 267- 297. 171  Hine, K. A., Porter, L. E., Westera, N. J., & Alpert, G. P. (2018). Too much or too little? Individual and situational predictors of police force relative to suspect resistance. Policing and society, 28(5), 587-604. Hollis, M. E. (2018). Measurement issues in police use of force: a state-of-the-art review. Policing: An International Journal, 41(6), 844–858. https://doi.org/10.1108/PIJPSM-11-2017-0137 Hom, P., Ellis, A, & Roberson, L. (2008). Challenging Conventional Wisdom About Who Quits: Revelations From Corporate America, Journal of Applied Psychology (February). Hough, L.M., Oswald, F.L., & Ployhart, R.E. (2001). Determinants, detection and amelioration of adverse impact in personnel selection procedures: Issues, evidence and lessons learned. International Journal of Selection and Assessment 9(1-2), pp. 152-194. Hunter, J.E. & Hunter, R.F. (1984) Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96(1), 72-98. Hyland, S. S., & Davis, E. (2019). Local police departments, 2016: Personnel (NCJ 252835). U.S. Department of Justice, Office of Justice Programs, Bureau of Justice Statistics. Jennings, J. T., & Rubado, M. E. (2017). Preventing the use of deadly force: The relationship between police agency policies and rates of officer‐involved gun deaths. Public Administration Review, 77(2), 217-226. Jones, D. (2011). Upper Versus Lower Body Contribution to the Rowing Stroke. https://engagedscholarship.csuohio.edu/cgi/viewcontent.cgi?referer=&httpsredir=1&article=1725& context=etdarchive. Klahm, C. F., & Tillyer, R. (2010). Understanding police use of force: A review of the evidence. Southwest journal of criminal justice, 7(2), 214–231. Klinger, D. A., & Bridges, G. S. (1997). Measurement error in calls-for-service as an indicator of crime. Criminology, 35(4), 705-726. Knox, D., W. Lowe, and J. Mummolo. 2020. Administrative Records Mask Racially Biased Policing. American Political Science Review. https://www.cambridge.org/core/journals/american-political- science-review/article/administrative-records-mask-racially-biased- policing/66BC0F9998543868BB20F241796B79B8 Knox, D., & Mummolo, J. (2020). Toward a general causal framework for the study of racial bias in policing. Journal of Political Institutions and Political Economy, 1(3), 341-378. Lautenschlager, R. & Omori, M. (2019). Racial threat, social (dis)organization, and the ecology of police: Towards a macro-level understanding of police use of-force in communities of color, Justice Quarterly, 36(6), 1050-1071. Lee, J. (2016). Police use of nonlethal force in New York City: Situational and community factors. Policing and Society, 26(8), 875-888. Lee, H., Vaughn, M. S., & Lim, H. (2014). The impact of neighborhood crime levels on police use of force: An examination at micro and meso levels. Journal of Criminal Justice, 42(6), 491-499. Lovrich, N., Gaffney, M., Mosher, C., Pickerill, M., & Smith, M. R. (2003). WSP traffic stop data analysis project. Pullman: Washington State University. Available: https://www.academia.edu/download/47942704/WSP_Traffic_Stop_Data_Analysis_Project20160 810-12920-1ghf7g5.pdf McConnell, E. H. & Scheidegger, A. R. (2001), “Race and Speeding Citations: Comparing Speeding Citations Issued by Air Traffic Officers With Those Issued by Ground Traffic Officers,” paper presented at the annual meeting of the Academy of Criminal Justice Sciences, Washington, DC, April 4–8. McDevitt, J. Farrell, A., & Wolff, R. (2008). Promoting Cooperative Strategies to Reduce Racial Profiling. COPS Evaluation Brief No. 1. Washington, D.C.: Office of Community Oriented Policing Services, U.S. Department of Justice. https://cops.usdoj.gov/RIC/Publications/cops-p146-pub.pdf 172  Montgomery County Department of Police (2002), “Traffic Stop Data Collection Analysis,” 3rd report. Moore, R., Sanchez, E., & Ofelia San Pedro, M. (2018). Investigating Test Prep Impact on Score Gains Using Quasi-Experimental Propensity Score Matching. ACT.org, 2018-6. Morgan, M. A., Logan, M. W., & Olma, T. M. (2020). Police use of force and suspect behavior: An inmate perspective. Journal of Criminal Justice, 67. https://doi.org/10.1016/j.jcrimjus.2020.101673 Morison, Kevin P. 2017. Hiring for the 21st Century Law Enforcement Officer: Challenges, Opportunities, and Strategies for Success. Washington, DC: Office of Community Oriented Policing Services. Office of Police Oversight. (no date). Learn how Austin Police Department’s policies are changing. Available at: https://alpha.austin.gov/en/police-oversight/learn-how-austin-police-departments- policies-are-changing/ Office of Police Oversight. (2021). Community Feedback and Final Recommendations for 8 Can't Wait Policy Initiatives. Available at: https://joplin3-austin-gov- static.s3.amazonaws.com/production/media/documents/final_doc_Community_feedback_8CW.p df Office of Police Oversight, Office of Innovation, and Equity Office. (2020, January). Joint Report: Analysis of APD Racial Profiling Data, 2015-2018. Available at: https://joplin3-austin-gov- static.s3.amazonaws.com/production/media/documents/Final_- 1.28.2020_Racial_Profiling_Report.pdf Office of Police Oversight, the Equity Office, and Office of Innovation. (2020, November). 2019 Joint Report: Analysis of APD Racial Profiling Data. Available at: https://joplin3-austin-gov- static.s3.amazonaws.com/production/media/documents/2019_Joint_Report_Analysis_of_APD_R acial_Profiling_Data_FINAL.pdf PERF (March 2016). Guiding Principles on use of force. Washington, DC: Author. https://www.policeforum.org/assets/30%20guiding%20principles.pdf PERF (2021). What Police Chiefs and Sheriffs Need to Know About Collecting and Analyzing Use-of- Force Data. Police Executive Research Forum. Pryor, M., Goff, P.A., Heydari, F., & Friedman, B. (2020). Collecting, Analyzing, and Responding to Stop Data: A Guidebook for Law Enforcement Agencies, Government, and Communities. Ramirez, D., McDevitt, J., & Farrell, A. (2000). A resource guide on racial profiling data collection systems: Promising practices and lessons learned. Washington, DC: U.S. Department of Justice. Reaves, B. A. (2012). Hiring and retention of state and local law enforcement officers, 2008–Statistical tables. Washington, D.C.: Bureau of Justice Statistics (BOJ). Ridgeway, G. (2007). Analysis of Racial Disparities in the New York City Police Department’s Stop, Question, and Frisk Practices (Technical Report). RAND Corporation. Ridgeway, G., & MacDonald, J. (2010). Methods for assessing racially biased policing. In S. K. Rice & M. D. White (Eds.), Race, ethnicity, and policing: new and essential readings (pp. 180–204). NYU Press. https://doi.org/10.18574/nyu/9780814776155.003.0007 Robinson, D. M., & Reio, T. G. (2012). Benefits of mentoring African-American men. Journal of Managerial Psychology, 27(4), 406-421. Ross, M. B., Kalinowski, J. J., & Barone, K. (2020). Testing for disparities in traffic stops: Best practices from the Connecticut model. Criminology & Public Policy, 19(4), 1289–1303. https://doi.org/10.1111/1745-9133.12528 Roth, P., Bobko, P., McFarland, L., & Buster, M. (2008). Work sample tests in personnel selection: a meta‐analysis of black–white differences in overall and exercise scores. Personnel Psychology, 61(3), 637-661. Sherman, L. W. (2013). The rise of evidence-based policing: Targeting, testing, and tracking. Crime and Justice, 42(1), 377-451. 173  Shjarback, J. A., White, M. D., & Bishopp, S. A. (2021). Can police shootings be reduced by requiring officers to document when they point firearms at citizens? Injury prevention. Shjarback, J. A., & Nix, J. (2020). Considering violence against police by citizen race/ethnicity to contextualize representation in officer-involved shootings. Journal of Criminal Justice, 66, 101653. https://doi.org/10.1016/j.jcrimjus.2019.101653 Shjarback, J. A., & Todak, N. (2019). The prevalence of female representation in supervisory and management positions in American law enforcement: An examination of organizational correlates. Women & Criminal Justice, 29(3), 129-147. Sklansky, D. A. (2005). Not your father's police department: Making sense of the new demographics of law enforcement. Journal of Criminal Law & Criminology, 96, 1209-1243. Smith, D. (1986). The neighborhood context of police behavior. In A. Reiss and M. Tonry (Eds.), Crime and justice: A review of research. Chicago: University of Chicago Press. Smith, M. R., Rojek, J., Tillyer, R., & Lloyd, C. (2017). San Jose police department traffic and pedestrian stop study. El Paso, TX: University of Texas at El Paso, Center for Law and Human Behavior. San Jose Police Department website. http://www. sjpd. org/records/utep-sjpd_traffic- pedestrian_stop_study_2017. pdf. Smith, J. M., Tillyer, R., Engel, R. S., & Cherkauskas, J. C. (2019). A Multi-Method Investigation of Officer Decision-Making and Force Used or Avoided in Arrest Situations: Tulsa, Oklahoma Police Department Administrative Data Analysis Report. IACP/UC Center for Police Research and Policy. Stacey, M., & Bonner, H. S. (2021). Veil of Darkness and Investigating Disproportionate Impact in Policing: When Researchers Disagree. Police Quarterly, 24(1), 55-73. Stöber, J., & Pekrun, R. (2004). Advances in test anxiety research (Editorial). Anxiety, Stress, & Coping, 17(3), 205-211. Stroshine, M. S., & Brandl, S. G. (2019). The use, effectiveness, and hazards associated with police use of force: the unique case of weaponless physical force. Police Practice and Research, 1–18. https://doi.org/10.1080/15614263.2019.1582342 Stroshine, M. S., & Brandl, S. G. (2011). Race, gender, and tokenism in policing: An empirical elaboration. Police Quarterly, 14(4), 344-365. Suboch, G., Harrington, C., & House, J. (2017). Why do female and minority police officers remain in law enforcement? Race, Gender & Class (Towson, Md.), 24(3-4), 100-118. Task Force. (2015). President’s Task Force on 21st Century Policing, Final Report of the President’s Task Force on 21st Century Policing. Washington, DC: Office of Community Oriented Policing Services. Available at: https://cops.usdoj.gov/pdf/taskforce/taskforce_finalreport.pdf. Terrill, W. (2003). Police Use Of Force And Suspect Resistance: The Micro Process Of The Police- Suspect Encounter. Police Quarterly, 6(1), 51–83. https://doi.org/10.1177/1098611102250584 Terrill, W., & Mastrofski, S. D. (2002). Situational and officer-based determinants of police coercion. Justice Quarterly, 19. Terrill, W., Paoline III, E. A., & Ingram, J. R. (2018). Beyond the final report. Policing: An International Journal, 41(2), 194–201. https://doi.org/10.1108/PIJPSM-04-2017-0047 Terrill, W., & Reisig, M. (2003). Neighborhood context and police use of force. Journal of Research in Crime and Delinquency, 43, 291-321. Tillyer, R., Engel, R. S., & Cherkauskas, J. (2010). Best practices in vehicle stop data collection and analysis. Policing, 33(1), 69. Tillyer, R., Engel, R. S., & Wooldredge, J. (2008). The intersection of racial profiling research and the law. Journal of Criminal Justice, 36(2), 138–153. https://doi.org/10.1016/j.jcrimjus.2008.02.004 Todak, N., Leban, L., & Hixon, B. (2021). Are women opting out? A mixed methods study of women patrol 174 officers’ promotional aspirations. Feminist Criminology, 16(5), 658-679. Tregle, B., Nix, J., & Alpert, G. P. (2019). Disparity does not mean bias: making sense of observed racial disparities in fatal officer-involved shootings with multiple benchmarks. Journal of Crime and Justice, 42(1), 18–31. https://doi.org/10.1080/0735648X.2018.1547269 Vito, A. G., Griffin, V. W., Vito, G. F., & Higgins, G. E. (2020). “Does daylight matter”? An examination of racial bias in traffic stops by police. Policing: An International Journal. Westlin, Joseph (2014). Physical Ability Testing: A Review of Court Cases 1992-2014, Masters Thesis, Western Kentucky University. Whichard, C., & Felson, R. B. (2016). Are suspects who resist arrest defiant, desperate, or disoriented? Journal of Research in Crime and Delinquency, 53(4), 564-591. Wolf, R., Mesloh, C., Henych, M., & Thompson, L. F. (2009). Police use of force and the cumulative force factor. Policing: An International Journal of Police Strategies & Management, 32(4), 739-757. Worden, R. E., McLean, S. J., & Wheeler, A. P. (2012). Testing for racial profiling with the veil-of- darkness method. Police Quarterly, 15(1), 92-111. Worrall, J. L., Bishopp, S. A., & Terrill, W. (2020). The effect of suspect race on police officers’ decisions to draw their weapons. Justice Quarterly, 1-20. 175 MSOAUTOSHAPETYPE About Kroll Kroll is the leading global provider of risk solutions. For more than 45 years, Kroll has helped clients make confident risk management decisions about people, assets, operations and security through a wide range of investigations, cyber security, due diligence and compliance, physical and operational security, and data and information management services. For more information, visit www.kroll.com.