The Bureau of Justice Statistics (BJS) maintains a research program known as the National Crime Victimization Survey (NCVS) that measures and tracks nonfatal victimizations in the United States. The NCVS is an official source of crime data and the primary source of unreported crime. It collects detailed information on the frequency and nature of several offenses, including: rape, robbery, aggravated assault, simple assault, larceny, burglary, trespassing, motor vehicle theft, and other types of household theft.
Recently, BJS has undertaken several efforts to increase the efficiency, reliability, and utility of the NCVS. First was a revamp in the urbanicity measure used to classify areas as rural, suburban, or urban. Second, the NCVS also began collecting subnational estimates to help make data more precise. Third, the NCVS survey was completely re-designed to help improve validity of the estimates. Based on recently released reports, it appears that the revised version of the NCVS shows clear improvements over the previous version. In this post, I will discuss these improvements (and why they matter) in more detail.
New Measures of Urbanicity
Historically, the NCVS has relied on very poor measures of urbanicity, which were based solely on an area’s proximity to the nearest Metropolitan Statistical Area (MSA), without taking population density or housing density into account. Areas within a principal city of an MSA were classified as urban, areas within an MSA but not within a principal city were classified as suburban, and areas outside of an MSA were classified as rural. MSAs were based on entire counties, and each county would be classified as entirely non-rural (if part of an MSA) or entirely rural (if not part of an MSA). Problematically though, it is rarely the case that a county is entirely rural or entirely non-rural, and this measure was likely never valid to begin with. The measure also did not take population density into account, so many small towns with low population densities would end of being classified as “urban” due to their distance from the nearest MSA.
For example, California’s San Bernardino County covers more than 20,000 square miles and includes much of the Mojave Desert, but it was previously classified as having “no rural areas” because the county is part of the Riverside-San Bernardino-Ontario MSA. In contrast, Colorado’s La Plata County was previously classified as “entirely rural” because it is not part of an MSA, despite Census Bureau data showing that 40% of the population lives in non-rural areas.
The new measure for urbanicity is based accounts for both housing density and the distance to the center of a major metropolitan area, which is a major difference. The new criteria classify an area as “urban” if it meets one of the following: 1) within a primary city of a large metropolitan area (500,000+ persons) and has a housing unit density of at least 3,000 units per square mile; 2) within a major (but not primary) city of a large metropolitan area and has a housing unit density of at least 4,000 units per square mile; 3) within a major city of a medium-sized metropolitan area (50,000-499,999 persons) and has a housing unit density of at least 5,000 units per square mile; and 4) within a major city of a small (10,000-49,999 persons) metropolitan area and has a housing unit density of at least 10,000 units per square mile. Suburban areas are characterized by lower housing densities, larger ratios of single-family homes to apartments, and layouts based principally on automobile transportation. These areas are a mix of “suburbia proper,” towns, and some smaller cities. Everything else is classified as rural.
According to the historical NCVS definition, the most urban region in the country was the West, and the most suburban region was the Northeast. Under the new definitions, the Northeast is by far the most urban region and the West is the most suburban region. An entire list of places classified as urban can be viewed in the appendix of this report.
These new definitions more closely fit U.S. residents’ own sense of where they live, as reflected in the Census Bureau’s American Housing Survey (AHS). With revised definitions, researchers now have the opportunity to accurately compare victimization rates across rural, suburban, and urban areas. The first of these reports was actually recently published, and it compared the prevalence of victimization across rural, suburban, and urban areas.
Redesigning the Instrument
The NCVS Instrument Redesign and Testing Project was a major multiyear effort undertaken by Westat, Inc. on behalf of the BJS to revamp the existing core survey instrument, which was last updated in 1992. The researchers from Westat developed and assessed the new instrument in a large-scale national field test. The project aimed to modernize the core NCVS instrument, which included changes to the victimization screener and flow and logic of the instrument, the addition of new measures of police performance and community safety, and expanded measures of correlates of victimization and help-seeking.
A recent report by Westat describes the large-scale national field test to assess the new NCVS instrument. The report describes the major changes made to the survey and assess the performance and burden of the old NCVS compared to the new NCVS, compares victimization rates derived from each instrument, and examines differences in data quality and respondent burden. The sample for their field test was drawn from the 48 contiguous states and the District of Columbia using a three-stage design that selected counties followed by census tracts followed by households. Data collection began in October 2019 and continued through March 2020.
One of the first major changes involved the Victimization Screener (NCVS-1). The old instrument presented long probes that were somewhat cumbersome to answer, and these long probes were broken up into multiple shorter probes within each screener series. Next, they updated content within the probes, such as adding “cell phone” to the list of items likely to be stolen during a burglary. They also expanded the probes asking about Rape and Sexual Assault (RSA). Lastly, while the basic types of crimes asked about remained mostly the same, Vandalism was added to the screener series.
Another major change involved computer-assisted interviewing capabilities. For people passing the screener series, computer-assisted interviewing was able to pull previously reported information to determine what types of follow-up questions for the second stage, i.e., the Crime Incident Report (CIR). To ensure that sufficient information would be carried from the initial screener to the CIR, the redesign added follow-up probes for each incident that was reported. This allowed someone to provide more details on each incident and in some cases helped people recall more information about what happened.
When comparing victimization estimates between instruments, the researchers found that the new instrument detected increased victimization estimates, especially for Simple Assault, Motor Vehicle Theft, and Other Theft. This difference is largely attributable to additional probes that were added into the instrument that helped people recall less memorable experiences.
When it comes to burden, the second version of the interview was longer due to added content, but this difference was only by a few minutes. However, obtaining information for the second stage (i.e., the CIR) was a little faster for people who were screened under the new protocol. This was likely because the initial screener responses could be routed through the CIR. One downside is that the new survey did have a small but significantly reduced response rate in comparison to the old survey (24.3% vs. 27.0%, respectively), which is somewhat indicative of greater burden.
Conclusion
The recent changes to the NCVS appear mostly beneficial. The re-classification of urbanicity is much more precise and accurate than previous measures. The re-design of the instrument (specifically, the additional probes scripted into the survey) also appeared to capture more victimizations than previous iterations, and seemed to help individuals with their recall of past events. Unfortunately, the new version of the survey did appear slightly more burdensome, with a reduced response rate of about 3%.
The post National Crime Victimization Survey re-design shows improvements over previous iterations appeared first on Crime & Consequences.