Most criminal justice activity is conducted under the auspices of state and local governments. Law enforcement at the state level is mostly decentralized to the counties, cities and towns. The state police exercise authority over the major state highways and over unincorporated rural areas. They often have other limited functions, including maintenance of criminal records. State attorneys general, unlike the U.S. attorney general, usually have little or no prosecutorial authority, although they may be responsible for arguing criminal appeals and defending post-conviction petitions. Prosecution is a county-level function. Most prosecutors, called district attorneys (DAs), are elected.

Each county has a jail that holds defendants awaiting trial as well as defendants convicted of minor crimes called "misdemeanors" (crimes punishable by a maximum jail term of one year or less). Probation departments are usually organized at the county level as well. There are more than 20,000 independent police departments that belong to local governments. Most of these departments serve small towns and have fewer than 20 officers. In contrast, big city police departments are huge. For example, the New York City Police Department, the nation's largest, has approximately 38,000 officers. Defendants in state court who are convicted of felonies and sentenced to imprisonment, are incarcerated in the state-operated prison system, usually called the "department of corrections."