Palm Beach County speeds court filings with AI

Florida’s Palm Beach County is keeping five recent recruits in the dark to boost productivity: The Lights-Out Court Document Processing project has five “robots” working 24/7 to process some of the thousands of documents the county court receives each week. It’s an example of how artificial intelligence and humans can work alongside one another — and shows how enterprises can enhance manual systems with AI.

Attorneys and other frequent filers at Palm Beach County Court are required to submit documents electronically, using a dedicated portal that’s available 24/7. Altogether, they file about 40,000 court documents a week this way, each of which must be docketed — that is, associated with the correct case, tagged and indexed so that it can be found in the court’s records.

It’s a laborious process, one that Sharon R. Bock, clerk and comptroller for Palm Beach County, saw as ripe for automation.

Bock, who oversees around 700 employees and a $70 million budget, is always looking for more efficient ways to perform statutory and constitutional duties. With payroll accounting for 95 percent of her spend, using automation technology to deal with some out-of-hours- filings and help staff do more during regular office hours was an obvious target.

Narrowing the field

The project, which received a 2019 Digital Edge 50 Award for digital innovation, was undertaken in collaboration with Florida software developer Computing System Innovations (CSI), which supplies the software the county uses to redact court documents. CSI developed a machine-learning system to automate the docketing and to access relevant court filings, says Cindy Guerra, the county’s chief operating officer for courts and official records, and the project’s business sponsor. “We used the same interface, so it was actually very seamless,” she says.

Palm Beach County first identified the teams that would work on the project, and the types of filings the county would entrust to the system. They chose to work with “documents that were very frequently docketed but did not pose a risk if something went wrong,” Guerra says.

They also aimed for documents that would allow the highest quality OCR (optical character recognition). “For that reason, we excluded anything coming from a pro se filer, or self-represented litigant, because those tend to be handwritten, and the OCR quality is not going to be as good,” Guerra says.

Those criteria narrowed the load to seven docket codes, three of which were selected to begin the project: notice of hearing, notice of taking a deposition, and notice of cancellation. None of these documents require further action, such as the court asking the clerk to issue a writ or a summons, nor do they involve money.

Training the new recruits

To train CSI’s machine learning software, the county’s business analysts worked with the employee with the most knowledge of the county’s processes and documents to index 500 to 600 documents of each type, extracting and tagging the relevant information from them.

The county set up five instances of CSI’s automated docketing software on its servers, giving them “human” names, with a hint of robot in each: Arnold Connor, Hal Isaac, Rosie Tober, Walley Bishop and Kitt Robbie. Documents processed by these robots are electronically stamped with their names rather than ‘docketed by a robot.’ Guerra jokes: “We didn’t want to alarm our judges.”

Once the software was trained, it was time to audit it.

“We don’t audit 100 percent of our humans, but we did audit 100 percent of the machines,” says Guerra. “I was 100 percent skeptical, that’s why we did 100 percent auditing.”  

The documents were presented to auditors in the same way, whether they had been processed by a robot or a human. The audit found that the machine learning system was 98 or 99 percent accurate with its three document codes, much better than humans.

“We were the first instance in using this type of software, so we were very apprehensive, and we didn’t know we could trust it. Now we know,” says Guerra. “Machine learning is more accurate than humans in that, once you teach the machine that it made an error, it will never ever make that error again. When we teach our clerks that they have made an error, sometimes it takes multiple trainings for them to actually get it right.”

After the project went live, Guerra received a call from a supervisor who had spotted a mistake in a docket. “She called up saying, ‘I can’t find the clerk who made this error, I can’t find her in our telephone list, her name is Walley Bishop.’ Well, we had to chuckle!” Guerra says. “It’s so seamless that it just looks like a clerk did it.”

Since the system went live in March 2018, it has been trained to read and process 26 docket codes and now dockets more than one in six electronically filed documents, with the goal of docketing more than one in three by July 2019.

Right now, the five robots are doing the docketing work of eight employees, and there’s capacity available to do the work of 19, says Guerra. To reach that, the robots would have to be retrained to deal with less common or more risky documents.

The impact of shifting to AI

Palm Beach County employees needn’t fear for their jobs, according to Bock. With robots available to do the work that is most repetitive and error-prone for humans, “We have been able to retrain our staff into more ‘thinking’ jobs,” she says. “We have also now raised the level, the career level, the experience level and the technical level of our employees so that we’ve now reclassified them into higher wages.”

That move to higher-paid roles has had the slight downside of eroding some of the cost benefits of increasing capacity through adding robots rather than more staff, says Bock, but “AI in our experience is actually allowing our workforce to move from a minimum wage environment for a certain group of people into not only a living wage but into a technical career that’s transferrable, that is exciting, that is thought-provoking.”

She expects that virtuous circle to continue as staff seize the opportunity to learn something new and see that their greater understanding allows for further innovation and the creation of new applications.

Other would-be adopters of AI shouldn’t expect instant results, though. This is not an off-the-shelf, plug-and-play product, Bock cautions. The project took several years from inception to completion, with CSI and county staff working together a lot of the way. “[Guerra’s] deep distrust as the business sponsor is one of the reasons that we were successful,” she says. “Setting up the auditing team meant that we caught the mistakes and retaught, taught, retaught.”

Looking to the future, Guerra wants to start working with more risky documents, “Those that require further actions, so that the machine can learn to make those further actions whether it’s email someone, prepare a writ or prepare a summons,” she says. Another goal is to have the robots issue receipts for documents submitted with payment.

Meanwhile, Bock hopes that the system, currently dealing only with civil filings, can be extended into the criminal segment.

Visitors from the Federal Bankruptcy Court have come to look at the system, and Bock and Guerra both hope other courts will adopt lights-out document processing too. “As other counties in the state of Florida go online, we benefit from the work that they do,” Bock says.

Bock, a lawyer by training, concludes: “This is the most exciting thing I’ve ever done in the 27 years of my career.”