Evaluating RPA Tools? 32 Questions IT Leaders Should Ask

Let’s start off with the bad news. Nearly half (40%) of RPA implementations never advance beyond the pilot phase. When they do, 87% are thwarted by bot failure, 50% say that bots are harder to deploy than expected, and only 39% of planned bots get deployed on time.

The good news? It doesn’t have to be this way.

Many organizations have found much smoother paths to RPA success. For them, RPA is already delivering impressive results that increase customer satisfaction while improving the organization’s bottom line. Notable achievements include improved compliance (92%), improved quality / accuracy (90%), improved productivity (86%), and reduced costs (59%).

By and large, the most successful organizations tend to be those that took the driver’s seat in the RPA evaluation process. They weren’t lured in by vendor visions of an idealistic future where cute robots automagically perform all the work that everyone hates. They first thought long and hard about what types of automation would really benefit their business and what they required from an RPA perspective—in terms of people, process, and technology. Then they selected a solution based on how well it met those needs.

As you begin your RPA tool evaluation process, you should ask each RPA tool vendor a broad array of questions to help you clearly understand their approach to RPA—and how that will impact your organization’s productivity, costs, and success if you select them.

With diligent probing, you’ll be able to zero in on red flags related to:

• Enterprise scalability
• Process fit
• Support
• Total cost of ownership

Want some guidance on what to ask? Here are some core questions in five key categories.

Automation

• What is the primary method used to achieve automation (screen scraping, scripting, model-based automation…)?
• To what degree is scripting required to achieve the desired automation?
• How often is scripting used in long-term production bots (beyond demos and pilots)?
• How much time does it take, on average, to create a production bot?
• How many people—and which people—are typically involved in creating, testing, and deploying a production bot?

Scalability

• What methods does the solution use to accommodate change?
• In what situations will your bots need to be updated?
• What variations do the bots adapt to (e.g., UI, data, and flow variations)?
• What skillset is required to fix broken bots?
• Can business process owners fix bot failure on their own?
• When is outside assistance required to update bots?
• What metrics can you share regarding mean time to resolution after bots break?
• Are there capabilities that help teams manage change and evolve the assets over time?

Cost

• Is there a “per-studio” cost that penalizes organizations for enabling more people to create bots?
• Is there a “per-orchestrator” cost that penalizes organizations for involving more people in bot deployment and management?
• What is your pricing structure and where is it published?
• How do I know that I’m not being charged more than other organizations?
• How do you factor bot maintenance costs into the TCO?

Enterprise Support

• How do you partner with enterprise app vendors such as SAP and ServiceNow to provide integration that suits the finer nuances of each platform (beyond surface-level screen scraping)?
• What’s required to deploy the solution?
• What amount of professional services is required?
• How is your support organization structured and managed?
• What types of customer success programs do you offer?
• How are customer success programs structured?
• What frameworks and benchmarks do you use to establish KPIs and help clients track/benchmark progress?
• What is the average tenure of your support and customer success team members?

Artificial Intelligence

• How do you leverage AI to enable core RPA functionality (leveraging a combination of UI and surface-level features to automate routine, predictable data transcription work)?
• How is AI used to optimize non-RPA processes (e.g. OCR)?
• What part of your native RPA stack actually leverages AI?
• What is the advantage of AI in those scenarios?
• Is there an additional cost for AI components?
• If so, what is the alternative and what value add does the AI option offer?

Asking each RPA tool vendor these questions may seem extreme at first glance. However, performing due diligence now will have a huge impact on productivity, costs, and success over the long term.

[See the RPA Resource Center for analyst reports, white papers, videos, and other resources]