Process Mining: All You Need To Know Now

Leave a Comment /

This is a Bot Nirvana Community report. Thanks to contributions from Amol Rajamane, David Francis, Doug Shannon, Ricardo Henriques, and Dr. Adam Bujak

What Is Process Mining And Discovery?

Problem: Understanding business processes and their variations are usually incomplete with traditional techniques. Manual Observation, Interviews, and recording are time-consuming, labor-intensive, and subjective. 

Solution: Use the power of data science to discover, monitor, and improve business processes. With data and AI, you get an objective view of your processes & variations.

Why it matters – current benefits

  • Gives you a good understanding of the End to End (E2E) Process View
  • Improve the identification of areas to be automated
  • Substantial impact on cost and time when used for digital transformation initiatives
  • A litmus test to validate the expected gains of a new approach
  • Improve the overall visibility and transparency of your workflows
  • Improve identification of root causes and KPI impact in process variations, outcomes, and non-compliance
  • Use the insights gained to reduce average cycle times
  • Reduced effort to identify process steps and their variations (compared to manual)
  • Improve upskilling or retraining efforts with precision training for individuals or teams
  • Improved business agility
  • Reliably predict the completion time of end to end runs

source: Everest group

Process Mining is a foundation to Digital Innovations program, this could help you drive many of your programs like IOT, RPA, AI/ML, etc.

— Amol Rajamane, Global Digital Automation Technology Leader, DuPont

Use Cases

Process Mining – A few use cases:

  • Order Management
    • Inventory management
    • On-time delivery
    • Cost per order
    • Net promoter score
    • Price Change or revenue leakage
    • Shipped not billed
    • Delayed billing
    • Delivery Blocks
    • Credit Blocks
    • Sales Order Rejections
    • Complaints
  • Accounts Receivable
    • Days sales outstanding
    • Time to invoice
    • Perfect invoice rate
    • Processing time of purchase requisitions
    • Inbound on-time delivery rate
    • Spend under management
  • Accounts Payable
    • Days payable outstanding
    • Paid-on-time rate
    • Touchless invoice rate
  • Procurement
    • Inbound on-time delivery rate
    • Processing time of purchase requisitions
    • Spend under management
  • IT Services
    • Service now ticketing
    • System migration
  • Manufacturing
    • Measure and optimize machinery performance 
    • Reduce Operational Cost
    • Analyze and improve production-related processes
  • Logistics
    • Source to Contract
    • Transport Order to Delivery
    • Invoice to Pay
  • Marketing and Sales
    • Sales opportunity management
    • Customer service

How It Works

Here is a typical flow for an enterprise-grade mining/discovery solution:

  1. Data Ingestion: This is the first and most important step in the overall discovery. It integrates with different data sources like Event data, Databases, and Screen-based data. This step is highly complex and sometimes includes real-time data.
  2. Process Intelligence: This is the step where you see your business process, and inefficiencies, validate data, and recommend solutions. It involves:
    • Understanding, documenting, and categorizing of work (Diagnose the current State)
    • Un-cover variation across teams via different means including application-level screen capture
    • Visualize E2E flows, using existing data to analyze process performance
  3. Solution/Improvements/Automations – Automate inefficiencies from your business process, this could be simple training to employees, awareness, standardizing or building integrations or RPA solutions to bridge the gap
  4. Monitor and realize value: Continuous monitoring of your business process after improvements and keep control over variations, repeat steps 2 to 4

Process Discovery is the cumulative effort of Process Mining and Task Mining to understand the end-to-end common path for an individual process. 

“For leaders to not make blindfolded decisions it is important that business processes are transparent, while working toward transparency in operations also identify inefficiencies and waste, as most organizations are on the Automation bandwagon, this transparency becomes more so important”

— David Francis, Smart Automation Architect, Schneider Electric

Process Mining: Challenges & Solutions

Current Challenges

  • Data Ingestion is the biggest challenge when you have large and legacy ERP systems
  • Monitoring and analyzing the work of thousands of people simultaneously and continuously 
  • Dealing with a large volume of events for mining/discovery analysis 
  • With too many stakeholders to handle, you need to involve not only business teams who understand their process but you also need experts from ERP systems who can do data validations, data modeling, IT security experts
  • Need buy-in from the leadership team, a top-down approach is key
  • Task mining is still not mature enough from a PII perspective, it takes time for large organizations especially when you have a presence in Europe due to different regulations.
  • Democratizing process analytics 
  • Monitoring the E2E execution and feedback loops, especially in new initiatives.

Possible Solutions

  • Start small, in an agile mode, with one business workflow, and a system at a time
  • Keep it simple, do not focus on building new interfaces or integrations
  • Governance, security, value validations, automation prioritization, and operating model are key to success
  • Have all important stakeholders on board right from the beginning
  • Have a specific objective like reducing Days sales outstanding (DSO) by certain days, then drill down into specific KPIs and identify inefficiencies.
  • Let IT drive the initiative along with the businesses
  • Have a good training plan for business users
  • Do not forget Change Management
  • Do not focus too much on real-time connectivity. If real-time connectivity does not work fast enough, go with manual data ingestion using flat files.
  • You also don’t need all historical data, you can always choose one quarter at a time.
  • Chose the right partner for your implementation
  • Build your own team with the right people
  • Use Automation to create some “lift”
  • Overall, focus on the speed to results that matter

“Process Mining gives you an x-ray of your business process in the fastest possible way.”

—  Doug Shannon, Intelligent Automation Leader, PSI CRO AG

How To Choose The Right Tool(s)

A few measures to consider while evaluating tools with Mining/Discovery technologies:

  1. Easy to aggregate logs into processes
  2. Desktop process discovery capability
  3. Integration between business and desktop data
  4. End-to-end visibility and transparency of workflows
  5. Embedded AI / Intelligence, ML-based root cause analysis
  6. Range of prebuilt/templated process analysis
  7. Easy Conformance/Compliance checking (with a UI)
  8. Recommendations for process improvement and re-engineering
  9. Can identify activities/tasks to be automated
  10. Integrated automation capabilities
  11. Analytics reporting and insights
  12. Easy to use for business users – No/low-code development
  13. The effort to identify workflow steps and variations
  14. Ability to identify root causes and KPI impact of process variations and outcomes
  15. Transparent Pricing models
  16. Training availability, Upskilling, or retraining efforts
  17. Resource availability for the skill(s) 
  18. Legal and security requirements
  19. Ability to handle “Big data computing” (dealing with millions of events)
  20. Social analysis
  21. What important capabilities do we want to build within the organization?
  22. What skills and resources do we have to support the initiative?
  23. What do we have to settle for? Ex: Software requirement, number of processes and users in the pilot phase, legal and security requirements, etc.
  24. Business agility
  25. Overall business impact

Process Mining Companies

Here is a round-up of some of the most popular tools (Updated Mar 2024).

Standalone platforms

Automation Platforms

Big Tech

Source: Everest group

Technologies And Features Used

Some of the technologies and features in the current mining/discovery tools:

  • Process mining that uses data from Event logs (and databases at times)
  • Task mining records and analyzes the interactions performed on the computer screen
  • Desktop data collection to collect streams of desktop work that include application data, environmental variables, and user interactions
  • AI-ML-based analysis of task data and/or event data
  • Connectors to extract, transform and load transactional data from systems for analysis and integration to third-party platforms
  • Conformance checking to understand how work is performed against organizational policies
  • Root cause analysis to find factors that are contributing to certain behaviors and outcomes
  • Data simulation to simulate scenarios of process transformation and understand potential impacts before making changes

Trends And Predictions

A few trends and predictions for Process Mining:

  • Use Digital Twins of the process instead of or in addition to the stand-alone tools
  • Continuous management of operational processes and the automated execution of actions to improve the process
  • Process discovery & mining deployments will become cloud-based
  • We will see more Process-Discovery-as-a-Service offerings
  • Proactive intervention using ML to trigger an automation (RPA or BPM)
  • Real-time process guidance on the best next steps for business users
  • Additional technologies like IoT will help provide more data and insights
  • We will see more advanced ML approaches being added for process change recommendations
  • More advanced analytics for predictive and prescriptive analytics
  • Healthcare, transportation, and logistics sector is expected to drive growth
  • Automated what-if scenario simulation
  • Automated root cause analysis

Process Mining: Market Size

  • The current global Process Discovery and Mining (PDM) market size is estimated by NelsonHall at ~$670m and will grow to ~$4.3bn by 2025, a growth of 45% CAAGR. (Nelson Hall, Dec 2021)
  • The process mining market will grow to $2.3 billion by 2025, with a double-digit CAGR of 33%. The key drivers continue to be accelerated digital transformation efforts, growing process visibility requirements due to remote working, and increasing demands for operational resilience. (Gartner, Oct 2021)
  • The global process mining software market size is expected to reach USD 11 Billion By 2030 from USD 373.9 Million in 2021, at a compound annual growth rate (CAGR) of 49.0% during the forecast period 2022 to 2030. (Polaris, Mar 2022)
  • The global process mining software market size was valued at USD 322.02 million in 2020 and is expected to expand at a compound annual growth rate (CAGR) of 50.1% from 2021 to 2028 to reach USD 7.95 billion. (Grand view research)

What’s The Bottomline?

In today’s world, the speed to analyze, automate and realize value is a key to digital success.

Process Mining can be one of the key foundations for your Digital Innovations program. It could help drive many of your programs like Automation (RPA, BPM), IoT, AI/ML, and more.

Using process and task discovery, organizations can obtain a more precise view of work that has strategic value versus work that is more transactional in nature. Such discovery reveals the complexity of processes and thus has a bearing on decisions about outsourcing, automating, or reconfiguring processes or any component activities.

By combining data science and analytics, process mining/discovery gives you a framework to find insights into business workflows. It provides a disciplined effort of showcasing transparency, consistency, and objectivity within the process and enables leaders to take informed decisions, both tactical and strategic. This is where Data Science meets Process Management.

Leave a Comment

Your email address will not be published. Required fields are marked *