Do you or anyone in your family eat oatmeal for breakfast? If so, have you ever thought about how efficient your oatmeal prep is? It seems pretty straightforward: just heat up milk or boil water, dump those flakes in a bowl, add fruit, berries, honey, or other yummies you like. Now, thinking about each step, is there any way you can make it faster? Say, how many extra moves are you making because the ingredients are scattered around the kitchen?
If you dig into it, you can try to optimize your morning routine, but those few seconds that you save probably won't be of earth-shattering importance to you. Now, scale it to the enterprise level and just imagine how many processes comprise the daily workflow -- and how many optimization opportunities there are IF you could break down and analyze every step of every process.
Doing it manually is like boiling the ocean, and that's where process mining techniques come into play. In this article, we'll explain what process mining is, how it can benefit businesses across different industries, and how you can implement it to optimize the workflows of your company.
What is process mining?
Process mining is a set of techniques for the analysis of operational processes based on event logs extracted from company's databases, information systems, or business management software such as enterprise resource planning (ERP), customer relationship management (CRM), electronic health records (EHR), etc. In simple words, it’s about finding out how the processes are actually performed to discover problems and areas for improvement.
Process mining сan also be described as a part of business process management (BPM) that applies data science (with its data mining and machine learning techniques) to dig into the records of the company’s software, get the understanding of its processes performance, and support optimization activities.
There are different perspectives that process mining can be focused on.
- The control-flow perspective is about the order of activities, and the goal here is to find the optimal path of performing a process.
- The organizational perspective targets the resources involved in a process, i.e., roles, departments, etc. The general purpose is to come up with an optimal structure for the organizational units.
- The time perspective is focused on the timing and frequency of events.
- The case perspective considers the properties that are inherent in different cases or types of processes. Further analysis defines relations and hidden dependencies between these properties, giving a deeper understanding of the problem and its causes.
Process mining can be beneficial for businesses of any size and in any industry as it can be applied to any workflow, with the purpose of process analysis and optimization. Some other reasons to implement process mining are conformance validation, process harmonization, and process automation support. We’ll describe some real-world examples in the following sections.
Main stages of process mining
As people (and software) interact with business IT systems, their actions are captured by these systems and can then be transformed into event logs and visualized with the help of process mining. That’s how it happens.
Stages of process mining
- The activity or interaction with the system takes place, creating a digital record. Some examples of such activities are receiving an order, submitting a piece of documentation, approving a loan, entering information into a health record, etc.
- Process mining software transforms the digital records into event logs. The most common format for these event logs is an XML-based format XES (eXtensible Event Stream) which was adopted by IEEE Task Force on Process Mining. Event logs have at least three main attributes: case ID, activity, and timestamp.
A sample event log of order handling, source: processmining.org
3. The visualization of a process is automatically created using event logs. It’s important to understand that unlike traditional BPM techniques, process mining shows the real process as it’s actually done, not the ideal model as it was meant to be.
4. Process analytics takes place. Here, KPIs can be created and monitored to uncover potential improvement areas; data mining and/or ML algorithms can be used to detect hidden patterns and dependencies; or conformance checking techniques can be applied to compare the process to a certain ideal model.
Process visualization and analytics, source: Celonis
Process mining techniques
According to the Process Mining Manifesto, there are three main groups of process mining techniques.
(Automated) Process discovery is a primary technique and implies extracting and visualizing process models from an event log like we described above.
Conformance checking compares the actual process with a predefined model to discover deviations. So, it’s used to check if the reality conforms to an existing pattern.
Conformance analysis, source: QPR
Enhancement goes beyond measuring and comparing. It’s focused on extending the model with additional information, such as location data, costs, timing, etc. Enhancing the model with such attributes helps improve its performance and conduct more advanced analysis.
Applying these techniques would allow you to map your processes and discover gaps and inefficiencies in your workflow. Let’s look at some real-world examples of how process mining is used to benefit companies in different industries.
Process mining applications
Italian management consulting company HSPI publishes a database of process mining projects and case studies annually. The purpose of this project is to create the most complete list of process mining adoptions, increase awareness, and explore business potential. In the 2020 application database, there are 551 case studies from 27 countries around the world, proving the spread of process mining adoption and growth of interest in these techniques.
Here, we’ll go over several application cases to demonstrate the value and effectiveness of process mining.
Process mining and RPA
Robotic process automation or RPA is focused on automating repetitive business processes to increase efficiency. QPR, one of the largest providers of process mining software, claims that process mining can reduce RPA implementation time by 50 percent and RPA project risk by 60 percent. And according to UiPath, one of the leading RPA companies, “78 percent of people who automate say process mining is key to enabling their RPA efforts.” Here’s why.
- Process mining discovers areas that need improvement and can benefit from automation.
- Processes can be optimized to ensure that you’re not automating a mess.
- Process maps can be used as a guide or template to train bots, outlining the sequence of necessary steps.
- Process mining helps assess results and monitor RPA KPIs.
- With the help of advanced tools, you can simulate the workflow to predict how RPA implementation impacts performance.
Process mining and RPA project lifecycle
Vodafone, a multinational telecom giant, partnered with Celonis, the world leader in process mining, for an RPA project. One of the aspects that required automation was purchase order (PO) processing as Vodafone wanted to improve its rate of 73 percent correctly completed POs. Before actually launching RPA, process mining was implemented to understand process variations and noncompliance and prepare for automation.
Alexander Rinke, co-founder and co-CEO of Celonis, emphasizes the importance of process analysis and optimization BEFORE starting an RPA project: “If a process is already flawed, RPA will only make a bad process faster. It’s essential that businesses take a transparent approach to RPA, ensuring that automated processes are first optimized to be the most efficient.”
As a result of process mining and RPA implementation, in the first six months Vodafone achieved a PO rate of 85 percent followed by a reduction in operational purchasing costs by 11 percent and a reduced time to market by 20 percent. Eventually, Vodafone managed to further increase its PO rate up to 92 percent.
Process mining and supply chain
Process mining offers a lot of optimization opportunities to the complex, multifaceted supply chain industry, including such aspects as manufacturing, warehousing, transportation, inventory management, retail management, etc. Some of the typical processes that are often performed inefficiently and require improvement include
- procure to pay,
- order to cash,
- production process,
- warehouse operations, and
- accounts payable/accounts receivable management.
Twenty-two percent of respondents to a PwC study believe that procurement is the area that would benefit most from process mining. For example, an ideal procure to pay process is supposed to involve a short sequence of specific steps.
- Purchase request or requisition is created.
- Request is approved.
- Purchase order (PO) is created and sent to the supplier.
- Goods are delivered and approved.
- The vendor’s invoice is received.
- Payment is processed.
However, in reality, there are a number of gaps, unnecessary activities, and other weak points that add complexity and increase time of completion, often due to manual operations and poor communication. And you find out that your actual process now looks like this.
How a procure to pay process may look like, source: Processand
Process mining can help uncover those bottlenecks by offering full transparency into how the processes are performed in your organization.
In 2016, Veco Precision, the world-leading manufacturer of precision parts, won the Process Miner of the Year award after successfully applying process mining techniques to their manufacturing workflow.
Veco partnered with Fluxicon, another major process mining vendor, to reduce their manufacturing lead times. Fluxicon combined process mining analytical capabilities with the traditional process management approaches. As a result, they managed to identify process variations and missing links, reduce factory lead times, and increase efficiency as the same amount of work could be performed with fewer workers.
That was the first stage of their process mining journey. Inspired by the results, Veco kept applying process mining tools to analyze its performance. They also hired a data scientist to further discover opportunities for process improvement and trained more people in big data.
In 2017, Veco reported on another success. As part of their development strategy, they wanted to produce new samples and deliver them to customers within 15 days -- a process that then took 52 days. So, after analyzing data from their internal systems, they found the reasons for delays and had to define a new process “Engineer to Order” that required a completely different management approach.
As Mick Langeberg, the supply chain manager in Veco, announced, “The pilot results were jaw-dropping and gave us the green light to implement the new design. After we mined the customer journey, Veco has been consistently growing new business at an accelerated speed.”
Process mining and finance
For financial institutions, such as banks, insurance companies, or loan associations, the biggest operational priorities are security, accuracy, and speed of transactions. However, just like in any other industry, a lot of organizations suffer from inefficiencies, bottlenecks, and poor process management.
In 2018, Piraeus Bank, a Greek multinational financial services company, encountered problems after adopting RPA to automate the consumer loan process. Since traditional analytical methods proved to be of little value, they implemented QPR Process Analyzer to gain a deeper understanding of their process problems and identify bottlenecks.
The process analysis quickly revealed that “the main pain point was a lack of standardization of the consumer loan process and multiple inefficient variants of the process”. After uncovering the problem sources, an optimization plan was developed and implemented, resulting in cutting the loan application process from 35 minutes to 5 minutes and cutting lead time of the entire loan process by 86 percent.
How to approach process mining implementation: related challenges and possible solutions
If you are reading this article, your company is probably experiencing some process problems that you’re trying to solve. Here, we’ll define the key points to think about if you consider implementing process mining in your organization.
Things to consider before starting a process mining project
Review your current IT landscape
First of all, it’s important to understand that organizations must have their processes digitized, at least partially, so that they can apply software to extract event logs from their systems. You must have an ERP, CRM, or any other business management software in use that records your activities. Most process mining tools can import data in CSV, XLS, MXML, and other formats.
A lot of organizations still rely heavily on paper documentation or legacy disconnected systems that are unable to provide reliable records for event log extraction. Here, the gradual, enterprise-scale digital transformation and system modernization is necessary to obtain expected results from process mining.
Another challenge concerns the lack of integration for cross-organizational process mining. Even though traditionally process mining is executed within one company, there are cases when the understanding of the end-to-end process requires extracting information from multiple separate systems.
For example, a supply chain can involve the cooperation of several companies (manufacturing, transportation, retail, etc.), and to analyze the entire order fulfillment process, event logs of different systems have to be merged. Establishing secure data exchange between systems would facilitate information collection and analysis.
Ensure data quality
The results of your process mining project directly depend on the quality of extracted data. If it is unorganized, stored in many different systems, and incomplete, it would be difficult to process it effectively.
According to IBM, preparing data for process mining is the most effort- and time-consuming part, taking 40 to 70 percent of the project time. Check our articles on data quality and data preparation for some insights on managing and preparing your data.
Build the team
Another question is: Who will run the project? As obvious as it may sound, it’s often unclear what department should be responsible. Besides, since such projects involve operating advanced software tools, it can turn out that companies lack the needed specialists and have to hire business analysts and data engineers.
The main deployment challenge cited by 52 percent of the companies is the lack of expertise or specialized tooling. The solution here can be involving consultants or technology partners with relevant experience. In other words, make sure you have the right people with sufficient knowledge, whether employed or contracted.
Estimate the budget
Now, let’s also mention the financial aspect. Clearly, any tech initiatives require significant investment. So, before you dive in, try to estimate the involved expenses related to
- process mining software and hardware,
- new employees payroll,
- consulting services for initial implementation,
- maintenance and support costs, and so on.
Try to take all the unobvious costs into account, such as training your employees to operate the new software or your current system modernization.
Choose the tools
Even though process mining is a relatively new discipline, there’s already a number of vendors that offer comprehensive software solutions. We’ve already mentioned several popular providers of such tools, i.e., Celonis, QPR, UiPath, and Fluxicon. There are more off-the-shelf platforms on the market, including several open source ones such as ProM Tools or the Community edition of Apromore.
Whatever you choose, remember that you’ll have to establish a seamless integration between your software and the process mining solution to enable flawless data extraction and ensure accurate analysis.
Consider predictive analytics
Originally, process mining was used to analyze historical data. However, with the increasing volume of real-time data and the necessity for continuous operational support, process mining techniques are also evolving to be able to create real-time predictive models and provide instant recommendations.
The biggest challenge here is the increased complexity of data architecture. It requires substantial expertise in data science to operate advanced technological solutions, efficiently manage the company’s data, and support decision-making with accurate predictions.