Generative Artificial Intelligence (GenAI) has rapidly become a transformative force in the technology sector, prompting major cloud service providers to make substantial investments in their infrastructure and GenAI capabilities. Companies like Amazon, Microsoft, Google, and Alibaba are allocating billions of dollars to enhance their GenAI offerings, aiming to meet the growing demand for AI-driven solutions. These investments include expanding data center capacities, developing specialized AI hardware, and acquiring pioneering AI firms. This strategic focus underscores the critical role of GenAI in shaping the future of cloud computing and the broader technological landscape.
Process mining has emerged as a pivotal tool for organizations across various industries seeking to enhance operational efficiency by identifying and addressing process delays, anomalies, and duplications. By analyzing event logs from information systems, businesses can gain accurate insights into their actual workflows, uncovering inefficiencies and opportunities for improvement. This data-driven approach enables the strategic implementation of automation, streamlining processes and boosting productivity. For example, in manufacturing, process mining can optimize production lines by pinpointing bottlenecks and facilitating better scheduling. In the financial sector, it accelerates loan processing and enhances fraud detection. By leveraging process mining, organizations can transform their operations, achieving greater transparency and effectiveness.
Infusing GenAI in process mining offers a transformative approach for various stakeholders—including end clients, consultants, architects, developers, and technology enthusiasts—to optimize business processes and harness the full potential of both technologies. Let us see how.
Problem Statement
The other day, during a client workshop, my team was demonstrating how Microsoft's Process Mining capabilities can help streamlining process redundancies, fallacies, anomalies, loops, variants, delays, etc., thereby significantly enhancing the efficiency of the process lifecycle. Process mining software heavily depends on the event logs documented in a structured manner. CSV is the universally accepted format for almost every Process Mining tool, technology agnostic. However, in most occasions, these event log data are generally quite large in volume.
One of the clients asked me a genuinely relevant question which was of immense significance, that I did not have a ready answer to. He said "Almost every process mining tool has similar capabilities that can help streamlining processes. The biggest challenge lies elsewhere. How do we accumulate the data and give them a structure in CSV format? This usually becomes a monumental task." I fumbled for a bit and responded with a submissive grin "This is something I will have to circle back to you. Upfront I can think of GenAI being a potential way to address. But I will do my research and come back."
Since then I had this thought swirling in my head, and finally I got an answer that this article makes an attempt to discuss.
Solution
The solution to the above problem statement is twofold, as illustrated below:
- Prepare CSV from structured or unstructured event log data.
- Prepare Process Mining Data Model (is discussed in the next article in this series).
Fig. 1 Architecture Diagram
The idea is to generate a CSV output from captured event logs, that could be in any format, stored in any storage. The below section explains the diagram in Fig. 1 (above):
- Event Loggers: Event logs can be generated by a wide range of applications like RPA, apps, manually by data entry operators, etc.
- Event Log Storage: Event logs can be captured in any format (both structured or unstructured, i.e. tabular or non-tabular) in any on-prem or cloud data storage.
- Application Service Layer: This are main components of the solution.
- Copilot Studio Agent: An agent initiates an interactive conversation with the Process Mining Engineer. In course of the chat thread the agent captures key information like "Process Name", storage information (like SharePoint list, Dataverse table, etc.) where the event logs are stored. The agent also finally provides hyperlink to the output CSV for the user to download and verify the data generated by the agent.
- Power Automate (The orchestrator): Flows in Power Automate constitutes the backbone of this solution, that
- Receives user input (see "Copilot Studio Agent" step above).
- Reads event log data stored in the data storage.
- Calls an Azure Function when it is the first time event log data processing in bulk (See "Azure Function" step below). Event data are passed as JSON.
- Appends newly created events in the output CSV file (See "Azure Function" step below).
- Azure Function: The Azure Function has two roles to play, as follows:
- CSV Author: An HTTP Triggered Flex Consumption Azure Function utilizes Azure OpenAI LLMs (as NLP) to give structures to the unstructured or semi-structured event log data read from their storage to CSV format. Finally the output CSV is stored in some storage container (can be any container like Azure BLOB Storage, SharePoint, Dataverse, etc.)
- Data Modeler (discussed in my next article)
- Security Layer and Responsible AI: The solution is conceived to be using Microsoft's cloud service stack. This makes all services and data secured by Microsoft Entra ID. Also Azure OpenAI and Copilot Studio complies to responsible AI principle.
Conclusion
The integration of Generative AI with process mining represents a significant leap forward in operational efficiency and data-driven decision-making. By automating the transformation of diverse event logs into structured formats, organizations can unlock deeper insights and streamline their workflows. This synergy not only addresses longstanding challenges in data preparation but also paves the way for more agile and responsive business processes.
Stay tuned for future updates where we'll delve deeper into real-world applications, share success stories, and explore advanced techniques in this evolving landscape. If you're passionate about the convergence of AI and process optimization, consider following me on LinkedIn. Your feedback and insights are invaluable—feel free to share your thoughts in the comments below.
What next?
Integration of GenAI and Process Mining is a wide area of discussion, and should take more than just a single article. This is the very first article of a series that will keep getting updated. You can visit my next article in this series here.
