Design Reliable, High Performing Fault-tolerant Systems with Anypoint MQ

Introduction

Designing high-performance, reliable, and fault-tolerant systems with Anypoint MQ is essential for any organisation that wants to send and receive messages between different applications securely. Anypoint MQ is a robust messaging service from MuleSoft that makes it easier than ever to implement this type of system.

System design becomes complicated in a diverse system with variable traffic and minimal control of incoming requests. As part of the system’s core design, a system should be fault tolerant with zero data loss and offer high performance. The system also should be reliable, ensuring reprocessing of low-quality data in a scalable manner.

In this blog, we will explore a case study of implementing Anypoint MQ for KYC (Know Your Customer) compliance that the government mandates for the financial services industry.

Case Study – Know Your Customer

The government mandates KYC compliance for the financial services industry. Key areas where Anypoint MQ would benefit the KYC are:

(i) Customer Identification & Acceptance Policy

(ii) Monitoring of Transactions and On-going Due Diligence

(iii) Risk Management

Financial institutes, which have a huge customer base in both commercial and retail sectors, are flooded with a high non-deterministic number of KYC requests that are initiated by various third parties, customers, and partners. Managing such an uncontrollable and non-measurable frequency of requests is a huge challenge for the financial services industry.

Let’s first understand the key capabilities required by such systems and how Anypoint MQ features fulfil these capabilities.

Operational Capabilities

Key Capabilities	Description	Anypoint MQ Feature
Zero loss of data	A system should be flexible enough to accept all the requests without loss of data and should also provide a resilient and reliable way to serve these requests	Cloud messaging service that performs asynchronous messaging and ensures data is not lost once received
Multicast the requests for parallel processing	The message received by the system should be multicasted and can be replayed into different systems for parallel processing based on the target system’s need	Message exchange binding with one or more message queues
Handle failure and reprocess data	If a message fails repetitively for a similar reason, it should be possible to refine it by cleansing the data from the knowledge base, correcting issues, and then re-messaging.	DLQ Messages can be reprocessed and injected into the exchange after reprocessing and re-messaging
Automation of MQ management	As the number of target systems expands, the MQ should be created or deleted in an automated manner instead of manually creating and managing it.	AMQ Admin API. AMQ provides APIs to create/delete/edit MQ

Functional Capabilities

Key Capabilities	Description	Anypoint MQ Feature
Support Non-deterministic requests without losing any requests	The system should accept the non-deterministic load and process it asynchronously such that all the requests are processed without loss	Asynchronous messaging with the option of FIFO or Standard Queue
Intelligent routing based on Request	Incoming message requests should route to a specific queue which is associated with the egress	Anypoint Exchange Message Routing Rule
Dispatch message to the functional queue	The message should be sent to one or more queues based on the message type	Binding exchange with one or more queue
Undelivered Request Retentions	The invalid request should not be lost, and there should be a possibility to retain it and replay it later	Dead-letter queue (DLQ) for undeliverable messages (Refer to Figure 2 for more details)

Non-Functional Capabilities

Key Capabilities	Description	Anypoint MQ Feature
Allow recovery from temporary failure of the back-end system	If the request is failed due to any back-end failure, the request should not be played recursively and instead should allow to re-play again after a specific interval	Enabling Circuit breaker pattern
High throughput for certain load	Up to a certain load of incoming requests, the request processing should be faster. The back-end throughput determines the incoming request load.	Subscribe with Prefetch Message
Controlled and authorised access to a Message queue	Message queue should be secured and highly accessible, considering the message consists of PII data	Role-based and client management
High Availability & Disaster Recovery	Ideally, a zero downtime of the messaging system is essential, considering the incoming requests are non-deterministic, so the downtime of the MQ would result in the loss of data	Cloud-based managed queue
Secured message for financial data	PII data within the message should be encrypted in transit to ensure the high security of financial data	Encryption of messages

Based on the above features of Anypoint MQ, let’s look at how we can design the system with the use case of KYC compliance in a Bank.

KYC Compliance: Technical Design (Anypoint MQ Perspective)

Consider that third-party systems send KYC compliance requests to the system. These are HTTP/HTTPS requests with collated data of the personnels whose KYC compliance checks are required. The personnels’ KYC data to the front channel are received non-deterministic and irregular in nature. These are authorised requests with the Access token to ensure the authenticity and identity of the source system.
The front channel/experience layer accepts the requests and passes them to the KYC Compliance Receiver MQ. For simplicity purposes, Gateway and Load Balancer are omitted in the diagram.
KYC requests are validated, filtered, and tagged before being sent to the exchange for specific egress for further processing. Filtration and tagging are done by the application to enrich the message.
KYC compliance exchange has various rule sets defined which decide to which MQ the message should be sent. A typical ruleset for KYC Compliance can be: If ‘reKYC’ is ‘required’ ? Send to KYC Processing MQ. If ‘kycTrackingOnly’ is ‘true’ ? Send to KYC Tracking MQ. If ‘physicalKYC’ is ‘required’ ? KYC Logistic MQ.
Each MQ can have one or more egress associated (for simplicity, single egress is mentioned in the diagram), which consumes messages from the respective MQ.
The target system is the consumer of the specific egress MQ, which reads messages, transforms them, and sends them to the respective third-party system.

This kind of dispatcher logic enables intelligent processing of the messages to the respective queue.

This way, we build a highly reliable, available, high-performance system using Anypoint MQ. Above design covers three business use cases of KYC..

Re-KYC – Personals whose KYC is required again due to expiry or change in of previous KYC.
KYC Tracking – Personals whose KYC Requests were submitted earlier and now require tracking.
Physical KYC- A new KYC request is submitted and requires logistic processing.

We have yet to discuss how to achieve fault tolerance in the solution. Anypoint MQ provides specific features to support this. Let’s look at the ‘KYC Compliance Receiver MQ’ from the previous diagram and understand it in more detail.

A KYC Compliant Receiver Queue (MQ) is backed by a Dead Letter Queue (DLQ). Messages that can’t be processed after several unsuccessful delivery attempts are placed into DLQ.
There can be various reasons for the message’s failure, varying from business validation failure to an invalid message. A message cleansing system listens to DLQ and corrects the message based on the defined business rules. In some cases, manual intervention is required.
Once the message is corrected, the system retrieves the original message, enriches, reformats and generates a new message with existing metadata.
The newly generated message is now placed into the original MQ to play it again.

This way, the system becomes fault-tolerant and ready to process clean requests.

Final Thoughts

In this blog, we explored a case study showing how Anypoint MQ could be used for KYC (Know Your Customer) compliance that the government mandates for the financial services industry.

From this, we can see that Anypoint MQ is a multi-tenant, cloud messaging service that enables customers to perform advanced asynchronous messaging scenarios between applications in a high-performance, reliable and fault-tolerant manner.

If you would like to explore how your organisation could use Anypoint MQ or if you would like to discuss how Devoteam could help you to implement solutions for your business. Please do not hesitate to contact us.