Scalable, Cloud-Native SaaS Platform for Multi-Source Data Curation

Introduction: 

This initiative focuses on building a Cloud-Native SaaS Platform for Multi-Source Data Curation, designed to simplify how financial institutions and end-users’ access and analyze dynamic data from multiple sources.

A Microservices banking architecture is used in the backend to bring this concept to life, including a mix of technologies such as Django, Laravel, and NestJs. Authentication is handled by Keycloak and Strapi, while the frontend is driven by SvelteJs and the database is Postgres. This architecture ensures the Home Loan Package Curation process is efficient, secure, and scalable.

Client Profile: 

The client operates a user-centric platform that curates and continuously updates a wide range of financial data from multiple sources, ensuring real-time market competitiveness and transparency.

Their focus on data confidentiality and accessibility enables users to make informed, secure, and confident financial decisions. The platform’s goal is to simplify complex data aggregation through an intelligent Cloud-Native SaaS framework that empowers users and institutions alike.

Challenges:  

The project presented several challenges, including: 

• Complex Data Aggregation: Curating and presenting a comprehensive list of home loan packages and interest rates from multiple banks in a user-friendly format required advanced data aggregation techniques. 

• Integration of Diverse Technologies: Coordinating and ensuring seamless communication among the diverse technologies employed in the backend posed a challenge, as it required expertise in Django, Laravel, and NestJs. 

• User Authentication and Security: Implementing secure and user-friendly sign-in authentication via Keycloak and Strapi, while safeguarding user data, was a critical concern. 

• Performance and Data Storage: Ensuring optimal performance and reliable data storage, especially when dealing with financial data, was of paramount importance. 

Technical Solution: 

To address these challenges, the following technical solutions were implemented: 

• Microservices Architecture: A modular Microservices design enabled independent development, deployment, and scaling of components, which was crucial for maintaining a resilient Cloud-Native SaaS Platform for Multi-Source Data Curation.

• Data Curation Algorithm: A proprietary data curation algorithm was developed to collect, organize, and present home loan package information in an easily digestible manner for users. 

• Technology Stack: Django, Laravel, and NestJs were employed as backend technologies to facilitate seamless data retrieval and processing. Keycloak and Strapi were integrated for secure authentication and user management. 

• Frontend Excellence: The frontend was built with SvelteJs, offering a responsive and user-friendly interface for users to browse and select home loan packages effortlessly. 

• Database Efficiency: Postgres served as the database of choice, ensuring efficient data storage and retrieval, vital for maintaining data integrity. 

• Data Storage and Delivery: Data was stored in S3 and delivered via CloudFront, providing reliable access to the curated information within the home loan comparison system.

Technical Stack: 

– Technologies: SvelteJS, Django, NestJs, Laravel, Postgres
– Infrastructure: AWS S3 for static hosting of web app and admin, Cloudfront for distribution, ECS for running multiple tasks, ECR for docker image, AWS ALB for load balancer and routing, AWS RDS Postgres for database 

Architecture diagram:

Results/Business benefits: 

The implementation of this solution yielded several notable results and business benefits: 

• Streamlined User Experience: Users can now easily navigate and compare home loan packages and interest rates, streamlining the decision-making process. 

• Data Security: Robust authentication via Keycloak and Strapi ensures user data remains secure, bolstering user trust and data protection compliance. 

• Efficient Data Management: The use of Postgres for data storage guarantees efficient data management, supporting the system’s performance and reliability. 

• Scalability: The microservices architecture enhances scalability, allowing for the future integration of additional features and financial institutions.

• User Engagement: The user-friendly frontend powered by SvelteJs has led to increased user engagement and satisfaction. 

Conclusion: 

This case study exemplifies how the integration of innovative technologies and microservices architecture can revolutionize the process of building a Cloud-Native SaaS Platform for Multi-Source Data Curation. The proprietary curation system has simplified the user experience while ensuring data security, efficient data management, and scalability for future growth.

The success of this project underscores the potential of advanced technology solutions in the financial services sector, providing users with the tools they need to make informed financial decisions through a next-generation Cloud-Native SaaS Platform for Multi-Source Data Curation.