DevOps Solution for AI-Based Financial Recommendation System
Due to the complexity and tough security requirements of AI-based financial recommendation and analytic system SoluLab DevOps team made great efforts to setup processes automation and migrate the system between different cloud platforms.
About the Project
The client is a FinTech startup developing an AI-powered financial
analysis and recommendation system for the stock market and wealth
management domain. The product is used by tier 1 banks and asset
The system processes various data feeds such as CRM data, news,
documentation archives, client portfolios, and transactions and
stock market data and recommends news and documents relevant to the
client and his portfolio and helps prioritize the clients according
to the stock market events.
DevOps Solution Highlights
Due to the enterprise nature of the project, its complexity, 5+
years duration, security requirements, cloud independence during its
course we had to cope with a very wide range of DevOps tasks on all
major cloud platforms.
Time & Materials
Machine Learning, NLP, Enterprise
Effort and Duration
Tech stack / Platforms
The complexity of the system, the necessity to support and update
multiple versions for different clients required full automation of the
development and deployment processes with strict version control and
release management. During the development process our team successfully
completed the following DevOps tasks:
Setup Continuous Integration and Continuous Delivery (CI/CD) pipeline
for Python applications with Jenkins supplemented with automated Unit
and Service level tests
Setup automated code quality analysis check as part of the CI process
Moved all infrastructure to Docker containers
Setup Database cluster manager
Setup site-to-site VPN access for the distributed development team
Implement an automated monitoring system based on Sentry. The
development team received automated alerts of any issues (peak
memory/cpu usage, data processing delays, errors, service
unavailability etc) happening with the production instance.
Codified the Infrastructure with Terraform (Infrastructure as a code
As a result of the automation, the team could release new software
versions much faster, of much better quality, without errors caused by
human factors and could significantly reduce the time and money spent on
integration/deployment and testing activities.
Migrations Between Cloud Providers
Major cloud providers have startup support programs providing their
cloud infrastructure for free for the first 1 or 2 years. Our client
could obtain such credits from AWS, Google Cloud, IBM Cloud and
Microsoft Azure and he didn’t have to pay a dime for his development
infrastructure for 5 years. However this required the development team
to keep the system cloud-agnostic and once in 1-2 years migrate up to 30
servers and data (Dev, QA and Production environments) from AWS to
Google Cloud, from Google Cloud to IBM Cloud and from IBM Cloud to
We used Docker, Ansible, Cloud Formation to perform this migration.
On-Premises Installation for Enterprise Clients
Enterprise clients required that the system is installed in their own
cloud and installation and support is handed over to their own DevOps
team. Access to the production environment for our team was restricted.
We created an Installation guide, Ansible Playbook, and Cloud Formation
Template allowing the clients to automatically create all infrastructure
and install the system and subsequent updates without involving our
engineers. We also conducted a few training sessions with screen sharing
to train the customer’s DevOps team to use the installation scripts.
All highlights of the project and the development process are described