Lift Ignitor
AI-driven Recommendation System for Ecommerce & Media Agencies
Overview
Folio3 helped implement a solution and provide consequent infrastructure management services for an AI-Driven Recommendations System, a machine learning-driven platform providing real-time personalized content recommendations to optimize and enhance user experience in the Ecommerce & Media/Entertainment industry. This was achieved through a Cloud-first approach through Google Cloud Platform (GCP).
The Challenge - Real Time Data Visualization
Folio3 recognized the customer’s need for powerful infrastructure to accommodate real time data processing, that could:
- Store & analyze large-scale datasets in real time, with flexible visualization options
- Ensure a robust infrastructure that allowed seamless log monitoring
- Ensure stable, high-frequency deployments on the Cloud with minimal downtime and automated backups
The Solution - Cloud-based, flexible infrastructure
The Folio3 team proposed Google Cloud Platform (GCP) as a Cloud-based solution that would allow usage of powerful advanced data analytics tools to achieve real time data processing and visualization.
Adopting a Cloud-first approach for the AI-Driven Recommendations System also allowed Folio3 to tackle these wider challenges:
- Usage of BigQuery, a fully managed, scale and serverless Data Warehouse that can analyze massive datasets quickly using SQL queries
- PubSub was used for large-scale activity logging and cleaning training data
- Compute Engine for backend service components
- Bigtable and Aerospike were used as NoSQL Database Services for their capability to handle large-scale data and reliability
- CloudSQL for fully managed database services, along with automated backups, replication and scaling, in addition to easy integration with other GCP services used
- Bucket (Cloud Storage) for storing and accessing unstructured data
- Apache Spark was used to process large-scale data and model training through its high-performance distributed computing framework
- Prometheus for real-time monitoring and alerting
- Terraform for a consistent and organized infrastructure setup, with streamlined deployments
- Zipkin for tracking end to end requests
- Apache Thrift for communication between different services
- Hotjar for session recording and investigating reported issues
- PagerDuty for incident management and response
Tools & Technologies Used
Infrastructure Stack
