Big Data Platform Speeds Up Signaling Data Analysis for BICS

Business Challenge 

Initially, BICS had a classical RDBMS-based Data Warehouse that demonstrated slow data processing and analysis. Given that BICS operated with massive volumes of data, it was decided to migrate the current workload to a new data storage and analytics system.  

The Big Data platform extension project was then launched with the following goals: 

• Migrate data and workloads to a highly scalable and flexible data platform to collect, process, store, and analyze big volumes of data (over 10 terabytes)  
• Enable the parallel processing of both signaling data coming from hardware switches/probs and data feeds related to business processes  
• Allow analyzing data in the reports in real-time to make decisions based on the latest data  
• Optimize network capacity and traffic behavior  
• Build a data platform that can meet Advanced Analytics needs, e.g., ensure customer behavior analysis and predictive modeling, using AI & ML approaches  
• Provide 24/7 data platform monitoring to meet its SLA metrics and enhance data accuracy  
• Optimize costs for storing and managing massive amounts of data  
• Ensure compliance of the data platform with GDPR 

Solution  

We contributed to the extension of the Big Data platform and helped BICS to perform a non-disruptive migration of their data from heterogeneous sources into a unified data platform built on the Apache Hadoop framework. 

Since the migration, we have been fully in charge of the Big Data platform of BICS, including the following parts: 

• Data preparation & processing: developed data preparation and ETL processing using Apache NiFi, Spark, and Hive  
• Provided batch and real-time stream data processing  
• Ensured parallel data processing with Hadoop to  
• prepare signaling data for further reporting in just 15 minutes  
• provide a platform that enables AI & ML predictions based on the existing data patterns  
• Operational layer for the solution monitoring and SLA reinforcing: overview and monitoring of international traffic behavior when accessed from the customer-facing portal  
• 24/7 data platform monitoring and support  
• BICS Big Data Platform Architecture 

As a result, we helped BICS to build a cost-efficient data platform that complies with GDPR data protection requirements and provides our client with extensive data reporting capabilities. 

Technologies & Tools 

• Hadoop 
• Spark 
• Kafka 
• Oozie 
• Elasticsearch 
• Qlik Sense BI tools 
• NiFi 
• Hive 

Business Value 

For BICS, the Hadoop-based data platform developed by Infopulse proved its ability to handle big data and convert it into actionable insights at the speed of their business. In fact, the effectiveness of the solution unfolded many other data processing use cases – for IoT, MVNO/MVNE, and billing, which are actively incorporated in the ongoing project.  

Here is why BICS big data platform has turned into a powerful data management and advanced analytics solution: 

• Optimized storage and analysis of massive volumes of data (up to 2 TB during traffic peaks)  
• Reduced time for data analysis from 30 to 15 minutes regarding 1.5X traffic increase in a 4-year period  
• Data tracking and analysis help optimize costs and margin  
• Increased visibility and availability of customer data  
• Granular reporting of real-time data allows for smarter decisions  
• Proactive troubleshooting of issues due to the monitoring of each process at the governance/operational layers and the ability to raise the alarm when needed  
• The data platform enables further data usage in ML-powered data analysis and predictive analytics (e.g., preventing network issues, fraud detection, quality prediction, and advanced analysis) by BICS data scientists