The data layer collects, organizes, governs, and prepares financial data for analysis and modeling. Its job is not just storage: it determines how data enters the system, what transformations it receives, and how privacy constraints are enforced. section 4.3.1 section 1.4.1
A credit system may divide the layer into components suited to different data needs:
- Analytical stores, such as data lakes or warehouses, retain historical records for model training and backtesting. section 4.3.1
- Streaming engines calculate real-time features for decisions that must happen quickly. section 4.3.1
- Segmented data marts curate datasets for tasks such as behavioral scoring and collections analytics. section 4.3.1
- Data quality, privacy, and governance controls help address incomplete or erroneous feeds and enforce constraints such as PII tokenization. section 4.3.1
From financial signals to features
Repayment histories and default events can support credit-risk modeling; real-time application activity can yield velocity features. In investment and fraud settings, engineered features might capture volatility or transaction velocity. section 1.4.1 section 4.3.1
Figure 4.4 In the data asset layer multiple data sources—ranging from internal transaction logs and repayment histories to external credit bureau scores and alternative signals—flow into a centralized environment. Tools such as Airflow and Kafka manage real-time ingestion, while data lakes and specialized marts support segmentation, behavioral scoring, and performance tracking. Privacy, data quality, and governance mechanisms ensure compliance with regulatory standards throughout the pipeline.
Follow how internal and external sources feed into ingestion, storage, specialized marts, and governance controls. section 4.3.1
Within a broader financial application, the data layer handles pipelines and feature engineering; modeling consumes the prepared inputs, while other layers manage strategy, monitoring, and application behavior. section 10.1.1
