Data ingestion collects and imports data from source systems so it can be stored, processed, and made available for financial decisions. In financial services, ingestion must account for differing source cadences while protecting data quality, privacy, and security. section 5.1.1 section 1.5.1
Financial ingestion pipelines may collect data from sources such as:
- Core banking transaction feeds, delivered in end-of-day batches. section 5.1.1
- BNPL or card approvals, streamed for near-real-time processing. section 5.1.1
- CRM records and watchlists, delivered as daily or weekly snapshots. section 5.1.1
- Credit bureau, utility-payment, and other external data used in borrower assessment. section 1.5.1
Choose batch or near-real-time ingestion according to operational needs and risk tolerance. A nightly pipeline can combine daily usage for risk modeling; partial real-time merges can support instant approvals before a daily batch is finalized. After ingestion, clean and validate data, apply domain rules and privacy controls, and organize it into domain-specific data marts for downstream modeling. section 5.1.1 section 1.5.1
Fraud detection events
A real-time fraud pipeline can ingest transaction events, login attempts, and behavioral logs through Kafka. Stream processing computes features, which are managed through a feature store and passed to model serving for a decision or risk score. section 7.4.2
Figure 1.4 In the data asset layer, raw data, including repayment histories, credit bureau snapshots, and alternative signals, flows into specialized data marts. Ensuring data lineage, access control, and regulatory compliance are critical steps before modeling. By creating consistent, domain-focused features, the data asset layer provides a solid foundation for accurate credit decisions.
Shows how raw repayment histories, bureau snapshots, and alternative signals flow into specialized data marts, with lineage, access control, and compliance addressed before modeling. section 1.5.1
