Chapter 8: Designing the federation layer

Introduction

The chapter introduces why an Iceberg lakehouse needs a federation layer and what that layer provides for data spread across systems.

• Need for data federation: Explain why an Iceberg lakehouse may need a federation layer even when centralization is a goal.
  Why it matters: Federation addresses data that the lakehouse cannot or need not physically centralize.
• Federation layer roles: Describe how the federation layer bridges data systems and harmonizes what analysts use.
  Why it matters: A common interface and consistent business meaning let analysts work across data sources without first moving everything.

Key terms: Apache Iceberg lakehouse, federation layer, semantic layer, business logic, unified, governed interface

8.1 What data federation is and why it matters

This section explains why an Iceberg lakehouse may need federation and how a hub-and-spoke approach lets queries use external data without moving it into the lakehouse.

• Federation around the lakehouse: Explain how federation keeps the Iceberg lakehouse as the central hub while allowing queries to use data from external systems without duplicating or relocating it.
  Why it matters: Federation lets the lakehouse remain the main analytics platform even when useful data must stay elsewhere.
• Reasons for federated access: Describe why valuable organizational data may remain outside the lakehouse and what federation makes possible for business users.
  Why it matters: An architecture must account for useful data that cannot reasonably be brought into Iceberg.

Key terms: data federation, hub-and-spoke approach, just-in-time data, remote sources, SaaS platforms, legacy databases, semantic layer, business logic

8.1.1 Common use cases and challenges driving federation needs

This section explains why organizations federate data instead of moving everything into Iceberg, and the challenges that decision creates.

• Reasons to federate data: Explain when querying data where it resides is preferable to replicating it into Iceberg, using examples from the section.
  Why it matters: It helps you decide when a source needs an ingestion pipeline and when direct access is more useful.
• Federation tradeoffs and source risks: Describe the performance, governance, consistency, and source-system risks that can accompany federated queries.
  Why it matters: Direct access is useful only if its effects on users and source applications are taken seriously.

Key terms: data federation, Iceberg, ETL, replication, query performance, governance, data freshness, source systems

8.1.2 How federation aligns with agility and accessibility

This section explains how federation speeds access to new data, broadens self-service analytics, and uses a semantic layer to keep interpretations consistent.

• Agility through direct access: Explain how federation reduces the delay between new data becoming available and people using it, compared with traditional ETL workflows.
  Why it matters: Direct access is especially valuable when data loses relevance quickly.
• Self-service across user groups: Describe how access to federated data supports different users without making every request dependent on engineering teams.
  Why it matters: Broader access lets more teams test ideas and act on data sooner.
• Shared semantic business context: Explain why accessible federated data needs a semantic layer and how shared definitions help users and tools interpret it consistently.
  Why it matters: Shared business definitions reduce conflicting results and make federated data more trustworthy.

Key terms: federation, direct access, traditional ETL workflows, self-service, federated views, feedback loops, semantic layer, metrics, dimensions, business logic

8.2 Key requirements for federation

The section connects platform-audit findings to three requirements for a federation layer tailored to an organization’s needs.

• Audit-driven federation design: Explain why federation-layer design should respond to the organization’s goals, constraints, and data landscape rather than follow a generic template.
  Why it matters: Starting with the audit keeps federation choices tied to real user and platform needs.
• Three federation requirements: Describe the three requirements identified here: connections to external sources that are impractical to ingest, shared definitions and governance through a semantic layer, and flexible interfaces for users’ tools.
  Why it matters: These requirements give you a starting framework for assessing whether a federation layer will be useful to its users.

Key terms: platform audit, federation layer, external data sources, semantic layer, shared definitions, governance, tool-friendly interfaces

8.2.1 Supporting diverse data sources without duplication

This section explains why a federation layer should query diverse data sources in place and what capabilities it needs to do so securely and efficiently.

• In-place access across sources: Explain how federation lets a lakehouse query data across different systems without requiring a uniform storage format or centralized ingestion.
  Why it matters: This is how a lakehouse can serve data spread across an existing ecosystem.
• Reasons to avoid copying: Justify querying data in place, especially when its analytical value is short-lived, and describe the implications for freshness and source-system authority.
  Why it matters: In-place access can avoid unnecessary copies while keeping analysis close to current, authoritative data.
• Federation engine capabilities: Describe the connector, security, pushdown, and workload-handling capabilities the section requires of a federation engine.
  Why it matters: Connectivity alone is not enough if cross-source queries cannot be secured or run smoothly.

Key terms: federation layer, centralized ingestion, source of truth, data freshness, storage duplication, native connectors, adapters, credential delegation, role-based access control (RBAC), pushdown

8.2.2 Ensuring consistent semantics and business logic

This section explains how a semantic layer gives federated data consistent, governed business definitions across sources and tools.

• Shared business definitions: Explain why a federation layer needs a semantic layer and how centrally defined metrics, dimensions, and calculations prevent teams from producing conflicting results. Describe how it gives users a consistent interface even when source systems differ.
  Why it matters: Shared definitions make federated query results more consistent and trustworthy across the organization.
• Governed reusable datasets: Describe the federation-platform capabilities that make shared semantic definitions usable and governed, and explain how a centrally defined revenue calculation can serve multiple analytics tools and accommodate a rule change.
  Why it matters: Reusable, governed definitions let analytics teams share trusted calculations without repeating or separately updating the logic.

Key terms: semantic layer, business logic, metrics, dimensions, calculations, reusable views, virtual datasets, versioning, governance, access controls

8.2.3 Providing seamless connectivity for analytics tools

This section explains how a federation layer connects analytics tools and AI agents through interfaces suited to their needs.

• Standards-based analytics connectivity: Explain why federation must make data accessible to users and tools, not merely connect to many data sources. Describe the roles of JDBC and ODBC for BI tools and REST APIs for applications and automation.
  Why it matters: Source coverage has limited value if people cannot use the data through their existing tools.
• Arrow-native data exchange: Explain how Apache Arrow, Arrow Flight, Arrow Flight SQL, and ADBC support faster exchange and querying of data in a federated lakehouse.
  Why it matters: These interfaces help move large query results between systems with less overhead.
• MCP access for AI agents: Explain how MCP connects AI agents to live, governed federated data and why it complements rather than replaces conventional access interfaces.
  Why it matters: A common, governed interface lets AI agents use enterprise data without a separate integration for every pairing.

Key terms: federation layer, JDBC, ODBC, REST APIs, Apache Arrow, Arrow Flight, Arrow Flight SQL, ADBC, serialization overhead, Model Context Protocol (MCP)

8.3 Introducing Dremio and Trino

Introduces Dremio and Trino as federated SQL platforms and contrasts the priorities that make each a different fit for a lakehouse federation layer.

• Dremio and Trino priorities: Distinguish Dremio’s Iceberg-centered, governed, self-service approach from Trino’s emphasis on broad source connectivity, flexibility, and distributed query scale when choosing a federation platform.
  Why it matters: The contrast helps you identify which platform’s priorities better match your federation needs.

Key terms: Dremio, Trino, federated SQL, Apache Iceberg, semantic layer, connector ecosystem

8.3.1 Dremio

Introduces Dremio as a federated query layer that queries existing data in place, accelerates cross-source analytics, and exposes familiar interfaces to users and applications.

• Dremio’s query-in-place role: Explain that Dremio is a federated SQL query layer over existing data sources, rather than a database that must store the data itself.
  Why it matters: This distinction explains how Dremio can provide access across an existing data platform without requiring an initial data migration.
• Federated query acceleration: Explain why performance is a concern when querying multiple systems and how Dremio’s stated acceleration features help address it.
  Why it matters: Performance determines whether federated data is responsive enough for interactive analytics and dashboards.
• Interfaces for federated access: Describe the interfaces Dremio offers and why they make federated data accessible through familiar tools and custom workflows.
  Why it matters: Familiar interfaces let different users and applications access distributed data without changing their usual tools or query patterns.

Key terms: federated query engine, query layer, query data in place, SQL, columnar execution engine, Apache Arrow, reflections, caching, predictive optimization, redundant scans

8.3.2 Dremio’s architecture

This section explains how Dremio’s distributed services, workload-specific engines, caching, and Iceberg catalog support responsive analytics across federated data sources.

• Direct federated query performance: Explain how Dremio queries diverse data sources without requiring data movement or transformation before analysis, and describe the performance features the section identifies.
  Why it matters: Direct querying and acceleration make analytics practical when data remains in different systems.
• Coordinator and executor responsibilities: Distinguish the planning and management responsibilities of coordinators from the distributed execution performed by engines and executors.
  Why it matters: Separating planning from execution helps explain how Dremio scales federated queries.
• Independent engine workload scaling: Explain why administrators can configure and scale Dremio engines independently for different workloads.
  Why it matters: Separate engine sizing lets a platform respond to different demands without treating all workloads alike.
• Native Iceberg catalog role: Describe what Dremio’s integrated Iceberg-based catalog contributes to its lakehouse capabilities.
  Why it matters: The catalog connects Dremio’s federation architecture to governed Iceberg lakehouse tables.

Key terms: distributed, SQL-based query engine, Apache Arrow, reflections, Cloud Columnar Cache, master coordinator, scale-out coordinators, engines, executors, high concurrency, workload isolation

8.3.3 Dremio’s connector ecosystem and Iceberg-centric focus

This section explains how Dremio federates queries across varied sources while concentrating its deepest integration and optimization on Apache Iceberg.

• Connector breadth and Iceberg focus: Explain how Dremio combines connectors for many kinds of sources with an Iceberg-first strategy, allowing organizations to query data where it resides while reserving its deepest integration for Iceberg.
  Why it matters: This distinction helps you evaluate Dremio for both multisource access and an Iceberg-centered lakehouse.
• Native Iceberg catalog capabilities: Describe what Dremio’s built-in Iceberg catalog enables and how this differs from the more external-catalog-dependent Iceberg support described for engines such as Trino.
  Why it matters: Native table management and standards-based interoperability are central to Dremio’s Iceberg-focused value.
• Cross-source query enhancements: Explain the two query capabilities the section names for advanced use cases: external query pushdown and runtime filtering.
  Why it matters: These capabilities show how Dremio can handle source-specific SQL and improve some cross-source joins.
• Connector coverage gap options: Describe the options the section gives when Dremio lacks a needed connector.
  Why it matters: Connector breadth is not unlimited, so an architecture needs a path for unsupported sources.

Key terms: connector ecosystem, federate queries, Iceberg-first approach, built-in Iceberg catalog, Apache Polaris, Iceberg REST Catalog API, compaction, vacuuming, metadata pruning, external query pushdown

8.3.4 Dremio’s performance enhancements

This section explains how Dremio improves federated query performance through columnar execution, reflections, caching, runtime filtering, and isolated execution engines.

• Arrow-based columnar execution: Explain how Dremio uses Apache Arrow for analytical execution and data exchange.
  Why it matters: Arrow helps Dremio process analytical queries efficiently and exchange data with compatible tools.
• Transparent Data Reflections: Describe how Dremio’s planner uses Data Reflections to accelerate queries and distinguish the two common reflection types.
  Why it matters: Reflections can speed repeated analytical queries without requiring users to rewrite them.
• Cloud Columnar Cache: Explain how Dremio’s Cloud Columnar Cache reduces repeated cloud-storage reads.
  Why it matters: Reusing nearby data can lower both query latency and cloud data-transfer costs.
• Early runtime filtering: Trace how runtime filtering cuts data movement in a federated join and explain the semantic layer’s contribution.
  Why it matters: Filtering before retrieval limits work on remote systems and across the network.
• Isolated workload engines: Explain how separate Dremio execution engines isolate workloads and can be configured and routed for different needs.
  Why it matters: Isolation keeps demanding jobs from disrupting interactive work while allowing compute spending to follow demand.

Key terms: Apache Arrow, columnar in-memory format, vectorized execution, zero-copy data exchange, Arrow Flight, Data Reflections, raw reflections, aggregation reflections, cost-based planner, Cloud Columnar Cache

8.3.5 Trino

Introduces Trino as a distributed SQL engine for federated analytics, explains its basic execution model and platform boundaries, and clarifies its Presto lineage.

• Trino’s federated analytics role: Explain why Trino suits interactive analytics across diverse data sources rather than transactional workloads. Describe how its connectors let users query data in place through one SQL interface.
  Why it matters: This explains when Trino is a useful federation-layer choice.
• Distributed query execution: Trace the roles of a Trino coordinator and its workers when a query runs, including how work is divided for parallel execution.
  Why it matters: The coordinator–worker division is the basic explanation this section gives for Trino’s performance.
• Query engine platform boundaries: Distinguish Trino’s built-in query execution and connectivity from the semantic, curation, and business-user capabilities that require complementary tools or managed offerings.
  Why it matters: A federation engine’s query capabilities should not be mistaken for a complete business-facing platform.
• Presto and Trino lineage: Describe how Trino arose from Presto and distinguish it from the separately maintained PrestoDB project.
  Why it matters: The lineage helps make sense of Presto and Trino names in tools and documentation.

Key terms: Trino, OLAP, OLTP, plugin-based connector model, unified SQL interface, coordinator, workers, stages, tasks, splits

8.3.6 Trino’s modular architecture for wide-source support

Explains how Trino’s extensible connectors and distributed query engine provide SQL access across diverse data sources.

• Modular connector and catalog design: Explain how Trino’s connector architecture makes data sources available for querying: developers can add connectors through its SPI, and each source is configured independently as a catalog accessed with standard SQL.
  Why it matters: This design lets teams add and combine sources without making users learn each source’s configuration.
• Federated queries across diverse sources: Describe how Trino uses its distributed query engine to query and join across remote systems, and why that makes it useful for fragmented data landscapes without extensive duplication.
  Why it matters: Federated queries let organizations analyze data across otherwise separate systems without first copying it all into one place.

Key terms: connector, service provider interface (SPI), catalog, standard SQL, distributed query engine, federated queries, federated analytics, minimal duplication

8.3.7 Trino’s flexibility and configurability for complex environments

This section explains how Trino’s distributed execution, routing, and tuning options support complex federated environments, alongside the operational expertise they require.

• Distributed query execution: Explain how Trino breaks a query into parallel work and why that helps when querying large datasets or federated sources.
  Why it matters: This explains why Trino can handle analytical queries spanning large datasets and multiple sources.
• Federated performance controls: Describe the performance techniques and configurable controls the section identifies for queries across varied sources and workloads.
  Why it matters: The available controls help teams adapt Trino to different query shapes and performance bottlenecks.
• Gateway routing and isolation: Explain where Trino Gateway sits and how it supports access to, routing among, and management of multiple Trino clusters.
  Why it matters: A gateway lets teams manage separate clusters without making users manage separate connections.
• Storage independence and operational tradeoff: Explain why Trino fits hybrid or complex deployments and what operational responsibility accompanies that flexibility.
  Why it matters: Trino’s deployment freedom is most useful when a team can also manage the resulting complexity.

Key terms: hierarchical execution model, queries, stages, tasks, splits, coordinator–worker architecture, filter pushdown, cost-based optimizer, dynamic filtering, broadcast joins

8.3.8 Trino’s community-led evolution and vendor extensions

This section explains how Trino’s open source community and commercial vendors jointly shape its capabilities and adoption options.

• Community-led Trino development: Explain how Trino’s contributors and open governance shape the engine’s evolution.
  Why it matters: This explains why Trino can gain support for new data platforms and formats as needs emerge.
• Vendor extensions and adoption options: Describe what commercial Trino offerings add and how organizations can choose between them and the open source engine.
  Why it matters: The distinction helps organizations match their Trino deployment to operational and governance needs.

Key terms: open source community, open governance model, connectors, open source core, commercial distributions, managed services, enterprise-focused features

8.3.9 Semantic layer considerations in Trino

Explains how to add semantic capabilities to Trino and what to consider when choosing an approach.

• Trino semantic layer gap: Explain that Trino has broad connector support and strong federated query performance but, unlike Dremio, does not include a built-in semantic layer. Describe why that matters for governed, business-friendly data experiences.
  Why it matters: Connector breadth alone may not meet the needs of people relying on consistent business definitions.
• Trino semantic capability choices: Describe the two approaches given for adding semantic capabilities to Trino, and explain what an organization should weigh when choosing between them.
  Why it matters: The right approach depends on both the semantic experience required and the cost and complexity of adding it.

Key terms: Trino, Dremio, semantic layer, standardized metrics, curated datasets, dedicated semantic layer tool, enterprise-grade managed services, semantic models, governed data access, metric standardization

8.4 Deployment models

This section frames deployment choice for Dremio and Trino as a balance between infrastructure control and managed-service convenience.

• Federation deployment tradeoffs: Explain why choosing a federation deployment model depends on organizational priorities, contrasting infrastructure control with the agility and lower operational overhead of managed services.
  Why it matters: These tradeoffs help explain why the same federation platform may suit different deployment choices at different organizations.

Key terms: deployment model, self-managed clusters, fully managed services, cloud-native SaaS

8.4.1 Deploying Dremio

This section compares Dremio’s self-managed editions with Dremio Cloud so readers can match deployment responsibilities and capabilities to their needs.

• Self-managed edition tradeoffs: Distinguish Dremio Community Edition from Enterprise Edition by where each can run and the capabilities Enterprise adds for production lakehouses.
  Why it matters: The editions differ in both deployment constraints and the capabilities available to a self-hosted lakehouse.
• Dremio Cloud responsibility split: Explain what Dremio and the customer each control in Dremio Cloud, and why an organization might choose it instead of a self-managed deployment.
  Why it matters: A managed service reduces deployment work without transferring control of the customer’s data sources and compute to Dremio.

Key terms: Dremio Community Edition, Dremio Enterprise Edition, Kubernetes, RBAC, FGAC, reflections, Dremio Cloud, SaaS, control plane, data sources

8.4.2 Deploying Trino

This section compares self-managed and managed Trino deployments and explains how their tradeoffs fit different organizational needs.

• Trino deployment tradeoffs: Explain how self-managed and cloud-managed Trino differ in control, customization, and operational responsibility, and why an organization might choose either.
  Why it matters: The deployment choice determines how much engineering work your team takes on to gain control.
• Self-managed deployment flexibility: Describe where self-managed Trino can run and what makes it adaptable to particular data sources, security needs, and performance goals.
  Why it matters: Deployment flexibility lets teams fit Trino to their existing infrastructure and requirements.
• Managed Trino offering differences: Distinguish the managed offerings described by the section using their stated operational models or notable use cases.
  Why it matters: The provider’s model matters as much as choosing managed Trino in the first place.

Key terms: self-managed deployment, cloud-managed services, operational overhead, customizability, open source, modular architecture, pluggable connector ecosystem, plugins and extensions, Amazon Athena, Cloudera

8.5 Federation platform decision scenarios

The section frames federation platform selection as a context-dependent decision rather than a fixed choice between tools.

• Contextual federation platform selection: Explain why choosing a federation platform requires weighing an organization’s needs rather than applying a universal rule.
  Why it matters: It keeps the scenarios from becoming blanket recommendations for particular tools.

Key terms: data federation platform, tradeoffs, capabilities

8.5.1 Fragmented multisource environment: Trino for connector breadth

This scenario explains why Trino is a practical federation choice when an organization needs analytics across fragmented data sources without first centralizing them.

• Trino for fragmented data: Explain why an organization with data spread across many systems might choose Trino to enable analytics before building a centralized lakehouse.
  Why it matters: Connector breadth can make analytics possible now while leaving room for a later move toward a consolidated lakehouse.

Key terms: Trino, connector ecosystem, federate queries, read-time access, minimal data movement

8.5.2 Building a native Iceberg lakehouse: Dremio for Iceberg-native features

This section explains why Dremio can be a strong federation choice for a lakehouse centered on Apache Iceberg.

• Iceberg centered Dremio fit: Justify choosing Dremio when Iceberg is the foundational table format, citing its native Iceberg integration, read/write support, performance enhancements, and integrated management capabilities.
  Why it matters: It gives you a reason to choose a federation platform based on your lakehouse’s core table format, not just its ability to connect to data.

Key terms: Dremio, Apache Iceberg, automatic table optimization, autonomous reflection management, Apache Polaris, partition pruning

8.5.3 Empowering business users with UI and governed datasets: Dremio

This section explains why Dremio fits organizations that want business users to explore data independently without sacrificing consistent definitions or governance.

• Centralized business definitions: Explain how Dremio’s semantic layer addresses inconsistent metrics and business logic across departments.
  Why it matters: Shared definitions make dashboards and ad hoc analysis easier to trust across departments.
• Governed self-service exploration: Describe how Dremio combines a no-SQL interface for business users with access controls maintained by technical teams.
  Why it matters: The combination lets more people work with data while preserving governed access.

Key terms: semantic layer, metric definitions, business logic, federation engine, user-friendly UI, virtual datasets, fine-grained access control, self-service

8.5.4 Lightweight querying of Hudi datasets: Trino via AWS Athena

This subsection explains why Athena, powered by Trino, fits occasional queries over existing Hudi data in S3 better than a dedicated engine or Dremio.

• Athena for occasional Hudi queries: Justify choosing AWS Athena for lightweight, ad hoc queries over existing Hudi datasets in S3, and explain why Dremio is not an option in this scenario.
  Why it matters: The best query option depends on the data’s existing format and how often the team needs to use it.

Key terms: AWS Athena, Trino, Apache Hudi, Amazon S3, serverless, pay-per-query

8.5.5 On-prem Cloudera modernization: Trino replacing Impala for performance

This scenario explains why managed Trino is a practical query-engine upgrade for an on-premises Cloudera deployment whose Impala workloads have outgrown its capabilities.

• Managed Trino upgrade fit: Explain why an enterprise using Impala on an on-premises Cloudera cluster might choose Cloudera’s managed Trino offering, and how that choice preserves its existing environment.
  Why it matters: It shows how a platform-native, vendor-supported upgrade can improve querying while protecting existing infrastructure investments.

Key terms: Impala, managed Trino, Cloudera, federated queries, Customized Professional Services (CPS)

8.5.6 Hybrid cloud Iceberg strategy: Dremio bridging on-prem and ADLS

This section uses an on-premises Hadoop-to-ADLS migration to show why Dremio can support a gradual, federated Iceberg lakehouse transition.

• Phased hybrid Iceberg migration: Explain how Dremio can bridge on-premises Hadoop and Azure Data Lake Storage (ADLS) during a gradual Iceberg lakehouse migration instead of requiring a disruptive lift-and-shift.
  Why it matters: Federation lets a team modernize without moving every workload at once.
• Hybrid deployment and platform benefits: Describe the Dremio capabilities the example cites as reasons it fits a hybrid Iceberg strategy, and the benefits attributed to them.
  Why it matters: The example connects platform choice to continuity, performance, and governance during modernization.

Key terms: hybrid architecture, Apache Iceberg, ADLS, lift-and-shift migration, hybrid query federation, first-class Apache Iceberg support, integrated catalog, self-managed, cloud-managed, autonomous reflection management

8.6 Federation alternatives

The section introduces architectural alternatives to traditional federation engines for cross-system analytics.

• Alternative paths to federation: Explain that Dremio and Trino are not the only way to enable cross-system analytics: emerging alternatives pursue similar goals through different architectural approaches.
  Why it matters: It frames federation as an architectural choice rather than a choice limited to Dremio or Trino.

Key terms: federation engines, cross-system analytics, data duplication

8.6.1 Virtualization via shortcuts in OneLake

Explains how OneLake shortcuts virtualize external storage and why their local-compute approach trades cross-system query flexibility for simplicity and predictable performance.

• OneLake shortcut query model: Explain how OneLake shortcuts make external data appear as native tables and distinguish their pushdown behavior from conventional federation.
  Why it matters: This distinction explains what OneLake virtualizes and where query work happens.
• Shortcut virtualization tradeoff: Describe the tradeoff between OneLake’s limited cross-system querying and its operational simplicity and predictable performance.
  Why it matters: The tradeoff helps explain when shortcut-based virtualization may be attractive.

Key terms: OneLake, shortcuts, push down, filters, column projections, distributed joins, aggregations, cross-system querying, operational simplicity, predictable performance

8.6.2 AI-native data virtualization with Spice.ai

This section presents Spice.ai as a federation alternative designed to serve AI-enabled applications through fast virtualized data views.

• AI-oriented data federation: Describe how Spice.ai combines federated SQL across diverse sources with building blocks for AI-enabled applications.
  Why it matters: It explains why Spice.ai is presented as more than a way to query across sources.
• Materialized views for applications: Explain how Spice.ai's small materialized views differ from querying remote sources for every request, and what they serve.
  Why it matters: This is the section's main distinction between Spice.ai and per-request remote querying.
• Query and production foundations: State the query technologies and platform capabilities the section identifies as supporting Spice.ai in production.
  Why it matters: These named technologies and controls explain the production-backend role the section attributes to Spice.ai.

Key terms: Spice.ai Cloud Platform, federated SQL queries, model inference, vector search, retrieval-augmented generation (RAG), virtualized data views, materialized views, cached, replicated, responsiveness

8.6.3 Choosing the right fit

Explains how to choose or combine specialized federation alternatives according to a lakehouse’s users, data sources, and workloads.

• Choosing federation approaches: Explain when shortcut-based virtualization, AI-oriented virtualization, and a general-purpose federation engine each fit, and why an organization might combine them.
  Why it matters: It helps you choose complementary tools instead of expecting one approach to serve every access pattern.

Key terms: general-purpose federation engines, shortcut-based virtualization, Spice.ai, broad analytical access, application-centric performance

8.7 Summary