Elasticsearch is a distributed, RESTful search and analytics engine for indexing, searching, and analyzing large volumes of structured or unstructured data. It supports full-text search and can also serve as a destination for searchable operational logs. section 11.1 section 21.3.2
Choose Elasticsearch when its search and analytics capabilities justify the operational trade-offs:
- Use it as a dedicated search engine when database-native search cannot accommodate sophisticated requirements, including searches across multiple nested fields. section 11.1
- Replicate database collections into Elasticsearch when applications need separate search infrastructure, but plan for synchronization, monitoring, configuration, and maintenance. section 11.1
- Send Atlas logs to Elasticsearch when fast log searching and analysis are useful, then use Grafana to visualize trends and identify database problems. section 21.3.2
Replicated search and monitoring
A MongoDB application can replicate collection changes to Elasticsearch through Monstache, which uses change streams to monitor updates and send them to the search cluster. A separate deployment can send Atlas logs to Elasticsearch and connect Grafana for dashboards. These arrangements improve search or observability, but add resource usage and consistency concerns. section 11.1 section 21.3.2
Figure 11.1 High-level data synchronization architecture. The application uses separate drivers and query languages to interact with the database and search clusters. Data is replicated from the database cluster to the search cluster to keep the index updated. This setup adds complexity, increases resource usage, and can lead to consistency issues. (Image © MongoDB 2024 CC BY-NC-SA 3.0)
Inspect the high-level architecture showing separate database and search clusters, drivers, query languages, and replicated index updates. section 11.1
