Information on this page is taken from Designing Data-Intensive Applications by Martin Kleppmann.
An application is data-intensive if data is its primary challenge - the quantity of data, the complexity of data, or the speed at which it is changing - as opposed to compute-intensive, where CPU cycles are the bottleneck. A data intensive application is typically built from standard building blocks that provide commonly needed functionality; these data systems include:
- Store data so that they, or another application, can find it again later (databases).
- Remember the result of an expensive operation, to speed up reads (caches).
- Allow users to search data by keyword or filter it in various ways (search indexes).
- Send a message to another process, to be handled asynchronously (stream processing).
- Periodically crunch a large amount of accumulated data (batch processing).