In Memory Database is a fascinating field for a programmer, for an architect, and for technology leaders. Let us try to understand in this article, what is In-Memory DB and why are these so useful.
As a programmer, designer or architect, whenever I work with traditional databases, one major area to be conscious about is, that database access should be designed carefully. This is because traditional database system store everything on disk, and hence any access or update to this data could be one of the slowest operations in application flow. Undoubtedly, a lot of research and improvements have been done to improve the performance of disk based database systems with improved storing, searching and retrieving logic. Still, disk access, being mechanical operation, always have limitations. And this is one of the important reasons to be fascinated about in-memory database systems.
Visualize a system where all data (or all data required for curernt context) is sitting in memory. Which means, we can expect a lot more efficient and faster access to data. Database software can have much simpler logic to manage the data, as there is no need to manage the loading/unloading of pages in memory. There is no need to work on time taking locking mechanism to safeguard the data on disk, as access in memory is much faster and chances of conflicts are extremly small which change the way we design database system algorithms. It also means that we can exploit the best available CPU power (which is increasing continuously) to process the data available in memory, having no limitation of disk access. This could also mean that there is no need of another layer of caching system. Rather, database itself will be working like in-memory cache. There are many more benefits if we design the system considering in-memory database system.
Let us understand what is In-Memory Database System (IMDS). It is a DBMS which maintain all data primarily in Main Memory. Data will be loaded in memory even if it is in GB or TB. A few highlights:
• With 64 bits computer architecture, systems are capable to address 16 EB of data (1 TB * 1000 * 1000). 82% of the enterprise application’s databases are below 1 TB, growing with an average speed of 10% per year, which means that In-Memory database systems can cater to most of the applications now and in coming future as well.
• Does not need write or read to Disk, hence no dependency on mechanical parts and their performance limitations. When all data is stored in single address space, it reduces the complexity of storage algorithms, no need now of loading and unloading the pages in memory
• Much faster than Disk based traditional DBMS. Having all data in memory means that data is available at our finger tips, only microseconds or nanoseconds away.
• And it supports ACID properties of Database, including D (durability)