One of the recurring themes I’m hearing at tech conferences this year is in-memory computing. At the core of these discussions is the need to make data-informed business decisions more quickly. Instead of taking a snapshot from your data sources and loading the data into a data warehouse for analysis, which can take hours or days to get a a number of solutions are promising analytics side-by-side with your transactional workloads in the form of in-memory computing.

What exactly is in-memory computing?

Instead of working with data stored to disk, in-memory computing is centered around data stored in RAM or the CPU cache, which gets the efficiency of the direct access to the system bus you simply can’t get from reading from storage.

The speed of access isn’t without it’s challenges. Memory is notoriously volatile, though purpose built systems designed for in-memory computing help mitigate some of the risk. The other potential challenge of in-memory computing is the tendency for data sets to grow over time. Some of the in-memory solutions currently on the market require memory to increase as the database increases in size.

Some of the more innovative in-memory solutions use columnar data storage in combination with data compression so that you don’t need the entire database in memory in order to get the full benefit of real time analysis. This translates to being able to do data analysis in near real time instead of relying on classic ETL approaches to data analysis where you are effectively doing analysis on a snapshot in time.