Breaking the Memory Wall: Light-weight Compression and Near-data Processing for Data-intensive Applications
In current systems, the transfer of data between the off-chip memory to the processor takes two orders of magnitude more time and consumes three orders of magnitude higher energy than the actual computation. A variety of big data and machine learning applications are hindered by this costly data movement.
To unlock the value of these applications, we need to alleviate the data movement overhead by departing from conventional computer architecture design. In my Ph.D. studies, I have explored two approaches: (i) enabling flexible-bitwidth value representation in the memory hierarchy of general-purpose processors and (ii) processing data inside the memory.
First, we proposed a flexible, narrow-bitwidth, and overflow-free value representation technique in the cache and main memory of general-purpose processors, which acts as a light-weight and data-aware compression method. It provides 1.23× speed up, in a 4-core system, compared to the state of the art cache compression techniques.
Second, we explored deviating from conventional Von Neumann architectures to minimize data movement by adding processing units closer to the memory cells, so that the data is processed in situ. Our in-situ accelerator, Fulcrum, outperforms a server-class GPU with three stacks of HBM2 and, on average, provides 70× speedup per memory stack and reduces the energy consumption by 96%.
Third, we optimized Fulcrum for extremely sparse operations. Our preliminary evaluations show that this optimized Fulcrum can outperform Gunrock, a GPU-based graph-processing framework, by 18×.
In my future projects, I will continue optimizing Fulcrum for more memory-intensive applications. Specifically, we are exploring the opportunities for accelerating database kernels (e.g., merge, sort, and filtering). Our preliminary results for LIP, a filtering kernel in database operations, show that our approach provides 3-9× speedup, depending on the processed query.
- Worthy Martin, committee chair (Department of Computer Science)
- Kevin Skadron, advisor (Department of Computer Science)
- Ashish Venkat (Department of Computer Science)
- Yangfeng Ji (Department of Computer Science)
- Mircea Stan (Department of Electrical & Computer Engineering)