EAGER: Towards Low-Power Low-Latency Heterogeneous Memory Access

Recent strides in multicore technologies have enabled embedded systems to continue to scale to better performance, but the long latencies and high power consumption of memory access have been an overwhelming bottleneck for the development of these systems. To overcome these memory wall and energy wall issues, various memory techniques have been developed, including non-volatile memories. However, as today’s embedded systems conduct numerous computation-intensive applications, requiring extensive memory access, the memory performance continues to lag behind. This proposal aims at rethinking the fundamental architecture of memory systems and revolutionizing the paradigm of memory architecture, by exploring heterogeneous memory architecture and novel data allocation algorithms. The basic rationale of our approach is to reduce total memory access costs (either energy consumption or latency), with the aid of scratch-pad memory (SPM). SPM is a software controlled on-chip memory, which is envisioned as a potential technique to replace cache memory, due to its advantages in size, power, and real-time predictability. Despite the obvious benefits that heterogeneous memory offers, several challenges remain: (1) Which types of memory are applicable for the heterogeneous architecture? (2) What kinds of algorithms are suitable for the proposed architecture? Can these algorithms achieve optimal solutions for data allocation? (3) Should these proposed algorithms be integrated into hardware or software? Can they satisfy the limited space of embedded systems? (4) Is the approach scalable to more types of memory or more number of memory technologies? These open questions make this research a high-risk but high-reward proposition. The success of the proposed research has potential to revolutionize the design and architecture for the next generation of high-performance and low-power computing systems. The research objective of this proposal is to develop an innovative heterogeneous memory hierarchy and corresponding memory management strategies to reduce memory access latency and energy consumption. The outcomes of this research include: 1) A well-designed on-chip heterogeneous memory hierarchy, which integrates Static Random Access Memory (SRAM), Magnetic Random Access Memory (MRAM), and Zero-capacitor RAM (Z-RAM) technologies. 2) A set of novel algorithms which can achieve optimal allocation or efficient space utilization. 3) A simulation toolkit that integrates of commonly used simulations, such as MiBench, PARSEC, and MediaBench. Intellectual Merit: The PI will investigate fundamental heterogeneous memory architecture and memory management strategies for embedded systems. A novel heterogeneous memory hierarchy which integrates multiple types of memory technologies will be developed. Based on the developed memory architecture, the PI will study an array of on-chip memory management schemes to optimize data allocation for embedded multicore systems with stringent power, real-time, and memory space constraints. A novel multi-dimensional dynamic programming algorithm will be developed to achieve optimal data allocation and efficiently reduce memory access latency and power consumption. To further improve memory space utilization, the PI will design a genetic algorithm to allocate data to different memory modules. The whole data allocation mechanism will be verified by using real embedded multicore systems, such as a commercial “off the shelf” embedded system. A simulation toolkit will be built to validation our approached through implementing the commonly used benchmarks such as MiBench. The technological innovations introduced in this proposal have the potential to provide a novel optimization approach towards low power and lower latency memory access for embedded systems. The PI is well-qualified to implement this project and has sufficient access to necessary resources. Broader Impacts: The impacts of this project on the performance enhancement of embedded multicore systems will be immediate and prominent. 1) Our optimal data allocation strategies can be seamlessly integrated into existing embedded system and will substantially boost memory access performance. 2) The PI will design a new course and revamp two courses based on this project, which will accelerate research infrastructure development, as well as train the next generation of well-qualified students in embedded system design. 3) Collaborating with the University of Kentucky, Minority and Woman in Engineering Office, the PI will attract additional underrepresented students and younger generations into the computer architecture field. 4) The embedded system community will also benefit from this project by the dissemination of the results of this project in leading journals, conferences, lectures, and tutorials. The success of this research will not only fundamentally reform the design and development of the next generation multicore architecture for embedded systems, high performance computing systems, and cloud systems, but will also offer unprecedented opportunities for further research and industrial applications. Keywords: Embedded system, multicore, heterogeneous memory, SPM, data allocation.