Mobile computing has led to paradigm shifts in user behavior, application semantics, and device features. However, existing “hasty solutions” that borrow legacy designs directly from conventional operating systems cannot be applied seamlessly to mobile systems, particularly in conjunction with cloud computing. We conduct scientific and engineering research that would affect the future designs of mobile systems in the interconnection of smartphones, wearables, and the cloud. To give consideration to both academic value and practical use, we are devoted to not only original design concepts and system implementations but also theoretical derivation and analysis. Our research focuses include cloud-based energy saving services, user-centric mobile systems, self-powered intermittent IoT Systems, and embedded deep learning.

Lab Director

Academia Sinica Postdoctoral Scholar

Postdoctoral Scholar

Research Assistant

Research Assistant

Research Assistant

Administration Assistant

The personal computing landscape is undergoing a massive transition from stationary desktops to mobile devices. Emerging hardware technologies, such as OLED displays to replace traditional LCD displays and big.LITTLE processors that allows mobile devices to combine computing performance and energy efficiency, have been developed to reflect the paradigm shift. However, existing “hasty software solutions” that borrow legacy designs directly from conventional operating systems cannot be applied seamlessly to mobile systems. For example, resource management in Android, which is built upon Linux, often overlooks a key factor – the user’s perception. We advocate that mobile systems should move toward user-centric designs that consider human factors. To this end, we tried to understand the interplay among end users, mobile applications, and system resources. Understanding the implications has facilitated better system resource management.

In particular, we exploited application sensitivity, which reflects the user’s perception, to redesign the CPU scheduler so that user experience and energy efficiency can be improved simultaneously. We have incorporated human visual attention and acuity into the quality-retaining power-saving design for mobile OLED displays. Besides, we introduced the concept of alarm similarity into wakeup management for mobile systems in connected standby, with the objective of reducing energy waste while maintaining the quality of the user experience. We have also developed an augmented-sensing framework for mobile systems so that smartphones can easily mount sensors on wearables to create multi-device applications.

Intermittent-Aware Neural Architecture Search

The increasing paradigm shift towards intermittent computing has made it possible to intermittently execute deep neural network (DNN) inference on edge devices powered by ambient energy. Recently, neural architecture search (NAS) techniques have achieved great success in automatically finding DNNs with high accuracy and low inference latency on the deployed hardware. We make a key observation, where NAS attempts to improve inference latency by primarily maximizing data reuse, but the derived solutions when deployed on intermittently-powered systems may be inefficient, such that the inference may not satisfy an end-to-end latency requirement and, more seriously, they may be unsafe given an insufficient energy budget. This work proposes iNAS, which introduces intermittent execution behavior into NAS to find accurate network architectures with corresponding execution designs, which can safely and efficiently execute under intermittent power. An intermittent-aware execution design explorer is presented, which finds the right balance between data reuse and the costs related to intermittent inference, and incorporates a preservation design search space into NAS, while ensuring the power-cycle energy budget is not exceeded. To assess an intermittent execution design, an intermittent-aware abstract performance model is presented, which formulates the key costs related to progress preservation and recovery during intermittent inference. We implement iNAS on top of an existing NAS framework and evaluate their respective solutions found for various datasets, energy budgets and latency requirements, on a Texas Instruments device. Compared to those NAS solutions that can safely complete the inference, the iNAS solutions reduce the intermittent inference latency by 60% on average while achieving comparable accuracy, with an average 7% increase in search overhead.

Heterogeneity-aware Multicore Synchronization for Intermittent Systems

Intermittent systems enable batteryless devices to operate through energy harvesting by leveraging the complementary characteristics of volatile (VM) and non-volatile memory (NVM). Unfortunately, alternate and frequent accesses to heterogeneous memories for accumulative execution across power cycles can significantly hinder computation progress. The progress impediment is mainly due to more CPU time being wasted for slow NVM accesses than for fast VM accesses. This work explores how to leverage heterogeneous cores to mitigate the progress impediment caused by heterogeneous memories. In particular, a delegable and adaptive synchronization protocol is proposed to allow memory accesses to be delegated between cores and to dynamically adapt to diverse memory access latency. Moreover, our design guarantees task serializability across multiple cores and maintains data consistency despite frequent power failures. We integrated our design into FreeRTOS running on a Cypress device featuring heterogeneous dual cores and hybrid memories. Experimental results show that, compared to recent approaches that assume single-core intermittent systems, our design can improve computation progress at least 1.8x and even up to 33.9x by leveraging core heterogeneity

Everything Leaves Footprints: Hardware Accelerated Intermittent Deep Inference

Current peripheral execution approaches for intermittently-powered systems require full access to the internal hardware state for checkpointing or rely on application-level energy estimation for task partitioning to make correct forward progress. Both requirements present significant practical challenges for energy-harvesting, intelligent edge IoT devices, which perform hardware accelerated DNN inference. Sophisticated compute peripherals may have inaccessible internal state, and the complexity of DNN models makes it difficult for programmers to partition the application into suitably sized tasks that fit within an estimated energy budget. This paper presents the concept of inference footprinting for intermittent DNN inference, where accelerator progress is accumulatively preserved across power cycles. Our middleware stack, HAWAII, tracks and restores inference footprints efficiently and transparently to make inference forward progress, without requiring access to the accelerator internal state and application-level energy estimation. Evaluations were carried out on a Texas Instruments device, under varied energy budgets and network workloads. Compared to a variety of task-based intermittent approaches, HAWAII improves the inference throughput by 5.7% to 95.7%, particularly achieving higher performance on heavily accelerated DNNs

Enabling Failure-resilient Intermittent Systems Without Runtime Checkpointing

This project develops an intermittent operating system (OS) to preserve computation progress across power cycles without checkpointing for energy-harvesting systems. The intermittent OS is endowed with several capabilities as follows: 1) Run multiple tasks concurrently to improve computation progress. 2) Achieve consistency between data and computing progress. 3) Recover the system instantly from power failures. 4) Accumulatively preserve computation progress across power cycles to avoid progress stagnation. The intermittent OS is built upon FreeRTOS, a real-time operating system supporting many kinds of commercial microcontrollers, running on MSP-EXP430FR5994 LaunchPad, a Texas Instruments platform featuring 256KB FRAM and 8KB on-chip SRAM. We add a data manager and a recovery handler in FreeRTOS, so that the system runtime can cope with intermittence and exempts application developers from this responsibility. Due to the limitation of the memory size, the current implementation supports up to 10 user tasks and 16 data objects.

Accumulative Display Updating for Intermittent Systems

Electrophoretic displays (EPDS) are ideal for self-powered systems, but currently require an uninterrupted power supply to carry out the full display update cycle. Conventional self-powered EPDs will either re-execute the update process if it failed in the previous power cycle or incur significant HW/SW/programmer overheads and energy buffering latency to perform an uninterrupted display update, and hence is not suitable for intermittently-powered systems. Our Accumulative Display Updating (ADU) technique allows the EPD update to be completed across power cycles, without loosing update progress state due to power failure and removes the need for sufficient energy for the entire display update. We also propose a context-aware updating policy (denoted as ADU+F) to handle new data that becomes available to the system during an ongoing display update. Experimental results on a Texas Instruments device (MSP430FR5994) with an integrated 1.44 inch EPD by Pervasive Displays Inc., show that, compared to re-execution based atomic display updating (denoted as DU), our design can increase accurate forward progress of display updating by up to 31% and using the data freshness handling updating policy, further average response time reductions of up to 47.4% can be achieved.

A Mobile Cyber-Physical System for Accessible Location Exploration

Users roaming cellular signal coverage with their mobile devices essentially form a mobile cyber-physical system (CPS). By modeling cyber human mentality and physical signal coverage, as well as their interplay, user mobility can be leveraged to improve users' mobile experience with limited wireless bandwidth. Through a real-world case study, we observed that numerous "null zones" and "hot zones" exist in cellular signal coverage areas, where mobile devices cannot obtain sufficiently high data rates for delay-sensitive applications. Over one third of the locations in a crowded area could have weak signal coverage and low bandwidth shares, resulting in poor mobile connectivity experience. This paper considers the practicality of a mobile CPS called Oasis, which guides users to leave those zones and move to nearby locations with better mobile experience. To realize the system, we model and maximize a user's willingness to travel to another location, where the willingness involves the compound impact of the travel distance and the improved perceptual quality. We also develop a prototype system that creates a feedback control loop to allow self-adaptation to users' needs. To evaluate the efficacy, we conducted a series of experiments based on the real data collected in downtown Taipei, Taiwan. The results demonstrate that our mobile CPS can further reduce the average distance per unit of quality improvement achieved with OpenSignalMaps by about 80%, and motivate further research.

HW/SW Co-Design for Program Atomicity on Self-Powered Intermittent Systems

Self-powered intermittent systems featuring nonvolatile proces- sors (NVPs) allow for accumulative execution in unstable power environments. However, frequent power failures may cause incor- rect NVP execution results due to invalid data generated intermit- tently. This paper presents a HW/SW co-design, called HomeRun, to guarantee atomicity by ensuring that an uninterruptible pro- gram section can be run through at one execution. We design a HW module to ensure that a power pulse is sufficient for an atomic section, and develop a SW mechanism for programmers to protect atomic sections. The proposed design is validated through the development of a prototype pattern locking system. Experimental results demonstrate that the proposed design can completely guar- antee atomicity and significantly improve the energy utilization of self-powered intermittent systems.

Learning Adaptive Hidden Layers for Mobile Gesture Recognition

This paper addresses two obstacles hindering advances in accurate gesture recognition on mobile devices. First, gesture recognition performance is highly dependant on feature selection, but optimal features typically vary from gesture to gesture. Second, diverse user behaviors and mobile environments result in extremely large intra-class variations. We tackle these issues by introducing a new network layer, called an adaptive hidden layer (AHL), to generalize a hidden layer in deep neural networks and dynamically generate an activation map conditioned on the input. To this end, an AHL is composed of multiple neuron groups and an extra selector. The former compiles multi-modal features captured by mobile sensors, while the latter adaptively picks a plausible group for each input sample. The AHL is end-to-end trainable and can generalize an arbitrary subset of hidden layers. Through a series of AHLs, the great expressive power from exponentially many forward paths allows us to choose proper multi-modal features in a sample-specific fashion and resolve the problems caused by the unfavorable variations in mobile gesture recognition.The proposed approach is evaluated on a benchmark for gesture recognition and a newly collected dataset. Superior performance demonstrates its effectiveness.

Dynamic OLED Power Shifting Based on Visual Acuity for Interactive Mobile Applications

OLED power management on mobile devices is very challenging due to the dynamic nature of human-screen interaction. This paper presents the design, algorithms, and implementation of a lightweight mobile app called ShiftMask, which allows the user to dynamically shift OLED power to the portion of interest, while dimming the remainder of the screen based on visual acuity. To adapt to the user’s focus of attention, we propose efficient algorithms that consider visual fixation in static scenes, as well as changes in focus and screen scrolling. The results of experiments conducted on a commercial smartphone with popular interactive apps demonstrate that ShiftMask can achieve substantial energy savings, while preserving acceptable readability.

A Win-Win Camera: Quality-Enhanced Power-Saving Images on Mobile OLED Displays.

Mobile systems will increasingly feature emerging OLED displays, whose power consumption is highly dependent on the image content. Existing OLED power-saving techniques change users’ visual experience or degrade images’ visual quality in exchange for power reduction. This paper posits the possibility of a win-win scheme that enhances image quality and reduces power consumption simultaneously. We define metrics to assess the profit and the cost for potential image enhancement and power reduction. Then, we propose fundamental algorithms that ensure the transformation of images into their quality- enhanced power-saving versions. Finally, the proposed scheme is realized as a practical camera application on mobile devices. The results of experiments conducted on a commercial tablet with a popular image database are very encouraging and provide valuable insights for future research and practices.

A Semantics-Aware Design for Mounting Remote Sensors on Mobile Systems.

Application paradigms will increasingly exceed a mobile device’s physical boundaries. This paper presents a system solution for a mobile device to mount remote sensors on other devices. Our design is generic to mobile senor stacks, thus supporting unmodified apps and commodity sensors. Furthermore, it uses an asynchronous access model to facilitate semantics passing and data reporting in between. Such semantic information allows the development of an energy-efficient reporting policy for remote sensing applications. The results of experiments conducted on commercial Android smartphones with popular apps demonstrate that our design is very efficient in terms of energy consumption and completion time.

Similarity-Based Wakeup Management for Mobile Systems in Connected Standby

Resident applications, which autonomously awaken mobile devices, can gradually and imperceptibly drain device batteries. This paper introduces the concept of alarm similarity into wakeup management for mobile systems in connected standby. First, we define hardware similarity to reflect the degree of energy savings and time similarity to reflect the impact on user experience. We then propose a policy that aligns alarms based on their similarity to save standby energy while maintaining the quality of the user experience. Finally, we integrate our design into Android and conduct extensive experiments on a commercial smartphone running popular mobile apps. The results demonstrate that our design can further extend the standby time achieved with Android’s native policy by up to one-third.

User-Centric Energy-Efficient Scheduling on Multi-Core Mobile Devices

Mobile devices will provide improved computing resources to sustain progressively more complicated applications. However, the design concept of fair scheduling and governing borrowed from legacy operating systems cannot be applied seamlessly in mobile systems, thereby degrading user experience or reducing energy efficiency. In this paper, we posit that mobile applications should be treated unfairly. To this end, we exploit the concept of application sensitivity and devise a user-centric scheduler and governor that allocate computing resources to applications according to their sensitivity. Furthermore, we integrate our design into the Android operating system. The results of extensive experiments on a commercial smartphone with real-world mobile apps demonstrate that the proposed design can achieve significant energy efficiency gains while improving the quality of user experience.

CURA: A Framework for Quality-retaining Power Saving on Mobile OLED Displays.

Organic light-emitting diode (OLED) technology is regarded as a promising alternative to mobile displays. In this paper, we introduce the design, algorithm, and implementation of a novel framework called CURA for quality-retaining power saving on mobile OLED displays. First, we link human visual attention to OLED power saving and model the OLED image scaling optimization problem. The objective is to minimize the power required to display an image without adversely impacting the user’s visual experience. Then, we present the algorithm used to solve the fundamental problem, and prove its optimality even without an accurate power model. Finally, based on the framework, we implement two practical applications on a commercial OLED mobile tablet. The results of experiments conducted on the tablet with real images demonstrate that CURA can reduce significant OLED power consumption while retaining the visual quality of images.

Extend Your Journey: Considering Signal Strength and Fluctuation in Location-based Applications

Reducing the communication energy is essential to facilitate the growth of emerging mobile applications. In this paper, we introduce signal strength into location-based applications to reduce the energy consumption of mobile devices for data reception. First, we model the problem of data fetch scheduling, with the objective of minimizing the energy required to fetch location-based information without impacting the application’s semantics adversely. To solve the fundamental problem, we propose a dynamicprogramming algorithm and prove its optimality in terms of energy savings. Then, we perform postoptimal analysis to explore the tolerance of the algorithm to signal strength fluctuations. Finally, based on the algorithm, we consider implementation issues. We have also developed a virtual tour system integrated with existing web applications to validate the practicability of the proposed concept. The results of experiments conducted based on real-world case studies are very encouraging and demonstrate the applicability of the proposed algorithm towards signal strength fluctuations.

Dynamic Backlight Scaling Optimization: A Cloud-Based Energy-Saving Service for Mobile Streaming Applications

With the increasing variety of mobile applications, reducing the energy consumption of mobile devices is a major challenge in sustaining multimedia streaming applications. This paper explores how to minimize the energy consumption of the backlight when displaying a video stream without adversely impacting the user’s visual experience. First, we model the problem as a dynamic backlight scaling optimization problem. Then, we propose algorithms to solve the fundamental problem and prove the optimality in terms of energy savings. Finally, based on the algorithms, we present a cloud-based energy-saving service. We have also developed a prototype implementation integrated with existing video streaming applications to validate the practicability of the approach. The results of experiments conducted to evaluate the ef¿cacy of the proposed approach are very encouraging and show energy savings of 15-49% on off-the-shelf mobile devices.

TAGS:Emclab 

<<< Thank you for your visit >>>

Hot Websites