In this work, we study the long-run dynamics of learning agents in Cournot game. Cournot game is the underlying market model for many demand response programs in electricity markets, where providers bid their available quantity, the service price is set by the total supply and each provider gets paid accordingly.
My research interests are in area of cyber-physical and energy systems, from the perspective of machine learning, optimization, and control. Here is a list of selective research projects that I have worked on. Click on the title link for more details!
Deep neural networks have proven to be successful in many identification tasks, however, from the model-based control perspective, these networks are difficult to work with because they are typically non-linear and non-convex. In this work, we bridge the gap between model accuracy and control tractability faced by neural networks, by explicitly constructing input convex neural networks (ICNN). It leads to significant energy savings for building HVAC management.
Energy storage offsets renewable fluctuation and is the key to a low-carbon future, but managing these assets at the system level is challenging. A central question is to understand the degradation of storage and reflect this cost in the operations. In this work, we prove that the cycle-based electrochemical degradation model is convex. Based on this cost model, we further develop an optimal online control algorithm for energy storage in a general pay-for-performance market via a novel change of basis.
Existing works on energy storage planning focus on using batteries for a single application. Our results suggest that battery can achieve much larger economic benefits than previously thought if they jointly provide multiple services.
An issue that is faced by many state-of-the-art Reinforcement Learning (RL) algorithms is the sensitivity (risk) of the learned policy to model uncertainties. In this project, we proposed a data-driven framework for incorporating robustness to the parametric uncertainties into continuous control RL algorithms.