Vous êtes ici

Learning to survive: Dynamic optimization for autonomous and heterogeneous IoT networks

Equipe et encadrants
Département / Equipe: 
Site Web Equipe: 
Directeur de thèse
Olivier Berder
Co-directeur(s), co-encadrant(s)
Matthieu Gautier
NomAdresse e-mailTéléphone
Olivier Berder
Sujet de thèse

To overcome battery-limited problems IoT (Internet of Things) nodes may rely on available environmental energy sources such as light, wind or heat. Due to the variety of IoT applications, these nodes form a very heterogeneous network, since they embed different sensors and radios, and therefore different computation abilities. To guarantee its sustainability, each node has to organize its different tasks (sense, process, transmit, receive, relay…) with respect to its environment, especially the energy that can be harvested. As a node has no prior knowledge of the energy cost of these controlled or event-based tasks, the latter has to be learnt in real time. The role of the power manager is then to prioritize these tasks and allocate an energy budget to each of them, such that the consumed energy is equal to the harvested energy over a long period, which leads to Energy Neutral Operations (ENO).
In previous works, we designed generic power managers able to deal with the different energy sources [le13pimrc]. To deal with the uncertainty linked to the energy sources, we recently proposed a new power manager based on fuzzy control, whose outstanding performances were evaluated through network simulations (Castalia/OMNeT++) and experiments on the PowWow WSN platform developed by the GRANIT team [granit13powwow].
Another way to design the power manager is to base on experience, especially thanks to reinforcement learning, which differs from standard supervised learning in that correct input/output pairs are never presented, nor sub-optimal actions explicitly corrected. There is a focus on on-line performance, which involves finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge), which particularly suits to energy harvesting mechanisms.
The node will thus be able to track the harvested energy but also the energy costs to execute its different tasks (that can also be time-varying).

The first phase of this work will be dedicated to the bibliography study on Reinforcement Learning principles and its applicability to energy harvesting nodes [xiao15icc]. Then, the second phase will be the design of a RL-based power manager, that will adapt the energy budget to the residual energy in the battery. A particular attention will be given to the approximation functions that represent the value functions, exploring different forms from weighted linear functions to outputs of neural networks. These different possibilities will be evaluated by simulations, and compared to state-of-the art power managers. Finally, the main contribution of this work will be to extend the proposed approach at a network level by learning the costs of the different tasks and by collaboratively deciding the best policies. The best strategy will be implemented on a real wireless sensor platform. A network composed of heterogeneous nodes will be deployed to confirm this superiority in terms of energy management and quality of service .


[akyildiz02jcn] I. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci. Wireless sensor networks: a survey. Journal on Computer Networks, 38(4):393–422, 2002.
[alam11jes] M.-M. Alam, O. Berder, D. Ménard et O. Sentieys, A Hybrid Model for Accurate Energy Analysis for WSN, EURASIP Journal on Embedded Systems, 2011.
[le13pimrc] T.N. Le, A. Pégatoquet, O. Sentieys, O. Berder, C. Belleudy. Duty-Cycle Power Manager for Thermal-Powered Wireless Sensor Networks, in 24th IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), Londres, United Kingdom, September 2013, pp. 1645-1649.
[granit13powwow] http://powwow.gforge.inria.fr/
[le13enssys] T.N. Le, M. Magno, A. Pégatoquet, O. Berder, O. Sentieys, E. popovici. Ultra Low Power Asynchronous MAC Protocol using Wake-Up Radio for Energy Neutral Wireless Sensor Networks, in 1st International Workshop on Energy-Neutral Sensing Systems (ENSsys), Rome, Italy, November 2013
[alam12ieeejetcas] M.-M. Alam, O. Berder, D. Ménard et O. Sentieys, TAD-MAC: Traffic-Aware Dynamic MAC Protocol for Wireless Body Area Sensor Networks, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2(1) :109-119, mars 2012.
[xiao15icc] Y. Xiao, Z. Han, D. Niyato, C. Yuen, Bayesian Reinforcement Learning for Energy Harvesting Communication Systems with Uncertainty, IEEE International Conference on Communications (ICC15), May 2015
[hsu12ieeetetc] R. C. Hsu, C. T. Liu, and H. L. Wang, “A Reinforcement Learning-Based ToD Provisioning Dynamic Power Management for Sustainable Operation of Energy Harvesting Wireless Sensor Node,” IEEE Transactions
on Emerging Topics in Computing, vol. 2, no. 2, pp. 181–191, June 2014.
[park16ieeeaccess] T. Park, N. Abuzainab and W. Saad, "Learning How to Communicate in the Internet of Things: Finite Resources and Heterogeneity," in IEEE Access, vol. 4, no. , pp. 7063-7073, 2016
[aitaoudia16icc] F Aït Aoudia, M Gautier, O Berder, Fuzzy Power Management for Energy Harvesting Wireless Sensor Nodes, IEEE International Conference on Communications (ICC16), May 2016.
[aitaoudia16js] F. Aït Aoudia, M Gautier, O Berder, OPWUM: Opportunistic MAC Protocol Leveraging Wake-Up Receivers in WSNs, Journal of sensors, 2016

Début des travaux: 
Mots clés: 
Reinforcement learning, wireless sensor networks, energy harvesting, MAC protocols, power management
IRISA - Campus universitaire de Beaulieu, Rennes