Context. The Internet of Things and its myriads of devices constantly producing data at the Edge of the Internet, has become a huge source of data to be collected, processed and analysed in a near real time fashion. In particular, AI applications are natural consumers of these data [1]. With the emergence of Edge computing [2], moving the learning process behind these applications closer to the Edge, where data are produced, is appealing, in turn raising mutliple challenges.
An appealing approach is Federated Learning (FL) [3], where each device learns based on locally produced data, creates a model and then sends it to a centralized server in charge of merging the locally obtained models into a single global model. This model is then sent back to devices so they can have the global model locally and restart learning with a model built based on data collected everywhere. Federated Learning provides benefits compared to a purely centralized learning approach by not having local data moved to the server, and thus enabling by-design privacy.
Yet, FL still relies over a centralized server as a coordinator and merger for the models. Without it, no aggregation is possible. Moreover, it suffers from the traditional limitations of centralized approaches: limited scalability and resilience. While it has been shown that adapting learning for more decentralized platforms is promising [4], the problem of efficiently learning over decentralized platform is still a widely open problem.
Thesis description. Different from the FL approach where the centralized component is vital for the mere functioning of the system, we plan to adopt the recently proposed alternative where devices learn locally and opportunistically exchange their model so as to build a more complete model. Finding the right trade-off between accuracy of the models within an area (composed of one or several interrelated nodes) and the network traffic generated will constitute the main driver for the solution.
Once deployed, the runtime system must face the changing conditions of the platform: disconnection of devices, bottlenecks in the network. More generally, adaptation is needed to adjust the parameters, hyperparameters and methods use to combine local models, so as to ensure the best tradeoff between accuracy of the model and resource usage.
A related problem is the ease with which a programmer can deploy, monitor and adapt at runtime the platform and its parameters. This relates to the notion of Programmability. Developing abstractions for the easy deployment and control of AI programs over decentralized platforms is an important step towards a larger adoption of the approach. This will constitute the thid aspect to get studied during the thesis.
The work will be experimentally validated over a real large scale platform such as the Grid'5000 platform [5]. To ease such a deployment, the E2Clab framework will be used. E2Clab [6,7] is a framework that implements a rigorous methodology that provides guidelines to move from real-life application workflows to representative settings of the physical infrastructure underlying this application in order to accurately reproduce its relevant behaviors and therefore understand end-to-end performance.
Qualifications:
- Good communication and writing skills
- Strong programming / scripting skills
- Knowledge and/or experience in one or more of the following areas: distributed systems, adaptive systems, cloud, edge, decentralized learning
[1] R. Singh and S. S. Gill, “Edge AI: a survey”, Internet of Things and Cyber-Physical Systems 2023, Vol. 3, Page 71-92, doi:10.1016/j.iotcps.2023.02.004
[2] Weisong Shi et al., "Edge computing: Vision and challenges", in: IEEE internet of things journal 3.5 (2016), pp. 637–646.
[3] H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. 2016. Communication-Efficient Learning of Deep Networks from Decentralized Data. https://doi.org/10.48550/arXiv.1602.05629
[4] Hegedűs, I., Danner, G., & Jelasity, M. Gossip learning as a decentralized alternative to federated learning. In Distributed Applications and Interoperable Systems: 19th IFIP WG 6.1 International Conference, DAIS 2019, Kongens Lyngby, Denmark, June 17–21, 2019, Proceedings 19 (pp. 74-90). Springer International Publishing.
[5] Daniel Balouek et al. Adding Virtualization Capabilities to the Grid’5000 Testbed, in: Cloud Computing and Services Science, ed. by Ivan I. Ivanov et al., vol. 367, Communications in Computer and Information Science, Springer International Publishing, 2013, pp. 3–20, isbn: 978-3-319-04518-4.
[6] Daniel Rosendo, Pedro Silva, et al. E2Clab: Exploring the Computing Continuum through Repeatable, Replicable and Reproduc Edge-to-Cloud Experiments. Cluster 2020 - IEEE International Conference on Cluster Computing, Sep 2020, Kobe, Japan.
- [7] The E2Clab project: https://team.inria.fr/kerdata/e2clab/ (https://team.inria.fr/kerdata/e2clab/).