Machine learning deployment increasingly relies on edge computing to meet the demands of latency-sensitive and privacy-critical environments, such as smart buildings and healthcare. However, deploying ML workloads on heterogeneous and resource-constrained IoT nodes remains challenging due to limited computational capacity, management complexity, and the absence of scalable orchestration mechanisms. In this study, we propose an architectural evolution from a cloud-coupled, serverless Function-as-a-Service (FaaS) platform to a Kubernetes-native, deviceless framework for deploying ML workloads at the edge. Our system leverages lightweight containers, function-based ML code injection, and dynamic scheduling driven by real-time CPU and memory usage metrics, as well as node availability. The proposed architecture introduces support for both inline and repository-based function deployment, enhancing portability and reducing operational overhead on constrained devices. It enables the injection of inference pipelines across diverse IoT nodes, supporting both real-time prediction and distributed learning scenarios. A smart room use case, Smart Healthy Intelligent Room, reveals real-time headcount estimation through environmental sensing (CO2, temperature), highlighting the feasibility and adaptability of our approach in realistic settings. The framework offers a deployable, low-latency solution for edge-based inference, enabling analytics with local execution, scalability, and minimal privacy risk.
Deviceless Edge-Centric Architectures for Serverless ML Workloads Deployment
Mirto, Fabio Orazio;Passaniti, Marco;Tricomi, Giuseppe;D'Agati, Luca;Merlino, Giovanni
2025-01-01
Abstract
Machine learning deployment increasingly relies on edge computing to meet the demands of latency-sensitive and privacy-critical environments, such as smart buildings and healthcare. However, deploying ML workloads on heterogeneous and resource-constrained IoT nodes remains challenging due to limited computational capacity, management complexity, and the absence of scalable orchestration mechanisms. In this study, we propose an architectural evolution from a cloud-coupled, serverless Function-as-a-Service (FaaS) platform to a Kubernetes-native, deviceless framework for deploying ML workloads at the edge. Our system leverages lightweight containers, function-based ML code injection, and dynamic scheduling driven by real-time CPU and memory usage metrics, as well as node availability. The proposed architecture introduces support for both inline and repository-based function deployment, enhancing portability and reducing operational overhead on constrained devices. It enables the injection of inference pipelines across diverse IoT nodes, supporting both real-time prediction and distributed learning scenarios. A smart room use case, Smart Healthy Intelligent Room, reveals real-time headcount estimation through environmental sensing (CO2, temperature), highlighting the feasibility and adaptability of our approach in realistic settings. The framework offers a deployable, low-latency solution for edge-based inference, enabling analytics with local execution, scalability, and minimal privacy risk.Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.


