NetConfEval: Can LLMs Facilitate Network Configuration?

Abstract

This paper explores opportunities to utilize Large Language Models (LLMs) to make network configuration human-friendly, simplifying the configuration of network devices & development of routing algorithms and minimizing errors. We design a set of benchmarks (NetConfEval) to examine the effectiveness of different models in facilitating and automating network configuration. More specifically, we focus on the scenarios where LLMs translate high-level policies, requirements, and descriptions (i.e., specified in natural language) into low-level network configurations & Python code. NetConfEval considers four tasks that could potentially facilitate network configuration, such as (𝑖) generating high-level requirements into a formal specification format, (𝑖𝑖) generating API/function calls from high-level requirements, (𝑖𝑖𝑖) developing routing algorithms based on high-level descriptions, and (𝑖𝑣) generating low-level configuration for existing and new protocols based on input documentation. Learning from the results of our study, we propose a set of principles to design LLM-based systems to configure networks. Finally, we present two GPT-4-based prototypes to (𝑖) automatically configure P4-enabled devices from a set of high-level requirements and (𝑖𝑖) integrate LLMs into existing network synthesizers.

Authors

CHANGJIE WANG, KTH Royal Institute of Technology, Sweden 
MARIANO SCAZZARIELLO, KTH Royal Institute of Technology, Sweden
ALIREZA FARSHIN, NVIDIA, Sweden
SIMONE FERLIN, Red Hat, Sweden
DEJAN KOSTIĆ, KTH Royal Institute of Technology, Sweden
MARCO CHIESA, KTH Royal Institute of Technology, Sweden

Citation

Changjie Wang, Mariano Scazzariello, Alireza Farshin, Simone Ferlin, Dejan Kostić, and Marco Chiesa. 2024. NetConfEval: Can LLMs Facilitate Network Configuration?. Proc. ACM Netw. 2, CoNEXT2, Article 7 (June 2024), 25 pages. https://doi.org/10.1145/3656296