Presenter: Matteo Boffa
Thursday, March 27th, 2025, 5:00 PM
Location: Sala Grande Covivio, Corso Ferrucci 112
ABSTRACT
Recent research has explored the constrained generation capabilities of Large Language Models (LLMs) when explicitly prompted by few task-specific requirements. In contrast, we introduce Large-Scale Constraint Generation (LSCG), a new problem that evaluates whether LLMs can parse a large, fine-grained, generic list of constraints.
To examine the LLMs’ ability to handle an increasing number constraints, we create a practical instance of LSCG, called Words Checker. We evaluate the impact of model characteristics (e.g., size, family) and steering techniques (e.g., Simple Prompt, Chain of Thought, Best of N) on performance. In addition, we propose FoCusNet, a small and dedicated model that parses the original list of constraints into a smaller subset to help the LLM focus on relevant constraints. Experiments reveal that existing solutions suffer a significant performance drop as the number of constraints increases, with FoCusNet showing at least an 8-13% accuracy boost.
BIOGRAPHY
Matteo is a PhD student at Politecnico di Torino (PoliTO), Italy, and member of the SmartData@Polito research center. He obtained the M.Sc. in ICT for Smart Societies at Politecnico di Torino in 2021. In his research, he focuses on the applications of Artificial Intelligence for cybersecurity.