Book · 2026
A Fuzzy Control Architecture for Small Language Models with Adaptive Expert Ensembles
A. F. Shaikhulova
Zenodo (CERN European Organization for Nuclear Research)
Abstract
This study presents a fuzzy control architecture for Small Language Models (SLMs) with adaptive expert ensembles. The proposed system addresses the limitations of static multi-domain SLM deployments through three interconnected mechanisms: a differentiable fuzzy knowledge graph for continuous domain membership modeling, a bus-based communication system for asynchronous expert interaction, and a reinforcement learning framework with Proximal Policy Optimization (PPO) for adaptive expert behavior. Introduction. The deployment of Small Language Models (SLMs) in specialized domains is hindered by a fundamental trade-off: narrow specialization limits cross-domain reasoning, while rigid architectures fail to adapt to evolving knowledge boundaries. This necessitates a system that maintains deep domain expertise while enabling emergent interdisciplinary collaboration.This study introduces a fuzzy control architecture for SLMs with adaptive expert ensembles that addresses the limitations of static, hard-partitioned multi-domain systems. Methods. The proposed architecture comprises: (1) a differentiable fuzzy knowledge graph modeling continuous domain memberships using trainable embeddings; (2) a bus-based communication system with four asynchronous buses (Control, Input, Output, Mechanism) for expert information exchange; and (3) a reinforcement learning framework combining Proximal Policy Optimization (PPO) for experts with a meta-controller for autonomous population management. Experiments were conducted on AG News dataset (4 domains, 4,500 training samples) using TinyLlama-1.1B-Chat as base model. Results. The system achieved 82.3% test accuracy with 6 autonomously created experts showing strong specialization (0.919 average). The meta-controller successfully evolved the expert population from 4 to 6 specialists. Training dynamics show accuracy improvement from 66.8% to 84.1% with specialization increasing from 0.888 to 0.919. While the system slightly underperforms compared to single-model baselines (91.3%), it demonstrates key advantages: autonomous expert creation, high specialization without catastrophic forgetting, and adaptive fuzzy routing. Conclusion. Integrating fuzzy logic, asynchronous communication, and reinforcement learning provides a scalable foundation for self-organizing SLM ensembles. The architecture maintains specialized expertise while enabling adaptive cross-domain reasoning. Future work will focus on closing the accuracy gap while preserving adaptive capabilities.