For people with limited mobility, navigating in cluttered indoor environment is challenging. In this work, we present a real-time localization system for closed-loop control of assistive furniture. The system enables intelligent coordination of mobile furniture pieces by assessing the state of an arbitrary living scenario so that the estimation can be used as a real-time feedback for reconfiguration. We propose a vision-based approach that accurately tracks the position and orientation of furniture modules, enabling them to autonomously rearrange and adapt to the user’s needs. The method is evaluated on real-world scenarios demonstrating reliable real-time performance for assistive living applications.