Research

Overview

My research at National Yang Ming Chiao Tung University (NYCU) focuses on Reinforcement Learning and Restless Multi-Armed Bandits (RMABs), particularly in dynamic and non-stationary environments.

Under the supervision of Prof. Stefano Rini (Institute of Communications Engineering) and Prof. Yu-Chih Huang (Department of Electrical and Computer Engineering), I proposed the Piecewise Stationary Restless Multi-Armed Bandit (PSRMAB) framework — an extension of RMABs that captures environment changes by segmenting time into stationary periods.

The study introduces a low-complexity adaptive algorithm that combines change detection with a restless bandit framework, enabling efficient adaptation without direct state observation.

This work has evolved through multiple stages — a poster presentation, a master's thesis, and a spotlight paper at AISTATS 2026 — representing the same line of research refined over time through collaboration with PhD students in our group.

Research Focus

Reinforcement Learning Multi-armed Bandit Markov Decision Processes Restless Bandits Change Detection Mathematical Analysis Resource Allocation

Piecewise Stationary Restless Multi-Armed Bandits (PSRMAB)

Restless Bandit Change Detection Resource Allocation AISTATS 2026 ICS 2024
Advisor: Prof. Yu-Chih Huang ; Collaborators: Kuan-Ta Li, Prof. Yu-Pin Hsu, Prof. Ping-Chun Hsieh
Abstract: Proposed a novel Piecewise Stationary Restless Multi-Armed Bandit (PSRMAB) model and developed a low-complexity adaptive algorithm for resource allocation in non-stationary communication systems. The method addresses change detection under dynamic environments while maintaining computational efficiency and theoretical performance guarantees. This line of work was later extended through collaboration with PhD student Kuan-Ta Li into a modular framework, culminating in a spotlight publication at AISTATS 2026.
Publication: Kuan-Ta Li, Chia-Chun Lin*, Ping-Chun Hsieh, and Yu-Chih Huang, “A Modularized Framework for Piecewise-Stationary Restless Bandits,” AISTATS 2026 — Spotlight Presentation (top ~3% of submissions; acceptance rate 28%)
Poster: Chia-Chun Lin*, Kuan-Ta Li, Ping-Chun Hsieh, Yu-Pin Hsu, and Yu-Chih Huang, “Design and Analysis of Non-Stationary Restless Multi-Armed Bandits,” ICS 2024.