Kernel Temporal Differences for EEG-based Reinforcement Learning Brain Machine Interfaces

Bhoj Raj Thapa, Daniel Restrepo Tangarife, Jihye Bae

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Kernel temporal differences (KTD) (lambda) algorithm integrated in Q-learning (Q-KTD) has shown its applicability and feasibility for reinforcement learning brain machine interfaces (RLBMIs). RLBMI with its unique learning strategy based on trial-error allows continuous learning and adaptation in BMIs. Q-KTD has shown good performance in both open and closed-loop experiments for finding a proper mapping from neural intention to control commands of an external device. However, previous studies have been limited to intracortical BMIs where monkey's firing rates from primary motor cortex were used as inputs to the neural decoder. This study provides the first attempt to investigate Q-KTD algorithm's applicability in EEG-based RLBMIs. Two different publicly available EEG data sets are considered, we refer to them as Data set A and Data set B. EEG motor imagery tasks are integrated in a single step center-out reaching task, and we observe the open-loop RLBMI experiments reach 100% average success rates after sufficient learning experience. Data set A converges after approximately 20 epochs for raw features and Data set mathrm{B} shows convergence after approximately 40 epochs for both raw and Fourier transform features. Although there still exist challenges to overcome in EEG-based RLBMI using Q-KTD, including increasing the learning speed, and optimization of a continuously growing number of kernel units, the results encourage further investigation of Q-KTD in closed-loop RLBMIs using EEG. Clinical Relevance-This study supports feasibility of noninvasive EEG-based RLBMI implementations and addresses benefits and challenges of RLBMI using EEG.

Original languageEnglish
Title of host publication44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022
Pages3327-3333
Number of pages7
ISBN (Electronic)9781728127828
DOIs
StatePublished - 2022
Event44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022 - Glasgow, United Kingdom
Duration: Jul 11 2022Jul 15 2022

Publication series

NameProceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
Volume2022-July
ISSN (Print)1557-170X

Conference

Conference44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022
Country/TerritoryUnited Kingdom
CityGlasgow
Period7/11/227/15/22

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

ASJC Scopus subject areas

  • Signal Processing
  • Biomedical Engineering
  • Computer Vision and Pattern Recognition
  • Health Informatics

Fingerprint

Dive into the research topics of 'Kernel Temporal Differences for EEG-based Reinforcement Learning Brain Machine Interfaces'. Together they form a unique fingerprint.

Cite this