Correntropy kernel temporal differences for reinforcement learning brain machine interfaces

Jihye Bae; Luis G.Sanchez Giraldo; Jose C. Principe; Joseph T. Francis

doi:10.1109/IJCNN.2014.6889958

Correntropy kernel temporal differences for reinforcement learning brain machine interfaces

Jihye Bae, Luis G.Sanchez Giraldo, Jose C. Principe, Joseph T. Francis

Electrical and Computer Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

This paper introduces a novel temporal difference algorithm to estimate a value function in reinforcement learning. This is a kernel adaptive system using a robust cost function called correntropy. We call this system correntropy kernel temporal differences (CKTD). This algorithm is integrated with Q-learning to find a proper policy (Q-learning via correntropy kernel temporal differences). The proposed method was tested with a synthetic problem, and its robustness under a changing policy was quantified. The same algorithm was applied to the decoding of a monkey's neural states in a reinforcement learning brain machine interface (RLBMI) in a center-out reaching task. The results showed the potential advantage of the proposed algorithm in the RLBMI framework.

Original language	English
Title of host publication	Proceedings of the International Joint Conference on Neural Networks
Pages	2713-2717
Number of pages	5
ISBN (Electronic)	9781479914845
DOIs	https://doi.org/10.1109/IJCNN.2014.6889958
State	Published - Sep 3 2014
Event	2014 International Joint Conference on Neural Networks, IJCNN 2014 - Beijing, China Duration: Jul 6 2014 → Jul 11 2014

Publication series

Name	Proceedings of the International Joint Conference on Neural Networks

Conference

Conference	2014 International Joint Conference on Neural Networks, IJCNN 2014
Country/Territory	China
City	Beijing
Period	7/6/14 → 7/11/14

Bibliographical note

Publisher Copyright:
© 2014 IEEE.

ASJC Scopus subject areas

Software
Artificial Intelligence

Access to Document

10.1109/IJCNN.2014.6889958

Cite this

Bae, J., Giraldo, L. G. S., Principe, J. C., & Francis, J. T. (2014). Correntropy kernel temporal differences for reinforcement learning brain machine interfaces. In Proceedings of the International Joint Conference on Neural Networks (pp. 2713-2717). [6889958] (Proceedings of the International Joint Conference on Neural Networks). https://doi.org/10.1109/IJCNN.2014.6889958

@inproceedings{9ea9196386fb4c38a1e4a2a3778920e0,

title = "Correntropy kernel temporal differences for reinforcement learning brain machine interfaces",

abstract = "This paper introduces a novel temporal difference algorithm to estimate a value function in reinforcement learning. This is a kernel adaptive system using a robust cost function called correntropy. We call this system correntropy kernel temporal differences (CKTD). This algorithm is integrated with Q-learning to find a proper policy (Q-learning via correntropy kernel temporal differences). The proposed method was tested with a synthetic problem, and its robustness under a changing policy was quantified. The same algorithm was applied to the decoding of a monkey's neural states in a reinforcement learning brain machine interface (RLBMI) in a center-out reaching task. The results showed the potential advantage of the proposed algorithm in the RLBMI framework.",

author = "Jihye Bae and Giraldo, {Luis G.Sanchez} and Principe, {Jose C.} and Francis, {Joseph T.}",

note = "Publisher Copyright: {\textcopyright} 2014 IEEE.; null ; Conference date: 06-07-2014 Through 11-07-2014",

year = "2014",

month = sep,

day = "3",

doi = "10.1109/IJCNN.2014.6889958",

language = "English",

series = "Proceedings of the International Joint Conference on Neural Networks",

pages = "2713--2717",

booktitle = "Proceedings of the International Joint Conference on Neural Networks",

}

Bae, J , Giraldo, LGS, Principe, JC & Francis, JT 2014, Correntropy kernel temporal differences for reinforcement learning brain machine interfaces. in Proceedings of the International Joint Conference on Neural Networks., 6889958, Proceedings of the International Joint Conference on Neural Networks, pp. 2713-2717, 2014 International Joint Conference on Neural Networks, IJCNN 2014, Beijing, China, 7/6/14. https://doi.org/10.1109/IJCNN.2014.6889958

Correntropy kernel temporal differences for reinforcement learning brain machine interfaces. / Bae, Jihye ; Giraldo, Luis G.Sanchez; Principe, Jose C. et al.

Proceedings of the International Joint Conference on Neural Networks. 2014. p. 2713-2717 6889958 (Proceedings of the International Joint Conference on Neural Networks).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Correntropy kernel temporal differences for reinforcement learning brain machine interfaces

AU - Bae, Jihye

AU - Giraldo, Luis G.Sanchez

AU - Principe, Jose C.

AU - Francis, Joseph T.

PY - 2014/9/3

Y1 - 2014/9/3

N2 - This paper introduces a novel temporal difference algorithm to estimate a value function in reinforcement learning. This is a kernel adaptive system using a robust cost function called correntropy. We call this system correntropy kernel temporal differences (CKTD). This algorithm is integrated with Q-learning to find a proper policy (Q-learning via correntropy kernel temporal differences). The proposed method was tested with a synthetic problem, and its robustness under a changing policy was quantified. The same algorithm was applied to the decoding of a monkey's neural states in a reinforcement learning brain machine interface (RLBMI) in a center-out reaching task. The results showed the potential advantage of the proposed algorithm in the RLBMI framework.

AB - This paper introduces a novel temporal difference algorithm to estimate a value function in reinforcement learning. This is a kernel adaptive system using a robust cost function called correntropy. We call this system correntropy kernel temporal differences (CKTD). This algorithm is integrated with Q-learning to find a proper policy (Q-learning via correntropy kernel temporal differences). The proposed method was tested with a synthetic problem, and its robustness under a changing policy was quantified. The same algorithm was applied to the decoding of a monkey's neural states in a reinforcement learning brain machine interface (RLBMI) in a center-out reaching task. The results showed the potential advantage of the proposed algorithm in the RLBMI framework.

UR - http://www.scopus.com/inward/record.url?scp=84908494286&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84908494286&partnerID=8YFLogxK

U2 - 10.1109/IJCNN.2014.6889958

DO - 10.1109/IJCNN.2014.6889958

M3 - Conference contribution

AN - SCOPUS:84908494286

T3 - Proceedings of the International Joint Conference on Neural Networks

SP - 2713

EP - 2717

BT - Proceedings of the International Joint Conference on Neural Networks

Y2 - 6 July 2014 through 11 July 2014

ER -

Correntropy kernel temporal differences for reinforcement learning brain machine interfaces

Abstract

Publication series

Conference

Bibliographical note

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this