Reinforcement learning via kernel temporal difference

Jihye Bae; Pratik Chhatbar; Joseph T. Francis; Justin C. Sanchez; Jose C. Principe

doi:10.1109/IEMBS.2011.6091370

Reinforcement learning via kernel temporal difference

Jihye Bae, Pratik Chhatbar, Joseph T. Francis, Justin C. Sanchez, Jose C. Principe

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

18 Scopus citations

Abstract

This paper introduces a kernel adaptive filter implemented with stochastic gradient on temporal differences, kernel Temporal Difference (TD)(λ), to estimate the state-action value function in reinforcement learning. The case λ=0 will be studied in this paper. Experimental results show the method's applicability for learning motor state decoding during a center-out reaching task performed by a monkey. The results are compared to the implementation of a time delay neural network (TDNN) trained with backpropagation of the temporal difference error. From the experiments, it is observed that kernel TD(0) allows faster convergence and a better solution than the neural network.

Original language	English
Title of host publication	33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2011
Pages	5662-5665
Number of pages	4
DOIs	https://doi.org/10.1109/IEMBS.2011.6091370
State	Published - 2011
Event	33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2011 - Boston, MA, United States Duration: Aug 30 2011 → Sep 3 2011

Publication series

Name	Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
ISSN (Print)	1557-170X

Conference

Conference	33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2011
Country/Territory	United States
City	Boston, MA
Period	8/30/11 → 9/3/11

ASJC Scopus subject areas

Signal Processing
Biomedical Engineering
Computer Vision and Pattern Recognition
Health Informatics

Access to Document

10.1109/IEMBS.2011.6091370

Cite this

Bae, J., Chhatbar, P., Francis, J. T., Sanchez, J. C., & Principe, J. C. (2011). Reinforcement learning via kernel temporal difference. In 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2011 (pp. 5662-5665). Article 6091370 (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS). https://doi.org/10.1109/IEMBS.2011.6091370

@inproceedings{c68549ff235741a385617a2db14560cb,

title = "Reinforcement learning via kernel temporal difference",

abstract = "This paper introduces a kernel adaptive filter implemented with stochastic gradient on temporal differences, kernel Temporal Difference (TD)(λ), to estimate the state-action value function in reinforcement learning. The case λ=0 will be studied in this paper. Experimental results show the method's applicability for learning motor state decoding during a center-out reaching task performed by a monkey. The results are compared to the implementation of a time delay neural network (TDNN) trained with backpropagation of the temporal difference error. From the experiments, it is observed that kernel TD(0) allows faster convergence and a better solution than the neural network.",

author = "Jihye Bae and Pratik Chhatbar and Francis, {Joseph T.} and Sanchez, {Justin C.} and Principe, {Jose C.}",

year = "2011",

doi = "10.1109/IEMBS.2011.6091370",

language = "English",

isbn = "9781424441211",

series = "Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS",

pages = "5662--5665",

booktitle = "33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2011",

note = "33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2011 ; Conference date: 30-08-2011 Through 03-09-2011",

}

Bae, J, Chhatbar, P, Francis, JT, Sanchez, JC & Principe, JC 2011, Reinforcement learning via kernel temporal difference. in 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2011., 6091370, Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, pp. 5662-5665, 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2011, Boston, MA, United States, 8/30/11. https://doi.org/10.1109/IEMBS.2011.6091370

Reinforcement learning via kernel temporal difference. / Bae, Jihye; Chhatbar, Pratik; Francis, Joseph T. et al.
33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2011. 2011. p. 5662-5665 6091370 (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Reinforcement learning via kernel temporal difference

AU - Bae, Jihye

AU - Chhatbar, Pratik

AU - Francis, Joseph T.

AU - Sanchez, Justin C.

AU - Principe, Jose C.

PY - 2011

Y1 - 2011

N2 - This paper introduces a kernel adaptive filter implemented with stochastic gradient on temporal differences, kernel Temporal Difference (TD)(λ), to estimate the state-action value function in reinforcement learning. The case λ=0 will be studied in this paper. Experimental results show the method's applicability for learning motor state decoding during a center-out reaching task performed by a monkey. The results are compared to the implementation of a time delay neural network (TDNN) trained with backpropagation of the temporal difference error. From the experiments, it is observed that kernel TD(0) allows faster convergence and a better solution than the neural network.

AB - This paper introduces a kernel adaptive filter implemented with stochastic gradient on temporal differences, kernel Temporal Difference (TD)(λ), to estimate the state-action value function in reinforcement learning. The case λ=0 will be studied in this paper. Experimental results show the method's applicability for learning motor state decoding during a center-out reaching task performed by a monkey. The results are compared to the implementation of a time delay neural network (TDNN) trained with backpropagation of the temporal difference error. From the experiments, it is observed that kernel TD(0) allows faster convergence and a better solution than the neural network.

UR - http://www.scopus.com/inward/record.url?scp=84055199049&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84055199049&partnerID=8YFLogxK

U2 - 10.1109/IEMBS.2011.6091370

DO - 10.1109/IEMBS.2011.6091370

M3 - Conference contribution

C2 - 22255624

AN - SCOPUS:84055199049

SN - 9781424441211

T3 - Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS

SP - 5662

EP - 5665

BT - 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2011

T2 - 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2011

Y2 - 30 August 2011 through 3 September 2011

ER -

Bae J, Chhatbar P, Francis JT, Sanchez JC, Principe JC. Reinforcement learning via kernel temporal difference. In 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2011. 2011. p. 5662-5665. 6091370. (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS). doi: 10.1109/IEMBS.2011.6091370

Reinforcement learning via kernel temporal difference

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this