THE REPRESENTATION JENSEN-RÉNYI DIVERGENCE

Jhoan Keider Hoyos Osorio; Oscar Skean; Austin J. Brockmeier; Luis Gonzalo Sanchez Giraldo

doi:10.1109/ICASSP43922.2022.9747350

THE REPRESENTATION JENSEN-RÉNYI DIVERGENCE

Jhoan Keider Hoyos Osorio, Oscar Skean, Austin J. Brockmeier, Luis Gonzalo Sanchez Giraldo

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

We introduce a divergence measure between data distributions based on operators in reproducing kernel Hilbert spaces defined by kernels. The empirical estimator of the divergence is computed using the eigenvalues of positive definite Gram matrices that are obtained by evaluating the kernel over pairs of data points. The new measure shares similar properties to Jensen-Shannon divergence. Convergence of the proposed estimators follows from concentration results based on the difference between the ordered spectrum of the Gram matrices and the integral operators associated with the population quantities. The proposed measure of divergence avoids the estimation of the probability distribution underlying the data. Numerical experiments involving comparing distributions and applications to sampling unbalanced data for classification show that the proposed divergence can achieve state of the art results.

Original language	English
Title of host publication	2022 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2022 - Proceedings
Pages	4313-4317
Number of pages	5
ISBN (Electronic)	9781665405409
DOIs	https://doi.org/10.1109/ICASSP43922.2022.9747350
State	Published - 2022
Event	2022 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2022 - Hybrid, Singapore Duration: May 22 2022 → May 27 2022

Publication series

Name	ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume	2022-May
ISSN (Print)	1520-6149

Conference

Conference	2022 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2022
Country/Territory	Singapore
City	Hybrid
Period	5/22/22 → 5/27/22

Bibliographical note

Publisher Copyright:
© 2022 IEEE

Funding

This material is based upon work supported by the Office of the Under Secretary of Defense for Research and Engineering under award number FA9550-21-1-0227. Austin Brockmeier’s effort were sponsored by the Department of the Navy, Office of Naval Research under ONR award number N00014-21-1-2300.

Funders	Funder number
Office of the Under Secretary of Defense for Research and Engineering	FA9550-21-1-0227
Office of Naval Research Naval Academy	N00014-21-1-2300
Office of Naval Research Naval Academy
U.S. Navy Air Systems Command

Keywords

Divergence
Imbalanced classification
Information theoretic learning
Kernel methods
Subsampling

ASJC Scopus subject areas

Software
Signal Processing
Electrical and Electronic Engineering

Access to Document

10.1109/ICASSP43922.2022.9747350

Cite this

Osorio, J. K. H., Skean, O., Brockmeier, A. J., & Giraldo, L. G. S. (2022). THE REPRESENTATION JENSEN-RÉNYI DIVERGENCE. In 2022 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2022 - Proceedings (pp. 4313-4317). (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings; Vol. 2022-May). https://doi.org/10.1109/ICASSP43922.2022.9747350

@inproceedings{2f15532273d943f1a2d50f8ee268bfa0,

title = "THE REPRESENTATION JENSEN-R{\'E}NYI DIVERGENCE",

abstract = "We introduce a divergence measure between data distributions based on operators in reproducing kernel Hilbert spaces defined by kernels. The empirical estimator of the divergence is computed using the eigenvalues of positive definite Gram matrices that are obtained by evaluating the kernel over pairs of data points. The new measure shares similar properties to Jensen-Shannon divergence. Convergence of the proposed estimators follows from concentration results based on the difference between the ordered spectrum of the Gram matrices and the integral operators associated with the population quantities. The proposed measure of divergence avoids the estimation of the probability distribution underlying the data. Numerical experiments involving comparing distributions and applications to sampling unbalanced data for classification show that the proposed divergence can achieve state of the art results.",

keywords = "Divergence, Imbalanced classification, Information theoretic learning, Kernel methods, Subsampling",

author = "Osorio, \{Jhoan Keider Hoyos\} and Oscar Skean and Brockmeier, \{Austin J.\} and Giraldo, \{Luis Gonzalo Sanchez\}",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE; 2022 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2022 ; Conference date: 22-05-2022 Through 27-05-2022",

year = "2022",

doi = "10.1109/ICASSP43922.2022.9747350",

language = "English",

series = "ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings",

pages = "4313--4317",

booktitle = "2022 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2022 - Proceedings",

}

Osorio, JKH, Skean, O, Brockmeier, AJ & Giraldo, LGS 2022, THE REPRESENTATION JENSEN-RÉNYI DIVERGENCE. in 2022 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2022 - Proceedings. ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, vol. 2022-May, pp. 4313-4317, 2022 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2022, Hybrid, Singapore, 5/22/22. https://doi.org/10.1109/ICASSP43922.2022.9747350

THE REPRESENTATION JENSEN-RÉNYI DIVERGENCE. / Osorio, Jhoan Keider Hoyos; Skean, Oscar; Brockmeier, Austin J. et al.
2022 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2022 - Proceedings. 2022. p. 4313-4317 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings; Vol. 2022-May).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - THE REPRESENTATION JENSEN-RÉNYI DIVERGENCE

AU - Osorio, Jhoan Keider Hoyos

AU - Skean, Oscar

AU - Brockmeier, Austin J.

AU - Giraldo, Luis Gonzalo Sanchez

PY - 2022

Y1 - 2022

N2 - We introduce a divergence measure between data distributions based on operators in reproducing kernel Hilbert spaces defined by kernels. The empirical estimator of the divergence is computed using the eigenvalues of positive definite Gram matrices that are obtained by evaluating the kernel over pairs of data points. The new measure shares similar properties to Jensen-Shannon divergence. Convergence of the proposed estimators follows from concentration results based on the difference between the ordered spectrum of the Gram matrices and the integral operators associated with the population quantities. The proposed measure of divergence avoids the estimation of the probability distribution underlying the data. Numerical experiments involving comparing distributions and applications to sampling unbalanced data for classification show that the proposed divergence can achieve state of the art results.

AB - We introduce a divergence measure between data distributions based on operators in reproducing kernel Hilbert spaces defined by kernels. The empirical estimator of the divergence is computed using the eigenvalues of positive definite Gram matrices that are obtained by evaluating the kernel over pairs of data points. The new measure shares similar properties to Jensen-Shannon divergence. Convergence of the proposed estimators follows from concentration results based on the difference between the ordered spectrum of the Gram matrices and the integral operators associated with the population quantities. The proposed measure of divergence avoids the estimation of the probability distribution underlying the data. Numerical experiments involving comparing distributions and applications to sampling unbalanced data for classification show that the proposed divergence can achieve state of the art results.

KW - Divergence

KW - Imbalanced classification

KW - Information theoretic learning

KW - Kernel methods

KW - Subsampling

UR - http://www.scopus.com/inward/record.url?scp=85131237147&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85131237147&partnerID=8YFLogxK

U2 - 10.1109/ICASSP43922.2022.9747350

DO - 10.1109/ICASSP43922.2022.9747350

M3 - Conference contribution

AN - SCOPUS:85131237147

T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

SP - 4313

EP - 4317

BT - 2022 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2022 - Proceedings

T2 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2022

Y2 - 22 May 2022 through 27 May 2022

ER -

THE REPRESENTATION JENSEN-RÉNYI DIVERGENCE

Abstract

Publication series

Conference

Bibliographical note

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this