Robust Rayleigh quotient minimization and nonlinear eigenvalue problems

Zhaojun Bai; Ding Lu; Bart Vandereycken

doi:10.1137/18M1167681

Robust Rayleigh quotient minimization and nonlinear eigenvalue problems

Zhaojun Bai, Ding Lu, Bart Vandereycken

Mathematics

Research output: Contribution to journal › Article › peer-review

16 Scopus citations

Abstract

We study the robust Rayleigh quotient optimization problem where the data matrices of the Rayleigh quotient are subject to uncertainties. We propose to solve such a problem by exploiting its characterization as a nonlinear eigenvalue problem with eigenvector nonlinearity (NEPv). For solving the NEPv, we show that a commonly used iterative method can be divergent due to a wrong ordering of the eigenvalues. Two strategies are introduced to address this issue: a spectral transformation based on nonlinear shifting and a reformulation using second-order derivatives. Numerical experiments for applications in robust generalized eigenvalue classification, robust common spatial pattern analysis, and robust linear discriminant analysis demonstrate the effectiveness of the proposed approaches.

Original language	English
Pages (from-to)	A3495-A3522
Journal	SIAM Journal on Scientific Computing
Volume	40
Issue number	5
DOIs	https://doi.org/10.1137/18M1167681
State	Published - 2018

Bibliographical note

Funding Information:
∗Submitted to the journal’s Methods and Algorithms for Scientific Computing section January 29, 2018; accepted for publication (in revised form) August 1, 2018; published electronically October 18, 2018. http://www.siam.org/journals/sisc/40-5/M116768.html Funding: The first author was supported in part by NSF grants DMS-1522697 and CCF-1527091, and the second author by SNSF project 169115. †Department of Computer Science and Department of Mathematics, University of California, Davis, CA 95616 (bai@cs.ucdavis.edu). ‡Department of Mathematics, University of Geneva, CH-1211 Geneva, Switzerland (Ding.Lu@ unige.ch, Bart.Vandereycken@unige.ch).

Funding Information:
The first author was supported in part by NSF grants DMS-1522697 and CCF-1527091, and the second author by SNSF project 169115.

Publisher Copyright:
© 2018 Society for Industrial and Applied Mathematics.

Keywords

Nonlinear eigenvalue problems
Rayleigh quotient
Robust optimization
Self-consistent-field iteration

ASJC Scopus subject areas

Computational Mathematics
Applied Mathematics

Access to Document

10.1137/18M1167681

Cite this

@article{3b26ac970fec42d4b3204a3fc5fad9c2,

title = "Robust Rayleigh quotient minimization and nonlinear eigenvalue problems",

abstract = "We study the robust Rayleigh quotient optimization problem where the data matrices of the Rayleigh quotient are subject to uncertainties. We propose to solve such a problem by exploiting its characterization as a nonlinear eigenvalue problem with eigenvector nonlinearity (NEPv). For solving the NEPv, we show that a commonly used iterative method can be divergent due to a wrong ordering of the eigenvalues. Two strategies are introduced to address this issue: a spectral transformation based on nonlinear shifting and a reformulation using second-order derivatives. Numerical experiments for applications in robust generalized eigenvalue classification, robust common spatial pattern analysis, and robust linear discriminant analysis demonstrate the effectiveness of the proposed approaches.",

keywords = "Nonlinear eigenvalue problems, Rayleigh quotient, Robust optimization, Self-consistent-field iteration",

author = "Zhaojun Bai and Ding Lu and Bart Vandereycken",

note = "Funding Information: ∗Submitted to the journal{\textquoteright}s Methods and Algorithms for Scientific Computing section January 29, 2018; accepted for publication (in revised form) August 1, 2018; published electronically October 18, 2018. http://www.siam.org/journals/sisc/40-5/M116768.html Funding: The first author was supported in part by NSF grants DMS-1522697 and CCF-1527091, and the second author by SNSF project 169115. †Department of Computer Science and Department of Mathematics, University of California, Davis, CA 95616 (bai@cs.ucdavis.edu). ‡Department of Mathematics, University of Geneva, CH-1211 Geneva, Switzerland (Ding.Lu@ unige.ch, Bart.Vandereycken@unige.ch). Funding Information: The first author was supported in part by NSF grants DMS-1522697 and CCF-1527091, and the second author by SNSF project 169115. Publisher Copyright: {\textcopyright} 2018 Society for Industrial and Applied Mathematics.",

year = "2018",

doi = "10.1137/18M1167681",

language = "English",

volume = "40",

pages = "A3495--A3522",

number = "5",

}

TY - JOUR

T1 - Robust Rayleigh quotient minimization and nonlinear eigenvalue problems

AU - Bai, Zhaojun

AU - Lu, Ding

AU - Vandereycken, Bart

N1 - Funding Information: ∗Submitted to the journal’s Methods and Algorithms for Scientific Computing section January 29, 2018; accepted for publication (in revised form) August 1, 2018; published electronically October 18, 2018. http://www.siam.org/journals/sisc/40-5/M116768.html Funding: The first author was supported in part by NSF grants DMS-1522697 and CCF-1527091, and the second author by SNSF project 169115. †Department of Computer Science and Department of Mathematics, University of California, Davis, CA 95616 (bai@cs.ucdavis.edu). ‡Department of Mathematics, University of Geneva, CH-1211 Geneva, Switzerland (Ding.Lu@ unige.ch, Bart.Vandereycken@unige.ch). Funding Information: The first author was supported in part by NSF grants DMS-1522697 and CCF-1527091, and the second author by SNSF project 169115. Publisher Copyright: © 2018 Society for Industrial and Applied Mathematics.

PY - 2018

Y1 - 2018

N2 - We study the robust Rayleigh quotient optimization problem where the data matrices of the Rayleigh quotient are subject to uncertainties. We propose to solve such a problem by exploiting its characterization as a nonlinear eigenvalue problem with eigenvector nonlinearity (NEPv). For solving the NEPv, we show that a commonly used iterative method can be divergent due to a wrong ordering of the eigenvalues. Two strategies are introduced to address this issue: a spectral transformation based on nonlinear shifting and a reformulation using second-order derivatives. Numerical experiments for applications in robust generalized eigenvalue classification, robust common spatial pattern analysis, and robust linear discriminant analysis demonstrate the effectiveness of the proposed approaches.

AB - We study the robust Rayleigh quotient optimization problem where the data matrices of the Rayleigh quotient are subject to uncertainties. We propose to solve such a problem by exploiting its characterization as a nonlinear eigenvalue problem with eigenvector nonlinearity (NEPv). For solving the NEPv, we show that a commonly used iterative method can be divergent due to a wrong ordering of the eigenvalues. Two strategies are introduced to address this issue: a spectral transformation based on nonlinear shifting and a reformulation using second-order derivatives. Numerical experiments for applications in robust generalized eigenvalue classification, robust common spatial pattern analysis, and robust linear discriminant analysis demonstrate the effectiveness of the proposed approaches.

KW - Nonlinear eigenvalue problems

KW - Rayleigh quotient

KW - Robust optimization

KW - Self-consistent-field iteration

UR - http://www.scopus.com/inward/record.url?scp=85056141612&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056141612&partnerID=8YFLogxK

U2 - 10.1137/18M1167681

DO - 10.1137/18M1167681

M3 - Article

AN - SCOPUS:85056141612

VL - 40

SP - A3495-A3522

IS - 5

ER -

Robust Rayleigh quotient minimization and nonlinear eigenvalue problems

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this