A parallel hybrid web document clustering algorithm and its performance study

Shuting Xu; Jun Zhang

doi:10.1023/B:SUPE.0000040611.25862.d9

A parallel hybrid web document clustering algorithm and its performance study

Shuting Xu
, Jun Zhang

Computer Science

Producción científica: Article › revisión exhaustiva

25 Citas (Scopus)

Resumen

Clustering web document is an important procedure in many web information retrieval systems. As the size of the Internet grows rapidly and the amount of information requests increases exponentially, the use of parallel computing techniques in large scale web document retrieval is unavoidable. We propose a parallel hybrid web document clustering algorithm, which combines the Principal Direction Divisive Partitioning (PDDP) algorithm with the K-means algorithm. Computational experiments were conducted to test the performance of the hybrid algorithm using three real life web document datasets, and the results were compared with that of the parallel PDDP algorithm and the parallel K-means algorithm. The experiments show that the quality of the clustering solutions obtained from the hybrid algorithm is better than that from the parallel PDDP or the parallel K-means. The parallel run time of the hybrid algorithm is similar to and sometimes less than that of the widely used K-means algorithm.

Idioma original	English
Páginas (desde-hasta)	117-131
Número de páginas	15
Publicación	Journal of Supercomputing
Volumen	30
N.º	2
DOI	https://doi.org/10.1023/B:SUPE.0000040611.25862.d9
Estado	Published - nov 2004

Nota bibliográfica

Funding Information:
∗The research work of S. Xu was supported in part by the U.S. National Science Foundation under grant CCR-0092532. †The research work of J. Zhang was supported in part by the U.S. National Science Foundation under grants CCR-9988165, CCR-0092532, and ACR-0202934, by the U.S. Department of Energy Office of Science under grant DE-FG02-02ER45961, by the Kentucky Science & Engineering Foundation under grant KSEF-02-264-RED-002, by the Japanese Research Organization for Information Science & Technology, and by the University of Kentucky Research Committee.

Financiación

\u2217The research work of S. Xu was supported in part by the U.S. National Science Foundation under grant CCR-0092532. \u2020The research work of J. Zhang was supported in part by the U.S. National Science Foundation under grants CCR-9988165, CCR-0092532, and ACR-0202934, by the U.S. Department of Energy Office of Science under grant DE-FG02-02ER45961, by the Kentucky Science & Engineering Foundation under grant KSEF-02-264-RED-002, by the Japanese Research Organization for Information Science & Technology, and by the University of Kentucky Research Committee.

Financiadores	Número del financiador
Japanese Research Organization for Information Science & Technology
University of Kentucky Research Committee
Kentucky Science and Engineering Foundation	KSEF-02-264-RED-002
U.S. Department of Energy Chinese Academy of Sciences Guangzhou Municipal Science and Technology Project Oak Ridge National Laboratory Extreme Science and Engineering Discovery Environment National Science Foundation National Energy Research Scientific Computing Center National Natural Science Foundation of China	CCR-0092532, ACR-0202934, 0202934, CCR-9988165
U.S. Department of Energy Oak Ridge National Laboratory U.S. Department of Energy National Science Foundation National Energy Research Scientific Computing Center	DE-FG02-02ER45961

ASJC Scopus subject areas

Theoretical Computer Science
Software
Information Systems
Hardware and Architecture

Acceder al documento

10.1023/B:SUPE.0000040611.25862.d9

Otros archivos y enlaces

Citar esto

@article{231f8937c2ee43329535a427774ce62d,

title = "A parallel hybrid web document clustering algorithm and its performance study",

abstract = "Clustering web document is an important procedure in many web information retrieval systems. As the size of the Internet grows rapidly and the amount of information requests increases exponentially, the use of parallel computing techniques in large scale web document retrieval is unavoidable. We propose a parallel hybrid web document clustering algorithm, which combines the Principal Direction Divisive Partitioning (PDDP) algorithm with the K-means algorithm. Computational experiments were conducted to test the performance of the hybrid algorithm using three real life web document datasets, and the results were compared with that of the parallel PDDP algorithm and the parallel K-means algorithm. The experiments show that the quality of the clustering solutions obtained from the hybrid algorithm is better than that from the parallel PDDP or the parallel K-means. The parallel run time of the hybrid algorithm is similar to and sometimes less than that of the widely used K-means algorithm.",

keywords = "Information retrieval, K-means, PDDP, Parallel document clustering",

author = "Shuting Xu and Jun Zhang",

note = "Funding Information: ∗The research work of S. Xu was supported in part by the U.S. National Science Foundation under grant CCR-0092532. †The research work of J. Zhang was supported in part by the U.S. National Science Foundation under grants CCR-9988165, CCR-0092532, and ACR-0202934, by the U.S. Department of Energy Office of Science under grant DE-FG02-02ER45961, by the Kentucky Science \& Engineering Foundation under grant KSEF-02-264-RED-002, by the Japanese Research Organization for Information Science \& Technology, and by the University of Kentucky Research Committee.",

year = "2004",

month = nov,

doi = "10.1023/B:SUPE.0000040611.25862.d9",

language = "English",

volume = "30",

pages = "117--131",

number = "2",

}

TY - JOUR

T1 - A parallel hybrid web document clustering algorithm and its performance study

AU - Xu, Shuting

AU - Zhang, Jun

N1 - Funding Information: ∗The research work of S. Xu was supported in part by the U.S. National Science Foundation under grant CCR-0092532. †The research work of J. Zhang was supported in part by the U.S. National Science Foundation under grants CCR-9988165, CCR-0092532, and ACR-0202934, by the U.S. Department of Energy Office of Science under grant DE-FG02-02ER45961, by the Kentucky Science & Engineering Foundation under grant KSEF-02-264-RED-002, by the Japanese Research Organization for Information Science & Technology, and by the University of Kentucky Research Committee.

PY - 2004/11

Y1 - 2004/11

N2 - Clustering web document is an important procedure in many web information retrieval systems. As the size of the Internet grows rapidly and the amount of information requests increases exponentially, the use of parallel computing techniques in large scale web document retrieval is unavoidable. We propose a parallel hybrid web document clustering algorithm, which combines the Principal Direction Divisive Partitioning (PDDP) algorithm with the K-means algorithm. Computational experiments were conducted to test the performance of the hybrid algorithm using three real life web document datasets, and the results were compared with that of the parallel PDDP algorithm and the parallel K-means algorithm. The experiments show that the quality of the clustering solutions obtained from the hybrid algorithm is better than that from the parallel PDDP or the parallel K-means. The parallel run time of the hybrid algorithm is similar to and sometimes less than that of the widely used K-means algorithm.

AB - Clustering web document is an important procedure in many web information retrieval systems. As the size of the Internet grows rapidly and the amount of information requests increases exponentially, the use of parallel computing techniques in large scale web document retrieval is unavoidable. We propose a parallel hybrid web document clustering algorithm, which combines the Principal Direction Divisive Partitioning (PDDP) algorithm with the K-means algorithm. Computational experiments were conducted to test the performance of the hybrid algorithm using three real life web document datasets, and the results were compared with that of the parallel PDDP algorithm and the parallel K-means algorithm. The experiments show that the quality of the clustering solutions obtained from the hybrid algorithm is better than that from the parallel PDDP or the parallel K-means. The parallel run time of the hybrid algorithm is similar to and sometimes less than that of the widely used K-means algorithm.

KW - Information retrieval

KW - K-means

KW - PDDP

KW - Parallel document clustering

UR - https://www.scopus.com/pages/publications/23944477313

UR - https://www.scopus.com/pages/publications/23944477313#tab=citedBy

U2 - 10.1023/B:SUPE.0000040611.25862.d9

DO - 10.1023/B:SUPE.0000040611.25862.d9

M3 - Article

AN - SCOPUS:23944477313

SN - 0920-8542

VL - 30

SP - 117

EP - 131

JO - Journal of Supercomputing

JF - Journal of Supercomputing

IS - 2

ER -

A parallel hybrid web document clustering algorithm and its performance study

Resumen

Nota bibliográfica

Financiación

ASJC Scopus subject areas

Acceder al documento

Otros archivos y enlaces

Huella

Citar esto