Towards Improved Network Security Requirements and Policy: Domain-Specific Completeness Analysis via Topic Modeling

Jane Huffman Hayes; Jared Payne; Emily Essex; Kelsey Cole; Joseph Alverson; Alex Dekhtyar; Dongfeng Fang; Grant Bernosky

doi:10.1109/AIRE51212.2020.00019

Towards Improved Network Security Requirements and Policy: Domain-Specific Completeness Analysis via Topic Modeling

Jane Huffman Hayes, Jared Payne, Emily Essex, Kelsey Cole, Joseph Alverson, Alex Dekhtyar, Dongfeng Fang, Grant Bernosky

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Network security policies contain requirements-including system and software features as well as expected and desired actions of human actors. In this paper, we present a framework for evaluation of textual network security policies as requirements documents to identify areas for improvement. Specifically, our framework concentrates on completeness. We use topic modeling coupled with expert evaluation to learn the complete list of important topics that should be addressed in a network security policy. Using these topics as a checklist, we evaluate (students) a collection of network security policies for completeness, i.e., the level of presence of these topics in the text. We developed three methods for topic recognition to identify missing or poorly addressed topics. We examine network security policies and report the results of our analysis: preliminary success of our approach.

Original language	English
Title of host publication	Proceedings - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020
Pages	83-86
Number of pages	4
ISBN (Electronic)	9781728183527
DOIs	https://doi.org/10.1109/AIRE51212.2020.00019
State	Published - Sep 2020
Event	7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020 - Zurich, Switzerland Duration: Sep 1 2020 → …

Publication series

Name	Proceedings - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020

Conference

Conference	7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020
Country/Territory	Switzerland
City	Zurich
Period	9/1/20 → …

Bibliographical note

Publisher Copyright:
© 2020 IEEE.

Funding

ACKNOWLEDGMENT We thank NSF for partially funding this work under grant CICI 1642134. We thank the class of CalPoly students for undertaking the study. We thank Dr. Tingting Yu for helpful discussions on experimental design.

Funders	Funder number
National Science Foundation (NSF)	1642134, CICI 1642134

Keywords

Requirements quality
completeness
empirical evaluation
machine learning
network security

ASJC Scopus subject areas

Artificial Intelligence
Software
Safety, Risk, Reliability and Quality

Access to Document

10.1109/AIRE51212.2020.00019

Cite this

Huffman Hayes, J., Payne, J., Essex, E., Cole, K., Alverson, J., Dekhtyar, A., Fang, D., & Bernosky, G. (2020). Towards Improved Network Security Requirements and Policy: Domain-Specific Completeness Analysis via Topic Modeling. In Proceedings - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020 (pp. 83-86). Article 9233035 (Proceedings - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020). https://doi.org/10.1109/AIRE51212.2020.00019

Huffman Hayes, Jane ; Payne, Jared ; Essex, Emily et al. / Towards Improved Network Security Requirements and Policy : Domain-Specific Completeness Analysis via Topic Modeling. Proceedings - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020. 2020. pp. 83-86 (Proceedings - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020).

@inproceedings{c9ed03432d90420a8097246c113c5364,

title = "Towards Improved Network Security Requirements and Policy: Domain-Specific Completeness Analysis via Topic Modeling",

abstract = "Network security policies contain requirements-including system and software features as well as expected and desired actions of human actors. In this paper, we present a framework for evaluation of textual network security policies as requirements documents to identify areas for improvement. Specifically, our framework concentrates on completeness. We use topic modeling coupled with expert evaluation to learn the complete list of important topics that should be addressed in a network security policy. Using these topics as a checklist, we evaluate (students) a collection of network security policies for completeness, i.e., the level of presence of these topics in the text. We developed three methods for topic recognition to identify missing or poorly addressed topics. We examine network security policies and report the results of our analysis: preliminary success of our approach.",

keywords = "Requirements quality, completeness, empirical evaluation, machine learning, network security",

author = "{Huffman Hayes}, Jane and Jared Payne and Emily Essex and Kelsey Cole and Joseph Alverson and Alex Dekhtyar and Dongfeng Fang and Grant Bernosky",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020 ; Conference date: 01-09-2020",

year = "2020",

month = sep,

doi = "10.1109/AIRE51212.2020.00019",

language = "English",

series = "Proceedings - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020",

pages = "83--86",

booktitle = "Proceedings - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020",

}

Huffman Hayes, J, Payne, J, Essex, E, Cole, K, Alverson, J, Dekhtyar, A, Fang, D & Bernosky, G 2020, Towards Improved Network Security Requirements and Policy: Domain-Specific Completeness Analysis via Topic Modeling. in Proceedings - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020., 9233035, Proceedings - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020, pp. 83-86, 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020, Zurich, Switzerland, 9/1/20. https://doi.org/10.1109/AIRE51212.2020.00019

Towards Improved Network Security Requirements and Policy: Domain-Specific Completeness Analysis via Topic Modeling. / Huffman Hayes, Jane; Payne, Jared; Essex, Emily et al.
Proceedings - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020. 2020. p. 83-86 9233035 (Proceedings - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Towards Improved Network Security Requirements and Policy

T2 - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020

AU - Huffman Hayes, Jane

AU - Payne, Jared

AU - Essex, Emily

AU - Cole, Kelsey

AU - Alverson, Joseph

AU - Dekhtyar, Alex

AU - Fang, Dongfeng

AU - Bernosky, Grant

PY - 2020/9

Y1 - 2020/9

N2 - Network security policies contain requirements-including system and software features as well as expected and desired actions of human actors. In this paper, we present a framework for evaluation of textual network security policies as requirements documents to identify areas for improvement. Specifically, our framework concentrates on completeness. We use topic modeling coupled with expert evaluation to learn the complete list of important topics that should be addressed in a network security policy. Using these topics as a checklist, we evaluate (students) a collection of network security policies for completeness, i.e., the level of presence of these topics in the text. We developed three methods for topic recognition to identify missing or poorly addressed topics. We examine network security policies and report the results of our analysis: preliminary success of our approach.

AB - Network security policies contain requirements-including system and software features as well as expected and desired actions of human actors. In this paper, we present a framework for evaluation of textual network security policies as requirements documents to identify areas for improvement. Specifically, our framework concentrates on completeness. We use topic modeling coupled with expert evaluation to learn the complete list of important topics that should be addressed in a network security policy. Using these topics as a checklist, we evaluate (students) a collection of network security policies for completeness, i.e., the level of presence of these topics in the text. We developed three methods for topic recognition to identify missing or poorly addressed topics. We examine network security policies and report the results of our analysis: preliminary success of our approach.

KW - Requirements quality

KW - completeness

KW - empirical evaluation

KW - machine learning

KW - network security

UR - http://www.scopus.com/inward/record.url?scp=85097000989&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85097000989&partnerID=8YFLogxK

U2 - 10.1109/AIRE51212.2020.00019

DO - 10.1109/AIRE51212.2020.00019

M3 - Conference contribution

AN - SCOPUS:85097000989

T3 - Proceedings - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020

SP - 83

EP - 86

BT - Proceedings - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020

Y2 - 1 September 2020

ER -

Huffman Hayes J, Payne J, Essex E, Cole K, Alverson J, Dekhtyar A et al. Towards Improved Network Security Requirements and Policy: Domain-Specific Completeness Analysis via Topic Modeling. In Proceedings - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020. 2020. p. 83-86. 9233035. (Proceedings - 7th International Workshop on Artificial Intelligence and Requirements Engineering, AIRE 2020). doi: 10.1109/AIRE51212.2020.00019

Towards Improved Network Security Requirements and Policy: Domain-Specific Completeness Analysis via Topic Modeling

Abstract

Publication series

Conference

Bibliographical note

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this