The role of feature selection in text mining in the process of discovering missing clinical annotations – Case study

Aleksander Płaczek; Alicja Płuciennik; Mirosław Pach; Michał Jarząb; Dariusz Mrozek

doi:10.1007/978-3-030-19093-4_19

The role of feature selection in text mining in the process of discovering missing clinical annotations – Case study

Aleksander Płaczek
, Alicja Płuciennik
, Mirosław Pach
, Michał Jarząb
, Dariusz Mrozek

Department of Applied Informatics (RAU/RAU7)

WASKO S.A.
Silesian University of Technology
Maria Sklodowska-Curie Institute of Oncology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

Vocabulary used by the doctors to describe the results of medical procedures changes alongside with the new standards. Text data, which is immediately understandable by the medical professional, is difficult to use in mass scale analysis. Extraction of data relevant to the given case, e.g. Bethesda class, means taking on the challenge of normalizing the freeform text and all the grammatical forms associated with it. This is particularly difficult in the Polish language where words change their form significantly according to their function in the sentence. We found common black-box methods for text mining inaccurate for this purpose. Here we described a word-frequency-based method for annotation of text data for Bethesda class extraction. We compared them with an algorithm based on a decision tree C4.5. We showed how important is the choice of the method and range of features to avoid conflicting classification. Proposed algorithms allowed to avoid the rule-base limitations.

Original language	English
Title of host publication	Beyond Databases, Architectures and Structures. Paving the Road to Smart Data Processing and Analysis - 15th International Conference, BDAS 2019, Proceedings
Editors	Stanisław Kozielski, Dariusz Mrozek, Paweł Kasprowski, Bożena Małysiak-Mrozek, Daniel Kostrzewa
Publisher	Springer Verlag
Pages	248-262
Number of pages	15
ISBN (Print)	9783030190927
DOIs	https://doi.org/10.1007/978-3-030-19093-4_19
Publication status	Published - 2019
Event	15th International Conference Beyond Databases, Architectures and Structures, BDAS 2019 - Ustroń, Poland Duration: 28 May 2019 → 31 May 2019

Publication series

Name	Communications in Computer and Information Science
Volume	1018
ISSN (Print)	1865-0929
ISSN (Electronic)	1865-0937

Conference

Conference	15th International Conference Beyond Databases, Architectures and Structures, BDAS 2019
Country/Territory	Poland
City	Ustroń
Period	28/05/19 → 31/05/19

Keywords

Feature selection
Inverse document frequency
Text mining
Text tiding
Unstructured medical text

ASJC Scopus subject areas

General Computer Science
General Mathematics

Access to Document

10.1007/978-3-030-19093-4_19

Cite this

Płaczek, A., Płuciennik, A., Pach, M., Jarząb, M., & Mrozek, D. (2019). The role of feature selection in text mining in the process of discovering missing clinical annotations – Case study. In S. Kozielski, D. Mrozek, P. Kasprowski, B. Małysiak-Mrozek, & D. Kostrzewa (Eds.), Beyond Databases, Architectures and Structures. Paving the Road to Smart Data Processing and Analysis - 15th International Conference, BDAS 2019, Proceedings (pp. 248-262). (Communications in Computer and Information Science; Vol. 1018). Springer Verlag. https://doi.org/10.1007/978-3-030-19093-4_19

Płaczek, Aleksander ; Płuciennik, Alicja ; Pach, Mirosław et al. / The role of feature selection in text mining in the process of discovering missing clinical annotations – Case study. Beyond Databases, Architectures and Structures. Paving the Road to Smart Data Processing and Analysis - 15th International Conference, BDAS 2019, Proceedings. editor / Stanisław Kozielski ; Dariusz Mrozek ; Paweł Kasprowski ; Bożena Małysiak-Mrozek ; Daniel Kostrzewa. Springer Verlag, 2019. pp. 248-262 (Communications in Computer and Information Science).

@inproceedings{829d3507b2824ad1b7f37d02990263c5,

title = "The role of feature selection in text mining in the process of discovering missing clinical annotations {\textendash} Case study",

abstract = "Vocabulary used by the doctors to describe the results of medical procedures changes alongside with the new standards. Text data, which is immediately understandable by the medical professional, is difficult to use in mass scale analysis. Extraction of data relevant to the given case, e.g. Bethesda class, means taking on the challenge of normalizing the freeform text and all the grammatical forms associated with it. This is particularly difficult in the Polish language where words change their form significantly according to their function in the sentence. We found common black-box methods for text mining inaccurate for this purpose. Here we described a word-frequency-based method for annotation of text data for Bethesda class extraction. We compared them with an algorithm based on a decision tree C4.5. We showed how important is the choice of the method and range of features to avoid conflicting classification. Proposed algorithms allowed to avoid the rule-base limitations.",

keywords = "Feature selection, Inverse document frequency, Text mining, Text tiding, Unstructured medical text",

author = "Aleksander P{\l}aczek and Alicja P{\l}uciennik and Miros{\l}aw Pach and Micha{\l} Jarz{\c a}b and Dariusz Mrozek",

note = "Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2019.; 15th International Conference Beyond Databases, Architectures and Structures, BDAS 2019 ; Conference date: 28-05-2019 Through 31-05-2019",

year = "2019",

doi = "10.1007/978-3-030-19093-4\_19",

language = "English",

isbn = "9783030190927",

series = "Communications in Computer and Information Science",

publisher = "Springer Verlag",

pages = "248--262",

editor = "Stanis{\l}aw Kozielski and Dariusz Mrozek and Pawe{\l} Kasprowski and Bo{\.z}ena Ma{\l}ysiak-Mrozek and Daniel Kostrzewa",

booktitle = "Beyond Databases, Architectures and Structures. Paving the Road to Smart Data Processing and Analysis - 15th International Conference, BDAS 2019, Proceedings",

address = "Germany",

}

Płaczek, A, Płuciennik, A, Pach, M, Jarząb, M & Mrozek, D 2019, The role of feature selection in text mining in the process of discovering missing clinical annotations – Case study. in S Kozielski, D Mrozek, P Kasprowski, B Małysiak-Mrozek & D Kostrzewa (eds), Beyond Databases, Architectures and Structures. Paving the Road to Smart Data Processing and Analysis - 15th International Conference, BDAS 2019, Proceedings. Communications in Computer and Information Science, vol. 1018, Springer Verlag, pp. 248-262, 15th International Conference Beyond Databases, Architectures and Structures, BDAS 2019, Ustroń, Poland, 28/05/19. https://doi.org/10.1007/978-3-030-19093-4_19

The role of feature selection in text mining in the process of discovering missing clinical annotations – Case study. / Płaczek, Aleksander; Płuciennik, Alicja; Pach, Mirosław et al.
Beyond Databases, Architectures and Structures. Paving the Road to Smart Data Processing and Analysis - 15th International Conference, BDAS 2019, Proceedings. ed. / Stanisław Kozielski; Dariusz Mrozek; Paweł Kasprowski; Bożena Małysiak-Mrozek; Daniel Kostrzewa. Springer Verlag, 2019. p. 248-262 (Communications in Computer and Information Science; Vol. 1018).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - The role of feature selection in text mining in the process of discovering missing clinical annotations – Case study

AU - Płaczek, Aleksander

AU - Płuciennik, Alicja

AU - Pach, Mirosław

AU - Jarząb, Michał

AU - Mrozek, Dariusz

N1 - Publisher Copyright: © Springer Nature Switzerland AG 2019.

PY - 2019

Y1 - 2019

N2 - Vocabulary used by the doctors to describe the results of medical procedures changes alongside with the new standards. Text data, which is immediately understandable by the medical professional, is difficult to use in mass scale analysis. Extraction of data relevant to the given case, e.g. Bethesda class, means taking on the challenge of normalizing the freeform text and all the grammatical forms associated with it. This is particularly difficult in the Polish language where words change their form significantly according to their function in the sentence. We found common black-box methods for text mining inaccurate for this purpose. Here we described a word-frequency-based method for annotation of text data for Bethesda class extraction. We compared them with an algorithm based on a decision tree C4.5. We showed how important is the choice of the method and range of features to avoid conflicting classification. Proposed algorithms allowed to avoid the rule-base limitations.

AB - Vocabulary used by the doctors to describe the results of medical procedures changes alongside with the new standards. Text data, which is immediately understandable by the medical professional, is difficult to use in mass scale analysis. Extraction of data relevant to the given case, e.g. Bethesda class, means taking on the challenge of normalizing the freeform text and all the grammatical forms associated with it. This is particularly difficult in the Polish language where words change their form significantly according to their function in the sentence. We found common black-box methods for text mining inaccurate for this purpose. Here we described a word-frequency-based method for annotation of text data for Bethesda class extraction. We compared them with an algorithm based on a decision tree C4.5. We showed how important is the choice of the method and range of features to avoid conflicting classification. Proposed algorithms allowed to avoid the rule-base limitations.

KW - Feature selection

KW - Inverse document frequency

KW - Text mining

KW - Text tiding

KW - Unstructured medical text

UR - https://www.scopus.com/pages/publications/85065903506

U2 - 10.1007/978-3-030-19093-4_19

DO - 10.1007/978-3-030-19093-4_19

M3 - Conference contribution

AN - SCOPUS:85065903506

SN - 9783030190927

T3 - Communications in Computer and Information Science

SP - 248

EP - 262

BT - Beyond Databases, Architectures and Structures. Paving the Road to Smart Data Processing and Analysis - 15th International Conference, BDAS 2019, Proceedings

A2 - Kozielski, Stanisław

A2 - Mrozek, Dariusz

A2 - Kasprowski, Paweł

A2 - Małysiak-Mrozek, Bożena

A2 - Kostrzewa, Daniel

PB - Springer Verlag

T2 - 15th International Conference Beyond Databases, Architectures and Structures, BDAS 2019

Y2 - 28 May 2019 through 31 May 2019

ER -

Płaczek A, Płuciennik A, Pach M, Jarząb M, Mrozek D. The role of feature selection in text mining in the process of discovering missing clinical annotations – Case study. In Kozielski S, Mrozek D, Kasprowski P, Małysiak-Mrozek B, Kostrzewa D, editors, Beyond Databases, Architectures and Structures. Paving the Road to Smart Data Processing and Analysis - 15th International Conference, BDAS 2019, Proceedings. Springer Verlag. 2019. p. 248-262. (Communications in Computer and Information Science). doi: 10.1007/978-3-030-19093-4_19

The role of feature selection in text mining in the process of discovering missing clinical annotations – Case study

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this