KNIME Text Processing

The KNIME Text Processing feature was designed and developed to read and process textual data, and transform it into numerical data (document and term vectors) in order to apply regular KNIME data mining nodes (e.g. for clustering and classification).

This feature allows for the parsing of texts available in various formats (e.g. Xml, Microsoft Word or PDF and the internal representation of documents and terms) as KNIME data cells stored in a data table. It is possible to recognize and tag different kinds of named entities such as names of persons and organizations, genes and proteins or chemical compounds, thus enriching the documents semantically. Furthermore, documents can be filtered (e.g. by stop word or named entity filters), stemmed by stemmers for various languages and preprocessed in many other ways. Frequencies of words can be computed, keywords can be extracted, and documents can be visualized (e.g.tag clouds).

To apply regular KNIME nodes to cluster or classify documents, they can be transformed into numerical vectors. To process texts with the KNIME Text Processing feature, usually six different steps need to be accomplished. These steps are: • IO -Text parsing occurs here;The text and the structure, if available,are extracted and represented in a data structure the Text Processing nodes are able to handle

• Enrichment -semantic information is added by named entity recognition and tagging
• Preprocessing- terms are filtered and manipulated in order to get rid of terms that do not contain content. Terms such as stop words, numbers, punctuation marks or very small words are filtered.
• Frequencies -After preprocessing is finished, frequencies of terms in documents and the complete corpus can be computed
• Transformation- The textual data is first transformed into numerical data before regular KNIME nodes are applied on the documents.
• Visualization - There are two nodes that visualize textual data, the “ Document Viewer ” and the “ Tag Cloud ” node. The “Document Viewer” node allows for a simple visualization of documents. The “Tag Cloud” node creates a typical tag cloud with additional options such as different arrangements of terms.