Summarize Text by Ranking Sentences and Finding Keywords
Extract the most important sentences which were identified with textra...
Get all combinations of sentences
Use locality-sensitive hashing to get combinations of sentences which ...
Calculate the distance between 2 vectors based on the Jaccard distance
Textrank - extract relevant keywords
Textrank - extract relevant sentences
The 'textrank' algorithm is an extension of the 'Pagerank' algorithm for text. The algorithm allows to summarize text by calculating how sentences are related to one another. This is done by looking at overlapping terminology used in sentences in order to set up links between sentences. The resulting sentence network is next plugged into the 'Pagerank' algorithm which identifies the most important sentences in your text and ranks them. In a similar way 'textrank' can also be used to extract keywords. A word network is constructed by looking if words are following one another. On top of that network the 'Pagerank' algorithm is applied to extract relevant words after which relevant words which are following one another are combined to get keywords. More information can be found in the paper from Mihalcea, Rada & Tarau, Paul (2004) <https://www.aclweb.org/anthology/W04-3252/>.