nltk.BlanklineTokenizer

class nltk.BlanklineTokenizer[source]

Tokenize a string, treating any sequence of blank lines as a delimiter. Blank lines are defined as lines containing no characters, except for space or tab characters.

Methods

__init__()
span_tokenize(text)
span_tokenize_sents(strings) Apply self.span_tokenize() to each element of strings.
tokenize(text)
tokenize_sents(strings) Apply self.tokenize() to each element of strings.
unicode_repr()