nltk.TreebankWordTokenizer.span_tokenize¶

TreebankWordTokenizer.span_tokenize(s)¶

Identify the tokens using integer offsets (start_i, end_i), where s[start_i:end_i] is the corresponding token.

Return type:	iter(tuple(int, int))