class gensim.corpora.WikiCorpus(fname, processes=None, lemmatize=True, dictionary=None, filter_namespaces=('0', ))[source]

Treat a wikipedia articles dump (*articles.xml.bz2) as a (read-only) corpus.

The documents are extracted on-the-fly, so that the whole (massive) dump can stay compressed on disk.

>>> wiki = WikiCorpus('enwiki-20100622-pages-articles.xml.bz2') # create word->word_id mapping, takes almost 8h
>>> MmCorpus.serialize('', wiki) # another 8h, creates a file in MatrixMarket format plus file with id->word


__init__(fname[, processes, lemmatize, ...]) Initialize the corpus.
get_texts() Iterate over the dump, returning text version of each article as a list of tokens.
load(fname[, mmap]) Load a previously saved object from file (also see save).
save(*args, **kwargs)
save_corpus(fname, corpus[, id2word, metadata]) Save an existing corpus to disk.