gensim 0.2

Python Framework for Topic Modeling

Gensim is a Python framework for unsupervised learning from raw, unstructured digital texts. It provides a framework for learning hidden (latent) corpus structure. Once found, documents can be succinctly expressed in terms of this structure, queried for topical similarity and so on.

Gensim includes the following features:

• Memory independence – there is no need for the whole text corpus (or any intermediate term-document matrices) to reside fully in RAM at any one time.

• Provides implementations for several popular topic inference algorithms, including Latent Semantic Analysis (LSA, LSI) and Latent Dirichlet Allocation (LDA), and makes adding new ones simple.

• Contains I/O wrappers and converters around several popular data formats.

• Allows similarity queries across documents in their latent, topical representation.

The principal design objectives behind gensim are:

1. Straightforward interfaces and low API learning curve for developers, facilitating modifications and rapid prototyping.

2. Memory independence with respect to the size of the input corpus; all intermediate steps and algorithms operate in a streaming fashion, processing one document at a time.