
TODO list for ngram tools
-------------------------

BUGS
----
<04-06-11>  The command line options must be given in-order
	    see if getopt can be made not to care.
<08-06-11>  Massive memory requirements for huge corpora.
	    Make SQLite and incremental loading algorithm.


FEATURES
--------

* Check the frequency sum calculator method
    See if the output can be normalised
    Check the algorithm is accurate
* Add support for multiple comparison metrics
* Optimise frequency sum calculations
* Analysis methods:
    - ngram frequency
    - loglik
    - ngram rank
    - conditional probability (maybe the same as freq?)
    - bayesian classifier
    - SVM
    - logistic regression
* Support for large corpora using SQLite
* Fast support for medium corpora using incremental loading.
* CSV format instead of JSON for incremental loading
