Java Information Dynamics Toolkit (JIDT)

Copyright (C) 2012-2014 Joseph T. Lizier; 2014-2016 Joseph T. Lizier and Ipek Özdemir; 2017- Joseph T. Lizier, Ipek Özdemir and Pedro Mediano

JIDT provides a stand-alone, open-source code Java implementation (also usable in Matlab, Octave, Python, R, Julia and Clojure) of information-theoretic measures of distributed computation in complex systems: i.e. information storage, transfer and modification.

JIDT includes implementations:

• principally for the measures transfer entropy, mutual information, and their conditional variants, as well as active information storage, entropy, etc;
• for both discrete and continuous-valued data;
• using various types of estimators (e.g. Kraskov-Stögbauer-Grassberger estimators, box-kernel estimation, linear-Gaussian), as described in full at ImplementedMeasures.

JIDT is distributed under the GNU GPL v3 license (or later).

Getting started

1. Download and Installation is very easy!
   1. Quick start: take a git clone or download the latest v1.5 full distribution (suitable for all platforms) and see the readme.txt file therein.
2. Documentation including: the paper describing JIDT at arXiv:1408.3270 (distributed with the toolkit), a Tutorial, and Javadocs (v1.5 here);
3. Demos are included with the full distribution, including a GUI app for automatic analysis and code generation (see picture below), simple java demos and cellular automata (CA) demos.
   1. These Java tools can easily be used in Matlab/Octave, Python, R, Julia and Clojure! (click on each language here for examples)

For further information or announcements:

• Join our discussion group: http://groups.google.com/d/forum/jidt-discuss
• See also the FAQs
• Follow @infodynamicstkt on twitter

Citation

Please cite your use of this toolkit as:

Joseph T. Lizier, "JIDT: An information-theoretic toolkit for studying the dynamics of complex systems", Frontiers in Robotics and AI 1:11, 2014; doi:10.3389/frobt.2014.00011 (pre-print: arXiv:1408.3270)

And please let me know about any publications resulting from its use!

See other PublicationsUsingThisToolkit.

News

26/11/2018 - New jar and full distribution files available for release v1.5; Changes for v1.5 include: Added GPU (cuda) capability for KSG Conditional Mutual Information calculator (proper documentation to come), brief wiki page and unit tests included; Added auto-embedding for TE/AIS with multivariate KSG, and univariate and multivariate Gaussian estimator (plus unit tests), for Ragwitz criteria and Maximum bias-corrected AIS, and also added Maximum bias corrected AIS and TE to handle source embedding as well; Kozachenko entropy estimator adds noise to data by default; Added bias-correction property to Gaussian and Kernel estimators for MI and conditional MI, including with surrogates (only option for kernel); Enabled use of different bases for different variables in MI discrete estimator; All new above features enabled in AutoAnalyser; Added drop-down menus for parameters in AutoAnalyser; Included long-form lecture slides in course folder;

26/11/2017 - New jar and full distribution files available for release v1.4; Changes for v1.4 include: Major expansion of functionality for AutoAnalysers: adding Launcher applet and capability to double click jar to launch, added Entropy, CMI, CTE and AIS AutoAnalysers, also added binned estimator type, added all variables/pairs analysis, added statistical significance analysis, and ensured functionality of generated Python code with Python3; Added GPU (cuda) capability for KSG Mutual Information calculator (proper documentation and wiki page to come), including unit tests; Added fast neighbour search implementations for mixed discrete-continuous KSG MI estimators; Expanded Gaussian estimator for multi-information (integration); Made all demo/data files readable by Matlab.

17/12/2016 - New book out from J. Lizier et al., "An Introduction to Transfer Entropy: Information Flow in Complex Systems" published by Springer, which contains various examples using JIDT (distributed in our releases)

21/10/2016 - New jar and full distribution files available for release v1.3.1; Changes for v1.3.1 include: Major update to TransferEntropyCalculatorDiscrete so as to implement arbitrary source and dest embeddings and source-dest delay; Conditional TE calculators (continuous) handle empty conditional variables; Added new auto-embedding method for AIS and TE which maximises bias corrected AIS; Added getNumSeparateObservations() method to TE calculators to make reconstructing/separating local values easier after multiple addObservations() calls; Fixed kernel estimator classes to return proper densities, not probabilities; Bug fix in mixed discrete-continuous MI (Kraskov) implementation; Added simple interface for adding joint observations for MultiInfoCalculatorDiscrete Including compiled class files for the AutoAnalyser demo in distribution; Updated Python demo 1 to show use of numpy arrays with ints; Added Python demo 7 and 9 for TE Kraskov with ensemble method and auto-embedding respectively; Added Matlab/Octave example 10 for conditional TE via Kraskov (KSG) algorithm; Added utilities to prepare for enhancing surrogate calculations with fast nearest neighbour search; Minor bug patch to Python readFloatsFile utility.

19/7/2015 - New jar and full distribution files available for release v1.3; Changes for v1.3 include: Added AutoAnalyser (Code Generator) GUI demo for MI and TE; Added auto-embedding capability via Ragwitz criteria for AIS and TE calculators (KSG estimators); Added Java demo 9 for showcasing use of Ragwitz auto-embedding; Adding small amount of noise to data in all KSG estimators now by default (may be disabled via setProperty()); Added getProperty() methods for all conditional MI and TE calculators; Upgraded Python demos for Python 3 compatibility; Fixed bias correction on mixed discrete-continuous KSG calculators; Updated the tutorial slides to those in use for ECAL 2015 JIDT tutorial.

12/2/2015 - New jar and full distribution files available for release v1.2.1; Changes for v1.2.1 include: Added tutorial slides, description of exercises and sample exercise solutions; Made jar target Java 1.6; Added Schreiber TE heart-breath rate with KSG estimator demo code for Python.

28/1/2015 - New jar and full distribution files available for release v1.2; Changes for v1.2 include: Dynamic correlation exclusion, or Theiler window, added to all Kraskov estimators; Added univariate MI calculation to simple demo 6; Added Java code for Schreiber TE heart-breath rate with KSG estimator, ready for use as a template in Tutorial; Patch for crashes in KSG conditional MI algorithm 2.

20/11/2014 - New jar and full distribution files available for release v1.1; Changes for v1.1 include: Implemented Fast Nearest Neighbour Search for Kraskov-Stögbauer-Grassberger (KSG) estimators for MI, conditional MI, TE, conditional TE, AIS, Predictive info, and multi-information. This includes a general (multivariate) k-d tree implementation; Added multi-threading (using all available processors by default) for the KSG estimators -- code contributed by Ipek Özdemir; Added Predictive information / Excess entropy implementations for KSG, kernel and Gaussian estimators; Added R, Julia, and Clojure demos; Added Windows batch files for the Simple Java Demos; Added property for adding a small amount of noise to data in all KSG estimators;

15/8/2014 JIDT paper finalised and uploaded to the website and arXiv:1408.3270

14/8/2014 - New jar and full distribution files available for our first official release, v1.0; Changes for v1.0 include: Added the draft of the paper on the toolkit to the release; Javadocs made ready for release; Switched source->destination arguments for discrete TE calculators to be with source first in line with continuous calculators; Renamed all discrete calculators to have Discrete suffix -- TE and conditional TE calculators also renamed to remove "Apparent" prefix and change "Complete" to "Conditional"; Kraskov estimators now using 4 nearest neighbours by default; Unit test for Gaussian TE against ChaLearn Granger causality measurement; Added Schreiber TE demos; Interregional transfer demos; documentation for Interaction lag demos; added examples 7 and 8 to Simple Java demos; Added property to add noise to data for Kraskov MI; Added derivation of Apache Commons Math code for chi square distribution, and included relevant notices in our release; Inserted translation class for arrays between Octave and Java; Added analytic statistical significance calculation to Gaussian calculators and discrete TE; Corrected Kraskov algorithm 2 for conditional MI to follow equation in Wibral et al. 2014.

20/4/2014 - New jar and full distribution files available for v0.2.0; Moved downloads to http://lizier.me/joseph/ since google code has stopped the download facility here :(. Changes for v0.2.0 include: Rearchitected (most) Transfer Entropy and Multivariate TE calculators to use an underlying conditional mutual information calculator, and have arbitrary embedding delay, source-dest delay; this includes moving Kraskov-Grassberger Transfer Entropy calculator to use a single conditional mutual information estimator instead of two mutual information estimators; Rearchitected (most) Active Information Storage calculators to use an underlying mutual information calculator; Added Conditional Transfer Entropy calculators using underlying conditional mutual information calculators; Moved mixed discrete-continuous calculators to a new "mixed" package; bug fixes.

11/9/2013 - New jar and full distribution files available for v0.1.4; added scripts to generate CA figures for 2013 book chapters; added general Java demo code; added Python demo code; made Octave/Matlab demos and CA demos properly compatible for Matlab; added extra Octave/Matlab general demos; added more unit tests for MI and conditional MI calculators, including against results from Wibral's TRENTOOL; bug fixes.

11/9/2013 - New CA demo scripts for several review book chapters we're preparing in 2013 have been uploaded - see CellularAutomataDemos.

4/6/2013 - Added instructions on how to use in python and several PythonExamples.

13/01/2013 - New jar and full distribution files available for v0.1.3; existing Octave/Matlab demo code made compatible with Matlab; several bug fixes, including using max norm by default in Kraskov calculator (instead of requiring this to be set explicitly); more unit tests (including against results from Kraskov's own MI implementation)

19/11/2012 - New jar and full distribution files available for v0.1.2, including demo code for two newly submitted papers

31/10/2012 - Jar and full distribution files available for v0.1.1 (first distribution)

7/5/2012 - JIDT project created and code uploaded

Acknowledgements

This project has been supported by funding through:

• Australian Research Council Discovery Early Career Researcher Award (DECRA) "Relating function of complex networks to structure using information theory", J.T. Lizier, 2016-19 DE160100630
• Universities Australia - Deutscher Akademischer Austauschdienst (German Academic Exchange Service) UA-DAAD Australia-Germany Joint Research Co-operation grant "Measuring neural information synthesis and its impairment", Wibral, Lizier, Priesemann, Wollstadt, Finn, 2016-17