Kolmogorov complexity
Noun
Meaning
Kolmogorov complexity (countable and uncountable, plural Kolmogorov complexities)
(computing theory) The complexity of an information object—such as a book or an image—informally defined as the length of the shortest program that produces that information object as output.
Examples
An example of score function include minimal compression length where a hypothesis with a lowest Kolmogorov complexity has the highest score and is returned.
C.S. Wallace and D.L. Dowe (1999) showed a formal connection between MML and algorithmic information theory (or Kolmogorov complexity).
It can be shown that the Kolmogorov complexity of any string cannot be more than a few bytes larger than the length of the string itself.
More precisely, the Kolmogorov complexity of the output of a Markov information source, normalized by the length of the output, converges almost surely (as the length of the output goes to infinity) to the entropy of the source.
Most strings are incompressible, i.e. their Kolmogorov complexity exceeds their length by a constant amount. 17 compressible strings are shown in the picture, appearing as almost vertical slopes.
Objective razor The minimum instruction set of a universal Turing machine requires approximately the same length description across different formulations, and is small compared to the Kolmogorov complexity of most practical theories.