Cognitive Scaffold

Preparing your thinking workspace

arrow_back_ios_new
MENTAL MODEL · M7564

Information Entropy

Information Entropy
TechnicalHigh supportResearch Methods
Included
account_tree

Version 1.0.0 · Updated 2026-07-28

CORE DEFINITION

In information theory, the entropy of a random variable quantifies the average level of uncertainty or information associated with the variable's potential states or possible outcomes. This measures the expected amount of information needed to describe the state of the variable, considering the distribution of probabilities across all potential states. Given a discrete random variable X.

SCAFFOLDING EFFECT

psychology

Reduce cognitive load

In information theory, the entropy of a random variable quantifies the average level of uncertainty or information associated with the variable's potential states or possible outcomes. This measures the expected amount of information needed to describe the state of the variable, considering the distribution of probabilities across all potential states.

anchor

Anchor fast decisions

Shannon defined entropy as H = -Σ p(x) log p(x), which measures the average uncertainty of a random variable. The more uniform the distribution (each outcome has similar probability), the higher the entropy and the greater the uncertainty; the more concentrated (one outcome has high probability), the lower the entropy. The minimum average amount of information needed to eliminate this uncertainty equals the entropy. It is typically measured in bits (log base 2).

MINIMUM ACTION

In progress 0/5

Practice this model in one real situation:

Check to track your progress (stored locally)
Learning progress0%
account_treeGenealogyexpand_more
menu_bookReferencesexpand_more

Source support: Explicit

  • link
    en.wikipedia.orghttps://en.wikipedia.org/wiki/Entropy_%28information_theory%29ZH · Explicit
    verified

RELATED MODELS

Related model

Kolmogorov Complexity

In algorithmic information theory (a subfield of computer science and mathematics), the Kolmogorov complexity of an object, such as a piece of text, is the length of a shortest computer program (in a predetermined programming language) that produces the object as output. It is a measure of the computational resources needed to specify the object.

arrow_forwardRelated model

Stochastic Resonance

Stochastic resonance (SR) is a mathematical mechanism and behavior of nonlinear systems (that is, systems in which the change of the output is not proportional to the change of the input) where random

arrow_forwardRelated model

5W1H Method

A checklist of six questions, What, Why, Who, When, Where, and How, used to describe an event or problem completely enough that no essential dimension is left unexamined.

arrow_forward