The Ugly Duckling Theorem
The ugly duckling theorem, introduced by Satosi Watanabe in 1969, is a fundamental result in pattern recognition and classification theory. It states that any two objects are equally similar to each other if all possible Boolean predicates (properties) are considered without bias.
Key Concepts
Binary Features
Each object is described by n binary features (0 or 1), yielding 2^n possible Boolean predicates.
Boolean Predicates
A predicate is any Boolean function of the features: individual features, AND, OR, XOR combinations, etc.
Equal Similarity
When all 2^n predicates are weighted equally, any two distinct objects share exactly 2^(2^n - 1) predicates.
Implications
The theorem implies that classification and similarity judgments always require a bias -- some features or predicates must be considered more important than others. Without such a bias, a duckling is just as similar to a swan as two swans are to each other.
Why It Matters
- Shows that similarity is not an objective measure but depends on chosen features.
- Fundamental to understanding inductive bias in machine learning.
- Explains why feature selection is critical in pattern recognition.
- Related to the "no free lunch" theorems in optimization and learning.