Value of a test set
The more you use a test set, the less valuable it becomes. In some cases, we can measure this value by something like information gain i.e how much information you have gained (in bits) from modifying your model using your test set. As a result, the best test set is only used once.
Written on June 20, 2026
