Sketch of the proof for information theoretic limitations for self-reflection in large language models. Tl;dr you can't bound the extent to which the output remains in distribution, thus you can't bound drift.
This page may contain third-party content, which is provided for information purposes only (not representations/warranties) and should not be considered as an endorsement of its views by Gate, nor as financial or professional advice. See Disclaimer for details.
Sketch of the proof for information theoretic limitations for self-reflection in large language models. Tl;dr you can't bound the extent to which the output remains in distribution, thus you can't bound drift.