Wiki · Theme
ai-limits
4 notes tagged with this theme.
All themes
Pattern AI's most dangerous failure mode is confident wrongness AI's most dangerous failure is not silence but fluent, authoritative output that is wrong — making error detection a skilled, human task that cannot be deferred to the tool. Pattern Context rot As AI-generated content feeds back into the organisation's context — documents, transcripts, summaries — today's hallucinations become tomorrow's training data, and the quality of the context degrades over time unless the cycle is actively broken. Heuristic Users assume AI has access to information it does not have Users routinely overestimate the information AI has access to, treating it as if it were working from a complete picture; this overestimate compounds with AI fluency to produce misplaced trust. Heuristic Useful AI is a context problem The difference between useful AI and dangerous AI is almost entirely about the context it has; output quality is bounded above by input quality.