Delegated Cognition and Autonomy Boundaries
SUMMARY
The boundary between cognitive-load reduction, supportive developmental guidance, dependency, and covert behavioral control.
DETAIL
The system reduces cognitive load by handling retrieval, routine decisions, task decomposition, contextualization, and pattern detection. This can preserve human attention for creativity, judgment, relationships, and difficult reasoning. Delegation becomes harmful when the user loses the ability to inspect, contest, or operate without the orchestration layer.
Every act of prioritization is also an act of steering. Recommendations, graph prominence, reminders, omitted alternatives, collaboration suggestions, and interpretations of identity all influence what the user notices and becomes more likely to do.
Supportive steering is legible and reversible. It exposes the objective being optimized, the evidence used, the alternatives considered, and the expected tradeoff. The user can narrow the scope, refuse the intervention, or change the objective. Manipulative steering hides its purpose, exploits behavioral vulnerabilities, increases engagement against long-term interests, or gradually removes meaningful alternatives.
Consent must be granular. Permission to organize notes does not imply permission to infer psychological traits. Permission to plan study sessions does not imply permission to reshape relationships, health behavior, or identity. Consent should be revisited when a transformation crosses domains or becomes more consequential.
Autonomy is not equivalent to requiring the user to make every low-level decision. A well-governed system can automate routine choices, reduce decision fatigue, and help maintain health or workload boundaries. The important distinction is whether delegation expands the user's effective agency or transfers control to objectives the user cannot see.
Dependency risk should be evaluated through resilience. The system should preserve comprehensible summaries, exportable structures, recoverable reasoning, manual override, and the possibility of reduced-function operation. Augmentation has succeeded when the user gains greater clarity and capacity, not merely when the system becomes indispensable.
WHY THIS EXISTS
Supports governance, consent design, explainability, wellbeing constraints, behavioral intervention, dependency analysis, and human-AI control allocation.
SOURCE CONTEXT POINTERS
- /concepts/ai-orchestrated-personal-development-operating-system/RISKS_AND_CONTRADICTIONS.txt
- /concepts/ai-orchestrated-personal-development-operating-system/DEEP.txt
- /concepts/ai-orchestrated-personal-development-operating-system/PRODUCT_BUSINESS.txt
EVIDENCE QUESTIONS
- No evidence query recorded