Helpful AI should not quietly erode human capability

Immediate performance and independent capability are different outcomes

A user can complete more work with AI and still become less able to evaluate or perform that work independently. Conversely, a user can take longer during a learning phase and become more capable later.

Measure the outcome you actually need.

For an emergency response, direct and immediate guidance may be appropriate. For a novice learning a critical skill, a direct finished answer may remove the reasoning practice that the organization wants to develop. For an experienced professional under time pressure, a concise draft with evidence may improve throughput without materially reducing competence.

Common metrics such as time on task, number of clicks, and immediate completion rate do not show:

• Whether the user can repeat the task without assistance
• Whether knowledge transfers to a new case
• Whether the user can detect a wrong AI answer
• Whether confidence is calibrated to actual understanding
• Whether persistence changes when the tool is unavailable
• Whether expertise is being built or merely bypassed

Emerging evidence deserves careful interpretation

The Liu and coauthors preprint describes randomized controlled trials across mathematical reasoning and reading-comprehension tasks. Participants with AI support performed better while assisted, then performed worse and gave up more often when assistance was removed. The effects appeared after brief exposure in the experimental setting. [S1]

Important limitations include:

• The work is a preprint rather than a settled body of replicated longitudinal evidence.
• The tasks do not represent every enterprise workflow.
• A short experiment cannot establish effects over months or years.
• Different interface designs may produce different outcomes.
• Expert users may respond differently from novices.
• Direct answers, critique, hints, collaboration, and automation are not equivalent forms of assistance.
• Accessibility needs can make reduced friction essential rather than harmful.

A broader research discussion on protecting cognition in the age of AI also notes that much of the literature is short-term and that definitive longitudinal conclusions are difficult. [S2]

The appropriate conclusion is therefore modest: some forms of frictionless AI assistance may create a tradeoff between immediate performance and later independent performance in some contexts. Teams should test for that possibility when human capability matters.

Productive struggle is not arbitrary difficulty

Productive struggle means the user performs meaningful reasoning with support calibrated to the task. It does not mean hiding information, delaying safety guidance, or forcing inaccessible interaction.

Useful difficulty has three properties:

1. It supports the intended skill or judgment.
2. It remains achievable with appropriate support.
3. It provides feedback that helps the user improve.

Arbitrary friction does none of those. Examples include confusing navigation, inaccessible forms, vague error messages, and repetitive administrative steps. Those should be removed.

The design question is not "How much friction can we add?" It is "Which part of the task should remain cognitively owned by the human?"

Use an assistance ladder

A practical interface can offer levels rather than one default response.

Level 0: Observe

The system records context or provides a neutral workspace without generating an answer.

Level 1: Ask

The system asks the user to state the goal, assumptions, or first interpretation.

Level 2: Hint

The system points to a relevant concept, source, or area of concern without completing the task.

Level 3: Scaffold

The system provides a structure, checklist, partial example, or sequence of questions.

Level 4: Draft

The system produces a proposed answer with sources, limitations, and required review.

Level 5: Execute under authority

The system performs a bounded approved action with explicit authorization and audit evidence.

Not every system needs every level. The value is in making the choice explicit and testable.

Require a first attempt when the goal is learning or judgment

A first-attempt pattern can preserve user reasoning:

1. Ask the user to state an initial answer or plan.
2. Ask for confidence and evidence.
3. Provide targeted critique or missing considerations.
4. Let the user revise.
5. Reveal a fuller model answer only when needed.
6. Ask the user to explain the final decision in their own words.

This pattern is useful for code review, architecture decisions, policy interpretation, training, and professional development. It is less appropriate when delay creates safety risk or the user's disability makes the first-attempt requirement an unnecessary barrier.

Use Socratic prompts selectively

Socratic interaction can help a user examine assumptions:

• What evidence supports this conclusion?
• Which case would make it false?
• What is unknown?
• Who is affected if this is wrong?
• Which part is policy and which part is observed behavior?
• What would you test before release?

Do not turn every interaction into an interrogation. Experienced users may need direct assistance. The system should allow the user or workflow policy to choose an appropriate mode.

Direct answers are sometimes the responsible design

Direct assistance is appropriate when:

• Safety or incident response requires speed
• The task is purely clerical and does not build a needed capability
• The user already demonstrates expertise
• Accessibility accommodations require reduced physical or cognitive burden
• The correct answer is standardized and low consequence
• The system is acting as a reference rather than a teacher
• Delay would create more harm than reduced learning

Accessibility is a core constraint. WCAG 2.2 describes requirements for perceivable, operable, understandable, and robust web content. [S3] A capability-preserving design must not use "productive struggle" as a rationale for inaccessible controls, excessive memory demands, or barriers to assistive technology.

Add reflection and verification

After AI assistance, ask the user to verify rather than merely accept.

Useful prompts include:

• Which source supports the key claim?
• What did you change from the draft?
• Which limitation remains?
• What would make you escalate this case?
• Can you explain the decision without reading the generated answer?
• Which part should be tested independently?

For code assistants, require tests, review, and explanation of security or data implications. For enterprise copilots, show source context and ask users to confirm whether the source is current. For reviewer tools, record disagreement and override reasons.

Measure independence and transfer

A capability-aware evaluation can include:

• Performance with assistance
• Performance after assistance is removed
• Transfer to a new but related task
• Error-detection ability
• Calibration between confidence and correctness
• Persistence after a difficult case
• Quality of user-authored reasoning
• Frequency of blind acceptance
• Need for escalation
• Accessibility and user burden

Do not optimize a single metric. Faster completion can be valuable, but it should be balanced against quality, learning, reviewer workload, and consequence.

Implications for enterprise systems

Enterprise copilots

Allow direct answers for routine lookup, but expose sources, freshness, and access boundaries. Use a staged mode for policy interpretation or high-consequence decisions.

Code assistants

Encourage user-authored intent, tests, and review. Treat generated code as proposed work. Measure whether engineers can explain and maintain the result.

Learning systems

Use hints, scaffolded steps, retrieval practice, and delayed reveal. Preserve accommodations and allow educators to choose learning goals.

Reviewer tools

Show evidence before conclusions, support disagreement, and avoid anchoring reviewers with an unexplained confidence score.

Operational automation

When the purpose is reliable execution rather than learning, direct automation may be appropriate, but human authority and failure handling still matter.

Practical design checklist

• [ ] Identify whether the primary goal is speed, safety, learning, judgment, or execution.
• [ ] Classify user expertise and accessibility needs.
• [ ] Map the consequence of a wrong answer or delayed action.
• [ ] Choose an assistance level rather than defaulting to a complete answer.
• [ ] Require a first attempt only where it supports the goal and remains accessible.
• [ ] Use hints, scaffolds, critique, and staged reveal for capability-building tasks.
• [ ] Provide direct answers where safety, urgency, expertise, or accommodation warrants them.
• [ ] Show sources, limitations, and unknowns.
• [ ] Add reflection or verification after consequential assistance.
• [ ] Measure independent performance, transfer, calibration, and persistence where relevant.
• [ ] Monitor blind acceptance and reviewer over-reliance.
• [ ] Review the design with affected users rather than assuming one assistance mode fits everyone.

Decision connection

A responsible AI interface should be clear about what it is optimizing. If the organization needs durable human judgment, the design should not measure only how quickly AI completes the task.

The Human Capability Safeguards for AI-Assisted Work helps teams choose an assistance level. The human-reviewed AI workflow capability addresses authority, evidence, and execution boundaries for consequential work.