Stop AI from Doing the Work: Graduated Hints for Better Learning in Coding Workshops

I've always liked the idea of giving workshop participants an AI coding assistant. Not because it makes the workshop easier to run, but because "learn to work with AI" is a useful skill on its own. But I recently learned the hard way that an unrestricted assistant in a learning environment does more harm than good.

I ran a 3-hour hands-on workshop with 21 data engineers. Four notebooks, 12 exercises, Snowflake CoCo in the sidebar. Within five minutes of the first exercise, half the room had clicked "Fix the error" and the AI gave them the complete solution. Exercise done. Nothing learned.

The research agrees: don't give the answer

A CHI 2025 paper (DBox) tested this with programming students and LLMs. The group that got graduated support, concept explanations first, then code skeletons, then full solutions, performed better than the group with unrestricted chat. This isn't new. Vygotsky's Zone of Proximal Development says the same thing without the LLM: there's a sweet spot between "can do it alone" and "needs the answer." A good tutor lives in that gap.

Three levels, escalating on repeat asks

I settled on a hint ladder with three levels. The AI only escalates when the participant asks again:

Level	What they see	When
L1	"This function takes a file reference and extracts text. Use TO_FILE() to point to your stage."	First ask
L2	AI_PARSE_DOCUMENT(TO_FILE('@stage', '<path>'), {'mode': 'LAYOUT'})	Second ask
L3	Complete working SQL with explanation	Third ask, or "just show me"

No full answer on first ask. No executing code for them. No rewriting their solution during validation, just feedback in plain language.

Not every cell is an exercise

Here's what took a few iterations: workshop notebooks have infrastructure cells too. DDL, setup, demo queries. If someone hits a permission error on a setup cell, coaching them through it wastes time. The tutor needs to tell the difference.

The rule is simple:

• Cells with -- YOUR CODE HERE markers are exercises. Coach with hints.

• Everything else gets fixed directly.

When a participant clicks "Fix" in Snowsight, the request routes through the tutor. The tutor reads the cell, decides if it's an exercise, and acts accordingly. From the participant's perspective it's all seamless. They don't notice the AI is deliberately withholding answers on the cells that matter.

Automating it: $create-tutor

Writing the tutor instructions by hand for 12 exercises (3 hint levels each, plus triage logic) got old fast. So I built a meta-skill that generates the whole thing from existing workshop content.

$create-tutor explores a workshop repo with the same Glob/Grep/Read pattern that powers Cortex Code itself. It finds exercises, matches them to solutions, and produces:

No folder conventions. Multi-notebook, single-notebook, SQL worksheets, whatever. If the structure is unclear, it asks.

There's also an EVOLVE mode: change your exercises, re-run, and only the affected hints get regenerated.

Passive progress tracking with QUERY_TAG

One more thing that made the workshop work. Each exercise cell sets a session tag before the participant writes code:

ALTER SESSION SET QUERY_TAG = 'sds26:hol_02:ex3_search';
-- YOUR CODE HERE

A [[Streamlit]] admin dashboard queries INFORMATION_SCHEMA.QUERY_HISTORY and shows a heatmap of who's on which exercise. No submit button, no API calls from the notebook. Participants don't notice it. They just run their cell.

As the instructor, I could see who was stuck on Exercise 5 and walk over. No need to ask "how's everyone doing?"

How it went

At SDS Zurich: 21 participants, 12 exercises across 4 notebooks. The tutor infrastructure was generated from $create-tutor in one session. The admin dashboard showed progress and AI usage in real time. People got coaching instead of answers, and the workshop ran without anyone getting stuck silently in a corner.

The framework is open source:

/skill add https://github.com/anthu/agent-skills.git/create-tutor

What I'd do differently

The hint ladder works well for SQL exercises where there's a clear right answer. For open-ended tasks (design a schema, write a prompt) it's less useful. I'm still figuring out what graduated support looks like when the exercise doesn't have a reference solution.

Also: 3 levels might not always be the right number. Some exercises only need L1 and L3. Others could use an L1.5. The current framework is rigid about the three levels, and I'd like to make that configurable.

If you run workshops with AI assistants and want the teaching to survive the AI's helpfulness, give the hint ladder a try. I'd be curious what you find.