Let Cully improve itself

Cully's own evaluation runs on a schedule. When a check fails, Cully proposes a change to itself. Nothing is written until you approve it. This is separate from the advisor, which watches a session in your project and, when you accept a suggestion, changes that project with cully apply. Self-improvement changes Cully's own guidance and nothing in your projects.

cully improve                      # open the window: select, read, apply or drop
cully improve --list               # print the last eval, trend and proposals instead
cully improve --run                # evaluate now instead of waiting
cully improve --apply 1 --dry-run  # preview what proposal 1 would write
cully improve --apply 1            # approve it; Cully writes it
cully improve --dismiss 1          # skip it

The window

In a terminal, cully improve opens a window listing the suggestions. Select one with the arrow keys or a click and press Enter, or click its title, to read the failing checks behind it. Then:

Key	Does
a or Apply	Asks your model for a plan and shows it. y writes it; n keeps the suggestion pending.
d or Drop	Asks first. Only y drops it, and a dropped suggestion stays dropped.
Enter, b or Back	Opens or closes the details.
q or Esc	Goes back, then quits.

Looking at a suggestion, cancelling an apply, a failed plan and quitting never drop anything. Only a confirmed Drop does. Without a terminal, or with --list, cully improve prints the proposals, and --apply N and --dismiss N still work in scripts.

How it works

1. The advisor daemon runs cully eval for the setup, harness, improve, tasks and advisor suites every five hours, twice each so a flaky check is reported. Set CULLY_EVAL_INTERVAL (for example 12h, never below 30 minutes) to change the interval, or off to disable it. The last run is remembered across restarts. The memory suite needs a server and is not scheduled.
2. Each run is saved to eval-history.jsonl with counts and the names of failing checks, never contents. cully improve shows the trend in failing checks over recent runs.
3. A failing suite becomes one pending proposal. The same failure is not proposed twice. A failure that goes away resolves its proposal, and an applied change that did not hold reopens it. A dismissed proposal stays dismissed. cully suggestions mentions pending proposals.
4. When you press Apply (or run cully improve --apply N), Cully asks a coding agent to plan a change from the failing checks: the agent running Cully (CULLY_AGENT) when its CLI is installed, otherwise the first installed of Claude Code, Codex and Cursor. It shows the plan, and writes it only after you confirm (or pass --yes).

What a change may write

An approved change can add rules to ~/.cully/skills/cully/SKILL.md and install one small skill under ~/.cully/skills/. The coding agents you have set up read those files.

It never adds MCP servers or runs commands: if the planner proposes either, Cully ignores them and says so. It refuses a skill name that is not a plain folder name, anything that looks like a credential, and plans over 12 KiB. It cannot change Cully's code. When the cause is a code defect, the plan's notes say so and the proposal stays pending.

Hooks are not written automatically: a hook runs in every session, so adding one stays a decision you make through cully setup.

How it is checked

The improve suite in cully eval tests this loop against planted failures: one proposal per failure, no duplicates, resolved and reopened proposals, dismissal, skipped suites, and the approval gate. Declining and a dry run must write nothing, and an approval must add no MCP server. The Go tests also confirm that the suite fails against a loop that never proposes, duplicates, never resolves or lets everything through.

What this does not prove

• A passing improve suite shows the loop is safe and keeps its books straight. It does not show that a proposal improves Cully. The next scheduled run does: the failing-check count in cully improve should fall after a change.
• The planner is a model, so two plans for the same failure can differ. You review each one before it is written.
• Evaluations that read local history (advisor replay follows the running Claude Code or Codex agent; task measurements still use Claude history) report without ground truth and never fail a run, so they never create proposals. The tasks suite also compares time before and after each applied change, which shows whether a change helped; it is uncontrolled and informs you without creating a proposal.