Skip to content

Engineering Design: Cross-Scene Directorial Preference Memory

0. System Diagram

flowchart TD
  A[guruChatSessions<br/>existing raw chat + tool calls] --> B[Data Harvester]
  C[Final scene state<br/>shots, key beats, filler strategy] --> B
  D[Future manual edit diffs<br/>dialogue edits, UI edits] -. later .-> B

  B --> E[directorialSceneMemories<br/>one compact memory per ready scene]
  E --> F[Preference Aggregator]
  F --> G[directorialPreferenceProfiles<br/>one current aggregate profile per story]

  G --> H[Runtime Preference Block<br/>calculated live, not stored]
  C --> H
  I[Current user steering] --> H

  H --> J[WorkbenchSceneGuru.getContext]
  H --> K[Planning prompt blocks]

  K --> L[recommend_key_beats]
  K --> M[recommend_filler_strategy]
  K --> N[budget_beat_packages]
  K --> O[generate_beat_package]
  K --> P[propose_edit_plan]

  L --> Q[New scene plan / shots]
  M --> Q
  N --> Q
  O --> Q
  P --> Q

  Q --> A
  Q --> C

1. Executive Summary

We will store preference memory separately from storyVideos. That is assumed throughout this design.

The system has three parts:

Part Responsibility Stored?
Data Harvester Reads guruChatSessions, toolCall.data, and final scene state; extracts a compact factual summary of what changed. Yes, in directorialSceneMemories
Preference Aggregator Looks across scene memories and produces a concise set of story-level preference cards. Yes, in directorialPreferenceProfiles
Runtime Consumption Converts the current aggregate profile into short prompt blocks for the scene guru and planning tools. No, calculated live

The raw source of truth remains guruChatSessions. We do not copy raw chat history or full toolCall.data into preference memory. The harvester consumes that verbose data and stores compact derived memory.

The design intentionally avoids a heavy ontology. Instead of storing many rigid tags like powerDynamic, affectedStages, riskOfOvergeneralizing, and unresolvedConflicts, we store a few natural-language fields that LLMs can reason over:

sceneContextSummary
editSummary
sceneLearnings
profileSummary
preferenceCards

The goal is to make the preference memory useful without turning it into a large rules engine.


2. What Changed from the Previous Version

Previous idea Updated decision Why
sourceFingerprint hash on profiles Removed. Store the list of source scene/memory ids instead. A hash is useful for cache correctness, but v1 can be event-driven: when scene memory changes, rebuild and overwrite the profile. The source list is easier to inspect and debug.
version: 1 on documents Removed. Use createdAt and updatedAt. We do not currently need multiple stored schema versions. If a migration is needed later, add a migration strategy then.
sceneFingerprint Removed. We can re-run the harvester when a ready scene changes. No need to store a hash until we actually need cache-diffing.
status: fresh/stale/partial/failed Removed. Do not store failed documents. Reharvest overwrites old memory. Use logs for failures.
diagnostics object Removed from primary schema. Debug data should go to logs/observability. It does not need to be part of the product-facing memory object.
unresolvedConflicts Removed. The aggregator should either resolve conflicts into conditional preference cards or omit weak/contradictory claims.
Complex SceneContextSummary object Replaced with sceneContextSummary: string. The downstream LLM can reason over a few sentences. We do not need a large scene taxonomy in v1.
Complex DirectorialLearning type Replaced with small SceneLearning objects. Store what we learned in normal language, with concise evidence and confidence.
Stored directorialScenePreferenceCapsules Removed. The current-scene preference block is cheap and context-dependent; calculate it live from the profile.
Synthetic/director/persona packs Out of scope. Useful later, but too much scope for this iteration.
Exhaustive tool mapping Removed. Planning tools will receive preference cards plus tool-specific examples in their prompts. The tool LLM decides how to use them.

3. Goals and Non-Goals

3.1 Goals

Goal Description
Learn from real user behavior Use changed, removed, inserted, reordered, merged, and accepted shots from completed scenes.
Keep memory compact Store summaries and preference cards, not raw chats or full shot snapshots.
Avoid overgeneralization Learn conditional preferences, not shallow global rules.
Preserve enough context Keep a short scene context summary so preferences can be interpreted correctly.
Avoid blocking UX Preference harvesting and aggregation are non-blocking background work.
Feed tools simply Pass concise learned preference blocks into existing planning prompts.
Keep storage simple Only two new durable collections in v1.

3.2 Non-goals for v1

Non-goal Reason
Synthetic/director/persona packs Useful later, but excluded from this design.
Global cross-project preference memory Start with story-scoped memory. Cross-project may come later.
Manual dialogue edit learning Design leaves space for it, but v1 focuses on guru/tool-call-based shot planning.
Stored runtime capsules Calculated live.
Complex preference ontology Use natural language preference cards and prompt examples instead.
Asking the user every scene whether to use preferences Apply softly by default. Ask only for meaningful creative forks.

4. Durable Collections

V1 adds only two collections.

Collection Granularity Purpose
directorialSceneMemories One document per storyId + sceneId Compact memory of what happened in one ready scene. preferenceTriggerUserId is metadata (who last triggered harvest).
directorialPreferenceProfiles One document per storyId Current aggregate preference profile for the story. A story has ONE directorial vision — collaborative edits converge.

No directorialScenePreferenceCapsules collection in v1.


5. Simplified Keys

5.1 Key philosophy

Use the smallest keys that support the product:

preferenceTriggerUserId + storyId + sceneId

for scene memories, and:

preferenceTriggerUserId + storyId

for aggregate profiles.

projectId can be added if storyId is not globally unique or if permission queries require it, but it should not be part of the core design unless necessary.

5.2 Why no hash?

A profile can store the actual source list:

sourceSceneIds: string[];
sourceMemoryIds: string[];

That is more understandable than a hash.

In v1, the profile is not a complex cache keyed by source state. It is simply the current best aggregate profile for a story (story-scoped, not user-scoped — a story has one directorial vision). When a scene memory changes, enqueue aggregation and overwrite the profile.

Approach Pros Cons V1 decision
Hash/fingerprint cache key Strong cache correctness, supports many historical profiles. More schema complexity and harder debugging. Do not use.
Source id list + overwrite current profile Simple, readable, easy to reason about. Less precise cache invalidation. Use.

5.3 Why no document version?

Do not add version: 1 unless we plan to store multiple versions or run migrations now.

Use:

createdAt: Date;
updatedAt: Date;

If we later need schema migrations, add a schemaVersion during that migration.


6. Collection: directorialSceneMemories

One document represents the harvested memory of one ready scene.

6.1 Document shape

interface DirectorialSceneMemoryDocument {
  _id: ObjectId;

  preferenceTriggerUserId: string;
  storyId: string;
  sceneId: string;
  sceneIndex?: number;

  source: {
    chatSessionId?: string;
    chatSessionUpdatedAt?: Date;
    planningStageAtHarvest: 'READY_FOR_SKETCHES';
    harvestedAt: Date;
  };

  /**
   * 2-5 sentences describing what kind of scene this was.
   * Example: "Contained confrontation scene where Lena is publicly embarrassed..."
   */
  sceneContextSummary: string;

  /**
   * Human-readable summary of what changed.
   * Example: "The user removed two generic setup shots, changed several medium shots..."
   */
  editSummary: string;

  /**
   * Compact factual events extracted from tool calls.
   * This is not raw toolCall.data.
   */
  editEvents: CompactEditEvent[];

  /**
   * Compact summary of final accepted shot state.
   */
  finalShotSummary: FinalShotSummary;

  /**
   * LLM-distilled learnings from this scene.
   */
  sceneLearnings: SceneLearning[];

  createdAt: Date;
  updatedAt: Date;
}

6.2 CompactEditEvent

Keep edit events readable and small.

interface CompactEditEvent {
  type:
    | 'initial_generation'
    | 'modify'
    | 'insert'
    | 'remove'
    | 'reorder'
    | 'merge'
    | 'bulk_edit'
    | 'acceptance';

  summary: string;

  /** Optional nearby user message or rationale. */
  userText?: string;

  /** Optional shot ids or active shot numbers at the time of edit. */
  shotRefs?: string[];
}

Example:

{
  "type": "modify",
  "summary": "Changed Shot 4 from a medium neutral reaction to a closer emotional reaction on Lena.",
  "userText": "Can this feel more uncomfortable?",
  "shotRefs": ["shot_abc123"]
}

6.3 FinalShotSummary

This should be compact and mostly deterministic.

interface FinalShotSummary {
  shotCount: number;
  keyShotCount?: number;
  fillerShotCount?: number;

  avgShotSeconds?: number;
  avgShotsPerKeyBeat?: number;

  commonShotSizes?: string[];
  commonCameraAngles?: string[];
  commonCameraMovements?: string[];
  commonShotJobs?: string[];

  summary: string;
}

Example:

{
  "shotCount": 14,
  "keyShotCount": 5,
  "fillerShotCount": 9,
  "avgShotSeconds": 2.8,
  "commonShotSizes": ["CLOSE_UP", "MEDIUM_CLOSE_UP", "MEDIUM_SHOT"],
  "commonShotJobs": ["reaction_emotion", "relationship_pressure", "physical_detail"],
  "summary": "Final scene favors close emotional coverage and short reaction/consequence support around the confrontation."
}

6.4 SceneLearning

A scene learning should feel like a note a director’s assistant could read. It does not need a large enum taxonomy.

interface SceneLearning {
  learning: string;

  /** When this learning might carry forward. Free text, not a rigid tag array. */
  appliesTo?: string;

  /** When not to apply it. Free text. */
  avoidApplyingTo?: string;

  /** What in this scene supports the learning. */
  evidence: string;

  confidence: 'low' | 'medium' | 'high';
}

Example:

{
  "learning": "The user seems to prefer framing humiliation through the person absorbing the emotional cost, rather than through the aggressor's superiority.",
  "appliesTo": "Humiliation, shame, vulnerability, private realization, or power-imbalance beats where the affected character's internal reaction is the story point.",
  "avoidApplyingTo": "Fast action, pure geography setup, comic buttons, or scenes meant to feel emotionally detached.",
  "evidence": "The user changed several medium/wide reactions into closer coverage on Lena and removed one neutral dominance-facing shot.",
  "confidence": "medium"
}

This is much simpler than storing:

kind + affectedStages + riskOfOvergeneralizing + portable + eventIds + appliesWhen arrays

If we later need more filtering, we can add optional tags. V1 should start with human-readable cards.

6.5 Indexes

Index Unique? Purpose
{ preferenceTriggerUserId, storyId, sceneId } Yes Upsert/fetch memory for a scene.
{ preferenceTriggerUserId, storyId, updatedAt } No Load all scene memories for profile aggregation.

7. Collection: directorialPreferenceProfiles

One document is the current aggregate preference profile for a story (story-scoped — collaborative edits converge into one profile).

It is overwritten whenever aggregation runs successfully.

7.1 Document shape

interface DirectorialPreferenceProfileDocument {
  _id: ObjectId;

  preferenceTriggerUserId: string;
  storyId: string;

  source: {
    sourceSceneIds: string[];
    sourceMemoryIds: string[];
    aggregatedAt: Date;
  };

  /** 3-8 sentence overview of the user's current learned directorial style. */
  profileSummary: string;

  /** Main reusable preferences. */
  preferenceCards: PreferenceCard[];

  /** Optional compact numeric grounding for budget/shot generation. */
  statsSummary?: PreferenceStatsSummary;

  createdAt: Date;
  updatedAt: Date;
}

7.2 PreferenceCard

interface PreferenceCard {
  preference: string;
  appliesTo: string;
  avoidApplyingTo: string;
  evidence: string;
  confidence: 'low' | 'medium' | 'high';
}

Example:

{
  "preference": "When a beat turns on shame, grief, vulnerability, or private realization, favor emotionally legible coverage on the character absorbing the cost.",
  "appliesTo": "Human consequence beats, humiliation, grief, dread, vulnerability, power imbalance, private realization.",
  "avoidApplyingTo": "Fast action, pure spatial setup, broad comedy, or beats intentionally staged with emotional distance.",
  "evidence": "Observed across three ready scenes: repeated user edits moved coverage toward closer reactions and away from neutral dominance coverage.",
  "confidence": "medium"
}

7.3 PreferenceStatsSummary

Keep stats lightweight. They are used mostly by shot budgeting and shot generation.

interface PreferenceStatsSummary {
  summary: string;

  avgShotsPerScene?: number;
  avgShotsPerKeyBeat?: number;
  avgShotSeconds?: number;

  commonShotSizes?: string[];
  commonCameraAngles?: string[];
  commonCameraMovements?: string[];
  commonShotJobs?: string[];

  commonUserChangedFields?: string[];
  commonInsertedShotTypes?: string[];
  commonRemovedShotTypes?: string[];
}

Example:

{
  "summary": "Accepted scenes tend to be moderately dense, with extra support around emotional consequence beats rather than uniform filler expansion.",
  "avgShotsPerScene": 15.3,
  "avgShotsPerKeyBeat": 2.8,
  "avgShotSeconds": 2.9,
  "commonShotSizes": ["CLOSE_UP", "MEDIUM_CLOSE_UP", "MEDIUM_SHOT"],
  "commonUserChangedFields": ["shotSize", "description", "timing"],
  "commonInsertedShotTypes": ["reaction_emotion", "physical_detail"],
  "commonRemovedShotTypes": ["generic_establishing", "redundant_reaction"]
}

7.4 Indexes

Index Unique? Purpose
{ storyId } Yes Fetch current aggregate profile for this story (story-scoped).
{ storyId, updatedAt } No Debug/admin lookup.

8. Why We Do Not Store Runtime Capsules

A runtime preference capsule is just the current prompt-ready view of the profile.

It depends on:

Runtime input Why it changes
Current scene text User may revise the scene.
Current user steering Every chat turn can change intent.
Planning stage Key-beat planning and shot generation need different emphasis.
Current beat/package generate_beat_package may need beat-specific interpretation.
Prompt token budget Number of preference cards may vary per tool call.

Because of that, storing capsules creates invalidation complexity without much value.

Instead:

Stored profile + current scene/tool context → live preference prompt block

8.1 Runtime object shape

This object can exist in memory, but not as a DB collection.

interface RuntimePreferenceBlock {
  profileSummary: string;
  preferences: PreferenceCard[];
  statsSummary?: PreferenceStatsSummary;

  /**
   * Static prompt policy, not a learned preference and not stored in DB.
   * It tells downstream LLM calls how to rank learned preferences against
   * current user intent, scene facts, continuity, pacing, and tool constraints.
   */
  overridePolicy: string;

  usageExamples: string[];
}

8.2 overridePolicy definition

overridePolicy is not learned, aggregated, or stored. It is a fixed prompt-policy string attached to each runtime preference block so every downstream prompt interprets learned preferences the same way.

Recommended constant:

const PREFERENCE_OVERRIDE_POLICY = `
Learned preferences are soft priors, not requirements. Use them to improve the default plan only when they fit the current scene, beat, and user request.

Priority order:
1. Explicit current user instructions.
2. Script facts, continuity, character/location truth, and safety/feasibility constraints.
3. Current scene intent, act pacing, geography/blocking clarity, and confirmed planning artifacts.
4. Learned preferences from prior ready scenes.
5. Generic cinematic defaults.

Apply a learned preference only when its appliesTo condition matches the current scene or beat. Do not apply it when avoidApplyingTo matches. If a learned preference conflicts with the current user request or the scene's dramatic job, ignore the learned preference. If two valid interpretations remain and the preference would materially choose one, ask the user a targeted question instead of silently forcing it.
`;

In prose, this means the learned profile should influence defaults, not override the creative direction of the current scene.

8.3 When to reconsider storing capsules

Condition Better first move
Runtime formatting becomes expensive Add in-memory or Redis cache.
A future LLM selector is used on every scene entry Cache selector output temporarily.
Need audit of what influenced generated shots Store profileId/preferenceCardIds on generated artifacts, not a full capsule collection.

9. Source Eligibility

A scene becomes eligible as a preference source when it reaches READY_FOR_SKETCHES.

Scene status Use as source? Reason
Before READY_FOR_SKETCHES No Shot planning is not stable enough.
READY_FOR_SKETCHES or later Yes The scene has a completed/accepted shot plan.

This status is accumulative. Once a scene is ready for sketches, it can be harvested and used as source memory.

9.1 Future scenes are allowed as preference sources

Using future scenes as story context can leak plot information. But using future scenes as abstract preference evidence is fine if the aggregate profile does not pass future plot details into prompts.

Safe:

The user tends to favor human consequence shots during humiliation or vulnerability beats.

Unsafe:

In Scene 9, Lena is betrayed, so frame Scene 3 with that in mind.

Therefore the aggregator should sanitize profile cards so they describe preferences abstractly.

Internal source memory may contain Runtime prompt should contain
Scene ids Usually no, unless debugging.
Specific edit evidence Usually no.
Specific future plot details No.
Abstract conditional preference Yes.

10. When the Harvester Runs

The harvester should run as a non-blocking job when a scene reaches READY_FOR_SKETCHES.

Scene planningStage transitions to READY_FOR_SKETCHES
  → enqueue DirectorialPreferenceHarvestJob
  → upsert directorialSceneMemory
  → enqueue PreferenceAggregationJob
  → overwrite directorialPreferenceProfile

10.1 Trigger table

Trigger Action Blocking?
Scene reaches READY_FOR_SKETCHES Enqueue harvest. No
A ready scene is edited again Enqueue reharvest. No
Existing story has missing memories Backfill asynchronously. No
Scene guru opens and no profile exists Initialize without preferences and enqueue build. No
Scene guru opens and an older profile exists Use the older profile and enqueue rebuild if needed. No

10.2 Job flow

sequenceDiagram
  participant Scene as Scene State
  participant Queue as Job Queue
  participant Harvester as Data Harvester
  participant Chat as guruChatSessions
  participant Memory as directorialSceneMemories
  participant Aggregator as Preference Aggregator
  participant Profile as directorialPreferenceProfiles

  Scene->>Queue: planningStage becomes READY_FOR_SKETCHES
  Queue->>Harvester: harvest scene memory
  Harvester->>Chat: load scene-level session
  Harvester->>Harvester: snapshot-diff tool calls
  Harvester->>Harvester: summarize final shots
  Harvester->>Harvester: distill scene learnings
  Harvester->>Memory: upsert scene memory
  Harvester->>Queue: enqueue profile aggregation
  Queue->>Aggregator: aggregate all ready scene memories
  Aggregator->>Profile: overwrite current profile

10.3 Why not wait for later?

READY_FOR_SKETCHES is the right trigger because the shot plan is accepted enough to generate sketches. At that point the user's directorial interventions are useful for future planning. The harvest is non-blocking, so it does not slow down sketch generation or scene navigation.


11. Data Harvester Design

The harvester converts verbose scene history into compact evidence.

11.1 Harvester inputs

Input Existing? Purpose
Scene-level guruChatSessions Yes Tool calls, params, post-tool result snapshots, user messages.
toolCall.data Yes Actual post-manipulation shot state after each tool call.
Final scene.shots[] Yes Accepted final outcome.
confirmedKeyBeats Yes Final structural beat plan.
confirmedFillerStrategy Yes Final filler/support plan.
Manual dialogue edit diffs Later Dialogue preference learning.

11.2 Harvester outputs

Output Stored? Description
editEvents Yes Compact natural-language edit events.
editSummary Yes Short summary of what changed.
finalShotSummary Yes Compact stats and prose summary of final shots.
sceneContextSummary Yes Few sentences about what kind of scene this was.
sceneLearnings Yes Small learning cards distilled from the scene.
Raw toolCall.data No Used during extraction only.

11.3 Snapshot diff, not replay

Do not replay tool params against final state. Use actual post-tool states.

before snapshot = current known active shots
actual after snapshot = compact shot list extracted from toolCall.data
diff = compare before/after using shotId
event = summarize diff using toolCall.params + nearby user text
current snapshot = after snapshot

Key safety rule:

Resolve shot indexes only against the immediate before snapshot.
Track identity by shotId, not final shotIndex.

This avoids bugs after inserts and reorders.

11.4 Tool extraction table

Tool Harvester behavior
generate_scene_shots_from_plan Capture initial generated baseline. Not a user preference by itself.
modify_shot Use params to know intended fields; use before/after snapshots to summarize what changed.
insert_shot Inserted shots are afterIds - beforeIds. Summarize inserted shot purpose and neighbor context.
remove_shots Removed shots are beforeIds - afterIds. Summarize what type of shot was rejected.
reorder_shots Record moved shot ids and before/after adjacency. Do not count every renumbered shot.
merge_shots Summarize compression: what was merged and what survived.
execute_edit_plan Summarize high-level bulk changes.
confirm_shots Acceptance signal. Weak by itself, stronger after explicit edits.

11.5 Fields ignored in diffs

The harvester should ignore mechanical/system fields:

Ignore Examples
Ordering/system fields shotIndex, createdAt, updatedAt
Asset fields images, videos, sketches, generatedImageURL
Processing fields shotProcessorPrompts, needsRerender, dialogueAudio
Soft-delete mechanics isDeleted, except for active/final filtering

For modify_shot, count only fields explicitly requested in tool params. This avoids learning accidental side effects from validation or redistribution.


12. Scene Context: Keep It as Sentences

We do not need a large structured SceneContextSummary in v1.

Instead, store:

sceneContextSummary: string;

Examples:

Scene context summary Why it is enough
“Contained confrontation scene where Lena is publicly embarrassed and the dramatic focus is her loss of control.” LLM can infer humiliation, vulnerability, power imbalance.
“Fast escape sequence with high physical momentum and limited dialogue.” LLM can infer action-forward pacing and avoid applying slow emotional filler preferences.
“New-location scene where spatial orientation matters because the blocking sets up a later reveal.” LLM can infer establishing/geography may be necessary.
“Quiet exposition scene where the dramatic work is mostly subtext and withheld reaction.” LLM can infer restrained coverage may be appropriate.

If we later need reliable analytics or deterministic filtering, we can add optional tags. But v1 should use prose.


13. LLM Usage Overview

LLMs should articulate and aggregate preference meaning. They should not discover factual edits from raw verbose data.

Stage LLM? Purpose
Tool-call parsing No Structured params/data.
Snapshot diff No Deterministic and safer.
Final shot stats No Deterministic.
Scene context summary Yes or heuristic + LLM Convert scene facts into 2-5 useful sentences.
Per-scene learning distillation Yes Turn compact edit evidence into scene learnings.
Cross-scene aggregation Yes Merge scene learnings into preference cards.
Runtime prompt block construction No in v1 Format profile cards and prompt examples.
Tool interpretation Yes, inside existing planning LLM calls Each tool LLM decides how to use preferences in its own context.

14. LLM Prompt 1: Scene Memory Distiller

This prompt runs once per harvested scene. It receives compact evidence and returns a concise scene memory.

14.1 Input

Input Purpose
Scene description/script summary Lets the LLM understand the dramatic situation.
Act/scene intent Helps describe scene context.
Compact edit events Ground truth of what changed.
Final shot summary/stats Shows what survived.
Nearby user messages Adds intent/rationale.

14.2 Output

interface SceneMemoryDistillerOutput {
  sceneContextSummary: string;
  editSummary: string;
  sceneLearnings: SceneLearning[];
}

14.3 Prompt sketch

SYSTEM:
You create a compact memory of what the user appears to have preferred in one completed scene.

The edit events are factual ground truth. Do not invent edits.
Do not produce a large taxonomy. Write concise, practical notes.
Prefer conditional lessons over broad claims.
Do not turn continuity fixes or model mistakes into style preferences.
If a lesson seems too scene-specific, say so in the learning text or give it low confidence.
Return valid JSON only.

USER:
SCENE DESCRIPTION / SCRIPT SUMMARY:
{sceneSummary}

ACT / SCENE INTENT:
{actAndSceneIntent}

COMPACT EDIT EVENTS:
{editEvents}

FINAL SHOT SUMMARY:
{finalShotSummary}

SELECTED USER MESSAGES:
{selectedUserMessages}

Return JSON:
{
  "sceneContextSummary": "2-5 sentences describing what kind of scene this was and what dramatic problem it solved.",
  "editSummary": "3-8 sentences summarizing what the user changed, removed, inserted, reordered, accepted, or emphasized.",
  "sceneLearnings": [
    {
      "learning": "A concise preference or lesson inferred from this scene.",
      "appliesTo": "When this might carry forward.",
      "avoidApplyingTo": "When not to apply it.",
      "evidence": "What in this scene supports it.",
      "confidence": "low | medium | high"
    }
  ]
}

14.4 Example output

{
  "sceneContextSummary": "This was a contained humiliation scene. The external action was simple, but the dramatic point was Lena absorbing public embarrassment and losing control in front of Marcus.",
  "editSummary": "The user removed generic setup coverage and shifted several reactions closer to Lena. The final scene spends more coverage on her discomfort than on Marcus's dominance. The user also tightened a few shots so the scene does not feel over-covered.",
  "sceneLearnings": [
    {
      "learning": "The user may prefer humiliation beats to be framed through the person absorbing the emotional cost, rather than through the aggressor's superiority.",
      "appliesTo": "Humiliation, shame, vulnerability, private realization, or power-imbalance beats where the affected character's internal reaction is the story point.",
      "avoidApplyingTo": "Fast action, broad comedy, pure geography setup, or scenes intended to feel emotionally detached.",
      "evidence": "Several medium/wide reactions were changed into closer emotional coverage on Lena, and one neutral dominance-facing shot was removed.",
      "confidence": "medium"
    }
  ]
}

15. LLM Prompt 2: Cross-Scene Preference Aggregator

This prompt reads multiple directorialSceneMemories and creates the current story-level profile.

15.1 Input

Input Purpose
Scene context summaries Show the preconditions of each scene.
Edit summaries Show what changed.
Scene learnings Main semantic evidence.
Final shot summaries Numeric/style grounding.
Source scene count Helps confidence.

15.2 Output

interface PreferenceAggregatorOutput {
  profileSummary: string;
  preferenceCards: PreferenceCard[];
  statsSummary?: PreferenceStatsSummary;
}

15.3 Aggregation behavior

Evidence pattern Aggregator behavior
Same lesson appears in several scenes Promote into a stronger preference card.
Lesson appears only in similar scenes Keep it conditional.
Lessons seem contradictory but contexts differ Write a contextual preference, not a conflict object.
Lessons seem contradictory and cannot be explained Omit the claim or make it low-confidence and cautious.
Lesson is clearly scene-specific Do not promote.
Continuity/model-error correction Do not promote.

15.4 Prompt sketch

SYSTEM:
You aggregate directorial scene memories into a concise preference profile.

You are not reading raw chat history. You are reading compact memories that were already harvested from completed scenes.
Your job is to merge repeated or related learnings into practical preference cards.

Rules:
- Keep the output compact.
- Prefer conditional preferences over broad global claims.
- Do not create an unresolvedConflicts section. If evidence conflicts, either condition the preference carefully, lower confidence, or omit it.
- Do not include future scene plot details in preference cards. Make preferences abstract and reusable.
- Do not promote scene-specific, continuity, or model-error corrections.
- Evidence should be summarized without leaking specific plot details unless necessary.
- Return valid JSON only.

USER:
SCENE MEMORIES:
{sceneMemories}

Return JSON:
{
  "profileSummary": "3-8 sentences summarizing the user's learned directorial style for this story.",
  "preferenceCards": [
    {
      "preference": "Reusable preference written as a directorial guideline.",
      "appliesTo": "When to apply it.",
      "avoidApplyingTo": "When not to apply it.",
      "evidence": "Short abstract evidence summary, e.g. 'Observed across three ready scenes...'.",
      "confidence": "low | medium | high"
    }
  ],
  "statsSummary": {
    "summary": "Short statistical/pacing summary.",
    "avgShotsPerScene": number,
    "avgShotsPerKeyBeat": number,
    "avgShotSeconds": number,
    "commonShotSizes": string[],
    "commonCameraAngles": string[],
    "commonCameraMovements": string[],
    "commonShotJobs": string[],
    "commonUserChangedFields": string[],
    "commonInsertedShotTypes": string[],
    "commonRemovedShotTypes": string[]
  }
}

15.5 Example output

{
  "profileSummary": "The user's edits tend to push scenes toward emotionally legible human consequence rather than neutral coverage. They often prefer closer reaction coverage when shame, grief, vulnerability, or private realization is the point of the beat. They appear to remove generic setup shots when geography is already clear, but this should not be generalized to spatially complex scenes. Shot density should increase around irreversible emotional turns, not uniformly across the whole scene.",
  "preferenceCards": [
    {
      "preference": "When a beat turns on shame, grief, vulnerability, or private realization, favor emotionally legible coverage on the character absorbing the cost.",
      "appliesTo": "Human consequence beats, humiliation, grief, dread, vulnerability, power imbalance, private realization.",
      "avoidApplyingTo": "Fast action, pure spatial setup, broad comedy, or beats intentionally staged with emotional distance.",
      "evidence": "Observed across multiple ready scenes where user edits moved coverage closer to affected characters and away from neutral/dominance coverage.",
      "confidence": "medium"
    },
    {
      "preference": "Avoid generic establishing or filler shots when the scene can enter through immediate character pressure or behavior.",
      "appliesTo": "Contained scenes, simple geography, emotional pressure, confrontations where location is already clear.",
      "avoidApplyingTo": "New locations, complex blocking, geography-dependent reveals, or action scenes where spatial clarity matters.",
      "evidence": "The user removed or compressed setup/filler coverage in ready scenes where geography was simple.",
      "confidence": "medium"
    }
  ],
  "statsSummary": {
    "summary": "Final accepted scenes are moderately dense, with extra support around emotional consequence rather than uniform filler expansion.",
    "avgShotsPerScene": 15.3,
    "avgShotsPerKeyBeat": 2.8,
    "avgShotSeconds": 2.9,
    "commonShotSizes": ["CLOSE_UP", "MEDIUM_CLOSE_UP", "MEDIUM_SHOT"],
    "commonShotJobs": ["reaction_emotion", "relationship_pressure", "physical_detail"],
    "commonUserChangedFields": ["shotSize", "description", "timing"]
  }
}

16. Deterministic Stats

The aggregator can compute stats deterministically and then let the LLM summarize them.

Metric Source Use
Average shots per scene Final accepted shots General density signal.
Average shots per key beat Final shots + confirmed beats Budgeting signal.
Average shot duration Final shots Pacing signal.
Common shot sizes Final shots + modified fields Shot generation signal.
Common camera angles/movements Final shots + modified fields Shot generation signal.
Common changed fields modify_shot events What user tends to correct.
Common inserted shot types insert_shot events Missing coverage patterns.
Common removed shot types remove_shots events Avoidance patterns.
Merge/reorder frequency edit events Compression/order preferences.

This can remain loose. The profile stores a compact statsSummary, not a full analytics dump.


17. Runtime Consumption

At scene guru initialization, load the current aggregate profile and build a live preference block.

directorialPreferenceProfile
  + current scene data
  + current user steering
  + current tool name
  → RuntimePreferenceBlock

17.1 Runtime flow

sequenceDiagram
  participant User
  participant GuruChatService
  participant PreferenceService
  participant Profile as directorialPreferenceProfiles
  participant Guru as WorkbenchSceneGuru
  participant Tool as Planning Prompt

  User->>GuruChatService: open scene
  GuruChatService->>PreferenceService: getRuntimePreferenceBlock(story, scene)
  PreferenceService->>Profile: load profile by story
  PreferenceService->>PreferenceService: format compact block
  PreferenceService-->>GuruChatService: runtime preference block
  GuruChatService->>Guru: initialize with preference block
  Guru->>Tool: pass tool-specific block to prompt

17.2 Runtime block shape

interface RuntimePreferenceBlock {
  profileSummary: string;
  preferences: PreferenceCard[];
  statsSummary?: PreferenceStatsSummary;

  /**
   * Static prompt policy, not a learned preference and not stored in DB.
   * It tells downstream LLM calls how to rank learned preferences against
   * current user intent, scene facts, continuity, pacing, and tool constraints.
   */
  overridePolicy: string;

  usageExamples: string[];
}

17.3 Runtime formatting rules

Rule Reason
Send max 3-7 preference cards Avoid prompt bloat.
Prefer high/medium confidence cards Keep guidance useful.
Do not pass source scene details Avoid leaking future story details.
Include appliesTo and avoidApplyingTo Lets the planning LLM decide relevance.
Include tool-specific examples Avoid hardcoding a massive rules matrix.
Current user steering overrides profile User's current request wins.
Use the fixed PREFERENCE_OVERRIDE_POLICY in every preference block Keeps all planning prompts consistent about when to apply or ignore learned preferences.

17.4 Override policy text

The runtime block should include the same fixed policy defined in Section 8.2. This is intentionally not a database field and not an LLM-generated value. It is a static instruction emitted by PreferencePromptFormatter so that all planning tools treat preferences as soft, conditional priors.


18. Do We Need Exhaustive Tool Mapping?

No.

We do not need every preference to specify exactly which tool it affects. That creates too much structure for too little learning.

Instead:

  1. Store simple preference cards.
  2. Pass a small set of cards into the planning prompts.
  3. Add tool-specific examples to each prompt explaining how preferences should be interpreted.
  4. Let the LLM behind that tool decide whether and how the preference applies.

This is simpler and probably more robust.

18.1 Minimal consumption table

Tool / stage What it receives How it should use preferences
WorkbenchSceneGuru.getContext() profileSummary, top preference cards, override policy General awareness and deciding when to ask a targeted question.
recommend_key_beats Preference cards + key-beat usage examples Consider whether learned preferences imply a different structural emphasis.
recommend_filler_strategy Preference cards + filler usage examples Decide what kind of support/filler the user likely values.
budget_beat_packages Preference cards + statsSummary + budget examples Use density/pacing preferences carefully and conditionally.
generate_beat_package Preference cards + shot generation examples Translate preferences into concrete subject, size, angle, movement, timing choices.
propose_edit_plan Preference cards + revision examples Anticipate repeated correction patterns.

19. Prompt Blocks for Existing Tools

Each planning template gets the same basic structure:

LEARNED DIRECTORIAL PREFERENCES

These preferences come from the user's prior completed scene edits.
Treat them as soft, conditional priors.
Apply them only when they fit the current scene, beat, act pacing, geography, and current user steering.
Explicit user instructions override learned preferences.
Do not force a preference just because it exists.

PROFILE SUMMARY:
{profileSummary}

PREFERENCE CARDS:
{preferenceCards}

OPTIONAL STATS:
{statsSummary}

HOW TO USE THESE PREFERENCES IN THIS TOOL:
{usageExamplesForThisTool}

19.1 recommend_key_beats examples

Preference usage examples for key-beat recommendation:
- If a preference says the user values emotional consequence, consider whether the affected character's realization or loss of control is a true structural beat.
- If a preference is about closer reaction coverage, do not turn that directly into more key beats. Key beats must still be story turns.
- If the preference's appliesTo/avoidApplyingTo conditions do not match this scene, ignore it.

19.2 recommend_filler_strategy examples

Preference usage examples for filler strategy:
- If prior edits favor emotional consequence, use filler to make the affected character's internal state legible.
- If prior edits removed generic setup shots, avoid filler that only repeats geography unless geography is necessary.
- Apply density preferences only to matching beats; do not uniformly increase filler across the scene.

19.3 budget_beat_packages examples

Preference usage examples for shot budgeting:
- Use learned density preferences only when the current beat's function matches the preference conditions.
- If the user previously expanded emotionally irreversible beats, allocate extra support there rather than increasing every package.
- If act pacing calls for compression, keep the budget lean even if prior scenes were denser.

19.4 generate_beat_package examples

Preference usage examples for shot generation:
- Translate preference cards into concrete shot choices: subject, shot size, camera angle, movement, timing, and shot purpose.
- If a preference applies to vulnerability or humiliation, consider favoring the character absorbing emotional cost.
- Do not apply emotional-close-up preferences to pure geography, action clarity, or intentionally detached beats.

19.5 propose_edit_plan examples

Preference usage examples for edit planning:
- If the scene feels generic, check for neutral setup shots that could be replaced by character-specific pressure or consequence shots.
- If the scene feels bloated, trim filler that does not escalate emotion, geography, power, or revelation.
- If preferences conflict with the user's current instruction, follow the current instruction.

20. Integration with GuruChatService

Preference loading happens in the async scene guru initialization path.

const preferenceBlock =
  await DirectorialPreferenceService.getRuntimePreferenceBlockForScene({
    preferenceTriggerUserId: userId,
    storyData,
    sceneData,
  });

const guru = this.initializeSceneGuru(
  storyData,
  sceneData,
  userId,
  chatSession,
  preferenceBlock
);

Keep initializeSceneGuru() synchronous. Do not do DB reads inside the constructor path.

private static initializeSceneGuru(
  storyData: StoryVideoType,
  sceneData: StoryVideoScene,
  userId: string,
  chatSession?: GuruChatSessionDocument | null,
  preferenceBlock?: RuntimePreferenceBlock | null
): WorkbenchSceneGuru {
  const guru = new WorkbenchSceneGuru(
    config,
    storyData,
    sceneData,
    userId,
    preferenceBlock
  );

  if (chatSession?.messages) {
    guru.loadHistory(chatSession.messages);
  }

  return guru;
}

21. Integration with WorkbenchSceneGuru

21.1 Constructor

class WorkbenchSceneGuru extends AgenticGuruBase {
  private preferenceBlock?: RuntimePreferenceBlock | null;

  constructor(
    config: AgenticGuruConfig,
    storyData: StoryVideoType,
    sceneData: StoryVideoScene,
    userId: string,
    preferenceBlock?: RuntimePreferenceBlock | null
  ) {
    super(config);
    this.storyData = storyData;
    this.sceneData = sceneData;
    this.userId = userId;
    this.preferenceBlock = preferenceBlock ?? null;
  }
}

21.2 getContext()

Add a compact context field:

priorScenePreferences: this.preferenceBlock
  ? {
      profileSummary: this.preferenceBlock.profileSummary,
      preferences: this.preferenceBlock.preferences,
      overridePolicy: this.preferenceBlock.overridePolicy,
    }
  : null

Do not include:

Do not include Reason
Raw guruChatSessions Too large/noisy.
Raw toolCall.data Too verbose.
Full edit events Useful for aggregation, not runtime planning.
Source scene plot details Avoid future-scene leakage.

22. Integration with Planning Templates

Planning templates make separate LLM calls, so they must receive the preference block explicitly.

Template Add parameter
RecommendKeyBeatsTemplate.getPrompt(...) preferenceBlock?: RuntimePreferenceBlock
RecommendFillerStrategyTemplate.getPrompt(...) preferenceBlock?: RuntimePreferenceBlock
BudgetBeatPackagesTemplate.getPrompt(...) preferenceBlock?: RuntimePreferenceBlock
GenerateBeatPackageTemplate.getPrompt(...) preferenceBlock?: RuntimePreferenceBlock
ProposeEditPlanTemplate.getPrompt(...) preferenceBlock?: RuntimePreferenceBlock

Example:

const preferenceBlock = RuntimePreferenceFormatter.forTool(
  this.preferenceBlock,
  'recommend_key_beats'
);

RecommendKeyBeatsTemplate.getPrompt(
  this.storyData,
  this.sceneData,
  params.steering,
  preferenceBlock
);

For beat generation, the formatter can add the generate_beat_package usage examples and optionally trim preferences to the most relevant few. It does not need a stored capsule.


23. Service Architecture

flowchart LR
  A[DirectorialPreferenceService] --> B[DirectorialDataHarvester]
  A --> C[DirectorialSceneMemoriesModel]
  A --> D[PreferenceAggregator]
  A --> E[DirectorialPreferenceProfilesModel]
  A --> F[RuntimePreferenceFormatter]

  B --> B1[ToolCallSnapshotDiffExtractor]
  B --> B2[FinalShotSummaryBuilder]
  B --> B3[SceneMemoryDistiller]

  D --> D1[StatsAggregator]
  D --> D2[LLMPreferenceAggregator]

  F --> F1[Tool Prompt Block Builder]

23.1 Proposed files

File Responsibility
directorialPreference.types.ts Shared lightweight types.
directorialPreference.service.ts Main entry point.
directorialDataHarvester.ts Builds one scene memory.
toolCallSnapshotDiffExtractor.ts Converts tool calls and post-tool snapshots into compact edit events.
finalShotSummaryBuilder.ts Builds final shot stats and summary.
sceneMemoryDistiller.ts LLM prompt for scene memory learning.
preferenceAggregator.ts Aggregates scene memories into profile.
runtimePreferenceFormatter.ts Builds prompt-ready blocks.
directorialSceneMemories.model.ts Mongo model.
directorialPreferenceProfiles.model.ts Mongo model.

24. Core Service APIs

24.1 Harvest scene memory

class DirectorialPreferenceService {
  static async enqueueHarvestForScene(params: {
    preferenceTriggerUserId: string;
    storyId: string;
    sceneId: string;
  }): Promise<void>;

  static async harvestSceneMemory(params: {
    preferenceTriggerUserId: string;
    storyData: StoryVideoType;
    sceneData: StoryVideoScene;
  }): Promise<DirectorialSceneMemoryDocument>;
}

24.2 Aggregate profile

class DirectorialPreferenceService {
  static async aggregateProfile(params: {
    preferenceTriggerUserId: string;
    storyData: StoryVideoType;
  }): Promise<DirectorialPreferenceProfileDocument | null>;
}

Aggregation loads all directorialSceneMemories for:

preferenceTriggerUserId + storyId

and overwrites the single profile for that same key.

24.3 Runtime prompt block

class DirectorialPreferenceService {
  static async getRuntimePreferenceBlockForScene(params: {
    preferenceTriggerUserId: string;
    storyData: StoryVideoType;
    sceneData: StoryVideoScene;
    currentUserSteering?: string;
  }): Promise<RuntimePreferenceBlock | null>;
}

25. Manual Dialogue Edits Later

Manual dialogue edits may bypass tool calls. Do not block v1 on this.

Later, add a dialogue diff harvester that produces the same kind of compact edit events.

Dialogue change Future compact event Possible preference
User trims dialogue dialogue_trim Less exposition, more visual/subtextual storytelling.
User moves a line earlier/later dialogue_reorder Different reveal timing or reaction timing.
User changes speaker dialogue_speaker_change Different POV or emotional ownership.
User removes explanatory line dialogue_remove More subtext.
User adds silence/pause dialogue_pause_insert More tension or reaction space.
User rewrites a direct line into an indirect line dialogue_rewrite More restraint or ambiguity.

Implementation options:

Option Pros Cons
Event-sourced dialogue edits Most accurate. Requires UI/backend instrumentation.
Snapshot diff at READY_FOR_SKETCHES Easier retrofit. Harder to infer intent.
Hybrid Best long-term. More implementation work.

26. Rebuild Strategy Without Status Fields

We do not need status: fresh/stale/partial/failed in documents.

Use an event-driven rebuild strategy:

Event Behavior
Scene reaches READY_FOR_SKETCHES Harvest and upsert scene memory.
Ready scene changes Reharvest and upsert scene memory.
Scene memory upsert succeeds Re-aggregate and overwrite profile.
Harvest fails Log failure; keep old memory if it exists.
Aggregation fails Log failure; keep old profile if it exists.
Scene guru opens before profile exists Initialize without preferences.

This is simpler than maintaining document status fields.

If we later need operational visibility, use logs/metrics or add a separate job-run table. Do not pollute the product memory document with internal diagnostics until needed.


27. Observability

Even though we are removing diagnostics from the stored documents, we still need logs.

Event Example payload
DIRECTORIAL_HARVEST_STARTED storyId, sceneId, preferenceTriggerUserId
DIRECTORIAL_HARVEST_COMPLETED editEventCount, learningCount, durationMs
DIRECTORIAL_HARVEST_FAILED error, sceneId, toolCallCount
DIRECTORIAL_PROFILE_AGGREGATION_STARTED storyId, sceneMemoryCount
DIRECTORIAL_PROFILE_AGGREGATION_COMPLETED preferenceCardCount, durationMs
DIRECTORIAL_PROFILE_AGGREGATION_FAILED error, storyId
DIRECTORIAL_PROFILE_USED storyId, sceneId, toolName, preferenceCount

Metrics:

Metric Purpose
Harvest success rate Reliability.
Profile aggregation success rate Reliability.
Average harvest latency Background processing health.
Average aggregation latency Cost/performance.
Preference cards passed per tool Prompt-size control.
User corrections after profile use Quality feedback.

28. Rollout Plan

Phase 1: Harvester and scene memories

Task Output
Add directorialSceneMemories model Stores scene memory.
Add scene-level chat session query Loads guruChatSessions for a scene.
Implement snapshot-diff extractor Produces compact edit events.
Implement final shot summary builder Produces compact stats and summary.
Run harvester at READY_FOR_SKETCHES Non-blocking source-memory creation.

Phase 2: Scene memory distillation

Task Output
Add LLM scene memory distiller Produces sceneContextSummary, editSummary, sceneLearnings.
Store simple learning cards No complex ontology.
Add logs/metrics Debuggability without diagnostics fields.

Phase 3: Aggregate profiles

Task Output
Add directorialPreferenceProfiles model One current profile per story.
Implement stats aggregation Lightweight statsSummary.
Add LLM preference aggregator Produces profileSummary and preferenceCards.
Trigger aggregation after scene memory upsert Current profile stays updated eventually.

Phase 4: Runtime consumption

Task Output
Add runtime preference formatter Builds prompt-ready block live.
Inject into WorkbenchSceneGuru.getContext() Main guru awareness.
Pass block into planning templates Actual planning/generation impact.
Add tool-specific usage examples Tool LLMs interpret preferences safely.

Phase 5: Manual dialogue diffs

Task Output
Instrument manual dialogue edits Future source data.
Convert dialogue diffs into compact edit events Same scene memory format.
Update distiller prompt Dialogue-related preferences.

29. Testing Plan

29.1 Harvester tests

Test Expected result
Modify shot after insert Resolves shot by immediate before snapshot, not final index.
Remove shot Summarizes removed shot from before snapshot.
Insert shot Summarizes inserted shot from afterIds - beforeIds.
Reorder shots Counts explicit moved shots only.
Merge shots Summarizes compression.
Modify shot field side effects Counts only requested fields.
Full tool result Stores compact event, not raw tool result.

29.2 Distiller tests

Test Expected result
Continuity correction Not turned into a broad style preference.
Model error correction Not promoted as a preference.
One-scene density expansion Written cautiously and conditionally.
Repeated close reaction edits Produces clear, carry-forward scene learning.

29.3 Aggregator tests

Test Expected result
Similar learnings across scenes Merged into one preference card.
Filler added in confrontation and removed in chase Preference becomes conditional, not contradictory.
Weak contradictory evidence Claim is omitted or low-confidence.
Scene-specific learning Not promoted.
Future source scene evidence Profile card remains abstract and does not leak plot details.

29.4 Runtime tests

Test Expected result
Profile exists Runtime block built and passed to scene guru.
No profile exists Scene guru initializes without preferences.
Key-beat prompt Receives preference cards and key-beat examples.
Budget prompt Receives preference cards, stats summary, and budget examples.
Shot generation prompt Receives preference cards and shot-generation examples.
User steering conflicts Current user instruction wins.

30. Final Recommendation

Implement v1 as a deliberately simple preference memory system:

guruChatSessions
  Existing raw source of truth.

Data Harvester
  Reads tool calls and final scene state, then creates compact scene memories.

directorialSceneMemories
  One memory per story/ready scene.

Preference Aggregator
  Merges scene memories into story-level preference cards.

directorialPreferenceProfiles
  One current aggregate profile per story.

Runtime Preference Block
  Calculated live from the profile and passed into the scene guru/planning prompts.

Do not store hashes, status fields, diagnostics, unresolved conflicts, runtime capsules, synthetic packs, or a large preference ontology in v1.

The stored memory should be readable and compact:

What kind of scene was this?
What did the user change?
What did the final accepted shot plan look like?
What practical lessons might carry forward?
What aggregate preference cards summarize those lessons across ready scenes?

The planning tools should receive concise preference cards plus examples for how to use them. The LLM behind each tool can decide relevance based on the current scene, beat, pacing, and user steering.

The desired learned rule is not:

User likes more filler.

It is:

The user tends to want denser support around emotionally irreversible pressure or humiliation beats, especially when the dramatic purpose is to make the affected character's internal state legible. Do not apply this globally to fast action, pure exposition, or compressed act pacing.

That level of preference memory is useful, compact, and flexible enough to improve future scene planning without making the system brittle.