---
name: mine-fiction-motif-corpus
description: Build a provenance-preserving fiction motif corpus from personal text archives. Use when the user wants to combine journals, drafts, platform exports, or AI-chat exports; audit source coverage; deduplicate exact copies and near rewrites; rank scene evidence; and turn recurring structures into evidence-backed motif cards and story engines without publishing private source material.
---

# Fiction Motif Corpus

## Goal

Turn scattered personal writing into an incremental, auditable corpus for fiction development. Original exports remain the source of truth. Normalized records, motif cards, timelines, and story proposals are derived artifacts.

Do not treat a frequent keyword as a motif. A motif must describe a recurring predicament that changes form across time, roles, places, or objects.

## Workflow

### 1. Register the sources

Copy [the synthetic configuration](references/sources.example.json) into a private working directory. Replace its paths and field mappings with the sources authorized for this run.

Classify every source as:

- `raw`: original exports, databases, journals, and drafts.
- `normalized`: an index that already preserves original references.
- `derived`: summaries, previous motif reports, and rewritten drafts.

Do not ingest `derived` material by default. It can turn an earlier interpretation into fake independent evidence. If a normalized index already covers the underlying archives, do not ingest both copies as separate sources.

Only claim complete coverage when every registered required source is `ok` and its date range has been checked. Report optional missing exports and access gaps explicitly.

### 2. Select the author's text

For chat exports, include the author's messages by default and exclude assistant or correspondent messages unless the user explicitly needs them as contextual evidence. Never transform another person's words into the author's memory or opinion.

Export unstable or proprietary databases to JSON or JSONL before ingestion. Preserve a conversation identifier, timestamp, speaker, source identifier, and text wherever the source provides them.

### 3. Normalize and deduplicate

Run:

```bash
python3 scripts/build_motif_corpus.py \
  --config "/absolute/path/to/sources.json" \
  --output "/absolute/path/to/run_YYYYMMDD_HHMMSS"
```

The script writes:

```text
00_coverage.json
01_records.jsonl
02_corpus.jsonl
03_dedupe.json
04_evidence_candidates.jsonl
state.json
```

- `01_records.jsonl` contains every ingested record.
- `02_corpus.jsonl` merges exact copies while retaining provenance and duplicate IDs.
- `03_dedupe.json` separates exact duplicates from likely rewrites. Near matches are never merged automatically.
- `04_evidence_candidates.jsonl` ranks records by visible task, deadline, object, cost, and change signals. It is a reading queue, not a literary verdict.

Create a new timestamped output directory for every refresh. Never overwrite original exports or an earlier run.

### 4. Build motif evidence cards

Read [the motif method](references/motif-method.md). Return from a candidate excerpt to its full source before making a claim.

Each motif card must include:

- a structural motif sentence;
- variants across time, roles, places, or media;
- at least two traceable `record_id` or `source_ref` items;
- counterevidence that does not fit the interpretation;
- a visible task, deadline, main object, real constraint, and irreversible cost;
- a privacy level and the factual changes required before publication.

A single strong scene can enter the story-development pool, but it is not a motif without independent recurrence.

### 5. Create story engines

Keep one core task and one main object per proposal:

```text
motif structure -> concrete task -> deadline -> main object
-> responsibility added by each step -> irreversible loss -> ending action
```

Do not turn a résumé into a plot summary. Do not make an AI character announce the theme. Draft a short scene first, then decide whether the material supports a full story.

### 6. Verify the delivery

A complete delivery normally contains:

```text
00_coverage_and_gaps.md
01_motif_evidence_cards.md
02_timeline_and_variants.md
03_story_development_pool.md
04_privacy_and_fact_boundaries.md
the six generated data files
```

Verify that:

- the coverage report matches every completion claim;
- every motif includes sources and counterevidence;
- exact copies are merged and likely rewrites remain reviewable;
- personal identifiers, private workplace material, and recognizable third-party timelines are not copied into publishable prose.

## Stop rules

- Do not delete an original export, database, conversation, or previous run.
- Do not ingest sources without the user's authority to process them.
- Do not count a cross-post as independent motif evidence.
- Do not place credentials, payment data, case records, contact lists, or private workplace material into a publishable corpus.
- Do not automatically publish or submit fiction generated from this corpus.