---
name: multimedia-generation
description: Use when generating audio, video, or images in a Lexigram application — TTS, music, video, image, upscale, interpolate, beat analysis, or timeline composition
---

# Multimedia Generation

## Overview

`lexigram-multimedia` wires **seven subsystems** into one DI graph: TTS, music, video, image, upscale, interpolate, and beat analysis. One module and one nested config tree; protocols for direct calls, accessors for storage, caching, events, and idempotent jobs.

## Setup

```bash
uv add lexigram-multimedia
# backends are opt-in per subsystem
uv add "lexigram-multimedia-tts[elevenlabs,openai]"
uv add "lexigram-multimedia-video[wan22-server]"
```

With zero extras the umbrella uses the default in-process / `local-http` backends.

## Module Registration

```python
from lexigram.app.base import Application
from lexigram.multimedia import MultimediaConfig, MultimediaModule

app = Application(name="media-app")
app.add_module(MultimediaModule.configure(
    config=MultimediaConfig(cache_results=True)
))
await app.start()
```

`MultimediaModule.configure()` with no arguments is zero-config.

## Resolve a Protocol

```python
from lexigram.contracts.multimedia import TTSProvider, TTSRequest

tts = await app.container.resolve(TTSProvider)
result = await tts.generate(TTSRequest(text="Hello from Lexigram"))
if result.is_ok():
    asset = result.unwrap()  # MediaAsset — has_bytes OR has_uri
    if asset.has_uri:
        print(asset.uri)
```

Always check `asset.has_uri` / `asset.has_bytes` — ElevenLabs/OpenAI return raw bytes; hosted backends return URIs.

## Accessors

Accessors add storage normalization, result caching, generation events, and idempotent submission. They are properties on the `MultimediaProvider`, reachable via `app.providers`:

```python
provider = next(p for p in app.providers if p.name == "multimedia")

result = await provider.image.generate(ImageRequest(prompt="mountain at dawn"))
```

### Queued Jobs

```python
from lexigram.contracts.multimedia import TTSRequest

handle = await provider.tts.submit(
    TTSRequest(text="Long narration for a video"),
    idempotency_key="narration-v1",
)
print(handle.job_id, handle.status, handle.is_duplicate)
```

`submit()` dispatches to `lexigram-tasks`, deduplicated by `idempotency_key`; the task handler normalizes the finished asset to URI-backed storage.

## Timeline Composition

```python
from lexigram.contracts.multimedia import SubtitleCue, TransitionSpec

timeline = Timeline()
timeline.add_clip(clip_a)
timeline.add_clip(clip_b, transition_in=TransitionSpec(kind="crossfade", duration=0.4))
timeline.add_captions([SubtitleCue(start=0.0, end=2.0, text="Intro")])
timeline.set_music(bgm, duck_under_narration=True)

result = await provider.compose.render(timeline)          # sync
handle = await provider.compose.submit_render(timeline)   # queued
```

## Config

```yaml
multimedia:
  cache_results: true
  storage_path_prefix: multimedia/
  tts:
    backend: local-http
  music:
    backend: local-http
```

Each subsystem has its own sub-config; installing a subsystem's extra selects a remote/GPU backend.

## Subsystem Reference

| Accessor | Protocol | Request |
|----------|----------|---------|
| `.tts` | `TTSProvider` | `TTSRequest` |
| `.music` | `MusicProvider` | `MusicRequest` |
| `.video` | `VideoProvider` / `VideoProcessor` | `VideoRequest`, `Trim`, `Concat` |
| `.image` | `ImageProvider` | `ImageRequest` |
| `.upscale` | `UpscaleProvider` | `UpscaleRequest` |
| `.interpolate` | `InterpolationProvider` | `InterpolationRequest` |
| `.beat` | `BeatAnalysisProvider` | `BeatAnalysisRequest` |

## Common Mistakes

- Checking neither `has_uri` nor `has_bytes` — providers differ in asset shape
- Expecting `submit()` to work without `lexigram-tasks` and `lexigram-storage`
- Importing sibling subsystem packages directly — resolve protocols from the container
- Calling `unwrap()` on a `Result` without an `is_ok()` guard
- Installing GPU/remote backends you don't need — `local-http`/`librosa` need no server