# dpg — Dynamic Page Generation Pipeline

**One-line description:** Converts a text prompt (or wireframe images) into a fully runnable FastAPI backend + Vite/React frontend by chaining five LLM-powered pipeline steps via AWS Bedrock.

---

## Tech Stack

| Library | Version | Purpose |
|---------|---------|---------|
| Python | ≥ 3.10 | Runtime |
| boto3 | ≥ 1.42.56 | AWS Bedrock SDK (direct `invoke_model` calls) |
| langchain-aws | ≥ 1.3.0 | `ChatBedrockConverse` LLM wrapper (IR + React steps) |
| anthropic[bedrock] | ≥ 0.95.0 | Anthropic SDK (edit pipeline, step-06) |
| pydantic | ≥ 2.12.5 | Typed schema validation for all IR structures |
| pymupdf | ≥ 1.24.0 | PDF text + embedded image extraction |
| python-pptx | ≥ 1.0.0 | PPTX slide text + image extraction |
| python-docx | ≥ 1.1.0 | DOCX paragraph + table + image extraction |
| python-dotenv | ≥ 1.0.1 | `.env` loading at startup |
| redis[hiredis] | ≥ 5.0.0 | Optional session cache (edit pipeline) |
| openai | ≥ 2.34.0 | Alternative LLM client (some sub-steps) |
| jsonschema | ≥ 4.26.0 | JSON validation for generated structures |
| uv | latest | Dependency management + virtual environment |

---

## Prerequisites

- Python ≥ 3.10
- `uv` (`pip install uv` or see https://docs.astral.sh/uv)
- AWS credentials with `bedrock:InvokeModel` permissions in `us-east-1` (or configured region)

---

## Local Setup

```bash
# 1. Clone the repository
git clone <repo-url>
cd dpg

# 2. Install dependencies
uv sync

# 3. Configure environment
cp .env.example .env
# Edit .env — fill in AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (minimum required)

# 4. Run the pipeline
uv run python main.py generate --prompt "A warehouse management system"

# 5. Start the generated project
cd runs/outputs/<project_slug>/backend && uv sync && uv run uvicorn main:app --reload --port 8000
cd runs/outputs/<project_slug>/frontend && npm install && npm run dev
```

---

## Folder Structure

```
dpg/
├── main.py                          # CLI entry point (generate / edit subcommands)
├── pyproject.toml                   # uv project + dependency declaration
├── .env.example                     # Environment variable template
├── .python-version                  # Pins Python version for uv
│
├── shared/                          # Cross-step utilities — imported by every step
│   ├── config.py                    # Single source of truth: paths, model IDs, timeouts
│   ├── bedrock_client.py            # LangChain ChatBedrockConverse factory
│   ├── bedrock_raw_client.py        # Direct boto3 Bedrock client (BedrockLLMClient)
│   ├── anthropic_client.py          # Anthropic SDK client (edit pipeline only)
│   ├── extractors.py                # extract_json_object / extract_code_block
│   ├── llm_logging.py               # timed_llm_call context manager + token logging
│   ├── logging.py                   # configure_logging / get_logger
│   ├── timing.py                    # log_timed_step context manager
│   ├── run_artifacts.py             # new_run_id() + step_output_dir()
│   ├── run_log.py                   # RunLog: per-run telemetry accumulator
│   ├── redis_config.py              # Redis connection settings (optional)
│   ├── exceptions.py                # NoRunsFoundError / RunNotFoundError
│   ├── media/
│   │   ├── images.py                # EncodedImage + base64 encoding utilities
│   │   └── prd_loader.py            # LoadedPRD + format-specific extractors
│   └── schemas/
│       ├── app_plan.py              # AppPlan, PageNode, DesignSystem
│       ├── ir_bundle.py             # IRBundle (12-section UI specification)
│       ├── app_ir.py                # AppIRBundle, IRPage, MultiPageModelConfig
│       ├── multi_page_bundle.py     # MultiPageBundle, PageBundle
│       └── plan.py                  # ExecutionPlan, PlanStep
│
├── pipeline/
│   ├── master-pipeline/pipeline/
│   │   ├── orchestrator.py          # Wires all 5 steps — main generate_project()
│   │   ├── master.py                # Debug aggregator (legacy)
│   │   ├── reporter.py              # Console + markdown run reports
│   │   ├── runner.py                # CLI for master pipeline subcommands
│   │   └── pptx_to_pngs.py         # PPTX → PDF → PNGs via LibreOffice + PyMuPDF
│   │
│   ├── step-01-input-ingestion/pipeline/
│   │   └── ingestion.py             # ingest() → IngestedInputs
│   │
│   ├── step-02-prd-generation/pipeline/
│   │   ├── prd_generator.py         # generate_from_prompt() / generate_prd() / generate_prd_from_ddl()
│   │   └── ddl_to_cir.py            # DDL → enriched CIR dict (LLM-assisted)
│   │
│   ├── step-03-backend-generation/pipeline/backend_gen/
│   │   ├── orchestrator.py          # 9-step backend generation orchestrator
│   │   ├── requirement_analyzer.py  # LLM: PRD/DDL → structured analysis
│   │   ├── fastapi_code_generator.py# LLM: analysis → FastAPI module files
│   │   ├── manifest_generator.py    # AST parse → api_manifest.json
│   │   └── ...                      # (output_writer, validation_runner, etc.)
│   │
│   ├── step-04-ir-generation/pipeline/ir_pipeline/
│   │   ├── services/multi_page_service.py   # generate_app_ir() entry point
│   │   ├── services/page_detection_service.py # detect_pages() via Opus vision
│   │   ├── services/ir_generation_service.py  # generate_ir_bundle_for_page()
│   │   └── prompts/                           # LLM prompt templates
│   │
│   ├── step-05-react-generation/pipeline/
│   │   ├── react_gen.py             # generate_react_pages() entry point
│   │   ├── scaffolder.py            # scaffold_frontend() + write_generated_files()
│   │   └── react_pipeline/services/ # generate_page_react_code()
│   │
│   └── step-06-edit/                # Agentic edit pipeline (currently commented out)
│
├── runs/outputs/                    # Final assembled projects (<project_slug>/<run_id>/)
│   └── <slug>_<run_id>/
│       ├── backend/                 # Generated FastAPI project
│       ├── frontend/                # Generated Vite/React project
│       ├── ir/                      # Saved IR bundles (JSON)
│       └── api_manifest.json        # Cross-step API contract
│
└── specs/                           # Drop-in input directory (images/, ddl/)
```

---

## Documentation Index

| File | Description |
|------|-------------|
| [ARCHITECTURE.md](ARCHITECTURE.md) | System overview, layer communication map, error handling strategy |
| [API.md](API.md) | Every Bedrock API call — request shape, response shape, error path |
| [SERVICES.md](SERVICES.md) | Every service function with full input/output/error trace |
| [COMPONENTS.md](COMPONENTS.md) | Pipeline step modules as documented components |
| [STATE.md](STATE.md) | All data contracts and state objects: shapes, transitions, consumers |
| [DATA_FLOW.md](DATA_FLOW.md) | 5 end-to-end user flows traced step-by-step |
| [flowcharts/data-pipeline.md](flowcharts/data-pipeline.md) | Left-to-right data pipeline diagram |
| [flowcharts/api-request-lifecycle.md](flowcharts/api-request-lifecycle.md) | Sequence diagram: component → Bedrock → response |
| [flowcharts/auth-flow.md](flowcharts/auth-flow.md) | AWS credential resolution + Bedrock authentication |
| [flowcharts/state-flow.md](flowcharts/state-flow.md) | Data contracts and how they flow between steps |
| [flowcharts/generate-flow.md](flowcharts/generate-flow.md) | Full `generate` command end-to-end |
| [INDEX.md](INDEX.md) | Quick-start + "where to find X" reference table |

---

## Related source files

- [main.py](../main.py) — CLI entry point
- [pipeline/master-pipeline/pipeline/orchestrator.py](../pipeline/master-pipeline/pipeline/orchestrator.py) — pipeline wiring
- [shared/config.py](../shared/config.py) — all configuration constants