Runbook: Add a New Sector to the Pipeline

Prerequisites

• Pipeline architecture v3 is deployed (campaign.json + pipeline-index.json + memory/businesses/*.md)
• Agent crons are running
• tools/pipeline_util.py is on the VPS

Quick Version (5 steps)

# 1. Generate business .md files (one per business)
#    Each file needs minimal YAML frontmatter: slug, sector, region, status, name, website
#    Put them in a local staging dir

# 2. Create a sector patch JSON
cat > /tmp/sector-patch.json << 'EOF'
{
  "sector": "restaurant",
  "slugs": ["joes-pizza-calgary", "sushi-bar-kensington", ...],
  "initial_status": "discovered"
}
EOF

# 3. Deploy business files to VPS
scp /tmp/staged-businesses/*.md openclaw@76.13.109.25:~/.openclaw/workspace-pawlo-outreach/memory/businesses/

# 4. Merge into the pipeline index (additive — never touches other sectors)
ssh openclaw@76.13.109.25 'cd ~/.openclaw/workspace-pawlo-outreach && python3 tools/pipeline_util.py merge --patch /tmp/sector-patch.json'

# 5. Update campaign.json to assign the sector (or tell agent via Telegram)

Detailed Steps

Step 1: Discovery

Run web searches to build a list of businesses in the new sector. For each business, collect at minimum:

• Business name
• Website URL
• City/region

Generate a slug for each: lowercase, hyphens, no special chars (e.g., joes-pizza-calgary).

Step 2: Generate Business Files

Each business gets a {slug}.md file with YAML frontmatter:

---
type: business
slug: "joes-pizza-calgary"
name: "Joe's Pizza"
sector: "restaurant"
region: "Calgary"
status: "discovered"
website: "https://joespizza.ca"
address: ""
phone: ""
email: ""
contact_name: ""
discovered_at: "2026-02-24"
enriched_at: ""
outreach_sent_at: ""
last_reply_at: ""
profile_completed_at: ""
thread_id: ""
follow_ups_sent: 0
follow_up_dates: []
next_follow_up_date: ""
last_email_type: ""
---

# Joe's Pizza

Restaurant in Calgary.

## Notes
No notes yet.

Use src/tools/convert_pipeline.py if converting from a JSON list, or write a quick script for the new format.

Step 3: Create Sector Patch

{
  "sector": "restaurant",
  "slugs": ["joes-pizza-calgary", "sushi-bar-kensington"],
  "initial_status": "discovered"
}

The slugs array must match the filenames (without .md).

Step 4: Deploy

# Business files (additive — won't overwrite existing files with same name)
scp /tmp/staged-businesses/*.md openclaw@76.13.109.25:~/.openclaw/workspace-pawlo-outreach/memory/businesses/

# Sector patch (to temp location)
scp /tmp/sector-patch.json openclaw@76.13.109.25:/tmp/sector-patch.json

# Merge into index (SAFE — only adds, never removes)
ssh openclaw@76.13.109.25 'cd ~/.openclaw/workspace-pawlo-outreach && python3 tools/pipeline_util.py merge --patch /tmp/sector-patch.json'

# Verify
ssh openclaw@76.13.109.25 'cd ~/.openclaw/workspace-pawlo-outreach && python3 tools/pipeline_util.py status --sector restaurant'

Step 5: Assign the Campaign

Option A — Deploy config:

Edit src/working/campaign.json, add the sector to active or queue, then:

scp src/working/campaign.json openclaw@76.13.109.25:~/.openclaw/workspace-pawlo-outreach/working/campaign.json

Option B — Tell the agent via Telegram:

Message: “New sector added: restaurant. 50 businesses in Calgary. Start enrichment.”

The agent will update campaign.json itself.

Step 6: Sector-Specific Templates (optional but recommended)

If the new sector needs custom outreach copy:

# Create intro email template
src/reference/templates/intro/restaurant.md

# Create follow-up template
src/reference/templates/follow-up/restaurant.md

# Create questionnaire
src/reference/templates/questionnaire/restaurant.md

Deploy to VPS:

scp src/reference/templates/intro/restaurant.md openclaw@76.13.109.25:~/.openclaw/workspace-pawlo-outreach/reference/templates/intro/
scp src/reference/templates/follow-up/restaurant.md openclaw@76.13.109.25:~/.openclaw/workspace-pawlo-outreach/reference/templates/follow-up/
scp src/reference/templates/questionnaire/restaurant.md openclaw@76.13.109.25:~/.openclaw/workspace-pawlo-outreach/reference/templates/questionnaire/

If no sector-specific templates exist, the agent falls back to generic pet-sector templates (which may not make sense for non-pet sectors — create templates for any non-pet industry).

Step 7: Add to Discovery Config (optional)

If you want the agent to discover MORE businesses in this sector on its own, add search terms to src/reference/discovery/config.json:

{
  "id": "restaurant",
  "label": "Restaurant",
  "search_terms": ["restaurant", "pizza", "sushi", ...],
  "active": true
}

Deploy:

scp src/reference/discovery/config.json openclaw@76.13.109.25:~/.openclaw/workspace-pawlo-outreach/reference/discovery/config.json

What NOT to Do

• Never overwrite pipeline-index.json directly — always use pipeline_util.py merge
• Never delete other sectors’ business files — the merge is additive
• Never deploy a pipeline.json or todo.json — these are deprecated
• Never scp a full pipeline-index.json — use rebuild if the index is corrupt

Recovery

If the index gets out of sync:

ssh openclaw@76.13.109.25 'cd ~/.openclaw/workspace-pawlo-outreach && python3 tools/pipeline_util.py rebuild'

This scans all memory/businesses/*.md files and rebuilds the index from scratch.