Building an Autonomous Apify Actor Factory with AI Subagents

TL;DR: I built a multi-agent AI system called Goblin (based on Hermes Agent) with a domain-expert subagent called ghoul that handles Apify actor knowledge, monetization strategy, pre-publish QA, customer support drafts, and weekly content monitoring. A separate builder agent reads GitHub issues to actually implement changes. I manage everything from Telegram.

The Problem

Building and selling Apify actors on the Store involves way more than just writing code. You need to:

• Stay current with PPE (pay-per-event) pricing rules, platform policies, and SDK changes
• Run pre-publish QA against a 20-item checklist before every release
• Draft thoughtful customer support replies without making commitments you can't keep
• Calculate margins: 0.8 × price − CU_cost > 0 or you're losing money
• Monitor Apify docs for policy changes that could break your actors

Doing all this manually is tedious and error-prone. Having an LLM do it in a generic chat window loses context and costs too much on every query.

What I needed was a specialized AI agent with curated knowledge, gated web access, a vector search engine for semantic retrieval, and strict guardrails, all accessible from my phone.

The Solution: Hermes Agent + Commander Architecture

The system is built on Hermes Agent (Nous Research), an open-source AI agent framework. It's the same category of tool as Claude Code or OpenAI Codex, but designed to be the persistent, long-running agent that connects to your messaging platforms and orchestrates specialized subagents.

Here's the architecture:

The Agents

Goblin: the orchestrator. A Hermes Agent instance running on an ARM64 VPS, connected to Telegram as a bot. Goblin is my single point of contact: I paste questions, customer emails, or requests, and Goblin routes them to the right subagent. Goblin also acts as the approval gate, no outbound action happens without my /approve.

ghoul: the Apify SME Commander. A delegate_task subagent that Goblin spawns on demand. It has five operating modes:

code-ghoul: the builder (separate VPS). Reads approved GitHub issues, implements the fix or feature, opens a PR, and comments back. This is a separate machine so build processes never touch the agent VPS.

The Handoff Protocol

The most important design decision: no conversational relay. The expensive payload (the technical diagnosis, the code spec, the support draft) is never piped through multiple agent context windows. It goes straight to GitHub:

You → Telegram → Goblin → delegate_task → ghoul
                                           │
                                           ├─ Returns diagnosis/draft
                                           ├─ Writes work-order as GitHub issue
                                           │
You ← Goblin (1-line summary + "approve?")
  │
  └─ /approve → Goblin flips label on issue
                                       │
                          code-ghoul (separate VPS)
                            polls approved issues
                            reads spec from GitHub
                            builds, opens PR, comments

The result: you control the approval, but the full technical payload lives in GitHub where it belongs, never re-transcribed through a chat window.

The Knowledge Strategy

Knowledge is organized in three tiers, ranked by cost:

Tier 0: Static Core Pack ($0/query) Five curated markdown files shipped with the repo: PPE model, pricing margins, QA checklist, support rules, and SDK patterns. These are loaded into the subagent's context on every invocation. Covers ~80% of questions.

Tier 1: Vector Store (cents/week) A Qdrant vector database running on the same VPS, populated by a weekly cron job that scrapes 11 canonical Apify URLs. Each scrape computes a SHA256 hash and diffs against the previous snapshot. If content changed, I get a Telegram alert + GitHub issue. Semantic search via sentence-transformers (all-MiniLM-L6-v2, 384-dim) returns relevant passages ranked by cosine similarity. Covers ~15% of questions.

Tier 2: Live Web Fetch (<$0.01/query) Only fires when the query explicitly asks for current information or the vector store has no match. Hard-capped at 3 fetches per query, token-trimmed to ~5k tokens each. Covers the remaining ~5%.

This three-tier design means the expensive path (live web fetch) is almost never hit. Most queries are answered from static context or the vector store.

The Tech Stack

Design Decisions That Mattered

1. Skills as Subagent Blueprints

Hermes Agent's skill system was the key enabler. The apify-ghoul skill contains the persona, knowledge pack references, operating mode logic, and guardrails. When Goblin calls delegate_task(skills=["apify-ghoul"]), the subagent spawns with all that context pre-loaded. No custom scaffolding.

2. GitHub as the Bus, Not Conversation

The single most important rule: never relay the full payload through the orchestrator's context window. If ghoul produces a 5,000-token diagnosis, Goblin does not read it. Ghoul writes it to a GitHub issue. Goblin shows me a one-line summary. I approve. code-ghoul reads it from GitHub. The diagnosis token cost is paid exactly once.

3. Gated Live Fetch

The cron job and vector store handle most "what's new" questions. Live fetch only triggers on explicit intent ("is X still true", "what's the latest", "check the current price") and is capped at 3 pages per query. This prevents the cost from ballooning on routine questions.

4. Advisory-Only by Default

Every output is a draft, not an action. Support replies are wrapped in [SUPPORT DRAFT: review before sending]. Code changes go to GitHub issues, not directly to repos. Approval is a single label flip or one-word reply, not a transcript review.

The Repository

The entire ghoul project is versioned in a single private repo:

apify-ghoul/
├── knowledge/             # Tier 0: PPE model, pricing, QA, support, SDK
│   ├── ppe-model.md
│   ├── pricing-margins.md
│   ├── qa-checklist.md
│   ├── support-rules.md
│   └── sdk-patterns.md
├── persona.md             # Role, 5 modes, ground rules
├── config.yaml            # Model routing
├── seed-urls.txt          # 11 canonical Apify URLs
├── refresh.cron           # Weekly scrape + drift detection + Qdrant embed
├── query_qdrant.py        # Semantic search CLI
├── create_issue.py        # Structured work-order issue creator
└── README.md

Plus two Hermes skills that live in ~/.hermes/skills/:

• apify-ghoul: the Commander skill loaded by the subagent
• commander-roster: the master routing table (pinned, never archived)

Guardrails

• Approval gate: every outbound action goes through me via Goblin's one-line summary
• No auto-send: support drafts return in an envelope, never sent
• Live-fetch cap: max 3 pages per query, token-trimmed
• No auto-commit: ghoul diagnoses and specifies; code-ghoul builds from GitHub issues
• Drift alerts: content changes on canonical URLs generate Telegram + GitHub alerts, never blind overwrites
• Token-burn prevention: no payload relayed through multiple context windows

What's Next

P4 (Watch Digest): A weekly cron that scrapes the Apify blog and changelog, summarizes changes, and delivers a structured digest to Telegram. This covers the "what's new with Apify" use case.

code-ghoul: A separate VPS agent that polls GitHub for approved issues, reads the spec directly, builds the actor, runs tests, and opens a PR. This completes the loop from "I need a fix" to "here's the PR" without any manual intervention.

Resources

• Hermes Agent: The open-source AI agent framework this is built on
• Apify Docs: Monetize: PPE pricing model docs
• Apify Docs: Pay Per Event: PPE reference
• Apify SDK: PPE for JavaScript: SDK patterns
• Apify SDK: PPE for Python: SDK patterns
• Qdrant: Vector database
• Nous Research: Creators of Hermes Agent