AAASP
Multi-Tenant Agent Execution Infrastructure Built on OTP
The Problem
Every team building AI agent products reinvents the same backend: connection pools to multiple LLM providers, stateful conversation storage across requests, reliable background job execution, per-tenant credential vaulting, and usage-based billing. This boilerplate is expensive to build, fragile to maintain, and has nothing to do with the agent logic itself. Most teams ship it badly or skip it entirely and pay the operational cost later.
The Solution
AAASP is a production-grade Agent-as-a-Service Platform built in Elixir/OTP. It exposes a clean REST API that lets developers register agents, deploy them with encrypted provider credentials, and submit runs — async, sync, or streaming — without owning any infrastructure. OTP's supervision model provides fault tolerance that no thread-based stack can match: crashed processes restart automatically, session state survives node failures via checkpointing, and the job queue is crash-safe by design.
Register an agent. Deploy with credentials. Submit runs via REST — async, sync, or SSE. OTP handles faults. You ship product.
Core Innovation
AAASP's performance is driven by a layered optimisation stack:
| Optimisation | Mechanism | Impact |
| RunBatcher GenServer | Buffers 200 run inserts, flushes with single `insert_all` | 15.9 → 65 req/s (+300%) |
| BatcherPool | 4 parallel batcher workers, round-robin via `:atomics` | 4× parallel DB flush capacity |
| ETS Tenant Cache | Named ETS table, 60s TTL, `read_concurrency: true` | Auth lookup: DB round-trip → sub-microsecond |
| Session GenServer | OTP process per active session, in-process context | Zero-latency context retrieval for multi-turn runs |
| Merkle Checkpointing | Checkpoint to PostgreSQL every 20 messages | Crash recovery without full history replay |
| Pro/Enterprise queue skip | Skips `SELECT COUNT(*)` for high-tier plans | Eliminates second DB query on hot path |
Market Opportunity
| Market Segment | Size (2026) | Growth Rate | Application |
| AI Agent Platforms | $12B | 45% CAGR | Backend execution infrastructure |
| API-First Dev Tools | $8B | 35% CAGR | Per-run billing, self-serve onboarding |
| Enterprise AI Deployment | $18B | 30% CAGR | Multi-tenant isolation, SLA-backed ops |
| LLM Infrastructure | $6B | 50% CAGR | Multi-provider routing and margin capture |
| Total Addressable Market | $44B |
Business Model
| Plan | Target | Revenue |
| Free | Evaluation, hobbyists | Credit allowance on signup; upsell on exhaustion |
| Pro | Indie developers, small teams | Monthly subscription + overage credits |
| Enterprise | AI product companies | Annual contract, SLA, dedicated infrastructure |
| White-label | AI platforms, integrators | License fee for private AAASP deployment |
Per-run credit billing aligns cost directly with value. Dodo Payments integration is stubbed and ready to activate. Compute margin is captured on every LLM call routed through the platform.
Technical Architecture
OTP Supervision Tree:
| Layer | Component | Role |
| Application | `Aaasp.Application` | Boot order, restart policy |
| Sessions | `DynamicSupervisor` + `Registry` | One supervised GenServer per active session, unique naming |
| Job queue | `Oban` (Postgres-backed) | Crash-safe background runs, billing, webhooks, cron |
| HTTP client | `Finch` (HTTP/2 pools) | Per-provider connection pools (Anthropic 50×2, OpenAI 50×2, others 20) |
| DB stack | Ecto → PgBouncer → PostgreSQL | 100 Ecto connections multiplex through transaction-mode pool |
| Encryption | `Cloak AES-GCM` (`Aaasp.Vault`) | Credential vaults encrypted at rest |
| Dashboard | `Phoenix LiveView` | Real-time OTP process tree, run inspection, billing |
Oban Queue Configuration:
| Queue | Concurrency | Workers |
| `runs` | 200 | `RunWorker` — LLM execution |
| `webhooks` | 20 | `WebhookWorker` — result delivery |
| `billing` | 10 | `BillingWorker` — credit deduction |
| `maintenance` | 2 | `SessionExpiryWorker` — idle cleanup (*/5 cron) |
Technical Differentiation
| Feature | AAASP | OpenAI API | AWS Bedrock | LangChain |
| Multi-provider routing | Yes (6 providers) | OpenAI only | AWS models only | Yes (library) |
| Session memory | OTP GenServer, in-process | None (stateless) | None | Manual |
| Fault tolerance | OTP supervision, crash recovery | N/A | Managed SLA | None |
| Background job reliability | Oban (Postgres-backed, crash-safe) | None | SQS integration required | None |
| Multi-tenancy | Built-in isolation + credential vaults | Single-tenant API keys | IAM-based | None |
| Billing | Per-run credit deduction, audit log | Per-token, no multi-tenant | Per-token, no multi-tenant | None |
| Deployment model | Self-hosted or Fly.io | Hosted only | Hosted only | Self-managed |
| Throughput (single node) | ~65–70 req/s @ 200 concurrency | N/A | N/A | N/A |
Traction & Milestones
- >Now: Full API live (runs, batches, sessions, agents, deployments, credentials, billing)
- >Benchmarked: 65–70 req/s at 200 concurrency, zero failures on single node
- >Production: Deployed on Fly.io, running agent workloads for Rising Sun portfolio
- >Q2 2026: Dodo Payments billing activation, public developer beta
- >Q3 2026: Team accounts, Pro plan self-serve, usage analytics dashboard
- >Q4 2026: Enterprise tier, SLA, white-label deployment option
- >2027: Standard execution substrate for Rising Sun ecosystem + external customers
Competitive Advantages
1. **OTP fault tolerance**: Elixir's supervision model provides crash recovery and process isolation that no thread-based agent platform matches
2. **Multi-provider from day one**: Six LLM providers behind a single API; load can shift to cheaper or faster models without client changes
3. **Batcher architecture**: The RunBatcher GenServer pattern (+300% throughput) is a structural advantage not easily replicated in request/response frameworks
4. **Session memory as OTP processes**: In-process context retrieval is orders of magnitude faster than round-tripping to Redis or a database on every run
5. **Credential security**: AES-GCM encryption at rest, bearer token auth, and per-tenant isolation satisfy enterprise security requirements out of the box
6. **Rising Sun ecosystem**: Internal adoption by portfolio projects provides production battle-testing and credible reference architecture before external launch
Why Now
- >Agent proliferation: Teams are shipping AI agents faster than infrastructure can keep up; the operations gap is widening
- >OTP moment: Elixir's concurrency model is purpose-built for the connection-heavy, stateful workloads that agent platforms demand — but few infrastructure projects are built on it
- >Provider fragmentation: Six major LLM providers and growing; multi-provider abstraction becomes more valuable with every new entrant
- >Enterprise readiness: Organizations want agent infrastructure with audit logs, credential isolation, and usage controls before they can approve production deployments
- >Rising Sun foundation: Internal demand from Eudaimonia, Kappa, and other portfolio projects creates immediate production load and reduces go-to-market risk
The Ask
Building the execution substrate for the AI agent economy.
As AI agents become standard components of software products, teams need a reliable, multi-tenant backend to run them — just as they use Stripe for payments or Twilio for messaging. AAASP is that layer: production-grade, provider-agnostic, and built on the most fault-tolerant runtime available.
**Opportunity**: Capture the agent execution infrastructure layer before the market consolidates around cloud-vendor offerings that lock teams into a single provider and model catalog.
**Rising Sun** · risingsun.name · March 2026