# The Edge Security And Logging Layer: Principles For A Gateway That Sits In Front Of Everything

**version** v0.27.58
**date** 18 May 2026
**from** Human (project lead)
**to** Architect, Developer (lead), Security, @Dev
**type** Arch brief

---

## What This Is

The foundational principles brief for a new piece of infrastructure: **a logging, routing, and security layer that sits in front of the entire solution**. It is a mix of routing, WAF (web application firewall), reverse-proxy, and observability, built the way we build everything else: substrate-independent, controllable, refactorable, and LLM-engineerable.

This is the first of a series of briefs on security tools built on top of our platform. It establishes the principles and the shape before the subsequent briefs detail specific capabilities. The voice memo was explicit that the principles matter most here, because they determine the whole design: **"this is what will determine the whole thing."**

The motivation is concrete and current. The project lead has two real problems today: **insufficient real-time visibility into website traffic**, and **a CloudWatch-plus-Firehose logging pipeline that is expensive and not cost-effective**. Layered on top is a broader frustration: **AWS WAF feels weirdly priced, rent-like, and over-complex** for what is actually needed. The goal is something simpler, more controllable, cheaper, and more capable, built on our own primitives.

This brief covers the problem, the principles, the capabilities, the "make the site hostile to bad traffic" goal, and the naming question. The specific implementation details come in the subsequent briefs in this series.

## The Problem Being Solved

Three concrete problems converge:

**Problem 1: Insufficient real-time traffic visibility.** Right now there is no good way to see what is hitting the website in real time. Who is requesting what, how often, from where, with what patterns. This is an operational blind spot that matters for cost, security, and performance.

**Problem 2: The CloudWatch + Firehose logging pipeline is expensive.** The current logging approach (CloudWatch logs piped through Kinesis Firehose) ends up costing more than it should for the value it delivers. The voice memo named it directly: **"in a weird way, it's actually not cost effective at all."** Logging should not be a major cost line; right now it is.

**Problem 3: AWS WAF feels rent-like and over-complex.** AWS WAF is priced per rule, per request, per web ACL, in a way that feels like a recurring rent rather than a tool you own. It also adds significant configuration complexity. The voice memo: **"feels really weirdly priced, and it's the whole thing of feels like a rent."** For our use case, a simpler, owned, controllable alternative is preferable.

Underlying all three is a cost-control concern with teeth. The voice memo named a specific fear: **a small JavaScript bug once caused runaway traffic (repeated redirects, repeated hits to the same page), and the worry is that another such bug could introduce a crazy amount of cost and skew everything.** The edge layer is partly insurance against this class of self-inflicted cost explosion.

## The Principles

The voice memo was emphatic that the principles come first. Here they are, in priority order.

### Principle 1: Substrate-Independent

**The layer must work anywhere, not just on CloudFront.** CloudFront is the first deployment target because it gives us an always-on edge presence, but the design must not depend on CloudFront. The same layer must run:

- On CloudFront (via CloudFront Functions or Lambda@Edge) for the always-on edge case
- On Lambda functions and cloud functions (GCP, etc.) for serverless deployment
- In containers and our own cloud-compute environment for the controllable case
- Anywhere a request can be intercepted and processed

This is the substrate spectrum principle from earlier work, applied to the edge layer. The first design may take CloudFront's specific capabilities and limitations into account, but the core must be **completely independent and isolated** from any single substrate.

The payoff: we can run the same security and logging logic on the cheapest appropriate substrate per workload, move it between substrates without rewriting it, and avoid lock-in to any provider's edge offering.

### Principle 2: It Sits In Front Of Everything

**This layer is the single entry point to the entire solution.** Every request passes through it before reaching any backend. That position is what makes it valuable: it is the one place where routing, caching, security, tagging, and logging can all be applied consistently.

The capabilities it provides from that position:

| Capability | What It Does |
|------------|--------------|
| **Routing** | Direct requests to the right backend (vault server, static site, ephemeral compute, etc.) |
| **Caching** | Serve cached responses where appropriate; reduce backend load |
| **Security** | Block, challenge, or allow requests based on rules; the WAF function |
| **Tagging** | Annotate requests with metadata for downstream processing |
| **Logging** | Capture what is happening, cheaply and in real time |
| **Cleanup** | Reject malformed, malicious, or wasteful traffic before it costs anything |

Each of these is a capability the subsequent briefs will detail. The principle here is that they all live in one layer, applied consistently, at the single entry point.

### Principle 3: Controllable And Refactorable

**We own this layer and can change it easily.** Unlike AWS WAF (where rules are configured through AWS's model and priced through AWS's meter), this layer is our code. We can:

- Refactor it freely as requirements evolve
- Test it thoroughly with our own test framework
- Reason about its cost precisely
- Extend it with custom logic AWS WAF would never support
- Move it between substrates as Principle 1 requires

The voice memo named the LLM-engineering angle: **we now have the ability to refactor very easily and to leverage the LLM's ability to engineer powerful solutions and test them.** A custom edge layer that would have been a major engineering investment five years ago is now tractable, because the build-and-refactor cost has dropped. This is the de-commoditisation pattern from earlier this week applied directly: AWS WAF is the commodity; our edge layer is the custom shield that absorbs its attrition (rent-like pricing, complexity, lock-in) and gives us a commodity-feeling experience we actually control.

### Principle 4: The Best Layer For Each Job

**Ask, per function, what is the best layer to execute it.** Some logic belongs at the CloudFront edge (fast rejection of obvious bad traffic). Some belongs in a Lambda (request-specific decisions). Some belongs in our compute (heavy analysis, LLM-driven decisions). The architecture should let us place each piece of logic at the layer where it runs best, and move it if the answer changes.

The voice memo named this as a key advantage: **"we can basically ask the question, what is the best layer to execute something? And we just apply the multiple different parties on it."** The edge layer is not monolithic; it is a set of functions that can each live at the appropriate layer, coordinated into a coherent whole.

## The "Make The Site Hostile To Bad Traffic" Goal

A specific goal worth stating prominently because the voice memo emphasised it: **make the site genuinely hostile to malicious traffic, buggy traffic, and wasteful traffic.**

This is partly security, partly cost control, partly resilience. The categories of traffic we want to be hostile to:

| Traffic Category | Why Hostile |
|------------------|-------------|
| **Malicious** | Attacks, probes, exploitation attempts, scrapers abusing resources |
| **Buggy** | Our own bugs (the runaway-redirect example); client bugs causing request storms |
| **Wasteful** | Bots hammering endpoints; misconfigured clients; retry storms |
| **Anomalous** | Traffic patterns that deviate sharply from normal; possible early signs of either attack or bug |

Being "brutal about what traffic we allow to hit our servers" serves three goals simultaneously:

- **Cost**: every request that gets rejected at the edge costs almost nothing; every request that reaches a backend costs compute
- **Security**: the fewer requests that reach backends, the smaller the attack surface
- **Resilience**: backends protected by an aggressive edge layer stay healthy under load that would otherwise overwhelm them

The runaway-cost insurance is worth dwelling on. **A single JavaScript bug that causes a redirect loop or repeated identical requests can, without an edge layer, drive enormous cost before anyone notices.** The edge layer should detect such patterns (the same client requesting the same thing at high frequency, redirect loops, abnormal request volumes) and cut them off automatically. This protects against our own mistakes as much as against attackers.

## Advanced Capabilities

Beyond the baseline (routing, caching, security, tagging, logging, cleanup), the voice memo named several advanced capabilities worth designing toward:

| Capability | What It Adds |
|------------|--------------|
| **PKI support** | Public-key infrastructure for authenticating requests; ties into the vault public-key work from earlier this week |
| **Extra security keys** | Custom key-based access checks for protected endpoints |
| **Custom security checks** | Arbitrary validation logic we define, beyond what a standard WAF offers |
| **Bot detection** | Identifying and handling automated traffic; allow good bots, block bad ones |
| **LLM-driven decisions** | The novel capability: using an LLM to make nuanced traffic decisions that rule-based systems cannot |

The LLM-driven angle is the genuinely new one. The voice memo framed the whole thing as **"an LLM agentic-driven cloud WAF"** (while acknowledging it is more than a WAF). The idea: for traffic that rule-based logic cannot cleanly classify, an LLM can make a nuanced judgement. This is not on the hot path for every request (too slow, too expensive), but it can be applied to ambiguous cases, to pattern analysis over logs, and to generating new rules from observed traffic. **Rules handle the common case at the edge; the LLM handles the ambiguous case and improves the rules over time.**

## The Naming Question

The voice memo explicitly flagged that this thing needs a good name and that "gateway" is not quite right. What it is:

- More than a WAF (it does routing, caching, logging, not just firewalling)
- More than a reverse proxy (it does security and analysis, not just forwarding)
- More than a gateway (the word is overloaded and does not capture the security and observability roles)
- Has nginx-like routing capabilities (the voice memo referenced past routing-rule work)
- Has a sentry / sentinel quality (it watches and guards)
- Is an edge entry point (it sits in front of everything)

Some naming candidates worth considering:

| Candidate | Connotation |
|-----------|-------------|
| **SG/Edge** | Already in use for the edge compute layer; might extend it or might clash |
| **SG/Gate** | Gateway-ish but shorter; the "gate" you pass through |
| **SG/Sentry** | Emphasises the watching/guarding role; possible trademark concern with the Sentry observability product |
| **SG/Sentinel** | Similar; guarding connotation; less trademark risk |
| **SG/Front** | Literal (it sits in front); clean |
| **SG/Warden** | Guarding connotation; distinctive |
| **SG/Ward** | Shorter; protective connotation |
| **SG/Perimeter** | Security connotation; the boundary |

The name should capture that it is the guarding, routing, observing layer at the perimeter. **SG/Sentinel or SG/Warden or SG/Ward** feel closest to the role; the decision is open and worth a short bikeshed. For this brief I will refer to it as **the edge layer** generically.

(Note: there is a naming relationship to be resolved with the existing SG/Edge compute layer. The edge security layer and the edge compute layer are related but distinct; whether they share a name or are clearly separated needs deciding.)

## How This Composes With Existing Work

The edge layer composes with several recent directions:

| Recent Work | How It Composes |
|-------------|-----------------|
| The substrate spectrum (day 65-66) | The edge layer runs on the appropriate substrate per function |
| SG/Edge compute + Edge Waker (day 66, day 67) | The edge layer routes to woken instances; the two edge concepts interrelate |
| Vault discovery + public keys (day 66) | PKI support in the edge layer uses the vault public-key infrastructure |
| The de-commoditising article (day 67) | The edge layer is the shield over AWS WAF/CloudWatch attrition |
| Unified observability session (day 65) | Edge-layer logs flow into observability |
| The TUI API (day 67) | The edge layer should expose a TUI and TUI API for operators to watch traffic |
| AppSec mini-tools (day 66) | The edge layer is itself a security tool; fits the mini-tools-on-vaults pattern |
| Pre-auth metering (day 64) | Edge-layer cost tracking integrates with metering |

The edge layer is not a standalone product; it is a layer that uses the platform's primitives and exposes its state through the platform's surfaces. The TUI connection is particularly natural: **real-time traffic visibility is exactly the kind of thing the SG/Edge TUI work from yesterday is built to show.**

## The Cost Argument

Worth being explicit about the cost case, since it is a primary motivation. The current state:

| Current Approach | Cost Shape |
|------------------|------------|
| CloudWatch logs + Kinesis Firehose | Per-GB ingestion, per-GB processing, per-GB delivery; adds up fast at volume |
| AWS WAF | Per-web-ACL monthly, per-rule monthly, per-million-requests; rent-like |
| No real-time visibility | Hidden cost: blind spots that allow waste and attack to go unnoticed |

The proposed edge layer:

| Proposed Approach | Cost Shape |
|-------------------|------------|
| Custom logging to our own storage | We control format, retention, destination; far cheaper at volume |
| Custom security rules in our code | No per-rule rent; the cost is the compute to run the rules, which is minimal at the edge |
| Built-in real-time visibility | The edge layer sees everything by definition; visibility is free once the layer exists |

The build cost is real (this is engineering work), but the run cost is dramatically lower, and the control is total. This is the classic build-the-shield calculation from the de-commoditising article: the commodity (AWS WAF + CloudWatch) introduces enough attrition (cost, complexity, blindness) that building the custom shield is justified, especially now that AI-augmented development drops the build cost.

## What This Series Will Cover

This brief establishes principles. The subsequent briefs in this security-tools series will detail:

- The specific routing capabilities and rule model
- The logging architecture (format, storage, real-time streaming, retention)
- The security rule engine (the WAF function: rules, challenges, blocks)
- The bot-detection approach
- The PKI and key-based access model
- The LLM-driven decision layer (for ambiguous traffic and rule generation)
- The cost-protection / runaway-detection mechanism
- The operator interface (TUI + TUI API for real-time traffic visibility)
- The CloudFront-specific first deployment and the path to substrate independence

Each gets its own brief. This one is the foundation they build on.

## What This Does Not Try To Be

Deliberate scope limits:

- **Not a full commercial WAF product (yet).** v1 is for our own infrastructure; productisation is a later question.
- **Not a CDN replacement.** CloudFront still does the CDN job; the edge layer adds logic on top.
- **Not a replacement for all AWS security services.** It addresses the WAF and logging cost specifically; other AWS security tooling stays where useful.
- **Not LLM-on-every-request.** The LLM is for ambiguous cases and rule generation, not the hot path.
- **Not a single-substrate solution.** Substrate independence is Principle 1; CloudFront-first does not mean CloudFront-only.

## Honest Risks

Three risks worth flagging:

**Risk 1: Building a custom edge security layer is security-sensitive work.** A bug in a WAF is a vulnerability. Mitigation: thorough testing (the test framework from day 66); start conservative (log-and-observe before block-and-reject); never make the edge layer a single point of failure that fails closed in a way that takes the whole site down.

**Risk 2: The edge layer becomes a performance bottleneck.** Everything passes through it; if it is slow, everything is slow. Mitigation: keep the hot path minimal; push heavy logic (LLM decisions, analysis) off the hot path; measure latency obsessively.

**Risk 3: Reinventing a mature commodity badly.** AWS WAF, while rent-like, is battle-tested. A naive custom replacement could miss attack classes AWS handles. Mitigation: do not try to match AWS WAF feature-for-feature; build the specific protections we need; lean on the LLM and observability to catch what static rules miss; be honest about what the custom layer does and does not protect against.

## Open Questions

| Question | Notes |
|----------|-------|
| The name? | SG/Sentinel, SG/Warden, SG/Ward, SG/Front candidates; bikeshed open |
| Relationship to the existing SG/Edge compute layer? | Related but distinct; naming and architecture boundary needs deciding |
| First deployment substrate: CloudFront Functions or Lambda@Edge? | CloudFront Functions cheaper but more limited; Lambda@Edge more capable but pricier; probably start with Functions for simple rules |
| Where do logs go? | Our own storage (S3 + vault?); format and retention to be defined in the logging brief |
| Fail-open or fail-closed on edge-layer error? | Probably fail-open for availability, with alerting; security-critical paths might fail-closed |
| How aggressive should default rules be? | Start conservative (observe), tighten based on real traffic |
| LLM decision latency budget? | Off-hot-path; seconds acceptable for ambiguous-case analysis |
| How does this interact with the multi-cloud direction? | Should run on GCP/Akamai edges too eventually; substrate independence enables this |

## Relationship To Previous Briefs

| Date | Document | Relationship |
|---|---|---|
| 17 May | `v0.27.55__article__de-commoditising-the-commodity.md` | The edge layer is the shield over AWS WAF/CloudWatch attrition |
| 17 May | `v0.27.55__arch-brief__tui-api-structured-surface-for-text-uis.md` | The edge layer exposes a TUI + TUI API for traffic visibility |
| 17 May | `v0.27.55__dev-brief__sg-edge-tui-first-five-screens.md` | The live-event-stream screen pattern applies directly to traffic monitoring |
| 16 May | `v0.27.45__arch-brief__vault-discovery-and-public-keys.md` | PKI support uses the vault public-key infrastructure |
| 16 May | `v0.27.45__strategy-brief__appsec-mini-tools-on-top-of-vaults.md` | The edge layer is a security tool fitting the mini-tools pattern |
| 16 May | `v0.27.45__dev-brief__on-demand-vault-provisioning-workflows.md` | The edge layer routes to woken instances |
| 16 May | `v0.27.45__strategy-brief__sg-compute-as-serverless-environment.md` | The substrate the edge layer can run on |
| 15 May | `v0.27.43__arch-brief__unified-observability-session.md` | Edge logs flow into observability |
| 14 May | `v0.27.41__strategy-brief__preauth-micropayments-monetisation-model.md` | Edge cost tracking integrates with metering |
| 13 May | `v0.27.40__arch-brief__cli-first-agent-architecture.md` | The edge layer follows the CLI-first, then-TUI pattern |

---

## Acceptance Criteria

| # | Criterion | Verification |
|---|-----------|-------------|
| 1 | The principles are documented and agreed as the foundation for the series | Team alignment |
| 2 | A name is chosen for the edge layer | Decision made; bikeshed closed |
| 3 | The substrate-independence principle is reflected in the v1 architecture | Core logic has no hard CloudFront dependency |
| 4 | The "sits in front of everything" position is established | Single entry point design |
| 5 | The six baseline capabilities (routing, caching, security, tagging, logging, cleanup) are scoped | Each has a follow-on brief |
| 6 | The runaway-cost-protection mechanism is specified | Self-inflicted-cost insurance designed |
| 7 | Real-time traffic visibility is achievable through the layer | The blind spot is addressed |
| 8 | The cost case versus CloudWatch + Firehose + WAF is quantified | Real numbers, not assertions |
| 9 | The relationship to the SG/Edge compute layer is clarified | Boundary defined |
| 10 | The series of follow-on briefs is planned | Roadmap for the security-tools thread |

---

This document is released under the Creative Commons Attribution 4.0 International licence (CC BY 4.0).