
# Composable AI Micro‑services Architecture for Scalable Security Questionnaire Automation

Enterprises are drowning in an ever‑growing tide of security questionnaires, vendor assessments, and compliance audits. Traditional monolithic tools struggle to keep up, especially when they must integrate with disparate product ecosystems, support multilingual requests, and provide real‑time audit trails.  

A **composable micro‑services architecture**, built around large language models (LLMs) and retrieval‑augmented generation (RAG), offers a way to scale automation while preserving the flexibility and governance that regulated industries demand. In this guide we’ll:

* Outline the core design principles that keep the system **secure, auditable, and extensible**.  
* Walk through a reference implementation diagrammed with Mermaid.  
* Show how each service can be deployed independently on Kubernetes, serverless FaaS, or edge runtimes.  
* Provide concrete best‑practice recommendations for data governance, observability, and continuous improvement.

> **TL;DR:** Break the questionnaire automation platform into small, well‑defined services, let LLMs sit behind a stateless inference layer, and use event‑driven pipelines to maintain a single source of truth for evidence and version control.

---

## 1. Why Compose Rather Than Build a Giant Monolith?

| Monolithic Approach | Composable Micro‑services |
|---------------------|---------------------------|
| Single codebase, hard to scale specific workloads (e.g., LLM inference). | Independent scaling – AI inference can run on GPU nodes, while storage stays on cost‑effective object stores. |
| Tight coupling makes updates risky; a bug in the UI may bring down the entire system. | Loose coupling through asynchronous events or HTTP APIs isolates failures. |
| Limited language‑agnostic integration – often locked to one stack. | Polyglot support – each service can be written in the language best suited for its task (Go for auth, Python for LLM orchestration, Rust for high‑throughput pipelines). |
| Auditing and compliance become a nightmare as logs are intertwined. | Centralized event store + immutable audit log provides a clear, queryable trail for regulators. |

The **Composable** model embraces the *“you build what you need, and you replace what you don’t”* philosophy. It matches the dynamic nature of security questionnaires, where new control frameworks (e.g., [ISO 27001 Rev 2](https://www.iso.org/standard/27001)) appear regularly and teams must quickly adapt.

---

## 2. Core Architectural Pillars

1. **Stateless API Gateway** – entry point for UI, SaaS connectors, and external tools. Handles authentication, request validation, and throttling.  
2. **Domain‑Specific Micro‑services** – each encapsulates a bounded context:
   * **Questionnaire Service** – stores questionnaire metadata, versioning, and task assignments.  
   * **Evidence Service** – manages artifacts (policies, screenshots, audit logs) in an immutable object store.  
   * **AI Orchestration Service** – composes prompts, runs RAG pipelines, and returns answer drafts.  
   * **Change‑Detection Service** – watches evidence updates, triggers re‑evaluation of affected answers.  
   * **Notification Service** – pushes Slack, Teams, or email events to stakeholders.  
3. **Event Bus (Kafka / Pulsar)** – guarantees at‑least‑once delivery of domain events (e.g., `EvidenceUploaded`, `AnswerDrafted`).  
4. **Observability Stack** – OpenTelemetry traces across services, Prometheus metrics, and Loki logs.  
5. **Policy‑as‑Code Engine** – evaluates compliance rules (written in Rego or OPA) before an answer is marked “final”.  

All services communicate via **gRPC** (for low latency) or **REST** (for external integrations). The design encourages *dumb pipes, smart endpoints*—the business logic lives where it belongs, while the bus merely transports messages.

---

## 3. Data Flow – From Question to Auditable Answer

Below is a Mermaid diagram that visualises a typical request lifecycle.

```mermaid
flowchart TD
    subgraph UI["User Interface"]
        UI1["\"Web UI\""] -->|Submit questionnaire| AG["\"API Gateway\""]
    end

    AG -->|Auth & Validate| QMS["\"Questionnaire Service\""]
    QMS -->|Fetch template| AIOS["\"AI Orchestration Service\""]
    AIOS -->|Retrieve relevant evidence| ES["\"Evidence Service\""]
    ES -->|Evidence objects| AIOS
    AIOS -->|Generate draft answer| RAG["\"RAG Pipeline\""]
    RAG -->|LLM output| AIOS
    AIOS -->|Store draft| QMS
    QMS -->|Emit AnswerDrafted| EB["\"Event Bus\""]
    EB -->|Trigger| CDS["\"Change‑Detection Service\""]
    CDS -->|Re‑run if evidence changed| AIOS
    CDS -->|Emit AnswerUpdated| EB
    EB -->|Notify| NS["\"Notification Service\""]
    NS -->|Push to Slack/Email| UI

    style UI fill:#f9f,stroke:#333,stroke-width:2px
    style AG fill:#bbf,stroke:#333,stroke-width:1px
    style QMS fill:#bfb,stroke:#333,stroke-width:1px
    style AIOS fill:#ffb,stroke:#333,stroke-width:1px
    style ES fill:#fbb,stroke:#333,stroke-width:1px
    style RAG fill:#fdd,stroke:#333,stroke-width:1px
    style CDS fill:#ddf,stroke:#333,stroke-width:1px
    style NS fill:#cfc,stroke:#333,stroke-width:1px
```

**Key moments in the flow**:

1. **User submits** a new questionnaire or selects an existing one.  
2. **API Gateway** validates JWT, checks rate limits, forwards to the Questionnaire Service.  
3. The **Questionnaire Service** pulls the questionnaire template and posts an event to the AI Orchestration Service.  
4. **AI Orchestration** performs a *retrieval step*—it queries the Evidence Service for all artifacts relevant to the current control (using vector similarity or keyword match).  
5. The retrieved contexts, together with the prompt template, feed a **RAG pipeline** (e.g., `openAI/gpt‑4o‑preview`).  
6. The draft answer is stored back in the Questionnaire Service, marked *“pending review.”*  
7. The **Change‑Detection Service** watches for new evidence uploads. If a policy is updated, it re‑triggers the RAG pipeline for impacted answers.  
8. Final reviewers accept or edit the draft; upon acceptance, the **Policy‑as‑Code Engine** validates that the answer satisfies all rule constraints before committing it to an immutable audit log.  

---

## 4. Implementation Details

### 4.1. API Gateway (Envoy + OIDC)

* **Routing** – `POST /questionnaires/:id/answers` → `questionnaire-service`.  
* **Security** – Enforce scopes (`questionnaire:write`).  
* **Rate limiting** – 100 requests/min per tenant to protect downstream LLM costs.

### 4.2. Questionnaire Service (Go)

```go
type Questionnaire struct {
    ID          string            `json:"id"`
    Version     int               `json:"version"`
    Controls    []Control        `json:"controls"`
    Drafts      map[string]Answer `json:"drafts"` // key = control ID
    AssignedTo  map[string]string `json:"assigned_to"` // userID
}
```

* Uses **PostgreSQL** for relational data, **EventStoreDB** for domain events.  
* Exposes gRPC methods `GetTemplate`, `SaveDraft`, `FinalizeAnswer`.

### 4.3. Evidence Service (Python + FastAPI)

* Stores files in **MinIO** or **AWS S3** with bucket‑level encryption.  
* Indexes content in **Qdrant** (vector DB) for similarity search.  
* Provides an endpoint `POST /search` that accepts a query and returns top‑k artifact IDs.

### 4.4. AI Orchestration Service (Python)

```python
def generate_answer(question: str, evidence_ids: List[str]) -> str:
    evidence = fetch_evidence(evidence_ids)
    context = "\n".join(evidence)
    prompt = f"""You are a compliance specialist.
    Using the following evidence, answer the question concisely:\n{context}\n\nQuestion: {question}"""
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role":"system","content":prompt}]
    )
    return response.choices[0].message.content
```

* **RAG** – Combine vector search with a *system prompt* that instructs the model to cite evidence IDs.  
* **Caching** – Store generated responses for 24 h to avoid duplicate LLM calls.

### 4.5. Change‑Detection Service (Rust)

* Subscribes to `EvidenceUploaded` events.  
* Computes a **hash** of the new artifact and runs a **diff** against existing evidence linked to each control.  
* If the diff exceeds a configurable threshold, it publishes `AnswerRequiresRegen`.

### 4.6. Notification Service (Node.js)

* Listens to `AnswerDrafted`, `AnswerFinalized`, `AnswerRequiresRegen`.  
* Formats Slack blocks, Teams Adaptive Cards, or email templates.  
* Supports **deduplication** – only notifies once per change per questionnaire.

---

## 5. Security & Governance

| Concern | Mitigation |
|---------|------------|
| **Data Leakage** – LLM prompts may contain sensitive policy text. | Use **on‑prem LLM inference** (e.g., Llama 3.2) behind a VPC. Mask PII before sending to external APIs. |
| **Unauthorized Evidence Access** | Enforce fine‑grained ACLs using OPA policies in the Evidence Service. |
| **Model Drift** – Answers degrade over time. | Schedule **periodic evaluation** against a benchmark corpus and retrain prompt templates. |
| **Auditability** | Every state transition is recorded in an immutable event log stored on **WORM S3**. |
| **Compliance with GDPR/CCPA** | Implement **right‑to‑be‑forgotten** workflow that purges user‑specific evidence from vector DB and object store ([GDPR](https://gdpr.eu/)). |
| **Compliance with ISO 27001** | Validate that evidence retention, encryption, and access‑control policies align with the ISO 27001 standard. |
| **HIPAA / SOC 2** | For health‑care or SaaS providers, extend OPA rules to enforce the required safeguards. |

---

## 6. Scaling Strategies

1. **Horizontal Pod Autoscaling (HPA)** – Scale the AI Orchestration pods based on GPU utilization (`nvidia.com/gpu`).  
2. **Burst‑able Queues** – Use Kafka partitioning to isolate high‑traffic tenants.  
3. **Cold‑Start Reduction** – Keep a warm pool of containers for the LLM inference server (e.g., using **KEDA** with a custom scaler).  
4. **Cost Controls** – Apply **token‑based budgeting** per tenant; throttle or charge over‑usage automatically.

---

## 7. Observability & Continuous Improvement

* **Distributed Tracing** – OpenTelemetry spans from UI request → API Gateway → AI Orchestration → RAG → Evidence Service.  
* **Metrics** – `answer_draft_latency_seconds`, `evidence_upload_bytes`, `llm_token_usage`.  
* **Log Aggregation** – Structured JSON logs with `request_id` propagated across services.  
* **Feedback Loop** – After answer finalization, capture reviewer comments (`review_score`). Feed these into a **reinforcement learning** model that adjusts prompt temperature or selects alternative evidence sources.

---

## 8. Step‑by‑Step Migration Path for Existing Teams

| Phase | Goal | Activities |
|-------|------|------------|
| **0 – Discovery** | Map current questionnaire workflow. | Identify data sources, define control taxonomy. |
| **1 – Build Foundations** | Deploy API Gateway, authentication, and base services. | Containerize `questionnaire-service` and `evidence-service`. |
| **2 – Introduce AI** | Run RAG on a pilot questionnaire. | Use a sandbox LLM, manually verify drafts. |
| **3 – Event‑Driven Automation** | Wire the Change‑Detection pipeline. | Enable auto‑regen on evidence update. |
| **4 – Governance Harden** | Add OPA policies, immutable audit logs. | Switch to production LLM (on‑prem). |
| **5 – Scale & Optimize** | Auto‑scale GPU pods, implement cost controls. | Deploy observability stack, set SLOs. |

By incrementally adopting the composable architecture, teams avoid the “big‑bang” risk and can demonstrate early ROI (often a **30‑50 % reduction** in questionnaire turnaround).

---

## 9. Future‑Proofing the Stack

* **Federated Learning** – Train lightweight adapters on each tenant’s data without moving the raw evidence offsite, enhancing answer relevance while respecting data sovereignty.  
* **Zero‑Trust Service Mesh** – Use **Istio** or **Linkerd** with mutual TLS to secure intra‑service traffic.  
* **Semantic Governance** – Extend the Policy‑as‑Code engine to validate not only answer content but also the **semantic similarity** between evidence and control language.  
* **Generative Traceability** – Store the exact LLM temperature, top‑p, and system prompt alongside each answer for forensic inspection.

---

## 10. Conclusion

A composable micro‑services architecture transforms security questionnaire automation from a **painful manual chore** into a **scalable, auditable, and continuously improving engine**. By decoupling responsibilities, leveraging LLMs via a stateless RAG layer, and wiring everything together with an event‑driven backbone, organizations can:

* Respond to vendor assessments in minutes instead of days.  
* Keep compliance evidence always up‑to‑date with automated change detection.  
* Provide regulators with a clear, immutable audit trail.  

Start small, iterate fast, and let the micro‑services philosophy guide you toward a future where compliance is a **feature**, not a bottleneck.

---

## See Also

- [OPA Policy-as-Code for Micro‑services](https://www.openpolicyagent.org/docs/latest/policy-language/)