Skip to content
Arxo Arxo

LLM Architecture

LLM Architecture

Section titled “LLM Architecture”

This guide shows how to run an end-to-end architecture hardening workflow for llm_integration.

Workflow

Section titled “Workflow”
  1. Audit the current system.
  2. Triage findings by severity and evidence weight.
  3. Apply targeted fixes by risk category.
  4. Enforce policy in CI.
  5. Prevent regression with baseline checks.

1) Run an Audit

Section titled “1) Run an Audit”
Terminal window
# Focused LLM architecture metric
arxo analyze --path . --metric llm_integration --format json
Terminal window
# Full AI preset (includes llm_integration)
arxo analyze --path . --preset ai --format json

For MCP-based workflows, use check_llm_integration.

2) Triage Findings

Section titled “2) Triage Findings”

Prioritize in this order:

  1. llm.pii_leakage_risk, llm.prompt_injection_surface
  2. llm.observability_gap, llm.fallback_absence, llm.rate_limit_absence
  3. llm.eval_harness_absence, governance/versioning gaps
  4. Cost and architecture optimization gaps

Then review the Remediation Playbook.

3) Apply Fixes by Track

Section titled “3) Apply Fixes by Track”
  • Safety track: PII, prompt injection, output handling, telemetry redaction.
  • Reliability track: retries/backoff, timeouts, fallback, idempotency/cache.
  • Governance track: templates, model pinning, tool policies, eval harness.
  • Cost track: token budgets, context controls, usage attribution.

4) Enforce in CI

Section titled “4) Enforce in CI”

Use policy gates from Policy and CI Gates.

Terminal window
arxo analyze --path . --preset ai --config arxo.yml --fail-fast

Recommended rollout:

  1. Start with warning-level thresholds.
  2. Fix recurrent hotspots.
  3. Promote key safety/reliability checks to errors.

5) Monorepo and Incremental Rollout

Section titled “5) Monorepo and Incremental Rollout”
  • Start on one critical service/workspace.
  • Apply baseline no-regression checks before strict absolute thresholds.
  • Expand enforcement to remaining workspaces once trend is stable.

Troubleshooting with Diagnostic Signals

Section titled “Troubleshooting with Diagnostic Signals”

llm.blast_radius_available = 0

Section titled “llm.blast_radius_available = 0”

Call graph context was unavailable; blast radius and taint-backed flows may be incomplete.

llm.pii_taint_used = 0

Section titled “llm.pii_taint_used = 0”

PII detector used fallback path instead of taint analysis.

llm.pii_fallback_reason

Section titled “llm.pii_fallback_reason”
  • 1: no call graph or no call sites
  • 2: taint propagation failed
  • 3: source detection failed

Use these signals before tuning policy thresholds.

Section titled “Related Reading”