Skip to content

PRODUCTION INVESTIGATION

Don't search dashboards.Build explanations.

AutoObserve investigates production incidents by gathering evidence, generating competing hypotheses and continuously testing which explanation best fits what actually happened.

Evidence in. Explanations out. Uncertainty included.

THE DISTINCTION

An investigation isn't finding a metric. It's building the best explanation supported by evidence.

Dashboards help engineers explore what they already know to ask. Investigation determines what should be asked next.

ALERTING

What changed?

  1. Signal
  2. Condition
  3. Situation

INVESTIGATION

What most likely happened?

  1. Evidence
  2. Hypotheses
  3. Reasoning
  4. Explanation

MANUAL INVESTIGATION

  1. Incident
  2. Choose dashboard
  3. Form question
  4. Write query
  5. Inspect result
  6. Switch telemetry
  7. Form hypothesis
  8. Test
  9. Repeat

AUTOBSERVE

  1. Incident
  2. Evidence
  3. Hypotheses
  4. Tests
  5. Explanation

01 — GATHER

Start with evidence, not assumptions.

AutoObserve gathers observations across telemetry, changes and system context before deciding which explanation deserves belief.

OBSERVATION

Latency increased after deployment.

EVIDENCE

Supports deployment hypothesis.

NOT CONCLUSION

Deployment caused latency.

Evidence

14:31:31

  • Evidence · Metric

    Checkout latency

    strong

    Baseline 241 ms · Observed 684 ms

    Observed
    684 ms
    Baseline
    241 ms
    Deviation
    +184% vs baseline
    Source
    Metrics
    Backend
    Prometheus
    DSL
    PromQL
    Coverage
    98%
    Time
    14:31:00
  • Evidence · Deployment

    checkout-api deployment

    verified

    83 sec before degradation

    Observed
    v2.14.2 → v2.14.7
    Source
    Deployment events
    Backend
    Kubernetes API
    DSL
    Event Query
    Coverage
    100%
    Time
    14:30:00
  • Evidence · Trace

    Failed traces

    strong

    Failed spans +312%

    Observed
    checkout → payment
    Source
    Traces
    Backend
    Tempo
    DSL
    TraceQL
    Coverage
    94%
    Time
    14:31:31

02 — HYPOTHESISE

Don't jump to a root cause. Build competing explanations.

WHY IS CHECKOUT FAILING?

CURRENT HYPOTHESES

  • Hypothesis

    checkout-api v2.14.7 caused checkout degradation

    Confidence 74%

    Supporting

    • Deployment 83 sec earlier
    • Failed traces originate at checkout-api

    Contradicting

    None material

    Missing / unavailable

    • CPU profiling unavailable

    Missing evidence

    • Need application logs

    Next test

    Inspect checkout-api logs after deployment

03 — TEST

Evidence matters only when it changes what we believe.

AutoObserve tests each hypothesis against expected versus observed order, topology, and missing indicators—not against a single correlating metric.

Hypothesis

Payment provider caused checkout failures

Expected if true

  1. Payment degradation
  2. Checkout degradation

Observed

  1. Checkout degradation
  2. Payment degradation

Result

Temporal order contradicts hypothesis.

Confidence 27% → 12%

Status · WEAKENED

Causal reasoning

Time →

  1. 14:31:00

    CHECKOUT

  2. 14:31:09

    PAYMENT

  3. 14:31:16

    ORDERS

Observed order

  1. Checkout
  2. Payment
  3. Orders

Dependency direction

  1. Checkout
  2. Payment
  3. Orders

Propagation is consistent with checkout as the origin.

Payment · DOWNSTREAM SYMPTOM

Not root cause.

Four epistemic states

  • SUPPORTING EVIDENCE

  • CONTRADICTING EVIDENCE

  • EXPECTED BUT ABSENT

  • MISSING / UNAVAILABLE

Database hypothesis

Supporting

  • Minor CPU increase

Contradicting

  • Query latency normal
  • Connection pool healthy
  • Replicas healthy
  • Failures begin upstream

Expected but absent

  • No lock contention

Missing / unavailable

  • Storage-level telemetry unavailable

04 — UPDATE

Confidence changes when evidence changes.

Confidence is an evidence-weighted belief score for the current investigation—not a claim of calibrated statistical certainty.

ConfidenceConfidence expresses how strongly the current explanation is supported by available evidence, including contradictions and gaps.

Confidence

0%25%50%75%100%38%57%74%68%82%91%

05 — EXPLAIN

The root cause is the current best explanation.

RCA is the output of investigation—not the product category. AutoObserve maintains the strongest explanation supported by evidence available right now.

CURRENT BEST EXPLANATION

checkout-api v2.14.7

likely caused checkout degradation

Confidence
91%
Strength
HIGH

WHY

  • Degradation began 83 sec after deployment
  • New checkout error signature appeared
  • Failed traces originate at checkout-api
  • Downstream propagation matches topology
  • Database hypothesis contradicted
  • Payment-provider hypothesis contradicted

UNCERTAINTY

  • CPU profiling unavailable

ALTERNATIVE EXPLANATIONS

IMPACT

Checkout → Payment → Orders

RECOMMENDED NEXT STEP

Review / rollback checkout-api v2.14.7

See how AutoObserve reached the conclusion.

  1. 14:31:18

    INCIDENT

    Checkout degradation detected

  2. 14:31:21

    EVIDENCE

    Latency +184%

  3. 14:31:24

    HYPOTHESES

    4 explanations generated

  4. 14:31:31

    TRACE EVIDENCE

    Checkout deployment 38% → 57%

  5. 14:31:42

    CHANGE CORRELATION

    Deployment found 57% → 74%

  6. 14:31:48

    DATABASE TEST

    Database contradicted 38% → 18%

  7. 14:31:50

    CONTRADICTORY PATTERN

    Inconsistent trace pattern 74% → 68%

  8. 14:31:58

    TOPOLOGY

    Propagation consistent 68% → 82%

  9. 14:32:11

    LOG EVIDENCE

    Error signature corroborates 82% → 91%

  10. 14:32:12

    EXPLANATION

    checkout-api v2.14.7 · 91%

INVESTIGATION SYSTEM

One workspace. The entire investigation.

Raw telemetry is drill-down evidence. The workspace defaults to explanation, hypotheses, and tests—not a wall of charts.

Explanation → Hypotheses → Evidence → Raw telemetry

Investigation

Checkout degradation

Current best explanation

checkout-api v2.14.7

likely caused checkout degradation · Confidence 91%

Supporting

  • Degradation began 83 sec after deployment
  • New checkout error signature appeared
  • Failed traces originate at checkout-api

Contradicting

None material

Confidence

38% → 57% → 74% → 68% → 82% → 91%

Next recommended action

Review / rollback checkout-api v2.14.7

Reason about where failures propagate.

Topology answers where failure can propagate, which symptoms are downstream, whether a causal path is plausible, and what the blast radius is—not merely a service map.

Checkout is the likely origin — not a declared root cause

  • checkout-api

    ORIGIN

  • Payment

    DOWNSTREAM

  • Orders

    DOWNSTREAM

Investigation asks the question. Multi-DSL finds the evidence.

Multi-DSL is the evidence acquisition layer for machine investigation—not a generic multi-query UI.

  1. Hypothesis
  2. What do we need to know?
  3. Metrics · Logs · Traces · Events
  4. PromQL · LogQL · TraceQL · EventQL
  5. Evidence
  6. Hypothesis test
  7. Confidence update

Autonomous evidence acquisition

Missing
Application logs
Ask
Find new error signatures after deployment.
Planner
Multi-DSL Planner
Query
LogQL
Result
New error signature found
Confidence
82% → 91%
Explore Multi-DSL →

ONE INTELLIGENCE SYSTEM

Investigation explains. The platform supplies evidence.

Competing hypotheses drive evidence requests through Multi-DSL—updated confidence returns to AIDDE for the next decision.

INVESTIGATION

What most likely happened?

  • INVESTIGATION

    What explains this?

    1. 01Confidence insufficient
    2. 02Request investigation
    3. 03Generate competing hypotheses
    4. 04Test against evidence
    5. 05Updated evidence
    6. 06AIDDE reassesses
    Explore Investigation →
  • TOPOLOGY

    What is related?

    1. 01Root vs downstream
    2. 02Blast radius
    3. 03Causal consistency
    4. 04Suppression with evidence retained
    Explore Topology →
  • MULTI-DSL

    What evidence do we need?

    1. 01Evidence request
    2. 02Investigation planner
    3. 03Metrics · Logs · Traces
    4. 04Unified evidence
    5. 05AIDDE reassessment
    Explore Multi-DSL →

Evidence returns

→ AIDDE reassesses

TECHNICAL ARCHITECTURE

An investigation system, not an AI answer box.

Enough architecture to reward technical visitors — no model-provider diagrams, vector databases, or agent-framework boxes.

Incident

  1. Investigation Context

  2. Hypothesis Engine

  3. Evidence Needed

  4. Multi-DSL

  5. Evidence Store

  6. Topology

    Reasoning Engine

  7. Confidence Evaluation

  8. Explanation Builder

CURRENT BEST EXPLANATION

Developer representation

Epistemic model — competing hypotheses and evidence gaps, not a single root_cause field.

{
  "incident": "checkout-degradation",
  "status": "investigating",
  "current_explanation": {
    "candidate": "checkout-api:v2.14.7",
    "confidence": 0.91
  },
  "hypotheses": [
    {
      "candidate": "checkout-api:v2.14.7",
      "confidence": 0.91,
      "state": "strengthened"
    },
    {
      "candidate": "postgresql",
      "confidence": 0.18,
      "state": "weakened"
    },
    {
      "candidate": "payment-provider",
      "confidence": 0.12,
      "state": "weakened"
    }
  ],
  "evidence_gaps": [
    "cpu_profile"
  ]
}

For Developers →