Back to Blog
Technical Deep DiveApril 8, 202615 min read

Building a 5-Signal Scoring Engine

NG

Nick Gallick

Founder

The Five Signals

Hipp0's retrieval engine scores every candidate decision against the current query using five weighted signals. Each signal captures a different dimension of relevance, and together they produce a composite score that dramatically outperforms single-signal approaches like pure semantic search.

  • directAffect (weight: 0.30) — Measures whether the decision directly impacts the current task, project, or component being worked on. This is the strongest signal because direct relevance trumps everything.
  • tagMatch (weight: 0.20) — Compares structured tags (technology, domain, scope) between the query context and candidate decisions. Provides fast, precise filtering.
  • personaMatch (weight: 0.25) — Scores how relevant the decision is to the querying agent's role. A frontend agent benefits more from UI decisions than database schema changes.
  • semanticSimilarity (weight: 0.25) — Cosine similarity between the query embedding and the decision embedding, computed via pgvector. The classic RAG signal, but here it is only one of five.
  • Temporal freshness (multiplier) — A decay multiplier that boosts recent decisions and attenuates stale ones. Configurable half-life per project.

Scoring Pipeline

The scoring pipeline executes in three phases. First, a coarse filter narrows candidates using tag overlap and project scope (eliminating 80-90% of the graph). Second, the remaining candidates are scored in parallel across all five signals. Third, scores are combined using the weighted formula and the temporal multiplier is applied. The entire pipeline completes in under 50ms for graphs with 10,000+ decisions.

// Composite score calculation
const score =
  directAffect  * 0.30 +
  tagMatch      * 0.20 +
  personaMatch  * 0.25 +
  semanticSim   * 0.25;

const finalScore = score * temporalDecay(decision.createdAt, halfLifeDays);

Benchmark Results

We evaluated the 5-signal engine against naive RAG (semantic-only), tag-filtered RAG, and a hybrid semantic+recency baseline across 2,400 retrieval queries drawn from real-world agent workflows.

  • Recall@5: 78% (+39% over naive RAG, +22% over hybrid baseline)
  • Recall@10: 99%
  • MRR (Mean Reciprocal Rank): 0.94

The directAffect signal alone accounts for roughly 40% of the improvement, confirming that structural relevance (does this decision affect what I am working on?) is far more valuable than embedding similarity for decision retrieval.

H0C Compression

Once decisions are ranked, Hipp0 compiles them into the H0C (Hipp0 Compiled) format — a structured, token-efficient representation that achieves 10-12x token reduction compared to raw decision text. H0C preserves all critical fields (title, rationale, confidence, tags, relationships) while eliminating redundancy. This means agents receive richer context within the same token budget.

// Example H0C output (abbreviated)
{
  "h0c_version": "1.0",
  "compiled_for": "backend-agent",
  "decisions": [
    {
      "id": "d_7f3a",
      "title": "Use event sourcing for audit trail",
      "confidence": 0.92,
      "signals": { "composite": 0.87 },
      "rationale": "Immutable log required for compliance..."
    }
  ],
  "token_count": 342
}