LLMs Are Smart, So Why Don’t They Know How Our Company Works? (opens in new tab)

LLMs handle public knowledge well but struggle with company-specific questions because relevant evidence is scattered across documents, code, meetings, and chat—and may be outdated or contradictory. The post argues that this is not merely a search problem: organizations need a shared layer for managing trustworthy context. Topic addresses this by extracting source-aware units, linking concepts and relationships, and verifying their freshness, consistency, and evidentiary support.

Why Search Alone Is Not Enough

• Search retrieves relevant text but cannot determine whether it is current, authoritative, or consistent with other sources.
• A retry-policy example might contain:
  • Documentation saying requests fail immediately
  • A meeting discussion proposing three retries
  • Code currently implementing two retries
• Agents must still determine:
  • Whether the meeting produced a final decision
  • Which source is newer
  • Whether the code reflects an intentional change or an unfinished implementation
  • Whether different sources describe the same behavior
• Topic provides a shared context layer so humans and LLMs use the same sources, relationships, freshness information, and conflict states.

Six Dimensions of Trust

Rather than compressing trust into one score, Topic evaluates six separate dimensions:

• Granularity: Whether the context is a meaningful, independently manageable unit
• Faithfulness: Whether the source actually supports the claim
• Staleness: Whether the evidence remains valid
• Canonicality: Whether different names refer to the same entity
• Consistency: Whether sources are compatible
• Coverage: Whether important evidence or perspectives are missing

Different checks use different methods: rules and hashes for deterministic validation, LLMs for semantic interpretation, and humans for ambiguous or high-impact decisions.

Ingesting Documents, Chat, and Code

Topic normalizes information into a common ContentUnit containing source type, unit type, original URI, content, hashes, timestamps, and source-specific metadata. It uses different boundaries for each source rather than splitting everything into fixed-size text chunks.

Structured Document Sections

• Markdown documents are divided by heading hierarchy.
• Parent headings are preserved to retain context.
• Long sections are split only when necessary.
• URLs, document paths, and creation or modification times remain attached to the unit.

Conversation Threads

• Entire messenger threads are treated as the semantic unit, not individual messages.
• Summaries preserve:
  • Technical identifiers such as function names and file paths
  • Questions, alternatives, and final outcomes
  • Decisions versus unresolved issues
• The system avoids inventing consensus and ignores threads containing only casual conversation.

Code Symbols and Semantic Cards

• Parsers extract functions, classes, file paths, line ranges, imports, and other symbols without using an LLM.
• Multiple symbols are then grouped into code semantic cards describing business behavior.
• Cards retain domain terms, code identifiers, source spans, and the relevant commit SHA.
• LLM-generated cards are checked against actual files, line ranges, supporting spans, and duplicate-card patterns.
• Cards are an intermediate layer for connecting code to business concepts, not a replacement for the code itself.

Extracting Concepts and Relationships

• Topic extracts concept candidates and supporting evidence from each content unit.
• It preserves the relationship between every concept and its original evidence.
• Similar names are not automatically merged merely because they appear close in meaning.
• Concepts can be consolidated into canonical entities only when sufficient evidence exists.

Human Review for Ambiguous Terminology

• Normalization and embeddings can identify obvious duplicates.
• Internal abbreviations and aliases may require organizational knowledge.
• Topic creates synonym proposals with their supporting context.
• Humans approve or reject ambiguous aliases; rejected proposals are remembered to prevent repeated suggestions.

Typed Document–Code Relationships

Topic distinguishes among:

• supported_by: code behavior supports the document’s claim
• contradicted_by: code behavior conflicts with the document
• mentions: both refer to the same area, but support or contradiction is unconfirmed

Embedding search first narrows possible matches, after which semantic verification is performed. Low-confidence or failed checks do not create relationships; an absent relationship means “not yet verified,” not necessarily “unrelated.”

Incremental Verification and Change Detection

• Stable identifiers and content hashes allow unchanged units to reuse previous extraction and relationship results.
• Deleted sources trigger cleanup of dependent relationships.
• Code anchors store the validating commit and span hash.
• If an anchor disappears, it is marked orphaned.
• If the span remains unchanged, semantic verification can be skipped.
• If the span changes, faithfulness must be checked again.
• Rule-based checks happen before LLM calls, reducing cost and limiting nondeterministic reasoning to cases that require it.

Topic’s practical recommendation is to treat trustworthy internal context as a managed system rather than a search result. Preserve source structure, keep evidence attached to every claim, use automation for deterministic work, and route ambiguous organizational judgments to people.