Automating Service Vulnerability Analysis using LLM #2

The post explains how Toss Security Research improved AI-driven vulnerability analysis in a research network. Its main challenges were efficiently providing large codebases to an AI and making analysis results consistent and complete. The solution combined a custom code-browsing MCP server with SAST tools used not to identify vulnerabilities directly, but to enumerate all input-to-function paths that the AI must review.

Efficiently Providing Large Codebases

• Tools such as Cursor and Claude Code can search large projects, but primarily rely on pattern matching with tools like ripgrep.
• Without prebuilt indexes, they may miss relevant code or waste tokens exploring unnecessary files.
• The team built an MCP server that:
  • Uses ctags to index symbol definitions.
  • Uses tree-sitter to parse function boundaries.
  • Allows AI to access code remotely, similar to IDE features such as “Go to Definition” and “Find References.”

SourceCode Browse MCP

The MCP server provides four main tools:

• find_references()

  • Searches for symbols or patterns using ripgrep.
  • Returns file paths, line numbers, snippets, total matches, and whether results were truncated.

• read_definition()

  • Looks up definitions through the ctags index.
  • Returns metadata such as file, line, symbol type, language, signature, and scope.
  • Uses tree-sitter to include the complete function body when requested.

• read_source()

  • Reads a configurable number of lines before and after a target line.
  • Lets the AI retrieve only the relevant local context instead of entire files.

• get_project_structure()

  • Returns the indexed project’s directory structure.
  • Provides the AI with a project “blueprint,” which is especially important in remote environments where it cannot inspect the repository locally.

The MCP workflow is to locate relevant symbols with find_references() and read_definition(), inspect nearby code with read_source(), and use get_project_structure() to understand the overall project.

Improving Consistency and Accuracy

• AI analysis produced inconsistent results: for example, it might find all 10 XSS vulnerabilities in one run but only 8 in another.
• This variability made the results difficult to trust.
• The team combined AI analysis with SAST tooling to ensure complete coverage.

Using SAST to Enumerate Review Candidates

• Rather than passing SAST-detected vulnerabilities directly to the AI, the team used SAST as a candidate-generation tool.
• This avoids limiting the AI to vulnerabilities that the SAST engine itself knows how to detect.
• SAST extracts every location where untrusted input enters the application and tracks its possible flow to function calls.
• Custom Semgrep taint rules identify sources such as:
  • Spring @RequestParam
  • @PathVariable
  • @RequestHeader
  • Fields read from @RequestBody DTOs
  • @RequestPart
  • @ModelAttribute
  • @RequestAttribute
• Potential sinks include generic function calls and object method calls.
• The AI then reviews every extracted source-to-sink path, combining the completeness of static analysis with the broader reasoning ability of an LLM.

The overall approach is to use deterministic indexing and SAST for coverage, while relying on AI for deeper vulnerability interpretation.