"What Happens If You Disable the LLM?" - Why Pentesterra Isn't an LLM Wrapper

A VC asked me during an interview: "You use LLMs, right? Which ones? And what data do you train them on?"

Yes, we use LLMs. But not as the main core engine of Pentesterra. We use them only where they actually make sense:

• for "consultation" in uncertain cases
• when the deterministic engine needs additional reasoning support
• for generating some summary blocks

But we do not train on customer data. For us, customer security data is sensitive data.

Our models and knowledge layers are trained and enriched using our own scans, our own pentests, controlled test environments, data collected into our internal Knowledge Base, and knowledge from real exploits, bug bounty cases, threat descriptions, and threat intelligence.

"What happens if you disable the LLM? Does the product stop working?"

No. Pentesterra does not stop working. Some quality may degrade in specific edge cases - maybe around 8-15% in certain scenarios - but the core product continues to function. Even some summaries can still be generated without an LLM. And this is intentional.

Because there are a lot of hidden traps in "AI-first" security products: latency, provider outages, fallback between models, local mini-models, structured output problems, cost control, runtime stability, reproducible results.

And there is one more very practical question: do you really want to send pentest data to an external LLM API? What if that data contains passwords, tokens, internal infrastructure details, credentials, or sensitive findings?

That is exactly why the LLM layer must be optional, controlled, and not required for the product to function.

You cannot call an LLM for every decision

Not for every small, obvious, banal runtime decision during a scan or pentest - even if you move it to separate threads.

• You need deterministic logic.
• You need rules and playbooks.
• You need evidence.
• You need reproducible results.
• You need structured output.
• You need the product to work when the AI layer is unavailable.

This is only a tiny part of the iceberg.

So when people ask me about "what if" cases, I'm actually glad that Pentesterra can work without LLMs. Otherwise it would be just another LLM wrapper - and that was never the goal.

At the same time, yes: in some specific cases LLMs can be extremely helpful for pentesting. They can assist with reasoning, help explore hypotheses, and speed up analysis in genuinely useful ways.

But that is exactly the point: assist, not replace.