What you need to Know about AI in Pentesting
AI is everywhere and in everything. For pentesters, that means new pentest assets, with localized and outsourced LLMs to test for cybersecurity. However, it also increasingly means seeing AI-powered pentest tools on the market. Like in other fields, the question of whether AI will replace human skillsets is important. Big organizations are facing cybersecurity talent shortages, with the World Economic Forum suggesting a global shortage of some 4 million cybersecurity professionals. New, AI-powered hacks are also driving adoption of AI-defense platforms, in what has been termed by Gartner as a “cold war”.
Eventually, AI will never be able to replace the added value of a human pentester. Human insight and human ingenuity remain the most valuable parts of a pentest. Ethical hackers who can figure out ways to exploit vulnerabilities, find new exploits, and who can test intelligently and critically will always add more value.
“We’re going to see more and more AI-driven pentesting,” says Luis Abreu, CEO of Cyver Core, “big organizations are increasingly using AI to drive attack surface monitoring and as part of CTEM and continuous pentesting. At the same time, it’s important that we focus on automating repetitive manual tasks and overhead, instead of the tasks that actually benefit from human expertise”
Automated Vulnerability Scanning
AI is good at repeating tasks. That makes it ideal for automating vulnerability scanning. For example, automating behavioral analysis to look for attackers changing signatures is one good use of AI. Network traffic analysis is another.
“AI currently lacks the nuance to understand context and consequence, especially in complex environments,” says Luis, “Therefore, even if you’re running continuous pentesting guided by AI-driven scanners, you’re going to need human oversight and review. For example, with Cyver, you can automatically assign pentesters to review results after scanners are finished, so you still have a human reviewing finding results.”
Some pentesters are also using AI to automate attack simulations, which can be useful as part of more complex assignments but still requires significant human oversight.
LLMs and OSINT
Good prompts can mean that LLMs take you quite far into OSINT. For example, AI can generate lists for fuzzing, it can generate edge cases like invalid input and common vulnerabilities, it can generate types of data accepted. It can scan API documentation for you. It can create lists of input parameters, it can create lists of tools you can use, it can provide examples of payloads for testing. All of that will have to be double checked manually, as AI is still prone to hallucinations – but can still give you a good starting point over performing research manually.
“As LLMs mature, you can use them to generate increasingly complex requests including your word lists, tool lists, input parameters, etc.,” says Luis, “prompt editing and editing results means you can simply generate fairly refined lists straight away. That will speed up OSINT tasks, because the AI can analyze available public information and summarize it for you.”
Don’t Forget about Overhead
Overhead tasks like reporting, work management, scheduling, and assigning tasks across the team can take as much time as the actual pentesting. Managing findings, planning pentesting, setting up pentesting, checking work, etc., also have heavy time loads and will take up time for skilled technical testers who could be better utilized doing testing instead. Those tasks are also ideal to automate, as most are repetitive tasks that involve pulling the same general data such as frameworks, repeat scoping information, asset information, etc.
“We are more and more often seeing pentest teams bringing in people to do the planning, client management, etc.” adds Luis, “Keeping planning inside a single system and automating as much of it as possible means that the technical data those people need to plan and schedule is there. You’ll be able to do more with a smaller core team, and your technical people will still have input on setting up the templates, processes, and customizable workflows that go into it.”
Automating Pentest Reports
Without report automation, pentest reporting can take up to two days. With it, Cyver Core users see timelines averaging 2-5 hours. For simple web-app scans, you might only have to click a button and be ready to go. When you add LLM, you can generate custom content including from your own bullet points, add recommendations, and reference the client’s technology without taking the time for custom writeups.
“Automating pentest reports makes more sense than automating pentesting ever will,” says Luis, “If you can automatically pull content from your vulnerability libraries and content libraries, generate custom content with LLM, and then review it, you can cut 50-80% of time off the pentest. Like other tasks, you’ll still want a manual review, but every one of our users is saving a significant amount of time on pentest reporting.
AI is a trend that isn’t going anywhere soon. However, even as it becomes more prevalent in cybersecurity, it’s important to take time to identify where AI adds value, where it doesn’t, and what makes sense to automate. “AI is increasingly capable of running scans and using tools on its own, you can use prompts for hacks, you can even use AI for OSINT,” adds Luis, “That’s going to be valuable for big organizations that need truly real-time security. However, we should be thinking of it like we do with DAST and SAST, a layer underneath your pentesting and human cybersecurity. You’re not going to get much out of AI without significant human guidance, which always means you’ll need a senior cybersecurity specialist to manage AI results – just like you would with a trainee who could later go on to add much more value than the AI. With that in mind, it’s important for us to think about what we automate and why – so AI reduces the burden of tasks around pentesting, rather than reducing expertise in our cybersecurity teams.”