Pursuit of Penetration Testing

Written by Duarte Castelo Grande de Carvalho (dcgc)

PEN

Cybersecurity looks very enticing from the outside, as it is told (very much so). It’s due to all the buzzwords, all the flash, all the infographics, etc. Usually, media outlets, people who do not work in the industry, and popular culture fantasize about this field. When students or people interested in Information Security start to get their feet wet and want to move in, belong to it, one of the first things they do is watch presentations from some popular worldwide conferences or listen to some podcasts. The presenters work on research, small consulting firms or big companies, with a rare free position inside it.

For me, it was the same: you get wowed by the things they talk about, by the jargon and by the flair they seem to have. It happens because you are unaware or ignorant of the subject matter. When I was in the Networking Academy long time ago, I started learning about routing/switching protocols, networking products, and even security in the network domain. Once you have a few years of hands-on experience and start to understand how things work around you, you begin to see that a lot of it is beautified, and sometimes, to be quite frank, is overblown.

The primary profession, position or role that I got struck by awe was of a Penetration Tester. There were several things about it: the fantasy of “hacking” as your job, the coolness and swag of most speakers (Pentesters), the hands-on, the knowledge, etc. They are seen as Rockstars of some sort, mainly due to the leverage (knowledge) they hold and the attention they get (considered as specialists or “experts”).

After six years of wandering around the Information Security field, I see that there are professionals… and then there are “professionals”. There are Penetration Testers… and then there are “Penetration Testers”. I’m not talking about good and bad professionals. I am talking about the level of engagement, the level of interest, and the level of belonging/responsibility regarding to what they do. A good and/or bad professional can be considered as such due to totally different reasons and usually it comes down to how good of a job their superiors think they have done (rightfully so, or wrongfully so).

I passed THE Penetration Testing test not too long ago, like many others. I attended training thinking that Penetration Tester would be a good path, especially after witnessing very few other Penetration Testers in a work environment. But after seeing “other” Penetration Testers and understanding what organizations seek in their corporate environment… it becomes clear that what these Rockstars were doing is not exactly the norm or another day’s job for most professionals practising this trade. There are other reasons for this conclusion, so it is better to separate them into categories.

Corporations: what do they want in a Penetration Tester?

There are places… and then, there are “places”. A Penetration Tester position in one organization means one thing… and the exact position means a different thing somewhere else. There are two types of organization (maybe… three) that will look for a “Penetration Tester”:

You have the typical Consulting company. This type of company is usually opened by a former employee from somewhere else (from a 10k plus employee environment), someone who wants to start a business (Small-Medium Business; SMB) dedicated towards “CyberSecurity” or even focused solely on Penetration Testing. It can also be an already grown multi-million (or not as big in comparison) consulting company with several years in the market. The main goal of both organizations is clear: have clients/customers, and if the consulting company offers consulting services for other technology domains, extend a contract to those. It creates what is called “vendor support group/contract” on large organizations out there.
- Some consulting companies have the name in the market and can quickly obtain a contract with a large organization, even if, in reality… the “service” they offer (the professional working with the customer) is subpar. Smaller consulting companies have to scratch the surface and cannot drill down into the market, so they have to be satisfied with crumbles from the ground left by the more prominent consulting companies, or, they have to rely on their network of contacts. It leads to two types of smaller consulting companies:
  - the ones who are ambitious (that are active in the Information Security community and have a name online (online presence))
  - the ones who are not ambitious (who do not understand the Information Security market and want to be in the space because… it equals money; they also like to sell snake oil).
- Usually, these companies want people who have done the legwork and wish to be Penetration Testers. Still, due to the shortage of professionals, more people are being admitted or considered for the position… even though they don’t have the credentials (that you can obtain with hard work, fair). But ultimately, that don’t have the profile (they don’t care or want to be bothered about how something “works”). Also, with the rise of more companies like these being born (like mushrooms) and the operations (HQ, HR, etc.) being done remotely, after the COVID-19 lockdown/confinement showing that what we do, does not need to be done being stuck in a physical office or a particular location, we see hiring being done all over-the-place. In the world, the demand for professionals and the lack of them (of proper ones, instead) will make the quality and level of performance decrease from stellar to mediocre.
- Because of the presence of all these consulting companies in the market, in this security space, it creates confusion to someone who would want to work as a Penetration Tester: what is the end goal, what is the motivation for the service you provide? Money, sure. It is a business, but what else? People applying already have to deal with vague and non-descriptive job postings and get pre-constructed answers in the interviews they attend, but how to know for sure what the modus operandi is when it comes down to the job, and how and what is conducted in the role when working with a customer? It can be considered disclosure that can be harmful to the consulting company (spilling the beans). Still, in the end, people who have invested hundreds of hours studying on several topics regarding Penetration Testing, a couple of grands on papers to show you have the skills, want to know. They want to know what they will be doing and whether that company is a good fit. Unfortunately, in most cases (unless you know someone on the inside or read reviews), you will not be able to understand this until you join the squadron and start devoting yourself to the cause.
- The other bullet point to add to the confusion when applying to these places is the marketing around consulting firms and penetration testing services they offer. Big 5 Consulting Firms and the market saturation of these services come to my mind. The website (from the marketing department), the job description (as mentioned before), the people working (people and their Linkedin profiles tell a different story because even they are confused), as well as with misconception on what a Penetration Tester job should be.
- There is also the fact that companies are now starting to see that the flowering green grass they are acquiring is, after all, not what it seems. Hiring a few dozen people and setting up the services themselves leads to a cheaper alternative in the long run. The rise of internal teams replacing consulting firms’ contracts (not to 100% due to audits and other regulatory requirements) leads to more job posts like these happening and showing up. Both roles are different as we can imagine, someone working on an internal team instead of someone working as a consulting pentester… but they don’t tell you that.
- In the end, there are two situations when going to consulting companies: you either get to go to a good place, where you are given a chance to develop yourself and contribute to the company (open-source projects, present at the conference (whatever) on behalf of the company, etc.) and are surrounded by good people (harsh but fair; knowledgeable, and willing to share), or you end up in a place where the level of engagement is so damn low and where it is in the interest of the company to grow based on the next client they get, instead of focusing on giving back to the Information Security community and developing the name of the company.
- You will be rejected because:
  - A) you don’t have the desired level of knowledge, as in, you don’t have the level of knowledge as someone experienced and they are not willing to let you develop it (in some rare occasions, they can admit you and let you develop it).
  - B) you don’t have a social connection, as in, you don’t know the right people (you didn’t talk with X, Y and Z)
  - C) you were not cheap enough for their wallets.
Mid-Big organizations, as mentioned before, are starting to create their own Information Security teams, and one of the roles being created is the Penetration Tester role. But what they mean is they want someone who will run a couple of tools, allowing the organization to be able to “tick the box” on the annual or quarterly audit and hand over the papers to the regulatory body. It is nothing more than a pro-form, rather than creating a security program inside the organization and promoting a shift in culture and mindset.
- When you are studying for a Penetration Tester position, and when you are done getting the required qualifications to apply to a “beginner” level position regarding it, you start looking for a job. You will find several Penetration Tester positions at companies where their core business is not even cybersecurity. You start wondering and picturing in your head that you will be doing penetration testing against mainframes in financial markets or against aircraft in airlines (usually, this nonsense is spoken on interviews). Not necessarily… most of these positions have to do with the corporate environment, and at most, you will be testing against web applications or server infrastructure. But the reality is this: you will not be doing Penetration Testing. For several reasons: first, you will not write reports or do reporting as you learn a Penetration Tester does; second, you will not use several tools you have used in your studies, you will not follow any pen-testing methodologies, you will not, for sure, work with other teams inside the organization regarding things such as threat modelling, asset inventory, technology stack, etc.; third, teams and people will not be aware you exist. It sounds disheartening because… it is. Organizations are creating positions for this and don’t know the requirements for the job post. They don’t know the “business as usual” for something like this. Applicants have expectations and they get crushed when they face a situation like this.
- You will find a lot of types of employees in these places: employees who do not give a shit, employees who are not cut out for the job, employees who do the bare minimum, employees who are dedicated but are fed up with all of it and want to change internally, or leave the company, employees who know “how to stay”. If you would be a Penetration Tester in this organization and would like to do a job well done, you would face several challenges thanks to this: people will not listen to you (you are not the “expert” from Consulting Firm / Vendor), you will not be given priority (Information Security is not a priority and/or investment in these places, it is a cost), and you will not be given the proper tools/equipment to perform your job (“budget” reasons, meaning they don’t want to invest in it fully and they don’t want to admit that they don’t know what to “get”)). It sounds disheartening because… it is. Organizations and leadership around these circles don’t have the Security mindset to be developing programs. They don’t have the experience, and most of all, don’t have the interest in the “stuff” creating a unit for “CyberSecurity” of the company.
- At the end of it all, what is more difficult to accept is that all the small efforts that are done in these places towards Penetration Testing and bettering the security posture of the company (whatever the hell that means,) will in fact, not be celebrated. They will be seen as a hindrance. You are not part of the business, period. You will also feel like a cog on the wheel, like an ant on the colony. It’s not about ego. It is about making a difference and seeing results. Even if some effects are achieved, this is not conveyed by the organization and by the leadership, you will feel left out and think that everything is pointless. To some extent, it is…
- You will be rejected because:
  - A) you were honest in the interviews.
  - B) the company (the hiring manager) opened up the position for someone else in mind.
  - C) you don’t know anything about Penetration Testing.
Another very interesting position to people who hear about it and are still trying to get into Penetration Testing is of a researcher and/or bug bounty. All of this, in the end, means the same thing (even if the type of work is different) because the workflow is based on the work you put in, the results you get, and the name you create and market for yourself… which means that if you get ‘squat’, you are screwed (no money, no survival). Unless you do this type of job (typically the first researcher) in a corporation, you will get regular and consistent income… and normally do jack shit.
- The great thing about something like research, applicable with research for other fields, is the flexible goals and time you have for it, thus allowing you to think of it as purely technical work. Self-satisfying and rewarding (you are “pentesting” technology for NEW vulnerabilities, insecurities, etc.), and in the end, feeling you are making some difference. New CVE’s (Common Vulnerabilities and Exploits) you might find might be, in the grand scheme of things, worthless. Still, at least it proves to yourself that you are “discovering something new”, either with known TTP’s (Tactics, Techniques and Procedures) or with new attack vectors. Depending on how you leverage it, it might put you on the map in the Information Security community, and if applying to known conferences, even being able to talk about it to an audience (besides an article on the Internet).
- Bug Bounty is the cousin of research: it is a form of research with clear and defined goals, methods and rewards (usually stipulated by the companies putting up the Bug Bounties). Several platforms (some more popular than others) host a plethora of bounties from different companies, usually surrounding Web Application. The stipulation, memorandum of understanding if you will, is stated on the Bug Bounty platform/website. People have to respect it and follow or act according to the scope demanded by the bounty poster. It leads to easy bounties (low hanging fruit) or very difficult ones. There are many people on these websites so… you can imagine how hard it can be to get a successful bounty: it is about timing and knowledge, indeed. The key takeaway from this is that for a newbie to solely start working/developing/studying on Penetration Testing in Bug Bounty platforms… is almost an impossible route.
- Both sound stressful because they are (at least for a beginner). There is a very high and throat-cutting competition in both fields, and you will find yourself thinking that you will have to cut sleep. The problem with Bug Bounty and Research is that some of it is dominated by veterans. People who have been making this trade for 10+ or 20+ years. Let’s face it; not much has changed in how technology is built and how testing is done. Research in a company (usually a product vendor) is potentially a lousy role because of the scope of Pentesting, the scope of the targets (only products of the vendor or technology it uses around that) and the disclosure (you are on a leash). If you are willing to be a YES man and not have the critical thinking, it might sound nice… if you are a normal human being, then not. Bug Bounty is a nice deviation from your main/daily job but not a full-time gig if you are starting. There are too many people doing this and rather qualified, so you will not have a steady financial income.
- You WILL be rejected for a research role because you don’t have the experience (meaning you are a noob). You WILL not be admitted into the Bug Bounty platform or you WILL not get a successful bounty because you don’t have the experience (meaning you are chobo).

The work: What will I do in a Penetration Tester role?

“Pentesting work” will be divided into few “domains” (type of targets) and will be divided by type of interaction (“ways of testing”). Usually, the main “domains” are Web Application (the main one), Mobile Application (the other type of application you might see, not as common) and Server (OS, Middleware (maybe), web services). Cloud and Containers are on the rise, but not as common, unfortunately. The “ways of testing” are all about the tools you use: dynamic (“automatic”) testing will typically involve scanning (network-based, application-based) and static (“manual”) testing will involve proxying, creating requests manually and trade-crafting (payloads), interacting with the targets.

The main problem while working with clients or internally has to do with what we are allowed to do, and a lot of times, it isn’t easy to answer because… the people requesting the Penetration Test also don’t know. It starts with the scope: what targets we want to “check”, what vulnerability families we want to focus on, what is the goal from this Penetration Test, why are we doing this in the first place? Then it continues with the tools: what tools are allowed, what tools to use, what type of access should we have (type of testing)? Then the doubts continue: should we define and state what should be done and how, because the client/stakeholder/internal team doesn’t know? Should I give my 2 cents? Should I spoon feed them on what they are seeking after? Instead of the pentester focusing on the job technically, the professional wastes time defining things that are not his priority or even responsibility. It uses a lot of their time on communication (which is part of the job, no doubt about it, but it is not supposed to be a long ironman activity). Frustration comes into play; shenanigans materialize, and then the shitshow commences.

It is what mostly happens in many places where Penetration Testing work is done: nothing substantial technically is done, and a lot of time is spent on doing the legwork for the customer (who should be educated when requesting the Penetration Testing service). Do I want to go for a position that was promised to me to be the ultimate seat for technical development in Information Security and end up doing Tenable.IO scans and Burp Suite proxying most of the time? Yep…

Pentester position: Why I was rejected

Before I did qualifications regarding Penetration Test, I tried to apply to the first aforementioned two types of companies: it didn’t end well, but it was fruitful to understand what these places, in particular, were looking after.

What position and chance does someone who has recreational activities regarding Penetration Testing, no “work experience” in Penetration Testing or in a “Red Team” position (whatever the hell that means) and with no “paper” or given proof in tradecraft or PoC (Proof of Concept) development? An unfavourable position and a slim chance. Penetration Tester is a position for seniority, for someone who has worked in different positions and has seen a bit of everything throughout their career, or for someone who is crazy enough to devote a lot of their time to the craft. In theory, this is what the position is. Still, it is possible (due to the opportunities that exist) to get to work as a Pentester.

So, what went wrong? First, you need a “paper”, the one where you put your hands on virtual machines, not the one where you practice nmap. It is enough proof that you have fundamental skills and understanding when it comes to this type of position, but it is not proof that you understand or have the mentality to be working in such a trade. Second, you need to be up-to-date with what is happening (news), meaning vulnerabilities, threats, products and projects. Some organizations and some projects will be given more spotlight due to popularity and history, but others might be highlighted depending on where you apply. Third, the technology you learn outside of security, you need to understand them, in ‘n out, and its applicability must be interchangeable between products, deployments, and use cases.

I didn’t have the paper; I didn’t completely follow the news and understand all of the concepts (all use cases). Do most companies recruiting for this position expect this? No, but the good ones will. I faced applying to a consulting company (and having an interview), which I didn’t face applying to a big-mid company (not having an interview). Why was I rejected to the second one? No paper.

Pentester position: Why I rejected it

After certifying (the paper), after devoting time to studying Penetration Testing and after understanding the status quo of the Penetration Testing job market for a newbie, I started applying to these positions again and hoping to see what the outcome would look like.

The results were not surprising: I got more interviews. But what was surprising was the interviewing posture of the interviewers on most interviews: because I got the paper, the interview was less demanding, less laborious, less detailed, and less technical. The interviewers thought that because the candidate had a piece of paper, he was in the “clear” to work for their organization. It is alarmingly serious and a red flag. It shows that some roles are dependent on a piece of paper, on pre-conceived notions of what Penetration Tester qualifications should ideally be, and not on the actual technical skills of someone, especially on the answers a candidate gives based on the inquiry. It also shows that some places are not mature enough to start offering Penetration Testing services and/or having an internal Penetration Testing team, don’t have the backbone to do so, and will not attract the right and correct people to be working on such a thing.

Now, it will depend on the individual: some people are ok with this and will accept. Good professionals and people who have passion for the field will discard such chances, go back to the drawing board and continue with self-development. It’s the outlook on what the talk about demand vs offer in “CyberSecurity” is about: you have a lot of positions being built and opened for different types of jobs inside Information Security, but most of them are goalless and unfocused, created by people with no “security vision” whatsoever. Then you have a lot of people wanting to jump onto the ship, but without any idea where they are going or with a lot of misconceptions on what is actually done. The previously mentioned Penetration Testing positions are created and filled, which is why the bar is set low.

Pentester position: Why I should reconsider it

CyberSecurity positions in corporations are overall entry-level and stale. The complexity of the work done and the effort put in, is most of the time minimal and not thought-provoking. So, if one tends to sought after Information Security work that is purely technical and requires hands-on in most of the business working hours, you end up looking for a position such as Penetration Tester. It is true that a lot of the activities done as a Penetration Tester require reporting (which consists of writing reports) and meeting (which consists of participating with meetings) with customers and/or internal teams, but it also requires to use tools, test systems and applications, and in the end, be exposed to new things. Unfortunately, not many roles in Information Security are like this, so even with all the faults described, you incline to look at the positive so as to see the big picture, which is to develop yourself through other means (e.g., Pentest-as-a-service companies, freelance, etc.). Should I stay or should I go?

What is the point?

Until higher technical scrutiny and screening isn’t done for Penetration Testing, and until there is no understanding of this position, we will have this situation. I rejected it. It didn’t feel right because it was not. A lot of it is because of what I detailed… Other things have to do with depositions from people such as s1m0n (it goes both ways; playing devil’s advocate). Penetration Testing work is uninteresting and non-rewarding because the audience you are talking to continue to poorly understand what they have and manage and over-complicate what is simple. Penetration Tester is continuously seen as a “fixer” and/or a “breaker” when it should be seen as an “enabler” (enables you to show what can be done differently, security-wise). We already know we are vulnerable (we can run the tools to check this ourselves), we already know the answer (configuration workaround, patching, etc.). We already know what we should’ve done (in the compliance we have to follow). Why is the Penetration Tester a messiah?

Maybe one day, I will have the possibility to enjoy being a Penetration Tester that I dreamed of and will be given the opportunity to explore the potential that someone can have, to work in Information Security solely as a technical person. But, until then, I will stay in my corner working as a “regular security professional”.

Blog posts about Information Technology, Information Security Industry and Life. Whatever comes to my mind.