Are cybersecurity certifications useful?
Lately you may have heard me say that "certifications are useless", and you may have thought: "This guy has gone crazy, why is he talking such nonsense? So I thought it would be a good idea to try to explain myself, more than to justify myself, as an exercise of reflection that I propose to the reader.
The first thing I would like to say is that a certification is not an objective in itself. The objective of certifications is to show someone (a stakeholder: customers, partners, shareholders, supervisor etc.) that cybersecurity requirements (the reference standard used by the certification) are met, thanks to the assessment of an independent third party. If we make "being certified" the objective, then we will be faced with a "compliance" approach and organizations will seek to achieve certification with minimum effort (in fact, to think that this is often not the case would be too naive on our part).
Certifications, yes or no?
So, are certifications useful or not? Well, it depends. They are useful for what I was saying, to show someone that specific security requirements are met. In an orderly and stable world, their usefulness is obvious, but in today's much more complex environment, where the speed of change is greater than ever, the usefulness of certifications is more questionable.
For those interested in delving into the differences between well-ordered and unordered environments, I recommend delving into the Cynefin decision-making framework, although I will give a hint: more complex environments promote experimentation and learning (agile), as opposed to instruction manuals (which are better in more stable environments).
To use a simile, let's think about antivirus. In the beginning they were based on blacklists, because that was enough, but nowadays the situation is very different and requires other technologies (whitelists, heuristics, etc.). I believe that, in the same way, certifications should evolve to adapt to a situation that is different.
Evolution
In my opinion, there are three main characteristics of certifications that justify the need to evolve them and look for other types of mechanisms. The first is that they rely on standardization bodies that define the requirements to be evaluated. This, which seems obvious, has important implications. Firstly, it forces a consensus to be reached on these requirements, which can be a very long process (think, for example, of the 12 years between versions of the National Security Scheme or the more than four years it has taken in Europe to reach a consensus on the cybersecurity certification scheme for cloud services; by the time it is published, it will already be obsolete).
And secondly, the fact that each use case generates a different referential leads to a segmentation of the market by sector, country and, in general, by each type of guarantee to be established. Doesn't it seem to the reader that there are too many certifications?
The second characteristic is that they are absolute. Certifications divide the world into two: those who have them and those who do not. This means that there is no incentive for excellence in compliance (a certification achieved by the skin of one's teeth is the same as one achieved in an outstanding manner), nor in evaluation (few prefer a more demanding examiner). In fact, most certifiers make extensive use of freelancers as auditors to avoid large staffs and to reduce personnel costs as much as possible, since the ones that end up winning the market are the cheapest (all things being equal, price is the determining factor).
The third is that they evaluate a static situation at a moment in time. Perhaps this is the factor that clashes most with the current reality. Whereas in the past, versions of a product or system could last for years.
While in the past versions of a product or system could last for years, with the application of agile methodologies, nowadays, there are software versions with a life span of a couple of weeks. Therefore, how can we know that what we are using has a certain correlation with what was tested and that the current version has approximately the same degree of security as the one that was tested?
And to make the situation even worse, how can we evaluate something that has a life cycle shorter than the time it takes to test it?
Basically, as Einstein said, "insanity is doing the same thing over and over again expecting different results. Well, we have been using certifications for over 20 years and we think we are going to solve the problems they have created by using even more certifications..... In short, it's crazy.
Understanding what they are
By this I do not mean that we should run away from certifications. I think we should understand what they are, what they are for and use them appropriately.
As I mentioned a few paragraphs ago, certifications serve to demonstrate compliance with stable requirements, so they can be very useful, for example, to ensure that the minimum requirements for operating in a market or for marketing a product are met. In this sense, they do not serve as an absolute guarantee of security, but they do serve to ensure that at least the minimum requirements are met.
However, just like antivirus software, they must evolve and incorporate other elements:
Agility in the definition of criteria.
Capacity for continuous supervision over time.
Compliance levels (to promote excellence).
Mechanisms oriented to interoperability.