One of the exciting parts of the “new world” of cloud is the ability to green field solutions. We don’t have the legacy requirements and so we’re free to do what we want.
Or so the evangelists would have you believe.
The past lingers on
The reality is that many people are closer to a brown field environment. The organisation their team is embedded into has a tonne of reporting (“is your machine patched?”, “who has access?”). These requirements came from years of systems mismanagement (the best appdev team in the world isn’t necessarily the best at running servers) and auditor questions. To provide adequate reporting a series of centralised reporting tools have been created to help prove to auditors that the environment is controlled; “we have 30,000 servers; 357 of them are running Struts; 3 of them are unpatched and have change records for next week” is a great story to be able to tell, but it requires a level of conformity to get there.
And now you come along with your dynamically scaling app running inside an environment that builds/destroys servers at will. You can’t even answer the question “what servers are your apps running on?” because the question is meaningless in your world; it changes from day to day, even hour to hour.
So new builds may start being encumbered with requirements that trickle down from the traditional compute environment. Of course the CMDB needs to be kept up to date; of course you need privileged access monitoring; of course you need configuration management tools; of course…
We need to prevent this encumbrance from crippling the benefits of the new world.
In some cases we might be able to kludge a solution. “If we give you a nightly update into CMDB of all our AWS EC2 instances, will you be happy?“. Beware, though; once a machine is listed in CMDB then it’ll be expected to have all the other corporate tools (monitoring, access management, backups…) installed. Be aware of the consequences of the kludge, and try to avoid it.
The harder, but ultimately (IMHO) better path is to go back to square one and design new controls and procedures that are better optimised for this design.
The corporate policies and standards are meant to be the distilled guidance of everything that’s needed, based on legal reading of regulations, from history of audit questions and issues, from experience. These become your go-to documents. You may also want to go to the source, because regulations can change without your standards. So read the PCI docs, read MAS, read HIPAA…
And then design solutions based around the requirements, not around the existing practices.
This is hard work on a political level and requires a fair amount of senior buy-in. It requires vision and leadership because the organisation is now taking a fair risk; will you build a good solution that will pass audit? Will you expose the company to data theft? Will the solution scale to multiple appdev teams (you’re not a unique snowflake; other teams want to do this as well).
If you have management that support you, and can demonstrate compliance to corporate and regulatory policies and standards then you may be greenfield enough and avoid too many encumbrances.
New encumbrances
Because this is new and because you’ve now designed new processes and procedures, the people reviewing your work will be looking for edge cases. How can this new thing go wrong? It’s a natural consequence of trying to protect the company, the data.
The problem is when a potential gap is found. Is this gap already present in the traditional compute space? Is it known? Has the company decided this is OK? If so, it may be a problem you don’t need to fix; acknowledge the gap and move on. Don’t encumber your work with things that aren’t necessary.
Imagine building a PaaS solution and being required to tell the application operate team everytime you are going to reboot a VM? Technically it’s possible… but it’s stupid, and negates some of the benefits of a PaaS.
We see this in various places, especially in new fields of automation; for example, self-driving cars. I’ve seen people ask “once we get to fully automated cars capable of taking voice commands from the driver, how do we make sure the driver has a license?“. This is a perfect example of “new technological requirement” based on an existing requirement (must have a license, must be insured) but isn’t currently enforceable. People get excited by “we’re building something new; we can add this feature”.
The idea may seem reasonable on the face of it, and it may be a solution that is technically feasible. But this is a case of scope creep and can drag down your project.
“Just because you can, doesn’t mean you should”.
Don’t encumber your new solution with requirements that go above and beyond the problem you’re trying to solve. Try to keep project scope under control.
Conclusion
Very few projects are really greenfield. There’s always some small amount of legacy that you need to keep up with. Much of that legacy is there for a good reason, and you can’t just ignore it with impunity. However you can replace it with better processes and procedures, with better solutions. Sometimes going back to the drawing board and understanding the rationale is important.
Don’t just say “we’re doing DevOps” and ignore everything that came before; that’ll get you into trouble. However “we’re doing DevOps with these controls, procedures, enforcements, auditing…“; now you’ve got a chance to build things without the encumberance of traditional procedures.