Being a great DevOps guy means a lot of things. Your systems are designed to be up and to stay up. Your pager sits quietly on your nightstand while you sleep soundly. You didn’t even realize you had a 1000x traffic spike this morning because nothing went down and no one had any problems. You may even have a great beard. Some people define this type of DevOps as rugged DevOps—using a rugged software development approach for your infrastructure so you can survive the attacks, outages, and misconfigurations you can’t predict.
So you’re rugged DevOps. But are you responsible?
While being a great DevOps guy is critical, you need to take it a step further. For every 10 developers out there, there is one rugged DevOps guy and it falls on that person to “police” and educate their growing team on in order to protect the organization. This is what I call responsible DevOps and it more often than not falls by the wayside.
What does a responsible DevOps guy police?
First, make sure you are not the police. Your team is working with you and it is not your job to stifle innovation. It is your job to make sure best practices are followed and to ensure policy violations don’t occur. In simple terms, your job as a responsible DevOps guy is to educate your peers on how to create rugged software that polices itself. But, in order to get your peers onboard for why this is so critical, you have to show them the ugly truth…
Policy violations are rampant! (And yes, that means you.)
While I was consulting I had the benefit of working with some of the most advanced cloud deployed organizations in the country. Some of them were, for lack of a better term, impressive. Continuous, human-less deployments were just the beginning. I’ve seen organizations where literally everything was self-healing, loosely coupled, massively scalable… the list goes on. But for these same impressive companies, I would audit their S3 bucket policies and find world accessible buckets!
When I would find configuration issues like this, there’s one response I heard a lot: “I didn’t know that was in the policy.”
“I didn’t know that was in the policy.”
I know. Some other team created it 25 years ago and everyone has been following that example since. It was supposed to be created by hand. My dog ate my configuration specs. It doesn’t matter how or why it happened, because violations like this are going to pop up again and again.
This is why CloudCoreo has the concept of Coreo Rules.
Coreo Rules are best practice statements we’ve created that say how your infrastructure should or shouldn’t be. You can create your own Coreo Rules (or modify ours) to fit your organization’s policy and processes. Then, you run these rules on a regular schedule and CloudCoreo will alert you whenever any of the rules have been violated.
We already have a Coreo Rule written to alert on the S3 situation we described above. It interrogates the AWS API and alerts on any world accessible buckets found. Written in our DSL, it looks like this:
coreo_aws_rule "s3-allusers-write" do action :define service :s3 level "High" objectives ["bucket_acl","bucket_acl"] audit_objects ["grants.grantee.uri", "grants.permission"] operators ["=~", "=~"] raise_when [/AllUsers/i, /\bwrite\b/i] id_map "modifiers.bucket" end
Now that’s just one rule. You can check out the git repo for our audit-aws-s3 composite, to see all the best practice rules we check your S3 buckets for when we audit your cloud account.
If you’re ready to educate your team on the need for rugged, self-policing software, we’re here to help. Give our free Cloud Audit a try, and start passing out those violations to your team like candy. (Candy that needs to be fixed. Promptly.)