How It Works

CloudCoreo provides continuous cloud automation that enables you to secure, automate, and control your AWS cloud.

COREO COMPOSITES

A Better Building Block

CloudCoreo enables users to create and layer inheritable and extensible cloud reference designs called Composites. Composites can outline a simple best practice design, a complex micro-service application, or an entire data center deployment policy.

All Things Declared

A Composite declares in code the desired end state of your cloud deployment. Composites include all required cloud infrastructure resources, deployment variables and service dependencies.

Single Source of Truth

All Composites are stored in Git, allowing you to control your cloud environment in the same way you manage software.

Unique Inheritance Unlocks True Reuse

Child Composites can be branched from a single parent to extend or override portions of a base design. Unlike existing template-based systems, Composites unlock true design reuse, because the connection to the parent design isn’t broken by making customizations.

The op_mongo Composite Design

├── config.yaml
├── extends
│ ├── config.yaml
│ ├── extends
│ │ ├── config.yaml
│ │ ├── services
│ │ │ └── config.rb
│ ├── stack-vpc-private-only
│ │ ├── config.yaml
│ │ ├── extends
│ │ │ ├── config.yaml
│ │ │ ├── services
│ │ │ │ └── config.rb
│ │ ├── overrides
│ │ └── services
│ │ └── config.rb
├── stack-nat
│ ├── boot-scripts
│ │ └── order.yaml
│ ├── config.yaml
│ ├── extends
│ │ ├── boot-scripts
│ │ │ ├── install_ha_nat.sh
│ │ │ └── order.yaml
│ │ ├── config.yaml
│ │ ├── extends
│ │ │ ├── boot-scripts
│ │ │ │ ├── expand_root_fs.sh
│ │ │ │ ├── install_cloudcoreo_repo.sh
│ │ │ │ ├── order.yaml
│ │ │ ├── config.yaml
│ │ │ ├── operational-scripts
│ │ │ │ ├── run_date.sh
│ │ │ │ ├── run_df.sh
│ │ ├── operational-scripts
│ │ ├── overrides
│ │ ├── services
│ │ │ └── config.rb
│ │ ├── shutdown-scripts
│ │ │ └── order.yaml
│ ├── operational-scripts
│ ├── overrides
│ ├── services
│ │ └── config.rb
│ └── shutdown-scripts
│ └── order.yaml
├── stack-vpn
│ ├── boot-scripts
│ │ └── order.yaml
│ ├── config.yaml
│ ├── extends
│ │ ├── boot-scripts
│ │ │ ├── install_openvpn.sh
│ │ │ ├── order.yaml
│ │ │ ├── restart_openvpn.sh
│ │ │ ├── setup_backups.sh
│ │ │ └── setup_vpn_config.sh
│ │ ├── config.yaml
│ │ ├── extends (repeats server-base)
│ │ ├── files
│ │ │ ├── client.conf
│ │ │ ├── template-server-config
│ │ │ └── vars
│ │ ├── overrides
│ │ ├── services
│ │ │ └── config.rb
│ ├── overrides
│ ├── services
│ │ └── config.rb
│ └── shutdown-scripts
│ └── order.yaml

COMPOSITE CODE SAMPLES

    CLUSTER_NAME:
        default: leadercluster
        description: the name of the cluster - this will become your dns record too
        type: string
        required: true
    CLUSTER_ELB_TRAFFIC_PORTS:
        default:
            - 27017
        description: ports that need to allow traffic into the ELB
        type: array
        required: true
    CLUSTER_ELB_TRAFFIC_CIDRS:
        default:
            - '10.0.0.0/8'
        description: the cidrs to allow traffic from on the ELB itself
        type: array
        required: true


VIEW MONGODB-CLUSTER VARIABLES IN GITHUB ›

coreo_aws_ec2_instance "${CLUSTER_NAME}" do
  action :define
  image_id "${CLUSTER_AMI}"
  size "${CLUSTER_SIZE}"
  security_groups ["${CLUSTER_NAME}"]
  role "${CLUSTER_NAME}"
  ssh_key "${CLUSTER_KEY}"
  disks([
         {
           :device_name => "/dev/xvdg",
           :volume_size => 25
         },
         {
           :device_name => "/dev/xvdh",
           :volume_size => 25
         },
         {
           :device_name => "/dev/xvdf",
           :volume_size => 250
         }
        ])
end

coreo_aws_ec2_autoscaling "${CLUSTER_NAME}" do
  action :sustain 
  minimum((${AUTOMATIC_ON_OFF} ? ((${AUTOMATIC_ON_HOUR}..(${AUTOMATIC_OFF_HOUR}-1)).include?(Time.new.getlocal("${TIMEZONE_OFFSET}").hour) ? ${CLUSTER_GROUP_SIZE_MIN} : 0) : ${CLUSTER_GROUP_SIZE_MIN}))
  maximum((${AUTOMATIC_ON_OFF} ? ((${AUTOMATIC_ON_HOUR}..(${AUTOMATIC_OFF_HOUR}-1)).include?(Time.new.getlocal("${TIMEZONE_OFFSET}").hour) ? ${CLUSTER_GROUP_SIZE_MAX} : 0) : ${CLUSTER_GROUP_SIZE_MAX}))
  server_definition "${CLUSTER_NAME}"
  subnet "${PRIVATE_SUBNET_NAME}"
  elbs ["${CLUSTER_NAME}-elb"]
  health_check_grace_period ${CLUSTER_HEALTH_CHECK_GRACE_PERIOD}
  upgrade({
            :upgrade_on => "dirty",
            :cooldown => ${CLUSTER_UPGRADE_COOLDOWN}
        })
  tags([
        "Name=${CLUSTER_NAME}"
       ])
end

VIEW MONGODB-CLUSTER RESOURCES IN GITHUB ›

coreo_aws_rule "ec2-get-all-instances-older-than" do
  action :define
  service :ec2
  link "http://kb.cloudcoreo.com/mydoc_ec2-alert-to-kill.html"
  display_name "Alert to Kill"
  description "EC2 instance was launched within the last 5 minutes that violates tag policy (does not have the necessary tags)."
  category "Policy"
  suggested_action "Review instance tags and terminate the instance if it does not comply to tagging policy."
  level "Warning"
  objectives ["instances"]
  audit_objects ["reservation_set.instances_set.launch_time"]
  operators ["<"&#93;
  raise_when &#91;"5.minutes.ago"&#93;
  id_map "object.reservation_set.instances_set.instance_id"
end

coreo_uni_util_jsrunner "tags-to-notifiers-array-kill-scripts" do
  action :run
  data_type "text"
  provide_composite_access true
  packages(&#91;
               {
                   :name => "cloudcoreo-jsrunner-commons",
                   :version => "1.8.4"
               },
               {
                   :name => "js-yaml",
                   :version => "3.7.0"
               }      ])
  json_input '{ "composite name":"PLAN::stack_name",
                "plan name":"PLAN::name",
                "violations": COMPOSITE::coreo_aws_rule_runner_ec2.advise-ec2-atk.report}'
  function <<-EOH
  
function setTagsLengthFromEc2Logic(EC2_LOGIC, EXPECTED_TAGS) {
    let tagLength = EXPECTED_TAGS.length;
    if(EC2_LOGIC === 'or') {
        tagLength = 1;
    }
    return tagLength;
}
function getSimilarNumber(tags, EXPECTED_TAGS) {
    let similarNumber = 0;
    EXPECTED_TAGS.forEach(EXPECTED_TAG => {
        EXPECTED_TAG = EXPECTED_TAG.toLowerCase();
        tags.forEach(tagElem => {
            if(tagElem.hasOwnProperty('tag')) {
                const tagToLowerCase = tagElem.tag['key'].toLowerCase();
                if(tagToLowerCase == EXPECTED_TAG) {
                    similarNumber++;
                }
            } else {
                const tagToLowerCase = tagElem['key'].toLowerCase();
                if(tagToLowerCase == EXPECTED_TAG) {
                    similarNumber++;
                }
            }
        });
    });
    console.log(similarNumber);
    if(EXPECTED_TAGS.length === 0) {
        similarNumber = 0;
    }
    return similarNumber;
}
function setTableAndSuppression() {
  let table;
  let suppression;
  const fs = require('fs');
  const yaml = require('js-yaml');
  try {
      suppression = yaml.safeLoad(fs.readFileSync('./suppression.yaml', 'utf8'));
  } catch (e) {
      console.log("Error reading suppression.yaml file: " , e);
      suppression = {};
  }
  try {
      table = yaml.safeLoad(fs.readFileSync('./table.yaml', 'utf8'));
  } catch (e) {
      console.log("Error reading table.yaml file: ", e);
      table = {};
  }
  coreoExport('table', JSON.stringify(table));
  coreoExport('suppression', JSON.stringify(suppression));
  
  let alertListToJSON = "['ec2-get-all-instances-older-than']";
  let alertListArray = alertListToJSON.replace(/'/g, '"');
  json_input['alert list'] = alertListArray || [];
  json_input['suppression'] = suppression || [];
  json_input['table'] = table || {};
}
setTableAndSuppression();
const JSON_INPUT = json_input;
const NO_OWNER_EMAIL = "${AUDIT_AWS_EC2_ATK_RECIPIENT}";
const OWNER_TAG = "${AUDIT_AWS_EC2_ATK_OWNER_TAG}";
const ALLOW_EMPTY = "${AUDIT_AWS_EC2_ATK_ALLOW_EMPTY}";
const SEND_ON = "${AUDIT_AWS_EC2_ATK_SEND_ON}";
const SHOWN_NOT_SORTED_VIOLATIONS_COUNTER = true;
const EXPECTED_TAGS = [${AUDIT_AWS_EC2_ATK_EXPECTED_TAGS}];
const EC2_LOGIC_LENGTH = setTagsLengthFromEc2Logic("${AUDIT_AWS_EC2_ATK_TAG_LOGIC}", EXPECTED_TAGS);
const sortFuncForViolationAuditPanel = function sortViolationFunc(JSON_INPUT) {
    let regions = JSON_INPUT['violations'];
    let counterForViolations = 0;
    let counterForSortedViolations = 0;
    const regionKeys = Object.keys(regions);
    regionKeys.forEach(regionKey => {
      const violationKeys = Object.keys(regions[regionKey]);
      violationKeys.forEach(violationKey => {
          const alertKeys = Object.keys(regions[regionKey][violationKey].violations);
          const tags = regions[regionKey][violationKey].tags;
          const similarNumber = getSimilarNumber(tags, EXPECTED_TAGS)
          alertKeys.forEach(alertKey => {
              if(similarNumber >= EC2_LOGIC_LENGTH) {
                  delete regions[regionKey][violationKey]['violations'][alertKey];
                  counterForSortedViolations--;
                  if (Object.keys(regions[regionKey][violationKey]['violations']).length === 0) {
                      delete regions[regionKey][violationKey];
                  }
              }
              counterForViolations++;
              counterForSortedViolations++;
          });
      });
    });
      
    JSON_INPUT['counterForViolations'] = counterForViolations.toString();
    JSON_INPUT['counterForSortedViolations'] = counterForSortedViolations.toString();
    console.log(JSON_INPUT);
    return JSON_INPUT;
};
const sortFuncForHTMLReport = function htmlSortFunc(JSON_INPUT) {
    let regions = JSON_INPUT['violations'];
    let counterForViolations = 0;
    let counterForSortedViolations = 0;
    const regionKeys = Object.keys(regions);
    regionKeys.forEach(regionKey => {
      const violationKeys = Object.keys(regions[regionKey]);
      violationKeys.forEach(violationKey => {
          const alertKeys = Object.keys(regions[regionKey][violationKey].violations);
          const tags = regions[regionKey][violationKey].tags;
          const similarNumber = getSimilarNumber(tags, EXPECTED_TAGS)
          alertKeys.forEach(alertKey => {
              if(similarNumber >= EC2_LOGIC_LENGTH) {
                  delete regions[regionKey][violationKey]['violations'][alertKey];
                  counterForSortedViolations--;
                  if (Object.keys(regions[regionKey][violationKey]['violations']).length === 0) {
                      delete regions[regionKey][violationKey];
                  }
              }
              counterForViolations++;
              counterForSortedViolations++;
          });
      });
    });
    JSON_INPUT['counterForViolations'] = counterForViolations;
    JSON_INPUT['counterForSortedViolations'] = counterForSortedViolations;
    console.log(JSON_INPUT);
    return JSON_INPUT;
};
const VARIABLES = { NO_OWNER_EMAIL, OWNER_TAG, 
    ALLOW_EMPTY, SEND_ON,
    SHOWN_NOT_SORTED_VIOLATIONS_COUNTER,
    sortFuncForViolationAuditPanel, sortFuncForHTMLReport};
const CloudCoreoJSRunner = require('cloudcoreo-jsrunner-commons');
const AuditEC2ATK = new CloudCoreoJSRunner(JSON_INPUT, VARIABLES);
const HTMLKillScripts = AuditEC2ATK.getHTMLKillScripts();
callback(HTMLKillScripts);
  EOH
end

VIEW EC2 ALERT TO KILL IN GITHUB ›

The Coreo engine scans, alerts, provisions, updates, restores, and enforces cloud resources.

COREO ENGINE

Automation and Policy Control

The Coreo Engine provides a powerful, cloud native automation solution that is able to quickly scan, provision and update vast cloud infrastructure in minutes. The Coreo Engine executes all alerting and deployment logic in a single, integrated pass, providing full transparency along the way.

Deep Cloud Inspection

The Coreo Engine understands the resources, dependencies and variables for your cloud deployment and continuously monitors your running infrastructure against your cloud policies. It can notify stakeholders of any differences, restore configuration to the desired declared state, as well as take other extensible remediation actions.

Future Cloud Portability

The Coreo Engine is designed to rapidly add support for new clouds and cloud primitives. Initially, CloudCoreo supports AWS only. Our unified DSL aims to provide an approach for multi-cloud management, enabling a single application infrastructure model to be translated to multiple cloud
resource types.

COREO INTERFACE

Scale Cloud Teams and Tasks

CloudCoreo provides a full GUI, CLI, API to manage your cloud designs, alerts, policies and running cloud resources. The CloudCoreo interface lets you manage access, create teams and invite colleagues to leverage your approved Composite designs to launch cloud infrastructure.

Integrate in CI/CD

Tie into CI/CD systems to automate dev, test and production cloud infrastructure. Use the API or CLI to initialize a Composite plan, feed in variables, and create consistent, automated cloud environments.

Operational Efficiency

CloudCoreo provides a flexible UI that enables users to create custom displays. Whether exposing operational scripts for common tasks or modifying your results dashboard, we give you the flexibility to visualize your code how you see fit.

Examples from CloudCoreo's Web App