Cloud Devops Automation

You can use the following Terraform script to Deploy an Instance in your AWS Account. Terraform will create a t2.medium instance from the official RHEL7.2 AMI using the AMI ID within the specified subnet. And will create a 30GB root block device and a 10GB Ebs volume. The instance will use a predefined key and will add the specified tags to the Instance Being Launched. provider "aws" { access_key = "AKXXXXXXXXXXXXXXXXX" secret_key = "2YXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXxx" region = "ap-south-1" } resource "aws_instance" "instance_name" { ami = "ami-cdbdd7a2" count = 1 instance_type = "t2.medium" security_groups = ["sg-f70674re"] subnet_id = "subnet-526bcb6d" root_block_device = { volume_type = "standard" volume_size = "30" } ebs_block_device = { device_name = "/dev/sdm" volume_ty...

As discussed in the previous post regarding the alarm plugins those plugins are used to push the metrics data to the cloudwatch using the cron running every minute or 5minutes depending upon your requirements. Next we have to create the alarms in the cloudwatch on the above metrics which works on the logic that if the metrics crosses the threshold value than an event is triggered which could be like send a mail through sns alerting that the value has crossed the threshold and if it agains comes below threshold than it state is changed from alarm to ok which is more like a recovery. But unlike from the console we are going to trigger this programmatically using the AWS CLI provided by the AWS. The script works sequentially and uses the array which runs in a loop and all the relevant alarms are created. The most important thing to be considered here is the name of the alarm which is to be created in the cloudwatch. Now you can put any name but the name based on programmatic assump...

The following Cloudwatch plugin can be used to determine the established tcp connections. #!/bin/bash # # About : Check TCP connections # # Name : cw_tcpconnection.sh DIR=$(dirname $0); PLUGIN_NAME='cw_tcpconnection'; # Include configuration file source ${DIR}/../conf/plugin.conf; #Get Current Instance ID INSTANCE_ID=(`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`); #Get Hostname HOST_ID=(`wget -q -O - http://169.254.169.254/latest/meta-data/hostname`); # Help usage() { echo "Usage: $0 [-n ] [-d ] [-m ] [-h ] [-p ]" 1>&2; exit 1; } # Logger logger(){ SEVERITY=$1; MESSAGE=$2; DATE=`date +"[%Y-%b-%d %H:%M:%S.%3N]"`; echo -e "${DATE} [${SEVERITY}] [${PLUGIN_NAME}] [${INSTANCE_ID}] [${HOST_ID}] ${MESSAGE}" >> ${DIR}/../logs/appcwmon.log; } # Process Arguments if [ $# -eq 0 ]; then # When no argument is passed logger ERROR "Invalid a...

The following Cloudwatch plugin helps to measure the number of messages in the Rabbitmq unack,Ready,Total Message on which alarms can be configured later using the cloudwatch API. #!/bin/bash # # About : Check RabbitMQ Queue Message Length # # Name : cw_rabbitmq.sh DIR=$(dirname $0); PLUGIN_NAME='cw_rabbitmq'; # Include configuration file source ${DIR}/../conf/plugin.conf; #Get Current Instance ID INSTANCE_ID=(`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`); #Get Hostname HOST_ID=(`wget -q -O - http://169.254.169.254/latest/meta-data/hostname`); # Help usage() { echo "Usage: $0 [-n ] [-d ] [-m ] [-u ] [-p ] [-q ]" 1>&2; exit 1; } # Logger logger(){ SEVERITY=$1; MESSAGE=$2; DATE=`date +"[%Y-%b-%d %H:%M:%S.%3N]"`; echo -e "${DATE} [${SEVERITY}] [${PLUGIN_NAME}] [${INSTANCE_ID}] [${HOST_ID}] ${MESSAGE}" >> ${DIR}/../logs/appcwmon.log; } # Process Argu...

You can monitor the number of process running for a service to determine whether the service is running or not on the Server using the following cloudwatch plugin. #!/bin/bash # # About : Check Process Running Status # # Name : cw_process.sh DIR=$(dirname $0); PLUGIN_NAME='cw_process'; # Include configuration file source ${DIR}/../conf/plugin.conf; #Get Current Instance ID INSTANCE_ID=(`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`); #Get Hostname HOST_ID=(`wget -q -O - http://169.254.169.254/latest/meta-data/hostname`); # Help usage() { echo "Usage: $0 [-n ] [-d ] [-m ] [-p ]" 1>&2; exit 1; } # Logger logger(){ SEVERITY=$1; MESSAGE=$2; DATE=`date +"[%Y-%b-%d %H:%M:%S.%3N]"`; echo -e "${DATE} [${SEVERITY}] [${PLUGIN_NAME}] [${INSTANCE_ID}] [${HOST_ID}] ${MESSAGE}" >> ${DIR}/../logs/appcwmon.log; } # Process Arguments if [ $# -eq 0 ]; then # ...

Cloudwatch can be used to monitor the established connection to the vm. This helps in tracking connections in case your application is network intensive #!/bin/bash # # About : Check Local and Foreign Network Connections # # Name : cw_netconnection.sh DIR=$(dirname $0); PLUGIN_NAME='cw_netconnection'; # Include configuration file source ${DIR}/../conf/plugin.conf; #Get Current Instance ID INSTANCE_ID=(`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`); #Get Hostname HOST_ID=(`wget -q -O - http://169.254.169.254/latest/meta-data/hostname`); # Help usage() { echo "Usage: $0 [-n ] [-d ] [-m ] [-s ] -t [ LOCAL | FOREIGN ] -p " 1>&2; exit 1; } # Logger logger(){ SEVERITY=$1; MESSAGE=$2; DATE=`date +"[%Y-%b-%d %H:%M:%S.%3N]"`; echo -e "${DATE} [${SEVERITY}] [${PLUGIN_NAME}] [${INSTANCE_ID}] [${HOST_ID}] ${MESSAGE}" >> ${DIR}/../logs/appcwmon.log; } # Process A...

The following plugin pushes the memory consumption of the vm to the cloudwatch which you can use to set the alarms and also can use for the autoscaling or taking actions when combined with the events. #!/bin/bash # # About : Check used memory in percentage # # Name : cw_memory.sh DIR=$(dirname $0); PLUGIN_NAME='cw_memory'; # Include configuration file source ${DIR}/../conf/plugin.conf; #Get Current Instance ID INSTANCE_ID=(`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`); #Get Hostname HOST_ID=(`wget -q -O - http://169.254.169.254/latest/meta-data/hostname`); # Help usage() { echo "Usage: $0 [-n ] [-d ] [-m ] " 1>&2; exit 1; } # Logger logger(){ SEVERITY=$1; MESSAGE=$2; DATE=`date +"[%Y-%b-%d %H:%M:%S.%3N]"`; echo -e "${DATE} [${SEVERITY}] [${PLUGIN_NAME}] [${INSTANCE_ID}] [${HOST_ID}] ${MESSAGE}" >> ${DIR}/../logs/appcwmon.log; } # Process Arguments ...

Below is the plugin for monitoring the diskusage of specific mount via the cloudwatch. This would go in the bin folder in the cloudwatch and you need to create a file name like cw_diskusage.sh with the following script #!/bin/bash # # About : Percent of Disk usage by Mount based on Mount name # # Name : cw_diskuage.sh DIR=$(dirname $0); PLUGIN_NAME='cw_diskusage'; # Include configuration file source ${DIR}/../conf/plugin.conf; #Get Current Instance ID INSTANCE_ID=(`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`); #Get Hostname HOST_ID=(`wget -q -O - http://169.254.169.254/latest/meta-data/hostname`); # Help usage() { echo "Usage: $0 [-n ] [-d ] [-m ] [-f Mount Point]" 1>&2; exit 1; } # Logger logger(){ SEVERITY=$1; MESSAGE=$2; DATE=`date +"[%Y-%b-%d %H:%M:%S.%3N]"`; echo -e "${DATE} [${SEVERITY}] [${PLUGIN_NAME}] [${INSTANCE_ID}] [${HOST_ID}] ${MESSAGE}" >...

The Cloudwatch is a hosted tool provided by the aws to monitor different resources in your Cloud Infrastructure. AWS provides you with various metrice(data) related to resources to determine its state on per minute basis which can be used to monitor and raise an alarm whenever a certain threshold is crossed. You can configure the cloudwatch with the SNS to send the notification once the state of the alarm changes. Further you can configure the events and take any action on these alarms. The only limitation is that AWS provides you with certain metrices to monitor but there are times when you want to monitor the resources which are not provided by AWS. Like your services, established connections, processes, memory etc. For this you need to create your own custom cloudwatch metrics which you can push to the cloudwatch using the AWS Cli. Once the metrice has been configured in the cloudwatch than you can put the alarms on these metrices. You need to push the metrice regularly using t...