Using aws config to monitor Stacks with Screenshots

Aws config can be used to monitor the instances, security group and other resources within you AWS Account. This is specially useful to monitor the compliance of the instances and to raisae alarm if someone creates an instance without the proper procedure like missing tags, changes in the security group, non compliant instance type etc.

Aws config will mark them as non-compliant under the config dashboard and can send a notification using the SNS service.It can easily be deployed using the cloudformation and helps in managing the resource effectively. This is specially recommended in case you have many users in your organization with the dashboard access who have privilege to create the instances.

Follow the procedure descripbed below to configure the aws config in your environment for the ec2 instances where you can define the instances which are compliant in your aws account any instance apart from them would be marked as non-compliant and you configure an alert for the same.

Start by selecting the Config service from the aws dashboard

If you haven't configured the Aws config start by clicking "Get Started"

Installing php

3.  PHP Installation

Version: 5.3.17

export CFLAGS=-m64

export LDFLAGS=-m64

 yum install libjpeg-devel

yum install libpng-devel

yum install libtiff-devel

yum install libtool-ltdl-devel

yum install freetype-devel

Install libmcrypt-devel

Install libmcrypt

tar -zxvf mm-1.4.2.tar.gz

make

make test

make install

 tar -xvzf php-5.3.17.tar.gz

 cd php-5.3.17

  './configure'  '--prefix=/usr' '--with-libdir=lib64' '--libdir=/usr/lib64/' '--with-config-file-path=/etc' '--with-config-file-scan-dir=/etc/php.d' '--with-apxs2=/opt/www/apache/2.2.23/bin/apxs' '--with-libxml-dir=/usr' '--with-curl=/usr' '--with-mysql' '--with-zlib' '--with-zlib-dir=/usr' '--enable-sigchild' '--libexecdir=/usr/libexec' '--with-libdir=lib64' '--enable-sigchild' '--with-bz2' '--with-curl' '--with-exec-dir=/usr/bin' '--with-openssl' '--with-libexpat-dir=/usr' '--enable-exif' '--enable-ftp' '--enable-magic-quotes' '--enable-sockets' '--enable-sysvsem' '--enable-sysvshm' '--enable-sysvmsg' '--enable-wddx' '--with-kerberos' '--enable-ucd-snmp-hack' '--enable-shmop' '--enable-calendar' '--with-xmlrpc' '--with-pdo-mysql=/usr' '--enable-pcntl' '--disable-debug' '--enable-inline-optimization' '--enable-mbstring' '--enable-safe-mode' '--enable-wddx=shared' '--with-regex=system' '--enable-xml'  '--enable-soap' '--with-mcrypt=/usr' '--enable-inline-optimization' '--enable-mbregex' '--enable-shmop' '--with-xsl' '--with-gd' '--with-png-dir=/usr' '--with-jpeg-dir=/usr' '--enable-gd-native-ttf' '--with-iconv' '--enable-sockets' '--enable-sysvsem' '--enable-sysvshm' '--enable-sysvmsg' '--enable-trans-sid' '--with-kerberos' '--enable-ucd-snmp-hack' '--enable-shmop' '--enable-calendar' '--enable-dbx' '--enable-dio' '--with-xmlrpc' '--enable-soap' '--enable-pdo' '--with-openssl' '--with-xsl' '--with-mm=shared' '--enable-mbstring' '--with-mcrypt=/usr' '--enable-zip' '--with-xml2' '--with-freetype-dir=/usr'

make

make install

remove dbase.ini & ldap.ini from /etc/php.d

change extension_dir in /etc/php.ini to  /usr/lib64/extensions/no-debug-non-zts-20090626/

Upgrading Kernel Version on Aws Ec2 instance

There are multiple requirements for upgrading the Kernel version on the Ec2 instance on the AWS instances. You will definitely want to upgrade for the bug fixes and the stability which comes with the updated version.

In our case we have to install the Antivirus required for the audit purpose but the issue arise when we were using the older version of the Kernel which was not supported by the Antivirus. This was particularly essential since antivirus provides you with active defense against unknown threats which are not included in the free antivirus. So we were required to update the kernel version.

We were running the 3.13.0-105-lowlatency version of the kernel however the following version was supported by the Antivirus. We were required to upgrade the kernel to 3.13.0-142-generic for the antivirus to work.

Upgrading the kernel version on the Aws ec2 instances can be little tricky because if you don't do it properly it would certainly cause the startup failures. We already faced issues with one of our instance when we initially tried to upgrade the kernel version through apt directly after the restart the status checks for the instance reachability got failed and this in-turned added the complexity during the upgrade. I am going to discuss in a separate post , how you can recover an instance in case you have corrupted the kernel upgrade and Ec2 instance fails to start due to Ec2 instance health check failures. 

Its good to take the snapshot of the volumes separately for such scenarios in case your instance fails to start you can create new volume out of the snapshot, replace the old volumes with new and you should be able to start the instance without any issues. This is particularly the best saviour you have in case something goes wrong during the kernel upgrade. 

Once the snapshot is completed you just need to install the package bikeshed. ∫Bikeshed comes with additional binaries which are not shipped with the Linux distros by default.

 # apt-get install bikeshed  

Machine Learning Use Cases in the Financial Services

With the rapid changing digital age and our more dependence on the digital services in every domain from banking, payments, medical, ecommerce, investments etc needs a technology delivery model that's suited to how the world and consumer needs are changing so as to allow companies to develop new products and capabilities that kind of fits with the digital age.

We are considering the Capital One's use case for the Machine learning

There were 3 main fields for the Capital one to improve there banking relationship - Fraud Detection, Credit Risk, Cybersecurity. Improving these areas involves distinguishing patterns, something the neural networks underlying machine learning accomplish far better than traditional , non-AI software. The goals were to Approve as many transactions as possible by identifying fraud only when it's very likely to happen; make better decisions making around credit risk, track constantly evolving threats. Applying machine learning to these areas is a big oppurtunity.

One recent innovation for the same was a tool called SECOND LOOK , which uses machine learning to alert customers about unusual spending patterns, such as double charges, a repeating charge that is higher than the previous month , or a tip that's higher than the norm. According to the company , Second look saved customers millions of dollars in unwanted charges. 

Using the Comments in the Python

You can use the #  for single line comments.

# This is comment  # this is also comment

There is no concept of multi line comments in the python so you would need to enter the # infront of every line you want to comment in multiline case.

Creating and Running python scripts

You can simply start by creating a file which ends in the extension .py like

vim hello.py

Than just start by entering the print command

print ("Hello, World!")

Than you can execute this file with the python as follows

python hello.py

If you want to make this executable use the # in the linux as follows

#!/bin/python
chmod +x hello.py 

afterwards you can simply execute it like
./hello.py

Now always keep the executables in the bin directory

mkdir bin

mv hello.py bin/hello

This way you can simply make it a command which can directly be executed in the linux terminal and this is how most of the commands actually work in the Linux

You just need to modify the source to match yours , it can be done as follows

source PATH=$PATH:$HOME/bin/

REPL in the python

REPL in python Read Evaluate print Loop . It helps you to work with the python the easy way when you start learning the python initially. You can simply login to the python by just writing the python on the terminal window. And once you evaluated than you exit out using the exit() function remember it neads () otherwise just writing the exit wouldn't allow you to log out.

History of the Python

1. Created by Guido van Rossum
2. First apperance in 1991
3. Used and supported by the tech giants like google and youtube.
4. Supported two major version for nearly a decade (python2 and python3)
5. Object oriented Scripting Language
6. Dynamic and Strong type system
7. Functional Concepts (map,reduce,filter,etc)
8. Whitespace delimited with pseudo-code like syntax
9. Used across a variety of descipilines, including:
a.) Academia
b.) Data Science
c.) Devops
d.) Web Development

10. Runs on all major operating systems.
11. Consistently high on the Tiobe index