Setting up a Jenkins Server on AWS EC2 instance

Let us first understand what is Jenkins? Jenkins is a self-contained, open source automation server which can be used to automate all sorts of tasks related to building, testing, and delivering or deploying software. Jenkins is a software that allows continuous integration.

Prerequisites:

Launch and SSH into EC2 Instance.
Don't forget to add Port 8080 in the Security Group to open for all incoming connections.
Java JDK should be installed.

Install Java 1.8 JDK

• sudo su
• yum update –y
• yum install java-1.8*

Update Bash_profile to set JAVA_HOME variable

• To get the path of java binary - find /usr/lib/jvm/ -name java
• Add below lines to your .bash_profile file in user's home directory. 
• JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64
• export JAVA_HOME
• PATH=$PATH:$JAVA_HOME
• save the file :wq 
• To make the changes effective immediately update your .bash_profile
• source ~/.bash_profile
• java -version (check the java version)

Install Jenkins

yum -y install wget
wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo

Import a key file from Jenkins-CI to enable installation from the package

rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
yum install jenkins -y

Start Jenkins as a service

## Start jenkins service
systemctl start jenkins

## Setup Jenkins to start at boot,
systemctl enable jenkins

Accessing Jenkins UI

Visit the following address in your browser, http://EC2-SERVER-PUBLIC-IP:8080

Things to do after first login:

• Default Password Location: cat /var/lib/jenkins/secrets/initialAdminPassword

• Change admin password
• Admin > Configure > Password

• Configure java path
• Manage Jenkins > Global Tool Configuration > JDK

• Now you are ready to create a job for your build. Test Jenkins Job

• Create “new item”
• Enter an item name – My-first-job
• Chose Freestyle project
• Under Build section - choose option - Execute shell : echo "Welcome to Jenkins Demo"
• Save your job
• Build job
• Check "console output" for successful completion of job.

Hurray!! you have just created and ran your first job.

Restarting Jenkins through browser (Only if you are logged in as Admin user)
http://<ip-address>:8080/restart

Designing HA Architecture in AWS part-3

Amazon Web Services (AWS) offers a wide range of services that we can leverage to implement High Availability (HA) to any web application that we deploy on the AWS cloud.

I am going to create a Web application which is highly available, resilient, optimized and secure. It delivers high performance and low latency everytime users are accessing it and from whereever the users are accessing it.
I have deployed this application with following parameters:

Region - ap-south-1 (Asia Pacific Mumbai)
Availability Zone - AZ-a and AZ-b
A highly available architecture that spans two Availability Zones. Multi-AZ deployment.
VPC - I have created 2 Virtual Private Clouds

• testVPC (10.0.0.0/16)
• new-testVPC (10.1.0.0/16)

Subnets - I have created 2 Subnets and 2 Subnets in this VPC.

Under Availability Zone AZ-b

• Public_subnet1b = 10.0.1.0/24
• Private_subnet1b = 10.0.2.0/24

Under Availability Zone AZ-a

• Public_subnet1a = 10.0.20.0/24
• Private_subnet1a = 10.0.21.0/24

WPS - I have provisioned the EC2 instances in both of the Public Subnets and installed wordpress on them and named them WPS. So at any point of time we will be having traffic served by atleast 2 intances (1 in each subnet) to a maximum capacity of 6 instances (3 in each subnet).

Route53 - Its a DNS service used for Domain Name Registration, maintaining DNS record sets and setting up different routing policies for Domain Names.

Schematic diagram of the HA architecture below:

I have tried to explain this in Problem and Solution fashion, means what problems I have faced during designing a HA architecture and how I have implemented the solution to that problem.


Problem - How to manage unpredictable load on webserver EC2 instances in our architecture?
Solution - Auto-Scaling Groups

• An Auto-Scaling group is dependent on 2 things: Launch Configuration and Scaling Policies.

Implementation -

• First choose the Launch Configuration and the VPC and Subnet in which we are implementing the Auto-Scaling group.
• Define the group size starting with 1 instance.
• Next step defining the minimum and maximum size of our group. (Min=1 and Max=3 instances)
• Create an alarm for Increase Group Size : Avg. CPU utilization >= 75% for consecutive period of 5 mins.
• You can optionally opt for sending Notification to a Topic name when the above alarm is triggered.
• Similarly create an alarm for Decrease Group Size : Avg. CPU utilization < 60% for consecutive period of 5 mins.
• Under the instances tab on selecting Auto-Scaling group, we can see the lifecycle/status of the instances.

Problem - How to get an auto-configured server everytime an instance is added in the Auto-Scaling group?

Solution - Snapshot

• A Snapshot is a backup of a single EBS volume. You can create an AMI from Snapshot. It is not a bootable copy but an AMI is.

Implementation -

• Provision a template EC2 instance -> Install Apache,Wordpress and all the key configurations
• Take Snapshot of the root volume of the EC2 instance, (we can terminate the template instance after taking snapshot).
• Create an AMI with this Snapshot and name it myWebAppAMI.
• Use this AMI as Launch Configuration in the Auto Scaling group to spawn an auto-configured EC2 instance in the subnets.
• In this manner we will be saving the boot time of the newly created EC2 instances and no need to do configurations remotely everytime a server boots up.

Problem - How to update/patch the EC2 instance in private subnet as there is no inbound nor outbound internet access?
Solution - NAT instance

• A NAT instance, allows your private instances outgoing connectivity to the internet while at the same time blocking inbound traffic from the internet.
• A NAT instance is similar to a normal EC2 instance with NAT Optimized HVM type AMI.

Implementation -

• In the public subnet (public_subnet1a), provisioned a NAT instance to allow outbound internet access for DB_server instance in the private subnet (private_subnet1a).
• After running this instance I have changed the Destination/Source check and disabled it. To do this, right click on your NAT Instance within the AWS Console and select ‘Networking > Change Source/Dest. Check > Yes, Disable’.
• NAT Gateways provide the same functionality as a NAT instance, however, a NAT Gateway is an AWS managed NAT service. As a result, these NAT Gateways offer greater availability and bandwidth and require less configuration and administration. This was a costlier option and moreover should be applied to a very large scale application.

Problem - How to have SSH access of EC2 instances in private subnets?
Solution - Bastion hosts

• A Bastion Host is a special purpose computer on a host designed and configured to withstand attacks.
• It acts as a jump server, allowing you to use SSH or RDP to log in to other instances within private subnets.

Implementation -

• As we know the servers in private subnets are not configured to talk to the outside network. I have updated the Private Security Group to accept all the inbound/outbound traffic from Public Security Group.
• In the public subnets, EC2 instances in an Auto Scaling group to allow inbound Secure Shell (SSH) access to EC2 instances in private subnets.

Problem - How to maintain performance of website with the increasing site traffic?
Solution - Elastic Load Balancer (ELB)
Implementation -

• An Application Load Balancer must be deployed into at least two subnets (Public_subnet1a & Public_subnet1b) to distribute HTTP and HTTPS requests across multiple WordPress instances (WPS1,WPS2,..,WPSn).

Problem - Increasing site traffic means increasing query load on Database, How to cope up?
Solution - ElastiCache

• Caching improves application performance by storing critical pieces of data in memory for low latency access.
• Cached information may include the results of I/O-intensive database queries or the results of computationally-intensive calculations.
• Suppose we are running an online business, customers continuously asking for the information of a particular product. Instead of making a call to DB and always asking information for that product, we can cache the product data using Elasticache.

Implementation -

• I have used Redis nodes for caching database queries.
• As the Redis Cache is dependent on just Memory, we should always create the Redis node with Memory Optimized Instance type – (X1, R5, R4, R3)
• Port should be kept default port - 6379
• We can set number of Replicas from 0 to 5 to be a part of the cluster.

Problem - How to control the inbound/outbound traffic in the VPC?
Solution - Security Groups and Network Access Control Lists

• Security Groups add a security layer to EC2 instances that control both inbound and outbound traffic at the instance level.
• NACL also adds an additional layer of security associated with subnets that control both inbound and outbound traffic at the subnet level.

Implementation -

• I have created 2 Security Groups - test_security_group and test_security_group_private.
• I have declared the routes in the route tables and associated the subnets with the respective table - public_route_table and private_route_table

Problem - Why not create a MySQL Database instance instead of a RDS instance?
Solution - RDS (MySQL)
Implementation -

• Creating a RDS endpoint instead of deploying mySQL Database on an EC2 instance is better in many ways.
• It frees you from managing the time-consuming database administration tasks such as provisioning, backups, software patching, monitoring, and hardware scaling.
• It supports "License-included" licensing model, so we do not have to care about purchasing license separately.
• Amazon RDS provides high availability of MySQL Server using multi-availability zone capability, and this reduces the risk to data loss.

Problem - How should I keep an active most recent backup of our WebApp?
Solution - Amazon S3

• S3 buckets store the data as objects. We have many advantages of backing up any webapp on these buckets because of their high availability and durability.

Implementation -

• I am using S3-IA or S3-RRS buckets instead of S3-Standard to decrease the costing of this model as we do not want to access it frequently.
• I have created a Cron job running on the WebApp EC2 instance to take backup of site hourly.
• * */1 * * * aws s3 sync --delete /var/www/html s3://<bucket_name>

Problem - How to minimize the high latency if users are accessing this web application from US/Europe or any other part of globe?
Solution - CloudFront
Implementation -

• I have implemented the CloudFront as the static Content Delivery Network because static content is heavy (such as JPG, media, Audio files) and unnecessary load on EC2 instances.
• All the static content is being served from the nearest edge locations to the users worldwide. This has helped a lot in maintaining a low latency irrespective of the placement of the EC2 instances.

Problem - The Web application needs to talk to an EC2 instance in a different VPC, how should we do it?
Solution - VPC Peering

• A VPC peering connection is a networking connection between two VPCs that enables you to route traffic between them using private IPv4 addresses or IPv6 addresses.
• Instances in either VPC can communicate with each other as if they are within the same network.
• You can create a VPC peering connection between your own VPCs, or with a VPC in another AWS account.
• The VPCs can be in different regions (also known as an inter-region VPC peering connection), it provides a simple and cost-effective way to share resources between regions.

Implementation -

• This peering was added to the architecture taking into account that some service might be required by WebApp running in main VPC to communicate/fetch data from some other EC2 instance in different VPC.
• I have created a VPC peering named test_peer between the 2 VPCs (testVPC and new-testVPC) which has made communication possible between these VPCs.
• Additionally I have to add an entry in route table for both of the public subnets in which the EC2 instances needs to communicate.
• public_route_table -> 10.1.0.0./16 test_peer
• new_public_route_table -> 10.0.0.0./16 test_peer

I hope this information will be helpful for upcoming solution architects. If you have any doubt/query please feel free to ask in the comment section below.

Components of AWS part-2

In this blog we are going to take a deeper dive into the components that make AWS instead of having a birds eye view of it.

You might be thinking of what could be the infrastructure of AWS which is providing infrastructure to millions of users and thousands of customers worldwide.

AWS Global Infrastructure

The following are the components that make up the AWS infrastructure:

• Availability Zones - Availability Zones as a Data Center,  An availability zone is a facility that can be somewhere in a country or in a city. Inside this facility, i.e., Data Centre, we can have multiple servers, switches, load balancing, firewalls. The things which interact with the cloud sits inside the data centers.
• Region - Region is a distinct geographical area & can have 2 or more AZ. A region is a collection of data centers which are completely isolated from other regions. Currently there are 22 regions across the globe.
• Edge locations - Edge Locations are endpoints for AWS which are used for caching content. Typically consists of CloudFront, Amazon's Content Delivery Network (CDN). They are mainly located in most of the major cities to distribute the content to end users with reduced latency. Currently there are more than 150 edge locations.
• Regional Edge Caches - Regional Edge cache lies between CloudFront Origin servers and the edge locations. A regional edge cache has a large cache than an individual edge location. Data is removed from the cache at the edge location while the data is retained at the Regional Edge Caches. When the user requests the data, then data is no longer available at the edge location. Therefore, the edge location retrieves the cached data from the Regional edge cache instead of the Origin servers that have high latency.

Below are the major services under various domains:

Networking and Content Delivery

1. VPC - A Virtual Private Cloud is your private section of AWS, it provides a logically isolated area where you can launch AWS resources, and allow/restrict access to them. 
• We can create Subnets (Private/Public) in a VPC and can assign custom IP address ranges in each subnet.
• Max 5 VPCs allowed in each AWS Region by default.


2. Route 53 - DNS Routing service (Manage DNS records of the domain) Routing of traffic to EC2 instances can be based on 
• Weighted percentages
• Latency based
• Failover by creating health checks on each Record sets 
• Geolocation based
• MultiValue Answer policy is Simple Routing with health checks.


3. API Gateway - It is a gateway which lets the incoming API calls communicates with a bunch of Lambda functions that create a serverless system and serve the users with response from those functions.


4. CloudFront - Content Delivery Network is a system of distributed servers that deliver web pages and other web content to a user based on the geographic locations of the user, the origin of the webpage and a content delivery server. CDN comprises of following components:
1. Edge Location: Edge location is the location where the content will be cached. It is a separate to an AWS Region or AWS availability zone.
2. Origin: It defines the origin of all the files that CDN will distribute. Origin can be either an S3 bucket, an EC2 instance or an Elastic Load Balancer.
3. Distribution: It is the name given to the CDN which consists of a collection of edge locations. When we create a new CDN in a network with AWS means that we are creating a Distribution. The distribution can be of two types:
1. Web Distribution: It is typically used for websites. When a user requests for content, the request is automatically routed to the nearest edge location so that the content is delivered with the best possible performance.
2. RTMP: It is used for Media Streaming.

Compute

1. EC2 - EC2 stands for Amazon Elastic Compute Cloud.
1. Amazon EC2 is a web service that provides resizable compute capacity in the cloud.
2. Amazon EC2 reduces the time required to obtain and boot new user instances to minutes rather than in older day which was a very time-consuming process.
3. You can scale the compute capacity up and down as per the computing requirement changes.
4. Amazon EC2 has changed the economics of computing by allowing you to pay only for the resources that you actually use. Rather than previously we use to use physical servers on lease or purchase them, so we have to plan for 5 years in advance. This end up in spending a lot of capital in such investments.
5. EC2 pricing options
1. On Demand - On Demand is perfect for the users who want low cost and flexibility of Amazon EC2 without any up-front investment or long-term commitment. It is suitable for the applications with short term, spiky or unpredictable workloads that cannot be interrupted.
2. Reserved - In a Reserved instance, you are making a contract means you are paying some upfront, so it gives you a significant discount on the hourly charge for an instance. It is used for those applications that require reserved capacity.
3. Spot Instances - It allows you to bid for a price whatever price that you want for instance capacity, and providing better savings if your applications have flexible start and end times. It is useful for those applications that are feasible at very low compute prices.
4. Dedicated Host - A dedicated host is a physical server with EC2 instance capacity which is fully dedicated to your use.


2. Elastic Beanstalk - It is a PAAS (Platform as a Service) used for deploying and scaling web applications/services developed with Java, PHP,Node.js on familiar servers like Apache, Nginx, Tomcat, IIS.
1. Elastic Beanstalk is one layer of abstraction away from the EC2 layer. Elastic Beanstalk will setup an "environment" for you that can contain a number of EC2 instances, an optional database, as well as a few other AWS components such as a Elastic Load Balancer, Auto-Scaling Group, Security Group. 
1. Load Balancing
2. Auto Scaling
3. Health Monitoring
2. EB offers two different Environment tiers:
1. Web Server Environment: Handles HTTP requests from clients
2. Worker Environment: Processes background tasks which are resource consuming and time intensive
3. Each environment runs only a single application version at a time. But it is possible to run same or different versions of an application in many environments at the same time.
4. After terminating an environment, You can restore it if terminated in the last six weeks.


3. Lambda - It is a pay-as-you-go serverless compute service. It is known as a Function as a Service(FAAS)
1. All lambda functions are stateless, meaning they cannot store persistent data.
2. You deploy some code, it gets invoked, processes some input, and returns a value.
3. It is always used in conjunction with API Gateways to create serverless model. Means it will always be invoked through an API gateway.
4. Lambda is used to encapsulate Data centres, Hardware, Assembly code/Protocols, high-level languages, operating systems, AWS APIs.
5. Lambda is a compute service where you can upload your code and create the Lambda function.
6. Lambda takes care of provisioning and managing the servers used to run the code.
7. While using Lambda, you don't have to worry about scaling, patching, operating systems, etc.

Storage

1. EBS stands for Elastic Block Store.
1. Amazon EBS allows you to create storage volumes and attach them to the EC2 instances.
2. Once the storage volume is created, you can create a file system on the top of these volumes, and then you can run a database, store the files, applications or you can even use them as a block device in some other way.
3. Amazon EBS volumes are placed in a specific availability zone, and they are automatically replicated to protect you from the failure of a single component.
4. EBS volume attached to the EC2 instance where windows or Linux is installed known as Root device of volume.
5. EBS Volume types fall into two parts:
1. SSD-backed volumes
2. HDD-backed volumes
6. SSD is further classified into two parts:
1. General Purpose SSD - General Purpose SSD is also referred as GP2. It is required where application uses less than 10,000 IOPS.
2. Provisioned IOPS SSD - It is also referred to as IO1. It is mainly used for high-performance applications such as intense applications, relational databases. It is used when you require more than 10,000 IOPS.


2. S3 stands for Simple Storage Service.
1. It is an Object-based storage, i.e., you can store the images, word files, pdf files, etc.
2. The files which are stored in S3 can be from 0 Bytes to 5 TB.
3. It has unlimited storage means that you can store the data as much you want.
4. Files are stored in Bucket. A bucket is like a folder available in S3 that stores the files. You can put the permissions individually on your files or on complete bucket.
5. S3 is a universal namespace, i.e., the bucket names must be unique globally. Bucket contains a DNS address.
6. If you create a bucket, URL look like: https://<bucket-name>.s3-<AWS-region>.amazonaws.com


3. Snowball - These are physical devices that help migrate large amounts of data into and out of the cloud without depending on networks.
1. Snowball is a suitcase-sized device, Snowball Edge is a rack mountable and clusterable suitcase sized device with compute capabilities, and Snowmobile is a shipping container moved with a tractor-trailer.
2. With Snowball service we can migrate data in amount ranging between 100 TeraBytes to 10 PetaBytes.


4. Storage Gateway - Storage Gateway is a service in AWS that connects an on-premises software appliance with the cloud-based storage to provide secure integration between an organization's on-premises IT environment and AWS storage infrastructure.
1. File Gateway (NFS) - It is used to store the flat files in S3 such as word files, pdf files, pictures, videos, etc
1. Files are directly stored as objects in S3 buckets, and they are accessed through a Network File System (NFS) mount point.
2. Ownership, permissions, and timestamps are durably stored in S3 in the user metadata of the object associated with the file.
2. Volume Gateway (iSCSI) - Volume Gateway is an interface that presents your applications with disk volumes using the Iscsi block protocol.
1. The iSCSI block protocol is block-based storage that can store an operating system, applications and also can run the SQL Server, database.
2. Data written to the hard disk can be asynchronously backed up as point-in-time snapshots in your hard disks and stored in the cloud as EBS snapshots 
3. Tape Gateway (VTL) - It is mainly used for taking backups.
1. Tape Gateway offers a durable, cost-effective solution to archive your data in AWS cloud.
2. The VTL interface provides a tape-based backup application infrastructure to store data on virtual tape cartridges that you create on your tape Gateway.
3. It is supported by NetBackup, Backup Exec, Veeam, etc. Instead of using physical tape, they are using virtual tape, and these virtual tapes are further stored in Amazon S3.

Database

1. RDS - stands for Relational Database Service. It supports six commonly used database engines. The Amazon RDS Free Tier provides a single db.t2.micro instance as well as up to 20 GiB of storage.


2. DynamoDB - It is a fast and flexible NoSQL database service.


3. ElasticCache - It is a web service used to deploy, operate, and scale an in-memory cache in the cloud. It improves the performance of web applications by allowing you to retrieve information from fast, managed in-memory cache instead of relying entirely on slower disk-based databases. Caching improves application performance by storing critical pieces of data in memory for low latency access. There are two types of Elasticache:
1. Memcached - Memcached keeps its data in memory by eliminating the need to access the disk.
1. Memcached uses the in-memory key-value store service that avoids the seek time delays and can access the data in microseconds.
2. It is a distributed service means that it can be scaled out by adding new nodes.
2. Redis - Redis stands for Remote Dictionary Server.
1. It is a fast, open-source, and in-memory key-value data store.
2. Its response time is in a millisecond, and also serves the millions of requests per second for real-time applications such as Gaming, AdTech, Financial services, Health care, and IoT.

Security, Identity & Compliance

1. IAM - Identity Access Management
1. IAM Policies are made up of documents called Policy Documents. These docs are in JSON format. 
2. Roles are made up of policies.
3. Roles can be assigned to users or to a group. Best practice is to assign roles to the groups and add users to that group.
4. SAML(Security Assertion Markup language) is a technique of achieving Single Sign-On (SSO) means that users have to log in once and can use the same credentials to log in to another service provider.
5. SAML provides security by eliminating passwords for an app and replacing them with the security tokens.
6. Two types of SAML providers: Service provider & Identity provider.

Management and Governance

1. CloudFormation - It is a tool from AWS that allows you to spin up resources effortlessly. You define all the resources you want AWS to spin up in a blueprint document, click a button, and then AWS will creates all of the components. This blueprint is called a template.
1. CloudFormation makes sure that dependent resources in your template are all created in the proper order. For example if DNS record points to an EC2 instance then the CF will provision the EC2 instance first, wait for it to be ready and then create the Route53 DNS record afterwards.
2. CF declare the template as JSON format.


2. CloudWatch - CloudWatch is a service used to monitor your AWS resources and applications that you run on AWS in real time. 
1. CloudWatch is used to collect and track metrics that measure your resources and applications.
2. Following are the terms associated with CloudWatch:
1. Dashboards: CloudWatch is used to create dashboards to show what is happening with your AWS environment.
2. Alarms: It allows you to set alarms to notify you whenever a particular threshold is hit.
3. Logs: CloudWatch logs help you to aggregate, monitor, and store logs.
4. Events: CloudWatch help you to respond to state changes to your AWS resources.


3. Auto Scaling - Scale your EC2 instances capacity automatically. enabled by Amazon CloudWatch. Scale In/Scale Out EC2 instance to/from Auto Scaling groups as per the launch configuration, when scheduled event is met or Cloud Watch event is triggered. We have to create Launch Configuration first (choice of AMI and EC2 instance type) then Auto-Scaling group.

Application Integration

1. SNS - SNS stands for Simple Notification Service.
1. It is a way of sending messages. When you are using AutoScaling, it triggers an SNS service which will email you that "your EC2 instance is growing".
2. SNS notifications can also trigger the Lambda function. When a message is published to an SNS topic that has a Lambda function associated with it, Lambda function is invoked with the payload of the message.
3. Amazon SNS allows you to group multiple recipients using topics where the topic is a logical access point that sends the identical copies of the same message to the subscribe recipients.
4. To prevent the loss of data, all messages published to SNS are stored redundantly across multiple availability zones.


2. SQS - SQS stands for Simple Queue Service.
1. Amazon SQS is a web service that gives you access to a message queue that can be used to store messages while waiting for a computer to process them.
2. Amazon SQS is a distributed queue system that enables web service applications to quickly and reliably queue messages that one component in the application generates to be consumed by another component where a queue is a temporary repository for messages that are awaiting processing.
3. Messages can contain up to 256 KB of text in any format such as json, xml, etc.
4. Used if the producer is producing work faster than the consumer can process it, or if the producer or consumer is only intermittently connected to the network.
5. The Default Visibility Timeout is 30 seconds. Visibility Timeout can be increased if your task takes more than 30 seconds. The maximum Visibility Timeout is 12 hours.
6. There are two types of Queue:
1. Standard Queues (default)
2. FIFO Queues (First-In-First-Out)


3. SWF - SWF stands for Simple Workflow Service.

So far we have covered all the major services that are useful for creating a high availability architecture in a cloud.

I have created a simple architecture for hosting a web app which I kept evolving while I was in the process of learning AWS. Now I have scaled it to a highly available, reliable and scalable architecture, which we will be covering in the next section : Designing HA Architecture in AWS part-3

Concepts of AWS part-1

Nowadays we have so many cloud services hosted on the internet by various providers like Apple iCloud, Google Cloud Platform, Microsoft Azure, Amazon Web Services , IBM Cloud, Salesforce and many others.
Everybody has there own perception of how things are uploaded/downloaded/accessed from a cloud. But people are not keen to learn about how things actually work in this process.
Because of the versatility and vastness of Amazon Web Services I have chosen this to understand and clear my concept of cloud architecture.

Lets start with understanding what is AWS and its services first. In the end we would be able to create a cloud architecture ourself by connecting all the dots together.

What is AWS?

AWS stands for Amazon Web Services.

Amazon's Cloud services provide great flexibility in provisioning, duplicating and scaling resources to balance the requirements of users, hosted applications and solutions.
Cloud services are built, operated and managed by a cloud service provider, which works to ensure end-to-end availability, reliability and security of the cloud.

AWS ensures the three aspects of security, i.e., Confidentiality, integrity, and availability of user's data.

There are three basic types of cloud services:

• Software as a service (SaaS)
• Infrastructure as a service (IaaS)
• Platform as a service (PaaS)

In addition to these services above, AWS also offers Function as a Service (FaaS), which is the concept on which Serverless computing is build.
These services are the building blocks that can be used to create and deploy any type of application in the cloud.
Currently there are around 165 services that are being offered by AWS.
If you want to know more about the services offered by AWS. Please feel free to follow this link.

History of AWS

• In 2003, Chris Pinkham and Benjamin Black presented a paper on how Amazon's own internal infrastructure should look like. They suggested to sell it as a service and prepared a business case on it. They prepared a six-page document and had a look over it to proceed with it or not. They decided to proceed with the documentation.
• In 2004, the first web service SQS which stands for "Simple Queue Service" was officially launched. 
• In 2006, AWS (Amazon Web Services) was officially re-launched, combining the three initial service offerings of Amazon S3 cloud storage, SQS and EC2.
• In 2007, over 180,000 developers had signed up for the AWS.
• In 2014, AWS claimed its aim was to achieve 100% renewable energy usage in the future.
• In 2015, AWS breaks its revenue and reaches to $6 Billion USD per annum. The revenue was growing 90% every year.
• By 2016, revenue doubled and reached $13 Billion USD per annum.
• In 2018, AWS launched a Machine Learning Speciality Certs. It heavily focused on automating Artificial Intelligence and Machine learning.

Advantages of AWS

1. High Availability and durability (99.9999999%)
2. High Scalability/Elasticity (expand/shrink on demand)
3. Fault tolerance (Reliable/Resilient)
4. Based on the concept of Pay-As-You-Go - Pay for the resources when you need them.
5. Cost-effectiveness - No long term commitments/huge investments in physical infrastructure.
6. Loosely coupled architecture - Best fit for adopting microservice architecture.

How to SignUp to the AWS platform

• First visit this website https://aws.amazon.com/, then click on the Complete Sign Up to create an account and fill the required details.
• Now, fill your contact information.
• After providing the contact information, fill your payment information.(Don't worry nothing will be deducted from your account)
• After providing your payment information, confirm your identity by entering your phone number and security check code, and then click on the "Contact me" button.
• AWS will contact you to verify whether the provided contact number is correct or not via a phone call.
• The final step is the confirmation step. Click on the link to log in again; it redirects you to the "Management Console".

AWS provides 4 plans, you can choose as per your usage/features: 

• Basic - Free
• Developer - Starting at $29 per month
• Business - Starting at $100 per month
• Enterprise - Starting at $15,000 per month

Wondering how much percentage of internet is comprised of AWS??

AWS hosts about 5% of all websites and accounts for about 40% of all Internet traffic.
You can check by blocking all the traffic coming from IP address ranges of AWS hosted servers, which is shared by AWS here
There is a simple script called AWS Blocker created by a developer which retrieve the official list of AWS IPv4 and IPv6 ranges, then block them all using iptables.
After running the above script on your linux machine, you won’t be able to listen to Spotify, book a flight on Expedia, or look at rooms on Airbnb & moreover not able to watch your favorite seasons on Netflix :'(
This is what the internet would look if Amazon Web Services suddenly ceased to exist.

In the next blog, we will deeply discuss about the components that make AWS such an amazing cloud platform in the coming future. Components of AWS part-2

Riders paradise - 5 B's of Bikerism

Becoming a rider is not an easy task, you have to be an engineer, a manager, a mechanic, a cook, a guide and many others qualities.
Its not just by riding a bike and having a destination in your mind will make you a rider. Somethings only comes with passion. So you don't require any degree just require that passion with few qualities and respect for other riders.

The basic requirements for becoming a rider are 5 B's, else you will be called a traveler.

5 B's of Bikerism and their significance

Bullet - All other bikes will serve the purpose of taking you from your start point to destination but if you are riding a Royal Enfield then whole route will become more travel worthy. That sturdy look of the bike itself will say that anything comes into my way I am gonna get through it. That thudding sound will make you feel like a roaring beast on the highway. When a rider controls the massiveness of a bullet while making turns, that feeling is something out of this world.

Beard - It's easier to grow but difficult to maintain. A normal man can grow a beard ranging from an inch to 2-3 feets depending upon his genes and surely on his strengthened thoughts. The beard lover likes the things raw, they don't like artificial make-ups. In ancient times, Beard was a symbol of royalness, strength and position in society.
It projects one's traits like Patience, Perseverance and exudes his wisdom.
Beard will not only completes a rider look but it will also protects him from cold chilly winds.
Trust me while riding in colder areas the snowflakes that are stuck in your beard will look like a diamond studded in some finest jewelry.

Boots - Boots can be of many types and brands, but a rider will always choose a pair of boots that are old school and rugged in looks at the same time can sustain the roughness of the route. You'll feel heavy initially but later when you'll ride, they will go best with the biker attire.
Boots make people look at you from bottom to top, instead of looking from top to bottom. Choose very wisely as this is an essential part of your biker attire.

Biker Jacket - Like in the war between 2 countries how soldiers distinguish the soldiers from other country, just by looking at their uniform.
Similarly for rider to be in a perfect attire, he needs to wear a jacket that must be different from his regular jackets, to distinguish him from the crowd.
It will not only provide you safety with the padded armour but will also gives a rider a sense of mission accomplishment. That pride ahhaa!!!

Biker Tattoo - Tattoo is an art, who doesn't loves art can not be a nature lover. It's a symbol of commitment to anything, whether it can be your loved ones, role model or your passion.
Don't rush in choosing one, even the tattoo artist matters a lot.

Good luck in finding that perfect tattoo design that you can wear for the rest of your life!


Information regarding my Leh-Ladakh expedition and route yet to be shared in next blog, stay tuned RIDERS!!

Configuring Raspberry Pi as NAS,VNC,Samba,Apache server

The Raspberry Pi is a small, credit card sized computer that doesn’t require a lot of power to use.
It can be used for solving many purposes in day to day technical life and building many IOT (Internet Of Things) systems.

Here we are going to discuss about the configurations to be done after you have bought the RPi. It comes without any OS to boot and test the device. First we need a 16 GB Class 10 micro SD card. (Preferably Sandisk Ultra UHS-I)

Creating bootable Raspbian OS image

Download the Raspbian Jessie OS image:
https://www.raspberrypi.org/downloads/raspbian/

For Linux: DD command is use for bit-by-bit copying the image on the memory card
umount /dev/sdx (For unmounting the SD card)
dd bs=10M if=~/2015-02-16-raspbian-jessie.img of=/dev/sdx

For Windows:
Install this software - Win32DiskImager

First thing First!

After installation of Jessie Raspbian first thing should be connecting to RPi via SSH and upgrading Package & Firmware
Default Username/Password : pi / raspberry

sudo apt-get update
sudo raspi-update
sudo raspi-config -> Expand File System (requires reboot)

Configuring Static IP Address on RPi

sudo nano /etc/network/interfaces
Replace iface eth0 inet dhcp with static
auto eth0
iface eth0 inet static
address 192.168.0.2
netmask 255.255.255.0
network 192.168.0.0
gateway 192.168.0.1

(Reboot again)  This will assign the RPi 192.168.0.2 IP everytime it boots.

Automatically connecting to a WiFi network

sudo nano /etc/network/interfaces
on wlan0
# allow-hotplug wlan0
         iface wlan0 inet manual
         wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
         iface default inet dhcp

sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

Add this in the end of file
network={
ssid="WiFi_name"
proto=RSN
key_mgmt=WPA-PSK
pairwise=CCMP TKIP
group=CCMP TKIP
psk="wifi_password"
}

(Reboot again)  This is very useful configuration when you don't have access to UI and you have to connect the RPi to a Wireless network everytime it boots. No need to connect it through LAN.

---

Now the networking part is over, we are ready to install the services on RPI now.

Services you can run on your RPi

Wireless printer using Rpi

Samba share drive on Rpi

VNC server on Rpi

Apache server with PHP & mysql

NAS server

Running boot scripts & commands at startup (Editing /etc/rc.local)

Installing Samba server

sudo apt-get install samba samba-common-bin

sudo vi /etc/samba/smb.conf

Paste in the end of Share Definitions :-

[pihome]

   comment= Pi Home

   path=/home/pi

   browseable=Yes

   writeable=Yes

   only guest=no

   create mask=0777

   directory mask=0777

   public=no

sudo smbpasswd -a pi (Adding a pi share user)

service samba restart

Installing VNC server (Remote Desktop)

sudo apt-get install xrdp

On Windows Run > mstsc (for Remote Desktop connection)

This service is very useful when you don't have a external keyboard/mouse to operate the UI of Raspbian OS, so we utilize laptop's keyboard via RDP connection.

Installing Apache MySql Server

sudo apt-get install apache2

sudo apt-get install mysql-server

verify the verison when installation is done

mysql –version

mysql -u root –p

mysql> show databases;

Install PHP & MySql driver for PHP

sudo apt-get install php5 php5-mysql

Testing PHP : 

sudo nano /var/www/hello.php

Converting Raspberry Pi into a NAS device

Add support to Raspbian for NTFS-formatted disks. To do so type the following command:
sudo apt-get install ntfs-3g (This is present on OS but still to upgrade the package)

Look for the unmounted partitions of the attached external hard drives.
sudo fdisk –l
umount /dev/sda1 (Suppose you external hard disk is sda1)
sudo mkdir /media/USBHDD

sudo mount -t auto /dev/sda1 /media/USBHDD

sudo nano /etc/samba/smb.conf


[NAS HDD]
comment = NAS Drive
path = /media/USBHDD
valid users = @users
force group = users
create mask = 0660
directory mask = 0771
read only = no

Creating a Pi share user

sudo useradd nas -m -G users (Adding a user “nas”)
sudo passwd nas

sudo smbpasswd -a nas (Adding “nas” user to smb)

When Rpi restarts it will automatically mount the external hard drives:

sudo nano /etc/fstab

/dev/sda1 /media/USBHDD auto noatime 0 0

How to extend the range of WiFi adapter

Hi friends, I feel enormously happy to explain this hack to all of you. Do you know how much is the range of a normal WiFi adapter (Internal / External), it's just a few meters 10-15 mts or for some dongles it may varies between 15-20 mts if nothing comes in between the router and the wireless adapter.

So I have decided to write this blog so that everyone should know how they can increase their WiFi adapter range but unfortunately this hack can't increase the internal WiFi adapter range, this is only for External WiFi adapter. For this hacks you require certain things like:

• Parabolic Dish (Unused Reliance Dish Antennae)
• External USB WiFi Adapter (TP Link TL-WN721N)
• USB extension cable Male to Female (1 to 5 mts)

The basic principle behind this hack is the principle of convergence of waves over a focal point of parabolic dish. You can learn about this in detail from the wiki link. (https://en.wikipedia.org/wiki/Parabolic_reflector)

To calculate the focal point of  the parabolic dish there is a formula, which gives a relationship between the diameter D, the depth d and the focal distance f of the dish.

f = D^2 / 16d

The above formula helps in positioning the feed of the parabolic antennas as it gives the focal distance f.

The focus of this hack is to find the focus of the parabola dish that you have and install your WiFi adapter on that spot. Please see these images how I made it.

You can install it on a camera tripod if u have one or adjust it between AC and the upper wall like me :P

So basically it's an uni directional or Line of Sight Communication system. And you can capture or hack wifi that are very far away from your range just by pointing the setup on the victim site. I can't disclose how many networks I own now :D

Proof Of Concept Screenshots:

Capturing packets through Airodump with WiFi dongle only

Capturing packets through Airodump with WiFi dongle installed in dish antenna, able to find so many distant Wireless signals

If you like my setup please do share it or comment over here.
Happy Hunting !!

Find saved wifi password on Android device

Hi Friends.. I am back with a new hack on Android devices. Few of my friends were asking how to find saved wifi password on Android device. So I decided to tell all of you via my blog. This trick will work on any android device with any version.

Whenever we connect to a wireless network and punch in the password. After successful connection we cant find the field where we have typed in the password because it has been stored on a configuration file. But the best part is that it get saved in a non-encrypted plaintext format.

Prerequisite to extract password : You got to have a rooted android device, as we have to search for a file in Root file system. And a good file explorer like ES Explorer.

Open the file explorer and grant the file explorer root privileges. Navigate back to the root directory "/".
Navigate to --> data > misc > wifi.

Open the file wpa_supplicant.conf in any text editor. And voila you can see all of the passwords of your previously connected networks.

network={
ssid="example wpa-psk network"
key_mgmt=WPA-PSK
proto=WPA
pairwise=TKIP
group=TKIP
psk="secret passphrase"
}

Or you can also download an app from the link below if you don't want to search manually for it. For running this app you must have root privileges.
http://www.mediafire.com/download/8lm66g3w1dbt3cw/wifi.passwords.apk

Do leave comments and likes for this post if it worked!!

How to get serial number of laptop from Command Prompt

Hi friends, recently one of my friend was asking about how to know the serial number of his laptop as his sticker pasted on the back of laptop has been ruined and was not visible from there.

So there is a remedy for that a small command you can run over your command prompt to know the serial number whether its a Dell, Compaq ,Hp or Acer laptop. (Brand doesn't matter only platform matters that it should be Windows)

In the Run windows (Windows + R) type "cmd" and press enter to open the command prompt window.
On cmd window write the following command :
>wmic bios get serialnumber (Press enter)

Voila you will get the serial number of your laptop.
Its a simple way to know your laptop's serial number instead of turning it around while you are working.

WMIC stands for Windows Management Instrumentation Command-line.

Distance Finder and Temperature meter using Arduino microcontroller

Hello friends!! Its been a long long time since I visited my blog now I have decided to write a post on Arduino technology.
So my techie friends who wish to work with electronic components here's a golden chance to understand and start your field of interest from here only.
To learn more about Arduino, please go through Wikipedia's link. http://en.wikipedia.org/wiki/Arduino

Click on the image to get a bigger view of what I have made.

Things you would be requiring before starting off the project:

1. Firstly installing the Arduino IDE for compiling and uploading the code on Arduino chip. Here is the link: http://arduino.cc/en/main/software

2. Arduino UNO R3 - Rs.1395 (http://fabtolab.com/UNO-R3?search=arduino%20uno)

3. 16x2 Character LCD Module - Rs.230 (http://fabtolab.com/16x2-LCD-Module-Green)

4. TMP 36 IC - Rs.85 (http://fabtolab.com/sensors/temperature-sensors/TMP36GT9Z-temperature-sensor)

5. HC-SR04 - Rs.135 (http://fabtolab.com/sensors/distance-proximity-sensors/HC-SR04-ultrasonic)

6. Buzzer - Rs.14 (http://fabtolab.com/5V-piezo-buzzer)

7. Male Headers(used for soldering on LCD) - Rs.6 (http://fabtolab.com/components/miscellaneous-components/male-headers)

8. Breadboard - Rs.125 (http://fabtolab.com/breadboard-850-pts)

9. Rest you require are few male to male cables to connect the components.- Rs.100 (Various sizes available on FabtoLab.com)

Plus Shipping - Rs.55

This project has costed me around 2100 bucks. But seriously spending this much amount on it is worthy.

Interfacing diagram of HC_SR04 which is the ultrasonic sensor used to find distance by sending and receiving ultrasonic waves and calculating their time. This is a high precision device with the precision of 0.3 cm and its range is: 2cm – 450 cm
Personally I have tested this device till 350 cms. If anybody has tested with more range ,please drop me a mail.

Interfacing diagram of LCD with Arduino. It can display 16 characters per line and there are 2 such lines.

Below is the LCD pinout for deep understanding the interfacing.

Interfacing diagram of TMP36 module

1st pin : 5V
2nd pin : A0 on arduino
3rd pin : GND on arduino

Interfacing diagram of Buzzer: It is optional for producing beep sound if distance becomes less than 20 cms.

+ve of buzzer - 9 Digital pin which is PWM pin
-ve of buzzer - GND of arduino

NOTE: This device can only work as temperature monitor or distance finder at once.
Here is the arduino code available for download.

Temperature Monitor: http://www.mediafire.com/view/1uo3x9gr7ge7huz/LCD_Thermometer.ino

Distance Finder: http://www.mediafire.com/view/93wmy47pkr5pkbc/LCD_distance_finder.ino

Likes and Shares are always appreciated. Keep reading for more Arduino projects.