How to import VM to AWS

One of the coolest features I like about AWS is it not only gives you the powerful images through AMI but also allows you to import your VM images running in your data center as well. In this, I would like to show you how simple it is to import the VM image into the AWS

The prerequisites for VM import are

For S3 Bucket I have created  “my-vm-imports” bucket

Creating IAM Role

You cannot create using the AWS management console. You have to follow the aws- only

  1. create a trust policy trust-policy.json
{

   "Version": "2012-10-17",

   "Statement": [

      {

         "Effect": "Allow",

         "Principal": { "Service": "vmie.amazonaws.com" },

         "Action": "sts:AssumeRole",

         "Condition": {

            "StringEquals":{

               "sts:Externalid": "vmimport"

            }

         }

      }

   ]

}

2. Using aws command line create a role vmimport

aws iam create-role --role-name vmimport --assume-role-policy-document file://trust-policy.json

3. Create a file named role-policy.json with the following policy

{

   "Version": "2012-10-17",

   "Statement": [

      {

         "Effect": "Allow",

         "Action": [

            "s3:ListBucket",

            "s3:GetBucketLocation",

            "s3:FullAccess"

         ],

         "Resource": [

            "arn:aws:s3:::my-vm-imports"

         ]

      },

      {

         "Effect": "Allow",

         "Action": [

            "s3:GetObject"

         ],

         "Resource": [

            "arn:aws:s3:::my-vm-imports/*"

         ]

      },

      {

         "Effect": "Allow",

         "Action":[

            "ec2:ModifySnapshotAttribute",

            "ec2:CopySnapshot",

            "ec2:RegisterImage",

            "ec2:Describe*",

            "ec2:FullAccess"

         ],

         "Resource": "*"

      }

   ]

}


4. Use the following command “put-role-policy” to the role we created before.

aws iam put-role-policy --role-name vmimport --policy-name vmimport --policy-document file://role-policy.json

Next steps :

  1. Upload the VM image to S3
aws s3 cp file_path s3://my-vm-imports

2. Create a container file which contains the s3 bucket name, format, description and key name in the s3 bucket. Save this file as JSON

[

  {

    "Description": “My VM",

    "Format": "ova",

    "UserBucket": {

        "S3Bucket": “my-vm-imports",

        "S3Key": "my-vm-imports/myVm.ova"

    }

}]

Note: Only OVA,VMDK image formats are supported in AWS

4. Finally, import the image from S3 with import-image command. After that, your image(AMI) will be ready for use

aws ec2 import-image —description “Linux or Window VM” —-disk-containers file://container.json

Thanks for Reading.

Best Regards

Naveen

Nagios Vs CloudWatch AWS

Monitoring is certainly an important thing in maintaining consistent infrastructure in cloud or on-premise network infrastructure. AWS CloudWatch is a native tool to monitor the AWS Services.CloudWatch can monitor AWS resources like EC2, DynamoDB, Lambda, RDS, and Custom Metrics generated by your application or service. We can use CloudWatch to have automated monitoring by use of ALARMS. With the use of ALARM’s in CloudWatch, you can send alerts to your team whenever the ALARM breaches.

On EC2 CloudWatch provides system-wide metrics like CPU Utilization, Brust balance for T2 type instances, Available Memory( with use of Custom Metrics) and it also provides metrics for EBS Volumes such as BytesRead, BytesWrite, VolumeQueueLength i.e Number of reading, writes operations waiting to be completed in a specific duration of time. By using these metrics we can analyze the performance of the applications and audit the logs for security and compliance.

There is no shortage of monitoring tools besides CloudWatch. One such popular tool is Nagios, it has its advantage of being an early player in monitoring the IT infrastructure. Nagios is trusted monitoring tool for enterprises who are working with on-premise networks.It can monitor the network, servers, and applications as well.

Nagios extends its usage to monitoring the Cloud Computing resources too. The following described highlights can be used for deciding which monitoring tool to be used based on your usage.

  • ALARMS:

AWS CloudWatch and Nagios both have capabilities of monitoring the deployment of Computing resources in AWS. When it comes to monitoring the metrics of other AWS Resources such as RDS, DynamoDB or other managed services, CloudWatch unsurprisingly leads over Nagios. CloudWatch is not interoperable with third-party messaging tools like slack. Instead, ALARM’s in CloudWatch uses SNS to publish Email Alerts. In this Nagios much matured with its wide community plugins.

It is to be noted that Nagios can not monitor the IOPS of EBS and Burst Balance of EC2 resources. These metrics provide good reasons to find out the application performance and system level metrics over given period of time.

  1.     Resource Metrics:

If comes to AWS Cloud resources, CloudWatch provides many detailed metrics for every resource in AWS. CloudWatch can even store these metrics over a period of 4 weeks. With these stored metrics you can analyze the performance of your application servers based on the generated load. CloudWatch is on top of Nagios in storing the Metrics for Cloud Services.

CloudWatch provides a rich set of graphics for all AWS resource metrics. 

3.Events

Events are set of rules that will trigger the other AWS resources like calling lambda, creating an Ec2 instance etc. This feature quite useful in the cloud to automate the things. Events in CloudWatch can also be used as Cron job as to trigger for every 5 minutes a lambda function or call some other service. Nagios lacks these kinds of the feature.

  4.Ease of Use

Nagios is a centralized monitoring system provides support for monitoring the network, storage and computing devices. Being a legacy monitoring system it is a bit hard to understand the things in Nagios.You need to have a learning curve in Nagios. CloudWatch is more user-friendly and provides graphs for each metric.

  5.Pricing

Nagios is free open sourced, but when it comes to using enterprise features you have to be prepared to pay from your pocket.Nagios Enterprise features price starts from $1995.

Comparative to Nagios enterprise edition CloudWatch is cost effective. Each Dashboard in CloudWatch cost about $3.00 per month where detailed monitoring cost $3.50 per month

Bottom Line: If you are enterprise and wants to move your existing infrastructure from on-premise to Cloud try with your existing monitoring tools and start using CloudWatch monitoring features and if it satisfies your monitoring needs replace Nagios with CloudWatch.

 

If you need to monitoring any network appliance which most telecom companies use in Cloud, I would strongly suggest using existing Nagios. Because CloudWatch does not provide application metrics as Nagios provides.

Best Regards

Naveen

Hybrid Cloud with CISCO CSR 1000v

Cisco CSR 1000v series is a router software appliance from Cisco. It provides enterprise routing, VPN, Firewall, IP SLA, and more.CSR 1000v can be used to connect multiple VPC across all-region in AWS Cloud and on-premise networks. Thus it can be used avoid managed VPN service from AWS.

In AWS, you can find Cisco CSR 1000v in AWS marketplace which has 30 days free trial to test it out. AWS Marketplace for Cisco. Be aware this is not cheap, it will cost you EC2 Instance charges. All instance types are not supported for CSR 1000v. It supports only m3 and c3 instance family types.

Cisco CSR 1000v Can be used in various network models in cloud like Transit VPC, multi-cloud Network.

Following is the Architecture I have used to connect multiple VPC.

 

The two VPC’s are one in N.Virginia region and other is in Ohio Region. And Each VPC has Internet Gateway and were connected over VPN. On Ohio region, we used AWS managed VPN service to connect VPC in N.Virginia region VPC. And On-Premise Edge Router we used Cisco RV110W small business router. In this Post, I would like to mention the steps to follow to establish VPN over two VPC’s spread in two different regions in AWS.

Steps to create VPC’s in two regions:

  1. Create VPC in N.Virginia Region with CIDR 10.0.0.0/16 and attach Internet Gateway to it. you can do it from CLI or through the management console.
    aws ec2 create-vpc --cidr-block 10.0.0.0/16 --region us-east-1
    
    Output:
    {
                "Vpc": {
                    "VpcId": "vpc-848344fd",
                    "InstanceTenancy": "dedicated",
                    "Tags": [],
                    "CidrBlockAssociations": [
                        {
                            "AssociationId": "vpc-cidr-assoc-8c4fb8e7",
                            "CidrBlock": "10.0.0.0/16",
                            "CidrBlockState": {
                                "State": "associated"
                            }
                        }
                    ],
                    "Ipv6CidrBlockAssociationSet": [],
                    "State": "pending",
                    "DhcpOptionsId": "dopt-38f7a057",
                    "CidrBlock": "10.0.0.0/16",
                    "IsDefault": false
                }
              }
    
    
    aws ec2 create-internet-gateway --region us-east-1
    
    Output:
    
              {
                  "InternetGateway": {
                      "Tags": [],
                      "InternetGatewayId": "igw-c0a643a9",
                      "Attachments": []
                  }
              }
    
    
    
    aws ec2 attach-internet-gateway --gateway-id <<IGW-ID>> --vpc-id <<VPC-ID>> --region us-east-1
  2. Create two subnets in N.Virginia Region VPC, one for CSR 1000v with CIDR 10.0.0.0/24 and another subnet with CIDR 10.0.1.0/24.
    aws ec2 create-subnet --cidr-block 10.0.0.0/24 --vpc-id <<VPC-ID>> --region us-east-1
    
    Output:
    {
                  "Subnet": {
                    "VpcId": "vpc-a01106c2",
                    "AvailableIpAddressCount": 251,
                    "MapPublicIpOnLaunch": false,
                    "DefaultForAz": false,
                    "Ipv6CidrBlockAssociationSet": [],
                    "State": "pending",
                    "AvailabilityZone": "us-east-1a",
                    "SubnetId": "subnet-2c2de375",
                    "CidrBlock": "10.0.0.0/24",
                    "AssignIpv6AddressOnCreation": false
                  }
              }
    
    aws ec2 create-subnet --cidr-block 10.0.1.0/24 --vpc-id <<VPC-ID>> --region us-east-1
    
    Output:
    {
                  "Subnet": {
                    "VpcId": "vpc-a01106c2",
                    "AvailableIpAddressCount": 251,
                    "MapPublicIpOnLaunch": false,
                    "DefaultForAz": false,
                    "Ipv6CidrBlockAssociationSet": [],
                    "State": "pending",
                    "AvailabilityZone": "us-east-1b",
                    "SubnetId": "subnet-2c2de375",
                    "CidrBlock": "10.0.1.0/24",
                    "AssignIpv6AddressOnCreation": false
                  }
              }
  3. Create Route Table in N.Virginia VPC which will have the default route to Internet Gateway.And associate CSR subnet to it.

4. Launch the CSR 1000v from AWS MarketPlace with the one-click launch. Link To AWS Marketplace, you can ssh into the CSR 1000v instance using ec2-user.Attach Elastic IP to the CSR instance which will act as Customer Gateway in N.Virginia Region VPC. In later steps, we will configure the router to add Static routes to other subnets in VPC and setting BGP to propagate routes over VPN Connection with other VPC.

5. In a similar fashion create VPC in AWS Ohio region with CIDR 10.1.0.0/16 And create two subnets with CIDR 10.1.0.0/24 and 10.1.1.0/24

Steps to Create VPN connection in AWS Ohio VPC

  1. Create Customer Gateway. Open VPC management console at console.aws.amazon.com. In navigation pane choose Customer Gateway and then create new Customer Gateway. Enter Name, Routing type as Dynamic and EIP of the CSR 1000v instance in N.Viriginia Region VPC. ASN number is 16-bit and must be in the range of 64512 to 65534.
  2. Create VPG and attach to the VPC.In the Navigation Pane choose Virtual Private Gateway and create VPG.
  3.  Now Create VPN connection. In Navigation Pane Choose VPN Connection, Create New VPN Connection. Enter the Name, VPG and Customer Gateway which we have created previously, select routing type as Dynamic and create VPN connection.

It will take few minutes to create VPN connection. When it is ready to download the configuration for Cisco CSR from the drop-down menu.

Steps to establish VPN Connection on CSR 1000v

  1. Add static routes of other subnets in VPC(N.Virginia) to CSR 1000v. Every subnet in AWS has a virtual router with IP address of Subnet CIDR +1. As CSR router will be in Subnet 10.0.0.0/24  the virtual router IP address will be 10.0.0.1. The Virtual Router on each subnet has a route to other all subnets in the VPC.
    >Configure terminal
    #ip route 10.0.1.0 255.255.255.0 10.0.0.1
  2. Configure BGP. Choose the ASN number which you gave while creating Customer Gateway in Ohio VPC. Above we gave 64512
    > Configure terminal
    (config)#router bgp 64512
    (Config-router)# timers bgp keepalive holdtime
    (Config-router)# bgp log-neighbor-changes
    (Config-router)# end

    This step might not be necessary. But as good practice, I have applied the above configuration before copying the configuration file that is downloaded before.

  3. Apply the Configurations that are downloaded previously when VPN Connections Created. After you have applied those setting on CSR you can see on the management console that both the tunnels of VPN as UP.

Testing to check connectivity between two VPC’s

  1. Launch an instance in subnet1 in Ohio region VPC’s with Public IPv4. SSH into the instance and ping the CSR 1000v instance private IP.
  2. Similarly, you can check connectivity with Ohio Region VPC by pinging the instance in subnet1 in Ohio region VPC with its Private IP.

Troubleshooting :

> Route Propagation must be added to the route table in Ohio Region VPC.

> You must configure CSR 1000v as NAT, so the subnets in N.Virginia region can access the hosts in Ohio region VPC via CSR 1000v. You need to Update the route table with target fo CSR 1000v instance-id after making it as NAT.

> Allow ICMP in Security groups on all instances.

Thanks and Regards

Naveen

Rappelling at Kajeguda

Great adventures are all around us. This weekend on 05-11-2017 i.e Sunday had a wonderful adventure and experience with rappelling at Kajeguda. Met some great people who are passionate about Travel, Adventures, and sports. Credits should be given to HATS(Hyderabad Adventurers & Trekkers Club) Organizers  Kishore, Sai and Hemanth for organizing such beautiful event.

What is Rappelling?

Rappelling, also known as Abseiling is a unique skill used for controlled descent from high vertical faces. Climbers use this method when the slope is too steep and dangerous to descend or want to save time. Well trained instructors in HATS will guide you to fun and safe adventure.

As this is a beginner camp only we rappelled 50-60 Feet Hard Rock Mountain.

HATS Organizers gathered all adventurers at Raidurgam Police Station and we were totally around 25 members. From Raidurgam Police station it is 3 KM drive to Kajeguda. We all started at 6:20 AM and reached the place around 6:40 AM.  And then all the fun started….:)

Sharing some of the pics in this event

Want to Join HATS Club? Please Join Here

 

VPC Design Principles

Virtual Private Cloud(VPC) creation is the first step in building your infrastructure in AWS Cloud. AWS gave the flexibility to create VPC based on RFC4632 . Major Components of VPC : VPC CIDR, Subnets, Route Table, ACL and Security Groups. The VPC creation is a straightforward method just grab a CIDR based on RFC4632  but subnetting the VPC can consider the  following principles.

Creation of Subnets:

Primary reasons to create Subnets

  1. You need hosts to be routed successfully.(Private facing or Public facing)
  2. Want to distribute Workload across multiple AZ’s( Availability Zones) for fault tolerance.
  3. Create Subnets for hosts that require additional layer of  Security using ACL ( Access Control List)

Subnet the network into smaller networks which can be considered as  Public Subnets, Private or VPN only subnets. These networks are supernets and not the actual subnets we create. Then subnet each supernet into smaller networks which you fit your hosts into it.

 

Note* : AWS reserves 5 IPs when you create a subnets. So more subnets you create more ips you will lose. For example for subnet 10.0.0.0/27 following IP’s are resolved

  1. 10.0.0.0 network address
  2. 10.0.0.1 Virtual Router address
  3. 10.0.0.2 DNS address
  4. 10.0.0.3 Reserved by AWS for future use.
  5. 10.0.0.31 Broadcast address

Route Tables

All the hosts within VPC can be routed to other hosts in the VPC using an implicit virtual router . A Default Virtual Router would be created when you create the subnet. For example a subnet with CIDR 10.0.0.0/27 will have Virtual Router with IP 10.0.0.1 ( Subnet CIDR + 1). This Router will utilize the route table entries of the subnet associated with.

Each Subnet should be associated with a Route Table for traffic to flow.If a subnet is not associated to any route table, it will use the default Main Route Table. Route Table can be associated with multiple subnets.

 

  1. Create Route Tables for Subnets that need different Routing requirements(Public facing or Private facing).
  2. Create Route Table for subnets that require more specific routing. For example a subnet may be needed to allow traffic only from a pool of IP address.

Access Control List(ACL)

ACL Provide security at Subnet Level. You can control what traffic to flow in and out of a subnet. ACL are stateful, i.e you have to define both ingress and outgress traffic in the rule list.

You can find more at ACL Overview

Create ACL if you want restrict any traffic to flow to the hosts in the subnets.

Network Address Translator (NAT)

A NAT is used to provide Outbound internet to the hosts inside Private Subnets. Route Tables for Private Subnets has to updated with logical id of NAT to provide Outbound Internet Connectivity to hosts inside private Subnet.

 

Based on the above principles ,a Concrete Example for  Creating VPC in Practice is below

  1. Subnet the VPC CIDR to Public facing or Private facing Subnets.
  2. All Private facing subnets would be associated with a single Route Table, and  ACL. The same would be applied for VPN Subnets and Public Subnets with different Route Tables and ACL
  3. Create a Subnet if more security is needed at subnet level using ACL and associate the subnet to Route Table.

The following figure shows the summary of VPC Design in AWS

vpc network desing

 

This Post has been originally written to CloudTern