Amazon AWS fundamentals

I will be sharing the basic concepts around Amazon web service. I believe this will be helpful for the new users who are usually overwhelmed by the array of service choices offered in AWS. We are discussing basic understanding in AWS. Detailed information can be found on their help content.

Storage:

EBS ( Elastic Block Storage):

Disks used for Virtual machines in the AWS cloud. Typical servers has computing power, storage and RAM.

However, in AWS compute and storage are separate.

We need to connect EBS(Storage) to EC2(Compute). We may have many EBS storage in your AWS account. Only, when you connect your EBS to the running instance makes a proper virtual machine.

Disk Performance:

Another key concept in AWS storage is performance of a disk. It is measured in Input Output per second (IOPS)

There are three types of EBS:

  • 1.General purpose SSD ( 3 IOPS;)
  • 2.Provisioned IOPS ( 40 to 200 IOPS)
  • 3.Magnetic (4000 IOPS)

A database or high performance required application has to be hosted in Magnetic EBS volume which delivers better read write speed.

Snapshot

A snapshot can be used as a backup of the EBS volume of an instance. This can be an incremental backup of the data stored in EBS. This can be used to provision a cloned instance.

S3 ( Simple storage service):

S3 is also related to storage in AWS world. However, It is not equivalent to EBS in terms of storage. It allows developer to store objects in the external storage. Static files such as media or a static web site. We have used to store user’s profile picture in S3 and display those pictures in a direct link.

S3 uses bucket concept to store object. Each bucket can be accessible using unique URL. These stored objects can be versioned. Access to these objects is private by default. It can manage complex access control.

Networking:

Security Groups:

Security group is equivalent to configuring the firewall in the server. Security group can be applied to as many as instance as possible. An instance can have many security groups. System admins keep security group per need and they apply according to the needs required. For example, Web server has to serve the content on port 80. So port 80 is opened in the Webserver-security group.

  1. Inbound : E.g for allowing SSH to the server
  2. Outbound : E.g. for exposing the database port to the outside world

When working with changes in security group, it is instant.

Elastic IPs

In AWS, by default dynamic ip address is assigned. For web server hosting and running website, static ip address is required. This can be solved by Elastic IP. Elastic IP can be attached to the running instance and we can map this static IP in MX record in DNS.

How to add volume in the EC2 instance?

  1. Create add volume under EC2 console.
  2. Attach the volume to the running instances.
  3. SSH to the running instance
  4. type <pre>dmsg</pre> command to see the unknown partition being added.
[584679.841428] blkfront: xvdf: barrier or flush: disabled; persistent grants: disabled; indirect descriptors: enabled;
[584679.845533] xvdf: unknown partition table

This means that partition is available for EC2 instance.

5. sudo fdisk /dev/xvdf

Type ‘p’ to see that there is no partition. ( just to confirm)

Empty table will be shown if it is empty.

Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1): 2
First sector (2048-4194303, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-4194303, default 4194303):
Using default value 4194303
Command (m for help): p
Disk /dev/xvdf: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders, total 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x213ea459
Device Boot Start End Blocks Id System
/dev/xvdf2 2048 4194303 2096128 83 Linux
Command (m for help): w
The partition table has been altered!

6. Next step is to create the file system on it.

sudo mkfs -t ext4 /dev/xvdf2

Here I am passing the -t option for filesystem and my partition

mke2fs 1.42.9 (4-Feb-2014)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
131072 inodes, 524032 blocks
26201 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=536870912
16 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912

Allocating group tables: done
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

7. Mounting the file system. sudo mkdir /ebsvolume

sudo mount /dev/xvdf2 /ebs_volume

8. Are we done? No. We need to mount this for every boot.

sudo /etc/fstab

Add the following line in the above file to make the mounting at every boot.

/dev/xvdf2	/ebs_volume 	ext4	defaults,discard	0	0
  1. sudo reboot
  2. Login in to machine again and enjoy cd’ing in to your volume.

How to add swap space in AWS instances?

Follow 1–5 steps in how to add disk in AWS instances.

  1. sudo swap /dev/xvdg1
  2. Run free command to see the available memory on the server. free
total       used       free     shared    buffers     cached
Mem: 1016324 171828 844496 372 9712 88128
-/+ buffers/cache: 73988 942336
Swap: 0 0 0

This shows swap is 0 and yet to add.

3. Activate the swap with following command. <code> sudo swap on /dev/xvdg1</code>

4. Edit the /etc/fstab file to make this swap available on every boot. Add the following line.

/dev/xvdg1	swap 	swap	defaults	0	0

5. Reboot and Run the free command again to see the swap space activated. Happy swapping!

How to change the timezone in the AWS instances?

  1. All the timezone related information are available in the following directory.
  2. cd /usr/share/zoneinfo/Greenwich
  3. Edit the system config to change the timezone.
  4. sudo vi /etc/sysconfig/clock
  5. ZONE=“Greenwich”
  6. UTC=false
  7. ls -sf /usr/share/zoneinfo/Greenwich /etc/localtime
Show your support

Clapping shows how much you appreciated dinesh’s story.