Advertisements

How to enable access to a S3 object in a secure way ?

December 5, 2017 1 Comment

By default all the objects in an AWS bucket are private. Sometimes we may come across the requirement to share an object (file) present in s3 to others and don’t want to share aws access credentials.

In these scenarios, AWS provides a simple way to provide access by generating a timebound signed URLs using the access credentials. These URLs will be active only for the specified time period and the access will be revoked after that time period. I came across a similar requirement and the solution is shared below.

Filed under cloud Tagged with , , , , , , , , , , , , ,

Get public and private IP address of an EC2 instance from the command line

July 24, 2017 Leave a comment

Sometimes we may get requirements to identify the public ip address of an ec2 instance from the command line. There are various ways to perform this operation.

  1. Using AWS CLI
  2. Using a simple REST call
aws ec2 describe-instances --instance-ids  --query 'Reservations[*].Instances[*].PublicIpAddress' --output text

If we are on the same EC2 host, we can obtain the details of the EC2 instance using the instance metadata service.

curl http://169.254.169.254/latest/meta-data/


ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
hostname
iam/
instance-action
instance-id
instance-type
local-hostname
local-ipv4
mac
metrics/
network/
placement/
profile
public-hostname
public-ipv4
public-keys/
reservation-id
security-groups
services/

Using this, for getting the private IP, the command will be

curl http://169.254.169.254/latest/meta-data/local-ipv4

For getting the public IP, the command will be

curl http://169.254.169.254/latest/meta-data/public-ipv4

Filed under cloud, Linux World, Tips and Tricks Tagged with , , , , , , , , , , , ,

Python code to list all the running EC2 instances across all regions in an AWS account

March 9, 2017 12 Comments

This code snippet will help you to get the list of all running EC2 instances across all regions in an AWS account. I have used python boto3 package for developing the code. This code will dynamically pick up all the aws ec2 regions. So the code will work perfectly without any modification even if a new region gets added to the AWS.

Note: Only the basic api calls just to list the instance details are mentioned in this program . Proper coding convention is not followed . 🙂

Filed under BigData World, Tips and Tricks Tagged with , , , , , , , , , ,

How to attach a new EBS to an EC2 instance

May 24, 2016 1 Comment

Nowadays majority of us are using some cloud services. Amazon Web Services is one of the popular provider among all the other cloud service providers. Just like we upgrade our harddisk or mounting new drives to physical machines, we can attach new block storages to Amazon EC2 also. Amazon provides a service called EBS (Elastic Block Storage). There are various types of EBS with various speed and cost. Example are magnetic, SSD etc.

Attaching a new EBS to a running EC2 instance is very simple. We can do this programatically as well as using the console. Here I am explaining the basic steps to perform this operation using the console.

  1. Launch an EBS in the same region and same availability zone as that of the EC2 instance
  2. Note down the instance id of the EC2 instance
  3. Attach the EBS to the EC2. This can be done by using the attach option available in the EBS. The EBS will be listed under the Volumes section in EC2 service page of AWS console.
  4. Login to the EC2 instance and switch to the root user
  5. Type lsblk to list all the block devices
  6. Identify the new block device.
  7. Create a new directory to mount the EBS.
  8. Format the newly mounted storage. The command is mkfs -t ext4 /dev/<device-name>
  9. Mount the EBS on the directory. The command is mount /dev/<device-name>  <mount-dir>
  10. Check for the new storage. The command is df -h

 

 

Filed under Linux World Tagged with , , , , , , , ,

Recovering a corrupted EC2 instance

April 5, 2016 Leave a comment

Amazon Web Services is one of the most popular cloud service providers. I am a customer of Amazon. I like the services provided by Amazon very much. Compared to other cloud service providers AWS is simple, secure and advanced. I use EC2 machines for my project related activities as well as my personal experiments. Since I mostly work on open source software, 99.99 % of my EC2 instances are Linux instances. The only way to access these instances is through ssh. I use putty as the ssh client. If something happens to the ssh server, we will not be able to access the server. Sometimes the ssh server crashes due to overload. This can be resolved by rebooting the instance.

Sometimes because of wrong configs in the sshd config file, the ssh server may stop. The ssh server will not start until we make that file proper and restart the service. But for making these changes we have to access the machine.

By default we don’t have direct root login into the machine. We usually login to one user which is a sudo user and using sudo privileges, we access the root. If something happens to the sudoers file or if some wrong entry made in sudoers file, the root access will be revoked.

These are some of the commonly occurred situations where users loose access or super user privilege in the ec2 machine. Most of the users terminate and leave the instance in this situation.

If the instance is an EBS backed instance, we don’t have to terminate and leave the machines in this kind of situations. We can recover these instances. It is simple and can be done in few steps. If the instance is with ephemeral storage, we cannot do anything, because shutting down will clear all the data in the instance.

  1. Start a new instance in the same availability zone as that of the EBS of the broken machine. Micro or nano instance type is fine. If you already have an instance, no need of this instance.
  2. Stop the broken machine. Note down the mount locations
  3. Detach the EBS from the instance.
  4. Attach the EBS to the second EC2 instance (The newly launched one)
  5. Mount the EBS to some directory in the second EC2 instance.
  6. Navigate through the files and directories and make the required changes.
  7. Unmount the EBS
  8. Detach the EBS from the second instance
  9. Attach the EBS to the first instance
  10. Use the same mount location as that of the orginal
  11. Start the instance.

This should fix the problem.

Filed under Tips and Tricks Tagged with , , , , , , , , , , , , , , , ,

Programmatic way to reboot EC2 instances

November 14, 2015 1 Comment

Sometimes we might have to reboot EC2 instances. If the requirement is to restart EC2 instances regularly, we can achieve it by writing a small piece of code. I also came across a similar requirement and a portion of the code I used is given below.

 

Filed under Tips and Tricks Tagged with , , , , , , , , , , , , , ,

← Older posts