How to enable access to a S3 object in a secure way ?

By default all the objects in an AWS bucket are private. Sometimes we may come across the requirement to share an object (file) present in s3 to others and don’t want to share aws access credentials.

In these scenarios, AWS provides a simple way to provide access by generating a timebound signed URLs using the access credentials. These URLs will be active only for the specified time period and the access will be revoked after that time period. I came across a similar requirement and the solution is shared below.

Common dependencies to install PyCrypto package in CentOS/RHEL

The installation of pycrypto package may fail with errors like

“error: no acceptable C compiler found in $PATH”

“RuntimeError: autoconf error”

“fatal error: Python.h: No such file or directory”

” #include “Python.h”
^
compilation terminated.
error: command ‘gcc’ failed with exit status 1″

The solution for this issue is to install the following dependent packages.

yum install gcc

yum install gcc-c++

yum install python-devel

pip install pycrypto

How to check the Java architecture from command line ?

To check the Java architecture whether it is 32 bit or 64 bit, the following commands will be helpful.

Execute the following commands in the command line and check the results

java -d32 -version

java -d64 -version

If any of the above command is giving an error message similar to “Error: This Java instance does not support a xx-bit JVM.” If it is not supporting 32 bit, then we can say it is a 64 bit Java. If it is not supporting 64 bit, we can say it is a 32-bit Java. The screenshot of the same is attached below.

java_screen.PNG

Hope this will help .. 🙂

Programmatic way to identify the status of the namenode in an HA enabled hadoop cluster

In an namenode HA enabled hadoop cluster, one of the namenodes will be active and the other will be standby. If you want to perform some operations on HDFS programmatically, some of the the libraries or packages need the details of active namenode (some of the packages in python need the details of active namenode, they will not support the nameservice). In this case, the easiest way to get the status is to issue a GET request similar to the one given below on each of the namenodes. This will help us to identify the status of each namenode.

GET REQUEST

curl 'http://namenode.1.host:50070/jmx?qry=Hadoop:service=NameNode,name=NameNodeStatus'

SAMPLE OUTPUT

{
"beans" : [ {
"name" : "Hadoop:service=NameNode,name=NameNodeStatus",
"modelerType" : "org.apache.hadoop.hdfs.server.namenode.NameNode",
"State" : "active",
"SecurityEnabled" : false,
"NNRole" : "NameNode",
"HostAndPort" : "namenode.1.host:8020",
"LastHATransitionTime" : 0
} ]
}

(13)Permission denied: access to /index.html denied

I was getting the error “(13)Permission denied: access to /index.html denied” after deploying a static website in Apache webserver.

Solution:

Apache could not access those directories because of the SELinux security settings. Execute the below command

chcon -R -t httpd_sys_content_t  

 

Python code to list all the running EC2 instances across all regions in an AWS account

This code snippet will help you to get the list of all running EC2 instances across all regions in an AWS account. I have used python boto3 package for developing the code. This code will dynamically pick up all the aws ec2 regions. So the code will work perfectly without any modification even if a new region gets added to the AWS.

Note: Only the basic api calls just to list the instance details are mentioned in this program . Proper coding convention is not followed . 🙂

Error: Cannot retrieve metalink for repository: epel. Please verify its path and try again

Problem:

Error: Cannot retrieve metalink for repository: epel. Please verify its path and try again

Faced this error in CentOS 6.3. This issue started after installing the epel-repo

Solution:

You need to update ca-certificates package. Before that disable all the repos with https that are failing.

Here in my case,  epel-repo is failing, so I have to disable only epel repo:

yum --disablerepo=epel -y update  ca-certificates

The above fix helped me to resolve the issue.

Unique ID in Raspberry Pi

Sometimes we might need a unique id from a raspberry PI. We don’t have to worry about generating a unique id for every device. There is a simple way to use an already existing unique number within the device. The serial number of the chip will be good enough to use as a unique key.

The following command will give the details of the cpu. We can find a serial number from this details and can be used as a unique id.

pi@raspberrypi:~ $ cat /proc/cpuinfo
processor   : 0
model name   : ARMv7 Processor rev 4 (v7l)
BogoMIPS   : 38.40
Features   : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer   : 0x41
CPU architecture: 7
CPU variant   : 0x0
CPU part   : 0xd03
CPU revision   : 4

processor   : 1
model name   : ARMv7 Processor rev 4 (v7l)
BogoMIPS   : 38.40
Features   : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer   : 0x41
CPU architecture: 7
CPU variant   : 0x0
CPU part   : 0xd03
CPU revision   : 4

processor   : 2
model name   : ARMv7 Processor rev 4 (v7l)
BogoMIPS   : 38.40
Features   : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer   : 0x41
CPU architecture: 7
CPU variant   : 0x0
CPU part   : 0xd03
CPU revision   : 4

processor   : 3
model name   : ARMv7 Processor rev 4 (v7l)
BogoMIPS   : 38.40
Features   : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer   : 0x41
CPU architecture: 7
CPU variant   : 0x0
CPU part   : 0xd03
CPU revision   : 4

Hardware   : BCM2709
Revision   : a02082
Serial      : 00000000xxxxxxxx

Hope this info is helpful. !!