Enabling LDAP authentication in Apache HTTPD server

Apache httpd server is a widely used open source webserver. By default, the applications deployed in this server will be open to the network and can be accessible without any authentication.

If we want to secure it using some credentials, what we will do .?

Apache httpd server provides several ways to add authentication.

Here I am explaining a basic configuration that enables ldap authentication with apache httpd server. Through this, we can integrate the applications deployed in the apache server with enterprise ldap. We can integrate apache server with LDAP in two steps. These steps are tested with apache httpd version 2.2

Step 1:

Open httpd.conf file and check for the below lines. If it is already present, we are good to go, else add these lines.

LoadModule ldap_module modules/mod_ldap.so
LoadModule authnz_ldap_module modules/mod_authnz_ldap.so

Step 2:

Add the following configuration at the end of the httpd.conf file

<Directory /var/www/html>
AuthType Basic
AuthName "Web Site: Login with user id"
AuthBasicProvider ldap
AuthzLDAPAuthoritative off
AuthLDAPURL ldap://ldap.myserver.com:389/ou=people,dc=unix,dc=myserver,dc=com
Require valid-user


Now this will allow all the users present in the LDAP directory to access your application.
Now restart the httpd server and try using it. You will be asked for credentials while accessing the webpages 🙂

Notification on completion of Mapreduce jobs

Heavy mapreduce jobs may run for several hours. There can be several jobs and checking the status of mapreduce jobs manually will be a boring task. I don’t like this  J. If we try to manage java programs using a script, it will not be a clean approach. Using scripts for managing java programs is bad approach. I consider these kind of designs as worst designs.

My requirement was to get notification on completion of mapreduce jobs. These are some critical mapreduce jobs and I don’t want to frequently check the status and wait for its completion.

Hadoop is providing a useful configuration to solve my problem. It is very easy to achieve this solution. Just a few lines of code will help us. Add these three lines to the Driver class

conf.set("job.end.notification.url", "http://myserverhost:8888/notification/jobId=$jobId?status=$jobStatus");
conf.setInt("job.end.retry.attempts", 3);
conf.setInt("job.end.retry.interval", 1000);

By setting these properties, hadoop sends an http request on completion of the job. We need a small piece of code for creating a webservice that accepts this http request and send email. For creating the webservice and email utility I used python language because of simplicity. Once the mapreduce job completes, it sends an http request to the URL mentioned by the configuration job.end.notification.url. The variables jobId and jobStatus will be replaced with the actual values. Once a request comes to the webservice, it will parse the arguments and call the email sending module. This is a very simple example. Instead of email, we can make different kind of notifications such as sms, phone call or triggering some other application etc. The property job.end.notification.url  is very helpful in tracking asynchronous mapreduce jobs. We can trigger another action also using this trigger. This is a clean approach because we are not running any other script to track the status of the job. The job itself is providing the status. We are using the python program for just collecting the status and making notifications using the status.

The python code for the webservice and email notification are given below.