Advertisements

Migrating Namenode from one host to another host

Namenode is the heart of the hadoop cluster. So namenode will be installed in a good quality machine compared to the other nodes. If we want to migrate namenode from one node to another node, the following steps are required. This is a rare scenario.

Manual Approach

Method 1: (By migrating the harddrive)

  • Stop all the running jobs in the cluster
  • Enter into Namenode Safe
    • hdfs dfsadmin -safemode enter
  • Execute the following command to save the currrent namespace to the storage directories and reset editlogs..
    • hdfs dfsadmin -saveNamespace
  • Stop the entire cluster
  • Remove the hard disk from the old namenode host and attach it to the new namenode host
  • Release the ipaddress from the old namenode host and assign it to the new namenode host
  • Start the new namenode (DO NOT PERFORM FORMAT)
  • Start all the services

Method 2: (New Harddrive)

  • Stop all the running jobs in the cluster
  • Enter into Namenode Safe
    • hdfs dfsadmin -safemode enter
  • Execute the following command to save the currrent namespace to the storage directories and reset editlogs..
    • hdfs dfsadmin -saveNamespace
  • Stop the entire cluster
  • Login to the namenode host.
  • Navigate to the namenode storage directories.
  • Copy the namenode metadata. Always better to keep this as a compressed file. Notedown the folder and file permissions & ownership.
  • Take a back up of the configuration files.
  • Install namenode of the same version as that of the existing system to the new machine.
  • Ensure that the ipaddress of the old host is taken and assigned to the new host.
  • Copy the configuration files and metadata to the new namenode host
  • Create namenode storage directory structure in the new host.
  • Maintain the same folder permissions and ownership in the new host also.
  • If there are any changes in namenode directory structure, make the corresponding changes in config files.
  • Incase of a kerberised cluster, create appropriate principles for the new host and place the proper keytabs.
  • Start the new namenode. (DO NOT PERFORM FORMAT)
  • Start the remaining services.
  • Test the working of the cluster by executing file system operations as well as MR operations.

Automated Approach in a cluster managed using Cloudera Manager (CM above 5.4)

If you are using cloudera manager 5.4 or above, there is a new feature known as Namenode Role Migration that helps us to migrate namenode from one host to another. This requires HDFS HA to be enabled.

Advertisements

How to Change the Hostname of Ubuntu Server ?

We can change the hostname of a machine by several ways. I am mentioning one way to change the hostname of Ubuntu server.

  1. Open the terminal or Login with putty.exe as root user (if you are working remotely)
  2. Goto /etc/
  3. Type nano hostname
  4. Change the HOSTNAME to your preferred machine name
  5. Press Cntrl+X
  6. Save the configuration by pressing Y
  7. Log off or reboot

Note: Add the new hostname and ipaddress in the /etc/hosts file also

How to Change the Hostname of CentOS or RedHat Linux systems?

We can change the hostname of a machine by several ways. I am mentioning two ways to change the hostname.

Method 1 :- Editting /etc/sysconfig/network file

  1. Open the terminal or Login with putty.exe as root user (if you are working remotely)
  2. Goto /etc/sysconfig/
  3. Type nano network
  4. Change the HOSTNAME to your preferred machine name
  5. Press Cntrl+X
  6. Save the configuration by pressing Y
  7. Log off or reboot

Method 2:- Editting  /proc/sys/kernel/hostname file

  1. Open the terminal or Login with putty.exe as root user (if you are working remotely)
  2. Goto /proc/sys/kernel/
  3. Type nano hostname
  4. Change the HOSTNAME to your preferred machine name
  5. Press Cntrl+X
  6. Save the configuration by pressing Y
  7. Close the terminal and login again

In this method, no reboot is required to get the change in effect

Note: Add the ipaddress and new hostname to /etc/hosts file also