After a recent update in CentOS 7 and RHEL 7, the cassandra daemon stopped working. I was getting the following error while trying to start the cassandra using systemd. Similar installations were working fine in the recent past and suddenly it stopped working.
Mar 20 13:22:34 localhost systemd[1]: New main PID 72596 does not belong to service, and PID file is not owned by root. Refusing.
Mar 20 13:22:34 localhost systemd[1]: New main PID 72596 does not belong to service, and PID file is not owned by root. Refusing.
Mar 20 13:22:34 localhost systemd[1]: Failed to start LSB: distributed storage system for structured data.
Root Cause
Solution
Update the /etc/rc.d/init.d/cassandra file. Either make the following patch manually or replace the entire file with the file that I provided below.
Option: 1 – Manual Patch
Open /etc/rc.d/init.d/cassandra file and make the modifications as per comments in the below script. The below snippet is not the complete script, it is only the portion which needs update. Do not copy paste and replace the file completely with this
case "$1" in start) # Cassandra startup echo -n "Starting Cassandra: " [ -d `dirname "$pid_file"` ] || \ install -m 755 -o $CASSANDRA_OWNR -g $CASSANDRA_OWNR -d `dirname $pid_file` # -Commented for fix #su $CASSANDRA_OWNR -c "$CASSANDRA_PROG -p $pid_file" > $log_file 2>&1 # +Added for fix runuser -u $CASSANDRA_OWNR -- $CASSANDRA_PROG -p $pid_file > $log_file 2>&1 retval=$? # +Added new chown root.root $pid_file [ $retval -eq 0 ] && touch $lock_file echo "OK" ;; stop) # Cassandra shutdown echo -n "Shutdown Cassandra: " # -Commented as per the fix #su $CASSANDRA_OWNR -c "kill `cat $pid_file`" # +Added for fixing the issue runuser -u $CASSANDRA_OWNR -- kill `cat $pid_file` retval=$? [ $retval -eq 0 ] && rm -f $lock_file for t in `seq 40`; do status -p $pid_file cassandra > /dev/null 2>&1 retval=$? if [ $retval -eq 3 ]; then echo "OK" exit 0 else sleep 0.5 fi; done
Option:2 – Replace the file
Replace the /etc/rc.d/init.d/cassandra file with the file present in the following link. This patch was made as per the JIRA issue CASSANDRA-15273.
Steps to replace the file are given below.
mv /etc/rc.d/init.d/cassandra /etc/rc.d/init.d/cassandra.old curl -o /etc/rc.d/init.d/cassandra https://gist.githubusercontent.com/amalgjose/74cf98e0110c27b6124b0adbb698d372/raw/c08ce3481e9cb0601e79e127c78a65bf82080e5f/cassandra systemctl daemon-reload systemctl restart cassandra
The Gist code is pasted below.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| # | |
| # /etc/init.d/cassandra | |
| # | |
| # Startup script for Cassandra | |
| # | |
| # chkconfig: 2345 80 20 | |
| # description: Starts and stops Cassandra | |
| # pidfile: /var/run/cassandra/cassandra.pid | |
| ### BEGIN INIT INFO | |
| # Provides: cassandra | |
| # Required-Start: $remote_fs $network $named $time | |
| # Required-Stop: $remote_fs $network $named $time | |
| # Should-Start: ntp mdadm | |
| # Should-Stop: ntp mdadm | |
| # Default-Start: 2 3 4 5 | |
| # Default-Stop: 0 1 6 | |
| # Short-Description: distributed storage system for structured data | |
| # Description: Cassandra is a distributed (peer-to-peer) system for | |
| # the management and storage of structured data. | |
| ### END INIT INFO | |
| . /etc/rc.d/init.d/functions | |
| export CASSANDRA_HOME=/usr/share/cassandra | |
| export CASSANDRA_CONF=/etc/cassandra/conf | |
| export CASSANDRA_INCLUDE=$CASSANDRA_HOME/cassandra.in.sh | |
| export CASSANDRA_OWNR=cassandra | |
| NAME="cassandra" | |
| log_file=/var/log/cassandra/cassandra.log | |
| pid_file=/var/run/cassandra/cassandra.pid | |
| lock_file=/var/lock/subsys/$NAME | |
| CASSANDRA_PROG=/usr/sbin/cassandra | |
| # The first existing directory is used for JAVA_HOME if needed. | |
| JVM_SEARCH_DIRS="/usr/lib/jvm/jre /usr/lib/jvm/jre-1.7.* /usr/lib/jvm/java-1.7.*/jre" | |
| # Read configuration variable file if it is present | |
| [ -r /etc/default/$NAME ] && . /etc/default/$NAME | |
| # If JAVA_HOME has not been set, try to determine it. | |
| if [ -z "$JAVA_HOME" ]; then | |
| # If java is in PATH, use a JAVA_HOME that corresponds to that. This is | |
| # both consistent with how the upstream startup script works, and with | |
| # the use of alternatives to set a system JVM (as is done on Debian and | |
| # Red Hat derivatives). | |
| java="`/usr/bin/which java 2>/dev/null`" | |
| if [ -n "$java" ]; then | |
| java=`readlink –canonicalize "$java"` | |
| JAVA_HOME=`dirname "\`dirname \$java\`"` | |
| else | |
| # No JAVA_HOME set and no java found in PATH; search for a JVM. | |
| for jdir in $JVM_SEARCH_DIRS; do | |
| if [ -x "$jdir/bin/java" ]; then | |
| JAVA_HOME="$jdir" | |
| break | |
| fi | |
| done | |
| # if JAVA_HOME is still empty here, punt. | |
| fi | |
| fi | |
| JAVA="$JAVA_HOME/bin/java" | |
| export JAVA_HOME JAVA | |
| case "$1" in | |
| start) | |
| # Cassandra startup | |
| echo -n "Starting Cassandra: " | |
| [ -d `dirname "$pid_file"` ] || \ | |
| install -m 755 -o $CASSANDRA_OWNR -g $CASSANDRA_OWNR -d `dirname $pid_file` | |
| #Commented as per CVSS Fix | |
| #su $CASSANDRA_OWNR -c "$CASSANDRA_PROG -p $pid_file" > $log_file 2>&1 | |
| #Added new | |
| runuser -u $CASSANDRA_OWNR — $CASSANDRA_PROG -p $pid_file > $log_file 2>&1 | |
| retval=$? | |
| #Added new | |
| chown root.root $pid_file | |
| [ $retval -eq 0 ] && touch $lock_file | |
| echo "OK" | |
| ;; | |
| stop) | |
| # Cassandra shutdown | |
| echo -n "Shutdown Cassandra: " | |
| #Commented and added new line as per CVSS fix | |
| #su $CASSANDRA_OWNR -c "kill `cat $pid_file`" | |
| runuser -u $CASSANDRA_OWNR — kill `cat $pid_file` | |
| retval=$? | |
| [ $retval -eq 0 ] && rm -f $lock_file | |
| for t in `seq 40`; do | |
| status -p $pid_file cassandra > /dev/null 2>&1 | |
| retval=$? | |
| if [ $retval -eq 3 ]; then | |
| echo "OK" | |
| exit 0 | |
| else | |
| sleep 0.5 | |
| fi; | |
| done | |
| # Adding a sleep here to give jmx time to wind down (CASSANDRA-4483). Not ideal… | |
| # Adam Holmberg suggests this, but that would break if the jmx port is changed | |
| # for t in `seq 40`; do netstat -tnlp | grep "0.0.0.0:7199" > /dev/null 2>&1 && sleep 0.1 || break; done | |
| sleep 5 | |
| status -p $pid_file cassandra > /dev/null 2>&1 | |
| retval=$? | |
| if [ $retval -eq 3 ]; then | |
| echo "OK" | |
| else | |
| echo "ERROR: could not stop $NAME" | |
| exit 1 | |
| fi | |
| ;; | |
| reload|restart) | |
| $0 stop | |
| $0 start | |
| ;; | |
| status) | |
| status -p $pid_file cassandra | |
| exit $? | |
| ;; | |
| *) | |
| echo "Usage: `basename $0` start|stop|status|restart|reload" | |
| exit 1 | |
| esac | |
| exit 0 |
This solution helped me. I hope this will help someone else also.
All About Tech 
Thanks, It worked fine, I just changed the pid to cassandra home dir.
Thanks it worked for me