Production deployment of a Python Web Service (Flask / Tornado Application)

Python Flask and Tornado are two of the most popular frameworks in python for developing RESTful services.

Do you know how to develop and deploy a production grade python application. ?

A sample python flask service is given below. This is a sample flask web service. This has only one endpoint (/requestme) at is a GET method. (sample_flask.py). I am not focusing on the coding standards. My goal is to show you the production implementation of a python application.

We can run this program in the command line by executing the following command.

> python sample_flask.py

The service will be up and running in port 9090. You will be able to make requests to the application by using the URL http://ipaddress:9090/requestme.

How many requests will this python web service can handle ? 

10 or 20 or 100 ?? … Any guess ??

Definitely this is not going to handle too many requests. This is good for development trials and experimental purpose. But we cannot deploy something like this in production environment.

How to scale python applications  ?

Refer to the below diagram. The diagram has multiple instances of flask applications with Gunicorn WSGI proxied and load balanced through Nginx web server.

haproxy_python

Production Deployment of Python Flask Application

Sample Nginx configuration that implements the reverse proxy and load balancing is given below. 

This is a sample configuration and this does not have the advanced parameters.

server {
listen 80;
server_name myserverdomain

location / {
proxy_pass http://upstream_backend/requestme;
  }
}

upstream backend {
server gunicornapplication1:8080;
server gunicornapplication2:8080;

}

 

The upstream section routes the requests to the two gunicorn backends and the requests are routed in round robin manner. We can add as many backend servers as we need based on the load.

How to run the python applications with gunicorn ?

First lets install gunicorn

> pip install gunicorn

Now it is simple, run the following command.

> gunicorn -w 4 app:app

Now the our application will run with 4 workers. Each worker is a separate process and will be able to handle requests. The gunicorn will take care of handling the requests between each of the workers.

We can start multiple gunicorn instances like this and keep it behind the nginx. This is the way to scale our python applications.

Hope this helps 🙂 

Gunicorn Connection in Use: (‘0.0.0.0’, 8000)

I develop web services using python flask. One of the common error that I see while deploying the application is “Gunicorn Connection in Use: (‘0.0.0.0’, 8000)”.

This means that the port 8000 is busy with some other running process. But when I check the status of the port with the following command, I get empty response. That means there are no active application using the port. Some stale process is making the port busy.

netstat -tulpn | grep 8000

I even tried with the ps command to see any active process, but that also did not help.

ps -aux

If the ps command list the process, we can kill the process directly using the kill command

kill -9 {PID}

In my case I do not have the PID. So the only option to kill these kind of zombie application by using the below command.

sudo fuser -k {PORT}/tcp

In my case, the port number is 8000, so the command will be.

sudo fuser -k 8000/tcp

This trick helped me several times, hope this helps someone else also.