Do we need additional encryption on top of HTTPS for a REST API?

This a good topic to understand. The original content is from StackExchange. Sharing it here for everyone’s reference and understanding.

Question

Let’s set up the environment before coming to my question.

  • We have a web application which will be accessible to the user over HTTPS.
  • Mostly only accessible in the intranet ( rarely over the internet )
  • Front-end was developed in Angular
  • Backend was developed in Java-EE

Now I came to an argument with my colleagues over a concern of mine.

The communication between the frontend and backend is completely unencrypted, resulting in the credentials of a user logging in send over the intranet/internet completely clear text. Also they will enter sensitive data which is a secret of the company (ingredients, proportions, etc.).

Adding the fact we integraded LDAP Login possibility to our application this fact seems highly risky to me, possibly resulting in an entry point to gain more informations off the intranet (getting LDAP credentials and accessing more services and machines).

My colleagues said thats nothing to worry about since the communication between client and server is established over HTTPS.

However they couldn’t convince me with only providing that one single argument at all, especially since there are known problems like SSL Stripping, MITM, or even traffic analysis. I know that I should only worry about those if we don’t configure everything correctly, but thats also a concern of mine.

I am on the point that we should encrypt the communication betweeen client and server additionally with something like Jose4J. That would ensure that even in an intranet breach or HTTPS Problem the sensitive data would still be a secret. It would also be more hard to analyse the traffic.

Now I wanted to get an answer which provides more facts and a better conclusion than only “but we are using https”.

Is only using HTTPS for a web application dealing with sensitive data “secure” enough?

Answer

The communication between the Frontend and Backend is completely unencrypted, resulting in the credentials of a user logging in send over the intranet/internet completely clear text.

…the communication between client and server is established over HTTPS.

You are contraditcting yourself. If you are using HTTPS, your data isn’t unencrypted.

…especially since there are known problems like SSL Stripping, MITM, or even traffic analysis.

SSL stripping is solved by HSTS. Use it. A MITM is exactly what TLS protect against, so I fail to see how that is a “known problem”. Not sure what you mean by “traffic analysis”, but implementing your own application level crypto is not going to solve it.

I think you are undervaluing what HTTPS gives you.

Is only using HTTPS for an Web application dealing with sensitive data “secure” enough?

What your question basically boils down to is this: Is transport layer encryption enough, or do I need application layer encryption as well?

For a web app designed to run in the browser, the security value of application layer encryption is basically zero. Why? Because the very code that does the application layer crypto will have to first be transported to the client. If transport layer crypto is broken, that code can be tampered with to the attackers benefit.

And anyway: You do not trust yourself to configure your own TLS. Why should you trust yourself with the much more complex task of setting up safe application layer crypto? I promise, it will be much easier to just read up on TLS and do that right.

Your colleagues provided you with only one argument, because it’s the right argument. Use HTTPS

Reference: https://security.stackexchange.com/questions/178315/do-i-need-additional-encryption-on-top-of-https-for-a-rest-api

How to convert or change the data type of columns in Pandas dataframe ?

Changing the datatype of columns in pandas dataframe is very easy. Here I am using stype() function to perform the typecase operation.  Refer to the following example. The type conversion is happening in the line number 10 of the code.

 

You can add as many columns as you want to convert the data type or typecast. For example if you want to typecast the columns emp_id and salary, use the following syntax.

> df = df.astype({‘salary’:‘int’, ’emp_id’:’int’})

 

rpm: /usr/bin/rpmspec: No such file or directory – CentOS RHEL

I have faced an issue while building an rpm in a CentOS machine. The error was rpm: /usr/bin/rpmspec: No such file or directory. To fix this issue we need to install the following package and re-run the build.

For CentOS 7 or RHEL 7 users

> sudo yum install rpm-build

For CentOS 8 or RHEL 8 users

> sudo dnf install rpm-build

 

How to install Python in CentOS 8 / RHEL 8 ?

Python is not available by default in CentOS 8 / RHEL 8. Read the following blog post to install Python and pip in CentOS 8 / RHEL 8.

To Install Python 3 in CentOS 8 or RHEL 8

> sudo dnf install python3

CentOS 8 and RHEL 8 does not have an unversioned python by default. We have to explicitly set it. So simply typing python will give you a “command not found” response.  To verify the installation,  use the following command

> python3 -V

The above command will print the version information. For me it printed Python 3.6.8

To install pip, execute the following command

> sudo dnf install python3-pip

Check the installation

> pip3 –version

If you simply type the command python in the shell, it will give you a response something like below

bash: python: command not found…

To enable the command python, execute the following command.

> sudo alternatives –set python /usr/bin/python3

This will enable command python. Now you can use python without explicitly typing the version.

Note: Follow the below steps only if you need Python 2. If your requirement if Python3, refer the steps described above.

 

To Install Python 2 in CentOS 8 or RHEL 8

> sudo dnf install python2

To install pip, execute the following command

> sudo dnf install python2-pip

Now check the installation

> pip2 –version

To set python2 as the default python across the system, execute the following command.

> sudo alternatives –set python /usr/bin/python2

 

In previous versions of CentOS  and RHEL , there were so much dependency in the system with the unversioned python. Installing Python 3 and Python 2 together creates so much mess in the system. Now in CentOS 8 and RHEL 8, it is very easy.

Hope this blog helps. Please comment below if you face any issues. 🙂

 

 

Python program to check the internet speed or bandwidth

Monitoring the internet speed in an office or a data center is a very critical requirement. The following simple program can help you to monitor the internet speed of a network. This will check the upload speed and download speed available in the network.

Note: Do not run this test continuously in a network with limited data package. 

The following program checks the internet speed and stores it an sqlite database. The speed gets monitored every 15 minutes. In this way you will be able to track the speed of the network at various points of time. The program internally uses speedtest python package for determining the speed of the network. This can be extended by storing the data in a proper database.

 

Python program to find the timezone from latitude and longitude ( geo coordinates )

We all know that there are several timezones in the world. While developing applications that are used by the people across the world, we have to consider the users timezone. So depending upon their location, we have to display the parameters or values. I am sharing a simple python code snippet that finds the timezone based on the latitude and longitude.

This is a very simple program. There is a powerful package in python called timezoneinfo. We are using this package for finding the timezone information. This package works with python versions above 3.6. This is the optimal and quick way to find the timezone using geo coordinates.

The following command installs the package

pip install timezonefinder[numba]

Sample Program

 

This package works offline. That means you do not need to be connected to the internet to get this working. This covers the entire earth. In this way  we can find the timezone information with few lines of code. Hope this helps.

Bubble chart using Python

Bubble chart is one of the powerful and useful chart for representing data with three or four dimensions.

The position of the bubble is determined by the x & y axis values. These are the first two properties.

The size of the bubble can be controlled by the third property.

The colour of the bubble can be controlled by the fourth property.

A Sample program to create a bubble chart using the python library matplotlib is given below.

import matplotlib.pyplot as plot
import numpy as npy

# create some dummy data using numpy random function.
# Bubble charts are used to represent data with three or four dimensions.
# X axis can represent one property, Y can represent another property,
# The bubble size can represent another properly, the color of the bubble can represent another property.

x = npy.random.rand(50)
y = npy.random.rand(50)
z = npy.random.rand(50)
colors = npy.random.rand(50)
# use the scatter function
plot.scatter(x, y, s=z * 1000, c=colors)
plot.show()

Here we are generating some random data using numpy and plotting the bubble chart using matplotlib.

A sample output is given below.

bubblechart

Bubble Chart using Python

 

Production deployment of a Python Web Service (Flask / Tornado Application)

Python Flask and Tornado are two of the most popular frameworks in python for developing RESTful services.

Do you know how to develop and deploy a production grade python application. ?

A sample python flask service is given below. This is a sample flask web service. This has only one endpoint (/requestme) at is a GET method. (sample_flask.py). I am not focusing on the coding standards. My goal is to show you the production implementation of a python application.

We can run this program in the command line by executing the following command.

> python sample_flask.py

The service will be up and running in port 9090. You will be able to make requests to the application by using the URL http://ipaddress:9090/requestme.

How many requests will this python web service can handle ? 

10 or 20 or 100 ?? … Any guess ??

Definitely this is not going to handle too many requests. This is good for development trials and experimental purpose. But we cannot deploy something like this in production environment.

How to scale python applications  ?

Refer to the below diagram. The diagram has multiple instances of flask applications with Gunicorn WSGI proxied and load balanced through Nginx web server.

haproxy_python

Production Deployment of Python Flask Application

Sample Nginx configuration that implements the reverse proxy and load balancing is given below. 

This is a sample configuration and this does not have the advanced parameters.

server {
listen 80;
server_name myserverdomain

location / {
proxy_pass http://upstream_backend/requestme;
  }
}

upstream backend {
server gunicornapplication1:8080;
server gunicornapplication2:8080;

}

 

The upstream section routes the requests to the two gunicorn backends and the requests are routed in round robin manner. We can add as many backend servers as we need based on the load.

How to run the python applications with gunicorn ?

First lets install gunicorn

> pip install gunicorn

Now it is simple, run the following command.

> gunicorn -w 4 app:app

Now the our application will run with 4 workers. Each worker is a separate process and will be able to handle requests. The gunicorn will take care of handling the requests between each of the workers.

We can start multiple gunicorn instances like this and keep it behind the nginx. This is the way to scale our python applications.

Hope this helps 🙂 

“The Zen of Python”, by Tim Peters

Every Python Developer should try and read these statements periodically. In the python interpreter, type the following statement.

> import this

You will see the following response. Read it and refresh.

python_zen

Switch Case Statements in Python

Switch case statements are very popular conditional control statements in almost all programming languages. But surprisingly this is not available in python.

Question: Is there any switch case statements in python ?

Answer: The direct answer is NO

Alternative options for switch case statements in python

Option 1: Using If – elif – else statements. An example is given below.

if case == "case1":
    execute_func_case1()
elif case == "case2":
    execute_func_case2()
elif case == "case3":
    execute_func_case3()
else:
    execute_default_func()

Wow. Excellent.  The above code looks good right ?. It works exactly like switch-case statements, then why need switch-case statements in Python ?

Have you noticed a problem ?. The above if-elif-else conditions are fine as long as we have less number of cases. Imagine the situation with 10 or more elif conditions. Now you got the problem right ?.

Lets try the second option

Option 2: Using List in Python as an alternative to switch case statements

An example is given below.

def add(a, b):
    return a + b

def sub(a, b):
    return a-b

case_funcs = [add, sub]

case_funcs[0](1, 2)
case_funcs[1](1, 2)

 

In the above program, we don’t have to use if-elif-else blocks, instead, we can call using the position or index of the list and call the function. This looks better than the previous option right ?. But what about the default case ?. Also what if someone types an option greater than the size of the list ?. It will throw exception and there is no way to handle default case.

Option 3: Using Dictionary as alternative to switch case statements in python

An example is given below.

def add(a, b):
    return a + b

def sub(a, b):
    return a-b

case_funcs = {'sum':add, 'subtract':sub}

case_funcs['sum'](1,2)

 

Here the implementation is much similar to the switch case statement. We use a key to identify or route to the required case or function. The keys can be anything and are not limited by the indices or positions.

Now lets talk about the drawbacks of the above implementation. The above method will throw KeyError if we pass an unknown key. Also there is no default case statement. How will we handle these problems?

Check the below program

def add(a, b):
    return a + b

def sub(a, b):
    return a-b

def default(a, b):
    return "Default Return"

case_funcs = {'sum':add, 'subtract':sub}

# sum is the key for the add(). 
# default is the default function that gets called for non existent keys
# (1, 2) are the arguments for the function
print(case_funcs.get('sum',default)(1,2))

 

Python dictionary has a get() method that returns the value based on the key. This has one more feature. We can configure a default value for non-existent keys. Wow now we got the solution.

So by using this feature, we can implement the switch-case like feature in python.