Do we need additional encryption on top of HTTPS for a REST API?

This a good topic to understand. The original content is from StackExchange. Sharing it here for everyone’s reference and understanding.

Question

Let’s set up the environment before coming to my question.

  • We have a web application which will be accessible to the user over HTTPS.
  • Mostly only accessible in the intranet ( rarely over the internet )
  • Front-end was developed in Angular
  • Backend was developed in Java-EE

Now I came to an argument with my colleagues over a concern of mine.

The communication between the frontend and backend is completely unencrypted, resulting in the credentials of a user logging in send over the intranet/internet completely clear text. Also they will enter sensitive data which is a secret of the company (ingredients, proportions, etc.).

Adding the fact we integraded LDAP Login possibility to our application this fact seems highly risky to me, possibly resulting in an entry point to gain more informations off the intranet (getting LDAP credentials and accessing more services and machines).

My colleagues said thats nothing to worry about since the communication between client and server is established over HTTPS.

However they couldn’t convince me with only providing that one single argument at all, especially since there are known problems like SSL Stripping, MITM, or even traffic analysis. I know that I should only worry about those if we don’t configure everything correctly, but thats also a concern of mine.

I am on the point that we should encrypt the communication betweeen client and server additionally with something like Jose4J. That would ensure that even in an intranet breach or HTTPS Problem the sensitive data would still be a secret. It would also be more hard to analyse the traffic.

Now I wanted to get an answer which provides more facts and a better conclusion than only “but we are using https”.

Is only using HTTPS for a web application dealing with sensitive data “secure” enough?

Answer

The communication between the Frontend and Backend is completely unencrypted, resulting in the credentials of a user logging in send over the intranet/internet completely clear text.

…the communication between client and server is established over HTTPS.

You are contraditcting yourself. If you are using HTTPS, your data isn’t unencrypted.

…especially since there are known problems like SSL Stripping, MITM, or even traffic analysis.

SSL stripping is solved by HSTS. Use it. A MITM is exactly what TLS protect against, so I fail to see how that is a “known problem”. Not sure what you mean by “traffic analysis”, but implementing your own application level crypto is not going to solve it.

I think you are undervaluing what HTTPS gives you.

Is only using HTTPS for an Web application dealing with sensitive data “secure” enough?

What your question basically boils down to is this: Is transport layer encryption enough, or do I need application layer encryption as well?

For a web app designed to run in the browser, the security value of application layer encryption is basically zero. Why? Because the very code that does the application layer crypto will have to first be transported to the client. If transport layer crypto is broken, that code can be tampered with to the attackers benefit.

And anyway: You do not trust yourself to configure your own TLS. Why should you trust yourself with the much more complex task of setting up safe application layer crypto? I promise, it will be much easier to just read up on TLS and do that right.

Your colleagues provided you with only one argument, because it’s the right argument. Use HTTPS

Reference: https://security.stackexchange.com/questions/178315/do-i-need-additional-encryption-on-top-of-https-for-a-rest-api

How to use password with special character in mount command linux ?

I faced a problem while using mount command with a password containing special characters. The error that I got was “permission denied”. On analyzing the root cause, I figured out that because of the special character, the authentication failed.

The syntax of the mount command is given below.

> mount -t cifs -o username= ,password=[password] [windows-share-dir-path] [dir-in-linux]

Here my password was something like below. It has so many special characters.

> i2_Gw$wF?Xs4zq??K

Solution

The solution is simple. Create a text file and enter the credentials into that file.

I have created a file with the name credentials.ini

username=[username]
password=[password]

Now save this file and use this credentials file in the mount command. The syntax is given below.

mount -t cifs -o credentials=credentials.ini [windows-share-dir-path] [dir-in-linux]

 

Now execute this command and your problem will be solved. !!!!

How to convert or change the data type of columns in Pandas dataframe ?

Changing the datatype of columns in pandas dataframe is very easy. Here I am using stype() function to perform the typecase operation.  Refer to the following example. The type conversion is happening in the line number 10 of the code.

 

You can add as many columns as you want to convert the data type or typecast. For example if you want to typecast the columns emp_id and salary, use the following syntax.

> df = df.astype({‘salary’:‘int’, ’emp_id’:’int’})

 

rpm: /usr/bin/rpmspec: No such file or directory – CentOS RHEL

I have faced an issue while building an rpm in a CentOS machine. The error was rpm: /usr/bin/rpmspec: No such file or directory. To fix this issue we need to install the following package and re-run the build.

For CentOS 7 or RHEL 7 users

> sudo yum install rpm-build

For CentOS 8 or RHEL 8 users

> sudo dnf install rpm-build

 

How to alter or modify the primary key in a mysql table ?

If we have to alter or modify the primary key of a table in an existing mysql table, the following steps will help you.

For a table without primary key

For single primary key

ALTER TABLE table_name ADD PRIMARY KEY(your_primary_key);

For composite primary key

ALTER TABLE table_name ADD PRIMARY KEY(key1, key2, key3);

 

For a table with already existing primary key

For single primary key

ALTER TABLE table_name DROP PRIMARY KEY, ADD PRIMARY KEY(primary_key);

For composite primary key

ALTER TABLE table_name DROP PRIMARY KEY, ADD PRIMARY KEY(key1, key2, key2);

 

Hope this is helpful 🙂

Python program to check the internet speed or bandwidth

Monitoring the internet speed in an office or a data center is a very critical requirement. The following simple program can help you to monitor the internet speed of a network. This will check the upload speed and download speed available in the network.

Note: Do not run this test continuously in a network with limited data package. 

The following program checks the internet speed and stores it an sqlite database. The speed gets monitored every 15 minutes. In this way you will be able to track the speed of the network at various points of time. The program internally uses speedtest python package for determining the speed of the network. This can be extended by storing the data in a proper database.

 

Python program to find the timezone from latitude and longitude ( geo coordinates )

We all know that there are several timezones in the world. While developing applications that are used by the people across the world, we have to consider the users timezone. So depending upon their location, we have to display the parameters or values. I am sharing a simple python code snippet that finds the timezone based on the latitude and longitude.

This is a very simple program. There is a powerful package in python called timezoneinfo. We are using this package for finding the timezone information. This package works with python versions above 3.6. This is the optimal and quick way to find the timezone using geo coordinates.

The following command installs the package

pip install timezonefinder[numba]

Sample Program

 

This package works offline. That means you do not need to be connected to the internet to get this working. This covers the entire earth. In this way  we can find the timezone information with few lines of code. Hope this helps.

How to set url length in Nginx Request (error code: 414, uri too large)

Today I got one annoying error after deploying the new version of the web application in nginx web server. Initially I thought the web app was buggy, but when I inspected the requests and response, I found the following error.

error code: 414, uri too large

On checking more details around this, I found that this issue can be fixed by adjusting few configurations in nginx. The parameter to modify is large_client_header_buffers.

This parameter sets the maximum number and size of buffers used for reading large client request header. A request line cannot exceed the size of one buffer, or the 414 (Request-URI Too Large) error is returned to the client. A request header field cannot exceed the size of one buffer as well, or the 400 (Bad Request) error is returned to the client. Buffers are allocated only on demand. By default, the buffer size is equal to 8K bytes. If after the end of request processing a connection is transitioned into the keep-alive state, these buffers are released.

Syntax : large_client_header_buffers number size ;

The default value is 4 and the size is 8 KB. You can increase this value to a higher value to fix this issue.

large_client_header_buffers 16 128k;

If you are facing issues even after making these changes, then add the following configuration to the server block in nginx.

fastcgi_buffers 8 16k;
fastcgi_buffer_size 32k;

client_max_body_size 24M;
client_body_buffer_size 128k;

client_header_buffer_size 5120k;
large_client_header_buffers 16 5120k;

 

Hope this helps 🙂  Please comment if you are facing any issues.

 

Production deployment of a Python Web Service (Flask / Tornado Application)

Python Flask and Tornado are two of the most popular frameworks in python for developing RESTful services.

Do you know how to develop and deploy a production grade python application. ?

A sample python flask service is given below. This is a sample flask web service. This has only one endpoint (/requestme) at is a GET method. (sample_flask.py). I am not focusing on the coding standards. My goal is to show you the production implementation of a python application.

We can run this program in the command line by executing the following command.

> python sample_flask.py

The service will be up and running in port 9090. You will be able to make requests to the application by using the URL http://ipaddress:9090/requestme.

How many requests will this python web service can handle ? 

10 or 20 or 100 ?? … Any guess ??

Definitely this is not going to handle too many requests. This is good for development trials and experimental purpose. But we cannot deploy something like this in production environment.

How to scale python applications  ?

Refer to the below diagram. The diagram has multiple instances of flask applications with Gunicorn WSGI proxied and load balanced through Nginx web server.

haproxy_python

Production Deployment of Python Flask Application

Sample Nginx configuration that implements the reverse proxy and load balancing is given below. 

This is a sample configuration and this does not have the advanced parameters.

server {
listen 80;
server_name myserverdomain

location / {
proxy_pass http://upstream_backend/requestme;
  }
}

upstream backend {
server gunicornapplication1:8080;
server gunicornapplication2:8080;

}

 

The upstream section routes the requests to the two gunicorn backends and the requests are routed in round robin manner. We can add as many backend servers as we need based on the load.

How to run the python applications with gunicorn ?

First lets install gunicorn

> pip install gunicorn

Now it is simple, run the following command.

> gunicorn -w 4 app:app

Now the our application will run with 4 workers. Each worker is a separate process and will be able to handle requests. The gunicorn will take care of handling the requests between each of the workers.

We can start multiple gunicorn instances like this and keep it behind the nginx. This is the way to scale our python applications.

Hope this helps 🙂 

“The Zen of Python”, by Tim Peters

Every Python Developer should try and read these statements periodically. In the python interpreter, type the following statement.

> import this

You will see the following response. Read it and refresh.

python_zen