Convert csv to json using pandas

The following sample program explains you on how to read a csv file and convert it into json data. Two programs are explained in this blog post. The first program expects the column names in the csv file and second program does not need column names in the file.

The first program expects the headers in the first line of the csv. In case of missing headers, we have to pass it explicitly in the program.

Sample Input

EMPID,FirstName,LastName,Salary
1001,Amal,Jose,100000
1002,Edward,Joe,100001
1003,Sabitha,Sunny,210000
1004,John,P,50000
1005,Mohammad,S,75000

Here the first line of the csv data is the header

Sample Output

[{"EMPID":1001,"FirstName":"Amal","LastName":"Jose","Salary":100000},{"EMPID":1002,"FirstName":"Edward","LastName":"Joe","Salary":100001},{"EMPID":1003,"FirstName":"Sabitha","LastName":"Sunny","Salary":210000},{"EMPID":1004,"FirstName":"John","LastName":"P","Salary":50000},{"EMPID":1005,"FirstName":"Mohammad","LastName":"S","Salary":75000}]

 

If the csv file contains a header row, then you should explicitly pass header=0 to override the column names. If headers are not present in the csv file, we have to explicitly pass the field names in a list to the argument names. Duplicates in this list are not allowed. A sample implementation is given below.

 

Functions as Objects in Python

Python is very powerful. It is easy to learn. Applications can be developed very quickly using python because of the simplicity.

Everything in python is an object. This includes functions also. Are you aware of the following features of functions in python. I was not aware during my initial few years.

  • Functions can be the elements inside various data structures like lists, dictionaries etc.

Few examples

Function as argument to another function

A Sample program in python to explain the implementation of using function as an argument of another function is given below.

Functions as elements within data structures like list or dict()

A simple implementation of passing list of functions as argument to another function is shared below.

I hope this will help someone. ūüôā

 

Programmatic Data Upload to Amazon S3

S3 is a service provided by Amazon for storing data. The full form is Simple Storage Service. S3 is a very useful service for less price. Data can be uploaded to and downloaded from S3 very easily using some tools as well as program. Here I am explaining  a sample program for uploading file to S3 using a python program.

Files can be uploaded to S3 in two approaches. One is the normal upload and another is the multipart upload. Normal upload sends the file serially and is not suitable for large files. It will take more time. For large files, multipart upload is the best option. It will upload the file by dividing it into chunks and sends it in parallel and collects it in S3.

This program is using the normal approach for sending the files to S3. Here I used the boto library for uploading the files.

Ways to find out the count of unique items in a list using python

Last day I was trying for a method to find the count of unique items in a list. I found two solutions. Thought like it is worth to share here.

Method 1:

This method works only with python versions 2.7 and above. The collections library is not available in python versions less than 2.7.

Method 2:

This is a very simple method using python dictionary. This will work in all versions of python.

Difference between / and // division in python

I have seen people using / and // for division in python. So for checking the difference I checked both of this in my system. I used python 2..7 for testing. I got same output for both the divisions. The reason for the same output is explained below.

Eg: 7 / 2   and 7 // 2  will give the same answer 3 in python 2.x

In Python 3.0, 7 / 2  will return 3.5 and 7 // 2  will return 3. The former is floating point division, and the latter is floor division, sometimes also called integer division.

In Python 2.2 or later in the 2.x line, there is no difference for integers. ie we will get 3 for both the divisions. We can enable the 3.x behavior by doing simple import.

from __future__ import division,

This causes Python 2.x to adopt the behavior of 3.0. The future import should be placed at the top of your program.