Chunking Large CSV file with pandas

When working with a large CSV file, instead of loading the entire file in memory for reading, you can read CSV files as chunks by reading only a part of the file at a time which uses very little memory.

With the pandas read_csv() method by specifying the chunksize parameter, you can divide a large CSV file into chunks:

# chunksize = number of line per chunk
for chunk in pd.read_csv("large_csv_file.csv", chunksize=10000):
	print(chunk)
	print("-----")

The above code will print CSV file content with 10000 lines per iteration.