If you have multiple CSV files with the same structure, you can append or combine them using a short Python script. Suppose you have several files which name starts with datayear. For instance, datayear1980.csv, datayear1981.csv, datayear1982.csv.
import pandas as pd
import glob, os
os.chdir("C:/Folder")
results = pd.DataFrame([])
for counter, file in enumerate(glob.glob("datayear*")):
namedf = pd.read_csv(file, skiprows=0, usecols=[1,2,3])
results = results.append(namedf)
results.to_csv('C:/combinedfile.csv')
In line 7 you have to specify the structure of the files’ name. Then, in line 8 you can select which columns you want to combine.
Thanks a lot, just an underscore is missing in the last line “to_csv”
Hi this for some reason worked for me on spider but not pycharm. I keep getting a passed header mismatch usecols error
I am not sure what is going on there. I am sure you verified that all the files have the same structure, right? I just use the Python IDLE to run this script.