2 years, 11 months ago
Combine a bunch of XLS files into ONE CSV file. List of all nonprofits in the US.
First person to do it TODAY (sunday) gets the tip.
All the XLS files from this page: http://www.irs.gov/taxstats/charitablestats/article/0,,id=97186,00.html combined into one CSV file, fields all in the right place.
email it to fortressgame@techhouseliving.com and let me know you have done so.
tips: use 'downThemAll' or some other extension to get all the xls files at once.
All the XLS files from this page: http://www.irs.gov/taxstats/charitablestats/article/0,,id=97186,00.html combined into one CSV file, fields all in the right place.
email it to fortressgame@techhouseliving.com and let me know you have done so.
tips: use 'downThemAll' or some other extension to get all the xls files at once.
Separate topics with commas, or by pressing return. Use the delete or backspace key to edit or remove existing topics.
You can leave an optional "tip" with Mahalo's virtual currency, Mahalo Dollars. If you are asking a difficult question that might require some research, or if you'd like a wide variety of feedback, a higher tip often leads to more answers to your question.
M$2 Answers
Working on it, this may take some time. When does Sunday end?
You can leave an optional "tip" with Mahalo's virtual currency, Mahalo Dollars. If you are asking a difficult question that might require some research, or if you'd like a wide variety of feedback, a higher tip often leads to more answers to your question.
M$
Wow! great job everyone!!!!
@Wdawe:
I don't know how you're doing this... but if it's taking you a ton of time, maybe this will help:
You're a web guy, right?
Do you have admin access to a mySQL database?
Save each file from that page as a csv and import each one mySQL (should take about 5 min of opening and saving... the import will only take a minute after converting to csv)
Then, export the combined database as a CSV.
All of this can be done pretty quickly with the point-and-click gui phpMyAdmin that's included with any cPanel host. Of course, the LOAD sql command will read a csv if you're on the command line.
I thought that I'd post this in case you were copying-and-pasting all that stuff... but you may have found a better way and if so... never mind :)
Nah, you earned it. I was doing it the hard way. :)
All done. The final file is 515M, zipped it's 99M. There certainly are a lot of nonprofits in the U.S.A.
Yeah, I just gave it a go... this is pretty quick and easy.
Giver WDAWE!!!!
You don't have to use a database... I thought about it more and that's overkill.
You could just save each file as a CSV and then (if you're using windows) use the copy command: copy *.csv master.csv
That'll merge all of the csv files into one master.csv file.
So really, the longest part of this task is opening them up and pressing save as. If you wanted to get really fancy and slip through this in under 10 minutes total, you could create a macro to save as, open all of the files and watch some TV while everything works away.
I'm using a perl script to do the conversion, it takes about 5 minutes per file so I'm browsing the web while it does the work. The biggest problem is that I've managed to kick the plug out twice (don't ask). If you want the money silverhammer you can have it, the hardest problem I had was finding the perl script and waiting long enough for it to do the conversion. I killed it off a couple of times because I thought it was hung. Otherwise I probably would have been done by now. I currently have 31 of the 53 files completed. Oops I meant 84 files.
The problem I've run into is that the fields are defined by position, not tabs so it's important to create the commas at the right point.
As a real world project this would take up to four hours (MD$10/4=$2.50/hr x $0.75 (real $) = $1.87/hr). Including a necessary accuracy check (cut and paste even to make one document to import is limited by max file size (depending on the text editor you use).
I'm an hour into it (doing it just for fun) and saw that wdawe is working on it so I'll step down.
I hope it's a generous tip. Good luck, wdawe!
P.S. Using the xls files is definitely the way to go. Easy to export as a CSV file too. :)