Merging files with pr
Tonight, I’ve been poring over a rather large data set that I want to get some useful information out of. All the data was originally stored in a .html file, but after some (very) crude extraction techniques, I managed to pull out just the data I wanted, and shove it into a comma-separated file. Earlier, I had given up on my tools at hand and typed up an entire list of row headings for my newly-gotten data. So I had two files like so:
headings.txt
Alpha
Bravo
Charlie
values.csv
1,2,3,4
5,6,7,8
9,10,11,12
I spent quite a bit of time trying to figure out how to combine the two columns into one file with what I knew, but none of my tools could quite do it without nasty shell scripting. It took me a while, but I eventually found this post that cracked the case for me. The pr command, ostensibly for paging documents, has enough horsepower to solve my problem in short order, like so:
$ pr -tm -s, headings.txt values.csv
The -t tells the program to omit headers and footers, and -m tells it to merge each line. The -s, tells it to use commas as field-separators. My desired result, like so:
headings.txt
Alpha,1,2,3,4
Bravo,5,6,7,8
Charlie,9,10,11,12
There are numerous other options to pr, and depending on your potential line lengths, one may have to experiment. But for me, this got the job done.
External Links
Interesting tool, pr. I’ll have to remember that one. You could’ve done this in Vim’s Visual Block mode or in Emacs via column editing too.
Really? If you happen to know how this would be done in vim, I’d be rather interested to see it. There’s a lot of functionality in vim I haven’t yet discovered. As a matter of fact, I’ve only ever used visual mode — I hadn’t even heard of visual block mode. More to learn, I have!
In the headings.txt buffer, first:
gg0^VG$A,^[
(Where ^V is Ctrl-V and ^[ is Esc.) This'll move the cursor to the top left, highlight everything in Visual-block mode, and add a comma at the end of each line. Then:
gg0^VG$y
This'll highlight and copy everything column-wise. Then in values.csv:
gg0P
This'll paste everything column-wise at the front of the existing lines. You might have a few extra spaces after the first comma, but you can clean it up with a regex e.g.
:%s/\v^[^,]+,\zs\s*//
(Where all the ^ are literal ^’s.) Visual-block mode is quite useful even though there are only a few operators you can use in that mode. See :h visual-block and :h blockwise-operators.