Using AWK for Text File Input

In some recent shell script development, I have fallen in love with an amazingly useful text processing language: AWK. Here are some of the basic command structures I used:

ls ~/music/ | awk '{ print "File" NR "=~/music/" $1 }'

Reading piped input from the directory listing, ls, the script prints "File", the current line number (determined using NR) of the input stream, and a path to the file located in the "music" folder located in the users home folder. For example, if the directory listing was:

jazz-music.mp3

rock-music.mp3

the script would output the following text:

File1=~/music/jazz-music.mp3
File2=~/music/rock-music.mp3

The $1 is used to retrieve the first field in the current line of the file (in this instance, there was only one field). I used this command, along with some other basic shell commands, in a script that listed files from a directory and created a media playlist file.

Here's another command I used:

php retrieve-media-url.php | awk '{ system("wget -O ~/music/" $2 " " $1) }'

While this command is a bit more complex, it uses only a few different elements. Instead of input from a directory listing, AWK receives piped input from the output of a PHP script. This script returns a simple text file containing two columns of data per line. The first column is the URL of a media file and the second column was the filename the download should be renamed to. The AWK system command was used to execute a shell command (wget in this case). For example, if the PHP script returned the following text:

http://xyz-music.com/classical-orchestra-music.mp3 classical.mp3

http://xyz-music.com/rock-and-roll-music.mp3 rock.mp3

AWK would execute the following commands:

wget -O ~/music/classical.mp3 http://xyz-music.com/classical-orchestra-music.mp3

wget -O ~/music/rock.mp3 http://xyz-music.com/rock-and-roll-music.mp3

Since there was two columns, $1 and $2 were used to retrieve the text from the first and second column of each line of input.

Pseudo advanced shell scripts is something that I plan to blog on much more in the future. Thanks for reading.

Under the Hood

Have you looked at a technological problem and wondered how it could be solved simply and efficiently? Have you ever tried to logic out the deep inner workings of a modern CPU? Do you have the habit of looking at anything technological and disassembling it in your mind? Are you a computer geek? If you answered yes to any of these questions, mainly the last one, then this is the blog to read. From Unix to Windows and back again, you will find reviews, tutorials, and stories that will inform, inspire, and amuse you.