Shell Brace Expansion

I would say that most developers that I know are comfortable with a shell/terminal but they are not proficient with the shell. I, for one, am constantly striving to improve my knowledge of the shell, customizing my environment, and trying to maximize my productivity with it. Even after years of constant usage I am always learning new tricks and techniques. It also doesn’t help that there are thousands of command line applications out there waiting to be absorbed.

My shell of choice, like most people I know, is bash. Most developers only use the basic features available in shells such as launching programs, command history, auto-completion, and the like. However, most shells have functionality people don’t even know about and could probably benefit from. Who knows, you might even be able to multitask and partake in foxy bingo online or pogo games while programming simultaneously. One such feature is Brace Expansion.

Take a look:

Brace Expansion

Its a little hard to see from the image so I’ll run down the examples here:

shell> echo hello{world,goodbye}
helloworld hellogoodbye

This example shows exactly what Brace Expansion is doing. Its taking the input line “hello{}” and replacing the “{}” section with each of the comma separated values, producing “helloworld” and “hellogoodbye.”

Its important to note that there is no space between “hello” and the opening brace. This is called the preamble and therefore it gets prepended to the front of both “world” and “goodbye.” Also, there is no space around the comma inside the braces. If there were a space then brace expansion doesn’t take place at all! The output itself will separate each expansion with a space, just like any other built-in shell expansion. If you really want spaces in the expansion portions you can escape them as you normally would and it will respect that.

shell> echo file.{txt,rb}
file.txt file.rb

This points out a couple things. First of all, I’m using echo to debug the brace expansion so you can see what it produced. What seemingly happens, like in the previous case, is that it ran the brace expansion when it parsed the input line, and turned it into the following:

shell> echo file.txt file.rb

Now some simple usages pop to mind. Need to rename a file? How about “mv filename.{txt,rb}.” Want to quickly backup a file, how about “cp filename.{txt,save}.” Whenever you feel yourself typing the same thing twice in a row that is a perfect candidate for brace expansion! Save yourself some keystrokes, some time, and potential errors by reusing what you’ve already typed in already!

The third usage is combining two separate but connected brace expansions “{}{}.” This works differently then two completely disjoint sets “{} … {}” and is still different then nested sets “{…{}…}.” Rather then discuss the semantics of those, which you can easily find out yourself by just trying them out, I’ll give you a glimpse of another trick you can do with brace expansion.

Here is a simple example of Sequences:

Brace Expansion Sequences

The syntax for sequences is “{#..#}.” You can even do simple character sequences like “{a..z}.” The second example from the picture shows a postscript, the opposite of the preamble. Its text that comes immediately after the braces (no space) and is thus appended to the end of each expansion.

I hope you give it try, you might end up liking it. I actually do some real neat tricks with it in one of my scripts. If you want to explore it further you should check out some of the more technical documentation:

Until next time, wrap your head around this crazy line. And no, there is no typo, its running a brace expansion once, which then outputs a very long command which includes brace expansion, and then runs that command with bash:

shell> echo echo {A..Z}\{0..{1..4}\} | bash

Updates to line.rb – Open to more suggestions

I was chatting with a friend the other day and he had some really good ideas concerning last week’s line.rb script. He wants to make an improved version. I look forward to seeing what he will come up with because he shared some great ideas. Well, I couldn’t help but implement them immediately in my version. His ideas also spurred a little creativity from me and I came up with a few new features as well. The usage is now looks like:

  # Check Cmd Line Args and Print Usage if needed
  if ARGV.size <= 1
    puts "usage: line [options] filename numbers"
    puts " options:"
    puts "   --silent or -s   Just print the line, without [#]"
    puts " number formats:"
    puts "   1-3              Prints lines 1 through 3"
    puts "   5                Prints line 5"
    puts "   -1               Prints the last line, (negative)"
    puts " extra formats:"
    puts "   ~5               Prints 2 (default) lines before and after 5"
    puts "   4~10             Prints 4 lines before and after 10"
    puts "   *7 or 8*         Prints all lines before or after the number"
    puts "   5/1              Prints 5 lines, then skips 1 line..."
    puts "   2:5/1            Starts at line 2, prints and skips..."
    exit 1

So there is some cool new syntax:

  • ~# – prints 2 lines before and after the given line number

  • #~# – prints the given number of lines before and after the given line number

  • *# – prints all of the lines before the given line number

  • #* – prints all of the lines after the given line number

  • #/# – is a print and skip option

  • #:#/# – allows you to provide an offset

Lets say you want to print every even line in a text file. With this script that is not a problem. The solution is: starting on the 2nd line, print 1 line, then skip 1 line. (line file 2:1/1):

Print only even lines using line.rb

I think these new formats are pretty neat. You can always access the latest version of the script on Github or in my ~/bin. I’m willing to change the syntax and introduce new concepts. Send suggestions my way by dropping a comment! Cheers.

Quickly Output Lines in a File

The other day I wanted a shell command to have somebody print out the 671st line of /etc/services on their computer. So I gave it some thought, then some more thought, scratched my head, and figured out that I couldn’t really think of a shell command that does that. A google search came up with a few `sed` and `awk` examples but I honestly found those to be a bit awkward for something that should be super simple. So I wrote my own script.

After writing the script to print out a single line, I soon found it made a lot of sense to include ranges. Taking advantage of Ruby I can even print lines from the end with negative numbers. So I spent a few minutes to clean up the script and make it a little more reusable, add some formatting, and cleaner. Here was the result:

line sample usage

line is now the latest addition to my ~/bin.

Also, in case you wanted to use this in a shell script or via piping, there is a `–silent` or `-s` switch that you can use that removes all of the special formatting, and prints only the specified lines. Much nice for scripts. (See the highlighted line in the image below). Enjoy!

line source

So what exactly is line 671 of /etc/services? On my mac it is:

line 671 of /etc/services

Dynamic Web URLs with ExpanDrive

Often when I work with ExpanDrive the files I am working on correspond to some website that I own. When I’m mounted with ExpanDrive each file is accessible via “my hard drive” in the mounted volume and, more importantly, from a web URL! I found myself repeatedly opening up my browser and manually typing the URL for files that I just uploaded or edited. This was error prone, especially if some of the characters needed encoding. So, I spent some time to write up a Ruby script that can read ExpanDrive’s preferences, build the file’s “web URL,” and open it in your default browser.

expanurl usage

Simple to Use

I followed along with ExpanDrive’s previous command line tool named expan and named my script expanurl. Given no arguments it will open the current directory via its web URL, or you can give a list of files and each will be opened at their web URLs. Its usage is pretty straightforward but there is a single catch: the server setting in an ExpanDrive Drive may not be a true one-to-one mapping with the web server’s address.

For example: I provide as the server value for one of my personal ExpanDrive drives. However, when I view files on that server (inside the public_html directory) they have a much different looking URL: The result? The script simply keeps its own mapping of ExpanDrive server values to their associated web page prefixes. When you use expanurl on a Drive you have never used it on before the script will prompt you for that mapping, store the value, and never ask again.

Here it is in action. I have removed all stored mappings so I can demonstrate what it would be like using expanurl for the first time. Here I use it on my BogoJoker ExpanDrive drive:

expanurl first usage

Notice that in the prompt it tells you:

  • the server that the ExpanDrive volume is linked to and the one you will be providing a web url prefix for
  • an example of a web url prefix (useful)
  • where the mappings are stored in case you need to edit them later

The script is available on GitHub, so feel free to contribute and improve. Here is a link to the always current version, and here is a snapshot of the current version at the time of writing:


#!/usr/bin/env ruby
# Author: Joseph Pecoraro
# Date: Saturday December 13, 2008
# Description: When I'm using ExpanDrive and I'm
# remotely logged into a server, I can use this
# script to "open filename" and it will open using
# the server's associated URL.

# For URL Escaping and Stored Mappings
require 'uri'
require 'yaml'

# Exit with a msg
def err(msg)
  puts msg
  exit 1

# Simple Class to handle the mappings
class UrlMap
  MAP_PATH = File.expand_path('~/.expanurl')
  def initialize
    @hash = load
  def load
    if File.exists?(MAP_PATH)
      YAML::load_file( File.expand_path(MAP_PATH) )
  def add_mapping(server, mapto)
    @hash[server] = mapto, 'w') do |file|
  def is_mapping?(server)
  def get_mapping(server)
  def path

# Local Variables
mapping =
url_prefix = nil
server = nil
volume = nil

# Check the if the current directory is an
# ExpanDrive Volume and a public_html folder
pwd = `pwd`
match = pwd.match(/^\/Volumes\/([^\/]+)/)
if match.nil?
  err("Not inside an ExpanDrive Volume")
elsif !pwd.match(/\/public_html\/?/)
  err("Not inside a public_html directory.")
  volume = match[1]
  defaults = `defaults read com.magnetk.ExpanDrive Drives`
  defaults.gsub!(/\n/, '')
  props = defaults.match(/\{[^\}]+driveName\s+=\s+#{volume}[^\}]+server\s+=\s+"([^"]+)"[^\}]+\}/)
  if props
    server = props[1]
    err("This Volume (#{volume}) is not an ExpanDrive Volume")

# Check if a mapping exists
# Otherwise create and store one
if mapping.is_mapping?(server)
  url_prefix = mapping.get_mapping(server)
  # Prompt
  puts "This is the first time you've used expanurl for #{volume}"
  puts "Please Provide us with a mapping for #{server}"
  puts "Mappings are stored in #{mapping.path}"
  puts "Example:"
  print ">> "
  # Store user input and proceed
  url_prefix = gets.chomp
  url_prefix += '/' unless url_prefix.match(/\/$/)
  mapping.add_mapping(server, url_prefix)
  # Terminal Output
  puts "Server: #{server}"
  puts "Maps to: #{url_prefix}"


# Build the URL
subpath = pwd.match(/public_html\/?(.*)/)[1]
subpath += '/' unless || subpath.match(/\/$/)
url_prefix += subpath

# If No Files, open the directory
# Otherwise,   open each provided file
if ARGV.size == 0
  `open #{url_prefix}`
  ARGV.each do |filename|
    `open #{url_prefix}#{URI.escape(filename)}`


How it Works

The Ruby Script grabs the current working directory using `pwd` and checks to make sure you’re in an ExpanDrive volume. ExpanDrive volume’s are dynamically generated by parsing the ExpanDrive preferences thanks to their foresight to make them accessible via the `defaults` command. So if you’re in an ExpanDrive volume and inside a public_html directory expanurl will then use its mapping to open a uri encoded web url in your default browser with the `open` command.

The mappings are stored in a hidden YAML file in your home directory (~/.expanurl). This style of storing preferences is just like dozens of other command line applications and scripts. YAML is just a lightweight textual data format popular with Ruby, similar to JSON and XML. Its so simple that you could edit the file yourself if you wanted/needed to. For instance here is what is in mine, just two simple key/value pairs:

~/.expanurl yaml mapping

The Future

Its just that simple. Being a Ruby Script you can call this from GUI applications, anything with built-in shell access, etc. It should play friendly with your usual Unix tools. I will likely make this script more and more robust if others find it useful, so I’d be happy to hear some feedback.


Mac OS X What is my IP Address?!

I’ve asked this question a lot. What is my IP address? This can be for any number of reasons. There are a number of ways that this can be done, but I was looking for a way to do this on the command line. Well… it didn’t turn out to be as simple as I had hoped.


The primary tool is ifconfig. However, that spits out way more information then I wanted. So a little bit of reading, some regular expressions, and I created the following script. Note that en1 is my wireless port because I am always on my wireless:

ifconfig en1 | awk '$2~/[0-9]+./{print$2}'

Update: I found a better solution:

# Update: Even Better!
ipconfig getifaddr en1

Here is some usage and me checking that it actually makes sense:

ipaddr usage


I’m paranoid… so let me double check at “What!?” That didn’t make sense. Or did it? A few minutes later, after running a traceroute and following the hops I noticed that sure enough the Frontier Wireless provider in this public CafĂ© was using that IP address. So I wrote a quick curl script to pull whatismyip’s opinion and quite craftily they must have planned for this:

whatismyip suggestion

Cool. That makes my job even simpler. The response is simply my ip address string. Add an echo to provide a newline and I’ve got a useful script.

curl --silent

Hopefully these two scripts can help save someone a few minutes. ipaddr to see what my machine’s ip address is and ipaddr2 to see through what ip address my computer is reaching the outer world. Cheers.

Ruby Process Controller – psgrep

Every once in a while a process will freeze and will be too stubborn to die when I try to “Quit” it. For those stubborn processes I tend to use the terminal to `kill` it. For a while I had been using a simple Perl script for searching through processes. The script would find me the processids and I could then kill it, using whatever power I need.

I found that it was taking far too long for me to do the search, and then carefully type out the process id, and hope I got the right one. I discovered killall, but the problem is that sometimes I don’t want to kill “all” of the processes with that name. So, I gave in and wrote up a Ruby script that did what I wanted. Here is psgrep: (Download)

#!/usr/bin/env ruby
# Start Date: Saturday December 6, 2008
# Current Version: 0.9
# Author: Joseph Pecoraro
# Contact:
# Decription: Quicker handling of process searching
# and killing.  Never type in a PID again, just regexes!

# -----------
#   Globals
# -----------
kill_mode = false
icase_mode = false
pattern = nil
targets = []
pids = []

# ---------
#   Usage
# ---------
def usage
  puts "usage: #{$0.split(/\//).last} [options] pattern"
  puts "  -k  or --kill   kills all processes"
  puts "  -k# or --kill#  kills the [#] process"
  exit 0

# -----------
#   Options
# -----------
if ARGV.size > 1
  ARGV.each do |arg|
    if arg.match(/^-(k|-kill)(\d*)$/)
      kill_mode = true
      targets << $2.to_i unless $2.empty?
    elsif arg.match(/^-(i|-ignore)$/)
      icase_mode = true
  ARGV.delete_if { |e| e.match(/^-/) }

# -------------------
#   Remaining Args
# -------------------
if ARGV.size != 1

if icase_mode
  pattern = ARGV[0], Regexp::IGNORECASE )
  pattern = ARGV[0] )

# ----------------------
#   Actual `ps` Output
# ----------------------
lines = %x{ ps -Au#{ENV['USER']} }.split(/\n/)
header = lines.shift

# ----------
#   psgrep
# ----------
puts "     #{header}"
count = 0
lines.each do |line|
  unless line =~ /psgrep/
    if line.match(pattern)
      count += 1
      puts "[#{count}]: #{line}"
      if targets.empty? || targets.member?(count)
        pids << line.strip.split[1]

# -------------
#   Kill Mode
# -------------
if kill_mode
  puts "Killing Processes"
  puts "-----------------"
  pids.each_with_index do |pid, i|
    print targets.empty? ? "[#{i}]:" :  "[#{targets[i]}]:"
    print " Killing #{pid}... "
    res = %x{ kill #{pid} }
    puts "Dead" if $?

# Always

So there it is. Less then 100 lines of ruby to get a pretty straightforward psgrep/kill program. Here is an example where I have three perl processes running on my machine. One of the is running as root (the userid is 0). I just type “!! –kill” or “[up-arrow] –kill” and it tries to kill them all. Note that the root perl process doesn’t terminate and there is an error message but psgrep continues as best as it can, and kills the two normal perl processes: [Note: I could have done `sudo psgrep perl -k` to kill the root process]

psgrep usage

Here is another good example. I have two python instances that I want to kill but there is another python instance running ExpanDrive in the background. I just ran psgrep and found the two I want to kill are [2] and [3]. Therefore, I can send -k2 and -k3 (or –kill2 and –kill3) to kill only those processes. Here is the result:

psgrep target

Note also that by default psgrep is case-sensitive. To ignore case just add the -i or –ignore switch. So there you have it. Usage is straightforward. Switches can go anywhere on the command line makes it easier to just use your history and tack a switch on the end of your previous command.

Feel free to improve it, it is on GitHub!

Dot Files For Your Shell and Even Ruby

I have come across a number of programmers who don’t know what dotfiles are. Thats a shame. Every programmer should know common dotfiles and actively seek to add aliases, functions, and other tidbits to increase their productivity and make the shell more usable. I recently went on a binge and updated a few of my dotfiles. I’ll share some useful tricks that I found on not only my ~/.bashrc file but also ~/.irbrc for my ruby IRB prompt!

I won’t show off my entire ~/.bashrc file, only because it is rather long. I just want to get across some of the usefulness of such a file:

# -----------
#   General
# -----------
alias ..='cd ..'
alias ll='ls -lh'
alias la='ls -la'
alias ps='ps -ax'
alias du='du -hc'
alias cd..='cd ..'
alias more='less'
alias mkdir='mkdir -p'
alias today='date +"%A, %B %d, %Y"'
alias yest='date -v-1d +"%A %B %d, %Y"'
alias recent='ls -lAt | head'
alias ebashrc='mate ~/.bashrc'
alias mbashrc='mate ~/.bashrc'
alias sbashrc='source ~/.bashrc'
alias htdocs='cd /Applications/MAMP/htdocs/'
alias mampmysql='/Applications/MAMP/Library/bin/mysql -u XXXXX -p'
alias desktoptopia='open /Users/joe/Library/Application\ Support/Desktoptopia/.Backgrounds/'
alias ql='qlmanage -p "$@" >& /dev/null' # Quick Look alias

# -------------
#   Shortcuts
# -------------
alias c="clear"
alias m="mate"

# --------
#   SSHs
# --------
alias rit="ssh -l XXXXX"
alias vega="ssh -l XXXXX"

# -------
#   Git
# -------
alias ga='git add'
alias gs='git status'
alias gd='git diff'
alias github="open \`git config -l | grep 'remote.origin.url' | sed -En 's/remote.origin.url=git(@|:\/\/)|\/)(.+)\/(.+).git/https:\/\/\/\3\/\4/p'\`"

# --------
#   Ruby
# --------
alias irb='irb -r irb/completion -rubygems'

# ---------------
#   Environment
# ---------------
export PATH="$PATH:/usr/local/bin:/usr/local/sbin:/usr/local/mysql/bin"
export PATH="$HOME/bin/:$PATH"
export HISTSIZE=10000
export HISTFILESIZE=10000
export PAGER=less
export CLICOLOR=1
export EDITOR="/usr/bin/mate -w"

Hopefully this wasn’t too overwhelming. But lets take a look at some of these. There are a bunch of aliases at the top which simply replace the older version? It just makes sense that when you do a ps you really want `ps -ax`. Likewise a few others there are printing disk usage with human readable output.

Nothing completely interesting however everything is extremely useful. I’m cutting my keystrokes in half and getting better output. I have a bunch of more exciting tricks in the rest of my ~/.bashrc. Its not just reserved for aliases and EXPORTS, take this function for example:

# cd directly to a dir and list contents
cdl() {
  if [ "$1" ]
  then builtin cd "$1" && ll
  else builtin cd && ll

Cd to a directory and list the directory. I even reference my ll alias up above to list long. You can take a peek at my entire .bashrc file on Lets move on to something you may not have known about. Let me walk you through parts of a nice ~/.irbrc file, which you might want for yourself:

# Load and Start Wirble, a gem to beautify irb
require 'wirble'

Probably one of the most popular inclusions in a .irbrc file. Wirble is a Ruby gem built for improving the irb interactive ruby console. That means you may also have to ‘require “rubygems”‘ in order for this to work. Take a look at the gem documentation for Wirble to find out more of its capabilities. But that also means you can include a number of other useful rubygems. For instance, what_methods, map_by_method, hpricot, yaml, the list goes on and on. Instead I’m going to point out some other neat additions you can add to your .irbrc!

# Awesome benchmarking function
# Source:
def time(times=1)
  require "benchmark"
  ret = nil { |x| { times.times { ret = yield } } }
alias bench time

# A cool way to index in a hash
# h = { :alpha => 'bet', :beta => 'blocker' }
# h/:beta #=> 'blocker'
class Hash
  def /(key)

# Simple regular expression helper
# show_regexp - stolen from the pickaxe
def show_regexp(a, re)
  if a =~ re
    "no match"

# Convenience method on Regexp so you can do
# /an/.show_match("banana") # => "b<>ana"
class Regexp
  def show_match(a)
    show_regexp(a, self)

There, now you’re building some really useful tricks! Check it out. Thanks to a few different sources for those code snippets I can easily benchmark any code run at any number of times. There is a neat way to pull an element from a hash, without having to put [brackets] around the key. Finally, a little helper for regular expressions which is useful every now and then.

But wait, there is more. This time a little more system specific:

# Textmate helper
# Source:
def mate *args
  flattened_args = {|arg| "\"#{arg.to_s}\""}.join ' '
  `mate #{flattened_args}`

# Clear
def c

You can now open up TextMate from within irb! I’ve also grown so used to using my ‘c’ alias to clear the terminal prompt that I added the same functionality to irb. Just make sure that you don’t name a variable c!

Now let me blow your mind:

# Why's aorta method to edit an object in YAML, awesome!
# Source:
require 'yaml'
def aorta( obj )
  tempfile = File.join('/tmp',"yobj_#{ }") tempfile, 'w' ) { |f| f << obj.to_yaml }
  system( "#{ ENV['EDITOR'] || 'vi' } #{ tempfile }" )
  return obj unless File.exists?( tempfile )
  content = YAML::load( tempfile ) )
  File.delete( tempfile )

Why is a famous member of the Ruby community. This function here actually takes a Ruby object, exports it as YAML to a file, opens the File for editing, and once saved reloads the file from YAML. Essentially it allows you to edit the contents of an object in YAML. Absolutely amazing, let that stir in your mind for a minute!

Now, I have started building my own library of functions that I have deemed semi-useful but not worthy of turning into a gem. These functions include some simple String extensions like my TXT helpers. I include all these files in a directory and I auto-load them like so:

# Load all my non-test libraries in '~/.util/irb'
util_dir = File.expand_path('~') + '/.util/irb/*'
Dir[util_dir].each do |f|
  require f unless File.basename(f) =~ /\Atest/

Notice how I ignore any files starting with “test”. I have gotten into the habit of creating test files for my libraries and the naming convention I use is the exact same as the majority of Ruby developers, you just create a new file called “test_library.rb” to test “library.rb”. Simple yes, and helpful in this case where I want to avoid loading these test files. This means I have all my useful functions pre-loaded whenever I open the irb. I’d like to see what additional stuff you have!

Finally here are links directly to my .bashrc and .irbrc files. Enjoy, and please give me additions!