Quickly Output Lines in a File

The other day I wanted a shell command to have somebody print out the 671st line of /etc/services on their computer. So I gave it some thought, then some more thought, scratched my head, and figured out that I couldn’t really think of a shell command that does that. A google search came up with a few `sed` and `awk` examples but I honestly found those to be a bit awkward for something that should be super simple. So I wrote my own script.

After writing the script to print out a single line, I soon found it made a lot of sense to include ranges. Taking advantage of Ruby I can even print lines from the end with negative numbers. So I spent a few minutes to clean up the script and make it a little more reusable, add some formatting, and cleaner. Here was the result:

line sample usage

line is now the latest addition to my ~/bin.

Also, in case you wanted to use this in a shell script or via piping, there is a `–silent` or `-s` switch that you can use that removes all of the special formatting, and prints only the specified lines. Much nice for scripts. (See the highlighted line in the image below). Enjoy!

line source

So what exactly is line 671 of /etc/services? On my mac it is:

line 671 of /etc/services

~/bin – Tilde Bin

So I wanted to do a little something over break. I had no idea what I would do, until I came across this article taking a look at the blueprint css framework. I had heard plenty of talk about this css framework, and other grid frameworks, and decided to give it a shot.

So I had something new I wanted to work with, what was the site actually going to be? I gave it a few minutes thought and ended up with an idea that satisfied my semi-recent Internet philosophy:

When I have something worth spreading I should write it on my blog or turn it into a webpage. This way when people ask me about it, or if I need to present the information to someone, I can just point them to that single source. This saves me from having to type and retype the same information over and over for different people. A single “master” source with the most up to date information is always optimal.

With that in mind, I decided to whip up a simple site to showcase the scripts that are in my “tilde bin” folder. Scripts that I wrote and use frequently that others might find useful. Sure I’ll still blog about them, but for those who want to download them its so much faster to point them to ~/bin instead of having them scour my blog.

~/bin

And thus ~/bin was born.

Sidenotes

I was quite satisfied with the result. It took only 24 hours from Conception to Creation. I made use of Blueprint and jQuery (AJAX and animation) to create a pretty fresh website. Sure its pretty basic but I felt comfortable the entire way through, and I loved every minute of it. I’m hoping that when Ruby on Rails 3 is released, if I have some spare time, I’ll convert it and include a small admin dashboard.

As of two years ago I’ve decided to get pretty serious with my New Years Resolutions. This year (like last year) I have a few ideas I’m tossing around. It will be a few weeks until I settle on one. Typically the resolutions will help me improve my health or lifestyle. One of the ideas I’m tossing around is blogging at least once per week. Compared to the other alternatives this is certainly the most fun. It has a hint of a challenge and its pretty doable. Let me know what you think!

Mac OS X What is my IP Address?!

I’ve asked this question a lot. What is my IP address? This can be for any number of reasons. There are a number of ways that this can be done, but I was looking for a way to do this on the command line. Well… it didn’t turn out to be as simple as I had hoped.

ipaddr

The primary tool is ifconfig. However, that spits out way more information then I wanted. So a little bit of reading, some regular expressions, and I created the following script. Note that en1 is my wireless port because I am always on my wireless:

#!/bin/sh
ifconfig en1 | awk '$2~/[0-9]+./{print$2}'

Update: I found a better solution:

#!/bin/sh
# Update: Even Better!
ipconfig getifaddr en1

Here is some usage and me checking that it actually makes sense:

ipaddr usage

ipaddr2

I’m paranoid… so let me double check at whatismyip.com. “What!?” That didn’t make sense. Or did it? A few minutes later, after running a traceroute and following the hops I noticed that sure enough the Frontier Wireless provider in this public Café was using that IP address. So I wrote a quick curl script to pull whatismyip’s opinion and quite craftily they must have planned for this:

whatismyip suggestion

Cool. That makes my job even simpler. The response is simply my ip address string. Add an echo to provide a newline and I’ve got a useful script.

#!/bin/sh
curl --silent www.whatismyip.com/automation/n09230945.asp
echo

Hopefully these two scripts can help save someone a few minutes. ipaddr to see what my machine’s ip address is and ipaddr2 to see through what ip address my computer is reaching the outer world. Cheers.

Ruby Process Controller – psgrep

Every once in a while a process will freeze and will be too stubborn to die when I try to “Quit” it. For those stubborn processes I tend to use the terminal to `kill` it. For a while I had been using a simple Perl script for searching through processes. The script would find me the processids and I could then kill it, using whatever power I need.

I found that it was taking far too long for me to do the search, and then carefully type out the process id, and hope I got the right one. I discovered killall, but the problem is that sometimes I don’t want to kill “all” of the processes with that name. So, I gave in and wrote up a Ruby script that did what I wanted. Here is psgrep: (Download)

#!/usr/bin/env ruby
# Start Date: Saturday December 6, 2008
# Current Version: 0.9
# Author: Joseph Pecoraro
# Contact: joepeck02@gmail.com
# Decription: Quicker handling of process searching
# and killing.  Never type in a PID again, just regexes!

# -----------
#   Globals
# -----------
kill_mode = false
icase_mode = false
pattern = nil
targets = []
pids = []

# ---------
#   Usage
# ---------
def usage
  puts "usage: #{$0.split(/\//).last} [options] pattern"
  puts "  -k  or --kill   kills all processes"
  puts "  -k# or --kill#  kills the [#] process"
  exit 0
end

# -----------
#   Options
# -----------
if ARGV.size > 1
  ARGV.each do |arg|
    if arg.match(/^-(k|-kill)(\d*)$/)
      kill_mode = true
      targets << $2.to_i unless $2.empty?
    elsif arg.match(/^-(i|-ignore)$/)
      icase_mode = true
    end
  end
  ARGV.delete_if { |e| e.match(/^-/) }
end

# -------------------
#   Remaining Args
# -------------------
if ARGV.size != 1
  usage
end

if icase_mode
  pattern = Regexp.new( ARGV[0], Regexp::IGNORECASE )
else
  pattern = Regexp.new( ARGV[0] )
end

# ----------------------
#   Actual `ps` Output
# ----------------------
lines = %x{ ps -Au#{ENV['USER']} }.split(/\n/)
header = lines.shift

# ----------
#   psgrep
# ----------
puts
puts "     #{header}"
count = 0
lines.each do |line|
  unless line =~ /psgrep/
    if line.match(pattern)
      count += 1
      puts "[#{count}]: #{line}"
      if targets.empty? || targets.member?(count)
        pids << line.strip.split[1]
      end
    end
  end
end

# -------------
#   Kill Mode
# -------------
if kill_mode
  puts
  puts "Killing Processes"
  puts "-----------------"
  pids.each_with_index do |pid, i|
    print targets.empty? ? "[#{i}]:" :  "[#{targets[i]}]:"
    print " Killing #{pid}... "
    STDOUT.flush
    res = %x{ kill #{pid} }
    puts "Dead" if $?.exitstatus.zero?
  end
end

# Always
puts

So there it is. Less then 100 lines of ruby to get a pretty straightforward psgrep/kill program. Here is an example where I have three perl processes running on my machine. One of the is running as root (the userid is 0). I just type “!! –kill” or “[up-arrow] –kill” and it tries to kill them all. Note that the root perl process doesn’t terminate and there is an error message but psgrep continues as best as it can, and kills the two normal perl processes: [Note: I could have done `sudo psgrep perl -k` to kill the root process]

psgrep usage

Here is another good example. I have two python instances that I want to kill but there is another python instance running ExpanDrive in the background. I just ran psgrep and found the two I want to kill are [2] and [3]. Therefore, I can send -k2 and -k3 (or –kill2 and –kill3) to kill only those processes. Here is the result:

psgrep target

Note also that by default psgrep is case-sensitive. To ignore case just add the -i or –ignore switch. So there you have it. Usage is straightforward. Switches can go anywhere on the command line makes it easier to just use your history and tack a switch on the end of your previous command.

Feel free to improve it, it is on GitHub!

Spaces, Exposé, Hot Corners, Dashboard Broken?

I ferociously make use of Spaces and Exposé. To utilize them I use QuickSilver, Cmd+Tab, the Dock, special Mouse Buttons, but most of all I use the Hot Corners! Occasionally, when I wake my Mac Book Pro these features won’t work. No reason given, just when I try I get the usual chime meaning “this can’t be done.” At the time the only sure solution was to reboot the computer. Not anymore. Just open up your terminal and run:

killall Dock

Since those major features (Spaces, Exposé, and the Dashboard) are tied to the Dock, this command will restart the Dock and, like magic, those features will work again. Killing the Dock and even Finder with `killall` are special cases. Those two will automatically restart on their own. However, for other programs, it will just quit. For instance if Firefox hangs you could type `killall Firefox` and Firefox would be force quit but would not reopen. Check the man page for more information.

Dot Files For Your Shell and Even Ruby

I have come across a number of programmers who don’t know what dotfiles are. Thats a shame. Every programmer should know common dotfiles and actively seek to add aliases, functions, and other tidbits to increase their productivity and make the shell more usable. I recently went on a binge and updated a few of my dotfiles. I’ll share some useful tricks that I found on not only my ~/.bashrc file but also ~/.irbrc for my ruby IRB prompt!

I won’t show off my entire ~/.bashrc file, only because it is rather long. I just want to get across some of the usefulness of such a file:

# -----------
#   General
# -----------
alias ..='cd ..'
alias ll='ls -lh'
alias la='ls -la'
alias ps='ps -ax'
alias du='du -hc'
alias cd..='cd ..'
alias more='less'
alias mkdir='mkdir -p'
alias today='date +"%A, %B %d, %Y"'
alias yest='date -v-1d +"%A %B %d, %Y"'
alias recent='ls -lAt | head'
alias ebashrc='mate ~/.bashrc'
alias mbashrc='mate ~/.bashrc'
alias sbashrc='source ~/.bashrc'
alias htdocs='cd /Applications/MAMP/htdocs/'
alias mampmysql='/Applications/MAMP/Library/bin/mysql -u XXXXX -p'
alias desktoptopia='open /Users/joe/Library/Application\ Support/Desktoptopia/.Backgrounds/'
alias ql='qlmanage -p "$@" >& /dev/null' # Quick Look alias

# -------------
#   Shortcuts
# -------------
alias c="clear"
alias m="mate"

# --------
#   SSHs
# --------
alias rit="ssh holly.cs.rit.edu -l XXXXX"
alias vega="ssh vega.it.rit.edu -l XXXXX"

# -------
#   Git
# -------
alias ga='git add'
alias gs='git status'
alias gd='git diff'
alias github="open \`git config -l | grep 'remote.origin.url' | sed -En 's/remote.origin.url=git(@|:\/\/)github.com(:|\/)(.+)\/(.+).git/https:\/\/github.com\/\3\/\4/p'\`"

# --------
#   Ruby
# --------
alias irb='irb -r irb/completion -rubygems'

# ---------------
#   Environment
# ---------------
export PATH="$PATH:/usr/local/bin:/usr/local/sbin:/usr/local/mysql/bin"
export PATH="$HOME/bin/:$PATH"
export HISTSIZE=10000
export HISTFILESIZE=10000
export PAGER=less
export CLICOLOR=1
export EDITOR="/usr/bin/mate -w"

Hopefully this wasn’t too overwhelming. But lets take a look at some of these. There are a bunch of aliases at the top which simply replace the older version? It just makes sense that when you do a ps you really want `ps -ax`. Likewise a few others there are printing disk usage with human readable output.

Nothing completely interesting however everything is extremely useful. I’m cutting my keystrokes in half and getting better output. I have a bunch of more exciting tricks in the rest of my ~/.bashrc. Its not just reserved for aliases and EXPORTS, take this function for example:

# cd directly to a dir and list contents
cdl() {
  if [ "$1" ]
  then builtin cd "$1" && ll
  else builtin cd && ll
  fi
}

Cd to a directory and list the directory. I even reference my ll alias up above to list long. You can take a peek at my entire .bashrc file on dotfiles.org. Lets move on to something you may not have known about. Let me walk you through parts of a nice ~/.irbrc file, which you might want for yourself:

# Load and Start Wirble, a gem to beautify irb
require 'wirble'
Wirble.init
Wirble.colorize

Probably one of the most popular inclusions in a .irbrc file. Wirble is a Ruby gem built for improving the irb interactive ruby console. That means you may also have to ‘require “rubygems”‘ in order for this to work. Take a look at the gem documentation for Wirble to find out more of its capabilities. But that also means you can include a number of other useful rubygems. For instance, what_methods, map_by_method, hpricot, yaml, the list goes on and on. Instead I’m going to point out some other neat additions you can add to your .irbrc!

# Awesome benchmarking function
# Source: http://ozmm.org/posts/time_in_irb.html
def time(times=1)
  require "benchmark"
  ret = nil
  Benchmark.bm { |x| x.report { times.times { ret = yield } } }
  ret
end
alias bench time

# A cool way to index in a hash
# h = { :alpha => 'bet', :beta => 'blocker' }
# h/:beta #=> 'blocker'
class Hash
  def /(key)
    self[key]
  end
end

# Simple regular expression helper
# show_regexp - stolen from the pickaxe
def show_regexp(a, re)
  if a =~ re
    "#{$`}<<#{$&}>>#{$'}"
  else
    "no match"
  end
end

# Convenience method on Regexp so you can do
# /an/.show_match("banana") # => "b<>ana"
class Regexp
  def show_match(a)
    show_regexp(a, self)
  end
end

There, now you’re building some really useful tricks! Check it out. Thanks to a few different sources for those code snippets I can easily benchmark any code run at any number of times. There is a neat way to pull an element from a hash, without having to put [brackets] around the key. Finally, a little helper for regular expressions which is useful every now and then.

But wait, there is more. This time a little more system specific:

# Textmate helper
# Source: http://dotfiles.org/~lattice/.irbrc
def mate *args
  flattened_args = args.map {|arg| "\"#{arg.to_s}\""}.join ' '
  `mate #{flattened_args}`
  nil
end

# Clear
def c
  system('clear')
end

You can now open up TextMate from within irb! I’ve also grown so used to using my ‘c’ alias to clear the terminal prompt that I added the same functionality to irb. Just make sure that you don’t name a variable c!

Now let me blow your mind:

# Why's aorta method to edit an object in YAML, awesome!
# Source: http://rubyforge.org/snippet/detail.php?type=snippet&id=22
require 'yaml'
def aorta( obj )
  tempfile = File.join('/tmp',"yobj_#{ Time.now.to_i }")
  File.open( tempfile, 'w' ) { |f| f << obj.to_yaml }
  system( "#{ ENV['EDITOR'] || 'vi' } #{ tempfile }" )
  return obj unless File.exists?( tempfile )
  content = YAML::load( File.open( tempfile ) )
  File.delete( tempfile )
  content
end

Why is a famous member of the Ruby community. This function here actually takes a Ruby object, exports it as YAML to a file, opens the File for editing, and once saved reloads the file from YAML. Essentially it allows you to edit the contents of an object in YAML. Absolutely amazing, let that stir in your mind for a minute!

Now, I have started building my own library of functions that I have deemed semi-useful but not worthy of turning into a gem. These functions include some simple String extensions like my TXT helpers. I include all these files in a directory and I auto-load them like so:

# Load all my non-test libraries in '~/.util/irb'
util_dir = File.expand_path('~') + '/.util/irb/*'
Dir[util_dir].each do |f|
  require f unless File.basename(f) =~ /\Atest/
end

Notice how I ignore any files starting with “test”. I have gotten into the habit of creating test files for my libraries and the naming convention I use is the exact same as the majority of Ruby developers, you just create a new file called “test_library.rb” to test “library.rb”. Simple yes, and helpful in this case where I want to avoid loading these test files. This means I have all my useful functions pre-loaded whenever I open the irb. I’d like to see what additional stuff you have!

Finally here are links directly to my .bashrc and .irbrc files. Enjoy, and please give me additions!

mgrep – Multiple Regex Grep – SotD

My newest Ruby utility is called mgrep. It is a multiple regular expression version of grep. Input is processed line by line and each regular expression is said by the user to match or not match. All lines that meet the user’s desires for matching/non-matching regular expressions are printed in the following format: “filename [line number]: line of text.”

This can be taken in a number of different directions. I’m thinking “Partial Matches” meaning one regular expression matches one line in the file and eventually a second regular expression matches a totally different line and if all of these conditions succeed by the end of the file then print out the filename. This sounds more useful and will most likely be in version 1.0.

Here is the current usage:

usage: mgrep [-#] ( [-n] regex ) [filenames]
  #         - the number of regular expressions, defaults to 1
  ( ... )   - there should be # of these
  regex     - regular expessions to be checked on the line
  filenames - names of the input files to be parsed, if blank uses STDIN

options:
  --neg, --not, or -n    line must not match this regular expression

special note:
  When using bash, if you want backslashs in the replace portion make sure
  to use the multiple argument usage with single quotes for the replacement.

The usage is a little confusing seeing as the number of regular expressions on the command line are variable based on another command line switch. All in all though it is rather clear. Options will likely come in the future, much like grep or awk if I get around to it.

Here is an example probably not useful but at least it shows functionality, the line must contain a number, a double letter, and end with a !: [input]

1 ab !
- aa !
1 aa -
1 aa !
oh yah! 1

And when I run my script, I’ll put all the regular expressions in /here/ to make it clearer, this syntax is allowed by mgrep for convenience. Here is what it looks like:

joe[~/sandbox]$ mgrep -3 '/\\d/' '/(\\w)\\1/' '/!$/' input
input [4]: 1 aa !

To show of the –neg or -n option this command will show all the lines that do not have a hypen and still end with a !:

joe[~/sandbox]$ mgrep -2 -n '/-/' '/!$/' input
input [1]: 1 ab !
input [4]: 1 aa !

The script surely be updated soon, but grab it now and try it out:
mgrep – Most Recent Version – Download
mgrep – changelog

search