Updates to line.rb – Open to more suggestions

I was chatting with a friend the other day and he had some really good ideas concerning last week’s line.rb script. He wants to make an improved version. I look forward to seeing what he will come up with because he shared some great ideas. Well, I couldn’t help but implement them immediately in my version. His ideas also spurred a little creativity from me and I came up with a few new features as well. The usage is now looks like:


  # Check Cmd Line Args and Print Usage if needed
  if ARGV.size <= 1
    puts "usage: line [options] filename numbers"
    puts
    puts " options:"
    puts "   --silent or -s   Just print the line, without [#]"
    puts
    puts " number formats:"
    puts "   1-3              Prints lines 1 through 3"
    puts "   5                Prints line 5"
    puts "   -1               Prints the last line, (negative)"
    puts
    puts " extra formats:"
    puts "   ~5               Prints 2 (default) lines before and after 5"
    puts "   4~10             Prints 4 lines before and after 10"
    puts "   *7 or 8*         Prints all lines before or after the number"
    puts "   5/1              Prints 5 lines, then skips 1 line..."
    puts "   2:5/1            Starts at line 2, prints and skips..."
    puts
    exit 1

So there is some cool new syntax:

  • ~# – prints 2 lines before and after the given line number

  • #~# – prints the given number of lines before and after the given line number

  • *# – prints all of the lines before the given line number

  • #* – prints all of the lines after the given line number

  • #/# – is a print and skip option

  • #:#/# – allows you to provide an offset

Lets say you want to print every even line in a text file. With this script that is not a problem. The solution is: starting on the 2nd line, print 1 line, then skip 1 line. (line file 2:1/1):

Print only even lines using line.rb

I think these new formats are pretty neat. You can always access the latest version of the script on Github or in my ~/bin. I’m willing to change the syntax and introduce new concepts. Send suggestions my way by dropping a comment! Cheers.

Quickly Output Lines in a File

The other day I wanted a shell command to have somebody print out the 671st line of /etc/services on their computer. So I gave it some thought, then some more thought, scratched my head, and figured out that I couldn’t really think of a shell command that does that. A google search came up with a few `sed` and `awk` examples but I honestly found those to be a bit awkward for something that should be super simple. So I wrote my own script.

After writing the script to print out a single line, I soon found it made a lot of sense to include ranges. Taking advantage of Ruby I can even print lines from the end with negative numbers. So I spent a few minutes to clean up the script and make it a little more reusable, add some formatting, and cleaner. Here was the result:

line sample usage

line is now the latest addition to my ~/bin.

Also, in case you wanted to use this in a shell script or via piping, there is a `–silent` or `-s` switch that you can use that removes all of the special formatting, and prints only the specified lines. Much nice for scripts. (See the highlighted line in the image below). Enjoy!

line source

So what exactly is line 671 of /etc/services? On my mac it is:

line 671 of /etc/services

~/bin – Tilde Bin

So I wanted to do a little something over break. I had no idea what I would do, until I came across this article taking a look at the blueprint css framework. I had heard plenty of talk about this css framework, and other grid frameworks, and decided to give it a shot.

So I had something new I wanted to work with, what was the site actually going to be? I gave it a few minutes thought and ended up with an idea that satisfied my semi-recent Internet philosophy:

When I have something worth spreading I should write it on my blog or turn it into a webpage. This way when people ask me about it, or if I need to present the information to someone, I can just point them to that single source. This saves me from having to type and retype the same information over and over for different people. A single “master” source with the most up to date information is always optimal.

With that in mind, I decided to whip up a simple site to showcase the scripts that are in my “tilde bin” folder. Scripts that I wrote and use frequently that others might find useful. Sure I’ll still blog about them, but for those who want to download them its so much faster to point them to ~/bin instead of having them scour my blog.

~/bin

And thus ~/bin was born.

Sidenotes

I was quite satisfied with the result. It took only 24 hours from Conception to Creation. I made use of Blueprint and jQuery (AJAX and animation) to create a pretty fresh website. Sure its pretty basic but I felt comfortable the entire way through, and I loved every minute of it. I’m hoping that when Ruby on Rails 3 is released, if I have some spare time, I’ll convert it and include a small admin dashboard.

As of two years ago I’ve decided to get pretty serious with my New Years Resolutions. This year (like last year) I have a few ideas I’m tossing around. It will be a few weeks until I settle on one. Typically the resolutions will help me improve my health or lifestyle. One of the ideas I’m tossing around is blogging at least once per week. Compared to the other alternatives this is certainly the most fun. It has a hint of a challenge and its pretty doable. Let me know what you think!

Dynamic Web URLs with ExpanDrive

Often when I work with ExpanDrive the files I am working on correspond to some website that I own. When I’m mounted with ExpanDrive each file is accessible via “my hard drive” in the mounted volume and, more importantly, from a web URL! I found myself repeatedly opening up my browser and manually typing the URL for files that I just uploaded or edited. This was error prone, especially if some of the characters needed encoding. So, I spent some time to write up a Ruby script that can read ExpanDrive’s preferences, build the file’s “web URL,” and open it in your default browser.

expanurl usage

Simple to Use

I followed along with ExpanDrive’s previous command line tool named expan and named my script expanurl. Given no arguments it will open the current directory via its web URL, or you can give a list of files and each will be opened at their web URLs. Its usage is pretty straightforward but there is a single catch: the server setting in an ExpanDrive Drive may not be a true one-to-one mapping with the web server’s address.

For example: I provide holly.cs.rit.edu as the server value for one of my personal ExpanDrive drives. However, when I view files on that server (inside the public_html directory) they have a much different looking URL: http://www.cs.rit.edu/~jjp1820/. The result? The script simply keeps its own mapping of ExpanDrive server values to their associated web page prefixes. When you use expanurl on a Drive you have never used it on before the script will prompt you for that mapping, store the value, and never ask again.

Here it is in action. I have removed all stored mappings so I can demonstrate what it would be like using expanurl for the first time. Here I use it on my BogoJoker ExpanDrive drive:

expanurl first usage

Notice that in the prompt it tells you:

  • the server that the ExpanDrive volume is linked to and the one you will be providing a web url prefix for
  • an example of a web url prefix (useful)
  • where the mappings are stored in case you need to edit them later

The script is available on GitHub, so feel free to contribute and improve. Here is a link to the always current version, and here is a snapshot of the current version at the time of writing:

 

#!/usr/bin/env ruby
# Author: Joseph Pecoraro
# Date: Saturday December 13, 2008
# Description: When I'm using ExpanDrive and I'm
# remotely logged into a server, I can use this
# script to "open filename" and it will open using
# the server's associated URL.

# For URL Escaping and Stored Mappings
require 'uri'
require 'yaml'

# Exit with a msg
def err(msg)
  puts msg
  exit 1
end

# Simple Class to handle the mappings
class UrlMap
  
  MAP_PATH = File.expand_path('~/.expanurl')
  
  def initialize
    @hash = load
  end
  
  def load
    if File.exists?(MAP_PATH)
      YAML::load_file( File.expand_path(MAP_PATH) )
    else
      Hash.new
    end
  end
  
  def add_mapping(server, mapto)
    @hash[server] = mapto
    File.open(MAP_PATH, 'w') do |file|
      file.write(@hash.to_yaml)
    end
  end
  
  def is_mapping?(server)
    @hash.has_key?(server)
  end
  
  def get_mapping(server)
    @hash[server]
  end
  
  def path
    MAP_PATH
  end
  
end


# Local Variables
mapping = UrlMap.new
url_prefix = nil
server = nil
volume = nil

# Check the if the current directory is an
# ExpanDrive Volume and a public_html folder
pwd = `pwd`
match = pwd.match(/^\/Volumes\/([^\/]+)/)
if match.nil?
  err("Not inside an ExpanDrive Volume")
elsif !pwd.match(/\/public_html\/?/)
  err("Not inside a public_html directory.")
else
  volume = match[1]
  defaults = `defaults read com.magnetk.ExpanDrive Drives`
  defaults.gsub!(/\n/, '')
  props = defaults.match(/\{[^\}]+driveName\s+=\s+#{volume}[^\}]+server\s+=\s+"([^"]+)"[^\}]+\}/)
  if props
    server = props[1]
  else
    err("This Volume (#{volume}) is not an ExpanDrive Volume")
  end
end

# Check if a mapping exists
# Otherwise create and store one
if mapping.is_mapping?(server)
  url_prefix = mapping.get_mapping(server)
else
  
  # Prompt
  puts
  puts "This is the first time you've used expanurl for #{volume}"
  puts "Please Provide us with a mapping for #{server}"
  puts "Mappings are stored in #{mapping.path}"
  puts "Example: http://bogojoker.com/"
  print ">> "
  
  # Store user input and proceed
  url_prefix = gets.chomp
  url_prefix += '/' unless url_prefix.match(/\/$/)
  mapping.add_mapping(server, url_prefix)
  
  # Terminal Output
  puts
  puts "Server: #{server}"
  puts "Maps to: #{url_prefix}"
  puts

end

# Build the URL
subpath = pwd.match(/public_html\/?(.*)/)[1]
subpath += '/' unless subpath.length.zero? || subpath.match(/\/$/)
url_prefix += subpath

# If No Files, open the directory
# Otherwise,   open each provided file
if ARGV.size == 0
  `open #{url_prefix}`
else
  ARGV.each do |filename|
    `open #{url_prefix}#{URI.escape(filename)}`
  end
end

 

How it Works

The Ruby Script grabs the current working directory using `pwd` and checks to make sure you’re in an ExpanDrive volume. ExpanDrive volume’s are dynamically generated by parsing the ExpanDrive preferences thanks to their foresight to make them accessible via the `defaults` command. So if you’re in an ExpanDrive volume and inside a public_html directory expanurl will then use its mapping to open a uri encoded web url in your default browser with the `open` command.

The mappings are stored in a hidden YAML file in your home directory (~/.expanurl). This style of storing preferences is just like dozens of other command line applications and scripts. YAML is just a lightweight textual data format popular with Ruby, similar to JSON and XML. Its so simple that you could edit the file yourself if you wanted/needed to. For instance here is what is in mine, just two simple key/value pairs:

~/.expanurl yaml mapping

The Future

Its just that simple. Being a Ruby Script you can call this from GUI applications, anything with built-in shell access, etc. It should play friendly with your usual Unix tools. I will likely make this script more and more robust if others find it useful, so I’d be happy to hear some feedback.

Cheers.

Ruby Process Controller – psgrep

Every once in a while a process will freeze and will be too stubborn to die when I try to “Quit” it. For those stubborn processes I tend to use the terminal to `kill` it. For a while I had been using a simple Perl script for searching through processes. The script would find me the processids and I could then kill it, using whatever power I need.

I found that it was taking far too long for me to do the search, and then carefully type out the process id, and hope I got the right one. I discovered killall, but the problem is that sometimes I don’t want to kill “all” of the processes with that name. So, I gave in and wrote up a Ruby script that did what I wanted. Here is psgrep: (Download)

#!/usr/bin/env ruby
# Start Date: Saturday December 6, 2008
# Current Version: 0.9
# Author: Joseph Pecoraro
# Contact: joepeck02@gmail.com
# Decription: Quicker handling of process searching
# and killing.  Never type in a PID again, just regexes!

# -----------
#   Globals
# -----------
kill_mode = false
icase_mode = false
pattern = nil
targets = []
pids = []

# ---------
#   Usage
# ---------
def usage
  puts "usage: #{$0.split(/\//).last} [options] pattern"
  puts "  -k  or --kill   kills all processes"
  puts "  -k# or --kill#  kills the [#] process"
  exit 0
end

# -----------
#   Options
# -----------
if ARGV.size > 1
  ARGV.each do |arg|
    if arg.match(/^-(k|-kill)(\d*)$/)
      kill_mode = true
      targets << $2.to_i unless $2.empty?
    elsif arg.match(/^-(i|-ignore)$/)
      icase_mode = true
    end
  end
  ARGV.delete_if { |e| e.match(/^-/) }
end

# -------------------
#   Remaining Args
# -------------------
if ARGV.size != 1
  usage
end

if icase_mode
  pattern = Regexp.new( ARGV[0], Regexp::IGNORECASE )
else
  pattern = Regexp.new( ARGV[0] )
end

# ----------------------
#   Actual `ps` Output
# ----------------------
lines = %x{ ps -Au#{ENV['USER']} }.split(/\n/)
header = lines.shift

# ----------
#   psgrep
# ----------
puts
puts "     #{header}"
count = 0
lines.each do |line|
  unless line =~ /psgrep/
    if line.match(pattern)
      count += 1
      puts "[#{count}]: #{line}"
      if targets.empty? || targets.member?(count)
        pids << line.strip.split[1]
      end
    end
  end
end

# -------------
#   Kill Mode
# -------------
if kill_mode
  puts
  puts "Killing Processes"
  puts "-----------------"
  pids.each_with_index do |pid, i|
    print targets.empty? ? "[#{i}]:" :  "[#{targets[i]}]:"
    print " Killing #{pid}... "
    STDOUT.flush
    res = %x{ kill #{pid} }
    puts "Dead" if $?.exitstatus.zero?
  end
end

# Always
puts

So there it is. Less then 100 lines of ruby to get a pretty straightforward psgrep/kill program. Here is an example where I have three perl processes running on my machine. One of the is running as root (the userid is 0). I just type “!! –kill” or “[up-arrow] –kill” and it tries to kill them all. Note that the root perl process doesn’t terminate and there is an error message but psgrep continues as best as it can, and kills the two normal perl processes: [Note: I could have done `sudo psgrep perl -k` to kill the root process]

psgrep usage

Here is another good example. I have two python instances that I want to kill but there is another python instance running ExpanDrive in the background. I just ran psgrep and found the two I want to kill are [2] and [3]. Therefore, I can send -k2 and -k3 (or –kill2 and –kill3) to kill only those processes. Here is the result:

psgrep target

Note also that by default psgrep is case-sensitive. To ignore case just add the -i or –ignore switch. So there you have it. Usage is straightforward. Switches can go anywhere on the command line makes it easier to just use your history and tack a switch on the end of your previous command.

Feel free to improve it, it is on GitHub!

Stack Overflow – Edit Summary Quicklinks

I requested a feature to make the few edit summary suggestions clickable. I, like many developers, don’t normally let my keys leave the keyboard. However, this was one case where I felt making those suggestions of “corrected spelling” and “fixed grammar” should automatically be inserted.

Showing the Usage

Well the suggestion was declined. I can’t blame the team. Nobody upvoted the suggestion. But I felt strongly enough about it, and knew that it was very simple to implement that I whipped up a GreaseMonkey script to do it myself. The script runs like a charm and even adds a few extra suggestions to the original three. It handles formatting and commas all automatically, so don’t worry about a thing, just click. Enjoy!

Script to Add Edit Summary Quicklinks

Script to Prevent Blank Edit Summaries

UPDATE: Fixed to use keng’s URL and added just plain old stackoverflow.com without the beta sub-domain in preparation for a launch. Thanks keng!

DOUBLE UPDATE: Sam put out a “wanted ad” for a Greasemonkey script to prevent blank edit summaries. I whipped that script up and linked to it up above. Thanks Sam! Quick Demo

JavaScript Sort an Array of Objects

I ran into an interesting problem today. I had an array of objects that I wanted sorted on a certain property. My obvious thought didn’t work! (Update: I got a comment below from Peter Michaux who points out a nicer solution, it is included here:)

// Array of Objects
var obj_arr = [ { age: 21, name: "Larry" },
                { age: 34, name: "Curly" },
                { age: 10, name: "Moe" } ];

// This doesn't work!
obj_arr.sort( function(a,b) { return a.name < b.name; });

// This does work! (Peter's update, very fast)
obj_arr1.sort(function(a,b) { return a.name < b.name ? -1 :
                                     a.name > b.name ?  1 : 0; });

That kind of frustrated me. Sorting is one of those things I expect to be available in all languages. I don’t want to have to write a sorting algorithm every time I need to sort. So I looked into things, pulled up a Javascript Quicksort Algorithm and manipulated it to support any compare function.

Now that I have the freedom to truly write a compare function that works for objects! I also changed around certain parts of the code I found online to actually extend the Array class and make the extra functions hidden. Take a look at the sample usage:

// Defaults to (a<=b) sorting.  Great for numbers.
var arr = [1234, 2346, 21234, 3456, 32134, 3456, 1234, 2345, 23, 42523, 1234, 345];

// Object Array
var obj_arr = [ { age: 21, name: "Larry" },
                { age: 34, name: "Curly" },
                { age: 10, name: "Moe" } ];

arr.quick_sort();
// => [23, 345, 1234, 1234, 1234, 2345, 2346, 3456, 3456, 21234, 32134, 42523]

obj_arr.quick_sort(function(a,b) { return a.name < b.name });
// => Curly, Larry, Moe

obj_arr.quick_sort(function(a,b) { return a.age < b.age });
// => Moe (10), Larry (21), Curly (34)

For those who want to see the code be glad, its free. I carried the copyright with it but its rather loose. Grab the JavaScript Source Here! Enjoy:

Array.prototype.swap=function(a, b) {
  var tmp=this[a];
  this[a]=this[b];
  this[b]=tmp;
}

Array.prototype.quick_sort = function(compareFunction) {

  function partition(array, compareFunction, begin, end, pivot) {
    var piv = array[pivot];
    array.swap(pivot, end-1);
    var store = begin;
    for (var ix = begin; ix < end-1; ++ix) {
      if ( compareFunction(array[ix], piv) ) {
        array.swap(store, ix);
        ++store;
      }
    }
    array.swap(end-1, store);
    return store;
  }

  function qsort(array, compareFunction, begin, end) {
    if ( end-1 > begin ) {
      var pivot = begin + Math.floor(Math.random() * (end-begin));
      pivot = partition(array, compareFunction, begin, end, pivot);
      qsort(array, compareFunction, begin, pivot);
      qsort(array, compareFunction, pivot+1, end);
    }
  }

  if ( compareFunction == null ) {
    compareFunction = function(a,b) { return a<=b; };
  }
  qsort(this, compareFunction, 0, this.length);

}

Update

Peter Michaux pointed out something very important. The sort() function can be made to work if it returns numeric output (-1,0,1). His approach is far superior. Here was a benchmark I took:

var obj_arr1 = [];
var obj_arr2 = [];
var filler = [ { age: 21, name: "Larry" },
               { age: 34, name: "Curly" },
               { age: 10, name: "Moe" } ];
for (var i=0; i<5000; i++) {
  rand = Math.floor( Math.random() * 3 );
  obj_arr1.push( filler[rand] );
  obj_arr2.push( filler[rand] );
}

var s = new Date();
obj_arr1.sort(function(a,b) { return a.name < b.name ? -1 : a.name > b.name ? 1 : 0; });
var e = new Date();
console.log(e.getTime()-s.getTime()); // => 75 ms

s = new Date();
obj_arr2.quick_sort(function(a,b) { return a.name < b.name });
e = new Date();
console.log(e.getTime()-s.getTime()); //  => 4444 ms

That shows drastic differences for arrays as large as 5000 elements (with not too random data). 75 ms versus 4444 ms (over 4 seconds). Doing the math: (4444/75) => 59.253 times better! Moral of the story, don’t rush into thinking something doesn’t exist!

So if that’s the way to do it, then I want to make it easier on me. My arrays are generally going to be under 100 in size, and at such a size building a function dynamically instead of writing a custom function works just about as well (although if you were using objects, polymorphism and a compare function would be the best way to go). Here is a simple function I can use to more quickly build compare functions in order to ascend sort an array on multiple properties!

function buildCompareFunction(arr) {
  if (arr && arr.length > 0) {
    return function(a,b) {
      var asub, bsub, prop;
      for (var i=0; i<arr.length; i++) {
        prop = arr[i];
        asub = a[prop];
        bsub = b[prop];
        if ( asub < bsub )
          return -1;
        if ( asub > bsub )
          return 1;
      }
      return 0;
    }
  } else {
    return function(a,b) { return a<=b; };
  }
}

Sample usage would be:

var obj_arr = [
  { name: 'Joe',   age: 20 },
  { name: 'Joe',   age: 10 },
  { name: 'Joe',   age: 30 },
  { name: 'Joe',   age: 40 },
  { name: 'Joe',   age: 20 },
  { name: 'Joe',   age: 15 },
  { name: 'Joe',   age: 35 },
  { name: 'Joe',   age: 25 },
  { name: 'Bill',  age: 5 },
  { name: 'Barry', age: 20 },
  { name: 'Paul',  age: 20 },
  { name: 'Peter', age: 1 },
  { name: 'Smith', age: 25 },
  { name: 'Kary',  age: 30 }
];

obj_arr.sort( buildCompareFunction(['name','age']) );

search