There are a few tree scripts available online. Some as simple as find | sed and others are slightly more advanced like a python script. I wasn’t pleased with the existing solutions, so I wrote my own. To get an idea of what I’m talking about take a look at this screenshot showing the listing of a Rails project:

Its a simple, clean listing of the directory tree. I will admit, the style is based off of another tree script that I’ve seen that I liked. Also, this isn’t really production quality code. I take the lazy way out and first get a directory listing and then work from there. This means that for large directories there may be an initial pause before it starts outputting. I wouldn’t suggest running this on your home directory. Although I can think of better algorithms its unlikely that I would want to run this on huge directories so I’m more then happy right now.

The usage is pretty bare bones:

This is one more script I’ve added to my ~/bin and its completely open source on GitHub. Its just straight Ruby, no extra packages, works with 1.8 and 1.9. Oh and did I mention its customizable?

I hope you like it.

I actually wrote a script where I wanted to change the command line arguments if I was in a pipe. This is probably confusing but its really useful! So there is actually an easy way to do this in Ruby:

#!/usr/bin/env ruby
# Prints if the input or output is regular or piped
puts "INPUT:  #{STDIN.tty?  ? "regular" : "pipe"}"
puts "OUTPUT: #{STDOUT.tty? ? "regular" : "pipe"}"

Proof that it works as advertised:

The documentation on IO#tty? says the following:

Returns true if [the IO stream] is associated with a terminal device (tty), false otherwise.

For those that don’t know, tty is short for teletype writer. This is an old Unix term for an interactive process that takes in user input. Nowadays it almost always means a shell/console/terminal.

I’m not going to go over HTTP Headers. thats already been done. Instead I’m going to focus on debugging and working with them a little bit. I’m going to assume you have already the general concepts.

The Problem

I wanted to add pagination to the skreemr shell. So you could run a search, then get the next page of results, etc. So a little experimentation with skreemr in my browser produced the following URLs:

http://skreemr.com/results.jsp?q=test

http://skreemr.com/results.jsp?q=test&l=10&s=10

http://skreemr.com/results.jsp?q=test&l=10&s=20

A simple pattern! “q” is the query string, “l” is the number per page, “s” is the number to start at, indexed from 0. So that last URL would produce results 21-30. This all worked well in my browsers, but it wasn’t working in Ruby:

require 'open-uri'

# Grab the pages and read the content
str1 = open( 'http://skreemr.com/results.jsp?q=test'      ).read  # 1-10
str2 = open( 'http://skreemr.com/results.jsp?q=test&s=10' ).read  # 11-20

#=> Should print false... but its printing true!
puts str1==str2

What this was saying was that the content being returned from both of those urls is EXACTLY the same. That couldn’t be… could it?

More Investigation

At this point I thought it was a problem in Ruby. I figured Ruby was doing some caching in the background that I was going to have to disable or work around. (I now know this is not true, but that was my first guess). To test that hypothesis I turned to my trusty friend curl and checked to see if that showed the proper behavior:

shell> curl 'http://skreemr.com/results.jsp?q=test'      -o 1.html
shell> curl 'http://skreemr.com/results.jsp?q=test&s=10' -o 2.html
shell> diff -q -s 1.html 2.html
Files 1.html and 2.html are identical

What!?! That stunned me. Curl was getting the exact same results as Ruby. I took a look at the html files, and indeed 2.html, which should have contained results 11-20 held 1-10. I opened both urls in my browser… they showed the correct results. Something weird was happening!

Take A Step Back

At this point you’ve got to know what is happening. Curl and Ruby (using Ruby’s Net:HTTP under the hood) are just making a simple GET request. Both my browsers Safari and Firefox are sending far more then just a GET request. They are sending a bunch of other headers. Lets take a look at what Firebug says Firefox sent:

Thats a mighty long list of headers! Its entirely possible that one of those might be influencing the server. Onto the drama!

Time to Act – Literally

The only difference between what curl sent in its request and what Firefox was sending is that header information and possibly any information contained in those cookies (unlikely in this case). So lets make curl act as if it is Firefox by send the same headers with curl! I took a quick peek at my curl reference for the proper switches/formatting and I was ready:

shell> curl 'http://skreemr.com/results.jsp?q=test' -o 1.html
shell> curl 'http://skreemr.com/results.jsp?q=test&s=10'            \
            -H 'Host: skreemr.com'                                  \
            -H 'User-Agent: Mozilla/5.0 (...) Firefox/3.0.6'        \
            -H 'Accept: text/html,application/xml;q=0.9,*/*;q=0.8'  \
            -H 'Accept-Language: en-us,en;q=0.5'                    \
            -H 'Accept-Encoding: gzip,deflate'                      \
            -H 'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7'     \
            -H 'Keep-Alive: 300'                                    \
            -H 'Connection: keep-alive'                             \
            -H 'Cache-Control: max-age=0'                           \
            -o 2.html
shell> diff -q -s 1.html 2.html
Files 1.html and 2.html differ

Jackpot! Some subset of those headers is indeed fixing my problem, because now it properly fetched the second page of results! Translating that back into Ruby works as well:

require 'open-uri'

# Headers
HEADERS = {
  'Host'            => 'skreemr.com',
  'User-Agent'      => 'Mozilla/5.0 (...) Firefox/3.0.6',
  'Accept'          => 'text/html,application/xml;q=0.9,*/*;q=0.8',
  'Accept-Language' => 'en-us,en;q=0.5',
  'Accept-Encoding' => 'gzip,deflate',
  'Accept-Charset'  => 'ISO-8859-1,utf-8;q=0.7,*;q=0.7',
  'Keep-Alive'      => '300',
  'Connection'      => 'keep-alive',
  'Cache-Control'   => 'max-age=0'
}

# Get the Pages
str1 = open( 'http://skreemr.com/results.jsp?q=test', HEADERS ).read
str2 = open( 'http://skreemr.com/results.jsp?q=test&s=10', HEADERS ).read

# Correctly Prints false
puts str1==str2

Problem Solved

Sure, we have a solution but its not pretty. How much is really needed? Finding that out is just a matter of eliminating the headers that do nothing. Remove one, test, remove another, test, remove another test… As long as it still works after you’ve removed an individual header you know that header wasn’t needed. It took me a few minutes but I narrowed it down to just a single header:

MAGIC = { 'Accept-Language' => 'en-us' }

This was a rather unique problem. It is rare for me to have to dip down to the HTTP Headers to see what is actually happening for such a high level problem. However, as this problem shows, its important to know what is going on under the hood. If I didn’t know about headers, I would not have been able to solve it.
]]> http://blog.bogojoker.com/2009/03/debugging-http-headers/feed/ 2 http://blog.bogojoker.com/2009/02/mini-mp3-searching-shell-skreemr/ http://blog.bogojoker.com/2009/02/mini-mp3-searching-shell-skreemr/#comments Mon, 23 Feb 2009 15:30:49 +0000 Joseph Pecoraro http://blog.bogojoker.com/?p=493 So its way to hard to download an mp3 in Safari. Right click the link and download? Pff, I want to ⌃S and be done with it. Well, this time I decided to avoid the problem all together. I use Skreemr to search for a particular song when it interests me.

In the past I wrote a little bash script, that makes use of curl, to download an mp3 to my desktop unique named so it wouldn’t have conflicts. This shell essentially wraps and drastically improves that to allow for searching, pagination, history, downloading, and opening mp3s off of Skreemr. It gives me just what I need. The functionality that I want without having to use torrents etc. I’m thinking of turning this into a gem.

This script requires the popular “escape.rb” script that gives some nice and safe shell escaping functions. You can download both from my GitHub scripts project.

Of course its available on my ~/bin and there will be another article later on that goes over a few aspects of this simple little script.

  # Check Cmd Line Args and Print Usage if needed
  if ARGV.size <= 1
    puts "usage: line [options] filename numbers"
    puts
    puts " options:"
    puts "   --silent or -s   Just print the line, without [#]"
    puts
    puts " number formats:"
    puts "   1-3              Prints lines 1 through 3"
    puts "   5                Prints line 5"
    puts "   -1               Prints the last line, (negative)"
    puts
    puts " extra formats:"
    puts "   ~5               Prints 2 (default) lines before and after 5"
    puts "   4~10             Prints 4 lines before and after 10"
    puts "   *7 or 8*         Prints all lines before or after the number"
    puts "   5/1              Prints 5 lines, then skips 1 line..."
    puts "   2:5/1            Starts at line 2, prints and skips..."
    puts
    exit 1

So there is some cool new syntax:

• ~# – prints 2 lines before and after the given line number

• #~# – prints the given number of lines before and after the given line number

• *# – prints all of the lines before the given line number

• #* – prints all of the lines after the given line number

• #/# – is a print and skip option

• #:#/# – allows you to provide an offset

Lets say you want to print every even line in a text file. With this script that is not a problem. The solution is: starting on the 2nd line, print 1 line, then skip 1 line. (line file 2:1/1):

I think these new formats are pretty neat. You can always access the latest version of the script on Github or in my ~/bin. I’m willing to change the syntax and introduce new concepts. Send suggestions my way by dropping a comment! Cheers.

After writing the script to print out a single line, I soon found it made a lot of sense to include ranges. Taking advantage of Ruby I can even print lines from the end with negative numbers. So I spent a few minutes to clean up the script and make it a little more reusable, add some formatting, and cleaner. Here was the result:

line is now the latest addition to my ~/bin.

Also, in case you wanted to use this in a shell script or via piping, there is a `–silent` or `-s` switch that you can use that removes all of the special formatting, and prints only the specified lines. Much nice for scripts. (See the highlighted line in the image below). Enjoy!

So what exactly is line 671 of /etc/services? On my mac it is:

Simple to Use

I followed along with ExpanDrive’s previous command line tool named expan and named my script expanurl. Given no arguments it will open the current directory via its web URL, or you can give a list of files and each will be opened at their web URLs. Its usage is pretty straightforward but there is a single catch: the server setting in an ExpanDrive Drive may not be a true one-to-one mapping with the web server’s address.

For example: I provide holly.cs.rit.edu as the server value for one of my personal ExpanDrive drives. However, when I view files on that server (inside the public_html directory) they have a much different looking URL: http://www.cs.rit.edu/~jjp1820/. The result? The script simply keeps its own mapping of ExpanDrive server values to their associated web page prefixes. When you use expanurl on a Drive you have never used it on before the script will prompt you for that mapping, store the value, and never ask again.

Here it is in action. I have removed all stored mappings so I can demonstrate what it would be like using expanurl for the first time. Here I use it on my BogoJoker ExpanDrive drive:

Notice that in the prompt it tells you:

• the server that the ExpanDrive volume is linked to and the one you will be providing a web url prefix for
• an example of a web url prefix (useful)
• where the mappings are stored in case you need to edit them later

The script is available on GitHub, so feel free to contribute and improve. Here is a link to the always current version, and here is a snapshot of the current version at the time of writing:

#!/usr/bin/env ruby
# Author: Joseph Pecoraro
# Date: Saturday December 13, 2008
# Description: When I'm using ExpanDrive and I'm
# remotely logged into a server, I can use this
# script to "open filename" and it will open using
# the server's associated URL.

# For URL Escaping and Stored Mappings
require 'uri'
require 'yaml'

# Exit with a msg
def err(msg)
  puts msg
  exit 1
end

# Simple Class to handle the mappings
class UrlMap

  MAP_PATH = File.expand_path('~/.expanurl')

  def initialize
    @hash = load
  end

  def load
    if File.exists?(MAP_PATH)
      YAML::load_file( File.expand_path(MAP_PATH) )
    else
      Hash.new
    end
  end

  def add_mapping(server, mapto)
    @hash[server] = mapto
    File.open(MAP_PATH, 'w') do |file|
      file.write(@hash.to_yaml)
    end
  end

  def is_mapping?(server)
    @hash.has_key?(server)
  end

  def get_mapping(server)
    @hash[server]
  end

  def path
    MAP_PATH
  end

end

# Local Variables
mapping = UrlMap.new
url_prefix = nil
server = nil
volume = nil

# Check the if the current directory is an
# ExpanDrive Volume and a public_html folder
pwd = `pwd`
match = pwd.match(/^\/Volumes\/([^\/]+)/)
if match.nil?
  err("Not inside an ExpanDrive Volume")
elsif !pwd.match(/\/public_html\/?/)
  err("Not inside a public_html directory.")
else
  volume = match[1]
  defaults = `defaults read com.magnetk.ExpanDrive Drives`
  defaults.gsub!(/\n/, '')
  props = defaults.match(/\{[^\}]+driveName\s+=\s+#{volume}[^\}]+server\s+=\s+"([^"]+)"[^\}]+\}/)
  if props
    server = props[1]
  else
    err("This Volume (#{volume}) is not an ExpanDrive Volume")
  end
end

# Check if a mapping exists
# Otherwise create and store one
if mapping.is_mapping?(server)
  url_prefix = mapping.get_mapping(server)
else

  # Prompt
  puts
  puts "This is the first time you've used expanurl for #{volume}"
  puts "Please Provide us with a mapping for #{server}"
  puts "Mappings are stored in #{mapping.path}"
  puts "Example: http://bogojoker.com/"
  print ">> "

  # Store user input and proceed
  url_prefix = gets.chomp
  url_prefix += '/' unless url_prefix.match(/\/$/)
  mapping.add_mapping(server, url_prefix)

  # Terminal Output
  puts
  puts "Server: #{server}"
  puts "Maps to: #{url_prefix}"
  puts

end

# Build the URL
subpath = pwd.match(/public_html\/?(.*)/)[1]
subpath += '/' unless subpath.length.zero? || subpath.match(/\/$/)
url_prefix += subpath

# If No Files, open the directory
# Otherwise,   open each provided file
if ARGV.size == 0
  `open #{url_prefix}`
else
  ARGV.each do |filename|
    `open #{url_prefix}#{URI.escape(filename)}`
  end
end

How it Works

The Ruby Script grabs the current working directory using `pwd` and checks to make sure you’re in an ExpanDrive volume. ExpanDrive volume’s are dynamically generated by parsing the ExpanDrive preferences thanks to their foresight to make them accessible via the `defaults` command. So if you’re in an ExpanDrive volume and inside a public_html directory expanurl will then use its mapping to open a uri encoded web url in your default browser with the `open` command.

The mappings are stored in a hidden YAML file in your home directory (~/.expanurl). This style of storing preferences is just like dozens of other command line applications and scripts. YAML is just a lightweight textual data format popular with Ruby, similar to JSON and XML. Its so simple that you could edit the file yourself if you wanted/needed to. For instance here is what is in mine, just two simple key/value pairs:

The Future

Its just that simple. Being a Ruby Script you can call this from GUI applications, anything with built-in shell access, etc. It should play friendly with your usual Unix tools. I will likely make this script more and more robust if others find it useful, so I’d be happy to hear some feedback.

Cheers.

I found that it was taking far too long for me to do the search, and then carefully type out the process id, and hope I got the right one. I discovered killall, but the problem is that sometimes I don’t want to kill “all” of the processes with that name. So, I gave in and wrote up a Ruby script that did what I wanted. Here is psgrep: (Download)

#!/usr/bin/env ruby
# Start Date: Saturday December 6, 2008
# Current Version: 0.9
# Author: Joseph Pecoraro
# Contact: joepeck02@gmail.com
# Decription: Quicker handling of process searching
# and killing.  Never type in a PID again, just regexes!

# -----------
#   Globals
# -----------
kill_mode = false
icase_mode = false
pattern = nil
targets = []
pids = []

# ---------
#   Usage
# ---------
def usage
  puts "usage: #{$0.split(/\//).last} [options] pattern"
  puts "  -k  or --kill   kills all processes"
  puts "  -k# or --kill#  kills the [#] process"
  exit 0
end

# -----------
#   Options
# -----------
if ARGV.size > 1
  ARGV.each do |arg|
    if arg.match(/^-(k|-kill)(\d*)$/)
      kill_mode = true
      targets << $2.to_i unless $2.empty?
    elsif arg.match(/^-(i|-ignore)$/)
      icase_mode = true
    end
  end
  ARGV.delete_if { |e| e.match(/^-/) }
end

# -------------------
#   Remaining Args
# -------------------
if ARGV.size != 1
  usage
end

if icase_mode
  pattern = Regexp.new( ARGV[0], Regexp::IGNORECASE )
else
  pattern = Regexp.new( ARGV[0] )
end

# ----------------------
#   Actual `ps` Output
# ----------------------
lines = %x{ ps -Au#{ENV['USER']} }.split(/\n/)
header = lines.shift

# ----------
#   psgrep
# ----------
puts
puts "     #{header}"
count = 0
lines.each do |line|
  unless line =~ /psgrep/
    if line.match(pattern)
      count += 1
      puts "[#{count}]: #{line}"
      if targets.empty? || targets.member?(count)
        pids << line.strip.split[1]
      end
    end
  end
end

# -------------
#   Kill Mode
# -------------
if kill_mode
  puts
  puts "Killing Processes"
  puts "-----------------"
  pids.each_with_index do |pid, i|
    print targets.empty? ? "[#{i}]:" :  "[#{targets[i]}]:"
    print " Killing #{pid}... "
    STDOUT.flush
    res = %x{ kill #{pid} }
    puts "Dead" if $?.exitstatus.zero?
  end
end

# Always
puts

So there it is. Less then 100 lines of ruby to get a pretty straightforward psgrep/kill program. Here is an example where I have three perl processes running on my machine. One of the is running as root (the userid is 0). I just type “!! –kill” or “[up-arrow] –kill” and it tries to kill them all. Note that the root perl process doesn’t terminate and there is an error message but psgrep continues as best as it can, and kills the two normal perl processes: [Note: I could have done `sudo psgrep perl -k` to kill the root process]

Here is another good example. I have two python instances that I want to kill but there is another python instance running ExpanDrive in the background. I just ran psgrep and found the two I want to kill are [2] and [3]. Therefore, I can send -k2 and -k3 (or –kill2 and –kill3) to kill only those processes. Here is the result:

Note also that by default psgrep is case-sensitive. To ignore case just add the -i or –ignore switch. So there you have it. Usage is straightforward. Switches can go anywhere on the command line makes it easier to just use your history and tack a switch on the end of your previous command.

Feel free to improve it, it is on GitHub!

1. RDoc – Ruby Autogenerated Documentation
2. RSpec – Ruby Test Framework aiding Behavior Driven Development
3. General Familiarity with Ruby Classes
4. General Familiarity with GitHub

I have to say that I was really impressed with how strikingly natural, easy, and fun it was to work with these tools. I already wrote about RDoc, hopefully to cover a “void” that I saw in the online documentation for it. I may look into writing about RSpec, however the current RSpec documentation was quite good so I may focus elsewhere. Finally GitHub and Ruby are mostly things that you have to personally practice with to get good at, and there are already plenty of great resources for them. The Ruby community has done a very good job!

The ARGFy Results

So, here are the results of my experiment:

• ARGFy GitHub Repository
• ARGFy RDoc Documentation

The GitHub README is very similar to the RDoc, but it goes in more depth by showing the output of the sample.rb script included. Its not too exciting, but here is what ARGFy does.

What ARGFy Does

ARGFy is a class. In the constructor it takes an Array of filenames. It then treats those files as one continuous stream of lines. If no filenames are provided, or if “-” is provided as a filename, that input is treated as STDIN. Everything so far makes ARGFy look and act just like ARGF except you can specify your own files instead of only relying on the command line arguments.

Using ARGFy is mostly like ARGF. If you call the ARGFy#each method (note that this allows for any Enumerable method!) it will exhaust all the lines of input from all the files as a single stream. At each line you can check the states of the ARGFy object itself. The states include filename and lineno like the normal ARGF, but they also include filelineno. Because there is a filelineno there is a guaranteed way to know if under-the-hood the stream is now processing a different file. Since this might be a common thing to check there is a ARGFy#new_file? helper method that does just that.

Finally, because its an object you can add a file to the list at any time. Although removing didn’t seem to make much sense considering what its purpose was. Just make use of ARGFy#add_file to add a file to the end of the sequence of input files to the stream.

In the background ARGFy is really just reading and buffering the files one at a time and returning the lines. Its nothing too exciting, just a little fun working with Ruby. The example nicely displays how ARGFy works:

# sample.rb
require 'ARGFy'

argf = ARGFy.new(ARGV)
argf.each do |line|

  # Per File Header
  if argf.new_file?
    filename = argf.filename
    filename = "STDIN" if filename == "-"
    puts '', filename, "-"*filename.length
  end

  # Print out the line with line numbers
  puts "%3d: %s" % [argf.lineno, line]

end
puts

Calling sample.rb with a few small input files creates some nicely formatted output:

shell> ruby sample.rb in1.txt in2.txt 

in1.txt
-------
  1: one
  2: two

in2.txt
-------
  1: alpha, beta, gamma
  2: 0987654321
  3:
  4: NOT BLANK!

Nothing complex. It works like you would expect it too. For more sample usage you can scan the RSpec test cases in the GitHub repository.

Consistency is Nice

The main advantage I see with automatically generated documentation is that it is consistent. Take Javadocs for instance. They are all the same. When a developer wants to work with a Java library, they expect Javadocs. Why? Because they are familiar with them. They can easily navigate them and quickly find whatever it is that they are looking for. Documentation in any other way would require wasting time learning how to use/navigate it searching for what you want to know.

RDoc is Ruby’s documentation generator. You see RDoc generated documentation all over the place in Ruby. See YAML, Hpricot, or even core classes like Array.

So, I felt if I want to continue using Ruby I should at least learn how its handled. It turns out that its easier then I thought. I’m a huge fan of Markdown syntax and RDoc turns out to be pretty close to that. So, here is what I think is all you need to know to handle producing some simple, yet thorough, documentation for a class.

RDoc Resources

Start by updating your rdoc. The latest version at the time of writing is 2.2.1. The gem provides you with the rdoc and ri tools so that you can both generate and display documentation from the command line. Here is how you can install them:

shell> sudo gem install rdoc

The best online resources I found were not surprisingly:

• RDoc’s documentation itself – an RDoc on RDoc… actually its RDoc’s classes but its overview is helpful.
• An Older Writeup by Dave Thomas – has a similar example
• Markdown Syntax – Very similar to what is used in RDoc comments

Structure

Here is a basic example that shows the structure of the RDoc as it describes a File, Class, Attributes, and Methods. The placement of the comments is important. RDoc comments are always on top of what they are documenting:

# Documentation for the file itself
# There should be a blank line between this and any class
# definition to separate the documentation about the file
# and the class.  If there is no space then the entire text
# is used for both the file and the class, no different.

# Documentation for the class itself.
# This will appear at the top of the page specific to this
# class, before any other content.
class Dice

  # Documentation for an attribute
  # To documentation each attribute you must make individual
  # calls to attr_accessor, attr_reader, and attr_writer.
  # Appears next to the attribute name in the attrs section
  attr_accessor :sides

  # Documentation for the constructor
  # Corresponds to the `new` method
  def initialize(sides)
    @sides = sides
  end

  # Documentation for a method
  def roll(times)
    Array.new(times).map { 1+rand(@sides) }
  end

  # Documentation for a method
  def beat(num)
    roll(1).first > num
  end

end

Running `rdoc` on that file creates this documentation.

Style

Rich documentation makes the important parts stand out. It makes use of HTML’s expressive power and enables lists, headers, links, bold/italics, code, and other presentation helpers. I’ll now document the Dice class and add some style and realistic content.

# == sample.rb
# This file contains the Dice class definition and it runs
# some simple test code on a 16 sided dice.  A 20 dice
# roll fight again the COMPUTER who always rolls 10s!

#
# Multi-sided dice class.  The number of sides is determined
# in the constructor, or later on by accessing the _sides_
# attribute.
#
# == Summary
#
# A #single_roll returns a single integer from 1 to the
# number of sides, _inclusive_.  However, if you want to
# roll multiple times you can can use the #roll method,
# specifying the number of rolls you want, and you will
# get an Array with the values of all the rolls!
#
# == Example
#
#    dice = Dice.new(8)   # An eight sided dice
#    four = dice.roll(4)  # An Array containing 4 rolls
#    sum  = four.inject(0) { |mem,i| mem+i } # Sum of rolls
#
# == Contact
#
# Author::  Joseph Pecoraro (mailto:joepeck02@gmail.com)
# Website:: http://blog.bogojoker.com
# Date::    Saturday November 29, 2008
#
class Dice

  # Number of sides on the dice
  attr_accessor :sides

  # Create a dice with `sides` of dice.
  # Defaults to 6.
  def initialize(sides=6)
    @sides = sides
  end

  # Returns an array of size `times` containing
  # a group of dice rolls.
  def roll(times)
    Array.new(times).map { single_roll }
  end

  # Returns the value of a single dice roll.  The
  # values are from 1 to @sides _inclusive_.
  def single_roll
    1+rand(@sides)
  end

  # A single roll challenge:
  # * makes a single_roll
  # * returns true if the roll was strictly greater
  #   then the given number
  # * returns false otherwise
  def beat(num)
    single_roll > num
  end

end

# Note that this is a constant, which is special
# and it is documented like a Class Attribute.
# This is in the RDoc generated documentation for
# the file.
COMPUTER = 10

# Note that these comments, for generic code
# are not in the RDoc generated documentation.
dice = Dice.new(16)
winCount = loseCount = 0
20.times do
  if dice.beat(COMPUTER)
    winCount += 1
  else
    loseCount += 1
  end
end

# Output
puts "You won #{winCount} times and lost #{loseCount} times!"
puts "Muhahah.  Try again later!!"           if winCount < loseCount
puts "Well Played.  I'll get you next time." if winCount > loseCount
puts "What a match!  Boy that was fun."      if winCount == loseCount

That generates this documentation.

Specifics

There are some subtle points that make this documentation format nicely. I’ll point them out and explain them. Most of this is straight from the above resources, however some of it I could not find documented anywhere.

• The file documentation links to the Dice class. Furthermore the Class documentation links down to the single_roll and roll methods. This is because:
  

  Names of classes, source files, and any method names containing an underscore or preceded by a hash character are automatically hyperlinked from comment text to their description.

  1. sample.rb was a filename and so it was automatically linked.
  2. Dice was the name of a class and so it was automatically linked.
  3. single_roll had an underscore and happened to be a method name so it was automatically linked in a few places.
  4. #roll had a hash character signifying that it should be linked.
• Sections begin with a “=” or a “==”. I prefer to use double, because it stands out more in the source code. Technically a single “=” becomes a level 1 header, and a double becomes a level 2 header. However, they both display the same.
• URIs like http://blog.bogojoker.com and mailto:email are automatically turned into links and formatted nicely.
• Bold, Italics, and Typewriter Text can be quickly formated much like Markdown:
  

  _italic_ or <em>italic</em>
  *bold* or <b>bold</b>
  +typewriter+ or <tt>typewriter</tt>

• Code is displayed if each line
• Tabular Labeled List, like the Contact information, are formatted like:
  

  label:: description 1
  label2:: both descriptions will line up

• Formatting source code is like Markdown. The code that you want formatted must be indented with a few spaces. As long as the indention is maintained the text will display as source code in the HTML documentation.
• Formatting lists is again like Markdown. Just use *’s or -’s and they will turn into bullet points. For numbered lists just use numbers followed by a dot and they will be formatted automatically.

Final Notes on `rdoc` itself

When I created the final documentation above I used a few of rdoc’s command line switches to customize the output. What I actually used was:

shell> rdoc --title="Dice Documentation" --line-numbers --tab-width=2

The title switch changed the <title> for the documentation page, and the other two deal with formatting the htmlized source code that RDoc shows when you click on the function name to view the source in the documentation. There are plenty of command line switches. To view the full list do:

shell> rdoc --help

A few useful switches are “–ri” to create ri documentation so you can access your classes from the command line. Also you can output to several formats. For instance you can make a PDF using “–format=texinfo” then using `texi2pdf` on the texinfo file. The PDF doesn’t look that bad, here is my example as a PDF.

NOTE: Finding the generators was tricky. I had to check out the rdoc source code and find the different generators. If anyone knows an easier way to check what generators are available, please let me know.

I hope this helps some people using RDoc for their classes. Enjoy.

DATA

Although this is mostly useless, its a neat trick. In any Ruby script, as soon as the __END__ symbol is matched, then the rest of the text in the file is no longer parsed by the interpreter. Whatever is after __END__ can be accessed via DATA. DATA acts like a File Object, so its like you’re reading the current script as though you’re reading from a File.

# DATA is a global that is actually a File object
# containing the data after __END__ in the current
# script file.
puts DATA.read

__END__
I can put anything I want
after the __END__ symbol
and access it with the
DATA global.  Whoa!

ARGF

ARGF takes each of the elements in ARGV, assumes they are filenames, and allows you to process these files as single stream of input. This is common with shell programs. Its a lot like cat. cat takes multiple files on the command line, and outputs them as a single stream. If you want to force input to come from STDIN then just provide a hypen “-”. Finally, if there is nothing in ARGV then ARGF defaults to STDIN.

Here is a simple example of ARGF mimicking cat.

shell> echo "inside a.txt" > a.txt
shell> echo "inside b.txt" > b.txt
shell> cat a.txt b.txt
inside a.txt
inside b.txt

Here is a Ruby script that can do just that:

# cat.rb
ARGF.each do |line|
  puts line
end

Example usage:

shell> ruby cat.rb a.txt b.txt
inside a.txt
inside b.txt

ARGF Confusion

What confused me when I first used ARGF was that it has no special class. It claims it is an Object. Take a look:

>> ARGF.class
# => Object

But at the same time it has so much more then a regular Object:

>> ARGF.methods - Object.methods
# => ["select", "lineno", "readline", "eof", "each_byte", "partition", "lineno=", "read", "fileno", "grep", "to_i", "filename", "reject", "readlines", "getc", "member?", "find", "to_io", "each_with_index", "eof?", "collect", "path", "all?", "close", "entries", "tell", "detect", "zip", "rewind", "map", "file", "any?", "sort", "min", "seek", "binmode", "find_all", "each_line", "gets", "each", "pos", "closed?", "skip", "inject", "readchar", "pos=", "sort_by", "max"]

The important things to note are accessors like lineno and filename. They can give you some information while you read the lines. Such as if you’re reading from a file or STDIN. You can easily give line numbers to everything being read. Like so:

# linenum.rb
ARGF.each do |line|
  puts "%3d: %s" % [ARGF.lineno, line]
end

Produces:

shell> ruby linenum.rb a.txt b.txt
  1: inside a.txt
  2: inside b.txt

ARGFy

As an exercise I wrote a little class to emulate what ARGF does and to make it more useful to me. For instance ARGF can’t tell you when it changes files. You can try and catch when the filename changes but what if the same file is repeated twice in a row? ARGFy has both a global lineno and a per file filelineno I’ll talk more about ARGFy later.

shell> ./configure --program-suffix=WHATEVER ...

That neat switch will make it so when you finally `make install` the output binaries are named rubyWHATEVER, irbWHATEVER, etc. Obviously you should choose something better then WHATEVER. I went with ‘–program-suffix=19′ so that all my Ruby 1.9 binaries are exactly the same as the 1.8 binaries but with “19″ on the end. Such as ruby19, irb19, etc. This will really save me some time as I experiment with Ruby 1.9.

Note when installing Ruby from source on Mac OS X you may run into problems, check out this article or the one linked above for the fixes.

Also, since I had now had a ruby19 I figured I’d make a ruby18 as well. I already have my own ‘~/bin’ directory in my shell’s path. So I added the following symbolic link which makes `ruby18` reference my default Ruby 1.8 interpreter:

shell> ln -s `which ruby` ruby18

The `which ruby` in my case evaluates to the default installation of Ruby. It may not be the case on your computer. That depends on how you’ve modified your path. So the explicit version that should work on any mac is:

shell> ln -s /usr/bin/ruby ruby18

Here the symbol link means I can use `ruby18` if I want to be dead sure I’m using Ruby 1.8, and likewise `ruby19` when I want to use Ruby 1.9. I know I’m not the first person to do this, but I’m happy and I want to spread the joy.

Other notable configure options are:

shell> ./configure --prefix=PREFIX ...

This makes it so when you run `make install` it will install the files to PREFIX/(here). An example would typically be ‘–prefix=/usr/local’ to install the files into the /usr/local dir. However, that is normally the default installation directory and therefore most people don’t bother changing it. I’ve used this prefix to install temporary things directly into my home directory. This made it easy to find, work with, and delete when I was done.

At the Big Picture it is very common for commenters to list the numbers of the images that they really liked. This intrigued me a little bit. Also, the comments on the main page are limited to just
100 comments, when there might actually be over 2000 comments on some collections.

So, I wrote a Ruby Script that I call bigpicvotes that takes in the URL to a Big Picture site and analyzes the comments to make a guess at what pictures were voted for the most!

Above is a sample usage on an article with now over 2600 comments. Notice that the URL you provide can be either the “all comments” URL or just the URL with the pictures that you are likely to be looking at. Also, the usage allows for a second optional parameter. This is to show the top N images instead of defaulting to the top 10.

One last thing. To use it you will need the Hpricot gem. That is simple enough to install:

shell> gem install hpricot

Again, the script is available here: bigpicvotes

Problem

Trying to install Ruby 1.8.7 (or even 1.9) on my Leopard machine produces warning messages on “make” that look like this:

gcc -I. -I../../.ext/include/i686-darwin9.1.0 -I../.././include
-I../.././ext/readline -DRUBY_EXTCONF_H=\"extconf.h\"    -fno-common
-g -O2 -pipe -fno-common   -o readline.o -c readline.c
readline.c: In function 'filename_completion_proc_call':
readline.c:659: error: 'filename_completion_function' undeclared
(first use in this function)
readline.c:659: error: (Each undeclared identifier is reported only once
readline.c:659: error: for each function it appears in.)
readline.c:659: warning: assignment makes pointer from integer without a
cast
readline.c: In function 'username_completion_proc_call':
readline.c:684: error: 'username_completion_function' undeclared
(first use in this function)
readline.c:684: warning: assignment makes pointer from integer without a
cast
make[1]: *** [readline.o] Error 1
make: *** [all] Error 1

Solution

I originally came across this problem right around new years and eventually found a solution thanks to Han Kessels on the Ruby Forums. It involved the following steps:

1. Download the newest version of readline (version 5.2 at the time of writing) at GNU.org. You may have to apply the following patch. Thanks to Michael Biven for showing a nice simple way to do this from the command line:

   $ curl -O ftp://ftp.gnu.org/gnu/readline/readline-5.2.tar.gz
   $ tar xzvf readline-5.2.tar.gz
   $ cd readline-5.2
   $ curl -O http://ftp.gnu.org/gnu/readline/readline-5.2-patches/readline52-012
   $ patch -p0 < readline52-012
   $ ./configure --prefix=/usr/local
   $ make
   $ sudo make install
   $ cd ..

2. Now you can download the 1.8.7 version of ruby, and install it but you should point to the version of readline that you just installed (to /usr/local) like so:

   $ curl -O ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.7.tar.gz
   $ tar xzvf ruby-1.8.7.tar.gz
   $ cd ruby-1.8.7
   $ ./configure --prefix=/usr/local/ruby1.8.7 --with-readline-dir=/usr/local
   $ make
   $ sudo make install
   $ cd /usr/local/ruby1.8.7/bin
   $ ./ruby -v
   ruby 1.8.7 (2008-05-31 patchlevel 0) [i686-darwin9.3.0]

The End Result

If you followed my terminal commands from above you should have a working Ruby 1.8.7 version installed in your "/usr/local/ruby1.8.7/bin". If you wanted ruby in a different directory then change the "--prefix=/usr/local/ruby1.8.7" option on ./configure to instead point to the directory you wanted it.

Why put it in its own directory at all? Well, I happen to have multiple version of ruby. By default I have 1.8.6, I have this version of 1.8.7, I even have a 1.9 version I installed a while ago. Its easy to remember where each is if the directory they reside in clearly states the name. This way I can test a script, maybe even run benchmarks, in each version of ruby. However, you might not want to take this extra measure.