Inspiration #2

Some more links that focus on a few topic areas. What does it mean to be an expert and how can you become one? Then, Hacker Culture, and how it means and is something totally different then what you think it means. Coupled with that is really the idea of stereotypes and making uneducated assumptions… leading to asking better questions and self-motivation. One last one at the end is just a recent standout I came across. Phew, that was a mouthful. Now on to the links:

★ Are You An Expert? – Coding Horror – Jeff Atwood – Jeff nails it right on the head and with some really awesome examples to illustrate the point. Quote: “Being an expert isn’t telling other people what you know. It’s understanding what questions to ask, and flexibly applying your knowledge to the specific situation at hand. Being an expert means providing sensible, highly contextual direction.” The comments are also pretty great.

Again on the Theme of Experts – James Bach – This talk masks itself on the subject of Software Testing but I believe that any person can benefit from what Bach is talking about. His focus is how someone becomes an expert and what an expert really is. There are some really valuable slides on just this subject. You just have to be sure to make it to the end of the video where it gets good. Question everything.

General Advice – Bill Gates – Straightforward and to the point. Life is unfair and you have to work in order to earn things. The sooner you learn that, the sooner you change your views on laziness and take responsibility for your life the better.

The Conscience of a Hacker – The Mentor – Short and sweet. I thought this was a brilliant depiction of the hacker self-motivation principle and the resistance and stereotypes encountered in doing so.

Hacker Culture and More – Eric Raymond – My definitive source to point people to when I need to educate them on Hacker Culture. Equally as important is his essay on “How to Ask Questions the Smart Way.” I’m an avid fan of encouraging people to ask smarter questions, now I have a place where I can point them to and demand they take initiative.

★ Your Shit Does Stink — Good Friends Are Hard to Find – Steven Bristol – When you really care about a subject or a project of your own and you share it with others you don’t just want a nod and the words “thats cool.” You want a critique… you want to prove its worth… you want a contrasting point of view! This short article sums it up barbarically. But its true.

The last article reminded me of a House episode, where House fires the one doctor who thinks exactly like him and “seemed” to help in some of the medical cases. Why? Because without friction the two didn’t feed off each other. By always agreeing and thinking the same the sum of the two doctors was no greater then either individually. Lesson learned.

Data URIs

I’m guessing that a lot of web developers don’t know about data URIs. Most probably won’t even have to know about them. However, I still want to go over them because I think they are clever and could prove useful. The main advantage I see is the possibility to avoid an HTTP Request, especially for a small image.

Wikipedia’s overview is quite thorough. The article also links to a number of places where you can quickly throw together some data URI’s for any mime type. The most well known is the data URI kitchen.

Quickly, the format of a data URI is as follows:

data:[<MIME-type>][;charset="<encoding>"][;base64],<data>

Note that almost every section is optional! The mime type option means you can do some very interesting things. The normal usage might be to to embed a small image, as a datauri, inside the html document. However, another usage might be to embed an entire HTML document inside a link!

I was looking at the code for some command line utilities that make datauri’s and I learned an easy way to find out the mime type of a file is to use the `file` command:

shell> file sym.png
sym.png: PNG image data, 76 x 78, 8-bit/color RGBA, non-interlaced

shell> file -i sym.png
sym.png: regular file

shell> file --mime sym.png
sym.png: image/png

So using `file –mime` and reading whats after the ‘: ‘ you can easily get a mime-type for any file. Then reading and base64 encoding the content you have a quick hack to quickly create datauris! Here goes:

#!/usr/bin/env ruby
# Quick & Dirty data URI for a file
#
require 'base64'
require 'uri'

# Filename and Contents
filename = ARGV[0]
contents = File.read(filename)
base64 = Base64.encode64(contents).gsub("\n",'')

# Mime Type for the Filename
output = `file --mime #{filename}`
mime = output.match( /: (.*)$/ )[1].downcase.gsub(/\s/,'')

# Make Data URI
datauri = "data:#{mime};base64,#{base64}"

# Output
puts datauri

Some better scripts are available. Here is a Perl implementation and a pretty neat Ruby Implementation.

My Mac Workflow – “How I Work”

I’ve seen a number of articles recently where people talk about their programming setups. Whenever people blog about their setups it always opens my eyes to some neat tips and tricks that end up improving my own workflow. So, I decided I’d post what I do. Hopefully that will give some people some ideas. Maybe I may even get some feedback on how I could improve things. For reference, this article was published in March 2009.

Ubiquitous Tools

These tools are typically Preference Panes or Menubar applications that I always have running. I asked myself if I could remove any of them and I thought… no, I don’t think I can.

QuickSilver

I have QuickSilver open 100% of the time I use my mac. If QuickSilver ever crashes I will drop what I’m doing and use SpotLight to reopen it. I occasionally use ⌘+Tab to switch between applications. However, typically I will use a custom global keyboard shortcut I have made to launch a particular application, or make use of the heavily weighted key combinations I’ve built up to open commonly used applications. I even use QuickSilver to browse though my hard drive for files!

QuickSilver with the HUD Theme

Key Settings for my QuickSilver are:

  • BezelHUD Theme
  • ⌃+Space Activation

  • ⌃⌥⌘N => Launch TextMate (therefore brings to the front if open)

  • ⌥⌘T => Launch Terminal (therefore brings to front if open)

  • ⇧⌘D => Launch Dictionary (therefore brings to front if open)

  • “w” => WebKit, “saf” => Safari, “m” = Mail, “i” = iTunes, “ff” = Firefox

  • Added Indexing Important Directories. This is important for School files, etc.

TextExpander

I’ve attested to the awesomeness of TextExpander in the past. It truly is remarkably useful and someone telling you how useful it is just never seems to be enough. You have to experience it. You also have to take control and make it really be an extension of yourself. Which I recently did.

I added a number of generic commands so that I can type ⇧⌘⌥⌃⌫⌦⌅⎋⏎♥ and more with exceptional ease. This has not only made me very happy that I didn’t have to search through the special character, but it also makes my messages to other mac users much more expressive and accurate. “cmd+alt+S” is hard to read. ⌘⇧S is not. End of the story. I have hundreds of other uses for this little gem but I won’t bore you with the details.

TextExpander

There are two things that I find with TextExpander. I never correct my spelling errors, because the words I find myself misspelling so often are auto-corrected!!! So I never learn… its now easier to type the typo! But seriously TextExpander isn’t have 100% perfect for expanding when you want it to. That can be customized and I’ll be trying these customizations soon.

GrabUp

The ability to take a picture of anything I can see on my computer, have it automatically uploaded to the web and a direct URL be put in my clipboard is so undeniably awesome that I can’t even mention this application without smiling. I struggled with this exact concept for years on Window, created such elaborate hot-key schemes to get it done. With GrabUp, its just the usual ⇧⌘4. I can’t recommend this enough.

GrabUp

Caffine

I’m a heavy laptop user. I have my settings set so that my laptop will dim quite quickly if it gets no user input. Sometimes, like watching videos or reading PDFs or long blog posts, I don’t want it to dim. A simple click on the menu bar icon and I know my Mac will be dim-free for as long as I want it.

Caffine

I mention this application because it truly is part of my workflow. It has its place, and I wouldn’t get rid of it. It gives me a super simple switch that I can “flip” on or off when I want to maximize my power usage, or when I don’t. That is important to me.

Spaces and Hot Corners

I’m spoiled. I exploit the crap out of Spaces to keep myself organized. I currently use 6 spaces, 3 for business, 3 for personal. Applications like Mail, iTunes, iCal, Adium are all tied to particular Spaces so I can find them without blinking. It is my favorite feature of Mac OS X Leopard.

Spaces

Secondly I make use of Hot Corners for Spaces, Expose, Dashboard, and Show Desktop. I make use of all four corners of my screen. Every corner has a purpose and every corner gets heavy usage. I also have a few shortcuts on my mouse for Spaces and Expose, but more then 90% of their usage is triggered by a wicked fast flick of the cursor to a corner. I’m told people are intimidated by it.

Development

TextMate

I’m a huge TextMate fan. Its my editor of choice. I’m not one of those people who wants to force you to change to “my editor.” I’ve learned that its just better to encourage people to choose an editor they like and put forth the effort to customize and enhance it for themselves. TextMate gives me plenty of room to write my own commands, add features, and make customizations to that make me look like a wizard doing any task.

TextMate

Shown above, what stole my heart with Textmate was that it could correctly syntax highlight php, javascript, css, and html all in the same file according to their respective languages, and it looked amazing doing it!! My jaw dropped, my neurons fired, and I’ve never looked back.

It normally goes without mentioning, but this is the “how I work” blog post so I should mention it. Learn the hot-keys and shortcuts for every application. Invent your own hot-keys. Customize, customize, customize. Find out what you do over and over, and automate it. Figure out what bothers you and fix it. 20 minutes spent customizing an application or learning its features can save you hours and hours in the long run. When you finally dedicate yourself to using an application, especially an editor, customize your environment!

Terminal

I always have a terminal open. Normally more then one, with different kinds of shells (bash, ruby, and javascript). Like I’ve mentioned before I have invested some time into making customizations that help make my terminal more comfortable.

terminal

Above are just some of my aliases, but my really good aliases are my single character ones. Click on the image above to check out my GitHub repository for my dotfiles (.bashrc, .bash_profile, .irb_rc, etc). I keep this updated frequently as I find more and more tricks to my liking.

As always here is a link to a site showcasing the scripts I’ve written that I use all the time. I call it my ~/bin. Its all free, so take a peek and dig in.

ExpanDrive

The most recent addition to my must have toolkit is ExpanDrive. I was fortunate enough to get this application while it was discounted and I’m really glad I did! I’m almost always connected to this web server (bogojoker) and my college account!

expandrive

With ExpanDrive and TextMate I can edit the live files on my webserver. Or seamlessly copy files from my local hard drive to my webserver in the terminal using a `cp` command. It just makes sense! I even hacked together a script that knows when I’m in an ExpanDrive directory and properly opens the file in a browser.

I try to tell other people that connecting to my web server is as though I just plugged in an external hard drive. I don’t think they believe me… but its true. I lied. Its easier.

Flow and Cyberduck SFTP

When I have larger SFTP requirements it helps to have a dedicated client. The two that I use are either Flow or Cyberduck.

flow

Nothing really special to say here. Actually I’ve had some rough luck with Flow lately and I’ve had to swing back to trusty old Cyberduck occasionally.

Some Go Without Saying

Really, what Mac User doesn’t use Adium and Growl? Its 2009…

Hot Keys. Seriously, learn them.

Sure, I ranted about it up above, but I’m sure a few people didn’t get it yet. Learn shortcut keys. Its irresponsible of you not to. I’ll run over the ones you should be comfortable with. Not the dull ones but the ones that probably confused you years ago when you first saw them. Read them, slowly put your fingers on the keys so you realize what the keys actually are (not what you read on this page but what you feel on the keyboard) and most of all start using them:

  • ⇧⌘] and ⇧⌘[ => Move between tabs. Safari, Adium, Terminal, …

  • ⌘] and ⌘[ => Indent and un-indent text. Mail, Text editors, Development editors, …

  • ⌥⌘T => Special Characters. Pretty much always, unless you’ve overloaded this (like me).

  • ⌘P => If you can Print, then you can save as PDF or save to Web Receipts folder. Good!

  • ⌘, => Open Preferences. Made you do it! Now set start customizing other apps!

Now I’ll walk through how I use some of the other everyday apps.

WebKit and Safari 4

I use the WebKit nightly. I personally find developing with the most bleeding edge features of HTML/CSS exciting. The Web Inspector is so cool that is even inspired me to download the Web Kit source and start contributing (more later). It keeps me on top of the changes and issues affecting the entire web development world.

Aside from the developer aspects, there is no doubt that WebKit/Safari is the primary web browser for Mac Users. So I’m not going to bore you with what has already been said, but try to explain my browser work flow. Its essential to be able to open Webkit, make a google search and display the results in a new tab in under a second. Why? Because I do it a hundred times a day. So here goes some of the Safari specific strokes:

  • ⌘L => Select/Goto the URL bar (the location bar). Essential for copying URLs.

  • ⌥⌘F => Select/Goto the Google Quick Search. Essential for quick google searches.

  • ⇧⌘⏎ => When in the Google Quick Search this runs the search and opens in a new tab.

  • ⌥⌘I => Open the Web Inspector.

  • ⌥⌘C => Open the Web Inspector Console.

  • Right Click → Inspect Element => Open the Web Inspector on that Element

  • ⌘ + Click link => Open in a New Tab in the background

  • ⇧⌘ + Click link => Open in a New Tab in the foreground

  • => Previous Page. Like Back one in History.

  • Drag and Drop file from the hard drive onto a file upload => Sets the file upload to that file!

Again, the problem with seeing these in bulk like this is that they don’t sink in unless you actually try them with your fingers. Muscle memory is way more powerful then visual memory (I have no evidence to back this up other then my own evaluations). Don’t get intimidated, just go character by character, press them on the keyboard, and you’ll actually start remembering and wanting to use them!

I still test in Firefox and most of the above work in Firefox as well.

NetNewsWire

There are a few alternatives for free Mac OS X feed readers. I’m a big App guy so websites like Google Reader just didn’t cut it for me. My favorite design wise, NewsFire, just didn’t cut have the proper user interface and keyboard shortcuts. I’ve been very happy with NetNewsWire for years. It has all of the features I need a nice free package.

netnewswire

I have tons of feeds. I wouldn’t recommend so many feeds for most people. But if you do decide to use NetNewsWire, learn how to use the arrow keys and space bar to traverse everything with unbelievable simplicity!

Entertainment

I almost always have music playing, and recently I’ve felt the need for a little variety. So I use iTunes and Last.fm’s player. I was also lucky and got SRS iWow, which is a really awesome improvement to iTunes normal sound output, even on my laptop. Finally, I use Twitterrific for my Twitter needs.

Ahhhh, Finally! In Closing

So, its taken me months to write this article, and its nowhere near the quality I had hoped it would be. But I’m too exhausted to entirely proofread this to the extent I normally do. I just hope that this article will inspire some of you readers to look at the apps that you use, and learn a new keyboard shortcut, or make an effort to tweak some settings that may have bothered them but they were too lazy to look into.

Cheers.

Freenode JSBot Command Line Script

So over my week break from college I spent a bunch of time in ##javascript learning and helping others with Javascript problems. This was to help me prepare for one of the projects that I’m working on (still to be announced).

One of the things I really liked in ##javascript was the freenode jsbot that could do all sorts of things. It was so useful in fact that I felt I had to have it for when I’m not using IRC. The website mentioned an API, so I dug in.

I wrote a command line jsbot script and added it to my ~/bin:

jsbot

A clip of the source code (yes its horrible… but its so compact!) shows how easy it is to work with JSON in Ruby. Just a few includes and its just as easy as Javascript, without the cross-site request issues:

#!/usr/bin/env ruby
# Author: Joseph Pecoraro
# Date: Friday March 6, 2009
# Description: Simple Interface for the
# really neat jsbot!

require 'rubygems'
require 'open-uri'
require 'json'
require 'cgi'
require File.dirname(__FILE__) + '/escape'


class JSBot

  JSON_PREFIX = 'http://fn-js.info/jsbot.xhr?'
  SITE_PREFIX = 'http://js.isite.net.au/jsbot?'

  def search(str)
    uri = url(str, JSON_PREFIX, "search=")
    JSON.parse( open( uri ).read )
  end

  def show(str)
    uri = url(str, JSON_PREFIX, "show=")
    JSON.parse( open( uri ).read )
  end

  def url(str, u=SITE_PREFIX, q='q=')
    u + q + CGI::escape(str)
  end

end

...

Ruby Readline Documentation

Before today the Ruby readline library lacked documentation! In order to find some decent documentation you would have to read the README packaged with the source. Michael Fellinger (manveru) from #ruby-lang offered to help me get some decent documentation online. Take a look!

readline

Debugging HTTP Headers

While writing the skreemr shell the other week I ran into an issue that required me to dig down to one of the lowest levels of web communication… HTTP Headers. I’m always shocked to learn that so many web developers don’t know much about headers. So I thought I would try to reenforce the point that HTTP Headers matter… and that knowing your stuff can help you debug and solve problems. Here is the story of my real world example.

I’m not going to go over HTTP Headers. thats already been done. Instead I’m going to focus on debugging and working with them a little bit. I’m going to assume you have already the general concepts.

The Problem

I wanted to add pagination to the skreemr shell. So you could run a search, then get the next page of results, etc. So a little experimentation with skreemr in my browser produced the following URLs:

http://skreemr.com/results.jsp?q=test
http://skreemr.com/results.jsp?q=test&l=10&s=10
http://skreemr.com/results.jsp?q=test&l=10&s=20

A simple pattern! “q” is the query string, “l” is the number per page, “s” is the number to start at, indexed from 0. So that last URL would produce results 21-30. This all worked well in my browsers, but it wasn’t working in Ruby:

require 'open-uri'

# Grab the pages and read the content
str1 = open( 'http://skreemr.com/results.jsp?q=test'      ).read  # 1-10
str2 = open( 'http://skreemr.com/results.jsp?q=test&s=10' ).read  # 11-20

#=> Should print false... but its printing true!
puts str1==str2

What this was saying was that the content being returned from both of those urls is EXACTLY the same. That couldn’t be… could it?

More Investigation

At this point I thought it was a problem in Ruby. I figured Ruby was doing some caching in the background that I was going to have to disable or work around. (I now know this is not true, but that was my first guess). To test that hypothesis I turned to my trusty friend curl and checked to see if that showed the proper behavior:

shell> curl 'http://skreemr.com/results.jsp?q=test'      -o 1.html
shell> curl 'http://skreemr.com/results.jsp?q=test&s=10' -o 2.html
shell> diff -q -s 1.html 2.html
Files 1.html and 2.html are identical

What!?! That stunned me. Curl was getting the exact same results as Ruby. I took a look at the html files, and indeed 2.html, which should have contained results 11-20 held 1-10. I opened both urls in my browser… they showed the correct results. Something weird was happening!

Take A Step Back

At this point you’ve got to know what is happening. Curl and Ruby (using Ruby’s Net:HTTP under the hood) are just making a simple GET request. Both my browsers Safari and Firefox are sending far more then just a GET request. They are sending a bunch of other headers. Lets take a look at what Firebug says Firefox sent:

firebug

Thats a mighty long list of headers! Its entirely possible that one of those might be influencing the server. Onto the drama!

Time to Act – Literally

The only difference between what curl sent in its request and what Firefox was sending is that header information and possibly any information contained in those cookies (unlikely in this case). So lets make curl act as if it is Firefox by send the same headers with curl! I took a quick peek at my curl reference for the proper switches/formatting and I was ready:

shell> curl 'http://skreemr.com/results.jsp?q=test' -o 1.html
shell> curl 'http://skreemr.com/results.jsp?q=test&s=10'            \
            -H 'Host: skreemr.com'                                  \
            -H 'User-Agent: Mozilla/5.0 (...) Firefox/3.0.6'        \
            -H 'Accept: text/html,application/xml;q=0.9,*/*;q=0.8'  \
            -H 'Accept-Language: en-us,en;q=0.5'                    \
            -H 'Accept-Encoding: gzip,deflate'                      \
            -H 'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7'     \
            -H 'Keep-Alive: 300'                                    \
            -H 'Connection: keep-alive'                             \
            -H 'Cache-Control: max-age=0'                           \
            -o 2.html
shell> diff -q -s 1.html 2.html 
Files 1.html and 2.html differ

Jackpot! Some subset of those headers is indeed fixing my problem, because now it properly fetched the second page of results! Translating that back into Ruby works as well:

require 'open-uri'

# Headers
HEADERS = {
  'Host'            => 'skreemr.com',
  'User-Agent'      => 'Mozilla/5.0 (...) Firefox/3.0.6',
  'Accept'          => 'text/html,application/xml;q=0.9,*/*;q=0.8',
  'Accept-Language' => 'en-us,en;q=0.5',
  'Accept-Encoding' => 'gzip,deflate',
  'Accept-Charset'  => 'ISO-8859-1,utf-8;q=0.7,*;q=0.7',
  'Keep-Alive'      => '300',
  'Connection'      => 'keep-alive',
  'Cache-Control'   => 'max-age=0'
}

# Get the Pages
str1 = open( 'http://skreemr.com/results.jsp?q=test', HEADERS ).read
str2 = open( 'http://skreemr.com/results.jsp?q=test&s=10', HEADERS ).read

# Correctly Prints false
puts str1==str2

Problem Solved

Sure, we have a solution but its not pretty. How much is really needed? Finding that out is just a matter of eliminating the headers that do nothing. Remove one, test, remove another, test, remove another test… As long as it still works after you’ve removed an individual header you know that header wasn’t needed. It took me a few minutes but I narrowed it down to just a single header:

MAGIC = { 'Accept-Language' => 'en-us' }

This was a rather unique problem. It is rare for me to have to dip down to the HTTP Headers to see what is actually happening for such a high level problem. However, as this problem shows, its important to know what is going on under the hood. If I didn’t know about headers, I would not have been able to solve it.

search