Web Analytics Taken to the Next Level

I came up with a neat idea the other night. Using localStorage and sessionStorage you could theoretically monitor the number of tabs or windows a visitor has opened for your site. As far as I know, this capability has never before been possible. Well, now it is.

Check out this crude example.

Once you open the monitor, leave that tab/window put. Use the “Spawn Tab” link to create new tabs and windows. The monitor will be notified and display some simple debug. There are 10 second updates per tab/window so that when they close the monitor can detect it. The monitor will detect a close within 15 seconds of the tab/window closing and will display the total time the tab/window plus or minus 10 seconds. Correct values are maintained as the tab/windows browse across pages as long as they stay on the domain! Just about everything you’d want or need.

Again, I mentioned this is rather crude. The fact that the monitor tab remains open is only due to the fact that I wanted to prototype the idea. The majority of the state is stored in localStorage, and each tab/window maintains a single identifier in its sessionStorage to remind the tab/window what id it was while it navigates to multiple pages. Because everything is stored in the storage this system has the capability to become completely distributed. Meaning no “monitor” tab is necessary, and the scripts can determine, and monitor, on their own the existence of all other tabs. Thus, this would be a viable option for the next level of web analytics.

As cool as this is, I don’t think it will provide too much value to the analytics. For the first time webmasters will be able to know how many windows or tabs a visitor opens (and to what pages they open). The webmaster will know more about how its user’s use the website, but I don’t think this statistic will be a game changer. Who knows!

So, how does it work? Very simple. Each tab includes the client.js code to handle updating the localStorage and maintaining its own “tab_id” in sessionStorage. Data it maintains can be whatever you want, I went with some simple information such as its start time, current url, and latest keepalive:

localStorage values

The upkeep for a Tab Client is to restore their session information when you navigate to any new page:

// Create or Restore tab_id
var myTabId = sessionStorage.tab;
if ( myTabId === undefined ) {
  var tabs = localStorage.tabs;
  if ( tabs === undefined ) {
    myTabId = 0;
    localStorage.tabs = "0";
  } else {
    var largest = parseInt( tabs.split(/,/).pop(), 10 );
    myTabId = largest+1;
    localStorage.tabs += "," + myTabId;
  }
  sessionStorage.tab = myTabId;
}

And to perform its keepalives:

// Update the Latest Timestamp
function setLatest() {
  var key = 'tab'+myTabId+'_latest';
  localStorage.setItem(key, +(new Date())); 
}

// Update Status every 10 seconds
window.setInterval(setLatest,10000);
setLatest();

I put a few more convenience functions in there to help it update these localStorage keys, and communicate with the monitor which was crudely done through localStorage. That is explained next.

The Tab Monitor as it stands right now receives messages through localStorage’s “storage” event. It also checks all the tab’s “lastest” keepalives to make sure they didn’t pass their 10 second limit. In the case of a tab being closed, it will remove references to that tab and output an approximation of the time the tab was open:

// Listener - receive messages from tabs
window.addEventListener('storage', function(e) {
  if ( e.storageArea === localStorage ) {
    if ( e.key == "tab_msg" ) {
      console.log( e.newValue );
      addMsg( e.newValue ); // Appends to the page
    }
  }
});

// Purger - clean out tabs that died for 15 seconds
window.setInterval(function() {
  console.log("purging");
  var now = +(new Date()),
      tabs = localStorage.tabs;
  if ( tabs !== undefined ) {
    var toRemove = [], toKeep = [];
    tabs = tabs.split(/,/);
    for (var i=0, len=tabs.length; i<len; i++) {
      var tabId = tabs[i],
          tabLatest = parseInt( localStorage.getItem("tab"+tabId+"_latest") );
      ( (now-tabLatest)>=15000 ? toRemove : toKeep ).push(tabId);
    }
    localStorage.tabs = toKeep.join(',');
    for (var i=0, len=toRemove.length; i<len; i++) {
      var tabId = toRemove[i],
          tabLatest = parseInt( localStorage.getItem("tab"+tabId+"_latest") ),
          tabStart = parseInt( localStorage.getItem("tab"+tabId+"_start") ),
          time = (tabLatest-tabStart)/1000;
      addMsg( "Tab " + tabId + " Closed after " + time + " seconds!" );
    }
  }
}, 5000);

This took a little under an hour to get working. There are still minor issues that I didn’t attempt to resolve. However, if there is interest this could be developed into a completely distributed peer-to-peer communication between tabs/windows on a single domain. However, a little warning. Web Storage is not set in stone. Not all browsers have implemented it and the specification is subject to change at any minute. There has been some rather heated debate on the subject of Web Storage recently, with good reason. All I know is that when its settled, this functionality will continue to exist!

Let me know what you think.

Handling the tab key in a <textarea>

Traversing through input elements with the tab key is important for accessibility reasons. However, every once in a while you come across a situation where traversal isn’t really important. Instead, you want the tab key to actually do something for you. Even still, you may want to do something fancy with the tab key. Wether its replacing it with spaces or something else.

I found an interesting website today that had an interesting idea. You could run some test code on the page to test their library. Their instructions said, “push tab to evaluate the code.” Sure enough you could tell it was working “onblur” for the textarea. The problem with this was that when you pushed tab you lost focus.

I thought about it, and figured you could do a rather simple trick to run some code and refocus on the textarea. It goes a little like this:

window.addEventListener('load', function() {
  var textarea = document.getElementById('txt');
  textarea.addEventListener('keydown', function(e) {
    if (e.keyCode === 9) {
      e.preventDefault();
      e.stopPropagation();
      // operation goes here
    }
  });
});

Note that to get the actual character you have to get the character from the event. There are many ways to do it, keyCode, charCode, which, even keyIdentifier. You’ll have to mix things up to work across all browsers. Basically 9 is the code for the tab key. So when you get the tab key, it prevents the default behavior and allows you to execute whatever code you want: run some functions, eval some code, display something, ajax request, whatever you want. Simple. I think it would improve a few interfaces. Neat idea to make use of the tab key to perform a function.

You can check out this example of what I mean.

Markdown => Tutorial in 1 Step

When I wrote my Ruby Readline tutorial I felt I came up with a cool concept. I started with a Markdown file, translated it to html, and I used the headers to generate a Table of Contents on the fly.

table of contents

It didn’t take me long to realize that I could turn this into a framework where I could turn any Markdown file into a tutorial exactly like this one. So with surprisingly little work I modified the scripts to work with any markdown file and the html automatically generated from the standard Markdown.pl script.

I called this markdownorial. Laugh all you want at the name, but I still think the concept is very cool. I’ll probably be using this more and more to automatically generate and format a pretty cool looking tutorial from a single markdown file. The start to finish time for a project like this has instantly dropped to just the raw content part, no design or coding needed!

Advantages include:

  • Writing Markdown is very fast and efficient.

  • Time is spent writing the content. Not messing with design,

  • Table of Contents is automatically built for you.

  • Useful permalinks are automatically generated. Very useful when passing around links.

  • Clean user interface that focuses entirely on the content but the Table of Contents is always available!

  • Git Repository means if I update the design its just a `git pull` away to get the update.

Right now the tutorials shows up elegantly in all standards compliant browsers. Safari/Webkit and Chrome display it perfectly. Opera has some very minor Unicode issues but displays everything perfectly. Firefox has some separate Unicode issues and if you don’t have the latest version it has some working but slow animation. Overall, its entirely usable for people using decent browsers.

Let me know what you think. Feel free to use it and improve it. Its all up on Github.

Freenode JSBot Command Line Script

So over my week break from college I spent a bunch of time in ##javascript learning and helping others with Javascript problems. This was to help me prepare for one of the projects that I’m working on (still to be announced).

One of the things I really liked in ##javascript was the freenode jsbot that could do all sorts of things. It was so useful in fact that I felt I had to have it for when I’m not using IRC. The website mentioned an API, so I dug in.

I wrote a command line jsbot script and added it to my ~/bin:

jsbot

A clip of the source code (yes its horrible… but its so compact!) shows how easy it is to work with JSON in Ruby. Just a few includes and its just as easy as Javascript, without the cross-site request issues:

#!/usr/bin/env ruby
# Author: Joseph Pecoraro
# Date: Friday March 6, 2009
# Description: Simple Interface for the
# really neat jsbot!

require 'rubygems'
require 'open-uri'
require 'json'
require 'cgi'
require File.dirname(__FILE__) + '/escape'


class JSBot

  JSON_PREFIX = 'http://fn-js.info/jsbot.xhr?'
  SITE_PREFIX = 'http://js.isite.net.au/jsbot?'

  def search(str)
    uri = url(str, JSON_PREFIX, "search=")
    JSON.parse( open( uri ).read )
  end

  def show(str)
    uri = url(str, JSON_PREFIX, "show=")
    JSON.parse( open( uri ).read )
  end

  def url(str, u=SITE_PREFIX, q='q=')
    u + q + CGI::escape(str)
  end

end

...

“Back and Forth” Greasemonkey For The Whole Web

Recently I wrote a Greasemonkey script to add keyboard shortcuts to The Big Picture, to improve on some of their already existing shortcuts. Once I started using some of the shortcuts I made I ended up wanting to use them all over the place at other blogs. This functionality is so tiny, but so useful, that I bundled it into its own script that runs on all web pages!

Grab it here:

//
// ==UserScript==
// @name          Back and Forth
// @namespace     http://blog.bogojoker.com
// @description   Keyboard Shortcut to Jump back and forth on a page. (esc key).
// @include       *
// @version       1.0 - Initial Version - Sunday February 15, 2009
// ==/UserScript==

(function() {
  
  // Global States
  var x = null;
  var y = null;

  // Add a new Global Key Listener for `esc`
  document.addEventListener("keypress", function(e) {
    if(!e) e=window.event;
    var key = e.keyCode ? e.keyCode : e.which;
    if ( key == 27 ) {
      var tempx = x;
      var tempy = y;
      x = Math.max(document.documentElement.scrollLeft, document.body.scrollLeft);
      y = Math.max(document.documentElement.scrollTop, document.body.scrollTop);
      if ( tempx != null ) { // First time it should be null
        window.scrollTo(tempx, tempy);
      }
    }
  }, true);

})();

On any webpage the first time you push the `esc` key position A gets stored. The next time you push `esc` position B gets stored and the browser jumps to position A. The next time you push it, A gets stored and you jump to B. So you always jump back to wherever you pushed `esc` last. Hence the name “back and forth.”

This is useful to me when I jump between comments and the content. When I’m reading a comment and I want to check back to the article, I just just push `esc` to save my position, go back to the article, and when I’m all set I just jump back to my saved position (the comments) with `esc`.

Short, Sweet, Simple: The Back and Forth Greasemonkey Script.

More Big Picture Keyboard Functionality!

I’ve mentioned before how I’m a big fan of The Big Picture blog. One of the things that makes it so great is that it has keyboard navigation! You can use ‘j’ and ‘k’ to automatically jump between pictures. Its so much nicer then scrolling because it jumps to the exact height to maximize the picture in the browser. Huge usability improvement!

Like before, the problem I had was that users were mentioning pictures in their comments. Jumping back to that picture was hard or annoying. So, I wrote a Greasemonkey script that allows you to type in a number and it will automatically jump to that picture! Click here to get the script!

Oh, and if you’re reading comments and you want to jump back and forth between images and comments that works too. Once you’ve jumped to the image, just hit ‘esc’ and you will be taken back to where you were before. Too cool!

//
// ==UserScript==
// @name          The Big Picture Keyboard Enhancements
// @namespace     http://blog.bogojoker.com
// @description   Keyboard Shortcut Enhancements
// @include       http://www.boston.com/bigpicture/*
// @version       1.1 - Added Back+Forth - Thursday February 12, 2009
//                1.0 - Initial Version - Monday February 9, 2009
//
//
//   This allows the user to type in numbers, and after about
//   a half second it will jump directly to that image.
//   For example:
//
//     Push '1'... '2'... User is taken directly to Image "12"
//     Push '9'... '9'... User is taken to the last picture.
//
//   Use 'esc' to jump back and forth between two positions.
//   For example if a comment mentions picture 4:
//
//     Push '4'...        User is taken directory to Image "4"
//     Push 'esc'         User is taken back to the comment!
//     Push 'esc'         User is taken back to Image "4"
//
// ==/UserScript==

(function() {
  
  // GreaseMonkey (Firefox - unsafeWindow) and GreaseKit (Safari - window)
  var w = ( /a/[-1]=='a') ? unsafeWindow : window;

  // Global States
  var x, y;
  var keypressnumber  = false;
  var builtupnumber   = '';
  var quicknumtimeout = null;
  var imgArr = document.getElementsByClassName("bpImage");
  
  // Keep the old and create a New Global Keypress listener on top of it
  document.addEventListener("keypress", function(e) {
    

    // Get the key
    if(!e) e=window.event;
    var key = e.keyCode ? e.keyCode : e.which;
    
    // Store the current x/y position
    function storePos() { 
      x = Math.max(document.documentElement.scrollLeft, document.body.scrollLeft);
      y = Math.max(document.documentElement.scrollTop, document.body.scrollTop);
    }
    
    // # Character => Jump to that image, Store Position
    if ( key >= 48 && key <= 57 ) {
      if ( e.target.nodeName.match(/TEXTAREA|INPUT/) ) return;
      clearTimeout(quicknumtimeout);
      keypressnumber = true;
      builtupnumber += (key - 0x30);
      quicknumtimeout = setTimeout(function() {
        w['currImg'] = parseInt(builtupnumber,10)-1;
        if (w['currImg'] >= imgArr.length) { w['currImg'] = imgArr.length-1; }
        storePos();       
        window.scrollTo(0,imgArr[ w['currImg'] ].offsetTop+174);
        keypressnumber = false;
        builtupnumber = '';
        quicknumtimeout = null;
      }, 300);
    }
    
    // Esc => Jump back to a Saved Position
    if ( key == 27 ) {
      var tempx = x; var tempy = y; storePos();
      window.scrollTo(tempx, tempy);
    }   

  }, true);

})();

All I do is register a new global keyboard listener to catch numeric keys and act accordingly. I tested thoroughly on Firefox and Safari to make sure it works correctly in all cases. It jumps to the correct image, it maintains the “current image” so j/k will still work, it won’t jump if you’re typing in a textfield/input, etc. It even properly handles situations that the current j/k functionality doesn’t. For instance mine allows the user to type in the comment box, click outside the box and reuse the keyboard shortcuts. When you type a j/k or even click inside the search box at the top the j/k functionality is gone. I didn’t feel like correcting that in this Greasemonkey script in case it gets fixed by the developers behind the Big Picture. (Note to those developers: reset isLoaded back to true or take a different approach.)

I’m open to New Ideas. I have some myself but I have to focus on schoolwork in these next few weeks. Let me know if you want anything.

Big Picture Keyboard Commands Greasemonkey Script!

Enjoy.

Playing With Some More jQuery Plugins

I decided to play with some more jQuery plugins. I’m building up some experience, learning a lot, and having a ton of fun doing it. So, what did I decide to do this time? I added a sidebar to ~/bin. When you click on the logo in the top right an instructions panel will slide out on the right hand side of the screen. There is also a little magic going on with the logo.

sidebar

This sidebar has some neat animation so it was fun to put into the page. It is a modified version of the jQuery pageSlide plugin. To my pleasant surprise the plugin already had the capability to run callback functions before and after the transition is run. I added a property allowing for static html so I wouldn’t need to depend on an Ajax call. Also, the default behavior of the plugin is that clicking anywhere on the document, except in the sidebar, will cause the page to slide back to normal. This is okay except that I wanted a button for usability purposes to close the sidebar and also links in the sidebar (normal anchor tags) to work. That took a little creativity but I eventually got it:

      // Make it so if the user clicks in the pageslide, it won't close
      // Exception for given closers that may be in the pageslide
      $("#pageslide-slide-wrap").click(function(e){
        if ( $('a, ' + settings.closers).index( $(e.target) ) == -1 ) {
          return false;
        }
      });

 

Protecting the Innocent

I also wanted to play around with some jQuery lightboxes. The one that always comes to mind is facebox. For once I didn’t have to make any major changes to the source of this plugin, instead I made a few small tweaks to the CSS.

My goal here was to provide IE6 users with a simple little “Please Upgrade Your Browser” message. I went to work, learned a lot, and came up with the following: (to view this yourself go to ~/bin and paste “javascript:if_i_were_ie6()” in your URL bar, without the quotes!):

facebox browser upgrade warning

Sure that looked great, and it had a nice message, but it ended up looking disgraceful in IE6 itself! Rather then scrap the work I left it there, to prove the point to whatever few IE6 visitors I get. The facebox works perfectly in IE8 (I haven’t tried IE7) and so I can make use of what I’ve learned elsewhere I just hope that IE6 will go away soon. I said my goodbye already.

search