Web Analytics Taken to the Next Level

I came up with a neat idea the other night. Using localStorage and sessionStorage you could theoretically monitor the number of tabs or windows a visitor has opened for your site. As far as I know, this capability has never before been possible. Well, now it is.

Check out this crude example.

Once you open the monitor, leave that tab/window put. Use the “Spawn Tab” link to create new tabs and windows. The monitor will be notified and display some simple debug. There are 10 second updates per tab/window so that when they close the monitor can detect it. The monitor will detect a close within 15 seconds of the tab/window closing and will display the total time the tab/window plus or minus 10 seconds. Correct values are maintained as the tab/windows browse across pages as long as they stay on the domain! Just about everything you’d want or need.

Again, I mentioned this is rather crude. The fact that the monitor tab remains open is only due to the fact that I wanted to prototype the idea. The majority of the state is stored in localStorage, and each tab/window maintains a single identifier in its sessionStorage to remind the tab/window what id it was while it navigates to multiple pages. Because everything is stored in the storage this system has the capability to become completely distributed. Meaning no “monitor” tab is necessary, and the scripts can determine, and monitor, on their own the existence of all other tabs. Thus, this would be a viable option for the next level of web analytics.

As cool as this is, I don’t think it will provide too much value to the analytics. For the first time webmasters will be able to know how many windows or tabs a visitor opens (and to what pages they open). The webmaster will know more about how its user’s use the website, but I don’t think this statistic will be a game changer. Who knows!

So, how does it work? Very simple. Each tab includes the client.js code to handle updating the localStorage and maintaining its own “tab_id” in sessionStorage. Data it maintains can be whatever you want, I went with some simple information such as its start time, current url, and latest keepalive:

localStorage values

The upkeep for a Tab Client is to restore their session information when you navigate to any new page:

// Create or Restore tab_id
var myTabId = sessionStorage.tab;
if ( myTabId === undefined ) {
  var tabs = localStorage.tabs;
  if ( tabs === undefined ) {
    myTabId = 0;
    localStorage.tabs = "0";
  } else {
    var largest = parseInt( tabs.split(/,/).pop(), 10 );
    myTabId = largest+1;
    localStorage.tabs += "," + myTabId;
  }
  sessionStorage.tab = myTabId;
}

And to perform its keepalives:

// Update the Latest Timestamp
function setLatest() {
  var key = 'tab'+myTabId+'_latest';
  localStorage.setItem(key, +(new Date())); 
}

// Update Status every 10 seconds
window.setInterval(setLatest,10000);
setLatest();

I put a few more convenience functions in there to help it update these localStorage keys, and communicate with the monitor which was crudely done through localStorage. That is explained next.

The Tab Monitor as it stands right now receives messages through localStorage’s “storage” event. It also checks all the tab’s “lastest” keepalives to make sure they didn’t pass their 10 second limit. In the case of a tab being closed, it will remove references to that tab and output an approximation of the time the tab was open:

// Listener - receive messages from tabs
window.addEventListener('storage', function(e) {
  if ( e.storageArea === localStorage ) {
    if ( e.key == "tab_msg" ) {
      console.log( e.newValue );
      addMsg( e.newValue ); // Appends to the page
    }
  }
});

// Purger - clean out tabs that died for 15 seconds
window.setInterval(function() {
  console.log("purging");
  var now = +(new Date()),
      tabs = localStorage.tabs;
  if ( tabs !== undefined ) {
    var toRemove = [], toKeep = [];
    tabs = tabs.split(/,/);
    for (var i=0, len=tabs.length; i<len; i++) {
      var tabId = tabs[i],
          tabLatest = parseInt( localStorage.getItem("tab"+tabId+"_latest") );
      ( (now-tabLatest)>=15000 ? toRemove : toKeep ).push(tabId);
    }
    localStorage.tabs = toKeep.join(',');
    for (var i=0, len=toRemove.length; i<len; i++) {
      var tabId = toRemove[i],
          tabLatest = parseInt( localStorage.getItem("tab"+tabId+"_latest") ),
          tabStart = parseInt( localStorage.getItem("tab"+tabId+"_start") ),
          time = (tabLatest-tabStart)/1000;
      addMsg( "Tab " + tabId + " Closed after " + time + " seconds!" );
    }
  }
}, 5000);

This took a little under an hour to get working. There are still minor issues that I didn’t attempt to resolve. However, if there is interest this could be developed into a completely distributed peer-to-peer communication between tabs/windows on a single domain. However, a little warning. Web Storage is not set in stone. Not all browsers have implemented it and the specification is subject to change at any minute. There has been some rather heated debate on the subject of Web Storage recently, with good reason. All I know is that when its settled, this functionality will continue to exist!

Let me know what you think.

Handling the tab key in a <textarea>

Traversing through input elements with the tab key is important for accessibility reasons. However, every once in a while you come across a situation where traversal isn’t really important. Instead, you want the tab key to actually do something for you. Even still, you may want to do something fancy with the tab key. Wether its replacing it with spaces or something else.

I found an interesting website today that had an interesting idea. You could run some test code on the page to test their library. Their instructions said, “push tab to evaluate the code.” Sure enough you could tell it was working “onblur” for the textarea. The problem with this was that when you pushed tab you lost focus.

I thought about it, and figured you could do a rather simple trick to run some code and refocus on the textarea. It goes a little like this:

window.addEventListener('load', function() {
  var textarea = document.getElementById('txt');
  textarea.addEventListener('keydown', function(e) {
    if (e.keyCode === 9) {
      e.preventDefault();
      e.stopPropagation();
      // operation goes here
    }
  });
});

Note that to get the actual character you have to get the character from the event. There are many ways to do it, keyCode, charCode, which, even keyIdentifier. You’ll have to mix things up to work across all browsers. Basically 9 is the code for the tab key. So when you get the tab key, it prevents the default behavior and allows you to execute whatever code you want: run some functions, eval some code, display something, ajax request, whatever you want. Simple. I think it would improve a few interfaces. Neat idea to make use of the tab key to perform a function.

You can check out this example of what I mean.

Markdown => Tutorial in 1 Step

When I wrote my Ruby Readline tutorial I felt I came up with a cool concept. I started with a Markdown file, translated it to html, and I used the headers to generate a Table of Contents on the fly.

table of contents

It didn’t take me long to realize that I could turn this into a framework where I could turn any Markdown file into a tutorial exactly like this one. So with surprisingly little work I modified the scripts to work with any markdown file and the html automatically generated from the standard Markdown.pl script.

I called this markdownorial. Laugh all you want at the name, but I still think the concept is very cool. I’ll probably be using this more and more to automatically generate and format a pretty cool looking tutorial from a single markdown file. The start to finish time for a project like this has instantly dropped to just the raw content part, no design or coding needed!

Advantages include:

  • Writing Markdown is very fast and efficient.

  • Time is spent writing the content. Not messing with design,

  • Table of Contents is automatically built for you.

  • Useful permalinks are automatically generated. Very useful when passing around links.

  • Clean user interface that focuses entirely on the content but the Table of Contents is always available!

  • Git Repository means if I update the design its just a `git pull` away to get the update.

Right now the tutorials shows up elegantly in all standards compliant browsers. Safari/Webkit and Chrome display it perfectly. Opera has some very minor Unicode issues but displays everything perfectly. Firefox has some separate Unicode issues and if you don’t have the latest version it has some working but slow animation. Overall, its entirely usable for people using decent browsers.

Let me know what you think. Feel free to use it and improve it. Its all up on Github.

First Nettuts Tutorial – .htaccess

My First Nettuts Tutorial was published today! For those who follow my blog every week, this is the reason why I haven’t been able to post the last few weeks… because I’ve been putting all the time I would normally be blogging into a series of Nettuts tutorials.


htaccess examples

This tutorial covers the basics of .htaccess, Apache’s Per Directory Configuration Files. Fortunately, with the basics out of the way, I can move on to the cooler features such as GZip encoding and mod_rewrite for the next article.

Also the examples can be viewed here:
http://bogojoker.com/htaccess/part1/

And the examples can be downloaded here:
http://bogojoker.com/htaccess/part1_examples.zip

Enjoy!

Favorite Quotes (Alpha Release)

Much like my ~/bin page that keeps track of shell scripts I’ve written I took the time to make a small site to host my favorite quotes. Again, this lets me experiment with some things. This is an alpha version showing some progress. Its just 2 hours of PHP, a text file backend, some CSS, and a bunch of minute JS animation. But its a neat site that I hope can become useful to me:

quotes

Some semi-novel concepts are that when you scroll 75% down the page the next 10 quotes are automatically loaded and added to the DOM in the background. This way you can continuously scroll forever and load quotes without ever noticing! Well, unfortunately you can’t quite test that yet, because there are only 10 quotes, and 75% is not perfect, it should really have a fixed size from the bottom. Minor details!

Also, this project allowed my to test my new auto-project-setup shell script. It worked great. it got me up and running in a matter of seconds!

BogoJoker Initial – Blog Redesign

About a week ago I decided I wanted to do something new and exciting. After some thought I figured I would take a go at upgrading my blog’s design. All in all I invest three days into the theme, and the remainder of the week to add final touches. The result is this, my first WordPress theme, tentatively named BogoJoker Initial.

As is normally the case with my projects, this one pushed me to the edge! I sweat the details on every pixel, every color, the layout, the font, and more for the design. On the backend I wanted to make sure it was compatible with widgets, plugins, extensions, templates, yada yada yada. In the end it was looking pretty good on my test server, better then I had hoped!

Unfortunately when I moved it to the real world data that was on my blog my previous articles and markup were fighting me. It took another few hours to clean up old content, or at least get it into a workable state. There are likely to be some problems, but rest assured I’ll be working out the kinks and focusing on the future content.

 

Front Page:

The focus here is on the content. I wanted a minimal header and sidebar, a standout footer, and the majority of real estate devoted to the actual articles themselves. I’m also experimenting with a larger font-size to make it easier on the eyes of visitors.

wordpress_final_front_small

 

Individual Articles & Comments

Nothing too special about an individual article, but here the responses, or the comments, are center stage. I spent a lot of time making the design on these minimal as well, but they certainly stand out just as much as the article does. Avatars, links, dates, and permalinks are all there as you would expect!

wordpress_final_front_small

 

404 Page:

But, don’t leave yet, I’m glad you’re here! If you just arrived try checking out the home page for something that interests you. Who knows, maybe this was a good thing!

If it was one of my links brought you here I’d appreciate a heads up on my broken link. Please drop me an email so that I can resolve the problem. Thanks for the notice.

 

Zero Search Results:

It looks like your search on asdf came back with no results. But, clearly you have an interest in the subject, so I’ll make you a deal!

Drop me an email asking me to write about the topic and I’ll see what I can do! Be sure to let me known what you are interested in learning, or at least what you want to hear about it. I’m always looking for things to write about, and having someone already interested in a topic gives me more incentive. Thanks!

 

Archives & Portfolio

Just a little uniqueness. There are a few built in page templates. This is one for Archives, and there is another that I have to complete for a Portfolio. These look similar to regular articles, except there is a custom icon/image instead of the date.

archives

 

Theme Screenshot

Finally I made sure to add a little jazz when someone is selecting the theme. Here is the screenshot image I decided on:

screenshot

 

Thoughts?

I’d like to know what you guys think. Both the good, and the bad. The more criticism the better, otherwise I’m not going to learn and improve. Cheers.

Data URIs

I’m guessing that a lot of web developers don’t know about data URIs. Most probably won’t even have to know about them. However, I still want to go over them because I think they are clever and could prove useful. The main advantage I see is the possibility to avoid an HTTP Request, especially for a small image.

Wikipedia’s overview is quite thorough. The article also links to a number of places where you can quickly throw together some data URI’s for any mime type. The most well known is the data URI kitchen.

Quickly, the format of a data URI is as follows:

data:[<MIME-type>][;charset="<encoding>"][;base64],<data>

Note that almost every section is optional! The mime type option means you can do some very interesting things. The normal usage might be to to embed a small image, as a datauri, inside the html document. However, another usage might be to embed an entire HTML document inside a link!

I was looking at the code for some command line utilities that make datauri’s and I learned an easy way to find out the mime type of a file is to use the `file` command:

shell> file sym.png
sym.png: PNG image data, 76 x 78, 8-bit/color RGBA, non-interlaced

shell> file -i sym.png
sym.png: regular file

shell> file --mime sym.png
sym.png: image/png

So using `file –mime` and reading whats after the ‘: ‘ you can easily get a mime-type for any file. Then reading and base64 encoding the content you have a quick hack to quickly create datauris! Here goes:

#!/usr/bin/env ruby
# Quick & Dirty data URI for a file
#
require 'base64'
require 'uri'

# Filename and Contents
filename = ARGV[0]
contents = File.read(filename)
base64 = Base64.encode64(contents).gsub("\n",'')

# Mime Type for the Filename
output = `file --mime #{filename}`
mime = output.match( /: (.*)$/ )[1].downcase.gsub(/\s/,'')

# Make Data URI
datauri = "data:#{mime};base64,#{base64}"

# Output
puts datauri

Some better scripts are available. Here is a Perl implementation and a pretty neat Ruby Implementation.

search