PicNet Custom Software Development

.Net and Javascript Development

Annotated, Google Closure Javascript Compiler - Visual Studio Snippets

Posted on clock June 3, 2010 09:12 by author guido

I have spoken in the past about my great respect for the closure project.  This project brings some semblance of order to the chaotic and dangerous world of large application javascript development.  However it also has some problems, the biggest of this being the fact that it is very verbose.  This small set of Visual Studio 2010 snippets aims to aliviate that issue when developing in VS2010.  I have been using a similar set of these snippets in production for many months (in the Mouse Eye Tracking service) and whilst not being industrial strength they are ready for you to get some efficiencies from.

Why Visual Studio?

At PicNet most of our server side code is .Net so when developing html and javascript we also use Visual Studio (saves opening up a new IDE).  However the snippet template language is just XML so if you are interested in doing an XSLT into another language please get in contact with me and we can bring that into the project also.

How to install

 

  • Download the vsi file
  • Run (double click - will be recognized by Visual Studio) and install
  • Ready to use (by typing the shortcut name followed by TAB)

 

Note: The VSI file is just a zip file with the extension renamed so if you are worried about running strange files just rename to .zip and import the snippets manually into the IDE.

Current Snippets

ShortcutContentDeclarations
jsclass
/**
 * @fileoverview $classcomments$
 *
 */
goog.provide('$namespace$.$classname$');
goog.require('$require$');
/**
 * @constructor
 * @param {string} $param1$ A sample parameter.
 */
$namespace$.$classname$ = function($param1$) {
	$contents$
};
  • classcomments: Comments describing this class
  • namespace: The name of this class' namespace
  • classname: The name of this class
  • require: Classes to import
  • param1: The name of the first constructor parameter.
  • contents: The contents of the class.
jsconst
/**
 * @const
 * @type {$typename$}
 */
$namespace$.$constname$ = $constvalue$;
  • typename: The type of this constant.
  • namespace: The namespace that this constant resides in.
  • constname: The name of this constant.
  • constvalue: The value of this constant.
jsconstructor
/**
 * @constructor
 * @param {string} $param1$ A sample parameter.
 */
$namespace$.$classname$ = function($param1$) {
	$contents$
};
  • namespace: The name of this class' namespace
  • classname: The name of this class
  • param1: The name of the first constructor parameter.
  • contents: The contents of the class.
jsenum
/**
 * $enumcomments$
 * @enum {string}
 */
$namespace$.$enumname$ = {
  ON: 'on',
  OFF: 'off'
};
  • enumcomments: Comments describing this enumeration
  • namespace: The name of this class' namespace
  • enumname: The name of this enumeration
jsextends
* @extends {$namespace$.$inherits$}
  • namespace: The name of this class' namespace
  • inherits: The name of the inherited class
jsimpclass
/**
 * @fileoverview $classcomments$
 *
 */
goog.provide('$namespace$.$classname$');
goog.require('$require$');
/**
 * @constructor
 * @implements {$namespace$.$implements$}
 * @param {string} $param1$ A sample parameter.
 */
$namespace$.$classname$ = function($param1$) {
	$contents$
};
  • classcomments: Comments describing this class
  • namespace: The name of this class' namespace
  • classname: The name of this class
  • implements: The name of the implemented interface
  • require: Classes to import
  • param1: The name of the first constructor parameter.
  • contents: The contents of the class.
jsimplements
* @implements {$namespace$.$implements$}
  • namespace: The name of this class' namespace
  • implements: The name of the implemented interface
jsinherits
goog.inherits($namespace$.$classname$, $namespace$.$inherits$);
  • namespace: The name of this class' namespace
  • classname: The name of this class
  • inherits: The name of the inherited class
jsinterface
/**
 * @fileoverview $interfacecomments$
 *
 */
goog.provide('$namespace$.$interfacename$');
goog.require('$require$');
/**
 * @interface
 */
$namespace$.$interfacename$ = function() {};
$namespace$.$interfacename$.prototype.$methodname$ = function() {};
  • interfacecomments:
  • namespace: The name of this interface's namespace
  • interfacename: The name of this interface
  • require: Classes to import
  • methodname: A method definition in this interface.
jsparam
/**
 * @param {string} $param1$ A sample parameter.
 */
  • param1: The name of this parameter.
jsparaminline
* @param {string|number=} $param1$ A sample parameter.
  • param1: The name of this parameter.
jsprivate
		/**     
		 * @type {$typename$}
     * @private
     */
    this.$attrname$ = $declaration$;
  • typename: The name of this attribute's type
  • attrname: The name of this attribute
  • declaration: The initial declaration of this attribute
jsprivateinline
* @private
 
jsprivatevar
		/**
		 * @type {$typename$}
     * @private
     */
    var $attrname$ = $declaration$;
  • typename: The name of this attribute's type
  • attrname: The name of this attribute
  • declaration: The initial declaration of this attribute
jsprotected
		/**     
		 * @type {$typename$}
     * @protected
     */
    this.$attrname$ = $declaration$;
  • typename: The name of this attribute's type
  • attrname: The name of this attribute
  • declaration: The initial declaration of this attribute
jsprotectedinline
* @protected
 
jsprotomem
		/**
 * @param {*=} $param1$
 */
$namespace$.$classname$.prototype.$methodname$ = function($param1$) {				
		return this.member * args;
};
  • namespace: The name of this class' namespace
  • classname: The name of this class
  • methodname: The name of this method
  • param1: The name of the sameple parameter
jsprovide
goog.provide('$namespace$.$classname$');
  • namespace: The name of this class' namespace
  • classname: The name of this class
jsrequire
goog.require('$namespace$.$require$');
  • namespace: The name of this class' namespace
  • require: Classes to import
jsreturn
* @return {string}
 
jssubclass
/**
 * @fileoverview $classcomments$
 *
 */
goog.provide('$namespace$.$classname$');
goog.require('$require$');
/**
 * @constructor
 * @extends {$namespace$.$inherits$}
 * @param {string} $param1$ A sample parameter.
 */
$namespace$.$classname$ = function($param1$) {
	$contents$
};
goog.inherits($namespace$.$classname$, $namespace$.$inherits$);
  • classcomments: Comments describing this class
  • namespace: The name of this class' namespace
  • classname: The name of this class
  • inherits: The name of the inherited class
  • require: Classes to import
  • param1: The name of the first constructor parameter.
  • contents: The contents of the class.
jstypedef
/** @typedef {$type$} */
$namespace$.$typedefname$;
  • namespace: The name of this class' namespace
  • type: The composite type of this definition
  • typedefname: The name of this typedef

License

MIT

Development

If you are interested in helping mantain the source code, let me know and we'll organise something.  

Mantaining the code is very straight forward simply:

 

  • Download the latest source.
  • Open the solution in visual studio
  • Edit the snippets in the \snippets directory
  • F5 to build a new VSI file and the supporting documentation
  • Install the VSI as described above to test your changes

 

 

Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd


Lean css in javascript

Posted on clock April 19, 2010 08:00 by author guido

This weekend I was looking at the lesscss.org project and found it very interesting.  There is also a .net implementation.  The basic features of these projects is to make css less verbose, they do this by offering features like mixins, variables and nested rules.  Now all of this is very straight forward lexical analysis to produce raw css so I thought to myself why does this need a server side component, what if I want to have a pure html site.  I don't want php, ruby, .net, etc just to compile some css.  I also don't want an additional deployment process to compile my css.  So... Why not javascript.

I thought there must be some limitations but a very quick test shows that it is entirely possible.  So here we go:

We want to prevent the bowser from doing 2 requests to css so change all your:


<link rel='stylesheet' href='style.css'/>

With:


<link rel='lss' href='style.css'/> <!-- I have called it lss for less but it can be anything -->

This will prevent the borwser from requesting this file, which is good as we will be manually requesting them. So now lets add a script file to load and parse our lss files:


<script src="jsless.js" type="text/javascript"></script> 

Content of jsless.js (Note I'm using jQuery)

 

$('link').each(function() {

  var lss = $(this);

  var lssurl = lss.attr('href');

  // Should we make this block?  Probably, but lets leave for now

  $.ajax({url: lssurl, dataType:"text", success: function(css) { getcsscallback(lss, css); }});

});

 

function getcsscallback(lss, cssText) { 

  var vars = getVariables(cssText); 

  for (var i in vars) {

  i = trim(i);

  var val = trim(vars[i]); 

  cssText = cssText.replace(/@[A-z^:]+:.*/g, ''); // remove var line

  cssText = cssText.replace(new RegExp(i,  'g'), val); // replace replace variables

  }

  cssText = '<style>' + cssText + '</style>'; // TODO: Add media, etc, etc from the attributes of lss into this tag

  $('head').append(cssText);

}

 

function getVariables(cssText) {

  return getMatches(/(@[A-z^:]+):([^;]+);/g, cssText);

}

/// Following code is just supporting regex and string manipulation code

function getMatches(regex, text) { 

  var matches = {};

  var match = null;

  while (match = regex.exec(text)) {  

  matches[match[1]] = match[2]; 

  }

  return matches;

}

 

function trim(str, chars) {

  return ltrim(rtrim(str, chars), chars);

}

 

function ltrim(str, chars) {

  chars = chars || "\\s";

  return str.replace(new RegExp("^[" + chars + "]+", "g"), "");

}

 

function rtrim(str, chars) {

  chars = chars || "\\s";

  return str.replace(new RegExp("[" + chars + "]+$", "g"), "");

}

 

Thats it, I have a test project set up that you can use to see this in action.  You can also just click here to see the html.  The css (lss) used is here and the js is here.

 

I personally think this project has some potential so if you are interested in working with me just email me and we can set up an OS project and start from there.

 

Note, I have tested the above in chrome, firefox and IE8 and it appears to work.

 

Anyways hopefully this inspires someone to do this properly.

 

Thanks

Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd


Javascript Tips, Tricks and Hacks

Posted on clock April 14, 2010 06:08 by author guido

JavaScript is by far the most expressive language I use today, that expresiveness is a result of the amount of flexibility that the language gives the developer.  This is also the same reason why it is so hard to develop in JS, however I will not be focusing on that here.  This article is solely about the art of JS development.  I will be following this article up with a series of usefull patterns in javascript and usefull utility classes, but for now lets focus on the sugar.

Note: This article is extremely subjective so if any of the following views offend please accept this as my sincerest of apologies.

Variables

Sometimes the way you do things has no impact on the results but it just 'feels' (or 'smells' if you want to Fowler it up) better.  One very clear example is variable declaration.  Example;

var i;

var x = 10;

var y =100;

var product = x*y;

I prefer this:

var i,

     x = 10, 

     y = 100.

     product = x*y;

Why? Don't know, I just do.

Also, you must always remember that assignment always returns the value of the assignment so why not do something like:

var product;

callFunction(product = x * y);

return product;

When doing multy variable assignment (separated by ,) then the last assignment is returned.  This is rarely usefull so no sugar there.

 

Conditions

Become familiar with falsy expressions.  Use them in your conditions.

So instead of:

if (typeof console !== 'undefined' && typeof console.trace !== 'undefined') console.trace(); 

Why not?

if (typeof console !== 'undefined'  && console.trace) console.trace(); 

Also, lazy conditions (or null-coalescing) are also cool.  Example:

x = x || getDefaultValueOfX(); // Basically only call getDefaultValueOfX if x does not have a value.  Synonimouse with ?? operator in C#

 

Loops

Iteration are a very personal thing some prefer to go backwards some forwards.  Some declare their 'i' before their 'for' some think this is very silly indeed.  This is how I loop.

for (var i = 0, e; e = array[i]; ++i) { ... }

Nice...  Oops, better mention a caveat here.  The termination expression here is e = array[i] which means that if the array contains: 0, '', null, undefined, false then the iteration will finnish prematurely.

But this syntax can also be used for lots of nice expressive statements like:

for (var i = 0, e, j = array.length - 1; e = array[i]; ++i; --j) { ... } 

How About:

for (var e; entries.length && (e = entries.shift());) { ... } // Awesome I hear you say?

etc, etc

 

Arrays

Arrays in javascript are very interesting, they are like an object with growths.  Some of those growths are very usefull but the fact that its still an object is pretty awesome also. So:

var x = [1, 2, 3 , 4];

x.max = function() {

 var m = Number.MIN_VALUE; 

 for (var i = 0, v; v = this[i]; ++i) { if (v > m) { m = v; } }

 return m;

}

How about (Continued from above):

x.length = 100;  // This actually increased the size of the array to 100 and filled in gaps with undefined

x.length = 2;  // This will shorten the array and 'delete' all enties above the specified index.

Something that needs to be mentioned here is the awesomeness of delete.  This marks a memory space for garbage collection so its a great way to ensure that something is infact 'deleted'.

Also, I highly suggest getting to know splice.  He can be a good friend.

 

Augmentation

The previous example (array x.max) is actually a form of augmentation but this concept can be taken even further and one of the pretties uses of this is a Crockfordian parasitic inheritance.  Let's have a look:

function Animal() {

 this.tallk = function() { return "Oink! Oink!"; }

}

 

function Dog() { // Dog will extend Animal parasitically (does that word exist?)

 var d = new Animal();

 d.talk = function() { return "Woooof, Grrrrrrrrrrr!"; }

 d.playDead = function() { return '.............'; }

}

alert(new Dog().talk());

There we go, we just implemented inheritance without ugly .prototype (s) and other nasties. 

 

Closures

If you ever have a chat to a Ruby developer they will probably bore you half to death with their preaching about the beaty of a closure, anyways JS has closures also, but javascript is a more practical language so we don't don't preach we just do.

Have you ever done this:

somearray.sort(function(a, b) { return a - b; });

We'll Array.sort takes in a cloure.  So closure is basically just a function that can be passed around and reused.  This is actually an amazingly powerful feature and can open a lot of doors when designing complex systems.  However, care must be taken when using closures, the context and hence the memory used up by the context of a function remains in memory as long as the closure is alive.  This is a common casue of JS memory leaks.

 

Hashtables (Dictionaries)

Objects are dictionaries.  This is actually a very usefull data structure so don't ignore it.

var dict = {};

dict['property1'] = prop1;

dict['property2'] = prop2;

dict['sub'] = function() { return this.propert1 + this.property2; };

 

Namespaces

Namespaces are usefull, they allow you to nicely modularise your code and prevent your code from clasing with other's code.  Namespacing in javascript is very easy.  I suggest the following approach:

// Create the namespace (using the uril finction below)

namespace('au.com.picnet.scripts');

 

// Add a class to the namespace

au.com.picnet.scripts.TestClass = function() {

 this.test = function() { alert('test'); }

};

 

// Instantiate the class and call test()

new au.com.picnet.scripts.TestClass().test();

 

// A safe namespace generator, can be called many times 

// with the same ns fragmens without fear of loosing objects.

function namespace(ns) {

 ns = ns.split('.');

 var current = window;

 for (var i = 0, n; n = ns[i]; ++i) {

 if (!current[n]) current[n] = {};

 current = current[n];

 }

}

 

Function Scope

The previous example of namespaces shows function scoping in action.  Function scope simply means that a function only has access its own members and its parent memebers (if it is nested).  So a parent funciton does not have access the the members of the nested function unless they are public.  This is actually a complex toic that I may explore further in its own article but for the time being I recommend reading this.

 

This!!

The this keyword is also a tricky beast especially when it can be manipulated with Function.apply and Function.call.  But generally this points to the current function (not the current function literal) or else the global scope.  Again too complex for this article but read this.

Note I generally try not to rely on 'this' very much.  If I need to access 'this' in future I generally create a temp variable that I can use:

var instance = this

This variable is created at a point I know that 'this' points to the right object and then don't worry about the technicalities.

 

Static Classes (Singleton, Utility Classes)

Object literals actually make great singletons.  Let see an example:

namespace('au.com.picnet.scripts');

au.com.picnet.scripts.Utilities = {

 add: function(x, y) { return x + y; },

 subtract: function(x, y) { return x - y; }

};

alert('add: ' + au.com.picnet.scripts.Utilities.add(1, 2) + ' subtrace: ' + au.com.picnet.scripts.Utilities.subtract(1, 2));

 

RegExp

The RegExp is an awesomly powefull object, it can make a 100 line parsing function vanish in a puff of elegance.  Howewer!!!!!  If you are anything like me, you need to use RegExps roughly 2-3 times a year.  The other times you just copy a previous use of the same regexp.  So when that time of year comes around you ask yoursel: How the hell do I do named groups again?

Well I have found that the best way to work with RegExs is to ignore them until needed.  When you do need a regex simply give yourself a 5 minute refresher here and make sure you test your regex thrououghly in places like here.  

I personally recommend not wasting too long getting good at RegExps as they are so rarely needed.  I have actually been good at RegExps several times but that all goes away... I have now accepted it.

 

Misc

- Always use '===' instead of '==' .

- Functions have a length attribute.  This is the length of expected parameters.

- Functions have an argumens array with all the arguments passed to the function, regardless of function signature.

 

Restraining the Beast

Javascript does give the developer too much rope, this is nice when trying to flex your creative powers but can become overpowering when working on large systems.  There are a number of tools which now allow you to try to tame this beast.  And at the top of the list is GWT (And other such projects).  However this is an extreme solution to this problem I personally like Javascript and don't really want to stop using it unless there are clear advantages on a project.  So some of the tools I use and look interesting are:

- Google's Closure Compiler : Adds some type safety to your beast (with annotations)

- FireBug : Don't leave home without it

- DOM / Ajax Helper Libs (jQuery, dojo, prototype, closure libs, etc): A must for cross browser development (I personally use jQuery as it seems to have won the popularity contest and regardless what the 'hardcore' tell you, this is actually very important)

 

The next generation of JS IDEs are also very exiting, leading this list are:

- WebStorm

- Visual Studio 2010

 

So Notepad++'s domination of the js development world may be comming to an end soon.  Ahhh I actually have dreams of accurate type inferrence in real time for a JS IDE.

 

Thanks

Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd


Google Explorercanvas (excanvas) for IE. Silverlight vs VML

Posted on clock March 15, 2010 05:08 by author guido

 

Many people do not realise that excanvas has a silverlight implementation.  This implementation is apparently less stable (I have had very few issues with it) but much much faster.  The only real draw back as I see it is that it requires clients to have silverlight installed (stat owl has this penetration rate at about 40%).

I use excanvas (silverlight) to allow support IE users to use the Mouse Eye Teracking Service.  Now this is a highly graphical intensive application so the demands I put on my canvas tag are huge.  The standard excanvas (VML) implementation just did not cut it at all however the silverlight version has allowed me to support IE.

The purpose of this article is to more intelligently compare the performance of both ex canvases.  I have used slightly modified versions of the excanvas test cases for performance measures.  You can download all the tests here.

 

All tests were run in IE8 on a poor spec Win XP box. This may be quite a limited but it suffices to give us a comparison between VML excanvas and silverlight excanvas.  The tests measure the time taken to do 200 iterations of the mentioned test.  Please see the source code of the tests for more details.

Note: I am using the dev versions of excanvas from here.


TestVML (ms)Silverlight (ms)x times faster
Arc 1250 562 2.22
Clearpath 1109 281 3.95
Colors 9141 1906 4.8
DrawImage 3156 234 13.49
DrawImageFlip 128812 n/a n/a
Gradient 828 406 2.04
Gradient2 1359 422 3.22
LineWidth 378669 18953 19.98
Overflow 6859 344 19.94
Overlay 2703 n/a n/a
Pattern 3140 n/a n/a
QuadraticCurve 5032 n/a n/a
Resizing n/a n/a n/a
Restorepath 2484 n/a n/a
StrokeScaleAndRotate 7860 n/a n/a
StrokeShouldNotClosePath 4766 n/a n/a
Text 6844 n/a n/a

Conclusions

We can see that the silverlight excanvas has very limited support for many features.  However the features it does support are many multiples faster (2 - 20 times faster).  So if you are not using the advanced features, use silverlight.

 

Thanks

 

Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd


Automatic Google Page Position Report

Posted on clock March 10, 2010 06:38 by author guido

Overview

The position of your website in the Google organic search is quickly becomming one of the most critical marketting measures.  This page/script will allow you to automate this process.

How To

  • Sign up for a google Ajax API Key (http://code.google.com/apis/ajaxsearch/signup.html)

- You will get a jsapi key from google, something like this

            'ABQIAXFEakaXw3_Fd-

zqqMhjDLzqaRTTser7lsytcEDBoz0jKRWQmOpxexR7x409podV88a5eoPr2KIvw8Ub3B'

 

 

            (Note this is not a valid key)

  • Download this html page/script and rename to .html (remove .txt extension)

  • Open this page in a text editor (note pad is just fine)

  • Replace <YOUR JSAPI KEY> (Line 53) with, you guessed it, your jsapi key.

  • On line 59 you will see the configuration for this file, you will need to change the following items

- lookfor (Line 66, 78, 87).  These tell the script what to look for in the url and youtube title.

 

- On line 75, replace "Sydney, NSW" with a location that you think some of your clients would be searching from. 

  • On line 228 the lookup table rows begin.  You can add as many rows as you wish here.  The rows must be in the format (replace <Search term or phrase>):


<tr><td><Search term or phrase></td></tr>

  • You can have headers by adding rows in this format:


<tr class='header'><th>This is a header row</th></tr>

  • Open the file in your browser and click 'Run Report'


Thats it, when the table is working you will get something that looks like this:

 

Let me know if you have any problems with this.

 

Thanks

Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd


How to prevent 'Stop running this script' message in browsers

Posted on clock March 4, 2010 07:23 by author guido

Avoiding the 'Script taking too long' (all browsers have some form or another of this) message in browsers is relatively simple. You just have to make sure the browser knows you have not created an endless loop or recursion. And the easiest way to do is is to just give the browser a breather in between long running tasks.

So how would we go about this.  Well the simplest solution is to just break up your task into mutliple smaller tasks with a setTimeout in between these tasks.  The utility class below does just this:

Utility Class


RepeatingOperation = function(op, yieldEveryIteration) {
var count = 0;
var instance = this;
this.step = function(args) { 
if (++count >= yieldEveryIteration) { 
count = 0;
setTimeout(function() { op(args); }, 1, [])
return;
}
op(args); 
}; 
};

So how do we use this class, lets say we have the following code which is giving us a jerky browser and occasionally displaying those horrible 'Stop running this script' message:


// initdata is just an array of numbers (a very very large array)
var test1 = new Array(initdata.length);
for (var i = 0; i < initdata.length; i++) { test1[i]  = initdata[i] * 2; } // Double each item in the initdata array
continueOperations();

To use the utility class above we would change the code to this:


var test2 = new Array(initdata.length);
var i = 0;
var ro = new RepeatingOperation(function() {
test2[i] = initdata[i] * 2;
if (++i < initdata.length) { ro.step(); }
else { continueOperations(); }
}, 100);
ro.step(); 

That's it, a little bit more code, an extra closure, so not pretty but I think a relativelly stylish solution to a nasty problem.  Note: Remember this will also give you a much more responsive browser during execution of this expensive code also.

Note: This solution is used in the Mouse Eye Tracking system.  So it is production ready (at your own risk tho).

Thanks

Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd

 


Introducing Mouse Eye Tracking - Tunnel Vision

Posted on clock February 19, 2010 08:46 by author guido

One question that has been asked of me a few times since the release of the PicNet Mouse Eye Tracking Service is that of accuracy. How acurate can the Mouse Eye Tracking service correlate eye activity from mouse activity. We have done a lot of testing on this topic and we believe that the correlation is significant. In fact it was significant enough for us to invest a huge amount of time and effor in producing this product. We are also highly encouraged by the academic research in this field that supports our own tests (See product home page for links to this research).

However, we are aware that the accuracy is not 100% and it will never be. If you want 100% accurate data about activity on your site you need to use eye tracking. However this also has many downsides. One of the ways that we try to increase the correlation between mouse and eye is by using an as yet undocumented feature called Tunnel Vision.

Tunnel Vision tries to encourage the user to use the mouse to view the page whilst attempting not to alter the normal behavior of the user on the page. Lets have a look at tunnel vision in action:

Tunnel Vision Off


Tunnel Vision On


As you can see, tunnel vision creates a tunnel around the user's mouse giving you a clear area of vision around the cursor but more importantly tunnel vision still allows the user's peripheral vision to look around the page in preparation for the next area of focus.

I encourage you to try tunnel vision in your closed testing (Do not put this on your live site!!!) and see the results for your self.

Turning On/Off
To turn on tunnel vision simply add (use-tunnel-vision="true") to your PicNetEyeTracker span. i.e:


<span id='PicNetEyeTracker' usercode='picnetpilot' use-tunnel-vision="true">...


To turn off tunnel vision simply remove this tag or set use-tunnel-vision to "false".

I hope this feature helps you get that extra level of accuracy with your mouse/eye tracking.

Thanks

Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd


Html Web Worker Woes - Data Analysis not an Option

Posted on clock February 18, 2010 04:25 by author guido

During development of the PicNet Mouse Eye Tracking service I carefully investigated the possibility of using Html Web Workers.

After careful reading of the specs I thought this would be great but after some initial tinkering I found that communications between threads was done through the postMessage feature supported by modern browsers.

This instantly raised some suspicions as I know that this is a text only feature, I know FF will auto serialize/deserialize to/from JSON but this as expected performs pretty poorly for large data sets. But I gave it a benefit of the doubt thinking that there was no way that someone would go to the extreme pains of implementing a threading framework and cripple it by not allow memory references to be passed between threads.

WRONG!!! And now I can say that the new threading features in browsers is only usefull for making the UI more responsive. Now don't get me wrong thats great and a huge relief not to be having to use setTimeout hacks everywhere but seriously?? Why would you break my heart like that?

Threading is critical in calculation intensive or data analysis intensive applications. The PicNet Mouse Eye Tracking service is a great example of this. The javascript engine gets a whole bunch of mouse coordinates and from this data we render heat maps, curved mouse tracks, etc, etc. This is a huge amount of calculations that lock the browser JS thread but it is impossible for me to use web workers. Why is that?

The cost of postMessage
postMessage can transfer strings between threads. But it is very rare that data requiring analysis is solely string based, mostly we are working with other primative types as well such as numbers, booleans, DateTimes, etc. And the cost of converting (serializing) strings to/from these data types is huge. After some early prototypes I tossed the idea of using Web Workers aside. I know this is very early days in the life of Web Workers but I was quite taken aback by this crippling flaw in the specs.

Lets look at numbers
K, lets support my whinging with numbers. Below is a table that shows an expensive operation on an array of numbers. We process the array using the specified number of threads. Note: Refresh to see it running in real time:

Run Test
1 Threads2 Threads3 Threads4 Threads5 Threads
IdleIdleIdleIdleIdle


Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd


GWT in a .Net environment

Posted on clock February 3, 2010 04:11 by author guido

Hi All,

As you know we have recently release the PicNet Mouse Eye Tracking service. This project is by far the largest Javascript centric application that PicNet has developed. This led us to do a lot of research on what technologies we would use for generating a maintainable javascript project.

We were restricted to having a .Net backend as that is what our existing infrastructure supports and its the technology where we live most comfortably. So the decision for the backend was simple, ASP.Net Mvc (w/ Spark View Engine) + NHibernate.

But now came the more complex decision of the client code (javascript). Do we stick to standard jquery code minified in our continous build? do we create our own bootstrapping js framework to easily allow OO in js? Do we leverage other such frameworks?

We ended up investigating a lot of different options but this article focuses on one of these which is GWT.

Javascript development is hard, anyone that tells you otherwise thinks javascript development is for creating roll over buttons. JS development is hard for various reasons.
  • The language is very flexible and it gives you all the rope in the world to ...
  • At the heart of the language is the function. This is a concept that is difficult for most OO devs to wrap their heads around
  • Performance is such a huge issue, compared to server languages
  • Code size (and design) is a big issue
  • Finally and most importantly is: The tools for javascript development are useless in comparison to other languages
The last item is huge. I mean with Java you use Eclipse or IntelliJ to manage your projects, builds, deployments, code analysis, error reporting, debugging, syntax highlighting, unit testing, code completion, refactoring, etc, etc, etc. For .Net you have Visual Studio giving the same support. Now try this as an experiment. Write a simple app in Java or .Net using notepad! That is what javascript development is like, regardless of language features, performance, etc. The tools issue is a killer.

GWT is a great framework and I would comfortably say, its the best framework for javascript development available. The reason it is the best framework is the fact that you can use good tools (Eclipse, IntelliJ, etc), it also removes alot of the Javascript hacky mentality required for writing fast javascript (but thats a secondary benefit in my opinion). Now for .Net developers learning to use Eclipse, etc is a bit of a pain but from a maintainability issue, if you were developing a solely javascript application it is definatelly worth the effort. HOWEVER!! Having a client (javascript) / server (.Net) application written in GWT / .Net has huge drawbacks. It is actually so painful that we finally decided to abandon the attempt.

How do you do it
Well our service is a very complex html page with a very large javascript component. The html file we generate using Spark View Engine. This lets us modularise the html into nice little controls that are easy to work with. This is where we encounter our first GWT hurdle. To use GWT you have to have the html in the hosted server, well that's not true you could compile the javascript and stick it in your ASP.Net application and not have html in Java however if you cannot use GWT in hosted mode it is not worth using (My own opinion).

What does this mean? Well, basically it means that you have to view the source of your html and copy into your GWT environment. Thats correct, change html you need to remember to copy paste into GWT env every time. We tried doing crazy things like having iframes pointing to the ASP.Net app from GWT environment with cross domain policies, etc but this always failed.

Ok, this is a huge maintenance issue, however if this were the only issue I think we still would have used GWT given its previously described benefits.

Second issue!! Talking to the server. This issue actually has a few parts, the first is serialisation. GWT offers a few optimised serialisation mechanisms such as XML and JSON (and a custom one). Since our application used plain strings for its communications, rolling up our own XmlHttpRequests from GWT was no problem (we do this for performance reasons). So this was no big issue at all. The second issue is authentication. To use our application you must be logged in so in development mode we had to add a hack that would recognise GWT development requests and simulate a user, Ughh!! Still not a huge issue but getting bigger. And finally the last issue is the fact that to use GWT with a .Net backlend in development you have to use a proxy server. No problem here is a great article describing how to set this up. Now it is difficult to describe the development process here but I will try.
  • Start the hosted mode server
  • Oh forgot to copy the new html
  • Copy the html
  • Restart hosted mode
  • Crash!!
  • Hughh?
  • Oh, a crash on the .net side
  • Done, try again
  • Hmmm yes it does look better with that border
So you get my pain?

So my conclusions. GWT is awesome!! Enough cannot be said about how good this toolkit is but it is still too painful to be a real alternative in a .net application.

Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd


PicNet Mouse Eye Tracking Service Limited Release

Posted on clock January 27, 2010 06:43 by author guido

You may have noticed that I have been a bit quiet over the last few weeks, this is because we have been working very hard to release a new product and it gives me great pleasure to announce that today we have flicked on the switch.

The product is called the Mouse Eye Tracking service. It is basically a web analytics product that records your visitors mouse interactions with your web site. You then use our product to analyse these interations and get value from them. You can think of it as a more visual 'Google analytics' (however the products are very different and supplement each other nicely).

Features
Heat Maps
Check out where our visitors are viewing our home page:


Or where they are 'clicking'


User Trackings
You can also just replay the user interactions on your site. Either one at a time or all at once:


Page Navigation
You can also visualize how your visitors are navigating around on your site:


Technology
One of the reasons I'm so proud of this product is because of the number of technical hurdles that had to be overcome to get a successful solution. I will be blogging about these in detail over the next few weeks in the hope that someone can learn from our trials and tribulations. But some of the issues we had to address were:

  • Capacity of a high volume, highly data driven application
  • Speed of visualisation generation on javascript
  • Threading in javascript
  • Best tools for large development team in a javascript project
  • Large javascript application programming practices
  • Html 5
  • IE Issues (Which we have deferred for now)
  • Unit testing in javascript (and continous testing) + multiple browsers


Limited Release
As mentioned above, this is a limited release. So no media releases yet, we are taking it slowly just to make sure we have nailed down all the capacity issues. But this is too exciting not to at least blog about it so feel free to subscribe and give it a try.
Please send me any feedback you may have, either through comments below, email or just use the built in feedback submission form in the system.

Links

Thanks All

Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd