JavaScript

Hack #82 Web Wokers: Basics of the Web Browser's UI Thread

This post is mirrored at: html5hacks.com

Single Threadin'

As we set out to build a highly responsive UI for our demo web application, we must fully understand how browsers manage processes. Perhaps the biggest challenge we will face has to do with browsers using a single thread for both JavaScript execution and user interface updates. While the browser's JavaScript interpreter is executing code, the UI is unable to respond to the user's input. If a script is taking a long time to complete, after a specified amount of time the browser will provide the user an option to terminate the script. To accommodate for the 'freeze' associated with scripts that exceed the browser execution time limit, web developers have traditionally created smaller units of work and used JavaScript timers to return execution to the next event in the queue. As you will see, web workers solve the locking of the UI thread by opening an additional thread, or even multiple threads, for execution of these long running, processor intensive tasks.

When designing your application, especially if you come from more of a 'server-side' or Java background, it is important to understand that non-blocking execution is not the same as concurrent threading. While not extremely complex, JavaScript's event driven style does take some getting used to for developers coming from other languages such as Java and C. Later, we will touch on a few examples where we pass a callback continuation function to take full advantage of JavaScript's non blocking design.

Thread Safety

Mozilla, in particular, provides a Worker interface which web workers implement. While the Worker interface spawns OS-level threads, web workers use the postMessage mechanism to send messages (with serializable objects) between the two execution contexts. To ensure thread safety the worker is only given access to thread safe components to include the following:

  • timers: setTimeout() and setInterval() methods
  • XMLHttpRequest
  • importScripts() method

The worker script can also make use of:

  • navigator and location objects
  • native JavaScript objects such as Object, String, Date

At the same time, the worker restricts access to DOM APIs, global variables, and the parent page. In Hack #84 Building the DOM with web workers and Handlebars.js, we will explore the restricted access to DOM APIs, and introduce JavaScript templating, importScripts, and the use of timers to poll for postMessage.

HTML5 Web Workers

As mentioned earlier, the Web worker spec defines an API for executing scripts in the background by spawning an independent execution context.

It is important to note that web workers are costly, having an impact on startup and overall memory consumption. So, they are intended to be used lightly and in conjunction with the some of the asynchronous techniques mentioned earlier. A well built client-side application would typically make use of one or two cases where tasks are expensive. Here are a few common uses cases:

  • Single Page Application bootstrapping large amounts of data during initial load
  • Performing long running mathematical calculations in the browser
  • Processing large JSON datasets
  • Text formatting, spell checking, and syntax highlighting
  • Processing multimedia data (Audio/Video)
  • Long polling webservices
  • Filtering images in canvas
  • Calculating points for a 3D image
  • Reading/Writing of local storage database

Long Running Scripts

Our first web worker hack will be a long running script with heavy computation. It will execute 2 loops that output a two-dimensional array. First, we will use this computation to lock up the browser's UI thread, and later we will move the task to a worker. To further demonstrate the performance strain, we will also animate a box horizontally across the screen. You will notice the animated box stop while the UI thread is executing the long running script.

	var box;

	function goRight() {
  		box.style.left = parseInt(box.style.left)+1+'px';
  		setTimeout(goRight,2); // call goRight in 200msec
	}

	function init() {
		var r = 1000;
		var c = 1000;
		var a = new Array(r);

		for (var i = 0; i < r; i++) {
		    a[i] = new Array(c);

		    for (var j = 0; j < c; j++) {
		        a[i][j] = "[" + i + "," + j + "]";
		    }
		}
		document.getElementById('result').textContent = a;
		
  		box = document.getElementById('box'); // get the "box" object
  		box.style.left = '0px'; // set its initial position to 0px
  		
		goRight(); // start animating to the right
	}
	
	window.onload = init;

Spawning a Worker

Now let's move our heavy computational task to a dedicated web worker, so that the user doesn't have to wait for the script to complete execution in order to interact with user interface. First, lets spawn a new worker:

  
  var worker = new Worker('highComp.js');

  worker.postMessage(JSON.stringify(message));

  worker.addEventListener('message', function(event){}, false);

Here, we define an external file that will contain the logic of our heavy computational task. The file, highComp.js will listen for a message that will receive the serialized JSON payload, and then we will set up an additional listener to receive a message back from highComp.js.

Now, we can move this cpu-intensive task to a separate file: highComp.js

var r = 1000;
var c = 1000;

var a = new Array(r);

for (var i = 0; i < r; i++) {
  a[i] = new Array(c);

    for (var j = 0; j < c; j++) {
     a[i][j] = "[" + i + "," + j + "]";
    }
};
postMessage(a);

In highComp.js, our two dimensional array is built and set to variable a. It is then passed back to our main script via the postMessage call.

In the next hack, we will mix our use of timers with the power of a dedicated worker. As we send messages (passing serializable objects as a parameter to postMessage) back and forth to code running in the shared UI thread, our timer will periodically check for new messages and use their contents to modify the DOM.

YUI3 Rails Application Template

I decided to put together a Rails template to generate a quick sqlite3 db driven web app to test out YUI3 functionality quickly. Rails 3 makes it super simple to quickly generate real JSON data for testing out various YUI 3 components such as DataSource.

The template takes care of removing the Prototype library, including yui-debug.js and the CSS framework including reset, and the new Grids - currently in beta. I also wanted to deliver my basic markup quickly so I've added Haml

Since I have been disciplining myself to follow Test Driven Development on the server, I included all of my Rails testing dependencies in the app template as well (RSpec, Cucumber, WebRat, Factory Girl) and I plan to continue to leverage YUI Test on the client as I run through browser validations of the datasource-polling sub-module. Clone the rails app template here.

# YUI3 Application Generator Template
# Generates a Rails app; includes YUI3, Haml, RSpec, Cucumber, WebRat, Factory Girl ...

puts "Generating a new YUI3 Rails app"

#----------------------------------------------------------------------------
# Create the database
#----------------------------------------------------------------------------
puts "creating the database..."
run 'rake db:create:all'

#----------------------------------------------------------------------------
# GIT
#----------------------------------------------------------------------------
puts "setting up 'git'"

append_file '.gitignore' do <<-FILE
'.DS_Store'
'.rvmrc'
FILE
end
git :init
git :add => '.'
git :commit => "-m 'Initial Commit of YUI3 Rails App'"

#----------------------------------------------------------------------------
# Remove files
#----------------------------------------------------------------------------
puts "removing files..."
run 'rm public/index.html'
run 'rm public/favicon.ico'
run 'rm public/images/rails.png'
run 'rm README'
run 'touch README'

puts "banning spiders from your site by changing robots.txt..."
gsub_file 'public/robots.txt', /# User-Agent/, 'User-Agent'
gsub_file 'public/robots.txt', /# Disallow/, 'Disallow'

#----------------------------------------------------------------------------
# Haml 
#----------------------------------------------------------------------------
  puts "setting up Gemfile for Haml..."
  append_file 'Gemfile', "\n# Bundle gems needed for Haml\n"
  gem 'haml', '3.0.18'
  gem 'haml-rails', '0.2', :group => :development

#----------------------------------------------------------------------------
# Set up YUI3
#----------------------------------------------------------------------------

puts "replacing Prototype with YUI3"
run 'rm public/javascripts/controls.js'
run 'rm public/javascripts/dragdrop.js'
run 'rm public/javascripts/effects.js'
run 'rm public/javascripts/prototype.js'
run 'rm public/javascripts/rails.js'

get "http://yui.yahooapis.com/combo?3.3.0/build/yui/yui-debug.js",  "public/javascripts/yui-debug.js"
get "http://yui.yahooapis.com/3.3.0/build/cssreset/reset.css",  "public/stylesheets/reset.css"
get "http://yui.yahooapis.com/3.3.0/build/cssbase/base.css",  "public/stylesheets/base.css"
get "http://yui.yahooapis.com/3.3.0/build/cssfonts/fonts.css",  "public/stylesheets/fonts.css"
get "http://yui.yahooapis.com/3.3.0/build/cssgrids/grids.css",  "public/stylesheets/grids.css"

#----------------------------------------------------------------------------
# Create an index page
#----------------------------------------------------------------------------
puts "create a home controller and view"
generate(:controller, "home index")
gsub_file 'config/routes.rb', /get \"home\/index\"/, 'root :to => "home#index"'
append_file 'app/views/home/index.html.haml'do <<-FILE
!!!
%h2{:class => "subtitle"} Get Started
%p{:class => "content"} Update application.js with your logic
%div{:class => "container", :id => "container"}
%div{:id => "testLogger"}
FILE
end

#----------------------------------------------------------------------------
# Generate Application Layout
#----------------------------------------------------------------------------

run 'rm app/views/layouts/application.html.erb'
  create_file 'app/views/layouts/application.html.haml' do <<-FILE
!!!
%html
  %head
    %title YUI3 App
    = stylesheet_link_tag "reset"
    = stylesheet_link_tag "base"
    = stylesheet_link_tag "fonts"
    = stylesheet_link_tag "grids"
    = stylesheet_link_tag "application"
    = javascript_include_tag :all
    = csrf_meta_tag
  %body{:class =>"yui3-skin-sam  yui-skin-sam"}
    = yield
FILE
end

#----------------------------------------------------------------------------
# Add Stylesheets
#----------------------------------------------------------------------------
create_file 'public/stylesheets/application.css' do <<-FILE
div.container {
  width: 100%;
  height: 100px; 
  padding: 10px;
  margin: 10px;
  border: 1px solid red;
}

#testLogger {
    margin-bottom: 1em;
}

#testLogger .yui3-console .yui3-console-title {
    border: 0 none;
    color: #000;
    font-size: 13px;
    font-weight: bold;
    margin: 0;
    text-transform: none;
}
#testLogger .yui3-console .yui3-console-entry-meta {
    margin: 0;
}

.yui3-skin-sam .yui3-console-entry-pass .yui3-console-entry-cat {
    background: #070;
    color: #fff;
}

FILE
end

#----------------------------------------------------------------------------
# Initialize YUI and add YUI Test
#----------------------------------------------------------------------------
append_file 'public/javascripts/application.js' do <<-FILE
  
  YUI({ filter: 'raw' }).use("node", "console", "test",function (Y) {

      Y.namespace("example.test");

      Y.example.test.DataTestCase = new Y.Test.Case({

          //name of the test case - if not provided, one is auto-generated
          name : "Data Tests",

          //---------------------------------------------------------------------
          // setUp and tearDown methods - optional
          //---------------------------------------------------------------------

          /*
           * Sets up data that is needed by each test.
           */
          setUp : function () {
              this.data = {
                  name: "test",
                  year: 2007,
                  beta: true
              };
          },

          /*
           * Cleans up everything that was created by setUp().
           */
          tearDown : function () {
              delete this.data;
          },

          //---------------------------------------------------------------------
          // Test methods - names must begin with "test"
          //---------------------------------------------------------------------

          testName : function () {
              var Assert = Y.Assert;

              Assert.isObject(this.data);
              Assert.isString(this.data.name);
              Assert.areEqual("test", this.data.name);            
          },

          testYear : function () {
              var Assert = Y.Assert;

              Assert.isObject(this.data);
              Assert.isNumber(this.data.year);
              Assert.areEqual(2007, this.data.year);            
          },

          testBeta : function () {
              var Assert = Y.Assert;

              Assert.isObject(this.data);
              Assert.isBoolean(this.data.beta);
              Assert.isTrue(this.data.beta);
          }

      });

      Y.example.test.ArrayTestCase = new Y.Test.Case({

          //name of the test case - if not provided, one is auto-generated
          name : "Array Tests",

          //---------------------------------------------------------------------
          // setUp and tearDown methods - optional
          //---------------------------------------------------------------------

          /*
           * Sets up data that is needed by each test.
           */
          setUp : function () {
              this.data = [0,1,2,3,4]
          },

          /*
           * Cleans up everything that was created by setUp().
           */
          tearDown : function () {
              delete this.data;
          },

          //---------------------------------------------------------------------
          // Test methods - names must begin with "test"
          //---------------------------------------------------------------------

          testPop : function () {
              var Assert = Y.Assert;

              var value = this.data.pop();

              Assert.areEqual(4, this.data.length);
              Assert.areEqual(4, value);            
          },        

          testPush : function () {
              var Assert = Y.Assert;

              this.data.push(5);

              Assert.areEqual(6, this.data.length);
              Assert.areEqual(5, this.data[5]);            
          },

          testSplice : function () {
              var Assert = Y.Assert;

              this.data.splice(2, 1, 6, 7);

              Assert.areEqual(6, this.data.length);
              Assert.areEqual(6, this.data[2]);           
              Assert.areEqual(7, this.data[3]);           
          }

      });    

      Y.example.test.ExampleSuite = new Y.Test.Suite("Example Suite");
      Y.example.test.ExampleSuite.add(Y.example.test.DataTestCase);
      Y.example.test.ExampleSuite.add(Y.example.test.ArrayTestCase);

      //create the console
      var r = new Y.Console({
          newestOnTop : false,
          style: 'block' // to anchor in the example content
      });

      r.render('#testLogger');

      Y.Test.Runner.add(Y.example.test.ExampleSuite);

      //run the tests
      Y.Test.Runner.run();

  });

FILE
end

#----------------------------------------------------------------------------
# Setup RSpec & Cucumber
#----------------------------------------------------------------------------
puts 'Setting up RSpec, Cucumber, webrat, factory_girl, faker'
append_file 'Gemfile' do <<-FILE
group :development, :test do
  gem "rspec-rails", ">= 2.0.1"
  gem "cucumber-rails", ">= 0.3.2"
  gem "webrat", ">= 0.7.2.beta.2"
  gem "factory_girl_rails"
  gem "faker"
end
FILE
end

run 'bundle install'
run 'script/rails generate rspec:install'
run 'script/rails generate cucumber:install'
run 'rake db:migrate'
run 'rake db:test:prepare'

run 'touch spec/factories.rb'
#----------------------------------------------------------------------------
# Finish up
#----------------------------------------------------------------------------
puts "Commiting to Git repository..."
git :add => '.'
git :commit => "-am 'Setup Complete'"

puts "DONE - setting up your YUI3 Rails App."


Understanding JavaScript Closures

So it took me awhile to truly understand JavaScript closures. There is limited documentation of the subject on the web, but here is a list of the resources I used to finally grasp not only what specifically creates a closure, but also why we would want to use them.

First, let's define closures:

  • A "closure" is an expression (typically a function) that can have free variables together with an environment that binds those variables (that "closes" the expression). - Jim Ley
  • ...JavaScript has closures. What this means is that an inner function always has access to the vars and parameters of its outer function, even after the outer function has returned. This is an extremely powerful property of the language. - Douglas Crockford
  • Things are different if you save a reference to the nested function in the global scope. You do so by using the nested function as the return value of the outer function or by storing the nested function as a property of some other object. - JavaScript: the Definitive Guide
  • Simply put, a closure is a variable, created inside a function, which continues to exist after the function has finished executing. - Patrick Hunlock

I should preface the more detailed explanation below with a few prerequisite concepts. If you don't truly grasp these concepts, then you may get hung up when trying to understand JavaScript closures. Fortunately, your quest to understand closures should force you to understand these concepts on a deeper level first.

  • global object and global scope
  • reference types and primitive types
  • lexical scope and the scope chain
  • the call object or the ECMA activation object
  • idea of persistent data
  • private variables
  • using return

To begin, we want to create a reusable function containing data that persists across invocations.

We don't want to hard code variables into the reusable function, and a local variable will not persist.

From what we have learned by our 'Responsible JavaScript for the Enterprise' guidelines and through our understanding of the call object, lexical scope, and namespacing: we also know that we want to greatly limit global variables in our global scope.

So instead, we want the developer to make two calls: One, to set up or 'configure' the function, and two, to invoke this 'pre-configured' function.

The power is in the first call. I like to refer to it as the configuration call.

During the configuration call, we 'freeze' the inner function by setting a reference to it in the global scope.

So this data persists. Why?

We have an external reference to this inner function.

The inner nested function retains its reference to the call object of the outer function.

The outer function's local scope resolves and the reference to its inner function remains.

This is an example from a friend that helped me finally wrap my head around the power of the technique.

function configEquation(A,B,C){
  //The equation: Ax^ex1 + Bx^ex2 + Cx^ex3
  //The exponents are ex1, ex2, and ex3
  //The coefficients are A, B, and C and are set

  var ex1 = 2;
  var ex2 = 1;
  var ex3 = 0;

//The exponents constitute a quadratic equation.
function theEquation(x){
	var result = A * Math.pow(x,ex1) + B * Math.pow(x,ex2) + C * Math.pow(x,ex3);
	return result; 
  }

	return theEquation; //notice we return the reference not the invocation of the function(no params)
  }

var myEquation = configEquation(2,3,0); //the configuration call
  console.log('myEquation, x = 2: ' + myEquation(2));
  console.log('myEquation, x = 5: ' + myEquation(5));

var myEquation2 = configEquation(1,1,1); //a 2nd configuration call
  console.log('myEquation2, x = 2: ' + myEquation2(2));
  console.log('myEquation2, x = 5: ' + myEquation2(5));

Learning JQuery

I'd like to promote a great resource for getting up to speed on JQuery syntax.

simile.mit.edu

Main Page - SIMILE

SIMILE seeks to enhance inter-operability among digital assets, schemata/vocabularies/ontologies, metadata, and services. A key challenge is that the collections which must inter-operate are often distributed across individual, community, and institutional stores. We seek to be able to provide end-user services by drawing upon the assets, schemata/vocabularies/ontologies, and metadata held in such stores.

SIMILE will leverage and extend DSpace, enhancing its support for arbitrary schemata and metadata, primarily though the application of RDF and semantic web techniques. The project also aims to implement a digital asset dissemination architecture based upon web standards. The dissemination architecture will provide a mechanism to add useful "views" to a particular digital artifact (i.e. asset, schema, or metadata instance), and bind those views to consuming services.

To guide the SIMILE effort we will focus on well-defined, real-world use cases in the libraries domain. Since parallel work is underway to deploy DSpace at a number of leading research libraries, we hope that such an approach will lead to a powerful deployment channel through which the utility and readiness of semantic web tools and techniques can be compellingly demonstrated in a visible and global community.

Powered by ScribeFire.

Syndicate content