Building Sproutcore with Garcon

We have started using Sproutcore to create rich web applications at Intellum. Recently, on a project, our Sproutcore build started to fail when using abbot (the offical Sproutcore build tool). After spending some time trying to figure out the issue with no success, we took at look at garcon: an alternative that uses node.js, instead of Ruby.

The driving Philosophy of garcon is that a build tool for a Javascript framework should be written in Javascript. The guys that are behind garcon believe someone that is learning Sproutcore shouldn't have to also learn and install Ruby to use Sproutcore.

The garcon codebase is smaller then abbot's which makes it slightly more accessible for a developer trying to understand what goes on during the build process of Sproutcore. Thus, after getting my build to work with garcon thanks to some help from erichocean, I looked into the source code for garcon to try to understand what it does. This blog post is a summary of what I learned. I hope someone else will find this useful in their quest to learn how Sproutcore gets built and to learn how garcon does it.

Differences between garcon and abbot

Some of the differences between garcon and abbot are:

• the biggest one: abbot uses Ruby, garcon uses Javascript and node.js
• garcon is stricter than abbot with file requirements so you might have to add some sc_require statements in your codebase for your project to build correctly with garcon. For example in our case, garcon was not automatically requiring up all our custom views located in our views directory before it loads main_page.js. We had to explicitely add sc_require statements to the top of main_page.js
• for our project, garcon builds our application must faster than abbot does. I believe this is due to garcon doing a lot of work in parallel thanks to the evented nature of node.js
• garcon does not support dynamically loaded bundles yet while abbot does. (Note: this should be an easy addition and could be added soon so keep monitoring the garcon project on github).
• garcon does not support Chance. Abbot does.
• abbot uses YUI Compressor to compress CSS and minify code whereas garcon uses YUI Compressor only for the CSS files and uses uglify.js for minifying the Javascript code.

Configuring your app to build with garcon

With garcon, you can either build the project into static files to deploy on your web server or start a development server which you can use while you are developing your application. Therefore, a typical application that uses garcon will have two config files: one to build the files and one to start the development server. You can find two sample config files in the garcon code base: sample_build_config.js and sample_server_config.js.
To start using garcon with your project, first clone garcon into your project:

git clone --recursive git://github.com/martoche/garcon.git

If you don't already have sproutcore into your application's framework folder, you should add it:

git clone git://github.com/sproutcore/sproutcore.git frameworks/sproutcore

Then copy the two sample config files into your application's root folder:

cp garcon/sample_build_config.js build_config.js
cp garcon/sample_server_config.js server_config.js

Now you are ready to modify your config files. First, replace all instances of myApp with the name of your application. Then, change the htmlHead to use your application name in the title of the page rather then the default 'My App'. Once that is done, add frameworks as needed by your application. For example, I use Ki with my application so I added the following line right after the comment that reads 'a third party framework':

{ path: 'frameworks/ki', combineScripts: true },

If you are using a server in your backend and want to proxy to it, you need to add a proxyHost and proxyPort when creating the server:

server = new g.Server({
proxyHost: 'localhost',
proxyPort: '4567'
});

By default, the garcon development server will start on port 8000, if you want to change the port to another port, you can add a 'port' option to the options hash when creating the server, as seen below. Note that you only need that in your server_config.js since your build_config.js will not start the development server:

server = new g.Server({
port: '4020',
proxyHost: 'localhost',
proxyPort: '4567'
});

It seems like the garcon codebase was updated by the sample config files weren't so you will need to change htmlBody and replace that with htmlBeforeScripts towards the end of the config files.
Your build_config.js and server_config.js files will be very similar except for a couple of differences. First, your build_config.js file will use some extra options to minify and combine the Javascript and CSS files:

combineScripts: true,
combineStylesheets: true,
minifyScripts: true,
minifyStylesheets: true

Second, the callback method passed on the the build() method will be different. For the server it will make a call to server.run() whereas the build_config.js will save the application using myApp.save()

Building your application with garcon

Make sure you've installed node. For our project, we use node.js version 0.4.5. Then building is very simple. Go to your application's root folder and run one of the following depending on if you want to build or launch the development server, respectively:

node build_config.js

node server_config.js

If you've build the server, the files will be stored in a build/ folder under your root folder. These are the files you need to serve from your web server.

Sometimes, you might run out of file descriptors when building with garcon. I didn't have this issue running Ubuntu but heard it could happen if you are on Mac OSX. If you do, try setting ulimit -n 1000 and try again

How garcon works: the garcon internals

I wanted to understand what is being done behind the scenes by garcon or abbot to build a Sproutcore application so I read through the garcon source code to get a good understanding of what goes on behind the scene during the build. This section will be harder to follow. You should only read if you want to learn about the garcon internals.

To follow what happens, we are going to start at the config_file which is the entry point of the build. So open your config files and follow along.

In the config files, a Server object is first created with the appropriate options. Then the application being built is added to the server using the addApp method of server. This method returns an App object which we will use in the remaining of the config file.

Once we have our App object, we add the frameworks that it uses, starting with the Sproutcore framework (using addSproutcore) and followed by the other third party frameworks, themes, and finally ending with the code for the application we are building.

Once the frameworks are added, a few configurations such as htmlHead and htmlBeforeScripts are set in the config file. These will be used to create the HTML for the page and are added here to the config file to allow you to change that to your liking. There is also a htmlAfterScripts that you can set if you want. The names should be self-explanatory as to where it sticks these fragments of HTML that you define. If it is not, look at the index.html page generated by the build and it will be obvious.

Finally, the build method is called. This is where all the preparatory work to build your Sproutcore application is done. It creates all the Javascript objects needed to then either save the project in your build/ folder or start the development server. This method has a callback that is invoked when everything is ready. That is where you decide what you want to do when garcon has gone through and prepared your build.

Let's look at what happens in this build method.

First, buildRoot() is called on the application object. This method does two things:

1. It creates the index.html file by calling rootContent().
2. It creates a symlink to index.html so that yourdomain.com/yourappname servers the index.html page

Sproutcore uses handlers to build the frameworks and the handlers can be found on handlers.js. In each build step, the appropriate handlers are called to process the framework as needed. The build() process does the following, in the order listed:

1. selectLanguageFiles is called and returns the file names for the language that we are building since Sproutcore supports internationalization
2. The stylesheets are built by buildStylesheets(). One by one they are first minified if minifyStylesheets was set to true. Then, they are joined into one file. If you use less, the resulting file then goes through the less processor. You will need to install less via npm if you want that to happen. Then the rewriteStatic handler kicks in and replaces calls to sc_static with the appropriate string. Finally, the resulting combined css file is saved using the file handler.
   (Handlers used in this step: ifModifiedSince -> contentType -> minify -> join -> less -> rewriteStatic -> file)
3. In buildResources(), the resources are created and added to the node http server to be served by the development server.
   (Handlers used in this step: ifModifiedSince -> contentType -> file)
4. In buildTests(), the tests are prepared and built.
   (Handlers used in this step: contentType -> rewriteFile -> wrapTest -> file)
5. In buildScripts() method, the javascript files for the project are first minified individually using uglify.js. Then for each of them, rewriteSuper is called to replace calls to sc_super() with 'arguments.callee.base.apply(this,arguments)' and rewriteStatic is called to replace sc_static with the appropriate strings. Once each file has been processed separately, they are joined into one. Then the handlebars templates gets compiled (if you use that) and the file is saved.
   (Handlers used in this step: ifModifiedSince -> contentType -> minify -> rewriteSuper -> rewriteStatic -> join -> handlebars -> file)
6. Once the buildScripts method is done, the callback method that you passed when you called app.build() is going to get triggered. In there, in general you will either have a call to app.save() if you were building the project (in build_config.js), or a call to server.run if you were starting the development server (in server_config.js).

At this point, garcon is ready to move on to the final step: which can be starting the server or saving the files to disk in the build/ folder.
Saving the file to the build/ folder

In that case, garcon will iterate over all the files saved in server.files object during the build step and create two arrays with them, depending on if the file is a script or a stylesheet. If the file is a resource, the file will be created and written to disk. Then, one stylesheet file will be created from all the files in the stylesheets array and it will be saved to disk. Same thing happens to the files from the scripts array. Finally, the HTML file is created and saved to disk.

In summary, all the files who were created and prepared during the build step are written to disk in the app.save() method.

Starting the development server

In the server.run() method, a node.js HTTP server is created to serve the individual files. Notice that in development mode, garcon does not minify and combine the CSS and script files (based on our config) so a handler is created for each of these files on the node server. Then a proxy is created in node.js. When a request is made to the HTTP server, it will first see if there is a file handler for it. If it finds it, it will serve the file, if not, it will proxy the request to the proxy server you have defined (if you did).

Summary

We are done with our overview of the Sproutcore build process and the garcon build tool. We hope it was useful to you and that this write-up will help others who are getting started with garcon. We wish your Sproutcore application tons of success.