Overview

Working on complex javascript projects usually means working with lots of javascript files. When it comes time to deploy this to production it can be be very tedious and sometimes dangerous to do this manually. This solution compiles all the javascript files in a directory to a single minified file ideal for release. This code will also automatically recompile this file if any of the files change.

Code

/// 

/// This class will compile (using Google's Closure Compiler) a directory of javascript files if:
///     - In Release Mode
///     - The release (minified) file is older than other files in the directory (stale)
///     - Not on localhost (Google cannot access your files)
/// 
public class JavascriptRuntimeCompiler {
    private static readonly ILog log = LogUtil.Logger(typeof (JavascriptRuntimeCompiler));

    private const string CACHE_MARKER = "CACHE_MARKER";
    private readonly string releaseFileName;        
    private readonly string baseScriptsUri;
    private readonly HttpContextBase context;

    private FileInfo releaseFile;

    /// 

    /// Creates the JavascriptRuntimeCompiler.  It is safe to create this per request as it is very efficient and
    /// will only create the new minified file if required.
    /// 
    /// The relative file name of the release minified file.  Eg: 'mydir\scripts\myscript.min.js'
    /// The relative uri of the directory holding the javascript files. Eg: '~/mydir/scripts/'.  It is safe
    /// for this to be the same directory as the one holding the release file.
    /// 
    /// The HttpContextBase object (Asp.Net Mvc)
    public JavascriptRuntimeCompiler(string releaseFileName, string baseScriptsUri, HttpContextBase context)
    {
        this.releaseFileName = context.Request.PhysicalApplicationPath + releaseFileName;            
        this.baseScriptsUri = context.Request.Url.GetLeftPart(UriPartial.Authority) + baseScriptsUri;
        this.context = context;            
    }
	
	/// 

	/// Wether to use the release script or not.  If true ensure your page points to your release file.  If false
    /// your page should reference all of the debug scripts.
	/// 
    public bool UseReleaseScript()
    {
        bool debug = false;
         #if (DEBUG)
        debug = true;
        #endif

        if (debug || baseScriptsUri.IndexOf("localhost") >= 0) { return false; }

        CheckReleaseScriptValidity();
        return true;
    }

    private void CheckReleaseScriptValidity()
    {
        releaseFile = new FileInfo(releaseFileName);            
        
        if (releaseFile.Exists && !IsReleaseFileOutOfDate()) { return; } 
        RebuildReleaseScriptFile();

        // This marker allows us to do a date check on all the files that the minified release file depends on
        context.Cache.Add(CACHE_MARKER, new Object(), new CacheDependency(GetJSFiles()), Cache.NoAbsoluteExpiration, Cache.NoSlidingExpiration, CacheItemPriority.Normal, null);       
    }

    private bool IsReleaseFileOutOfDate()
    {
        if (context.Cache[CACHE_MARKER] != null) { return false; }
        DateTime releaseFileDate = releaseFile.LastWriteTime;
        foreach (string s in GetJSFiles()) {
            if (File.GetLastWriteTime(s) > releaseFileDate) { return true; }
        }
        return false;
    }

    private void RebuildReleaseScriptFile()
    {
        string uri = "http://closure-compiler.appspot.com/compile?compilation_level=SIMPLE_OPTIMIZATIONS&output_format=text&output_info=compiled_code";
        foreach (string f in GetJSFiles()) {
            uri += "&code_url=" + baseScriptsUri + f.Substring(f.LastIndexOf("\\") + 1);           
        }
        log.Debug("Requesting Compiler: " + uri);
        WebRequest r = WebRequest.Create(uri);
        r.Method = "POST";
        r.ContentLength = 0;
        using (Stream s = r.GetResponse().GetResponseStream()) {                     
            using (StreamReader sr = new StreamReader(s, Encoding.UTF8)) {
                string content = sr.ReadToEnd();
                log.Info(content);
                FileUtils.WriteFileContents(releaseFileName, Encoding.UTF8.GetBytes(content));            
            }                
        }                             
    }

    private static string[] cached_js_files;
    private string[] GetJSFiles()
    {
        if (cached_js_files != null) return cached_js_files;
        List jsFiles = new List();
        if (releaseFile.Directory == null) throw new ApplicationException();

        foreach (FileInfo f in releaseFile.Directory.GetFiles()) {
            if (f.Name.Equals(releaseFile.Name) || f.Extension != ".js") { continue; }
            jsFiles.Add(f.FullName);                
        }
        return cached_js_files = jsFiles.ToArray();
    }
}

Using this Class

This example uses Asp.Net Mvc and the Spark view engine. However it should be trivial to change the above file or this example to use any other framework.

<if condition="new JavascriptRuntimeCompiler('resources\\scripts\\custom\\scripts.min.js', Url.Content('~/resources/scripts/custom/'), Context).UseReleaseScript()">
    <script language="javascript" src="${ Url.Content('~/resources/scripts/custom/scripts.min.js') }"></script>            
</if>
<else>
    <script language="javascript" src="${ Url.Content('~/resources/scripts/custom/Util.js') }"></script>    
    <script language="javascript" src="${ Url.Content('~/resources/scripts/custom/Class1.js') }"></script>    
    <script language="javascript" src="${ Url.Content('~/resources/scripts/custom/Class2.js') }"></script>    
    <script language="javascript" src="${ Url.Content('~/resources/scripts/custom/Class3.js') }"></script>                
</else>

Thanks

Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd

I needed to do this recently so I searched the interweb and quickly found this great article by Brent Edwards:

http://blog.edwardsdigital.com/post/Rendering-a-Spark-partial-view-to-a-string-or-JSONP-with-ASPNET-MVC.aspx

Now don't get me wrong, this works and is explained very well but I don't see the point in re-creating the ViewEngine? So I cleaned this up and ended up with:

 

Done, now for anyone that has tried to do this with ASP.Net MVC View engine, I pity you.

Thanks

Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd

One of my pet hates is working on projects where previous developers have decided to dump all of their settings in the web.config file.  Some of the reasons why I dislike this practice is:

• It looks horrible
• You cannot modify settings at run time (it restarts the application)
• You are mixing application deployment config info with your own configuration
• Did I mention that it looks and smells horrible?
• It is so simple to do it nicely

So now lets look at the nice way of doing this.

XML File

A settings file should look like this:

Not like this (Did I mention ugly?):

So lets go ahead and assume that our settings file will look like the first one above.

Accessing Application Settings

Accessing settings like this:

Is a million times nicer that accessing the like this:

 

Basic Implementation

So now that I've made it easy to see how ugly the alternative is lets look at a simple implementation for this design.

Allow Changes at Runtime

The above design is very tidy but it has a flaw that it does not allow changes to the settings file at run time.  To accomplish this we need

a file aware cache.  Lukily the ASP.Net cache is ideal for this.  So lets expand the above to take advantage of this.

 

 

Securing the Settings File

We may have sensitive data in this settings file such as passwords, etc.  This kind of information should obviously not be delivered to the web.  So how can we secure this.  Well, there are 3 main ways.

 

IIS

IIS can be set not to deliver xml files.  Or if you want just change the file name to settings.appsettings and then block the .appsettings filetype.

 

Web.config

This option is probably preferred to the IIS option as it will allow you to deploy to any server without having to remember to change IIS settings.  

To do this simply add the following handler to your web.config:

Note: You will need to ensure that .xml files are being handled by the aspnet IIS handler.

 

 

Global.asax.cs

Finally and by far the best (in my opinion) is to hide the file by Filtering in Global.asax.cs

Note: You will need to ensure that .xml files are being handled by the aspnet IIS handler.

 

That's it, and again, there is no good reason to have your settings in that decaying Web.config file.

 

Enjoy

 

Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd

This is something I have been wanting to do for a while in the ASP.Net world however its not untill recently (with MVC) that this has been a 'clean' possibility. This article demonstrates how to do this.

Inspiration

At PicNet are always trying to deliver quality products at the cheapest possible cost to the user.  The way we do is is by having 'template' driven projects. This allows us to generate data access layers very quickly (see previous articles) and we also copy template web projects, windows projects, mobile projects that give us a good starting point.  We also levarage custom libraries heavily.  However we could never put any of the view code (aspx, ascx) in these libraries as it was just not clean.

Embedded Views

To store your shared views in a library project simply copy the view code (aspx, ascx, master) into your library project.  Please ensure that the view code compiles.  By this I mean that all references are still valid (without requiring Web.config namespaces).  Tools like Resharper will show wether the view is valid or not.  Then mark the view as an 'Embedded Resource'.  To do this just right click on the file name -> Properties and set the Buidl Action to 'Embedded Resource'.  You will then need to rebuild the project to embed the resource in the DLL.

 

The View Engine

Add the following View Engine to your library project.

 

This view engine simply gets the views that have been stored as embedded resources and dumps them out to the ~/tmp/Views directory.

 

Register the View Engine

In Global.asax.cs just add:

That's It

That's it all the embedded views now simply behave like shared views.

 

 

Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd

Overview

Well, we're getting to the business end of these series of articles. In this article we will complete our PicNet.CodeGeneratedDALDemo.Data implementation, which means we will generate all of the required POCO files. We will also create a few tests to test the whole thing out.

Generating the NHibernate POCO Objects

To generate the POCOs download this articles source code. You can then just run nant dal as we described in Article 1.

The Generated POCOs

Lets have a look at the objects I create.

FIELDS

You will notifce a FIELDS enumeration at the top. I use this instead of passing magic strings around the place and simply serves the purpose of trying to remove the use of these magic strings and give a little bit of compile time checking for field names.

Constructors

I generate 3 constructors. These are: - The default constructor (required for serialisation, etc). - Required fields constructor (All fields that do not allow nulls) - All fields constructor

GetPropertyValueImpl and SetPropertyValueImpl

These methods are just implementations of IGetSetPropertyValue which provide a lightning fast alternative to reflection.

Properties

All of the object properties will follow. These save their dirty state and have a few NHibernate hacks that make life a little easier.

Misc Members

Finally we have the miscellaneous members ToString, GetHashCode, Equals, Clone, etc.

Compiling the Project

Once the POCOs have been generated you have to ensure that they are all included in the project. You also have to ensure that the hbm.xml files are marked as 'Embedded Resource'. Once you do this the entire project should compile.

Creating Tests

I have included only one very simple test class in the testing project but more importantly I have included the class 'InMemeoryDataTests' . This class allows you to create in memory NHibernate tests using SQLite.

Conclusion

This article wraps up this series of articles on using code generated data access layers. I hope that if you have taken the time to work through these things you will now appreciate the ease and speed of development that this approach gives. This approach is not for all projects but in my experience suits a vast majority of them. Some things that you may consider when working with this code.

• GetHashCode is not ideal as it uses the ID of an object for hashcode and this causes a plethora of well known issues when working with NHibernate.
• Nullables are not supported (have never gotten around to adding it in)
• This is a highly canabilised version of this code and is missing a lot of important features such as encryption, meta data helpers, serialisation helpers, etc.

Enjoy

Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd

PicNet and PicNet.Data Projects

Hi All,

In this article we will look at some of the libraries/classes required by the generated DAL.  These libraries simply have base classes and utility classes that make life a little easier.

PicNet.Data.IEntity

IEntity is the core interface of the data layer.  This interface defines all common behaviour for all entities generated by the code gen.  This helps with polymorphism and generally helps keep things nice and neat.

PicNet.Data.AbstractEntity

AbstractEntity is just a default implementation of many of the methods/properties defined in the IEntity interface.  All generated entities will extend this class to get some default behaviour.  Abstract entity also contains some utiltity methods that help in getting some meta information about the entities.

PicNet.IGetSetPropertyValue

You will notice that IEntity extends IGetSetPropertyValue.  This interface defines a way of setting and getting property values.  One of the beauties of code generated code is that you do not have to rely on reflection for dynamic property access.  Simply have your code gen generate a SetPropertyValue method and you have lightning fast dynamic property access. i.e (This is taken from the generated DVD.cs class which we will see in the next Article).

 

PicNet.Data.DataFacade

The DataFacade is the brains of the whole operation.  I'm not sure if DataFacade is an actual Facade, I mean, is it an implementation of the Facade pattern?  Who knows but I like the name because its the entry point to the data layer.  You can think of the DataFacade as a generic Repository which I believe is much neater then the repository pattern which would in this example lead to 3 classes (+ all the interfaces and test implementations) to access the data layer, these are:

- DVDRepository

- StorageUnitRepository

- StorageUnitTypeRepository.

Image a complex data layer with hundreds of entities.  Horrible.

Utility Classes

The PicNet project has a lot of utility classes that support the PicNet.Data classes and the generated objects.  Some of these classes include CollecitonUtils, LogUtil, UReflection, Utils, XMLUtils, etc.  Please read through these classes if you like they are very self explanitory.

Note

It is important to note that these projects have been around for years and for the purpose of this article I have just gone through and canabilised them, I removed:

- All code I did not want to share (encryption, etc)

- All code that was perhaps not directly required buy the basic DAL functionality I want to show you

However you may still find archeological evidence of these other classes which you should just ignore, again I am doing this to show you how you can build your own code gen DAL not really to provide you lots of great quality code.

Downloading the Solution

You can download the entire code for this article here.

Next Article

Generating the hibernate POCO objects.

 

Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd

Email: guido.tapia@picnet.com.au

Hi All,

This is the first in a series of posts about building a code generated data access layer.  Over the next few weeks I will be publishing articles that will allow you to understand why the approach taken here at PicNet produces such high quality systems.  Basically this approach is to use a code generated data access layer.

Code Generated DAL

Let me begin by defining what I mean by ‘code generated DAL’.  Basically I want to have an XML file like:

And then I can just fire up my code generator and create all the classes, configuration files, database schema, etc required to support this data layer.

Disadvantages
Code generation removes a lot of the power of the tools you are using.  Most modern ORM tools allow you to map any data schema to your domain model.  Working with a nice domain model has well documented advantages so I will not go into them here.  Generating your data layer will usually result in a one-to-one map of your data schema (hence not an nice abstract domain model).  This is probably the biggest drawback to this approach, however I hope that you will find that this is very quickly overcome by the huge gains in efficiency.

Advantages
- Fast, fast, fast.  I can build a complex data layer in a couple of days (fully tested).  This allows PicNet to provide customers with extremely cost effective solutions that are built on rock solid design principles and technologies.
- Code generated DAL lends itself to later more comlpex code generation, i.e. The user interface, user input validation, etc.
- Once you are comfortable with your own code generator you can pretty much do anything with it.
- A one-to-one map between database schema and domain is actually quite a positive thing.  The last thing you want to do is abstract the database too much.  You need to know when a join is being sent to the database, trying to hide this information can have very severe performance implications.
- Built on ORM technologies so all their benefits are inherited (caching, performance, security, etc).
- Many more which hopefully will be shown over the course of these articles.

The Schema
The schema is basically the XML file that defines the schema of the database and DAL.  This XML file will then be read by the code generator to create the required files.  A few things I want to point out about this schema:
- It should be as simple as possible relying on default values wherever possible.
- It should clear (not verbose)
- It should be natural

Let's go into a sample application.  I will use this application as the basis of my demo for the rest of these articles.  Its going to be a super basic app that will allow me to catalogue my DVD collection.  I will use Nant as my build tool, XSLT as my code gen engine and NHibernate as my ORM.  However you can sue whatever you want.

So, lets see whats going on.  At the top of the schema we define namespaces, assembly name and output directory, pretty straight forward.  Entities are then defined with the following format:

First thing we notice is the entity/@name attribute.  This will map to the object, class and table name.

Let me interrupt myself here.  I think its important to note that I'm not doing this to provide an open source DAL library.  I'm trying to explain and teach how and why you would do this yourself.  This means that if for example you want to have your object be called something totally different to your table then change this schema.  I.e.:

Ok, back to the post.

We then notice the fields.  Some things I want to point out is:
- type="int" is the default and can be left out
- type="string" or "byte[]" must have a length attribute
- null="true" - allows null values
- <name>true</name> simply means that a generic string GetName() method will return the value of this field.

Apart from that I think its all pretty self explanitory.

The Project Layout
I will lay my project out in this fashion:
CodeGeneratedDALDemo\
 lib\
 PicNet\
 PicNet.Data\
 PicNet.CodeGeneratedDALDemo.Data\
 PicNet.CodeGeneratedDALDemo.Data\demo-dal.xml
 PicNet.CodeGeneratedDALDemo.Data\Generated\
 PicNet.CodeGeneratedDALDemo.Data.Test\
 nant.bat
 demo.build

Description of Project Layout
lib: directory houses all my... libs.
PicNet and PicNet.Data: Simply some reusable custom written libraries I use.  I will be showing you the necessary code as we move along (next article actually).
PicNet.CodeGeneratedDALDemo.Data: Houses the PicNet.CodeGeneratedDALDemo.Data project which is mostly code generated.
PicNet.CodeGeneratedDALDemo.Data\demo-dal.xml: The schema file.
PicNet.CodeGeneratedDALDemo.Data\Generated\: The drirectory where to put the generated files.
PicNet.CodeGeneratedDALDemo.Data.Test: The Unit tests directory of the PicNet.CodeGeneratedDALDemo.Data namespace.
nant.bat, demo.build: The NAnt build files

The Code Generator
I will be using XSLT driven by NAnt to generate my code, you can use other tools such as NVelocity, MyGeneration or CodeSmith or whatever you choose.  I personally like XSLT and I think its a technology that every senior developer should be comfortable using.  Note: If you are in the java world just drop the prefix 'N's from all the technologies mentioned and you will find the same tools in the java world, in fact they all came from the Java world.  I will be using NHibernate as my ORM.

NAnt Build File
What I like to do with NAnt is have a batch file to execute my builds, this allows me to have my NAnt executables somewhere tidy and not in my path and still accessible in a convenient location in my project.  So lets add Nant.bat in the root directory of the project.  In this batch file just put:

Please change to the location of your nant executables (if different to this).

Lets create the demo.build file.

Generating the Hibernate config files
Ok, Enough with the theory lets build our hibernate config file XSLT code generator.  In the nant build file above we see that this file is called: dal-to-hibernate.xsl so lets see what it looks like.

This XSLT needs common.xsl for some common funcitonality.

You can download the code samples of this entire article here:

I suggest you download the zip above (here) and lets have a go at generating the NHibernate config files.
- Unzip into your dev directory
- Open a windows explorer and go to: CodeGeneratedDALDemo\PicNet.CodeGeneratedDALDemo.Data\Generated (It should be empty)
- Open a command prompt
- Go to the CodeGeneratedDALDemo directory
- Run: nant dal
- The CodeGeneratedDALDemo\PicNet.CodeGeneratedDALDemo.Data\Generated should now have the hbm.xml files.

These are the files that were just generated.  You can review these files and change the code generator (xslt files) as you see fit.  You may want to generate entities with a better identity generator for instance.  Or change caching hints, etc.  I also encourage you to review the 2 xsl files I have included.  These files have been put together over a few years and have had pplenty of hacky work done on them so please feel free to keep and throw away what ever you need.  If you want to play around with these files let me give you a few suggestions:
- I think it would be great if the 'pk' field was optional.  I.e. If it is not specified then simply generate it (EntityName + 'ID').
- Relationships should have the name attribute optional.  Just call it <RelatedEntityType>ID if not specified.
- Use a better, ORM friendly identity generator. Note: I like native because in SQL (which is the DB I use most) the identity column gives me a lot of nice information, like what order were rows added into the table.  Or how many inserted records have been in a table, etc.  So I am willing to put up with the draw backs of this identity generator (hashcode issues, db performance, etc).

Next Article
That will do for now.  In the next article we will look at some portions of the PicNet and the PicNet.Data projects. 

Guido Tapia

Manager - Custom Software Development

PicNet Pty Ltd