Modularity

Background

One of the questions that came to the dev group earlier was about using Spark in a web host that had a modular design.

I have been frequently stunned at the power and flexibility of the extensibility model in some products like Drupal and Wordpress. I recently added Forums to the Drupal CMS the Spark support site runs on and it was remarkable how much alien functionality could be added by dropping in packages.

Several ingredients should be present for a modular site architecture to succeed. The first sample was fairly trivial and dealt only with rendering embedded Spark files. A huge chunk of the problem was off the table because it didn't deal with the view engine.

This is a follow-up that speaks to the original question in greater detail: how would you create a modularity framework that allows you to extend a web site.

Key ingredients

The problem domain can be pretty well defined by a looking at CMS platform's architecture.

  • Url routing of incoming requests can be done by modules
  • Controllers are exposed for page actions as well as json, ajax, or dynamic graphics
  • View templates from modules render in combination with with partial and layout templates of the site
  • Management of additional static content files
  • Blocks of active content provided by a module placed on arbitrary pages
  • Composition of the inter-module service capabilities and dependencies
  • Packaging of these concerns into atomic deliverables

This is article is not about sample CMS - it does not deal with an information model and storage system suitable for content management. If you were designing a CMS however I believe this would be a good starting point for these concerns.

Last thing first - Packaging and Composition

In the Modules.sln sample there are two core projects which provide the "top" and the "bottom" of the stack. The Modular.WebHost is at the top; it provides all of the artifacts of a web application and will be the start-up project for debugging. Them Spark.Modules class library is at the bottom and all of the core interfaces and service classes for things like packages and rendering blocks.

A key interface is the IWebPackage. It's very complex. :)

public interface IWebPackage
{
    void Register(
        IKernel container,
        ICollection<RouteBase> routes, 
        ICollection<IViewEngine> viewEngines);
}

Now let's jump to the other side of the universe and take a look at the web host's Application_Start

protected void Application_Start(object sender, EventArgs e)
{
    var container = new WindsorContainer(Server.MapPath("~/config/windsor.config"));
 
    var app = new Application();
    app.RegisterFacilities(container);
    app.RegisterComponents(container);
    app.RegisterViewEngine(ViewEngines.Engines);
    app.RegisterPackages(container, RouteTable.Routes, ViewEngines.Engines);
    app.RegisterRoutes(RouteTable.Routes);
 
    ControllerBuilder.Current.SetControllerFactory(container.GetService<IControllerFactory>());
}

A lot of the work is delegated to a simple class called Application. It performs the general purpose tasks like initializing the IoC container and adding the universal {controller}/{action} route. It also calls on the component which will load all of the packages it can discover.

public void RegisterPackages(IWindsorContainer container, ICollection<RouteBase> routes, ICollection<IViewEngine> engines)
{
    var manager = container.Resolve<IWebPackageManager>();
    manager.LocatePackages();
    manager.RegisterPackages(routes, engines);
    container.Release(manager);
}

You'll notice there's a heavy reliance on the Windsor and MicroKernel containers from Castle Project. They provide support for composition and inter-module service utilization.

So let's create a new package! There's already a Game metaphor that's extensible - so let's make a package that adds a new type of game. You would start by creating a class library and making creating something based off of IWebPackage or WebPackageBase. I'm going to make some sort of simple star fleet type of game and I'll rely on the conventional use of folders and embedded resources so I'll use the standard implementation for registering a standard area.

using System.Collections.Generic;
using System.Web.Mvc;
using System.Web.Routing;
using Castle.MicroKernel;
using Spark.Modules;
 
namespace Modular.Fleet.WebPackage
{
    public class FleetWebPackage : WebPackageBase
    {
        public override void Register(
            IKernel container, 
            ICollection<RouteBase> routes, 
            ICollection<IViewEngine> viewEngines)
        {
            RegisterStandardArea(container, routes, viewEngines, "Fleet");
        }
    }
}

There will also be a ton of references to add: Castle.MicroKernel, Castle.Core, System.Web.Mvc, System.Web.Routing, and a project reference to Spark.Modules if you're in the sample Modules solution.

This is going to do three things we care about for right now. First it will add all of the of the classes that inherit from IController, IService, and IBlock to the MicroKernel. Secondly it will add a route for "{area}/{controller}/{action}" with the constraint that {area} is "Fleet". Finally it will create an overlay of the .spark embedded resources in the class library to add to the Views directory.

Url routes and controller actions

So let's jump right in to adding a controller to this new module. Simplest one in the world:

using System.Web.Mvc;
 
namespace Modular.Fleet.WebPackage.Controllers
{
    public class HomeController : Controller
    {
        public ActionResult Index()
        {
            return View();
        }
    }
}

So let's build-all, run the web host, and see what happens: Nothing! http://localhost:59497/Fleet/home/index is 404 because the web package isn't detected.

Just to make sure it's really acting as it's supposed to let's try copying Modular.Fleet.WebPackage.dll into \Modular.WebHost\bin and hitting refresh on the browser. And... Victory!

Exception Details: System.InvalidOperationException: The view 'index' or its 
    master could not be found. The following locations were searched:
~/Views/Fleet/home/index.aspx
~/Views/Fleet/home/index.ascx
~/Views/Shared/index.aspx
~/Views/Shared/index.ascx
Fleet/home\index.spark
Shared\index.spark

Let's verify it's really {area}==Fleet route in effect here working and the HomeController from that project is being invoked. Add a project reference from the WebHost to the Fleet.WebPackage project (for development copying a dll every build is too inconvenient), set a breakpoint in the Index, and run the debugger and point a browser to /Fleet/home/index (or simple /Fleet for that matter).

The breakpoint in the fleet HomeController does in fact get hit, and if you inspect the contents of the RouteData member you'll see the following:

RouteData.Values

Notice that the name of the {controller} has been changed! It's "Fleet/home" instead of simply "home". That's very important because it's the name of the controller that's making the view engines look into a Fleet sub-directory when they're finding the view.

View templates on disk and embedded

When the Package registered itself it injected the assembly's embedded files into a subdirectory of the site's Views folder. They won't appear on disk, but because the Spark view engine doesn't access the file system directly these virtual files work in exactly the same way a normal file would.

In the Fleet.WebPackage assembly let's add a Views folder with a Home folder with a Index.spark file. Be sure to change the Build Action to Embedded Resource.

<h2>Fleet</h2>
<p>A game created for the purpose of writing a tutorial.</p>
<set Title="'Fleet Game - ' + Title"/>

Because it's an embedded resource you'll need to rebuild before you can refresh the browser to see changes. It can be a pain. So ctrl+shift+B to build and point to /Fleet and you'll see you've just extended a web site by adding nothing more than a single self-contained dll.

Package dependencies and site composition

Although it's interesting - so far it isn't very practical. A site needs to compose itself into a meaningful interconnected system otherwise it's just a bunch of unrelated pages you can't reach without knowing the url in advance.

The capabilities of an IoC container like Castle Project's MicroKernel or Windsor are invaluable at this point. When they create the IWebPackage and IController classes they will also create and interconnect any number of necessary components out to any depth.

To take advantage of this you'll need an assembly reference for the interface of a needed service. Then simply add a constructor parameter which accepts that interface. That will make the service mandatory. Otherwise when there's a service which is optional it can be added as a public assignable property. If a component for that service has not been registered by any modules you'll have a default (null) values on the property.

The game module uses an IGameRegistry interface declared in the Modular.Common assembly. The name of that sample project should really be something like Modular.Game.Services but let's ignore that for now.

Let's add a reference to the assembly, take the interface as a constructor argument on the package, and add an entry to the registry when the package is installing itself.

public class FleetWebPackage : WebPackageBase
{
    private readonly IGameRegistry _gameRegistry;
 
    public FleetWebPackage(IGameRegistry gameRegistry)
    {
        _gameRegistry = gameRegistry;
    }
 
    public override void Register(
        IKernel container, 
        ICollection<RouteBase> routes, 
        ICollection<IViewEngine> viewEngines)
    {
        RegisterStandardArea(container, routes, viewEngines, "Fleet");
 
        _gameRegistry.AddGame("Star Fleet", new { area = "Fleet", controller = "Home" });
    }
}

In this example the game registry assumes there will be an action named "Play" and additional values for the Html.ActionLink are provided as object. If you look at /Games now you'll see this has been added to the list and the a href will point to a Fleet HomeController.Play action we need to create.

Let's add that - and at the same time put in logging. This sample uses Castle Project's ILogger and is to configured to use System.Diagnostics.TraceSource to log to the debug window. That can be changed in the WebHost's Config/Diagnostics.config file (again, beyond the scope of this article).

public class HomeController : Controller
{
    private ILogger _logger = NullLogger.Instance;
    public ILogger Logger
    {
        get { return _logger; }
        set { _logger = value; }
    }
 
    public ActionResult Index()
    {
        return View();
    }
 
    public ActionResult Play()
    {
        Logger.Info("Game's Play invoked");
 
        return View("Index");
    }
}

This example was to show how services may be used at the time the site is starting up, as well as when they're needed for any controllers. The modular framework doesn't provide any rules about how they are designed or where stateful information should be persisted. If you want to create and expose a service from a package assembly and use an interface which is based on Spark.Modules.IService it will be registered automatically by the WebPackageBase method RegisterStandardArea.

It's easier than it sounds. Here's the GameRegistry that's automatically exposed by Modular.Common's web package.

public interface IGameRegistry : IService
{
    void AddGame(string name, object playLinkValues);
    IEnumerable<GameDescriptor> ListGames();
}
 
class GameRegistry : IGameRegistry
{
    private readonly IDictionary<string, GameDescriptor> _games = new Dictionary<string, GameDescriptor>();
 
    public void AddGame(string name, object playLinkValues)
    {
        _games.Add(name, new GameDescriptor { Name = name, PlayLinkValues = playLinkValues });
    }
 
    public IEnumerable<GameDescriptor> ListGames()
    {
        return _games.Values.ToArray();
    }
}

Static css, js, and images files

You can only do so much in html so the question of static assets is bound to come up eventually. You could deploy them as companion files to the package dll, or refer to them on a CDN, or really any number of things would work. Because the goal here is to provide an example of a single installable package let's embed them in the package to serve dynamically.

To do this the WebPackageBase route the pattern "Content/{area}/{*resource}" onto a handler which will binary-write out any embedded resource. It's a really trivial example that's provided in Spark.Modules, so you may want to look there if it's not meeting your needs. For example I think it's case-sensitive, so that'll need to change.

But it's simple and it works: so after adding a Content\images\ships.jpgembedded resource to the fleet package assembly we can add <img src="~/content/fleet/images/ships.jpg"/> to the Index.spark and see that it's there.

Again that's being written from the dll so no files on disk are needed - so that's great for literally drop-in package dll's - but in practice you may have different requirements about performance of static files or end-user hackable css so you may want to explore an alternative.

Rendering Blocks of html

The name Block comes from the Drupal CMS for a hunk of html content which can be placed at a location on a page. In the CMS the theme defines several named Regions where these can be placed. What you'll find in the Modules sample projects is an example of how a package can register blocks, which include a class based on IBlock, and how another page or layout can give them control to render at a particular location.

So let's make a block that's a teaser badge linking to the fleet controller. To use a view engine (rather than response-write to product html) the RenderPartial extension method works great.

using System.Web.Mvc;
using System.Web.Mvc.Html;
using Spark.Modules;
 
namespace Modular.Fleet.WebPackage.Blocks
{
    public class FleetTeaserBlock : IBlock
    {
        public HtmlHelper Html { get; set; }
 
        private static int _counter;
 
        public void RenderBlock()
        {
            // evil! just a demo! honest!
            _counter++;
 
            Html.RenderPartial(@"Fleet\Home\Teaser", new { Counter = _counter });
        }
    }
}

The HtmlHelper Html property is being assigned via IoC and any other services and data sources you need can be properties or constructor arguments.

Then create a Views\Home\Teaser.spark file in the fleet package assembly. Make sure it's marked as an embedded resource. The use of the Fleet area prefix is again to avoid collisions because if you wanted to you could render anything.

<viewdata Counter="int"/>
<div class="badge">
    <p>
      Fleet! Nagging people at least ${Counter} times!<br/>
        !{Html.ActionLink("Play now!", "Play", new {controller="home", area="fleet"})}
 </p>
</div>

And to display this block first let's add it hard-coded to the site. That's just to make sure it's working. In the WebHost/Views/Home/Index.spark file you would add something like this.

<use namespace="Spark.Modules.Html"/>
#Html.RenderBlock("fleetteaser");

The namespace only has to be present once for the RenderBlock extension method to be available. It's already in _global.spark so this was just for show. The RenderBlock method itself works a bit like RenderPartial. It returns void rather than a string - so you call it with <% stmt; %> or #stmt;\r\n instead of <%= expr %> or ${expr}.

There are a few reasons why you would use a Block instead of Html.RenderPartial or an underscore partial. The underscored partial works entirely at the time the templates are parsed and compiled. They're very lightweight and clean, but can't be invoked abstractly.

That's an advantage of RenderPartial and RenderBlock - the named of the view or block can be treated as data at runtime.

An example of doing that can be found in the SideBlock in the Modular.Navigation.WebPackage assembly. There is an ISideRegistry service which packages use to add entries into a list, and the SideBlock iterate that list and render all of those blocks in turn.

So let's add the teaser to the sidebar when the fleet package registers itself.

public class FleetWebPackage : WebPackageBase
{
    private readonly IGameRegistry _gameRegistry;
    private readonly ISideRegistry _sideRegistry;
 
    public FleetWebPackage(IGameRegistry gameRegistry, ISideRegistry sideRegistry)
    {
        _gameRegistry = gameRegistry;
        _sideRegistry = sideRegistry;
    }
 
    public override void Register(
        IKernel container,
        ICollection<RouteBase> routes,
        ICollection<IViewEngine> viewEngines)
    {
        RegisterStandardArea(container, routes, viewEngines, "Fleet");
 
        _gameRegistry.AddGame("Star Fleet", new { area = "Fleet", controller = "Home" });
 
        _sideRegistry.AddItem(new SideItem { BlockName = "FleetTeaser", Weight = 8 });
    }
}

Future work

There are clearly areas which need further work. The IBlock rendering should take the normal ViewData and object parameters. Also the Content material that's accumulated in the <content name="..."> when you RenderBlock or RenderPartial won't bubble up into the view or layout contexts. That makes it much less practical to use content areas to aggregate css and js including html.

Finally this is an example of a set of modular site design techniques rather than an implementation of a modular site framework. Of course if you do want to make a CMS that incorporates these techniques I think that's a fabulous idea.