The Template System -- Design Document

by Christian Brechbühler Templating System : Design

I. Essentials

II. Introduction

III. Historical Considerations

Karl Goldstein designed the templating system. First it was called "Karl's Templates" or "The New Templating System" to distinguish it from the obsolescent templates or "Styles" by Philip Greenspun. An extended and improved version was named "Dynamic Publishing System". It wasn't part of the ACS yet, but client projects like iluvCAMP used it successfully. Newcomers were consistently puzzled by the .data files, which specified the datasources in an apparently unfamiliar XML syntax. (The .form files specified elements in an HTML form similarly.) To mitigate this initial shock, Karl redesigned templates to let the programmer specify datasources and forms in a .tcl script. The system is present as packages templates and form-manager in ACS 3.4. Both these packages are now merged and appear as acs-templating starting in ACS 4.0. The architecture of the package was changed several times to meet the emerging coding/style constraints of ACS 4.0.

V. Design Tradeoffs

As indicated above, the primary attribute that the page tries to achieve is the separation of code and layout. The primary sacrifice is simplicity; in the typical case there will be two files (a .adp templage and a .tcl script) instead of a single .tcl page.

Management of data sources. Through the various past versions of the package, it evolved that data sources should be set as Tcl variables and arrays. Earlier they were kept as lists of ns_sets, which was significantly less efficient. The datasources are not being copied around; they reside in a specific stack frame. Using the uplevel Tcl command, we make sure that the data file (tcl part of a page) executes in the same stack frame as the compiled template (the adp part), so the latter can make use of the data sources prepared by the former.
        Thus, we decided for performance, simplicity, and ease of use at the cost of using the (standard Tcl) commands upvar and uplevel, which is considered confusing and error-prone by reviewers (of 4.0). The use of these constructs has been reduced in 4.01, and the code is clearer now.

Other attributes are affected as follows. In parentheses the estimated priorities are listed, not the degree to which the attributes are being achieved:

VI. API

Details are in the developer guide. Here we give an overview, and then the more obscure aspects of the current implementation.

The most important abstraction is the data source, of which there are several kinds:

Currently ad_page_contract does not allow specifying the latter two.

Process Flow

In a simple case, the following is the sequence of steps that serving a templated page involves.
  1. The request processor gets a url and maps it to a .adp or .tcl file. As both invoke the same handler, it doesn't matter that adp take precendence.
  2. If a .tcl file is present, its ad_page_contract in the -properties block indicates a set of data sources that will be made available to the template.
  3. The rest of the tcl script executes, defining these data sources. It may change the name of the page being served by calling template::set_file directly or through the wrapper ad_return_template.
  4. The corresponding template (file stub.adp) is interpreted and the data sources from the .tcl script are interpolated into the template.
  5. The HTML is streamed to the client by the handler.
Less simple cases involve dependent templated pages. They are requested with the <include> and <master> tags. In these cases, TCL and/or ADP parsing happens recursively.

Tcl Call Stack

Below is a diagram of the typical call stack when processing a page without dependent pages. To conform to the Tcl notion of what's up and down (as in upvar), the stack grows down.

Level Procedure Arguments
#1rp_handler
#2rp_serve_abstract_file /web/service/www/page
#3rp_serve_concrete_file /web/service/www/page.adp
#4adp_parse_ad_conn_file
#5template::adp_parse /web/service/www/page {}
(6) template::adp_prepare
#5 template::code::tcl::/web/service/www/page

Levels #1 to #3 exposed here are request processor internals. In the case shown, datasources reside in level #5. Due to the uplevel command, the frame of the sixth procedure is not accessible in the call stack at this moment, and the seventh runs in stack frame #5. If the <include> or <master> tags are used, adp_parse will be invoked recursively. Datasources always reside in the stack frame of an instance of adp_parse.

To keep track of data sources of several page components, the templating system maintains a stack of their stack levels in the variable template::parse_level. In our case, it just contains 5. But if this page included another page or designated is as its master, the level of the next adp_parse would be pushed to the list, and popped when that proc returned. This next level will appear as #6, due to the repeated upleveling.

Caching and Template Compilation

To improve performance, adp pages are compiled into a tcl proc, and thus cached for future use. Tcl pages are also cached in a proc; this saves the trouble of reading and parsing the file the next time. The template system remembers the modification times of the adp and tcl sources, and re-processes any requested file if the cached version is no longer current. Consequently, this cacheing is transparent in normal use.

To emphasize that "normal" use essentially always applies, here's a scenario for abnormal use: Save version n of a file at 11:36:05.1; request a page that uses it at 11:36:05.3; modify and save version n+1 of the file at 11:36:05.9. If you work that fast (!), the new version will have the same modification time -- kept with 1 second resolution in Unix --, and will not be refreshed.

For timing measurements and performance tuning, you can set the parameter RefreshCache in section template to never or always. The former suppresses checking mtime and may improve performance on a production server, where the content pages don't change. The latter is only inteded for testing.

VII. Data Model Discussion

This packages doesn't need a data model.

It comes with its own database interfaces, one for using ns_ora, the Oracle driver from ArsDigita, and one for ns_db, the builtin database interface of the AOL server. If you are programming under the ACS, you should use neither of these, but rather the db_* interface, in particular db_multirow.

VIII. User Interface

This packages doesn't have a user interface. It is the substrate of all user interfaces, be it user or admin pages.

IX. Configuration/Parameters

There are two parameters.
      [ns/server/yourservername/acs/template]
      ; the site-wide Master template
      DefaultMaster=/www/default-master
      ; anything other than "never" or "always" means normal operation
      RefreshCache=as necessary
    

X. Future Improvements/Areas of Likely Change

Passing datasources by reference is new. The acs-templating syntax &formal="actual" is different from the independent ATS, which used formal="@actual.*@". The latter is phased out.

We intend to add a <which>, <switch>, or <case> tag, to complement sequential nested <if>/<else> constructs.

Authors

XII. Revision History

Document Revision # Action Taken, Notes When? By Whom?
0.1 Brought into the form suggested by Finkler, McLoghlin and Wu 18 Jul 2000 Christian Brechbühler
0.2 Adapted to acs-templating as distributed with ACS/Tcl 4.01 22 Nov 2000 Christian Brechbühler


Christian Brechbuehler
Last modified: $Id: design.html,v 1.1 2001/03/13 22:59:27 ben Exp $