Index: openacs-4/packages/acs-core-docs/www/apm-design.html =================================================================== RCS file: /usr/local/cvsroot/openacs-4/packages/acs-core-docs/www/apm-design.html,v diff -u -r1.34 -r1.34.2.1 --- openacs-4/packages/acs-core-docs/www/apm-design.html 7 Jun 2008 20:28:48 -0000 1.34 +++ openacs-4/packages/acs-core-docs/www/apm-design.html 10 Jun 2009 22:24:06 -0000 1.34.2.1 @@ -1,33 +1,33 @@ - -Package Manager Design

Package Manager Design

By Bryan Quinn

+ +Package Manager Design

Package Manager Design

By Bryan Quinn

OpenACS docs are written by the named authors, and may be edited by OpenACS documentation staff. -

Essentials

Introduction

-In general terms, a package is a unit of software that +

Essentials

Introduction

+In general terms, a package is a unit of software that serves a single well-defined purpose. That purpose may be to provide a service directly to one or more classes of end-user, (e.g., discussion forums and file storage for community members, user profiling tools for the site publisher), or it may be to act as a building block for other packages (e.g., an application programming interface (API) for storing and querying access control rules, or an API for scheduling email alerts). Thus, packages fall into one of two categories: -

  • OpenACS Applications: a "program or group of programs -designed for end users" (the Webopedia +

    An installation of the OpenACS includes the OpenACS Kernel, some services that extend the kernel's functionality, and some applications intended for -end-users. Packages function as individual pieces of subsites. A subsite can contain multiple +end-users. Packages function as individual pieces of subsites. A subsite can contain multiple application and service instances that provide the end-user with capabilities and content customized to the particular subsite.

    This architecture supports the growth of collaborative commerce. For example, Jane User starts a forum focusing on the merits of View Cameras by @@ -36,48 +36,48 @@ it in his subsite. As interest in Jane's forum grows, she creates a subsite specializing in providing information about View cameras. This subsite now includes several package instances beyond Forum; it could -potentially include its own Ecommerce capabilities (ala Yahoo! Shopping). This could include a +potentially include its own Ecommerce capabilities (ala Yahoo! Shopping). This could include a knowledge management application that allows users to spread expertise about view cameras and a portal application that links to reliable camera models and resellers. Any subsite enabled package that is added to the OpenACS installation through APM is another potential package instance that can become part of Jane's View Camera subsite.

    The APM provides an architecture for packaging software, making instances of that software available to subsites, specifying configuration parameters -for each instance, and managing the creation and release of new packages.

Historical Considerations

+for each instance, and managing the creation and release of new packages.

Historical Considerations

Prior to ACS 3.3, all packages were lumped together into one monolithic distribution without explicit boundaries; the only way to ascertain what comprised a given package was to look at the top of the corresponding documentation page, where, by convention, the package developer would specify where to find: -

  • the data model

  • the Tcl procedures

  • the user-accessible pages

  • the administration pages

Experience has shown us that this lack of explicit boundaries causes a -number of maintainability problems for pre-3.3 installations:

  1. Package interfaces were not guaranteed to be stable in any formal way, so +

    • the data model

    • the Tcl procedures

    • the user-accessible pages

    • the administration pages

    Experience has shown us that this lack of explicit boundaries causes a +number of maintainability problems for pre-3.3 installations:

    1. Package interfaces were not guaranteed to be stable in any formal way, so a change in the interface of one package would often break dependent packages (which we would only discover through manual regression testing). In this context, any of the following could constitute an interface change: -

      • renaming a file or directory that appears in a URL

      • changing what form variables are expected as input by a page

      • changing a procedural abstraction, e.g., a PL/SQL or Java stored -procedure or a Tcl procedure

      • changing a functional abstraction, e.g., a database view or a PL/SQL or -Java stored function

      • changing the data model

      This last point is especially important. In most cases, changing the data +

      • renaming a file or directory that appears in a URL

      • changing what form variables are expected as input by a page

      • changing a procedural abstraction, e.g., a PL/SQL or Java stored +procedure or a Tcl procedure

      • changing a functional abstraction, e.g., a database view or a PL/SQL or +Java stored function

      • changing the data model

      This last point is especially important. In most cases, changing the data model should not affect dependent packages. Rather, the package interface should provide a level of abstraction above the data model (as well as the rest of the package implementation). Then, users of the package can take advantage of implementation improvements that don't affect the interface (e.g., faster performance from intelligent denormalization of the data model), without having to worry that code outside the package will now -break.

    2. A typical ACS-backed site only uses a few of the modules included in the +break.

    3. A typical ACS-backed site only uses a few of the modules included in the distribution, yet there was no well-understood way to pick only what you needed when installing the ACS, or even to uninstall what you didn't need, post-installation. Unwanted code had to be removed manually. -

    4. Releasing a new version of the ACS was complicated, owing again to the +

    5. Releasing a new version of the ACS was complicated, owing again to the monolithic nature of the software. Since we released everything in the ACS together, all threads of ACS development had to converge on a single deadline, after which we would undertake a focused QA effort whose scale increased in direct proportion to the expansion of the ACS codebase. -

    6. There was no standard way for developers outside of ArsDigita to extend +

    7. There was no standard way for developers outside of ArsDigita to extend the ACS with their own packages. Along the same lines, ArsDigita programmers working on client projects had no standard way to keep custom development cleanly separated from ACS code. Consequently, upgrading an already installed @@ -88,43 +88,43 @@ packages for other OpenACS users to download and install.

      For a simple illustration of the difference between ACS without APM (pre-3.3) and ACS with APM (3.3 and beyond), consider a hypothetical ACS installation that uses only two of the thirty-odd modules available circa ACS -3.2 (say, bboard and e-commerce):

      APM itself is part of a package, the OpenACS Kernel, an OpenACS -service that is the only mandatory component of an OpenACS installation.

    Competitive Analysis

    The OpenACS is a platform for web-based application software, and any software +3.2 (say, bboard and e-commerce):

    APM itself is part of a package, the OpenACS Kernel, an OpenACS +service that is the only mandatory component of an OpenACS installation.

    Competitive Analysis

    The OpenACS is a platform for web-based application software, and any software platform has the potential to develop problems like those described above. Fortunately, there are many precedents for systematic solutions, -including:

    Borrowing from all of the above, OpenACS 3.3 introduces its own package -management system, the OpenACS Package Manager (APM), which consists of:

    • a standard format for APM packages (also called -"OpenACS packages"), including:

      • version numbering, independent of any other package and the OpenACS as a -whole

      • specification of the package interface

      • specification of dependencies on other packages (if any)

      • attribution (who wrote it) and ownership (who maintains it)

    • web-based tools for package management:

      • obtaining packages from a remote distribution point

      • installing packages, if and only if:

        1. all prerequisite packages are installed

        2. no conflicts will be created by the installation

      • configuring packages (obsoleting the monolithic OpenACS configuration -file)

      • upgrading packages, without clobbering local modifications

      • uninstalling unwanted packages

    • a registry of installed packages, database-backed and +including:

      Borrowing from all of the above, OpenACS 3.3 introduces its own package +management system, the OpenACS Package Manager (APM), which consists of:

      • a standard format for APM packages (also called +"OpenACS packages"), including:

        • version numbering, independent of any other package and the OpenACS as a +whole

        • specification of the package interface

        • specification of dependencies on other packages (if any)

        • attribution (who wrote it) and ownership (who maintains it)

      • web-based tools for package management:

        • obtaining packages from a remote distribution point

        • installing packages, if and only if:

          1. all prerequisite packages are installed

          2. no conflicts will be created by the installation

        • configuring packages (obsoleting the monolithic OpenACS configuration +file)

        • upgrading packages, without clobbering local modifications

        • uninstalling unwanted packages

      • a registry of installed packages, database-backed and integrated with filesystem-based version control -

      • web-based tools for package development:

        • creating new packages locally

        • releasing new versions of locally-created packages

    Design Tradeoffs

    +

  2. web-based tools for package development:

    • creating new packages locally

    • releasing new versions of locally-created packages

Design Tradeoffs

The design chosen for APM was meant to satisfy the following constraints: -

  • The process of authoring a package must be as simple as possible.

  • Strict conventions must be established that provide a set of canonical +

    • The process of authoring a package must be as simple as possible.

    • Strict conventions must be established that provide a set of canonical locations and names for files and patterns, for OpenACS application -development.

    • The processes of installing, upgrading, and using packages must be -straightforward and accessible through a web-based UI.

    • Package instances must be able to have subsite-specific content available +development.

    • The processes of installing, upgrading, and using packages must be +straightforward and accessible through a web-based UI.

    • Package instances must be able to have subsite-specific content available at an easily configurable URL.

    All of these requirements were met, but at the cost of development -simplicity. As Packages demonstrates, a set of strict directory conventions are +simplicity. As Packages demonstrates, a set of strict directory conventions are required in order for a package to use APM. This contrasts with the apparent simplicity available to developers of the OpenACS 3.3 system. However, while the system has become more complex for developers to build packages, this complexity is easily managed and is compensated for by additional capabilities.

    For example, to make a new application available to the system, a -developer must:

    1. Create the necessary files to support the data model, Tcl API, and UI -pages.

    2. Put the files in the correct locations for APM to be aware of them.

    3. Use APM to create a new package and enable it.

    4. Use the Site Map facility to create an instance of the package, mount it +developer must:

      1. Create the necessary files to support the data model, Tcl API, and UI +pages.

      2. Put the files in the correct locations for APM to be aware of them.

      3. Use APM to create a new package and enable it.

      4. Use the Site Map facility to create an instance of the package, mount it on an appropriate URL, and set parameters for that particular instance.

      While this is complex, especially to a new OpenACS developer, the documentation walks the developer through each of these steps. Moreover, from following these steps, the package can be subsite specific, available to subsites across the system, and be available for distribution to other OpenACS -installations without doing a monolithic upgrade or reinstall.

    API

    The APM is composed of systems for accomplishing a set of package-related +installations without doing a monolithic upgrade or reinstall.

    API

    The APM is composed of systems for accomplishing a set of package-related tasks. Each of these tasks comprise a feature area that has an API, data -model, and a UI:

    • Authoring a Package

    • Maintaining Multiple Versions of a Package

    • Creating Instances of the Package

    • Specifying Configuration Parameters for each Instance

    Authoring a Package

    Full instructions on how to prepare an OpenACS package are available in Packages. The API here can be invoked manually by a package's data model -creation script, but need not to be used. This API is part of the APM PL/SQL +model, and a UI:

    Authoring a Package

    Full instructions on how to prepare an OpenACS package are available in Packages. The API here can be invoked manually by a package's data model +creation script, but need not to be used. This API is part of the APM PL/SQL package.

     
     -- Informs the APM that this application is available for use.
    @@ -144,9 +144,9 @@
     

    The procedure above registers an OpenACS application in the APM. It creates a new OpenACS object and stores information about the package, such as its name, in the APM data model. There is an analogous procedure for OpenACS services, called -apm.register_service.

    To remove an application from the system, there are the calls -apm.unregister_application and -apm.unregister_service.

    +apm.register_service.

    To remove an application from the system, there are the calls +apm.unregister_application and +apm.unregister_service.

     
     -- Remove the application from the system.  
     procedure unregister_application (
    @@ -155,22 +155,22 @@
         cascade_p       in char default 'f'  
     );
     
    -

    Use the cascade_p only if you want to completely remove the +

    Use the cascade_p only if you want to completely remove the package from the OpenACS.

    In order to determine if a particular package exists in the system, use -the register_p predicate. It returns 1 if the specified -package_key exists in the system, 0 otherwise.

    +the register_p predicate. It returns 1 if the specified
    +package_key exists in the system, 0 otherwise.

     
     function register_p (
         package_key     in apm_package_types.package_key%TYPE
     ) return integer;
     
    -

    Maintaining Multiple Versions of a Package

    While the package authoring API provides a means for registering a +

    Maintaining Multiple Versions of a Package

    While the package authoring API provides a means for registering a package, some information about a package is version dependent. For example, between versions, the owner of a package, its vendor, its URI, and its dependency information may change. The API for package versions allows this -information to be specified. All of these APIs are part of the -apm_package_version PL/SQL package.

    To create a new package version, use the -apm_package_version.new constructor function.

    +information to be specified. All of these APIs are part of the 
    +apm_package_version PL/SQL package.

    To create a new package version, use the +apm_package_version.new constructor function.

     
     function new (
         version_id          in apm_package_versions.version_id%TYPE
    @@ -191,26 +191,26 @@
                             default 'f'
     ) return apm_package_versions.version_id%TYPE;
     
    -

    In order to use this function, an existing package_key must -be specified. The version_name parameter must follow a strict -convention:

    1. A major version number

    2. at least one minor version number. Although any number of minor version +

    In order to use this function, an existing package_key must +be specified. The version_name parameter must follow a strict +convention:

    1. A major version number

    2. at least one minor version number. Although any number of minor version numbers may be included, three minor version numbers is sufficient and is the -convention of software developers.

    3. One of the following:

      • The letter d, indicating a development-only version

      • The letter a, indicating an alpha release

      • The letter b, indicating a beta release

      • No letter at all, indicating a final production release

    In addition, the letters d, a, and -b may be followed by another integer, indicating a version +convention of software developers.

  • One of the following:

    • The letter d, indicating a development-only version

    • The letter a, indicating an alpha release

    • The letter b, indicating a beta release

    • No letter at all, indicating a final production release

In addition, the letters d, a, and +b may be followed by another integer, indicating a version within the release.

For those who like regular expressions:

 
 version_number := ^[0-9]+((\.[0-9]+)+((d|a|b|)[0-9]?)?)$
 
-

So the following is a valid progression for version numbers:

0.9d, 0.9d1, 0.9a1, 0.9b1, 0.9b2, 0.9, 1.0, 1.0.1, 1.1b1, -1.1

To delete a given version of a package, use the -apm_package_version.delete procedure:

+

So the following is a valid progression for version numbers:

0.9d, 0.9d1, 0.9a1, 0.9b1, 0.9b2, 0.9, 1.0, 1.0.1, 1.1b1, +1.1

To delete a given version of a package, use the +apm_package_version.delete procedure:

 
 procedure delete (
     package_id      in apm_packages.package_id%TYPE  
 );
 
 

After creating a version, it is possible to edit the information -associated with it using apm_package_version.edit.

+associated with it using apm_package_version.edit.

 
 function edit (
       new_version_id        in apm_package_versions.version_id%TYPE
@@ -244,8 +244,8 @@
 );
 
 

Files associated with a version can be added and removed. The path is -relative to the package-root which is -acs-server-root/packages/package-key.

+relative to the package-root which is
+acs-server-root/packages/package-key.

 -- Add a file to the indicated version. 
 function add_file(
     file_id             in apm_package_files.file_id%TYPE 
@@ -326,7 +326,7 @@
     version_name_two        in apm_package_versions.version_name%TYPE
 ) return integer;
 
-

Creating Instances of a Package

Once a package is registered in the system, it is possible to create +

Creating Instances of a Package

Once a package is registered in the system, it is possible to create instances of it. Each instance can maintain its own content and parameters.

 
@@ -353,7 +353,7 @@
 
 

Just creating a package instance is not sufficient for it to be served from the web server. A corresponding site node must be created for it. As an -example, here is how the OpenACS API Documentation service +example, here is how the OpenACS API Documentation service makes itself available on the OpenACS main site:

 
 declare
@@ -381,15 +381,15 @@
 show errors
 
 
-

Specifying Configuration Parameters for each Instance

A parameter is a setting that can be changed on a package instance basis. -Parameters are registered on each package_key, and the values +

Specifying Configuration Parameters for each Instance

A parameter is a setting that can be changed on a package instance basis. +Parameters are registered on each package_key, and the values are associated with each instance. Parameters can have default values and can be of type 'string' or 'number.' There is support with this API for setting a number of minimum and maximum values for each parameter, but for most instances, the minimum and maximum should be 1. It is useful to allow or require multiple values for packages that need to store multiple pieces of information under one parameter. Default values are automatically -set when instances are created, but can be changed for each instance.

All of the functions below are in the APM PL/SQL +set when instances are created, but can be changed for each instance.

All of the functions below are in the APM PL/SQL package.

 
 -- Indicate to APM that a parameter is available to the system.
@@ -463,65 +463,65 @@
     attr_value          in apm_parameter_values.attr_value%TYPE
 );  
 
-

Data Model Discussion

The central piece of the data model is the apm_package_types +

Data Model Discussion

The central piece of the data model is the apm_package_types table where each package is registered. When a new application or service is installed on an OpenACS instance, a corresponding row in this table is inserted -with information about the type of package, e.g. if the forum package is installed on your OpenACS server, a row -in apm_package_types will be created, noting that it's an -application package type.

The apm_packages table is used to contain information about +with information about the type of package, e.g. if the forum package is installed on your OpenACS server, a row +in apm_package_types will be created, noting that it's an +application package type.

The apm_packages table is used to contain information about the instances of packages currently created in the system. The -package_key column references the apm_package_types +package_key column references the apm_package_types table to ensure that no package instance can be created for a type that does -not exist.

The apm_package_versions table contains information specific +not exist.

The apm_package_versions table contains information specific to a particular version of a package. Several tables reference this one to -provide further information about the particular version:

  • apm_package_owners +provide further information about the particular version:

    • apm_package_owners Stores information about the owners of a particular version of a package. -

    • apm_package_files +

    • apm_package_files Stores information about the files that are part of a version. -

    • apm_package_dependencies +

    • apm_package_dependencies Stores information about what interfaces the package provides and -requires.

    Parameter information is maintained through two tables:

    • apm_parameters +requires.

    Parameter information is maintained through two tables:

    • apm_parameters This table contains the definition of each of the parameters for a package. -

    • apm_parameter_values +

    • apm_parameter_values This table holds all of the values of parameters for specific package instances.

    A number of views are available for obtaining information about packages -registered in the APM.

    • apm_package_version_info +registered in the APM.

      • apm_package_version_info Provides information about all of the versions in the system with -information available from the apm_package_types table. +information available from the apm_package_types table. -

      • apm_enabled_package_versions +

      • apm_enabled_package_versions A view (subset) of the above table with only enabled versions. -

      • apm_file_info - Provides a public interface for querying file information.

    User Interface

    The APM's user interface is part of the -OpenACS Administration Service. The UI is the primary +

  • apm_file_info + Provides a public interface for querying file information.

User Interface

The APM's user interface is part of the +OpenACS Administration Service. The UI is the primary point of contact with APM by developers and administrators. It is part of OpenACS Administration, because only the site-wide administrator should be able to access it. Thus in order to develop a package, the developer must be granted -site-wide administration.

Configuration/Parameters

APM has two parameters for configuring how it interacts with the UNIX -filesystem, accessible via the Site Map admin +site-wide administration.

Configuration/Parameters

APM has two parameters for configuring how it interacts with the UNIX +filesystem, accessible via the Site Map admin page. These parameters need not be changed under most circumstances, but may -need to be tweaked for Windows compatibility.

  • GzipExecutableDirectory - This directory points to where the gunzip program can be found -for uncompressing gzip archives. This is needed for the -installation of .apm files which are simply gziped -tarballs. Default is /usr/local/bin +need to be tweaked for Windows compatibility.

    • GzipExecutableDirectory + This directory points to where the gunzip program can be found +for uncompressing gzip archives. This is needed for the +installation of .apm files which are simply gziped +tarballs. Default is /usr/local/bin -

    • InfoFilePermissionsMode +

    • InfoFilePermissionsMode This sets the default UNIX permissions used when creating files using the -APM. Default is 775.

Future Improvements/Areas of Likely Change

APM has been in production since OpenACS 3.3, and as of version 4.0 offers a +APM. Default is 775.

Future Improvements/Areas of Likely Change

APM has been in production since OpenACS 3.3, and as of version 4.0 offers a stable set of features. One major feature planned is integration with the OpenACS Package Repository for automatic dependency satisfaction. When a user tries to install a package that depends on other packages, the APM will contact the @@ -539,6 +539,6 @@ repositories worldwide.

Another anticipated change is to split the APM UI into separate systems for authoring, maintaining, and installing packages. The current UI presents all of this functionality in one interface and it can be confusing from a -usability perspective.

Authors

  • System creator: Bryan Quinn, Jon Salz, Michael Yoon, Lars Pind, Todd -Nightingale.

  • System owner: Bryan Quinn

  • Documentation author: Bryan Quinn, building from earlier versions by Jon -Salz, Michael Yoon, and Lars Pind.

Revision History

Document Revision #Action Taken, NotesWhen?By Whom?
0.1Creation9/25/2000Bryan Quinn
0.8Ready for QA9/29/2000Bryan Quinn
0.9Edited for ACS 4 Beta release10/02/2000Kai Wu
1.0Edited for OpenACS 4.5 Beta release03/02/2002Roberto Mello
View comments on this page at openacs.org
+usability perspective.

Authors

  • System creator: Bryan Quinn, Jon Salz, Michael Yoon, Lars Pind, Todd +Nightingale.

  • System owner: Bryan Quinn

  • Documentation author: Bryan Quinn, building from earlier versions by Jon +Salz, Michael Yoon, and Lars Pind.

Revision History

Document Revision #Action Taken, NotesWhen?By Whom?
0.1Creation9/25/2000Bryan Quinn
0.8Ready for QA9/29/2000Bryan Quinn
0.9Edited for ACS 4 Beta release10/02/2000Kai Wu
1.0Edited for OpenACS 4.5 Beta release03/02/2002Roberto Mello
View comments on this page at openacs.org