Table of Contents
Table of Contents
+
Automated Testing is part of acs-core, and therefore should already be installed in any OpenACS system. Individual automated tests are stored within each package in package/tcl/test. -
Automated Testing provides a framework for executing tests of all varieties and for storing and viewing the results.
| Req # | Status in 5.0 | Priority for 5.1 (A=required, B=optional) | Description | ||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Done | Done | Execute TCL tests. Execute a sequence of TCL code is executed and determine the correctness of the results. | ||||||||||||||||||||||||||||||||||||||||||||||||
| 1.1 | partial | A | Execute HTTP tests. Execute tests that can interact with a the webserver via the external, HTTP interface, including retrieving pages, following links, and submitting forms. (This is partially done in the sense that we can make http calls from tcl api, but there is no framework for doing anything complicated.) | ||||||||||||||||||||||||||||||||||||||||||||||||
| 1.1.1 | � | A | Execute tclwebtest scripts. A test can contain tclwebtest commands. If tclwebtest is not installed, those commands fail gracefully. | ||||||||||||||||||||||||||||||||||||||||||||||||
| 1.1.1.1 | partial | A | tclwebtest is easy to install. Tclwebtest installation is fully documented and can be installed with less than five steps. (Install is documented in 5.0, but there's a can't-find-config error; also, some new work in tclwebtest HEAD needs to packaged in a new tarball release.) | ||||||||||||||||||||||||||||||||||||||||||||||||
| 2 | � | A | Tests have categories. Individual tests can be marked as belonging to zero, one, or many of these categories. The UI provides for running only tests in selected categories, and for viewing only results of tests in selected categories.
-Testing Mode + | ||||||||||||||||||||||||||||||||||||||||||||||||
| 2.1 | � | A | Each test can be associated with a single OpenACS.org bug (ie, store bug id as in integer, or store full url so that this can point to other bugs) | ||||||||||||||||||||||||||||||||||||||||||||||||
| 3 | � | B | Tests can be ordered lists of other tests. minimal: verify that a test proc can call other test procs. Better: A test can be created within the GUI by selecting other tests. This test is stored in the database and can be exported. (This is related to a bigger issue of storing test scripts in some format other than tcl procs.) | ||||||||||||||||||||||||||||||||||||||||||||||||
| 4 | � | C | Test scripts can be imported and exported. It should be possible to import a test into the database from a file, and to export it to a file. These files should be sharable by different OpenACS installations. It should be possible to import/export directly between running OpenACS sites. (We should look at what did and didn't work in acs-lang catalog files and work from there.) | ||||||||||||||||||||||||||||||||||||||||||||||||
| 5 | � | B | Macro Recording. End users can create and run tests from the web interface without writing code.
- 1) UI to turn on macro mode. 2) basic recording: when you fill out a form while macro mode is on, the submit is caught and displayed as tclwebtest code, and then executed. 3) UI for creating aa_true tests automatically, based on the content of the page. (For example, a form that says "the returned page must contain [ type regexp here] that spits out aa_true "test X" [string regexp blah blah] | ||||||||||||||||||||||||||||||||||||||||||||||||
| 6 | � | A | Notification subscriptions are available for "email me whenever this test fails" and "notify me whenever a test in this category fails" | ||||||||||||||||||||||||||||||||||||||||||||||||
| 7 | � | A | The results of an automated test are optionally written to an xml file. |
Forum Posting: tclwebtest with openacs-4/etc/install tests -- help getting started
Forum Posting: Berlin bug bash proposal
Forum Posting: tclwebtest with openacs-4/etc/install tests -- help getting started
Forum Posting: Berlin bug bash proposal
Here's the entire chain of code used to set up auto-rebuilding servers on test.openacs.org
The master server shows the status of all other servers. For test.openacs.org, it listens on port 80.
The acs-automated-testing parameter IsInstallReportServer is set to 1
The acs-automated-testing parameter XMLReportDir is set to /var/log/openacs-install. This is arbitrary - it just needs to be somewhere all the servers can write to.
For each server that will be monitored:
Suppose the first test server is service1. Set up a dedicated user and automated install script.
To run automated testing automatically each time the server is rebuilt, add this to /home/service1/install/install.tcl:
set do_tclapi_testing "yes"
Get the results of the automated tests dumped where the master server can see them - in this example, the same directory as above, /var/log/openacs-install, by adding this to install.tcl (requires 5.1):
set install_xml_file "/var/lib/aolserver/service0/packages/acs-core-docs/www/files/install-autotest.xml"
This will copy in the file install-autotest.xml:
<?xml version="1.0"?> + +<!-- This is an install.xml which can be used to configure servers for reporting their automated test results. Requires acs-automated-testing 5.1.0b2 or better --> + +<application name="acs-automated-testing" pretty-name="Automated Testing" home="http://openacs.org/"> + + <actions> + + <set-parameter package="acs-automated-testing" name="XMLReportDir" value="/var/log/openacs-install"/> + </actions> + +</application> +
which will, during install, configure that parameter in acs-automated-testing on the monitored server.
To enable the 'rebuild server' link, edit the file /usr/local/bin/rebuild-server.sh:
#!/bin/sh +# script to trigger a server rebuild + +# hard-coding the valid server names here for some minimal security +case $1 in + service1) ;; + service2) ;; + *) + echo "Usage: $0 servername" + exit;; +esac + +sudo /home/$1/install/install.sh 2>&1
and allow the master user to execute this file as root (this is a limitation of the automatic install script, which must be root). In /etc/sudoers, include a line:
master ALL = NOPASSWD: /usr/local/bin/rebuild-server.sh