================================ OpenACS 5.8, released 2013-08-30 ================================ Summary: * Compatibility with PostgreSQL 9.2: The new version installs without any need for special parameter settings in new PostgreSQL versions. This makes it easier to use e.g. shared or packaged PostgreSQL installations. * Compatibility with NaviServer 4.99.5 or newer * Various performance and scalability improvements * Various bug fixes * Altogether, OpenACS 5.8.0 differs from OpenACS 5.7.0 in more than 18.000 modifications (781 commits) contributed by 7 committers. Performance Improvements: * Re-implementation of permissions lookup based on PostgreSQL recursive queries (supported by PostgreSQL versions 8.4 or newer). Previously, larger installations were forced to use PostgreSQL 8.3). * Improved scalability - Reduce mutex locking times: the new version requires less mutex locks (e.g. for secret token handling) and avoids long locking times. Previously, many cache maintenance operations retrieved all keys from the cache, which can cause long mutex lock times. The new version avoids many such situations by using more efficient NaviServer operations when available. - Reduce mutex-stress on util-memoize cache for the most frequent operations and use per-thread variables when possible (for ns_cache and ns_v). * Reduced per-request overhead. The new version of the rp_filter (which is run on every non-resources request) is often faster by a factor of 2. * Produce more efficient code from the adp-compiler (using byte compiled functions, using local vars, C-implemented functions) Bug Fixes: * acs-bootstrap-installer: - the filter "install_handler" was fixed to return "filter_return" instead of "". - Change ad_returnredirect to ns_returnredirect, otherwise the installer returns an error (tries to access non-existing tables) when the browser asks for /favicon.ico. - Ensure that bootstrap_write_error works on AOLServer and NaviServer - Ensure sorted loading order of bootstrap files (Tcl's glob does not guarantee order) * acs-tcl: - Store secret tokens as described in the comments per thread (previously it did not) - Fixed broken site_node::conn_url see: http://openacs.org/forums/message-view?message_id=3653550 - request-processor: Reset "extra_url", to avoid looking for internal redirects on errors. - Fixed bug to flush ::party::email_not_cached in case party email has been updated. - During installation process database-init.tcl is loaded before installing the kernel, therefore one has to avoid using parameter::get_from_package_key which would cause us to cache a acs_kernel package_id of 0, which has negative impacts when trying to install .lrn for example (when executing install.xml statements that change kernel package's parameter values). - Catch mime initialization when processing incoming messages so corrupted messages don't blow up queue sweeper run. * apm-procs: - The process of installing acs-authentication's service contract implementations ends up trying to reload /packages/acs-service-contract/tcl/acs-service-contract-init.tcl, for which there is no entry in the apm_library_mtime nsv_array, therefore this logging message aborts the installation. - Fixed bug to avoid duplicate load of acs-lang - Fixed incorrect variable in apm-create.sql - apm_get_package_files has new boolean switches "-include_data_model_files" and "-all" in order to allow control of the number of returned files. - use ns_http for file transfer when available for loading apm-packages * acs-content-repository: - Fixed missing index and foreign key constraints on cr_item_publish_audit - Fixed bug to make sure we exclude null paths from cr_files_to_delete * acs-templating: - Fixed memory leak in acs-templating. The original code had at least two OpenACS caused memory leaks: it led to an unbounded growth for __template_cache_value and did not unset the nsv variables __template_*_timeout. This bug broke openacs.org (forums posting lead to the message "Not enough memory to allocate the list internal rep"). The new code makes use of the per entry expire feature of ns_cache and emulates the behavior otherwise. - Fixed bug with XSS Security Vulnerability in Enhanced Text or HTML Textarea - Renamed Xinha plugin CSS to CSSDropdowns to fix potential breakage for Firefox 22 and Chrome 28 as suggested by http://trac.xinha.org/ticket/1609 - Updated to Xinha (to version from Dec 3, 2012) due to reported problems with IE9 - Fixed bug in tinymce init code regarding booleans - Fixed bug in date-procs widgets, where, in display mode, minute and second fragments should be the same size as the hidden widget value (usually 2). - Fixed incorrect syntax of regexp in element handling code. - Fixed quoting problem with "tiny-plain" form template and checkboxes (bug #3210) - Detect broken ns_quotehtml in adp-compiler and use ad_quotehtml instead in these cases. - Don't quote boolean vars when initializing tinymce init params * json-procs: Fixed ""-must-be-null problem for results returned from the database. Added a utility in json-procs.tcl to help with this, but since I also needed to convert PostgreSQL intervals to the "PT01H02M03.5S" format used by the SCORM RTE in actuality it's fixed in the scorm player package. But the JSON proc may be useful for someone else so I left it in. * acs-lang: Fixed bug #3203 where timeanddate.com regexp needed updating. Made regexp more forgiving. Also cleaned up related parsing to help minimize future maintenance. * acs-admin: - Applied "no quote" in user-management to the return URL since it is not displayed in the browser, and it's quoted in the form under display mode, causing a double quoting. - Provide compatibility with NaviServer * ref-language: fixed catch syntax and removed unique index so that upgrade to openacs.org runs without problems. Probably other db's out there have the same issue. * acs-api-browser: fixed bug #3212 where API Browser link to show procedures didn't work. * acs-automated testing: don't produce hard errors in case, no reports are available. * acs-mail-lite: If processing of email message causes an error, keep it locked to prevent repeated attempts by sweeper. * tsearch2-driver: - Made search callback compatible with definition in search/tcl/search-procs.tcl - remove empty parens from search query to avoid tsearch errors * misc: - removed duplicate files that caused troubles under macOS. For some reason, several files existed with uppercase and lowercase names, where the latter was a previous convention in Xinha. cvs on macOS reported "cvs update: move away...; it is in the way" even on new checkouts. - convert ppt now via catppt to text, part of catdoc package on most distros - apm__register_parameter/10 was missing on openacs.org; added a update script, since other installations might miss this as well - Added missing message keys. Documentation improvements: * Provided release summary * Added complete raw changelog * acs-core-docs: Extended Makefile for doc generation such it works with MacPorts * acs-core-docs: Regenerated documentation * acs-kernel: Clarified unit size of MaxSize in description Configuration Changes: * Adding config parameter ReverseProxyMode in order to let know openacs that it runs behind a proxy that handles encrypted connections and that it should always use the https protocol. * etc/config.tcl: Removed duplicate lines in config file Code Changes: * General: - PostgreSQL overhaul: * Made OpenACS compatible with PostgreSQL 9.* (get rid of backslash escaping in strings, or use extended string syntax E'...') * Use $$ quoting in SQL files * Changed code to use modern-style argument passing instead of aliases. * Fixed function args in many places * Added missing upgrade scripts to change some PL/pgSQL functions in order to be compatible with PostgreSQL 8.x. - Use Tcl 8.5 idioms on several places instead of old quirks + use expand operator instead of dangerous "eval", + use "in" operator instead of "lsearch" when possible + use "lassign" instead of multiple "lindex", + use "unset -nocomplain ..." instead of "info exists" + unset", + get rid of empty_string_p and use byte-compiled built-ins - Use absolute (namespace prefixed) names to avoid namespace confusions, reduce number of "global" statements - Switch from "regexp" to "match" since latter is much faster - Use "info command" rather than "info proc" to allow for C-coded replacements. - Reduce usage / get rid of deprecated code: + Getting rid of last calls to deprecated ad_table. Instead using the list-builder. There is still one call to ad_table within acs-subsite but this page is not linked from anywhere anymore (probably we remove it later). + Getting rid of usage of "template::util::multirow_foreach" + Reduce usage of deprecated calls acs_sc_call, ad_user_new, validate_integer in core components + Replaced deprecated "ns_mkdir" proc with "file mkdir" + Replaced "ns_info tcllib" by $::acs::tcllib since "ns_info tcllib" is deprecated since it assumes a single server + Replaced "ns_info pagedir" by $::acs::pageroot since "ns_info pagedir" is deprecated since it assumes a single server - Improved NaviServer compatibility - Improve scalability * acs-tcl: - security-procs.tcl: Reduced hard-coded config-section names in code - Treat ad_set_cookie's boolean switches as such. * acs-kernel: on PostgreSQL version 9.0 (and lower) the function string_to_array takes only two parameters * apm_file loader: Various changes to improve server startup time significantly - ignore directories www/resources/, catalog/, upgrade/ - ignore extensions .html .gif .png .jpg .ico .pdf .js .css .xsl .tgz .zip .gz .java .sql - Added git to the list of ignored directories for apm loading * acs-bootstrap-installer: - Aligned implementation of apm_backup_file_p with documentation - Used usual style non-positional arguments in ad_proc - use Tcl 8.5 operations - acs-bootstrap-installer: Turned on warning for using deprecated commands, at least during the release phase * acs-templating: - Added "@var;literal@" notation: such variables are neither quoted nor localized. This leads to more sane code in case of comparisons in .adp files * acs-content-repository: added several mime-types (application/x-zip, application/x-sh, application/rdf+xml) for PostgreSQL and Oracle * openacs-default-theme: Moved "list-style: none" from UL to LI to avoid circle-bullets in tabbed master with yui-base * acs-subsite: Changed ad_return_error to ad_return_exception_page 404 with same message, because error is not with the system, but with the request. This helps to prevent bots from retrying when users no longer exist etc.