Index: openacs-4/Announce-OpenACS-5.9.1
===================================================================
RCS file: /usr/local/cvsroot/openacs-4/Announce-OpenACS-5.9.1,v
diff -u -N
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ openacs-4/Announce-OpenACS-5.9.1	7 Aug 2017 23:47:43 -0000	1.2
@@ -0,0 +1,511 @@
+
+OpenACS Version 5.9.1
+
+Release of OpenACS 5.9.1:
+
+This is the announcement of the release of the OpenACS 5.9.1.  This
+release contains many security and performance improvements and
+includes new functionalities.
+
+The release of OpenACS 5.9.1 contains the 88 packages of the oacs-5-9
+branch.  These packages include the OpenACS core packages, the major
+application packages (e.g. most the ones used on OpenACS.org), and
+DotLRN 2.9.1.
+
+The quanitative summary of the changes since the release of
+OpenACS 5.9.0 is as follows:
+
+    3548 files changed, 113292 insertions(+), 90507 deletions(-)
+
+contributed by 5 committers (Michael Aram, Gustaf Neumann, Antonio
+Pisano, Hector Romojaro, Thomas Renner) and 8 patch/bugfix providers
+(Frank Bergmann, Günter Ernst, Brian Fenton, Felix Mödritscher, Marcus
+Moser, Franz Penz, Stefan Sobernig, Michael Steigman). All packages of
+the release were tested with PostgreSQL 9.6.* and Tcl 8.5.*.
+
+Below is a short summary of the changes in this release. For details,
+consult the changelog of the release.
+
+Refactoring of rich-text editor integration
+
+   - Driving force: Debian packaging (e.g. js minified code is not
+     allowed) 
+   - Moved out code from acs-templating, provided interfaces to add
+     many different richtext editors as separate packages
+   - New OpenACS packages:
+      *  richtext-xinha
+      *  richtext-tinymce
+      *  richtext-ckeditor4 (has ability to choose between CDN and
+         local installation via web interface)
+
+Improving admin interface
+
+   - New theme manager:
+      * Goals:
+           + Make it easier to keep track of themes with local
+             modifications 
+           + Make it easier to create local modification a new
+	     themes and to update these
+           + Show differences between default theme parameter (in DB)
+             and actual settings (in subsite parameters)
+           + Allow to delete unused themes
+           + Give site admin hints, which theme is used at which
+             subsite 
+           + Ease theme switching
+	   
+      * Added a subsite::theme_changed callback to be able to handle
+        theme changes in custom themes (was also necessary for proper
+        integration with DotLRN theming)
+	
+      * Added support for these features under subsite admin (/admin/)
+        Improved support for themed templates via
+        [template::themed_template]
+   
+   - Improved (broken) interface to define/manage groups over web
+     interface 
+   - Allow to send as well mail, when membership was rejected
+   - New functions [membership_rel::get_user_id],
+     [membership_rel::get] and [membership_rel::get_group_id] to
+     avoid code duplication
+   - Added support to let user include %forgotten_password_url%
+     in self-registration emails (e.g. in message key
+     acs-subsite.email_body_Registration_password)
+      
+   - Improved subsite/www/members
+      * Make it possible to manage members of arbitrary groups
+      * Improved performance for large groups
+      * Improved configurability: when ShowMembersListTo is set to
+        "3", show list to members only, when this is not the whole
+        subsite
+   
+   - Improved user interface for /admin/applications for large number
+     of applications 
+   - Various fixes for sitewide-admin pages (under /acs-admin)
+   - Update blueprint in "install from repository"
+     (currently just working in NaviServer)
+
+SQL
+
+   - Further cleanup of .xql files (like what as done for acs-subsite
+     in OpenACS 5.9.0): 
+      * 36 files deleted
+      * Removed more than 100 obsolete named queries
+      * Stripped misleading SQL statements
+
+   - Marked redundant / uncalled sql functions as deprecated
+   - Replaced usages of obsolete view "all_object_party_privilege_map"
+     by "acs_object_party_privilege_map"
+
+   - Removed type discrepancy introduced in 2002:
+      * acs_object_types.object_type has type varchar(1000), while
+      * acs_object_types.supertype has type varchar(100)
+      * ... several more data types are involved, using
+        acs_object_types.object_type as foreign key
+
+   - Simplified core sql functions by using defaults:
+      * Number of functions reduced by a factor of 2 compared to
+        OpenACS 5.9.0 (while providing compatibility for clients using 
+	old versions),
+      * Reduced code redundancy
+        Affected functions:
+         +  Reduced content_item__new from 12 versions to 6,
+         +  Reduce content_revision__new from 7 to 4
+         +  Similar in image__new, image__new_revision,
+            content_item__copy, content_item__get_title,
+	    content_item__move
+         + PostgreSQL 9.5 supports named parameter in the same syntax
+           as in Oracle. Further reduction of variants will be
+           possible, once OpenACS requires at least PostgreSQL 9.5
+
+   - Reduced usage of deprecated versions of SQL functions
+     (mostly content repository calls)
+   - Reduced generation of dead tuples by combining multiple DML
+     statements to one (reduces costs of checkpoint cleanups in
+     PostgreSQL) 
+
+   - Permission queries:
+      * Improved performance
+      * Support PACKAGE.FUNCTION notation for PostgreSQL to allow
+        calls permission queries exactly the same way as in Oracle
+       (e.g. "acs_permission.permission_p()"). This helps to reduce the
+       number of postgres specific .xql files.
+       
+   - Modernize SQL:
+       * Use real Boolean types instead of character(1)
+         (done for new-portal, forums, faq, attachments, categories,
+	 dotlrn, dotlrn-forums, evaluation)
+       * Use real enumeration types rather than check constraints
+         (done for storage_type text/file/lob)
+
+
+CR hygienics (reduce cr bloat)
+
+   - Provided means to avoid insert/update/delete operations in the
+     search queue:
+
+     OpenACS adds for every new revision often multiple entries to the
+     search_queue, without providing any means to prevent this. This
+     requires for busy sites very short intervals between queue sweeps
+     (otherwise too many entries pile up). Another consequence is that
+     this behavior keeps the PostgreSQL auto-vacuum daemons
+     permanently active. Many of these operations are useless in cases
+     where the content repository is used for content that should not
+     be provided via search. The changed behavior should honors a
+     publish-date set to the future, since it will not add any content
+     with future publish dates to the search-queue. 
+
+   - Reduced number of insert cr_child_rels operations, just when needed:
+
+     cr_child_rels provide only little benefit (allow to use roles in
+     a child-rel), but the common operation is a well available in
+     cr_items via the parent_id. cr_child_rels do not help for
+     recursive queries either. One option would be to add an
+     additional argument for content_item__new to omit child-rel
+     creation (default is old behavior) and adapt the other cases. 
+
+Security improvements
+
+   - Added support against [[CSRF]] (cross site request forgery)
+      * OpenACS maintains a per-request CSRF token that ensures that
+        form replies are coming just from sites that received the form
+      * CSRF support is optional for packages where CSRF is less
+        dangerous, and such requests are wanted (e.g. search and
+        api-browser) 
+
+   - Added Support for W3C "Upgrade-Insecure-Headers"
+     (see https://www.w3.org/TR/upgrade-insecure-requests/):
+     For standard compliant upgrade for requests from HTTP to HTTPS
+
+   - Added support for W3C "Subresource Integrity" (SRI; see
+     https://www.w3.org/TR/SRI/) 
+
+   - Added support for W3C "Content Security Policy"
+     ([[CSP]]; see https://www.w3.org/TR/CSP/)
+      * Removed "javascript:*" links (all such urls are removed from
+        the 90 packages in oacs-5-9, excluding js libraries
+        (ajaxhelper) and richtext code)
+     
+      * Removed "onclick", "onfocus", "onblur", "onchange" handlers
+        from all .adp and .tcl files in the 90 packages in oacs-5-9 
+	(excluding js libraries (ajaxhelper) and richtext code)
+	
+      * Added optional nonces to all <script> elements with literal
+        JavaScript content 
+
+   - Removed "generic downloader", which allowed to download arbitrary
+     content items, when item_id was known (bug-fix)
+     
+   - Improved protection against XSS and SQL-injection (strengthen
+     page contracts, add validators, added page_contract_filter
+     "localurl", improve HTML escaping, and URI encoding)
+     
+   - Fixed for potential traversal attack (acs-api-documentation-procs)
+
+Improvements for "host-node mapped" subsites
+
+   - Fixed links from host-node mapped subsite pages to swa-functions
+     (must be always on main subsite) 
+   - Made "util_current_directory" aware of host-node-mapped subsites
+   - Added ability to pass "-cookie_domain" to make it possible to use
+     the same cookie for different domains 
+   - Fixed result of affected commands "util_current_location",
+     "ad_return_url", "ad_get_login_url" and "ad_get_logout_url" for
+     HTTP and HTTPS, when UseHostnameDomainforReg is 0 or 1.
+   - Improved UI for host-node maps when a large number of site nodes
+     exists 
+
+
+Reform of acs-rels
+
+   - Made acs-rels configurable to give the developer the option to
+     specify, whether these are composable or not (default fully
+     backward compatible). This is required to control transitivity in
+     rel-segments 
+   - The code changes are based on a patch provided by Michael
+     Steigmann. For details, see:
+      * http://openacs.org/forums/message-view?message_id=4031049
+      * http://openacs.org/forums/message-view?message_id=5330734
+
+Improved status code handlers for AJAX scenarios
+
+   - Don't report data source errors with status code 200
+     (use 422 instead)
+   - Let "permission::require_permission" return forbidden (403) in
+     AJAX calls (determined via [ad_conn ajaxp]) 
+
+Improved Internationalization
+
+   - Extended language catalogs for
+      * Russian (thanks to v v)
+      * Italian (thanks to Antonio Pisano)
+      * Spanish (thanks to Hector Romojaro)
+      * German (thanks to Markus Moser)
+
+   - Added (missing) message keys
+   - Improved wording of entries
+   - Added message keys for member_state changes, provide API via
+     group::get_member_state_pretty 
+
+
+
+
+Improved online documentation (/doc)
+
+   - Fixed many broken links
+   - Removed fully obsolete sections
+   - Improved markup (modernize HTML)
+   - Updated various sections
+
+
+Misc code improvements:
+
+   - 18 issues from the OpenACS-bug-tracker fixed
+   - Made code more robust against invalid/incorrect input
+     (page_contracts, validators, values obtained from header fields
+     such as Accept-Language)
+     
+   - Fixed quoting of message keys on many places
+   - Improved exception handling (often, a "catch" swallows to much,
+     e.g. script_aborts), introducing "ad_exception". 
+
+   - Generalized handling of leading zeros:
+      * Fixed cases where leading zeros could lead to unwanted octal
+        interpretations 
+      * Switch to use of " util::trim_leading_zeros" instead of
+        "template::util::leadingTrim", "dt_trim_leading_zeros" and
+        "template::util::leadingTrim", marked the latter as deprecated
+
+   - URL encoding
+      * "ad_urlencode_folder_path": new function to perform an
+         urlencode operation on the segments of the provided folder
+         path  
+      * "export_vars": encode path always correctly, except
+         no_base_encode is specified 
+      * Fixed encoding of the URL path in "ad_returnredirect"
+
+   - Improvements for "ad_conn":
+      * Added [ad_conn behind_proxy_p] to check, whether the request
+        is coming from behind a proxy server
+      * Added [ad_conn behind_secure_proxy_p] to check, whether the
+        request is coming from behind a secure proxy server
+      * Added [ad_conn ajax_p] to check, whether the request is an
+        AJAX requests (assumption: AJAX request sets header-field
+	Requested-With: XMLHttpRequest")
+      * Added [ad_conn vhost_url] to obtain the url of
+        host-node-mapped subsites 
+
+   - Added various missing upgrade scripts (missing since many years)
+     of changes that were implemented for new installs to reduce
+     differences between "new"-and "old" (upgraded) installations 
+
+   - Templating
+      * Get rid of various pesky "MISSING FORMWIDGET:
+        ...formbutton:ok" messages 
+      * Improved support for javascript event handlers in template::head
+      * New functions "template::add_event_listener" and
+        "template::add_confirm_handler" 
+      * Fix handling, when "page_size_variable_p" is set (was broken
+        since ages) 
+
+   - Improved location and URL handling:
+      * Refactored and commented "util_current_location" to address
+        security issues, handle IPv6 addresses, IP literal notation,
+        multiple drivers,  
+      * Improved "security::get_secure_location" (align with documentation)
+
+      * New functions:
+         + "util::configured_location"
+         + "util::join_location", "util::split_location"
+        for working on HTTP locations to reduce scattered regexps
+        handling URL components 
+      * Improved IPv6 support
+      * Use native "ns_parseurl" when available, provide backward
+        compatible version for AOLserver 
+
+   - MIME types:
+      * Added more Open XML formats for MS-Office to allowed content types
+      * Modernized entries to IANA recommendations
+      * New function "cr_check_mime_type" centralizing the retrieval
+        of the mime_type from uploaded content 
+
+   - Finalized cleanup of permissions (started in OpenACS 5.9.0):
+     Get rid of "acs_object_context_index " (and therefore on
+     acs_object_party_privilege_map " as well) on PostgreSQL.
+     Reasons:
+       * huge table,
+       * expensive maintenance, used only in a few places,
+
+   - Misc new functions:
+      * "lang::util::message_key_regexp": factor out scattered regexp
+         for detecting message keys
+      * "ns_md5" and "ns_parseurl": improve compatibility between
+        AOLserver and NaviServer 
+      * "ad_dom_sanitize_html": allow to specify different sets of
+        tags, attributes and protocols and "ad_dom_fix_html", which is a
+        light weight tidy variant.
+
+   - Improved HTML rendering (acs-api-browser), provide width and
+     height to speed up rendering 
+   - Improved ADP files (e.g. missing doc(title))
+   - Added usage of "ad_include_contract" on more occasions
+   - Modernize Tcl and HTML coding
+   - Reduced dependency on external programs (use Tcl functions instead)
+   - Improved robustness of "file delete" operations all over the code
+   - Improved documentation, fix demo pages
+   - Aligned usages of log notification levels (distinction between
+     "error", "warning" and "notice") with coding-standards
+
+   - Cleaned up deprecated calls:
+      * Removed usage of deprecated API functions
+       (e.g. "cc_lookup_email_user",
+        "cc_email_from_party", "util_unlist", ...)
+      * Moved more deprecated procs to acs-outdated
+      * Marked remaining (and unused) "cc_*" functions as well as
+        deprecated. 
+
+   - Improved Oracle and windows support
+   - Fixed common spelling errors and standardize spelling of product
+     names all over the code (comments, documentation, ...)
+   - Many more small bug fixes
+
+
+Packages:
+
+   - New Package Parameters
+   
+      * acs-kernel:
+         + MaxUrlLength: remove hard-coded constant in request
+           processor for max accepted url paths 
+         + SecureSessionCookie: Let site admin determine, whether or
+           not to use secured session cookies (useful, when not all
+           requests are over HTTPS) 
+         + CSPEnabledP: activate/deactivate CSP
+
+      * acs-kernel (recommended to be set via config file in section
+        "ns/server/${server}/>acs" 
+         + NsShutdownWithNonZeroExitCode: tell NaviServer to return
+           with a non-zero return code to cause restart (important under
+           windows) 
+         + LogIncludeUserId: include user_id in access log
+
+      * acs-api-browser:
+         + ValidateCSRFP: make checking of CSRF optional (default 1)
+
+      * acs-content-repository:
+         + AllowMimeTypeCreationP: Decides whether we allow unknown
+           mime types to be automatically registered (default: 0} 
+
+      * news-portlet:
+         + display_item_lead_p: Should we display news leads in the
+           portlet? (default 0) 
+
+      * search:
+         + ValidateCSRFP: make checking of CSRF optional (default 1)
+
+      * xotcl-request-monitor:
+         + do_track_activity: turn activity monitoring on or off (default 0)
+
+   - New OpenACS packages:
+      * richtext-xinha
+      * richtext-tinymce
+      * richtext-ckeditor4 (has ability to choose between CDN and
+        local installation via Web interface) 
+      * openacs-bootstrap3-theme (as used on openacs.org)
+      * dotlrn-bootstrap3-theme
+
+   - xotcl-core:
+      * Improved XOTcl 2.0 and NX support (e.g. api-browser)
+      * Added "-debug", "-deprecated" to ad_* defined methods
+        (such as e.g. "ad_instproc")
+      * Make use of explicit "create" statements when creating
+        XOTcl/NX objects (makes it easier to grab intentions and to
+        detect typos) 
+      * Added parameter to "get_instance_from_db" to specify, whether
+        the loaded objects should be initialized 
+      * Added support for PostgreSQL prepared statements of SQL
+        interface in ::xo::dc (nsdb driver) 
+
+   - xowiki:
+      * Named all web-callable methods www-NAME (to make it clear,
+        what is called, what has to be checked especially carefully) 
+      * Moved templates from www into xowiki/resources to avoid naming
+        conflicts 
+      * Improved ckeditor support
+      * Added usage of prepared statements for common queries
+      * Improved error handling
+      * Better value checking for query parameter, error reporting via
+        ad_return_complaint 
+      * Added option "-path_encode" to methods "pretty_link" and
+       "folder_path" to allow to control, whether the result should be
+        encoded or not (default true)  
+
+      * Form fields:
+         + Improved repeatable form fields (esp. composite cases),
+           don't require preallocation (can be costly in composite
+           cases) 
+         + Added signing of form-fields
+         + Added HTML5 attributes such as "multiple" (for "file") or
+           "autocomplete" 
+         + Fixed generation of "orderby" attribute based on form-field
+           names 
+         + richtext: allow to specify "extraAllowedContent" via options
+         + Improved layout of horizontal check boxes
+
+      * Menu bar:
+         + Added dropzone (requires bootstrap): drag and drop file
+           upload 
+         + Added mode toggle (requires bootstrap)
+         + Extended default policies for handling e.g. dropzone
+           (file-upload method) 
+         + Distinguish between "startpage" (menu.Package.Startpage)
+           and "table of contents" (menu.Package.Toc) 
+
+      * Notifications:
+         + Added support for better tailorable notifications:
+           introduced method "notification_render" (similar to
+           "search_render") 
+         + Added support for tailorable subject lines (method
+           "notification_subject") 
+
+      * Improved bootstrap support, use "bootstrap" as
+        PreferredCSSToolkit 
+      * Switched to ckeditor4 as PreferredRichtextEditor
+      * Improved handling of script-abort from within the payload of
+        ::xowiki::Object payloads 
+      * Added parameter to "get_all_children" to specify, whether the
+        child objects should be initialized 
+
+   - xowf:
+      * Added property "payload" to "WorkflowConstruct" in order to
+        simplify customized workflow "allocate" actions 
+      * Internationalized more menu buttons
+
+   - xotcl-request-monitor
+      *  Added class "BanUser" (use. e.g. ip address to disallow
+         requests from a user via request monitor) 
+      *  Added support for optional user tracking in database
+      *  Added support for monitoring response-time for certain urls
+         via munin 
+      *  Increased usage of XOTcl 2.0 variable resolver (potentially
+         speed improvement 4x) 
+      *  Performed some refactoring of response-time handling to allow
+         site-admin to make e.g. use of NaviServer's dynamic
+         connection pool management (not included in CVS) 
+      *  Added support for partial times in long-calls.tcl to ease
+         interpretation of unexpected slow calls 
+      *  last100.tcl: Don't report hrefs to URLs, except to SWAs
+
+   - chat:
+      * Introduced new options to set chat rooms so login and/or
+        logout message are not issued every time a user enters/exits a
+       chat-room (important for chats with huge number of participants) 
+      *  Parameterized viewing of chat-logs
+      *  Fixed cases of over-/under-quoting
+      *  Fixed JavaScript for IE, where innerHTML can cause problems
+
+   - file-storage:
+      * Don't show action keys, when user has no permissions
+      * Added support for copying of same-named files into a folder
+        (adding suffix) 
+      * Fixed old bugs in connection with "views" package
+