• last updated 15 hours ago
Constraints: committers
Constraints: files
Constraints: dates
reduce verbosity

Added parameter to define a default dbn to a database connection

By this change, one can now define a default dbn at the creation time

of a database connection object. Before, it was necessary to pass

the "-dbn" value to every single command. The parameter can still be

used for particular queries as before to overrule the default.

Example for defining a connection context to a pool named "legacy"

using the PostgreSQL database interface

::xo::db::DB-postgresql create ::xo::dc1 -dialect postgresql -dbn legacy

lappend _ [::xo::dc1 get_value . {select count(*) from acs_objects}]

lappend _ [::xo::dc get_value . {select count(*) from acs_objects}]

#> 660 51606

free explicitly answer ns_set in "sets" method

free ns_set storage more eager (when e.g. large queries are used in longer loops)

Undo part of the recent cache-deactivation for attribute definitions

When all calls for "db_attribute_defined" are performed without

caching, this results in a huge amount of additional SQL queries,

since these tests are performed during blueprint definition as well

during every blueprint reload of every db attribute. On a small site

with just 6 connection threads (2 monitor, 2 default, 2 slow), this

results during startup with 969 additional SQL queries, a reload of

xowiki causes an additional 1063 of such queries. When a site defines

much more connection threads ((the LEARN site has e.g. 85 connection

threads defined), these DB attribute testing operations will result in

10K+ of mostly useless SQL queries. As a consequence of this, the

startup of the server will become slower, reloads will become slower

etc., which is bad especially for large sites.

The new implementation seems to fix the original problem case (running

xotcl_core_tutorial_4 multiple times). If there is still a problem

with installing xolp [2] this problem should be analyzed and fixed

probably there. If necessary, a proper bug report would be appreciated

to reduce guessing work.

[1] https://fisheye.openacs.org/changelog/OpenACS/?cs=oacs-5-10%3Aantoniop%3A20230310183055

[2] https://fisheye.openacs.org/changelog/OpenACS/?cs=oacs-5-10%3Aantoniop%3A20230317160241

As it turns out, even caching only by request has consequences e.g. when one installs xolp from scratch

Disabl caching and prepare the statement instead

Improve robustness when a class is created/deleted multiple times (e.g. by repeating xotcl-core.xotcl_core_tutorial_4 automated test)

imporved log message

Use an idiom without expr to fill up the multirow after the body execution

Performance improvements:

- add with_headers flag to ::xo::dc list_of_lists behaving like the db_list_of_lists counterpart

- use ::xo::dc list_of_lists as internal for ::xo::dc foreach and ::xo::dc multirow to reduce the need for ns_sets

Collect all query results before executing the code to avoid the out-of-pools bug

-prepare flag must not be supplied when not available

Fix regression: the reimplementation of ::xo::dc foreach reintroduced the old "out of pools bug"

The fix makes again use of the ns_set api to free the handle before the code is executed

Remove hack and prefer to document limitation in the automated test

  1. … 1 more file in changeset.
Fix typo and include again the hack for strings with colon char in the fallback implementation of ns_pg_prepare

Fixes xotcl-core.test_prepared_statements automated test

improved spelling

use the built-in bindvars parser (when available) for prepared statements

Do not extend the existing multirow, we only enforce that this call will have the same columns

Do not enforce that body is a list

Fix typo

Performance improvements:

- set variables from the ns_set and extended variables separately

- collect the values and append to the multirow in one sweep

- when no code body is there, just bulk append the values

This appears to be ~5% faster than db_multirow when both are invoked with a code body and ~30% faster when invoked without (with no prepared statements)

Simplify idiom

Fix behavior of continue in the multirow code block, make the generic fallback behave the same as the postgres version with respect to appending

Cleanup leftover line

Provide an ::xo::dc api to generate multirows

Notable differences with the classical db_multirow:

- a multirow will always be appended when it already exists. The constraint that the two multirows must have the same columns remains.

- no "if_no_rows_code_block"

- no unclobber

- no subst, do it yourself :-)

- no cache stuff

- support for prepared statements

The remaining behavior has been kept the same, e.g. variables will always be reset to empty string, even if they existed outside of the code block. Compatibility has been checked with knowns idiosyncrasies.

Provide, as for other interfaces, a Postgres implementation of database foreach that will support prepared statements won't just wrap the db_* api

Make sure the original SQL stays unchanged, as it is used e.g. in the nsv storing the statement and in log messages

Improve the approach with strings containing colons in prepared SQL statements:

we first normalize all strings with a safe placeholder, substitute the variables, then put the strings back in place.

Improve the regexp detecting variables in a prepared statement, so that a prepared variable must not be preceded by a semicolon (as before), but also by any character allowed for a variable name

Allow to prepare a statement with no parameters (See https://www.postgresql.org/docs/11/sql-prepare.html)