OpenACS 4 Porting Resources

Goal

This is an attempt to compile in one location all the resources that a porter would need to get started (and unstuck from time to time). With your help, this document will get updated as we come upon and squash more problems.

Essentials

These documents are absolute must reads.

Porting Guidelines

This document describes the basic changes that will be needed in the Oracle datamodel

OpenACS Package Manager Wimpy Point

This is a multipage document that describes how the package manager functions. It also describes the first steps that the porter needs to take.

The Query Dispatcher

The Query Dispatcher is the key to db-independence in OpenACS 4 and this document will help you understand it. The last section on dynamic queries is not completed, but that info is in the OpenACS Package Manager Wimpy Point above.

Oracle to Postgres Porting Guide

Explains major differences between Oracle 8i and Postgres.

FAQ

How do I install OpenACS 4?

The Brief OpenACS 4 Installation Guide provides a skeleton framework to getting OpenACS 4 working on your machine

How do I upgrade to Postgres 7.1?

Read Pascal Scheffer's guide to upgrading Postgres from 7.0 to 7.1

How do I get the latest CVS? How do I commit changes?

OpenACS CVS HOWTO

Where do I get the Query Extractor?

Download the Query Extractor. This tool will extract queries from aD packages and do a lot of the porting work for you. Documentation on how to use it is in the /doc directory once you extract the tar file.

How do I use the new Content Repository?

Read the Content Repository HOWTO

What packages are available for porting?

Read the OpenACS 4 Status document to see which packages still need to be ported. You can also find out how far along other packages are coming. Be sure to update your status with Don frequently.

What goes in which .xql file again?

Try this post

Are we using any naming/style conventions?

In PL/PGSQL functions, we're using 'p_' prefixes for arguments to the function and 'v_' prefixes for other local variables. Also, for function calls, we're trying to keep comments that explain the purpose of the arguments. For example:

workflow.add_place (
    workflow_key => 'ttracker_wf',
    place_key    => 'end',
    place_name   => 'Closed',
    sort_order   => 4
);

becomes

select workflow__add_place (
    'ttracker_wf',	-- workflow_key 
    'end',		-- place_key    
    'Closed',		-- place_name   
    4			-- sort_order   
);

I found a 'connect by'. What do I do with it?

Read this document

How do I port outer joins?

Check out these bboard posts: Post 1 Post 2 Post 3

How do I deal with dynamic queries?

Well you should have read this in the WP. Plus here's a bboard post. And another.

One of the functions that I have to port has more than 16 parameters, but PG's limit is 16.

Bummer, huh? Hopefully PG will increase the default number of parameters, but until that happens, we need to workaround it. Look throughout the code to see how the function is called. Are some of the parameters always the same? Then go ahead and take those parameters out and set them explicitly in the function's body. If that's still not enough, then you'll have to overload the function with different sets of parameters. See packages/acs-content-repository/sql/postgresql/content-item.sql to see how Dan ported content_item__new.

How should I name my sequences so they're consistent between Oracle and PG

See this post.

How do I call PL/PGSQL functions?

Check out this post

Can I see a function's source code while in psql?

Yup, this has the advantage of showing you line numbers, which are sometimes spit up by PG error messages:
openacs=# select * from acs_func_defs where fname='acs_object__new'

I've looked at some ported packages and I've noted that ported queries are in the .xql files, but are also still in the .tcl files. Is that right?

Yup, at least for now. At some point we may take the queries out and perhaps replace them with some other information, such as which variables get set with that particular query. As long as the proper queries are in the .xql file, the query dispatcher will ignore the query in the .tcl file.

How does the permissions system work?

ACS 4 Permissions Tediously explained

I just wanna see how work is progressing.

You can browse the CVS or download a nightly tarball.

Where are all the porters?

Why on IRC of course. We're on #openacs on the OpenProjectsNet. Feel free to drop by, whether or not you're a porter! You can read some old logs.

A Day in the Life of a Porter

Let me make a feeble attempt at describing how to port a package.

1. Install OpenACS 4, Pick an unported package and ask Don if anyone else is porting it.
2. Go into /packages/my-package/sql. If there are already 2 directories (oracle and postgresql), then you're all set. If not copy all the files in sql/ to sql/oracle/ and duplicate the sql/oracle directory to sql/postgresql. Your tree should now look like:

   my-package
     sql
       oracle
         my-package-create.sql
         my-package-drop.sql
       postgresql
         my-package-create.sql
         my-package-drop.sql
   

3. Port the datamodel scripts: my-package-create.sql and my-package-drop.sql. It is to your advantage to make sure that my-package-drop.sql is working. It'll make your life easier when you want to test your package.
4. Download and untar the query extractor. Read the docs. In short, edit the file config/openacs_tools.conf. Edit the following parameters to match your installation:
   • packages_dir: /web/openacs-4/packages - Where is your packages directory
   • packageslist: my-package - Which packages you want to extract
   • sql_xml_outputdir:/home/nsadmin/openacs_tools/xml - Where you want the output to go
5. Run the extractor: bin/AcsSqlExtractor
6. The queries will now be in xml/my-package and you can copy them to your OpenACS installation
7. Now fire up OpenACS in your browser and make your way over to /acs-admin/apm. Click on 'Install Packages'. It may take a while to build a list of available packages.
8. In emacs, open up a shell and do a tail -f on your OpenACS 4 error log
9. Install and enable my-package. The APM will then run your create script. If you're lucky, everything will work fine and you can move to the next step. If not, try to figure out how far along it got, using either the output in the browser or the output in your error log. Once you find what choked, fix the create script. Then go into psql and drop everything that was created before it choked so that your install is free of all remnants of 'my-package'. Then go back to the browser and try to reinstall the package. If it still doesn't work, debug, purge and retry. Again and again until it works. This is the first place where an effective drop script comes in handy. If the installer chokes, just find where it chokes, fix the bug, feed your drop script to psql and reinstall.
10. Restart OpenNSD/AOLServer
11. Go back to the APM (/acs-admin/apm). Your package should be listed there. Click on it.
12. Check that PostgreSQL shows up in the 'Database Support' column. Then click on 'Manage File Information'
13. At the bottom of the page, click on 'Scan for new files' and then add the new files to your project. When you're done, go back to my-package APM info page and click on 'Write an XML Spec File...'. This updates the info file.
14. Also on the packages APM page, there's a link to 'Manage Dependency Info'. Make sure that this is up-to-date and correct otherwise users will have difficulty loading or using your package. So, if while porting, you realize that your package uses acs-mail or has templating queries from acs-templating, be sure to add those packages as dependencies.
15. Next, go through each and every .xql file that the query extractor created and do all your fixes. Fix outer joins, fix decode statements, fix pl/sql blocks, etc. Feel free to post to the bboard or on IRC for help. Remember the goal is to make queries SQL92 compliant and get them in the script.xql file. If they have any PG specific features, they go in -postgresql.xql. Oracle specific features -> -oracle.xql
16. Once you're done with all the .xql files, do a quick run through all the .tcl files to make sure that there are no unported queries. Some queries do not get picked up by the query extractor, and you'll have to port those yourself.
17. Once you're done, load your package on the site-map and bang away. If you can, test the oracle and postgres versions. This is where a functioning drop script really comes in handy because you'll want to drop and recreate your package repeatedly.

Throughout this process, you'll occasionally come to the point when you want to alter a table or alter a function. If so, go ahead and do it in psql. You don't need to drop your db and reload the entire datamodel. For example, if you just want to change one function, drop that function and recreate it with your fix. The package will now run your new function. If you want to drop a table, it may be a little more complicated, but at most, you can drop your package with your package drop script and then reload your fixed create script. There's no need to reload the datamodel or even reinstall from the APM. Of course, once you're done, it's probably a good idea to do a full fresh install to make sure that you haven't inadvertently broken anything.

When to commit changes: After getting the datamodel-create script working properly and then extracting queries, I update the .info file and make my first commit. Then I gradually commit as I'm porting the .xql files.

This is a very rough draft and I'm sure all of you have tips that you're just dying to impart, so please feel free to let 'em out!

History

Date	Action
2001-07-13	First Revision