We have our own file-storage application because we want all users to be able to collaboratively maintain a set of documents. Specifically, users can save their files to our server so that they may:
We want something that is relatively secure, and can be extended and maintained by any ArsDigita programmer, i.e., something that requires only AOLserver Tcl and Oracle skills.
In ACS 4, File Storage can be implemented on top of the Content Repository. Thus, there is no data model associated with File Storage. It is only a UI and a small set of Tcl and PL/SQL library procedures. The actual storage and versioning is relegated to the Content Repository.
File Storage was created to provide a mechanism for non-technical users to collaborate on a wide range of documents, with minimum sysadmin overhead. Specifically, it allowed clients to exchange design documents (often MS Word, Adobe PDF, or other proprietary desktop file formats) that changed frequently without having to get bogged down by sifting through multiple versions.
Why is a file-storage application useful?
If you simply give everyone FTP access to a Web-accessible directory, you are running some big security risks. FTP is insecure and passwords are transmitted in the clear. A cracker might sniff a password, upload Perl scripts or ADP pages, then grab those URLs from a Web browser. The cracker is now executing arbitrary code on your server with all the privileges that you've given your Web server.
The File Storage application is not a web-based file system, and can not be fairly compared against such systems. The role of File Storage is to provide a simple web location where users can share a versioned document. It does not allow much functionality with respect to aggregate file administration (ex. selecting all files of a given type, or searching through specified file types).
Previous versions of File Storage have not included folder permissions. (However, they did have a concept of private group trees.) The reasons for this were to simplify the code and the user experience. However, this system actually caused some confusion (e.g., explicitly granting permission to an outsider on a file in a group's private tree did not actually give that person access to the file) and was not as flexible as people desired. The ACS 4 version includes folder read, write and delete permissions.
Note that this can create some funny results. For example, a user might have write permission on a folder, but not on some of its parent folders. This can cause the select box provided for moving and copying files to look odd or misleading.
Previous versions of File Storage allowed only administrators to actually delete content (although users could mark content as "deleted" using a toggle in the data model, deleted_p.) However, the proper use of versioning should allow users to avoid accidentally losing their files. So, in this version, if a person asks to delete a version or a file, we really delete it.
Basing this system on the Content Repository provides a wealth of useful functionality for File Storage with no additional development costs. However, it may also constrain the system somewhat.
The Content Repository's datamodel has been extended to include an attribute to store the filesize. Unfortunately, the Content Repository does not automatically do this, since files may be stored on the filesystem (the Content Repository thus serving as a catalog to keep track of file location and some metadata, but not the filesize). The filesize is therefore calculated whenever a file is inserted in the Content Repository by the external program (the webserver's database driver) doing the insertion into the database..
The content_revision is subtyped as a "file-storage-item" to allow site-wide search to distinguish file storage objects in its search results. This feature is not implemented yet, however.
Permissions were chosen to make as much use as possible of the predefined privileges while keeping the connotative value of each privilege clear. The permissions scheme is vaguely modeled off Unix file permissions, with somewhat less overloading. In particular, we define a delete privilege rather than overloading the write permission. Also, execute privileges have no meaning in this context.
| Folder | File | Version | |
|---|---|---|---|
| read | view and enter folder | view file information | view and download version |
| write | add new files / folders | upload new versions | ----- |
| delete | delete folder | delete file | delete version |
| admin | modify permission grants and read, write and delete privileges | ||
Some notes: the admin privilege implies the read, write and delete privileges. It may be the case that a user has delete permission on a folder or file, but not on some of its child items. This will block attempts to delete the parent item. Finally, the write permission does not have any meaning for versions.
For the most part, File Storage will provide wrappers to the Content Repository APIs.
File Storage provides public PL/SQL APIs either as wrappers to the Content Repository API, or more involved functions that calls multiple Content Repository PL/SQP functions. One reason for doing this is to abstract from the Content Repository datamodel and naming conventions, due to the different way File Storage labels its objects.
The main objects of File Storage are "folders" and "files". A "folder" is analogous to a subdirectory in the Unix/Windows-world filesystem. Folder objects are stored as Content Repository folders, thus folders are stored "as is" in the Content Repository.
"Files", however, can cause some confusion when stored in the Content Repository. A "file" in File Storage consists of meta-data, and possibly multiple versions of the file's contents. The main meta-data of a "file" is its "title", which is stored in the Content Repository's "name" attribute of the cr_items table. The "title" of a file should be unique within a subdirectory, although a directory may contain a file and a folder with the same "title".
Each version of a file is stored as a revision in cr_revisions table of Content Repository. The Content Repository also allows some meta-data about a version to be stored in this table, and indeed File Storage uses attributes of the cr_revisions table are used. However, this is where the confusion is created. The name of the filename uploaded from the client's computer, as a version of the file, is stored in the "title" attribute of cr_revisions. Note that "title" is also used as the (unique within a folder) identifier of the file stored in cr_items. Thus, wrappers to the Content Repository API makes sure that the naming convention is correct: cr_items.name attribute stores the title of a file and all its versions, while the cr_revisions.title attribute stores the filename of the version uploaded into the Content Repository.
Meta-data about a version of a file stored in Content Repository are the size of the version (stored in cr_revisions.content_length) and version notes (stored in cr_revisions.description).
There are two internal PL/SQL functions that do not call the
Content Repository API, however: get_root_folder and
new_root_folder, defined in the
file_storage PL/SQL package
children_have_permission_pchildren_have_permission_p [ -user_id user_id ] item_idprivilege This procedure, given a content item and a privilege, checks to see if there are any children of the item on which the user does not have that privilege. |
fs_context_bar_listfs_context_bar_list [ -final final ] item_id Constructs the list to be fed to ad_context_bar appropriate for item_id. If -final is specified, that string will be the last item in the context bar. Otherwise, the name corresponding to item_id will be used. |
fs_file_downloaderfs_file_downloader connkey Sends the requested file to the user. Note that the path has the original file name, so the browser will have a sensible name if you save the file. Version downloads are supported by looking for the form variable version_id. We don't actually check that the version_id matches the path, we just serve it up. |
fs_file_pfs_file_p file_id Returns 1 if the file_id corresponds to a file in the file-storage system. Returns 0 otherwise. |
fs_folder_pfs_folder_p folder_id Returns 1 if the folder_id corresponds to a folder in the file-storage system. Returns 0 otherwise. |
fs_get_folder_namefs_get_folder_name folder_id Returns the name of a folder. |
fs_root_folderfs_root_folder [ -package_id package_id ] Returns the root folder for the file storage system. |
fs_version_pfs_version_p version_id Returns 1 if the version_id corresponds to a version in the file-storage system. Returns 0 otherwise. |
File Storage uses only the Content Repository data model. There
is one additional table, fs_root_folders, which maps
between package instances and the corresponding root folders in the
Content Repository.
Inserting a row into the table fs_root_folders occurs the first time the package instance is visited. The reason is that there is no facility in APM to insert a row in the database every time a package instance is created (technically, there is no "on insert" trigger imposed by APM on Content Repository, since they are separate packages even though they are both part of the core). The solution to this deficiency is a bit hack-ish, but seems to be the only solution available (unless APM allows trigger functions to be registered, to be caled at package instance creation). Whenever the package instance is first visited, it calls a PL/SQL function that calculated the "root folder" of the File Storage. If this function detects that there is no "root folder" yet for this instance (as would be the case when the instance is first visited), it inserts the package id and a unique folder_id into the fs_root_folder table to serve as the root folder identifier. It also inserts meta-data information about this folder in cr_items table. Finally, it returns the newly created folder identifier as the root folder for this package instance. Subsequent visits to the package instance will detect the root folder, and will then return the root folder identifier.
There is an "on delete cascade" constraint imposed on the package_id attribute of fs_root_folders. The reason for this is that whenever the package instance is deleted by the site administrator, it automatically deletes the mapping between APM and the Content Repository (i.e, the package identifier and the root folder identified), and presumably the particular instance of File Storage. Unfortunately this has an undesirable effect. There is no corresponding "on delete cascade" on the Content Repository objects so that deleting the root folder will cause deletion of everything under the root folder. Left on its own, the "on delete cascade" on the package identifier attribute of fs_root_folders will cause all objects belonging to the instance of File Storage deleted to be orphaned in the database, since the root folder is the crucial link from which all content is referenced!
The solution is (hopefully) more elegant: an "before on delete" trigger that first cleans up all contents under the root folder identifier before the root folder identifier is deleted by APM. This trigger walks through all the contents of the instance of File Storage, and starts deleting from the "leaves" or end nodes of the file tree up to the root folder. Later improvements in Content Repository will allow archiving of the contents instead of actually deleting them from the database.
The user interface attempts to replicate the file system metaphors familiar to most computer users, with folders containing files. Adding files and folders are hyperlinked options, and a web form is used to handle the search function. Files and folders are presented with size, type, and modification date, alongside hyperlinks to the appropriate actions for a given file. Admin functions will be presented alongside the normal user action when appropriate.
There are two configuration parameters in this version of File Storage. The first parameter MaximumFileSize is the maximum size of uploaded files, which should be self-explanatory. The other parameter is a flag that indicates to the package whether files are stored in the database or in the webserver's filesystem. This second parameter StoreFilesInDatabaseP uses the new capability in Content Repository to use the Content Repository as a mere catalog to store file information while the actual file contents are stored in the webserver's filesystem. Note that when files are stored in the filesystem, backups of the database will only store the catalog, but not the contents. Thus, it is important for the site administrator to store the entire directory containing the Content Repository files (in particular, pageroot/content-repository-content-files) when storing files in the fiesystem.
When a file is stored in the Content Repository, it first queries the parameter StoreFilesInDatabaseP to determine how the new file will be stored. Thus, it is important that this parameter should be changed only at package instance creation, or before any operation that uploads a file into Content Repository. Otherwise, the package instance will have files of different storage types, depending on the value of the parameter at the time the file is uploaded. Although all functionality provided by File Storage will continue to work (copy, move, delete, etc.), backing up the contents will be more complicated if the parameter is changed.
All of the other parameters in previous versions have been made obsolete by ACS 4 features like site-nodes and templating.
cr_mime_types
if they aren't there already. However, we don't currently
have a way of entering the description at the same time, so we have
to display "application/msword" instead of "MS Word
Document", for example. We could use a method of determining
the canonical long form of a MIME type.3.x : David Hill and Aurelius Prochazka
4.x : Kevin Scaldeferri
Kevin Scaldeferri
Kevin Scaldeferri
| Document Revision # | Action Taken, Notes | When? | By Whom? |
|---|---|---|---|
| 0.1 | Creation | 11/6/2000 | Kevin Scaldeferri |
| 0.2 | Revisions and Additions after Implementation | 11/15/2000 | Kevin Scaldeferri |
| 0.2 | Revised after review by Josh | 11/16/2000 | Kevin Scaldeferri, Josh Finkler |