The REBOL IOS Distributed Filesystem

Like the REBOL scripting language, REBOL IOS was developed by Rebol Technologies (http://www.rebol.com/). (For more information, see "The REBOL Scripting Language," by Carl Sassenrath, DDJ, July 2000.) Currently, there are three versions of REBOL IOS:

• IOS Express works by tunneling encrypted information through your existing web server. The tunneling method lets administrators run the system without needing to change network configuration and firewalls.
• IOS Lite can be used on a LAN and does not require a web server. Access is limited to clients within the LAN.

• Hosted services. Rebol Technologies provides hosting services for organizations that do not have a web server (or don't want to maintain one) and need external access to their IOS system.

Evaluation copies of REBOL IOS Express client and user accounts on the Express server are available at http://www.rebol.com/.

The REBOL IOS Filesystem

The REBOL IOS distributed filesystem was designed to address problems such as:

• How do clients and servers on disparate platforms share a common filesystem?
• How does the filesystem remain synchronized across all clients?

• How easy is it to set permissions, sharing, and access privileges?

• Can applications and data be packaged and distributed across the collaborative network?

Actually, the term "filesystem" is not precisely correct since servers and clients both use their native filesystems. IOS builds on top of those filesystems with what REBOL refers to as "filesets" — collections of files and metadata describing those files. The IOS filesystem architecture is a collection of these filesets, defined as standard REBOL blocks. REBOL uses those sets to keep clients and servers in sync.

According to REBOL CTO Carl Sassenrath, "The fileset design was oriented toward allowing automatic distribution of a set of files, along with a set of tags for those files. The tags include a wide range (unlimited) of attributes. Its history and objectives were not at all traditional in the filesystem sense, and the filesystem's distributed representation is in REBOL, so any machine with REBOL can deal with it."

Dissecting a Fileset

The fileset is the descriptive mechanism REBOL IOS uses for its own distribution and synchronization purposes, without conflicting with the host filesystem. Because the fileset system is described by REBOL itself, it is free of platform-specific filesystem issues.

Filesets are human readable and simple to set up and change. The fileset in Listing One starts with a name (used to identify it to the system), a block of files (to be associated with the set), and a block of descriptive tags (the metadata). The structure consists of the name and two REBOL blocks, contained by square brackets ([]).

The fileset name must be a valid REBOL word, as defined by the language. Names cannot include spaces, REBOL delimiters, or start with a number. For example, the fileset name for an interactive REBOL language dictionary application on the REBOL IOS desktop that comes with the standard distribution is "dictionary." The files block for the dictionary fileset contains a list of the files to be included. All file paths are designated relative to the root directory of the REBOL/Link client. So, for example, the root directory by default might be C:\rebol\link\developer\ on a Windows installation of the REBOL/Link client that links to a server called "developer." Instead of coding this absolute path, the filelist block keys off that folder as its root and indicates the path from the root — in this case, the apps/dictionary folder. In fact, you are forbidden from pointing to any files above the root directory.

%apps/dictionary/dictionary.r

%apps/dictionary/dict-html.r

%apps/dictionary/install.r

%apps/dictionary/words.r

%apps/dictionary/comments.r

There are no restrictions on the types of files included. The block can be composed of application programs, data files, or even nonREBOL files. Packaging applications along with data is straightforward because applications are synchronized the same as files — shared applications are always kept up-to-date for all users.

There are two sync-friendly ways to store data created by applications:

• The application stores data in separate files and each file is synchronized.
• The application stores all data in one file as distinct records (separate REBOL blocks). For efficiency, two files can be used: A master data file and a delta file that overlays the data file. The master/delta approach decreases the time it takes for synchronizations to download to the client.

Metadata Tags

Metadata provides information — access privileges, properties, icons, and folders — about the files included in the set. The tags are key to the synchronization features of REBOL IOS. The server looks at the tags to determine which users should receive the fileset, who has permission to modify its parts, and so on. Clients use the tags to determine display parameters, or where and how files should be displayed on the desktop.

Tags are structured as simple name/value pairs. You take the name of the tag and set it to a value. You can also use nested name/value pairs inside other values via REBOL blocks. Listing Two is a tag-list block for the dictionary fileset that shows three metadata tags: install-date, access, and icons. Values for the access and icons tags are composed of nested blocks of values. Posting privileges for the fileset have been assigned to all users. You can assign a cascading series of privileges (properties, rights, icons, server-code, delete, and change) to a block of users; in this case to ''admin" and "joe." Two icons have been defined: Dictionary and Dict-HTML. Each icon designation has an associated item (the file it points to), a folder (where the icon will sit on the REBOL/Link client desktop), and an info field (which displays when a user "mouses-over" the icon). Put together, the complete fileset for the dictionary application and data looks like Listing Three.

Each REBOL IOS client caches its filesets in a file called "registry.r." To examine how filesets are configured, you can load registry.r into a text editor. Administrators can edit the registry with a REBOL program called REG EDIT, included as part of IOS's admin tools.

Metadata tags, such as those in Table 1, provide flexibility. These tags are optional; if no ACCESS tag is set, access permissions default to the system administrator. If no USERS or GROUPS tag is set, all users receive the fileset. If users and groups are specified, only those users and groups "see" the fileset. Thus, a fileset can be public (shared by all users), private (seen by only one user), or shared (by a set of users or groups). Since custom tags are allowed, you are not confined by the standard tag names in Table 1.

Setting Access Permissions

Filesets exhibit two methods for controlling access to their files — synchronization and access. Synchronization determines whether users even know of the existence of a set of files. There could be dozens or hundreds of files and folders being shared that a user is not aware of, because that user has not been assigned to the workspace for the fileset (via the users tag).

Access privileges, which control who can modify the fileset itself, are established via the access tag within the metadata block for the fileset. It is composed of the tag and a block of name/value pairs for the different privileges you can set. A range of flexibility can be accomplished with it. For instance, the most important privilege in Listing Four is the rights permission because it gives users complete control over all the fileset's other rights. The other privileges designate who is empowered to control, change, or add/remove desktop tags and services, folder tags, icon tags, and property tags such as version, install-date, and priority. The change tag lets users add, remove, or modify the files within the fileset. The literal word 'all can be used to assign privileges to all users within a single stroke. If a privilege is assigned to an empty block ([]), by default only those with system-admin privilege have the power to modify it. Because these privileges are encoded in standard REBOL blocks, they are relatively easy to debug and modify, and confusion is reduced.

System administrators might typically want to setup access privileges for the fileset like this: One user owns the fileset, while all other users can modify its contents. (Note the use of cascading assignments in Listing Five.)

Programmatically Creating Filesets

Because filesets are represented in REBOL itself, programs written in REBOL can be used naturally to build filesets. Like all REBOL scripts, the program must begin with the standard script header. To include the correct environment for the IOS Link client, both title and type fields must be added too.

REBOL [Title: "Sample-Set" Type: 'link-app]

Users must have the NEW-APP privilege to create a fileset. These permissions are set by the user administrator. NEW-APP, the command for creating a fileset, takes a block that specifies the name, tags, and files to be included in the set. The script (Listing Six) uses the built-in send-server function, which takes a command argument (new-app) and data argument (compose/deep followed by a block). The compose command sends the unevaluated block to the server; it evaluates only what is inside the parentheses. This preserves the block structure demanded by the fileset mechanism. You have set one metatag called "users" to the username of the person launching the script, as defined by the user-prefs system object. One file is designated, called "testfile.r", which is created when the script is executed.

Once you are comfortable with this simple command structure, you can extend it by adding code that launches REBOL's built-in file requester, which reads the file and uploads it to the server; see Listing Seven. The second split-path file line takes the file you requested, splits off the filename from the rest of the path, and pulls the filename into the word "name." The name is then combined with the new path "%examples/". You could extend this example to publish multiple files, read and publish web pages or e-mail, and so on.

To publish a public fileset viewable by all users, you would: Change the users tag to an access, and assign the access privileges accordingly. When no users tag is set, all users share the fileset by default.

You can share a fileset containing an application and many files in such a way that only an icon for the application is visible to the users, while the other files are hidden. This can be done through an installer script or custom script. Alternately, you can use the UpFile reblet (one of the admin tools) to upload files you don't want to show as icons.

DDJ

Listing One

fileset-name
[
   metadata-tags
]
[
  list-of-files
]

Back to Article

Listing Two

[
  install-date: 8-Dec-2001/1:19:12-8:00
  access: [
     post: 'all
     properties: rights: icons: serve-code: delete: change:
["admin" "joe"]
  ]
  icons: [
     [name: "Dictionary"
      item: %apps/dictionary/dictionary.r
      folder: %documents/
      info: "Interactive REBOL Dictionary"
     ]
     [name: "Dict-HTML"
      item: %apps/dictionary/dict-html.r
      folder: %documents/
         info: "Generate a printable REBOL Dictionary in HTML"
      ]
   ]
]

Back to Article

Listing Three

dictionary
[
   install-date: 8-Dec-2001/1:19:12-8:00
   access: [post: 'all properties: rights: icons: serve-code:
                                   delete: change: ["admin" "joe"]]
   icons: [[name: "Dictionary" item:
%apps/dictionary/dictionary.r folder:
          %documents/ info: "Interactive REBOL Dictionary"]
          [name: "Dict-HTML" item:
%apps/dictionary/dict-html.r folder:
          %documents/ info: "Generate a printable REBOL Dictionary in HTML"]]
][
    %apps/dictionary/dictionary.r
    %apps/dictionary/dict-html.r
    %apps/dictionary/install.r
    %apps/dictionary/words.r
    %apps/dictionary/comments.r
]

Back to Article

Listing Four

access: [
   rights: ["joe" "admin" "sally"]
   desktop: ["joe"]
   folders: []
   icons: []
   properties: []
   change: []
   delete: []
   post: []
   server-code: []
]

Back to Article

Listing Five

access: [
  rights: properties: delete: ["joe"]
  folders: icons: change: 'all
]

Back to Article

Listing Six

send-server new-app compose/deep [
  sample-set

  [
     users: [(user-prefs/name)]
  ][
     [%examples/testfile.r (compress "One fine day.")]
  ]
]

Back to Article

Listing Seven

file: request-file/only
if none? file [quit]

name: second split-path file
insert name %examples/

data: compress read/binary file

send-server new-app compose/deep [
   sample-set
   [
     users: [(user-prefs/name)]
   ][
    [(name) (data)]
   ]
]

Back to Article