Previous: Multithreaded Web Applications, Up: Design Patterns


6.6 Real-World Application Examples

This section contains a collection of case studies or overviews of read-world applications that have exploited Elephant.

6.6.1 Konsenti

Elephant is used by Konsenti(tm), a for-profit company of Robert L. Read, one of the maintainers of Elephant. It can be visited at http://konsenti.com.

Konsenti uses the Data Collection Management (DCM) package, found in the src/contrib/rread directory. DCM provides prevalence-style in-memory write-through caching. The most enjoyable feature about Elephant for this project is that new Business Layer objects can be created without having to deal with an Object-Relational Mapping, enabling extremely rapid development.

All Business objects are persisted via a director in DCM (which sits on top of Elephant.) Many of these business objects are in fact finite state machines decorated with functions. The functions are represented by lambda s-expressions stored in slots on the business objects. A complete Message Factory and double-entry accounting system are also implemented as DCM objects. Binary objects, such as uploaded PDFs, can be attached to objects as comments and are stored directly in Elephant. Konsenti is based on utf-8, and unicode characters outside of the ISO-8859-1 character set are routinely stored in Elephant. Konsenti uses Postgres as a backend for licensing reasons; but use of other data stores is possible.

6.6.2 Conceptminer

Conceptminer is an Elephant-based web-mining framework developed by Ian Eslick (http://www.media.mit.edu/~eslick) that performs large-scale text analysis over the web to identify semantic relationships such as “PartOf”, “DesireOf” and “EffectOf” between English phrases.

Elephant's persistence capability is used to keep full records of all source material, extracted relationships and search queries so that it is always possible to trace the source of a learned relation and to avoid repeated queries to web search engines. Conceptminer used Elephant 0.6.0 and the development branch of Elephant 0.9 to perform months of analysis consisting of millions of pages and a page/query database of over ten gigabytes.

There are several interesting uses and extensions of Elephant in Conceptminer:

The most interesting use of Elephant was extending its transactional architecture to cover in-memory lisp operations. PCOMP (Process Components) is a framework for constructing and managing simple, dataflow-style multi-threaded applications in Common Lisp. The goal is to simplify the process sufficiently so that the ordinary user can hide from many of the details associated with aborting transactions. To this end, the model provides for safe, asynchronous communications among a set of components which may be scheduled together in a single process or communicate across separate threads (and potentially processes). Components are packaged into a system inside a Container object which schedules execution and mediates communications.

Communications between components can be in a dataflow style or using messages. Each component has a single port for receiving incoming data items. These items, if access is shared among components, should have the proper synchronization protections on mutating accesses. There is also an asynchronous communications method allowing you to send messages to components with particular names.

The basic building block is a component. Components are defined using the defcomponent form and contain several major elements, such as:

     (defcomponent counter
       (:vars (count 0) end (increment 1))
       (:initialize (assert end))
       (:body
          (when (>= (incf count increment) end)
             (terminate))))

The arguments to defcomponent behave as follows:

The body and messages are evaluated in a very specific environment. Within the body certain variables and functions are bound:

Each component execution is bound in a transactional framework. No variables are written, messages consumed or messages sent until the body or control handler has exited normally. Users can tap into this transactional framework by overriding start-transaction, commit-transaction and abort-transaction methods for the component class. Transactional variables are implemented via :after methods on these generic functions.

When signals are asserted by the body or a message handler, they are also wrapped in restart handlers called: