Conrad Vermeulen

This last week saw the release of KnowledgeTree 3.5.2. From a functional perspective, we’ve had some great developments regarding search that I’m wanting to comment on.

KnowledgeTree now includes a search grammar/language, allowing users to construct search queries on almost any field relating to a document. The new search engine provides great flexibility and leverages the power of the database to query document metadata, and the new full text search system based on the Apache Lucene project to query document content.

In previous versions of KnowledgeTree, search functionality has been accessible via a ‘quick search’ dashlet on the dashboard and in a little portlet when browsing through the repository. This has been replaced with an easy to access search widget on the top of each page:

Entering a value in the search widget searches through the metadata, document id, document text, discussion text, filename and full path fields.

The advanced search query builder has been modeled similarly to previous versions. The advanced search has improved grouping to allow improve the user experience in selecting search fields.

Search results are returned similar to search engine results, ranked by relevance:

All the bulk actions available when browsing are available via the search results.

Improvements to search also include searching based on full path - allowing you to create search query based on a specific folder and it’s child folders. Further, you can now also perform wild card searches. Searching terms such as ‘th*’ will return documents containing ‘thread’, ‘threat’, etc. Wildcards can be incorporated between letters, such as ‘t*t’ meaning to match words starting and ending with ‘t’. A question mark (?) may be used as a single character wildcard.

There should also be a noticeable improvement to performance when adding a document as the indexing process is now done as a background task. A number of administrative pages are provided to help administrators diagnose problems with indexing.

Lastly, the search using the grammar syntax is supported via web services - allowing third party applications to locate documents in KnowledgeTree more easily.

In time we’ll be able to provide considerably more improvements, with many other features such as records management utilising some of search capabilities. Further extensions in the ’search pipeline’ include word proximity matching and others made available via Apache’s Lucene full text search.

Bjarte Kalstveit Vebjørnsen has contributed a port of the KTWSAPI library to Borland Delphi. The KTWSAPI is an object model based on the functions exposed via the KnowledgeTree SOAP Webservice.

Bjarte has created a port of the PHP KTWSAPI object model to Delphi 2006 using components from the Indy Project. His contribution can be found in the KnowledgeTree installation in the folder called ktapi/delphi.

Although Delphi developers could easily access the webservice by creating SOAP proxy object using the WSDL - by having a Delphi port allows developers to work with KnowledgeTree documents and folders as object orientated constructs, making the development process far richer.

3.4 STABLE is about to be released and includes the KnowledgeTree webservice. The webservice exposes key functions that exist within KnowledgeTree via SOAP so that third parties can easily integrate their applications and KnowledgeTree.

The webservice is based on the ktapi object model, which wraps the internal functionality into a layer that simplifies development.

Key functionality currently exposed includes:

• add document
• checkin
• checkout
• update metadata
• start workflow
• change workflow state

In upcoming releases, the webservice functionality will be extended to include include search functionality.

Download the KnowledgeTree Visual Basic Webservice Client Demo, a sample Visual Basic .NET application created in Visual Studio 2005. In a few lines of code, I demonstrate how to authenticate, get folder contents, and add a document. More information is available via the documentation site.

The KnowledgeTree developer documentation is slightly lacking, and we are undertaking an exercise to improve this situation. Over the next few weeks, we will be trying to create tutorials to cover key areas that KnowledgeTree developers need to know about.

Recently, I have written two tutorials:

• How to write a dashboard dashlet
• How to write an authentication provider

The latest stable release of KnowledgeTree has been released. This release includes minor bug fixes and a security issue.

The security issue is related to installations authenticating with Active Directory. It is advised that users of the Active Directory authentication upgrade as soon as possible.

The latest development release of KnowledgeTree, 3.4.5 DEV, now exposes a SOAP web service. This is opens up the platform so that 3rd parties can easily integrate their applications into the document repository.

Documentation can be found at http://docs.knowledgetree.com.

A quick start guide is provided with a quick overview of the functionality available. As part of the development we have encapsulated the core KnowledgeTree functionality into an object model making it possible to separating much of the core logic away for the user interface, making the web service possible. This object model is found in the directory source/ktapi folder.

Furthermore, an object model was developed for the web service layer, closely resembling the structure provided by the core object model. With the help of the community, we hope to port the web service object model to different languages. The key concepts behind the web service object model in the long term is to provide an infrastructure that logically resembles the object model on the KnowledgeTree core, but will provide caching capabilities and better validation mechanisms so that server side processing can be minimised.

Currently, a PHP version of the web service object model is provided. Development on a C# .NET version is underway and will be available shortly. The web service object model for various languages is found in the source/ktwsapi folder.

Example of adding a document to the repository

The following sample illustrates the process of adding a document to the Knowledge Tree repository via the PHP web service object model.

require_once('../ktwsapi.inc.php');

$ktapi = new KTWSAPI('http://ktdms/ktwebservice/?wsdl’);

$ktapi->start_session(’admin’,'admin’);

// get a reference to the folder we want to work on
$root = $ktapi->get_root_folder();

// adds a document to the root folder
$root->add_document('c:/temp/test.doc');

$ktapi->logout();

The upgrade utility has been improved to provide a database backup and restore functionality.

Before attempting any upgrade, it is always best to make a backup. The upgrade utility provides a very easy ‘click-through’ backup procedure, and the restore process is just as easy.

I’ve just been working on two little prototype admin plugins for KnowledgeTree Administrators.
The first one is something to monitor disk space, and the other is to assist in cleaning up temporary files.

Running out of disk space can cause many serious headaches. Although system administrators should be doing regular maintenance and monitoring system resource utilisation - sometimes obvious things are simply overlooked.

These two plugins should be in the next release.

KnowledgeTree is quite hungry of disk space. For each document that you may have in the KnowledgeTree repository, a copy of each change is also preserved. If you are working with 50 megabyte files, make 5 changes, that implies KnowledgeTree needs to 250 megabytes of storage for that one document. Then you also need to take into consideration the index data needs to be taken into consideration, being the searchable text, which is generally to be less than the latest file size - which would mean around 300 megabytes is required.

This implementation may sound slightly crazy from a resource utilisation perspective - but it is the safest mechanism to ensure the document state can be restored.

Hopefully these utilities will save someone’s life one day.

I found that KnowledgeTree works out of the box on PHP 5.2.0 and 5.2.1 on Windows XP. I tried to use it on 5.1.6 a few weeks back and all I had was continuous headache with require_once() not working properly and classes being redeclared.

So what does this mean? Well, we will be migrating to PHP 5 slowly. It is one thing getting to use the Zend Engine 2 with an existing code base, but we need to take on many new features the language has to offer which can provide many benefits from the perspective of performance and reliability.

In the next few weeks we will be updating the stack installer so that the newer deployments will include this.

I’ve added a script to the KnowledgeTree Forge to assist users in importing files and meta data from other systems.

It relies on meta data files based on a .ini configuration file format:

invoice and client are the names of field sets, and invoice_date, invoice_no and client_name are fields linked to the field sets.

The script allows you to import a folder structure from the file system, where sub folders are created in KnowledgeTree with the same name. The script allows for an option where the meta data must reside in a file with exactly the same name, but with an extra “.metadata” extension. e.g. somefile.doc must have a meta data file called somefile.doc.metadata.

Field sets and related fields will be created if they don’t already exist, so they don’t have to be created prior to the import.

There is also an option to select ‘copy’ or ‘move’ into the KnowledgeTree repository. I havn’t tested the ‘move’ functionality across drives under windows, so I’m not sure what to expect. Using the ‘move’ should reduce I/O time of transfer. However, if something goes wrong…

Remember to make a backup prior to doing anything!