Assignment A06: BerkeleyDB code katas

The purpose of the following entry is to introduce the use of the Oracle Berkley DB via an exercise and describe how it works in combination with Restlet and Wicket.

The Berkeley DB library is a building block that provides the complex data management features found in enterprise class databases. These facilities include high throughput, low-latency reads, non-blocking writes, high concurrency, data scalability, in-memory caching, ACID transactions, automatic and catastrophic recovery when the application, system or hardware fails, high availability and replication in an application configurable package. Simply configure the library and use the particular features available to satisfy your particular application needs. Berkeley DB is a reliable solution with over 15 years of production use in products ranging from cell phones to e-commerce. The primary goal of Berkeley DB is to deliver fast, scalable and flexible data management services to your application while remaining transparent to your end-user.

Kata 1: Timestamp as secondary index (Completed: True, Time: ~ 1 hour)

Part I: Implementation
The example system we were introduced to by Professor Johnson provides a single key for retrieval of a Contact object. This "primary key" is the object's unique ID. For this Kata, we first implemented a "secondary key" alternative for Contact. The secondary key consists of a timestamp represented as a long value that can either be inputted directly, or generated dynamically.

Once the secondary index was implemented we had to extend the ContactClient system with two new commands:

get-timestamp: returns the Contact with the given timestamp, if such a Contact exists.
get-range: returns the set of Contacts with timestamps between two specified values.

I should have used the cursor function to iterate through the contacts currently stored in the database, but instead I used a naive for loop. This accomplishes the functionality by going through the range of timestamps specified by the user. This would be very bad, especially if there is significant differences between the times in which the timestamps are created.

For example:

If the user specifies 1 1000 for the range, but there are only 3 secondary keys between this range, the current method is inefficient. When I have time later, I would like to research more on the cursor function.

Part II: Testing
When one creates the timestamps for a contact dynamically upon creation, it can be problematic for JUnit testing. Creating two Contacts, one right after the other, can happen so fast that they will have the same time stamp. It is because of this that the implementor may wish to add the timestamp in just before it's entry into the database.

Kata 2: Wicket, REST, and BerkeleyDB: A match made in heaven (Completed: True, Time: ~ 6 hours)

For this example we created a client-server (i.e. web application) system for managing contacts. We provide a Wicket client front end using Restlet to communicate with BerkleyDB. The Wicket client consist of a single page, and allows the user to store a contact given its information. The functionality to retrieve a contact given its unique ID is also there. The Wicket client code essentially invokes an underlying Restlet-based module to send a request to a ContactDB server to put or get data.

This Kata was highly dependent on the fact that you already knew how to "get" and "put" contacts using Restlet's framework and were very comfortable dealing with Documents and DomRepresentations. I had to cut and paste a few methods from the previous Kata assignment to make everything work smoothly.

The use of sessions to store the data was a neat review and is important to have all transactions on the web application independent of each other no matter what user is on it.

It should be noted that I had to disable the checkstyle check in verify.build.xml as

Missing package-info.java file

is causing the verify to fail and I couldn't readily find a solution. I hope to figure this out for a later blog post.

Here is a link to my distribution

Assignment A05: Restlet Code Katas, Part II

The second part of the Restlet Code Katas involved learning how the service works with XML. This was a very interesting exercise in that I've never really worked with XML. It's nice to finally understand how information is dynamically made and stored in this format.

The Katas are as follows:

Kata 5: Contacts resource (~3 hours)

For this particular Kata, we we modified the Restlet system to support a new resource called "contacts" and its associated URL: http://<host>/contactservice/contacts.  This resource only supports the GET operation and when it is invoked, it returns an XML representation of all of the contacts in the system.  To support the RESTful style of hypertextual links, the XML had to look like the following:

<contacts>

  <contact>http://host/contactservice/contact/ID</contact>

  <contact>http://host/contactservice/contact/ID</contact>

</contacts>

The actual <host> and actual <ID> for all <contact> elements are substituted and returned in the XML.
This requires the system to programmatically determine the URL of the current host.  We accomplished this by use of this type of call inside of the ServerResource method:

getReference().getHostDomain() ... etc.

We also needed to extend the createInboundRoot() method to support the new resource, and add a new package, class, and associated unit test. Overall, there was quite a bit of code to be added. One had to understand the use of objects:

• DomRepresentations (org.restlet.ext.xml)
• Document (org.w3c.dom)
• Element (org.w3c.dom)

to implement the changes required to access the resource "contacts" and have it display the information "put" into memory storage. I did not know that chrome and safari automatically translated XML when viewed. I wasted much time on this trying to figure out why my pages weren't showing up.

Kata 6: Command line client manipulation of Contacts resource (~3 hours)

After we implemented the contacts resource, we were required to extend the command line client to support a "get-all" and "delete-all" operation.

The get-all operation had to invoke the GET contacts command to get the XML representation containing links to all the contacts, then extract the URL of each contact and perform a GET to retrieve its representation. Then print out the contact info associated with that URL (I.e. first name, last name, info).  The main changes had to be done in ContactClient.java.

The next step, which used the get-all operation, was delete-all operation. This operation had to use the GET contacts command to get the XML representation containing links to all of the contacts, then extract the URLs, then call DELETE to delete them. Sample output can be seen below.

I had a few odd thread errors during this kata (i.e. concurrentHashMap was having issues). I'm still not exactly sure how I fixed it, honestly. I ended up having to wipe my set up at least once and rebuild. The (only) good part about having to redo some work is that it reinforced quite a bit of knowledge imparted to us by Professor Johnson.

Kata 7: Add a telephone number to the Contact resource (~1.5 hours)

The final kata required the extension of the Contact resource to include a telephone number.  The telephone number must contain only numbers and the "-" character.  The following regular expression validates that the user has inputted a number-number format.

Pattern p = Pattern.compile("^\\d+(-\\d+)+");

Thus, when clients do a PUT of a Contact representation in XML format, the server checks that the telephone number field has the correct format and set the Status field to indicate an error if the telephone number is incorrectly formatted. Both unit tests to check valid input and invalid input are provided in my distribution.

A link to my distribution file can be found here

Assignment A04: Restlet Code Katas

Code katas are exercises in programming that help you to refine your skills. In this assignment, we  actively worked with an example REST architecture: the restlet-dateservice.

The RESTLET tutorial proved to be the initial step to solving most of these katas:

Kata 1: Time resources (~1 hour)
For this Kata, I was required to add three new resources called "hour", "minute" and "second" that return the current hour, minute and second.  I then wrote JUnit tests for each of these resources and made sure they passed. The tests simply required that I make simple cases that checked to see if the data returned by the server matched the curent java.util.Calendar date. Since all of these sample tests passed in flying colors and I deleted the test cases as they weren't worth having. My test case for Kata 4 is very similar to these.

Kata 2: Logging (~3 hours)
This particular Kata required that I figure out a way to enable a logging feature for Restlet.
By default, Restlet logs data about each request to the command line using the Java Logging API.  I had to read a few tutorials to get familiar with the API. The trickiest part, I found, was deciphering what exactly I had to put into the logging properties file and what all the commands entailed. This step took a little trial and error while I defined a ~/.dateservice/logging.properties file that enables the user to specify (among other things) that the logging information should go to a file, not the command line.
tell the restlet-dateservice application to read and use the logging properties file in ~/.dateservice/logging.properties

These are the resources I looked for inspiration to solve this particular Kata.
http://www.naviquan.com/blog/restlet-cookbook-log
-This link hand the most concise break down of the appropriate steps to achieve the required functionality.
http://download.oracle.com/javase/6/docs/technotes/guides/logging/
http://onjava.com/pub/a/onjava/2002/06/19/log.html

Kata 3: Authentication (~1 hours)
The current restlet-dateservice system, given to us by Professor Johnson, performs no authentication.  This again required that I vist the user guide (and/or google) for guidance. From here, implementing the HTTP Basic authentication was pretty straight forward. Since all resources stem from the server resource, they all should naturally be should be available due to the fact that it only required that I modify DateServer.java. Thus, HTTP Basic authentication is enabled in my distribution. and passes the username "guest" and password "pw". These were hardcoded to work in the form:

http://guest:pw@localhost:8111/dateserver/...

Kata 4: Wicket (~3 hours)

This was the most tricky kata. This task required that we create a new client for the DateService server that is a web application using Wicket and not a default command line application.  The web application brings up a single page with a single form that requests what aspect of the date (year, month, day, hour, minute second) the user desires. When the user submits the form, the web application uses Restlet to obtain the appropriate information from the DateService server, then presents the results to the user in the page.

It required that one really understand how the restlet http get protocol works in regards to server challenges. If it were not for the authentication feature added in the previous kata, the task would have been extremely straight forward. However, I noticed I was getting authentication errors (401)
The following code describes what must be modified to get the proper retrieval of resources in which a server issues challenges for authentication:

    Reference ref = new Reference("http://guest:pw@localhost:8111/dateserver/" + props.get("Unit"));
    string[] userinfo = ref.getUserInfo().split(":"); // "username:password"
    string username = userinfo[0];
    string password = userinfo[1];
    ClientResource clientRes = new ClientResource(ref);
    clientRes.setChallengeResponse(ChallengeScheme.HTTP_BASIC, username, password);
  
    try {
      add(new Label("ValueLabel", clientRes.get().getText()));
    }
    catch (ResourceException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
    }

A test case similar to those required in the first kata is provided in the distribution. It simply tests the submission and retrieval of the dateservice/hour.

A link to my distribution file can be found here.