Release 0.1

One unimaginably small step for mankind, but one giant leap for myself.
The first rendition of my plugin is out there!

Decide you must

It's inevitable, one day I have to get the plugin out in the open. The whole idea of a plugin is that your code could serve someone else in the community and hence, it should be out there for someone else to find it. But programming is an art as much as it is a science and there's no accounting for taste.
While I do have experience with peer-code-reviews and I do have written numerous little code examples to explain concepts or teach courses, this plugin is something else. And I've been doing tailor-made software for a living, never before did I try to contribute to other tailors.

Pro's

In it's current state, it does have value to other developers wanting to use the Flickr API to get some images and / or users. So that would be a reason to do an "initial public offering" at this point in time. Furthermore, it generally is a good thing to release early and release often. This might give me some feedback about the approach or heaven forbid, my Groovy coding style / knowledge.
The code works for me, but nobody notices that warm feeling (like wetting your pants in a dark suit) and I'm curious to find out if more people are interested in using it, if the approach will work out in the end, if other people can use the exception handling, if they would need more methods or less, if ...

Con's

It's not finished.

No really, I promise. I want to implement way more of the Flickr API, specially the methods that need  authentication are missing badly in the current state.

Also, I'm not a 100% sure if the architecture will hold up once I start implementing more scenario's and methods. And what impact an internal architecture change will have on external users of the plugin.

And I should add regression tests for each and every function. And I ...

Permission you seek?

Finally last night I revisited the instructions in the Grails User Guide, wrote some documentation, read the Release Plugin User Guide and the permission procedure detailed by Yoda Graeme Rocher, bit the bullet and emailed "Seeking permission for initial publication of my Glickr plugin" to the Jedi Grails Developers.
Waiting anxiously for the Force feedback.

Exceptionally

Trying to come up with a strategy for handling errors and exceptions, we came across some interesting discussions and/or Holy Wars. I'll try to elaborate a bit on my thoughts and the strategy that was decided for the plugin.

Keeping under Wraps

First of all the purpose of the Glickr plugin (or any plugin the implements some interface I guess) is to make the translation of the native language constructs and classes (objects) to the particular API and vice versa.

Ideally, the user of the plugin should not be bothered with the specification or workings of the Flickr API at all. Furthermore, the user of the plugin will actually be a programmer creating an application that will or will not have a humanoid working with it. How this application deals with errors and exceptions should not be up to me (the developer of the API) but up to my user (the programmer of the application).
So I should keep as much under wraps as possible, but not too much. Still with me?

To make the rest of this post easier to read, let's agree on the 4 different actors layered on top of each other in our scenario;

• Human; i.e. the user of an application using the plugin. Might or might not be applicable in every scenario
• Programmer; i.e. a software developer implementing an application using the plugin (Programmer != Human, we all know that)
• Me; i.e. He that Has to Hide the API details
• API; i.e. the Flick API, which is beyond the control of the other three actors and reportedly constructed in a presumably nice and happening office somewhere downtown San Francisco

To Err is Human

According to a dictionary, a 'Recoverable Error' is "an error that does not prevent program execution" which is interpreted by me as "not meeting the business rules", which are in this case, some rules of the Flickr API, like "at least 1 result should match your search criteria".
Real world business rules of an application are beyond the scope of the plugin.

Example:
A Human or Programmer searches for "Obi-Wan Kenobi" in the Star Trek database. Result? Not found, but a recoverable error (search for another person or in another database). An Error but not an Exception.
Humans can make Errors, but it is up to the Programmer to decide how to interact with the Human about this unfortunate situation (if there is a Human involved at all, think good old-fashioned batch programs).
We still agree it's an Recoverable Error not an Exception, don't we?

Next Example:
The API could report back to Me that the "Service is temporary not available".
While this does give Me a hint that this error could be recoverable, it is very hard for Me to deal with with the underlying problem. The (help-desk of the) Programmer might figure out when Flickr finishes their backup of 7 billion images or if there was a hostile takeover by Buy N Large, but that's as close to recovery we could get.
The API will respond this Recoverable Error every now and then, but neither Me nor a Human could actually deal with the possible recovery (such as: when and/or how often to retry).

Exceptional Behaviour

We've dealt with the Human and the API so far, they both are allowed to make errors, and we can not blame 'em for it. What's left to deal with are errors that are introduced by one of the other two actors.
As the developer of the plugin I do not know who the Programmer will be, so I need to provide her/him with the information needed to blame Me for the error that should be solved in the plugin and will friendly inform her of errors that she can solve herself (syntax problems mostly, specially since the interface will probably not be 100% type-safe and will use plenty of late-binding).

To prevent error handling code from messin' up the undoubtedly beautiful code produced by the Programmer I will be throwing Groovy Exceptions where applicable. Here's what I decided to implement for the different types of errors we've just elaborated on;

• Not meeting the business rules, which does not prevent program execution and therefor won't produce a Groovy Exception, but will return a NULL rather than the suspected object
• Recoverable API situations will be throwing a FlickrServiceApiException
• Errors that could be solved by the Programmer will be throwing a FlickrServiceSyntaxException
• All other problems messing up program execution (but can only be solved by the developer of the plugin) will be throwing a FlickrServicePluginException

This should enable the programmer to write code that just checks for business logic (is there an object returned or a NULL value) and catch FlickrServiceException (all three flavours will inherit from it) at a more sensible place in the application. Last but not least, while the thrown Exception will include detailed technical information about the actual source of the problem, one should never, and I do mean NEVER, bother a normal human being with this cryptical programming stuff. Promised?

Git-it-is

To host the source-code for this little project I decided on Google Code, since I already had an account for Blogger etc. Since I had been using a lot of Subversion in the day-job, and SVN was supported by Google Code, this seemed like the obvious choice too. How ignorant I was ... after fiddling in vein for literally a couple of hours to get this to work to my taste in combination with the IntelliJ IDE I threw in the towel. Then I found out about this nifty (and free, I'm Dutch) SCM client for OSX called SourceTree and decided to give it a try, forget about SCM in the IDE for now and reconfigure Google Code to use a Git repository. Happiness returned ...

Test images

In need for a versatile set of test images for the development of the Glickr Plugin I decided to create a dedicated Flickr account and uploaded some professional imagery.
In 1992 Eastman Kodak Company launched their impressive system for digitising and encoding images on a CD-ROM. While the whole concept was quite impressive in those days, the PhotoCD was never a big success in the consumer market. Roughly 10 years later Kodak abandoned the whole concept and never published the encoding format, thus leaving all their customers in the dark.

Anyhow, in 1995 Kodak released a PhotoCD sampler that was delivered complimentary a CDi player (another stillborn technology) which was my introduction to high-end digital images. For this 6-megapixel images to become useful at the Intel-clone, a few more years of Moore's Law passed, but I did end up ordering piles of this PhotoCD's at the local laboratory in this pre-digicam era.
Kodak granted the right "to use, manipulate, store, transmit, reproduce and display" the images on their PhotoCD sampler, "for any purpose associated with electronic imaging". So after collecting dust for 17-years, I pulled out the PhotoCD, had to use ImageMagick to translate the proprietary PCD files, and uploaded the sampler set as test-images on the new Flickr account.

Trough of Disillusionment

In the first weeks after GR8Conf I enthusiastically started with the blog and development of the plugin. Sadly we had to deal with an event in real life that thoroughly consumed all our energy and enthusiasm.

While the last mumbling on this blog is already a couple of months away, today seems like a good day to drop some thoughts on the internets. On a positive note I could argue I'm perfectly compliant with the Gartner Hype Cycle, so nothing is lost while I commence the Slope of Enlightenment.

REST assured

With slightly over 200 interface methods, the Flickr API is quite an impressive beast to tame. What I aim to achieve (given I'll arrive somewhere mimicking the Plateau of Productivity), is a full implementation of the API as a Grails service, using native (and probably extensive) Groovy classes for the relevant entities, like Photos, People, Places, etcetera. The REST format seems to be the most sensible way to approach the API, so essentially I'll be implementing a Facade for the REST.request and corresponding XML response.

private def apiCall

After some tire kicking I started with an abstraction of what all API calls should do:

1. Validate the parameters before doing the call to Flickr
2. Well eh, do the actual call
3. Process the response or handle any errors

In Pseudo Groovy Code it looks something like this:

void apiCall(method, params) {
   if (validator(params)) {
      try {
         def rsp = doApiCall(method,params)
         return process(rsp)
      } catch (Exception ex) {
         ...
      }
   } else {
      // todo: raise validation exception
   }
}

Non disclosure

Now using the power of Groovy Closures I made the params, the validator and the processor variable for each different API method. Furthermore, to help in the maintenance of what will potentially be over 200 different sets of params, validators and processors, I decided to wrap the Closures in a Class. The whole concept will likely evolve over time, but generally the code looks like this:

//
//  abstracting the whole call handling with closures implemented in individual classes and connected below
//
private def apiCall(def apiImplementation, def apiModel = {} ) {
    def validator = apiImplementation.validatorClosure
    def params    = apiImplementation.paramsClosure
    def processor = apiImplementation.processorClosure

    if (validator(apiModel())) {
        try {
            def rsp = doApiCall(apiImplementation.apiMethod, params(apiModel()))
            return processor(rsp,apiModel())
        } catch (FlickrException ex) {
            FlickrExceptionHandler.handleApiCallException(ex)
        }
    }
    // todo: raise validation exception
    return apiModel()
}

Now the definition of a single Facade method looks like this:

FlickrPhoto photosGetInfo(FlickrPhoto photo) {
    return apiCall(
        new org.glickr.api.photos.photosGetInfo(),
        { photo }) as FlickrPhoto
}

And the actual implementation of the Flickr API method is done in this class:

class photosGetInfo implements FlickrApiMethod{
    static final String apiMethod = 'flickr.photos.getInfo'

    Closure validatorClosure = { FlickrPhoto photo ->
        if (!photo || photo?.id <= 0) { return false } // missing model
        return true
    }

    Closure paramsClosure = { FlickrPhoto photo ->
        [photo_id:photo?.id, secret:(photo?.secret ?:'')]
    }

    Closure processorClosure = { GPathResult response, FlickrPhoto photo ->
        photo.title       = response.photo.title.toString()
        photo.description = response.photo.description.toString()
        photo.isPublic    = (response.photo.visibility.@ispublic?.toString() == '1')
        // etcetera              
        
        return photo
    }
}

That'll be it for today. Any ideas, suggestions or blatant criticism will be greatly appreciated while I code my way up the Slope of Enlightenment towards an initial alfa release of the Glickr plugin.

GR8Conf 2012 friday report

The first session on the last day of GR8Conf was led by Ivo Houbrechts, was about his big scale Grails project (1000 users, 75 screens, 14 developers), and quickly got my attention. The problems they had to deal with where quite recognizable to our current project (even to past projects in other platform besides Grails). Funny enough this brought some sort of realization that "if we run in the same limitations, we are probably understanding what the capabilities of the platform are and aren't. That we are starting to get past the newbie stage" (ok, we might have to admit here that we did enjoy the specially brewed conference beer last night, so thinking might have been a bit foggy).

Ivo continued to show they had created two Grails plugins to implement some solutions for the problems they (and we) ran into. Thanks for that! Another two more items were added on our post-conference todo list;

• Discover the GSP taglib plugin that makes it possible to define tags in GSP syntax.
• Play with the Rich Domain plugin that adds functionality like validation support for POGO's and instance constraints and constraint-groups.

The next presentation by Netflix' Joe Sondow was easily the most polished one of the conference. They have opensourced their mission-critical Grails application, used by Netflix engineers to deploy code changes and manage resources in the Amazon cloud. Not something we have at hand on a day-to-day basis, but nice to store in background memory nevertheless.

During the remainder of the day we saw Burt Beckwith of SpringSource explaining the Spring Security Plugin and showing more Spring stuff under the Grails hood.
Every now and then we fantasize about using a NoSQL database (but never found a real need/advantage so far), so the MongoDB session got easily absorbed too.
Groovier testing with Spock brought what the title promised, and Robert Fletcher clearly showed the advantages of Spock by side-to-side comparism with plain tests.

But the last few days, and certainly this mornings' session had certainly planted a seed. Development of a Grails plugin should be the next step in my exploration of Grails and would be the main subject (besides beautiful Danish women) of our discussions during the travel back home. We arrived home a bit delayed (our plane was struck by lighting when inbound CPH), but full of new ideas, inspiration and enthusiasm for the Grails ecosystem.

See y'all next year.