Midterm Review and Update

Thursday (7/10/08), I joined the developer's call to review the current status of my project. I got a lot of really good feedback from everyone and a new sense of direction for the project. The following is a basic list of some of the major points from the call and followup mailing discussions:

• Keep it simple and introduce complexity later if time allows. It's better to have a simple solution that actually works than a really feature rich solution that doesn't do anything.
• As we add more functionality to the ODA, just creating an initial data set is a lot to throw at the user. When a user initially creates the data set, we just want to provide the first page that allows for token selection and the rest of the pages should not be seen. Then, the user should be able to go to the other pages via the edit data set interface to further refine the query.
• The Modifier page is going to be redone. The top piece will basically allow a user to add an aggregate to the beginning (just FIRST and LAST for now). LAST will be the default. Then, the user will use the bottom half of the page to choose conditions to add to the query if desired.
• A new query format is needed to support all of these new additions. We're moving more towards a SQL looking query where the SELECT chooses the tokens and the FROM is the cohort. For token in the SELECT, there will first be an aggregate (LAST), then the token name in curly brackets, followed by an optional condition. After the "aggregate {token} condition", there are pipe delimiters to indicate how to split the token (date, location, etc.). Finally, the desired cohort is in the FROM clause. More will probably required later but this is the simple format for now.

A lot of other details behind adding aggregates and modifiers to tokens were discussed. Check this out which is the latest mock up of the interface. The modifier page (2) still needs more work to provide a better way to indicate how the conditions are applied.

One of the great breakthroughs this week was moving away from the Mock Logic Web Service to the "real" Logic Web Service. The main problem was the data I had from the sample data set had some concepts that were missing names (more details in the bug here). There is also another problem where the dynamic cohorts don't have an identifiable name so I've removed these types of cohorts from the Logic Web Service for now and just the static cohorts are available. Anyways, it's great to be using the ODA and getting back real lists of tokens and actual data instead of the hard coded values I was working with.

As far as coding this week, I added support on both the Logic Web Service and BIRT ODA side to add four columns to every token that is chosen:

1. Observation Time
2. Observation Location
3. Encounter Time
4. Encounter Type

So far, the user has no choice and all four of these columns are added no matter what is chosen through the interface (this will be worked on when adding the split selection page to the data set wizard). There's a major problem with this so it doesn't work quite like it should. Casting the Result object from the data query to an Obs object does not work so I can't actually get any of the times, location, or type. Right now, it's just returning the Result's date and the rest of the values are null. There's also a problem with datetime data types working (had to use Text for now).

I also fixed a nasty bug where the underlying class that holds the information regarding the token, filter, etc. would never be flushed. This problem isn't noticeable unless you create a brand new data set and notice that all your selections from the previous data set are selected. I added cleanup() methods to all the data set pages that destroy the shared InformationHolder and added logic to reload the InformationHolder when saving the page.

In the immediate future, I'll be working on all the refactoring required to use the new query format. Hopefully soon, I can also create a page for choosing how and which columns to split.