Wednesday, July 7, 2010

I've started contributing to the Android Cookbook, a brand new project of O'Reilly which is being run by my friend, Ian Darwin. Since you retain ownership of the material that you submit and simply license it to O'Reilly under the Creative Commons cc-by license, you have every reason to contribute something you may have already written–and no excuse not to!

Here is the first recipe I submitted:

Problem

Android's embedded SQLite3 database supports date and time data directly, including some useful date and time arithmetic. However, getting these dates out of the database is troublesome: there is no Cursor.getDate() in the Android API.

Solution

Use SQLite's strftime() function to convert between SQLite timestamp format and the Java API's "milliseconds since the epoch" representation.

Discussion

This recipe demonstrates the advantages of using SQLite timestamps over storing raw milliseconds values in your database, and shows how to retrieve those timestamps from your database as java.util.Date objects.

Background

The usual representation for an absolute timestamp in Unix is time_t, which historically was just an alias for a 32-bit integer. This integer represented the date as the number of seconds elapsed since UTC 00:00 on January 1, 1970 (the Unix time epoch.) On systems where time_t is still a 32-bit integer, the clock will roll over partway through the year 2038.

Java adopted a similar convention, but with a few twists. The epoch remains the same, but the count is always stored in a 64-bit signed integer (the native Java long type) and the units are milliseconds rather than seconds. This method of timekeeping will not roll over for another 292 million years.

Android example code that deals with persisting dates and times tends to simply store and retrieve the raw milliseconds since the epoch values in the database. However, by doing this, it misses out on some useful features built in to SQLite.

The Advantages

There are several advantages to storing proper SQLite timestamps in your data: you can default timestamp columns to the current time using no Java code at all; you can perform calendar-sensitive arithmetic such as selecting the first day of a week or month, or adding a week to the value stored in the database; and you can extract just the date or time components and return those from your data provider.

All of these code-saving advantages come with two added bonuses: first, your data provider's API can stick to the Android convention of passing timestamps around as long values; second, all of this date manipulation is done in the natively-compiled SQLite code, so the manipulations don't incur the garbage collection overhead of creating multiple java.util.Date or java.util.Calendar objects.

The Code

Without further ado, here's how to do it.

First, create a table that defines a column of type timestamp.
CREATE TABLE current_list (
item_id INTEGER NOT NULL,
added_on TIMESTAMP NOT NULL DEFAULT current_timestamp,
added_by VARCHAR(50) NOT NULL,
quantity INTEGER NOT NULL,
units VARCHAR(50) NOT NULL,
CONSTRAINT current_list_pk PRIMARY KEY (item_id)
);
Note the default value for the added_on column. Whenever you insert a row into this table, SQLite will automatically fill in the current time (accurate to the second) for the new record.
sqlite> insert into current_list (item_id, added_by, quantity, units)
...> values (1, 'fuerth', 1, 'EA');
sqlite> select * from current_list where item_id = 1;
1|2010-05-14 23:10:26|fuerth|1|EA
sqlite>
See how the current date was inserted automatically? This is one of the advantages you get from working with SQLite timestamps.

How about the other advantages?

Select just the date part, forcing the time back to midnight:
sqlite> select item_id, date(added_on,'start of day')
...> from current_list where item_id = 1;
1|2010-05-14
sqlite>
Or adjust the date to the Monday of the following week:
sqlite> select item_id, date(added_on,'weekday 1')
...> from current_list where item_id = 1;
1|2010-05-17
sqlite>
Or the Monday before:
sqlite> select item_id, date(added_on,'weekday 1','-7 days')
...> from current_list where item_id = 1;
1|2010-05-10
sqlite>
These examples are just the tip of the iceberg. You can do a lot of useful things with your timestamps once SQLite recognizes them as such.

Last, but not least, you must be wondering how to get these dates back into your Java code. The trick is to press another of SQLite's date functions into service—this time strftime(). Here is a Java method that fetches a row from the current_list table we've been working with:
 Cursor cursor = database.rawQuery(
"SELECT item_id AS _id," +
" (strftime('%s', added_on) * 1000) AS added_on," +
" added_by, quantity, units" +
" FROM current_list", new String[0]);
long millis = cursor.getLong(cursor.getColumnIndexOrThrow("added_on"));
Date addedOn = new Date(millis);
That's it: using strftime's %s format, you can select timestamps directly into your Cursor as Java milliseconds since the epoch values. Client code will be none the wiser, except that your content provider will be able to do date manipulations for free that would take significant amounts of Java code and extra object allocations.

See Also

SQLite's documentation for its date and time functions
This recipe in the Android Cookbook

Sunday, February 28, 2010

Guaranteed Definite Assignment, Bugs, and You

Here is a great technique to adopt when you must initialize a variable to one of two or more values. It's a simple technique that's easy to apply even when you're feeling lazy, but I'm surprised at how few people are aware of the advantages.

The scenario is simple enough: while revisiting a little parser I wrote a few months ago which turns strings such as "2010-02.1m" into corresponding typesafe immutable objects with a start date and end date, a small addition I made to it had the fortunate side effect of revealing a boneheaded bug I had baked into the original version.

Quick–can you spot the bug in this code, which compiles with no warnings and no errors?

Calendar start = (Calendar) end.clone();
if (periodType.equals("d")) {
start.add(Calendar.DAY_OF_MONTH, 1 - periodScalar);

} else if (periodType.equals("m")) {
start.set(Calendar.DAY_OF_MONTH, 1);
start.add(Calendar.MONTH, 1 - periodScalar);

} else if (periodType.equals("y")) {
start.add(Calendar.DAY_OF_MONTH, 1);
start.add(Calendar.YEAR, -periodScalar);
}

return new FilingPeriod(
code,
start.getTime(),
end.getTime());

It's missing the else part that throws an exception for unknown period types. Did you see it, or were you too distracted by the calendar manipulation to notice the obvious problem?

This code fragment comes from an accounting-related application, and sooner or later someone is going to have to add "q" for Quarter. If you gave the above code a periodType of "q," it would happily return an incorrect result rather than throw an exception. Besides, we're essentially building a parser for a little language here. It ought to throw an informative exception when you give it invalid input.

It was easy to justify the need for the catch-all else clause here, but truthfully, it's simply part of the idiom, and it's a waste of time to contemplate whether the catch-all is likely to actually do anything in practice. You should just get in the habit of always including it, and consider its omission in existing code to be an error. Unfortunately, even though it's such an important thing to remember, the compiler didn't issue any warnings or errors about its absence.

The good news is, your Java compiler can enforce the need for an else block that throws an exception (or returns normally, but your muscle memory for this idiom should have you typing "throw new …" before you've even thought about it.)

By the way, even if you did "immediately" spot the missing else, I bet it still took you more than .0001 seconds. It would have been more efficient, and safer, to let your Java compiler pick it up.


Let the compiler help you

So how do you enlist javac's help? Here's the change I made that gave javac the chance to throw my previous mistake in my face (sure, it sounds irritating, but it's actually a very very good thing:)

Calendar start = (Calendar) end.clone();
int monthsCovered;
if (periodType.equals("d")) {
start.add(Calendar.DAY_OF_MONTH, 1 - periodScalar);
monthsCovered = periodScalar / 30; // approximate

} else if (periodType.equals("m")) {
start.set(Calendar.DAY_OF_MONTH, 1);
start.add(Calendar.MONTH, 1 - periodScalar);
monthsCovered = periodScalar;

} else if (periodType.equals("y")) {
start.add(Calendar.DAY_OF_MONTH, 1);
start.add(Calendar.YEAR, -periodScalar);
monthsCovered = periodScalar * 12;
}

return new FilingPeriod(
code,
start.getTime(),
end.getTime(),
monthsCovered);

Now, the code won't compile: referencing monthsCovered is now an error because it might not be initialized. If I had initialized monthsCovered to 0 (or any other value) then the compiler would not have found the missing else for me. This is the key of what I'm saying: don't initialize your local variables to some garbage value. Leave them blank, and the compiler will help you to write better code.


But you can do even better

In most cases such as this, you can gain even more safety by marking the uninitialized variable final. This causes the compiler to guarantee that the variable has been assigned exactly once before it is first referenced.

Marking your uninitialized local variables final makes it virtually impossible for a future maintainer[1] to accidentally introduce subtle bugs into your multi-path initialization code–and this is the code where bugs experience long incubation periods before they rear their ugly heads[2].

Calendar start = (Calendar) end.clone();
final int monthsCovered;
if (periodType.equals("d")) {
start.add(Calendar.DAY_OF_MONTH, 1 - periodScalar);
monthsCovered = periodScalar / 30; // XXX approximate

} else if (periodType.equals("m")) {
start.set(Calendar.DAY_OF_MONTH, 1);
start.add(Calendar.MONTH, 1 - periodScalar);
monthsCovered = periodScalar;

} else if (periodType.equals("y")) {
start.add(Calendar.DAY_OF_MONTH, 1);
start.add(Calendar.YEAR, -periodScalar);
monthsCovered = periodScalar * 12;

} else {
throw new ParseException(
"Unrecognized period type in period " +
"specifier: " + code);
}

return new FilingPeriod(
code,
start.getTime(),
end.getTime(),
monthsCovered);

This third revision is robust code that's relatively safe in the hands of future maintainers. In terms of word count, it's a small change from the first example, but it's code I'd be far more comfortable having inside any system I'm responsible for.


The next step

Is it possible to make this type of code even more robust in the face of future modification? Yes! The typesafe enum pattern, introduced by Josh Bloch, is undoubtedly safer than what I've recommended above. It has even become a language feature in Java 5, which eliminates much of the boilerplate in simple applications of the pattern. However, unlike leaving your variables uninitialized and marking them final, applying the typesafe enum pattern to this type of problem is a big tradeoff in terms of lines of code you will write to solve the same problem. Is the tradeoff worthwhile? In many cases, I think it is.

My next post will go through the process of applying a typesafe enum to this same problem, and I will give you my insights on when I would go the extra mile versus sticking with the more succinct solution you see above. Ultimately, though, it's up to you to decide for yourself on a case-by-case basis.

[1] And let's be honest–this hypothetical future maintainer will be you, six weeks from now.
[2] This will happen either in production, or, more likely, during a demo.

Saturday, August 8, 2009

T-Shirt Update

The t-shirts arrived safely in California. Joe was nice enough to send a picture of the 'posse wearing them:


The guys spoke briefly about receiving the shirts right at the end of Episode 269.

Hope you enjoy the shirts, guys! And thanks for continuing to produce such an enjoyable podcast.

Friday, June 26, 2009

My t-shirts

Joe Nuxoll of the Java Posse recently tweeted that he was a huge Commodore 64 fan, "back in the day." Well, I happened to be wearing a C64-themed t-shirt that I made a year back, so I tweeted a picture of it and offered to send him one.

But then, to optimize shipping costs, I figured I might as well package up a set of 4 shirts of my own design and outfit the whole Posse in (hopefully) unique geeky wit, in t-shirt form. And for that matter, why not publish my work so everyone can enjoy my sense of humor (or implore me not to quit my day job).

So here's the collection so far. I have lots more ideas, but this is a catalog of shirts I've actually made and worn (in public, no less!).

This was my contribution to a larger t-shirt design at work. I think it stands on its own. The code snippets are real things we found in the code base at work. :)

Yes, I was a huge Commodore 64 fan back in the day too.

It's true!

This one that gets people commenting pretty frequently. Reactions run the gamut from "don't get it even after explanation, which is fine 'cause it's not funny once you try to explain it," through to overly deep analysis ("aren't the letters themselves just symbols that replace words?")

This one probably gets more comments than any other. I've become somewhat of an expert at explaining what rot13 is to non-geeks. I'm happy to say several people have decoded it from across the room without my help.

Plus, it's the only t-shirt design I'm aware of that comes with a secret decoder key under the front panel!


Shirts for other people

Ok, so I haven't worn all the shirts I made. Here are some I've given away as gifts:

This one was a birthday present for my friend Chris, who seizes every opportunity to remind us in no uncertain terms that he doesn't like to eat mushrooms.

This is the only one that wasn't my idea. Someone at BSDCan told me he always wanted a shirt like this, so I made it for him. What does it mean? Ask Locutus of Borg.

My sister's boyfriend Tom enjoys boasting about the time they visited some friends and their new baby was quite content to be held by him, but kept crying every time my sister took a turn. I'm not sure if this shirt was intended as a jibe or just more fuel for the fire, but he wears it proudly.

This one was for my father-in-law, Billy. Yes, he's a bit of an IKEA fan and so are we.

Monday, June 8, 2009

PGCon 2009: The Missing Lightning Talk

The week before JavaOne, I traveled to Ottawa to attend PGCon, the PostgreSQL Conference. I had the opportunity to spend the weekend following the conference with my wife, Linda, and our 6-month-old son, Elliot. We enjoyed ourselves! Ottawa is a great place to be a tourist for the weekend.

As usual with PGCon, I learned a lot at the talks, and I met interesting like-minded people in the hallway and after hours at the social events.

The last session of the conference was a general session which consisted of a series of lightning talks. Some of the talks were funny, others were informative, and some skillfully combined those two qualities. Very late in the game (Friday morning), I decided I should give a lightning talk about Power*Architect, an open source project I have the privilege to maintain as part of my day job. I had spotted a Power*Architect screenshot on Thursday in Steve Singer's OpenStreetMap session. I got to thinking there were probably several people at the conference who could use an ER diagramming tool with forward and reverse engineering, so I got up early on Friday and put together a small set of slides for a sub-5-minute talk introducing the Architect.

I sent off the slides to Selena Deckelmann, who was in charge of that session, and got back an answer right away: the lightning talk schedule was already full! Selena invited me to blog about it instead, so here I am.

Without further ado, an introduction to Power*Architect in 5 minutes or less:




E-R Diagramming
What is Power*Architect? In a nutshell, it's a visual data modeling tool that lets you work with E-R diagrams that look like this:



Reverse Engineering
We have intentionally made Power*Architect easy to learn and easy to use. For example, reverse engineering an existing database complete with foreign keys, data types, default values, sequences and autoincrement columns, indexes, and remarks is a simple drag-and-drop (or copy-and-paste) operation. There are no knobs to turn. It just works like you expect!

Forward Engineering (DDL Generation)
Similarly, forward engineering your data model into a database is a simple affair. Simply tell the Architect you want a create script, and tell it which dialect of SQL to use. If you connect to the database where you want to create all those objects I mentioned above, the Architect will execute the script for you (you can also save it to a file if you prefer).

The reverse- and forward-engineering features together make the Architect a really simple tool for migrating an existing database from one platform to another--hopefully from other platforms to PostgreSQL, but that's ultimately your choice to make. :)

Compare Data Models
The Architect can also compare databases to each other (structure only—not data) and create alter scripts for you. You can compare database to database, database to project, or project to project. You can even compare two databases of different platforms. This is really handy if you happen to write a database-independent application.

Multiplatform - OS
Power*Architect works on all platforms where a full Java 5 runtime environment is available. We develop and test it on Mac OS X, Linux, and Windows. We also know it works great on Free/Net/OpenBSD and Solaris.

Multiplatform - DB
The Architect also works great across database platforms. As I already mentioned, you can even directly compare, for example, a MySQL database to a PostgreSQL schema. The Architect handles implementation details such as sequences vs. autoincrement columns automatically. The current list of fully supported databases is: DB2, HSQLDB, MySQL, Oracle, PostgreSQL, and SQL Server. Reverse engineering is possible on a larger number of platforms, but forward engineering to those platforms is unlikely to work.

Free, in the wider sense

Power*Architect is distributed under the GPLv3. We develop it completely in the open. There is no hidden backend source code repository or anything.

Some Numbers
From July 2007 to May 2009, we've had:
3 community contributors
8 releases*
1773 forum posts
160427 downloads

*the eighth release, 0.9.13, came out on the day I had planned to give this talk.

For more information
See the Architect's product page for the download link and more marketing.

We have two other free and useful database tools, too! One does data cleansing (de-duping, reformatting/augmenting, and address correction), and the other helps you to produce ad-hoc reports. See the SQLPower software page for details.