Thorough architecture-- remember testability!

When architecting software, be sure to document which test utilities you anticipate will be needed to validate your application.

Think about each operational step that is required-- then think if you had just performed that step, how would you be assured everything went well?

As an example, let's pretend you've just architected an application that should help you load some data to a backend data store (maybe a database, maybe an indexed file system, maybe something else)

• How would you be sure the data you've loaded is really there?
• How would you be sure it's *all* there?
• How do you know it's available?
• How do you know it wasn't mangled in the process of loading it?
• Can you access it from all the places you need to access it?

If I were the operator charged with running these processes (loading the data), I'd want some sort of script I could run to make sure the data got where it was supposed to go. The time to think about this is in planning your architecture-- thinking of such things will lead you to consider the needs of your stakeholders and will give you insights into parts of your application you may have overlooked.

Continuous Integration for C++

I've used Continuous Integration with Java for quite a while-- usually using CruiseControl, JUnit, and PMD. I've had good luck with these running them against PVCS, CVS and Subversion. Lately, though, I've had an interest in using CI for C++ development, which was a whole new story.

This time around, I thought I'd try Hudson instead of CruiseControl. I was pleasantly surprised by the ease of use-- basically, just untar Hudson someplace and run the web application, which is a .jar. Once Hudson's running, you configure it through web forms. It's really easy!

I wanted to use Subversion this time around, it was trivial to enter the URL, user and password.

Next, we had to find a way to cause our build to happen. I already had a Make-style C project to work with, so I started out using Hudson's 'run a script' configuration to invoke Make. This worked great-- right out of the box Hudson will check your project out and invoke Make.

I wanted to use a JUnit-style unit test framework, and decided to go with CPPUnit based on internet reviews. I'd had an earlier failure trying to get CPPUnit to compile under Cygwin, but that was quite a while ago, so I got the most recent build and started building with 'configure, make, make install'. (By the way, I used the relocation option to give 'configure' a hint to build CPPUnit in a local directory. I didn't want to muddy up this shared Linux server until I was confident things were going to work out well.) CPPUnit built well and I had a sample program (based on a 'Net tutorial) running in a short while.

To run CPPUnit from Hudson, I altered my script to invoke the Unit test driver right after 'make clean' and 'make'. I also redirected the output from CPPUnit to a file and echoed the contents of the file at the end of the script. (In this way, you can see everything in the 'console output' view of your Hudson build. No need to reformat the XML output!)

Next up was civilized logging. This really wasn't a necessary part of CI for C++, but I've been so spoiled by Log4J that I had to have it. So I got Log4Cxx (which also required Apache's runtime 'apr' and 'apr-utils') then built all 3 in the order apr, apr-util, then Log4Cxx. All this was done with configure (with relocation), make, make install. Log4Cxx also worked right out of the box-- woohoo!

I wanted to see how well my unit tests were covering the code, so I added new CXXFLAGS to my Make file. I had the Hudson build script use sed to toggle the flags on (which enabled me to run gnu's gcov with the Unit test driver), then I got stats which naturally got redirected into the same file CPPUnit redirects to, and thus is echoed at the end of the Hudson build script. Now our console output for the build shows Unit tests and coverage.

As a final step, I wanted to gather some stats on how my code was performing, so I invoked Valgrind in the Hudson build script and redirected the output again to the tell-all echoed file. I ended up taking this one out in the end, because it got very wordy and made for a pretty large file to view in the scrollable browser Hudson gives you. Maybe I'll try to cut that down sometime and re-add it, but for now I'm just doing without.

So there we have it! I found the whole experience totally satisfying and well worth the minimal effort it took to get the whole toolset going. I hope you have as much luck in your efforts!

Rick

Stax Rox!

I just got done taking my first test drive of Stax, the Java Cloud provider. I haven't gone very far into it yet, but so far I've been *very* impressed with the development model. Here are the highlights:

• Just went beta, no cost for a dev account

• Easy to follow 5 minute videos to get you started writing your Cloud Java Web app

• Very nice local dev environment, with easy to use tools to slog code from your desktop to the cloud

• Trivial button-push creation of a sample Web App you can modify

• The sdk downloads your app to your desktop for modification, it provides a built-in Eclipse project that fires up with no modifications

• Trivial button-push to make a database (MySQL) for your application. (I haven't yet built my schema, I'm looking forward to watching the video for that.)

All things considered, this is the easiest Cloud I've had a look at. For Java devs, I can't imagine it getting much easier than this. (Although there are persistent rumors Google App Engine is bringing Java soon. Google also strongly understands the power of low-barrier-to-entry, so that'll be interesting.)

One downside: the User Agreement forbids several flavors of Open Source, notably the GNU stuff. For now, I'm sticking with the prepackaged stuff Stax gives you, so I shoudn't get in trouble that way.

If you've got 30 minutes to burn, you can have your own Cloud app up and running. The URL is here: http://www.stax.net/appconsole

Great Erlang podcast with Joe Armstrong

I've recently taken an interest in Erlang, as a way of preparing for massive muticore machines. So far, I've found the language interesting and somewhat easy to use. (This based on not much practice and some of the excellent tutorials available on the web.) Today, I'd like to share a top-notch resource for others in the same boat: an interview with Joe Armstrong on Software Engineering Radio.

For those who've never heard it before, SER is a series of podcasts on all sorts of interesting topics. The URL is here:

http://www.se-radio.net/

Joe's interview is #89. Happy listening!

Rick

Tech update-- New Tools in practice

Wow, it's been a while since I've written. That's because it's been busy, busy, busy working on that very data-intensive application I wrote about last spring.

Here's an update on some lessons learned:

ELT (Doing transformations in the database instead of marshalling data in and out) is working out well. It really does save lots of time and makes the whole process more reliable.

Message oriented middleware (through Apache's ActiveMQ) is working out nicely. I really like using the 'Competing Consumer' model to scale services horizontally. Just have your service nodes read from a single queue, add more services to add more horsepower!

Mule has allowed us to easily switch back and forth between desktop and distributed modes. This helps a lot in development. Note that we still have some sore spots-- our database isn't one that lends itself to 'one per developer', so we development and testing isn't as easy as we'd like in that respect.

Automated unit testing is great. There are some parts of the application (especially those around external data sources) that aren't so easy to test. There are various techniques (i.e. dbunit, mock objects) that can be of use, but they all seem a little cumbersome. We need a better answer here.

Agile development has worked out well. This was a first exposure to rapid iterative development for some team members, so it was a bit of a slow start for some of us. Not everybody is convinced yet-- some of our developers like more complete specs before starting their work. I like it, though.

That's it for today!

Book Review: Java EE 5 Development using GlassFish Application Server by David R. Heffelfinger

When I first picked up this book, I wasn’t sure what to expect. I’m usually an Eclipse/JBoss kind of guy, but occasionally I’ll take a look at other App Servers and IDEs just to see where things are going. It had been a while since I looked at Sun’s application server, so I admit I had preconceived ideas about GlassFish when I started. Boy, was I in for a surprise.

The book starts out explaining the installation process for GlassFish, then how to deploy and undeploy applications. After that, it goes into administration of domains, database connectivity, and connection pools. Not too shabby for the first chapter, especially since the book makes generous use of screen shots, something I always appreciate when I’m doing administrative work via GUI.

I tore into the next section and was rewarded with a re-introduction to servlets, HTML forms, HTTP redirection, and other useful similar topics. At one time I did a fair amount of work in this domain, so there really wasn’t anything astonishing for me. I was pleasantly surprised by the clarity and brevity of the author’s style, though—not too much detail, just enough to quickly get me back up to speed and feeling like I understood the core mechanisms of the workspace I was working around. This excellent style was repeated throughout the book, another big plus. The third chapter was similar, except it gave this once-through big pieces introduction to JSPs instead of servlets. I made use of much of the sample source code and found it very easy to work with.

The next chapter covered JDBC, JPA, and JPQL. Once again, just the no frills basics, with excellent and clear examples of how to use the tools under examination. By this point I was beginning to understand how the book came to be titled as it was—it really is a book that gives an excellent overview of the JEE 5 stack, as utilized on GlassFish. In my humble opinion, Sun would be well served to study this author’s manner of presentation—the text doesn’t drown you in corner cases and obscure functionality, but shows how to use the meat and potatoes of every important component of the stack. Comparing the JEE tutorial to this book would be like comparing a meandering trail through a forest to an empty 8 lane superhighway. This book takes you exactly where you want to go, pronto.

The next two chapters covered the JSP standard tag libraries and JavaServer Faces. I haven’t done any production UI work since the advent of JSF, but chapter 6 did a more than adequate job of explaining how it works. I left the chapter confident I could whip up a decent UI in little time if I was pressed to do so. This reinforces my impression of the book—there are enough simple examples to lend the reader confidence that they can implement a working example quickly.

Chapter 7 covered JMS, something I’ve been doing some work in lately. No surprises in the core technology here, but plenty of helpful screen shots showing how to establish your environment. There were excellent minimalist code snippets, of course.

Security was up next, demonstrating how to set up various realms, including custom ones. Of course there was sample source code to validate your environment, something that’s not always present in explanations of this type.

Chapter 9 was EJBs. All the types are covered, as well as transactions, security, the bean lifecycle, and authentication of a client. If anyone’s studying for the Sun Certified Enterprise Architect exam, this chapter could be of use.

Web Services were covered next. It was in this chapter that I had the thought “Wow, Sun really gets ease-of-use!” Whipping up web services with GlassFish is so easy it’s laughable, especially when using NetBeans. (By this time I was so wrapped up in running samples of things that I started using NetBeans, something I do from time to time if I’m working on a Swing UI. This time around I was picking it up to make use of the excellent IDE/Application Server integration. The build cycle, complete with free build scripts, are really handy.) After reading this chapter, I realized my only remaining gripe against GlassFish was use of the command-line interface, which I credited to my own lack of practice. This app server is for real, and given the simplicity of use I have to expect it’s going to be gaining marketshare. Back to the chapter review, we were shown how to quickly produce web services and clients to consume them, all very easily.

Next up was a chapter on technologies complimentary to but outside JEE, including Facelets, Ajax4jsf, and Seam. By this time I was anticipating the result-- another chapter that lent me confidence that all of these were within a few hour’s reach if ever I needed to get started. This is no small feat, considering I don’t consider myself a client-UI coder in the least, at least not for now.

The book includes two appendices that tell how to configure GlassFish for email and how to get started using either NetBeans or Eclipse.

My overall impression of this book is extremely favorable. I’m an IT generalist, which means I’m called upon to work all over the JEE stack, but more often in some places than others. With a book like this on the shelf, I’ll never have to worry about getting blindsided by a request to work on some part of JEE that I haven’t visited for a while. Kudos to the author and to Packt publishing—this book has earned a spot on my bookshelf.

Great new book for ESB users

I've recently had the opportunity to look over a yet-to-be-published book from Manning titled "Open Source ESBs in Action". If you're an ESB user, you owe it to yourself to check this one out as soon as it's available.

In an unusual twist, the authors of this book presented ESB best practices and techniques using *two* open source ESBs, Mule and ServiceMix. I'm already a Mule user, so I was pretty happy to see several new ideas to employ in my Mule use. I haven't tried ServiceMix yet, but may someday, time allowing. (I'm confident I can run several scenarios right out of the box, armed with this book.) The authors cover more than a few patterns straight out of "Patterns of Enterprise Integration". Kudos to Manning for allowing their authors to acknowledge this excellent resource, even if it comes from a different publisher. I really do appreciate that!

I liked several ideas I think I'll apply somewhere down the line-- including use of XML on Mule to facilitate validation and transformation, use of JibX to help make object-to-XML transformations, and use of a BPM product to oversee non-integration parts of my ESB. (I suspect that'll de-tangle the configuration a great deal, or at least make it easier to understand.)

By using two ESBs to implement every use case, I think the authors give you some insights you might otherwise have missed if you saw just one side of the story. As I said previously, unusual, but interesting.

I won't go on and on-- let's just suffice it to say this book will have a spot on my bookshelf when it becomes available in the near future. If you use an ESB, or think you might like to, you might consider looking at this informative work.