Tag Archives: Programming


I had to write an XML transform today that included CDATA, passed though exactly as received without encoding or escaping characters. It had been awhile, so I dug through an old project to find the incantation – ah: cdata-section-elements.  Just include that magic word in xsl:output, listing the elements to pass through, like so:

   cdata-section-elements="myPassThroughData" />

BTW, this tool is quite helpful when fiddling with XSLT fragments: http://xslttest.appspot.com/.

You Again

Remember back in the JDK 1.5 days when some genius changed BigDecimal.toString and broke half the universe? I’ve felt the impacts in DB2, Oracle, VoiceObjects, and other places. I’m over that now (and all the cool kids have updated or switched to toPlainString), but this thing just keeps coming back to haunt me.

Like this week, when old DB2 JDBC (JCC) drivers kept showing up on some platforms, prompting these intermittent errors:

[ibm][db2][jcc][converters] Overflow occurred during numeric data type conversion of …

This error can sneak up on ya’ because it often takes just the right numbers to kick in. Like the lottery, and sometimes with the same odds.

Fortunately, the fix is easy: just update JDBC drivers to something from the modern era.

Storyboards in any Language

Since the advent of NSLocalizedString, iOS localization practices haven’t changed much. But the new storyboards are easier to translate, particularly if you enable internationalization from the outset. To do this in Xcode 5:

  1. From the project settings (Info page), check Use base internationalization.
  2. For each storyboard, go to the File Inspector (utilities pane), enable Localization, and check at least Base and English.  This will extract strings files that others can translate or customize.
  3. Un-install and then re-install the app from target simulators or devices.

After initial setup, adding languages and dropping in strings files is easy, and quicker than ibtool-importing strings into individual xibs.

Open for Repairs

Although Android Libraries are the recommend way to package and use larger components, there’s no clean way to handle them in a Maven-based Eclipse or Jenkins environment. Sure, JARs are easy enough, but not the co-requisite resource files. Checking these into source control is just wrong, and full Maven support for these libs in incomplete, even with m2e, m2e-android, and android-maven-plugin in place.

Apklibs provide a decent stopgap, so I settled on that approach: loading the zips (ahem, apklibs) to Nexus and creating scripts to link things up during build. It requires one manual step in Eclipse, but no biggie. Yet there are greener pastures ahead…

The upcoming new Android Build System addresses this problem with AARs. Of course, it goes much further, ditching Maven for Gradle, ADT for Android Studio, Eclipse for IntelliJ, and XML for Groovy. I like new things, but while we’re waiting on the new house, I wish Google would finish fixing the plumbing in the current one.


Code analyzers like FindBugs, PMD, and Sonar provide a nice safety net for catching potential quality issues early, during code and build.

By default, PMD over-emphasizes style, leading to a lower signal-to-noise ratio, but FindBugs is quite good at catching real issues that slip past standard, compiler, lint, and IDE checks. Surprisingly good at times, given that there’s only so much you can learn from static analysis.  So I was skeptical that any commercial static analyzer could “take it to the next level” and provide qualitative improvements over open source tools.

That is, until I tried Coverity.

I recently got the chance to try Coverity’s Advisors hands-on, and met with two of their sales engineers to learn more. Coverity is not cheap, but its cost can often easily be justified for commercial code based on time savings and quality improvements.

I ran Coverity against three projects: two server apps with UIs and web services, and an Android app. In all three cases, I was impressed with the findings, particularly the High and Medium Impact ones like resource leaks and the various null dereference checks. These are the value add checks that you don’t get from its embedded FindBugs or from other open source tools. Of course, the UI presentations like code paths (in both Eclipse and the web interface) were also very helpful.

There were quite a few false positives, but not enough to be a problem. Many of the issues were deliberate fail-fasts and mocks in JUnit test classes, and could be marked as ignores. Some of the more interesting finds were null dereferences of results from calls to third party libraries – where I expected a method to always throw an exception on error, but in some situations it could return null instead. This demonstrates how the tool “looks” into underlying JARs to find undocumented behaviors.

Coverity also provides a broad range of security advisor and webapp-security checks, looking for vulnerabilities to SQL injection, XSS, and other exploits in the OWASP and CWE lists. Coverity can receive feeds from other complementary analyzers such as Android Lint, clang, PMD, etc., and include them on its dashboards. This is a nice feature to manage all issues from one common database.

Full runs with most checks enabled took time, often over 30 minutes, so these wouldn’t be done on every continuous integration build. If Coverity ever adds incremental analysis, that would help there.

It’s impossible for developers to catch all potential weaknesses through manual code inspections alone, so when it comes to static quality analyzers, good tools are important. Coverity is top-notch, and this evaluation demonstrated to me the value of a purchased tool.

Droid Units

JUnit testing of model and service classes under Android/Dalvik is straightforward, but when it comes to testing UIs (activities and fragments), the basic support is just too low-level. Fortunately, Robotium (think Selenium for droids) provides a most-excellent adjunct for this.

Yet Robotium should be used with care to create maintainable tests that aren’t brittle. To that end, I’ve developed some UI automation practices:

  • Write true unit test classes that cover only a single activity or fragment. Of course, many UI actions will take you to the next activity (and you should assert that), but leave multi-activity testing for separate integration tests.
  • Stub back-end behaviors using standard mock and dependency injection techniques. For my current app, I kept it simple/lightweight and wrote my own code, but I’ve also used Mockito (with dexmaker) and Dagger (with javax.inject); these are nice Android-compatible frameworks.
  • Rather than repeating the same raw Robotium calls directly from your tests, wrap them in descriptive helper methods like enterUserID(String id), enterPassword(String password), clickLoginButton(), etc. This DRY approach makes for more readable tests and simplifies updates when your UI changes.
  • Since you probably use common superclasses for your activities and fragments, also create parent test case classes to factor common testing behaviors. See below for snippets from one of mine.

I haven’t found a good tool for measuring code coverage for apps (Emma under Android is flakely), so I’d love to hear your recommendations.

public abstract class BaseActivityTest<T extends Activity> 
                             extends ActivityInstrumentationTestCase2<T> {
	protected Solo mSolo;	
	public BaseActivityTest(Class<T> activityClass) {
	protected void setUp() throws Exception {
		// Reset your DI container and common mocks here...		
		mSolo = new Solo(getInstrumentation(), getActivity());
	protected void tearDown() throws Exception {
	public void testLayoutPortrait() {
	public void testLayoutLandscape() {
	// ...


There are several options for over-the-air distribution of in-progress apps to internal testers. For Android, you can start simple with a link to the APK on a web site, but this misses the benefits of the Google Play store: automatic updates, consistent store listings, crash reports, support links, etc. Fortunately, we now have Google’s new Private Channel Distribution: a protected area for distributing apps as they develop and get ready for the larger “real world” of public distribution.

The service is easy to use, but does require a few setup steps:

  1. You must be a Google Apps user with administrator access. If you don’t already have it, sign up and pay the man.
  2. You must be registered as a Google Play developer with an account belonging to your Google Apps domain. Here again, sign up and pay the man.
  3. Sign into your Google Apps Console and enable the Google Play Private Channel. The process changed again recently with the new admin console, but you can find the latest instructions here.
  4. Log in to the Google Play Developer Console and create, configure, and publish your app. Check the box under Pricing & Distribution – Restrict Distribution.

To install, add your Google Apps account to your phone (for example, Play Store – menu – settings –  Account – Add Account – OK – Existing).  You should then see the private apps for your Google Apps domain under Apps – Categories (this, too, has changed recently).

You can use the Recent changes section of the app listing to describe updates to testers, and include a link to a video walk-through. Perhaps soon WebEx’s Android app will provide mobile screen sharing for live demos; until then Android Screencast works well with WebEx PC desktop sharing.  It’s not as nice as Reflector, but it is free and lightweight.

Worth the Weight

With many Android apps, there often seems to be little correlation between the size of the package and the value it provides. Multi-megabyte apps that do almost nothing leave me wondering, “what’s in there?”

Unpacking with dex2jar and JD-GUI often provides answers, and frequently it just means the developer forgot to enable Proguard when building. An overabundance of ill-compressed drawables are another common source. But beyond these, the habits of server-side re-use (freely expanding POMs and dropping in FOSS JARs) are a key source of bloat.

I try to be stingy when it comes to Android app libs, often taking a tougher route to avoid bringing in large JARs that might otherwise be useful. Such was the case recently when I needed to do a multi-part post to a REST web service that consumed a mix of binary image data and JSON. Multipart HTTP is conceptually simple, but the markup is obscure enough to make generating it directly from a business app just wrong.

Fortunately, though, Apache HttpMime is just 26K, and makes the process simple. For example:

MultipartEntity entity = new MultipartEntity();
entity.addPart("request", new StringBody(request));
entity.addPart("image", new ByteArrayBody(image, "image/jpeg", filename));

To avoid duplicate class errors at Proguard time, exclude the dependent httpcore in your pom.xml, like so:


I don’t always add JARs to Android apps, but when I do, I prefer light ones.

Just Click

Distributing in-progress iOS apps is tedious enough: setting up signing certs, creating provisioning profiles, adding device IDs, etc. I wanted at least the end-user install step to be friendlier than syncing with iTunes and remembering the .mobileprovision file. That is, I wanted to provide the same simple install for my iOS apps as I do for Android apps, with a web page that folks can visit from their phone browser and simply click a link to download and install.

Fortunately, I came across the itms-services protocol: Apple’s solution for wireless app installs. All it takes is the distribution IPA (with embedded mobileprovision file), plist, and a web page with the link. It’s not widely known, but there are a couple of good tutorials on the web, such as Apple’s instructions and Aaron Perecki’s walkthrough. No more tangles, no more wires!

Out of the Box

Business travels and extra duties lately have brought some unexpected surprises. I find such “out of the box” adventures refreshing: pulling the rug out from under typical routines forces me to challenge assumptions and improvise. Here are some examples from the past two days.

The Right Sequence

During group architecture sessions, I needed to quickly create, modify and display sequence diagrams to demonstrate interactions among several new web services. Lacking my usual tools (and convinced there had to be a better mousetrap), I searched around and found websequencediagrams.com. This STTCPW site was exactly what I needed. Using it was faster than tools such as Visio Pro, and even faster than drawing and editing on whiteboards.

Weekend Jobs

Today I got word that some Quartz jobs weren’t running as expected during pre-production testing, where typical weekday processes were run as a logical business day on Saturday. The cron-expressions were correct, so I needed to see more. A look at the full jobs.xml revealed the culprit: this environment had been configured to use WeeklyCalendar rather than the usual AnnualCalendar. A quick temporary switch had Quartz working weekends with the rest of us.

Fog of WAR

Despite version labels and other indicators, a web app was behaving as if it was down-level in a key area. Since the WAR file hadn’t been obfuscated, I grabbed a copy of the JD-GUI decompiler for a closer look. Sure enough, the class in question was at an older level. When sources aren’t available or are in question, the JD Project tools are indispensable for moving past assumptions and getting to facts.

Decimal, with an E

New calls to my web services from VoiceObjects were sending certain decimal data in E notation.  The underlying call flow objects were TTA – Literals/Digits with a digits?minlength=1;maxlength=14 grammar, so it should have been decimal from end to end.  But there turned out to be an unexpected feature of the platform to convert to float whenever any arithmetic is done, with no access to DecimalFormat or other mechanisms to convert back. Since there was fortunately no loss of precision and since a problem correctly stated often solves itself, I knew exactly what to do: modify the web service to accept the format. Voxeo provides a great VoiceXML platform, but it has plenty of surprising nuances like this.


We corp-rat developers typically do most real work in some big honkin’ desktop IDE, yet dream of popping into a browser or lightweight shell to get things done. For example, with much of my PHP and Ruby code, I can ssh and vim from anywhere, but my Java work is hopelessly tied to a single laptop. If only we could combine the rich features of an IDE with the shareability and convenience of a cloud environment.

That’s what projects like OrionHub and Codenvy seek to accomplish. Orion seems to have stalled a bit, but since Codenvy is quickly moving forward (and ambitiously taking on Java EE development), I gave it a try.

Codenvy supports a variety of languages, frameworks and cloud platforms, but I only needed Java: for a standalone JAR, and some web code with Spring and Tomcat.

Codenvy works hard to re-create an IDE-style environment in the browser, and does a good job of it. It has an Eclipse-like layout and features, and even many of the same keyboard shortcuts. There were plenty of gaps and differences to remind me that I wasn’t in Kansas anymore, yet I could adjust to those. Even required periodic browser refreshes were OK; they always took me back to some reasonable point.

But it was just too sluggish and buggy for anything but small projects. Granted, Eclipse desktop itself is infamous for stalls, crashes, and sudden disappearances, but this is another level. I suspect some of that can be addressed with faster servers and networks. Indeed, to use this for real work, I’d need the on-premise version where I could control that along with back-end interfaces.

I’ll keep an eye on Codenvy and use it where it fits. It’s great for sharing and collaborating on small bits of code, like a jsFiddle for Java. And maybe soon it’ll be the envy of every developer tied to a desktop IDE.

VoiceObjects by Proxy

During today’s action, I needed to verify new custom HTTP header fields (#HeaderName# properties) in my SOAP web service calls originating from a Voxeo VoiceObjects (VO) call flow.  This meant pointing VO to my favorite trace tool (Fiddler2) running as a proxy on localhost / 8888.  This usually requires setting JVM parameters (proxyHost and proxyPort) on the embedded app server, overriding post URL ports, or using proxycfg, but I suspected VO had an easier way.  Sure enough, after some poking around, I found it:

Repository – Services – (service name) – Proxy ParametersHTTP proxy host and HTTP proxy port

Yet another nice convenience in this market-leading platform.

Careful, They Byte

My son recently updated me on his latest work project, and we chatted about the dangers of treating binary data as characters or words, rather than just streams of bytes.  Nearly everyone must deal with character encoding, but these days byte ordering (endian word) concerns are usually the domain of systems programmers.  Consider:

“A rose is a rose, unless it grows in an Intel garden, in which case it’s an ores or an esor.”


Back in the day, I tossed that around frequently to mock the byte swapping in 16-bit and 32-bit words that we Intel assembly programmers dealt with.

Entropy Atrophy

derek williams (gatech) has the 2nd best hash.  See the full standings at http://almamater.xkcd.com/best.csvKudos to the Carnegie Mellon alum for the closest-fit solution to xkcd’s externally-controlled April Fools comic and Skein hash collision contest (always read the alt text).  I was far too hardware-deprived to be nerd-sniped by this one, but there were plenty others who jumped right on it, all motivated by challenge rather than “money.”

That’s encouraging because it seems information theory isn’t taught much anymore (my own alma mater has long since dropped “ICS” for “CS”). Although we now need it most (in our big data and security-starved era), our collective entropy-sophy has atrophied.

In areas like security policy and algorithm design, the cold reality of the pigeonhole principle is too often forgotten.  We often regard hashes as magic, forgetting they’re just bit-twiddling mapping functions and that when the domain is bigger than the range, there will be collisions.  Simple truths like this are ignored in spots ranging from the XBox to ReiserFS.

The winner of xkcd’s contest was still 384 bits shy of a total collision, but every imperfect hash has plenty of clashes.  So be careful out there to win that battle between bit length and processing power, at least while GPUs and quantum computers develop.

BTW, it seems Wikipedia got enough donations from the effort to be good sports about the xkcd-hacking.

Tight Spring

I still recall the wise advice of a friend while I was designing and building a set of application frameworks over two decades ago: “create as many hooks as possible and be quick to add new ones.”  So for each service flow I noted every interesting step and created extension points for each.  I didn’t think most of these would be be used but, over time, nearly all were.  And as users requested new hooks, I quickly added them.

The Spring frameworks, with their IoC underpinnings, are built for extensibility.  The application context is a software breadboard allowing all sorts of custom wirings and component swapping.  And namespace configuration elements make for much clearer and cleaner XML.  Where done right, the namespace configuration captures most core extension points, documents them well, and makes them easy to use. But shortcuts, inflexible designs, and atrophy can cause parts of Spring to be too tight. Here are just a couple of examples I encountered with this week.

Spring Security – Session Management Filter

A customer’s single sign-on (SSO) flow required tweaks to SessionManagementFilter, so I built my own: a subclass with just a simple override.  But swapping it in turned out to be quite clumsy, and basic things like setting the invalid-session-url in the namespace no longer work as documented.  Since I now have to specify the complete bean construction in the security context anyway, I decided to just replace the SimpleRedirectInvalidSessionStrategy.  Here again, I just needed one method override, but, alas, it’s a final class.  So, with some copy and paste re-use and ugly XML I finally got what I needed; here’s the gist:

 <http use-expressions="true" ...>
    	<!-- Disable the default session mgt filter: /-->	
	<session-management session-fixation-protection="none" />
    	<!-- ... and use the one configured below: /-->		
	<custom-filter ref="sessionManagementFilter" position="SESSION_MANAGEMENT_FILTER" />
 <!-- Configure the session management filter with the custom invalid session re-direct. -->  
 <beans:bean id="sessionManagementFilter" 
	<beans:constructor-arg name="securityContextRepository" 
		ref="httpSessionSecurityContextRepository" />
	<beans:property name="invalidSessionStrategy" 
		ref="myInvalidSessionStrategy" />
 <beans:bean id="httpSessionSecurityContextRepository"
 <beans:bean id="myInvalidSessionStrategy"
  	<beans:constructor-arg name="invalidSessionUrl" value="/basic/authentication" />
	<beans:property name="myProperty" value="myvalue" />  		

I wish JIRAs like SEC-1920 weren’t ignored.

Spring Web Services – HTTP Headers

I built web service calls (using Spring-WS) to a site that requires authentication fields in the SOAP header. It’s pretty pedestrian stuff except there are no direct Spring public methods to set header fields or namespaces. Instead I had to use the recommended WebServiceMessageCallback like so:

webServiceTemplate.sendSourceAndReceiveToResult(getUri(), source, 
        					getWebServiceMessageCallback(), result);
private WebServiceMessageCallback getWebServiceMessageCallback() {
   return new WebServiceMessageCallback() {
	   public void doWithMessage(WebServiceMessage message) {
		   try {
			SoapMessage soapMessage = (SoapMessage) message;
	           	SoapHeader header = soapMessage.getSoapHeader(); 
	           	StringSource headerSource = 
					new StringSource(getAuthenticationHeaderXml());
	           	Transformer transformer = 
	           	transformer.transform(headerSource, header.getResult());
		   } catch (Exception e) {
			// handle exception

That’s too verbose for what should have been one line of code.  I agree with JIRAs like SWS-479 that this should be simplified and extended.  Is an addHeader convenience method too much to ask?

BTW, when developing web service calls, I usually start with XMLSpy and soapUI to get the XML down first.  Once I switch over to coding, I typically set JVM system properties (proxySet, proxyHost, and proxyPort) to point to Fiddler2 so I can examine request and response packets.  It’s a nice arrangement, but I’m always looking for new ideas.  If you prefer a different approach, write me.

Just Ask the Mayans

“When in doubt, predict that the present trend will continue.”

Murphy’s Laws

It’s now an annual tradition here to calculate year-end NFL power rankings using tools we developed as Friday Fragments.  So I grabbed 2012 NFL regular season records and ran them through our tournament matrix calculator.  Of the resulting top 12 teams (by power ranking), 10 are in the playoffs:

Rank Team Power
1 Falcons 91
2 Texans 79
3 Broncos 77
4 Colts 76
5 Ravens 76
6 49ers 76
7 Packers 72
8 Patriots 67
9 Bears 65
10 Seahawks 59
11 Bengals 59
12 Rams 55

The Vikings and Redskins replace the Bears and Rams in real life.

These rankings look at the season as a whole, which is why my Falcons come out strongly on top.  But it’s a valid argument to weigh recent games more heavily; for example, if you remove the first 5 weeks, you wipe out the Broncos’ 3 losses, and they come in first.  Feel free to visit the page and run your own partial season calculations.

Of course, “past performance does not guarantee future results,” so that’s why they play ’em.  But one thing seems sure: it’s been an exciting regular season, and the postseason promises to be just as good.

jQuery Mobile

This week, I needed a quick mobile (smart phone) proof-of-concept for one of our web app products.  Since speed of development and broad platform support were more important than native device features, I took the mobile web route.  And as a jQuery fan, I chose jQuery Mobile (JQM).

JQM emphasizes doing as much as possible with simple HTML5 and CSS.  It uses data-* custom attributes to know how to add extra behaviors, rendered and tailored as needed for the target device.  Indeed, I found I could do most things with just markup and server-side code, and very little additional JavaScript.

JQM makes AJAX calls by default, but normally expects the results to be new page divs to add to the navigation stack.  This is different from my standard web app, where $.ajax calls return only data (XML and JSON), with bits of JavaScript to update DOM components from the results.  I decided to play along and add div wrappers around results, letting JQM navigate to them as simple linked pages (href=, action=) and dialogs (data-rel=”dialog”).  The end result was cleaner code (and smoother flow than standard postbacks would be), although I did give up a bit in look and feel.

Native app development requires real devices or emulators (such as the Android Virtual Device, AVD), but with a pure mobile web app, I was able to just use Chrome with the nice Ripple extension.  This removed all delays from the code-run cycle: yet another benefit of the mobile web / HTML5 world.

I went with a standard theme and did not (yet) need the extra step of packaging for offline use or adding a native wrapper (like PhoneGap/Cordova).  But those processes appear straightforward when the time comes.  The were a few little glitches along the way, but nothing I couldn’t work around with a little JavaScript.

jQuery Mobile is perhaps the go-to framework for HTML5 mobile web apps.  Certainly for those of us who are sold on the benefits of jQuery.

New Tricks

I recently resumed working a bit on a VA Smalltalk-based system I first created over a dozen years ago. VA Smalltalk is a fantastic programming environment, but I quickly realized how much I missed automated build packaging in that world. Each build takes only a few minutes, but it’s manual, requiring clicks through a GUI and waiting for results. I wanted this to now be automated like the rest of the continuous integration free world.

Fortunately, some folks have recently built tools to help.

I started with Ernest Micklei’s Melissa tool, a handy EpPackager front-end for scripting builds.  Using it to automate builds for our headed (GUI) images was straightforward: I just created a workspace and CMD file for my particular requirements.

But scripting headless (cross-development, XD) packaging was not so simple, due to the nature of passive images and the fact that the controls are hopelessly embedded in the XD development UIs.  Eventually, I found that Thomas Koschate cracked that nut with his nice HqaAutomatedBuildSupport tools (available at VASTGoodies.com).  To use it, I mainly just created my own AbtBuildSpecification to specify the maps, subsystems, and features to match what I used in my XD Image Properties.

For consistency’s sake, I wrote some code to invoke AbtBuildSpecification build from MelissaBuilder. When done, I set up the CMD files as Windows Scheduled Tasks (schtasks) to run overnight.

It’s a handy process, but clearly something that could benefit from standardization by the VA Smalltalk vendor.  For example, there are different ways to script startup loads: abt.cnf, Melissa, AbtImageStartup, or some combination of these.  Perhaps our friends at Instantiations will soon include this in the base product.  Our venerable community could certainly benefit from a common way of doing these new tricks.

The Pareto Lamp

I was asked over the Thanksgiving break to create an online store for soccer fundraising.  This planned Christmas Catalog was behind schedule and needed something fast.  With turkey on deck, there was no time to code anything from scratch, so I jumped into Softaculous and installed the highly-rated OpenCart. Configuration was quick, and I soon had the 25 product store online and ready for business.

It can be just that easy in the LAMP Stack world.

That’s largely because most of what we do on the web has already been done countless times before. There’s rarely good reason to code a custom solution when open source options are readily available. That’s a very good thing, because an occupational hazard for code monkeys like me is that we often get asked to help with “computer things” like this.  We like to be able to knock out these common problems quickly and reserve most of our time for the uncharted territories that require more invention and programming horsepower.

I did have to browse the OpenCart source, but that was only to answer my questions about how to configure some things (like sales tax) that didn’t work quite right at first.  Since it follows a familiar MVC structure, navigating the PHP was straightforward.  In the end, I didn’t have to write a single line of code.

Nearly 80% of the server-side web is written in PHP.  And certainly less than 20% of overall programming work goes toward maintaining it.  I’m thankful for the LAMP stack and the Pareto principle that powers it, and will keep OpenCart and other ready Softaculous solutions handy.