I ran into a bit of a quandary the other day. I had to modify a piece of legacy code in our system - one that has no tests written against it. My first task was to write as many JUnit tests as I could to document the behavior of the component to help ensure that I didn’t break existing functionality with my changes.

After a few tests, however, I began to notice something troubling. The component I was writing tests against had some bugs in it, and in order to make my unit tests pass, I had to code to those bugs. This in itself did not bother me too much, since I had no expectation that the component I was testing was error-free.

What bothered me is that unit tests are often the best documentation of how a system works, and I would be creating misleading documentation. So, I did what anyone in this situation would do: I emailed someone a lot smarter than me. 

The reply email contained several suggestions, all of which I plan to implement. The first suggestion is to carefully name the test methods to make it clear which ones work because of defects. The second suggestion is to create an annotation that can be added to a method to indicate that it works because of a defect, coupled with some way to report on all uses of the annotation.

After a few hours of coding, I’m happy to announce that I’ve created a Java annotation named KnownDefect as well as a detector plugin for FindBugs to collect all of the annotation instances.

In the parlance of “Working Effectively with Legacy Code“, I have been creating “characterization tests“. These tests look like normal unit tests, but they document the current state of the system, warts and all. Using the KnownDefect annotation helps make the warts stick out better.

Here’s an example:

public void testValidateNullResponse() {
   try {
      ProtocolParser.validateResponse(null);
      fail();
   } catch (InvalidRequestException expected) {
      // Caught expected exception
   }
}

This test documents the system as it currently exists, but the behavior is obviously not correct. The ProtocolParser class should be throwing an InvalidResponseException, not an InvalidRequestException. To document this defect, I change the method name and add the KnownDefect annotation to the test method, resulting in the following:

@KnownDefect("Should throw InvalidResponseException")
public void testValidateNullResponseShowsKnownDefect() {
   try {
      ProtocolParser.validateResponse(null);
      fail();
   } catch (InvalidRequestException expected) {
      // Caught expected exception
   }
}

The correct behavior is now documented, and the test still passes.

Instead of writing a new tool to manage the KnownDefect instances, I ended up writing a new plugin for FindBugs. The FindBugs detector will collect all instances of the annotation and group them under the “Correctness” heading so they’re all in one place. Bug reports can then be created and the defects fixed or not as business needs dictate. (Sadly, the FindBugs plugin for Eclipse does not find the annotation in Groovy code. The standalone version of FindBugs works fine, so I assume there’s a resource filtering issue with the Eclipse plugin.)

Instructions for using the annotation and the detector are included in the download file. Try it out - hopefully it will be useful to somebody other than myself.

(Download KnownDefects) (Subversion: http://subversion.megatome.com/projects/KnownDefects/trunk)

(Click to get the shirt)

I’ve been coding with an agile mindset for quite some time now. Some practices have become second nature to me over the years. One of these practices is aggressive code refactoring, coupled with test driven development (TDD).

I’ve been adding new functionality to our system at my new job, and as usual I’ve been following my standard techniques. My part of the system involves handling certain requests and responses. I wrote unit and functional tests for the request half of the equation, then began to tackle the response side. As I added (and tested) functionality, I began to notice a large amount of similarity in the code. This in turn led me to create an abstract base class for both request and response, pulling up all of the shared functionality. This resulted in the actual request and response classes being quite small, with just a few method implementations from the base class.

So far, this activity should seem pretty normal to most developers. As I said, it’s pretty much second nature for me, and I don’t even remember consciously making the decision to make the abstract class. Imagine my surprise when my team lead admonished me for creating and abstract class, and warned me not to do it again without explicit approval.

Of course I had to ask why we would not refactor code in this manner when the opportunity presented itself. My lead’s answer involved several parts, none of which have convinced me that refactoring is bad:

1. The abstract class is not consistent with the rest of the system. This, to me, seems to be a fault of the system. I’m not knocking consistency, but at some point you have to realize that you may be building something that’s consistently bloated/overcomplicated/etc.
2. The abstract class changes the design. Huh? Sure, if I go generate a UML diagram of the system, there’s going to be this abstract class and inheritance there. This is the only place the design is different. Requests and responses are being correctly handled just like the legacy ones in the system.
3. Too much refactoring is bad, and each case should be thoroughly examined before code is changed. This is how religious wars start. My lead claims that developers start out on the “no refactoring” end of the spectrum, simply because they don’t know any better. Once they learn about refactoring, they then swing to the “refactor everything” end of the spectrum where anything and everything is a potential target. Seasoned, knowledgeable developers fall somewhere in the middle. I got the impression from this speech that my lead considers me to be on the radical end of the spectrum.

Why are people like my lead so scared of refactoring? The arguments given really seem more like rationalizations than actual reasons - they don’t really stand up to scrutiny. I have a hard time believing, for example, that my lead truly thinks that the system is better because each request and response is a totally separate class, albeit with most of the code copied and pasted from one to the other. If anything, he should know that this approach leads to more problems, no matter how pretty it keeps the class diagrams.

(Click for Larger)

Yup, they sent me a faceplate for my Wii guitar, along with a note explaining that the gift is a thanks for being patient and waiting to get my remastered GH3 disc.

While this probably isn’t the design I would have picked on my own, I have yet to find faceplates for the Wii guitars in stores yet, so beggars can’t be choosers.

This gesture is nice and aimed at keeping customers. I appreciate the gift, but I’d rather have an apology for the way they screwed up and shared a bunch of email addresses that should have been kept private. I guess customers have been more concerned about how many speakers their game’s music comes out of than who has their email address.

The bill, however, did not pass by the two-thirds majority required to override a presidential veto, which is inevitable. What will likely happen now is that this issue will be delayed until the next president is sworn in, who will hopefully be somebody who agrees that telecoms need to be held responsible for their actions.

If you’d like to see the actual breakdown of the vote, here’s the official tally, which unfortunately does not list Representative’s states.

I’ll close with a quote from Ted Kennedy regarding the retroactive immunity fiasco and our president’s insistence on it:

"The President has said that American lives will be sacrificed if Congress does not change FISA. But he has also said that he will veto any FISA bill that does not grant retroactive immunity. No immunity, no new FISA bill. So if we take the President at his word, he is willing to let Americans die to protect the phone companies."

(Full text here.)

Previous Megatome entries:

• Colorado Senators Side with Telecoms
• Senator Salazar Responds - Still Doesn’t Get It

Co-Worker (who does not believe in unit tests): Okay. I just checked my changes in.

Me: There’s a bug in your code.

Co-Worker: How do you know? I just checked it in!

Me: It broke my unit tests. It looks like you’re not setting the flibbergibbet value correctly.

Co-Worker: What? How can you tell?

Me: JUnit showed me the difference.

Co-Worker: Oh. I guess I’ll fix it now. I would have found it eventually…

For what it’s worth, we’re handling Electronic Data Interchange documents - things that look like a page or so of line noise at a time. The bug was that one character wasn’t getting written to the output. My co-worker tests his code with the toString() and visual grep method. I’m not sure he would have ever caught the bug on his own. I keep trying to explain that unit tests are a Good Thing…

I’m a sucker for puzzles. The problem with most puzzle games is that they usually focus on one kind of puzzle and I lose interest. (Yeah, Brain Age. I’m looking at you. I enjoy Sudoku, but solving 85 jabillion Sudoku puzzles gets old fast.)

Professor Layton, luckily, does not suffer from this problem. Some of the puzzle themes are repeated, but there’s never more than four or five puzzles for each theme. For example, one of the themes is "Matchstick Puzzles", and there are half a dozen or so total matchstick related puzzles in the game. This variety really helps, as there will surely be some puzzle types that people like less than others. (For me, my bane is magic squares. Luckily, there are only a couple in the game.)

Professor Layton does a good job of wrapping the puzzle solving into an actual story. Where the developers could have easily thrown something simple around the puzzles, the story is actually quite entertaining. I was very near the end of the game before I figured out the village’s secret.

The game went quickly for me, taking about eleven and a half hours to finish. I intend to let it sit for a while and replay it to try to use as few hints as I can. I also need to make sure I have plenty of free time, because this game is very hard to put down.

Not only are there well over 100 puzzles in the game to solve, there are additional puzzles available weekly for download via the Nintendo Wi-Fi Connection.

Dear XXXX:
Thank you for contacting me with regard to S.2248, the Foreign Intelligence Surveillance Act (FISA) Amendments Act of 2007. I appreciate hearing from you.
As a U.S. Senator, my primary responsibilities are to uphold the Constitution and protect the American people. These responsibilities have guided me during the recent Senate debate on reforming and modernizing the rules governing America’s surveillance and intelligence-gathering system.
As you know, S.2248 passed the Senate on February 12, 2008 by a vote of 68-29. While I was disappointed the Senate did not adopt several amendments that would have gone further in strengthening civil liberties protections, I ultimately supported final passage of the bill.
I believe S.2248 gives intelligence officials the tools they need to pursue foreign threats, and, furthermore, institutes stronger oversight mechanisms to preserve the privacy rights of American citizens. Specifically, S.2248:
· Declares that the FISA court is the sole authority for the approval of electronic surveillance procedures, in response to the Bush administration’s five-year warrantless surveillance program outside of FISA;
· Implements a six-year sunset of the program to allow Congress to evaluate how the new authorities are carried out;
· Requires FISA court approval of foreign targeting procedures for determining that the target of the surveillance is reasonably believed to be outside the United States;
· Grants the FISA court oversight of the “minimization” procedures governing the protection of the identities and private information of U.S. citizens incidentally collected during the monitoring of a foreign target;
· Requires FISA court approval, on an individual basis, of the targeting of Americans overseas based on the court’s review of whether there is probable cause to believe that the person is an agent of a foreign power;
· Requires the FISA court to provide Congress with judicial opinions and interpretations pertaining to the new surveillance program;
· Requires the Attorney General and the Director of National Intelligence to assess overall compliance with targeting and minimization procedures and submit their findings every six months to the FISA court, as well as the House and Senate Intelligence Committees.
Taken together, these reforms represent a significant improvement over previous FISA laws in terms of oversight and accountability.
During the debate, the Senate also addressed the issue of retroactive immunity for telecommunications companies. As you may know, in the wake of the September 11th attacks, it is alleged that a number of telecommunications companies in the United States were asked by the National Security Agency (NSA) to turn over the personal data of their American customers for examination without a warrant. Subsequently, a number of Americans have filed lawsuits against these companies for violating their Fourth Amendment right to privacy.
S.2248 as reported to the full Senate provides narrowly circumscribed, retroactive immunity to the telecommunications companies in question. In response to this, I cosponsored Senate Amendment 3858, which would have referred the lawsuits to the FISA Court for review. Under this approach, if the court determined that the companies acted in good faith and had a reasonable belief they were abiding by the law when they complied with the government’s requests, the lawsuits would have been thrown out of court; if not, they would have proceeded as planned. Unfortunately, this amendment did not garner enough support to be included in the final Senate bill.
S.2248 now awaits action in a House-Senate conference committee. Please rest assured that I will keep your thoughts in mind as my colleagues and I continue work on this legislation.
Again, thank you for taking the time to share your views.
Sincerely,
Ken Salazar
United States Senator

While it was nice to actually hear back from Senator Salazar, I’m a bit disappointed at the content. In my correspondence, I specifically called out the failure to kill retroactive immunity as the reason for my writing. As you can see above, the response does a good job of skirting the issue until the last paragraph or so.

I am relieved that the proposed amendment 3858 did not pass, as it sounds like it would have been too easy for telecoms to get off the hook. All they would have to do is assure the court that they weren’t aware of any laws that were broken, and they would be off the hook.

I think Senator Salazar’s response can be summed nicely by this line: "While I was disappointed the Senate did not adopt several amendments that would have gone further in strengthening civil liberties protections, I ultimately supported final passage of the bill." I find it rather amusing that the good senator was disappointed - considering he voted against one of the amendments that would have strengthened our civil liberties.

In this case "certain assistance" means the activity that we’ve been hearing about that violates our Fourth Amendment rights - telecoms assisting the NSA in warrantless wiretapping activities.

Apparently my Colorado Senators don’t seem to think that holding telecoms accountable for their actions is a good idea - they both voted against the amendment. I certainly am not surprised by Republican Wayne Allard’s "nay" vote, but I’ll have to admit that I expected better from Democrat Ken Salazar.

Want to see how your Senator voted? Look here for the official tallies. If your Senator voted "nay", I strongly urge you to contact him or her and express your disappointment. I have already sent emails to both Allard and Salazar - hopefully enough other people will do the same to make our Senators realize that they work for their constituents and not large companies.

——-

The Electronic Frontier Foundation has some great resources regarding retroactive immunity and warrantless wiretapping.

Depending on which reports you read, anywhere from four to five undersea communications cables have been "compromised". I refrain from saying "cut" because very few of the accounts line up with each other. Reports range from cut lines to breakage due to seismic activity to power problems.

No matter what, all of the affected cables are in the Middle East, affecting the UAE.

This is where the conspiracy theory angle comes into play. If you look at the map put together by ILoveBonnie.net, you’ll see that it’s very unlikely that a ship’s anchor or even seismic activity could account for all of the cables.

Let’s don our tinfoil hats and see if we can come up with a reason why anybody would want to disrupt communications in the UAE. Aha - here it is. Iran plans on making their oil bourse operational in the first weeks of February.

A bourse is defined as a "European stock exchange", and an oil bourse obviously is an exchange devoted to oil trading. What’s important about Iran’s bourse is that, unlike all of the other oil markets in the world, it will not be tied to the US dollar.

There is a fascinating article at Energy Bulletin by Krassimir Petrov outlining the effects that Iran’s oil bourse would have on the American "Empire". It’s a wonderful read, and explains very clearly why having an international oil exchange that is not tied to the US dollar will affect the influence that the American economy has on the rest of the world. Changes to this influence will result in the US dollar being devalued significantly, leading to either deflation or hyperinflation in the American economy. Either of these actions will have significant negative impact, possibly causing another depression.

I hope you’ve taken the time to read Petrov’s article. I also hope that while you were reading it, you noticed it publication date: January 17 2006. That’s right - nearly two years ago the bourse was being seen as a threat. Do you remember hearing about it then? I don’t, either. What happened was that the bourse was delayed and is now on track to open in February of 2008. As of December 2007, Iran has already stopped accepting US dollars for oil.

Now, it shouldn’t take too much to put two and two together. The operation of the Iran oil bourse has the potential to collapse the American economy, and communications systems in the same geographic area have begun to fail, possibly due to sabotage.

It is also worth mentioning that these cable failures have not taken Iran offline, as some people have noted. Indeed, according to the Internet Traffic Report, it appears that Iran is totally offline, but remember that this information is based on the availability of a single router. (At the time this post was written, the single router listed for Florida in the US had 100% packet loss, but that doesn’t mean that Florida was offline.)

Still, broken cables offer a wonderful opportunity for communications listening devices to be added with very low risk of discovery. It may be that there is a plan to disrupt communications so the bourse will not be able to operate normally. It may also be that the cables end up having absolutely nothing to do with the bourse. I’m more of the paranoid type, and with our current president, I’m leaning towards there being a definite connection.

Regardless of any connection between the cables the bourse, if the US does end up going to war with Iran at some point, I think it will behoove us to question if the action is simply to help the US dollar retain its international value. After all, according to Petrov, a man named Saddam Hussein was the first to demand euros for oil (in 2000) - and look where he and Iraq ended up.