Syncing..., Part 3

In a couple of prior posts, I described some of my requirements for syncing my iPhone with my laptop computer. A couple of months ago, I solved the problem (and a whole bunch of other ones, too). I'm very pleased with the solution I chose, which I'll share with you here.

I bought a new MacBook Pro. Not one of the new solid-brick-of-aluminum ones, but one of the slightly older aluminum ones held together with screws. It has a beautiful 15-inch matte-finish monitor, and it's faster than any portable I've ever used. It runs Windows XP applications in my Fusion VM about 33% faster than the native XP laptop I replaced.

It was a big investment, but I love it. Every time I open up my MacBook, I feel good. Thirty seconds later after it has booted completely from scratch, I love it even more.

VMware Fusion, by the way, is a brilliant penetrant to the barriers I had perceived about moving from Microsoft Windows to Mac OS X. With Fusion, you can make your whole Windows PC run on your Mac if you want to. As you gain comfort in OS X, you can migrate applications from Windows to the native Mac operating system. I digress...

So, now I use iCal, which gives me the ability to subscribe to iCal feeds like the one from TripIt. And ones like webcal://ical.mac.com/ical/, which automagically populate your calendar with US holidays, so you won't accidentally book a course on Labor Day. (Visit iCal World for more.)

My new system completely solves the travel problems I was talking about. Now, I do this before I travel:

1. Book travel.
   
2. Forward the confirmation emails to plans@tripit.com.
   
3. Print my TripIt unified itinerary for my briefcase. (Or not.)
4. Sync my iPhone with my laptop.

And that's it. No data double-entry. None.

One more requirement I had, though, was syncing my iCal calendar with Google Calendar. I found a solution to that one, too. I tried using the Google Calaboration, but I really didn't like the way it forced me to deal with my separate calendars. The tool I chose is called Spanning Sync. I used their 15-day trial, liked it a lot, and bought a subscription. I love the way I can map from my list of Google calendars to my list of iCal calendars. However, I don't like the way it syncs my contacts, so I just turn that feature off.

I'm intrigued by the Spanning Sync business model as well. You can save $5 on Spanning Sync by clicking here. It works like this (I'm quoting here a Spanning Sync web page)...

After a 15-day free trial period, Spanning Sync usually costs $25/year, but you can save $5 by using my discount code if you decide to buy it:

39PKXV

Also, if you use my code I'll get a $5 referral fee from Spanning Sync. Once you're a subscriber you'll get a code of your own so you can make money every time one of your other friends subscribes to Spanning Sync. Pretty cool!

Anyway, I'm very pleased with the new system, and I'm happy to share the news.

Performance as a Service, Part 2

Over the holiday weekend, Dallas left a comment on my July 7 post that begins with this:

One of the biggest issues I run into is that most of my customers have no SLAs outside of availability.

It's an idea that resonates with a lot of people that I talk to.

I see the following progressive hierarchy when it comes to measuring performance...

1. Don't measure response times at all.
2. Measure response times. Don't alert at all.
3. Measure response times. Alert against thresholds.
4. Measure response times. Alert upon variances.

Most people don't measure response times at all (category 1), at least not until there's trouble. Most people don't measure response times even then, but some do. Not many people fit into what I've called category 2, because once you have a way to collect response time data, it's too tempting to do some kind of alerting with it.

Category 3 is a world in which people measure response times, and they compare those response times against some pre-specified list of tolerances for those response times. Here's where the big problem that Dallas is talking about hits you: Where does that list of tolerances come from? It takes work to make that list, and preceding that work is the motivation to make that list. Many companies just don't have that motivation.

I think it's the specter of the difficulty in getting to category 3 that prevents a lot of people from moving into category 2. I think that is Dallas's situation.

A few years ago, I would have listed category 3 at the top of my hierarchy, but at CMG'07, in a paper called "Death to Dashboards...," Peg McMahon and Justin Martin made me aware of another level: this notion of alerting based on variance.

The plan of creating a tolerance for every business task you execute on your system works fine for a few interesting tasks, but the idea doesn't scale to systems with hundreds or thousands of instrumented tasks. The task of negotiating, setting, and maintaining hundreds of different tolerances is just too labor-intensive.

Peg and Justin's paper described the notion that not bothering with individual tolerances works just as well—and with very low setup cost—because what you really ought to look out for are changes in response times. (It's an idea similar to what Robyn Sands described at Hotsos Symposium 2008.) You can look for variances without defining tolerances, but of course you cannot do it without measuring response times.

Dallas ends with:

I think one of the things you might offer as part of the "Performance as a Service" might be assisting customers in developing those performance SLAs, especially since your team is very experienced in knowing what is possible.

I of course love that he made that point, because this is exactly the kind of thing we're in business to do for people. Just contact us through http://method-r.com. We are ready, willing, and able, and now is a great time to schedule something.

There is a lot of value in doing the response time instrumentation exercise, no matter how you do alerting. The value comes in two main ways. First, just the act of measuring often reveals inefficiencies that are easy and inexpensive to fix. We find little mistakes all the time that make systems faster and nicer to use and that allow companies to use their IT budgets more efficiently. Second, response time information is just fascinating. It's valuable for people on both sides of the information supply-and-demand relationship to see how fast things really are, and how often you really run them. Seeing real performance data brings ideas together. It happens even if you don't choose to do alerting at all.

The biggest hurdle is in moving from category 1 to category 2. Once you're at category 2, your hardest work is behind you, and you and your business will have exactly the information you'll need for deciding whether to move on to category 3 or 4.

Cuts, Thinks About Cutting, Part 2

Thank you all for your comments on my prior post.

To Robyn's point in particular, I believe the processes of rehearsing, visualizing, and planning have extraordinarily high value. Visualization and rehearsal are vital, in my opinion, to doing anything well. Ask pilots, athletes, actors, public speakers, programmers, .... Check out "Blue Angels: A Year in the Life" from the Discovery Channel for a really good example.

But there's a time when the incremental value of planning drops beneath the value of actually doing something. I think the location of that "time" and the concept of obsessiveness are related. I'm learning that it's earlier than I had believed when I was younger.

The one sentence in my original blog post that captures the idea I want to emphasize is, "I'm not advocating that design shouldn't happen; I'm advocating that you not pretend your design is build-worthy before you can possibly know."

Thinking of all this reminds me of a "tree house" I wanted to build when I was about 10 years old. I use quotation marks, because it wasn't meant to have anything to do with a tree at all. Here was the general idea of the design:My dad was a pilot, so when I was 10, I was too. The front of this thing was going to have an instrument panel and some windows, and I was going to fly this thing everywhere I could think of.

I drew detailed design drawings, and I dug a hole. (If you have children, by the way, make sure they dig a hole sometime before they grow up. Trust me.) My plan called for a 2x4 frame with plywood walls screwed to that frame. It's when I started gathering the 2x4 lumber that my plan fell apart. See, I had naively planned that 2x4 lumber is 2 inches thick by 4 inches wide. It's not. It's 1-1/2 inches thick by 3-1/2 inches wide. That meant I'd have to redraw my whole plan. Using fractions.

I don't remember exactly what happened next, but I do remember exactly what didn't happen. I didn't redraw my whole plan, and I didn't ever build the "tree house." Eventually, I did finally fill in that hole in the yard.

Here are some lessons to take away from that experience:

• Not understanding other people's specs well enough is one way to screw up your project.
• Actually getting started is a sure way to learn what you need to know about those other people's specs.
  
• You learn things in reality that you just don't learn when you're working on paper.
• Not building the "tree house" at all meant that I'd have to wait until I was older to learn that putting wood on (or into) the ground is an impermanent solution.
• The problems that you think during the design phase are your big problems may not be your actual big problems.

Cuts, Thinks About Cutting

This xkcd comic I saw today reminded me of a conversation a former colleague at Oracle and I had some years ago. During a break in one of the dozens of unmemorable meetings we attended together, he and I were talking about our wood shops. He had built several things lately, which he had told me about. I was in the planning phase of some next project I was going to do. He observed that this seemed to be the conversation we were always having. He argued that it would be better for me to build something, have the fun of building it, actually use the thing I had built, and then learn from my mistakes than to spend so much of my time hatching on what I was going to do next.

I of course argued that I didn't want to waste my very precious whatever-it-was that I would mess up if I didn't do it exactly right on the first try. We went back and forth over the idea, until we had to go back to our unmemorable meeting. I remember him shaking his head as we walked back in.

Sometime after we sat down, he passed a piece of paper over to me that looked like this:It said "cuts" with an arrow pointing to him, and "thinks about cutting" with an arrow pointing to me.

Thus was I irretrievably labeled.

He was right. In recent years, I've grown significantly in the direction of acquiring actual experience, not just imagined experience. I'm still inherently a "math guy" who really enjoys modeling and planning and imagining. But most of the things I've created that are really useful, I've created by building something very basic that just works. Most of the time when I've done that, I've gotten really good use out of that very basic thing. Most of the time, such a thing has enabled me to do stuff that I couldn't have done without it.

In the cases where the very basic thing hasn't been good enough, I've either enhanced it, or I've scrapped it in favor of a new version 2. In either case, the end result was better (and it happened earlier) than if I had tried to imagine every possible feature I was going to need before I built anything. Usually the enhancements have been in directions that I never imagined prior to actually putting my hands on the thing after I built it. I've written about some of those experiences in a paper called "Measure once, cut twice (no, really)".

Incremental design is an interesting subject, especially in the context of our Oracle ecosystem, where there is a grave shortage of competent design. But I do believe that a sensible application of the "measure-once, cut-twice" principle improves software quality, even in projects where you need a well-designed relational database model. I'm not advocating that design shouldn't happen; I'm advocating that you not pretend your design is build-worthy before you can possibly know.

Some good places where you can read more about this philosophy include...

• Getting Real, a 91-essay e-book written by 37signals.
• "Extreme programming: a gentle introduction", an introduction to a discipline for building things, wherein you can embrace the advantages of incremental design.
• "Is design dead?", a paper by Martin Fowler that supports the idea that evolutionary design becomes viable in the presence of certain risk-mitigating practices.

A Small Adventure in Profiling

Tonight I'm finishing up some code I'm writing. It's a program that reports on directories full of trace files. I can tell you more about that later. Anyway, tonight, I got my code doing pretty much what I wanted it to be doing, and I decided to profile it. This way, I can see where my code is spending its time.

My program is called lstrc. It's written in Perl. Here's how I profiled it:

23:31:09 $ perl -d:DProf /usr/local/bin/lstrc

The output of my program appeared when I ran it. Then I ran dprofpp, and here's what I got:

23:31:23 $ dprofpp
Total Elapsed Time = 0.411082 Seconds
User+System Time = 0.407182 Seconds
Exclusive Times
%Time ExclSec CumulS #Calls sec/call Csec/c  Name
64.1    0.261  0.261     18   0.0145 0.0145  TFK::Util::total_cpu
18.1    0.074  0.076    176   0.0004 0.0004  TFK::Util::timdat
5.65    0.023  0.052      9   0.0026 0.0058  main::BEGIN
1.72    0.007  0.007   1348   0.0000 0.0000  File::ReadBackwards::readline
0.98    0.004  0.022      6   0.0007 0.0036  TFK::Util::BEGIN
0.74    0.003  0.011     18   0.0002 0.0006  TFK::Util::tim1
0.74    0.003  0.004      6   0.0005 0.0006  ActiveState::Path::BEGIN
0.74    0.003  0.014      7   0.0004 0.0019  Date::Parse::BEGIN
0.74    0.003  0.359      2   0.0015 0.1797  main::process_files
0.49    0.002  0.002      4   0.0005 0.0006  Config::BEGIN
0.49    0.002  0.002    177   0.0000 0.0000  File::Basename::fileparse
0.49    0.002  0.002    176   0.0000 0.0000  File::Basename::_strip_trailing_sep
0.49    0.002  0.002      3   0.0005 0.0005  Exporter::as_heavy
0.49    0.002  0.002      6   0.0003 0.0004  File::ReadBackwards::BEGIN
0.25    0.001  0.002     24   0.0001 0.0001  Getopt::Long::BEGIN

What this says is that the function called TFK::Util::total_cpu accounts for 64.1% of the program's total execution time. The thing you couldn't have known (except I'm going to tell you) is that this program is not supposed to execute the function TFK::Util::total_cpu. At all. It's because I didn't specify the --cpu command line argument. (I told you that you couldn't have known.)

Given this knowledge that my code was spending 64.1% of my time executing a function that I didn't even want to run, I was able to add the appropriate branch around the call of TFK::Util::total_cpu. Then, when I ran my code again, it produced exactly the same output, but its profile looked like this:

23:33:07 $ dprofpp
Total Elapsed Time = 0.150279 Seconds
User+System Time = 0.147957 Seconds
Exclusive Times
%Time ExclSec CumulS #Calls sec/call Csec/c  Name
50.0    0.074  0.076    176   0.0004 0.0004  TFK::Util::timdat
15.5    0.023  0.053      9   0.0026 0.0058  main::BEGIN
4.73    0.007  0.007   1348   0.0000 0.0000  File::ReadBackwards::readline
2.70    0.004  0.022      6   0.0007 0.0036  TFK::Util::BEGIN
2.70    0.004  0.013     18   0.0002 0.0007  TFK::Util::tim1
2.03    0.003  0.004      6   0.0005 0.0006  ActiveState::Path::BEGIN
2.03    0.003  0.013      7   0.0004 0.0019  Date::Parse::BEGIN
2.03    0.003  0.100      2   0.0015 0.0499  main::process_files
1.35    0.002  0.002      4   0.0005 0.0005  Config::BEGIN
1.35    0.002  0.002    177   0.0000 0.0000  File::Basename::fileparse
1.35    0.002  0.002    176   0.0000 0.0000  File::Basename::_strip_trailing_sep
1.35    0.002  0.002      6   0.0003 0.0004  File::ReadBackwards::BEGIN
1.35    0.002  0.002      3   0.0005 0.0005  Exporter::as_heavy
0.68    0.001  0.002     24   0.0001 0.0001  Getopt::Long::BEGIN
0.68    0.001  0.005    176   0.0000 0.0000  File::Basename::basename

Yay.

Let me summarize:

Total Elapsed Time = 0.411082 Seconds — before profiling
Total Elapsed Time = 0.150279 Seconds — after profiling

That's about a 64% improvement in response time, in return for about 30 extra seconds of development work.

Profiling—seeing how your code has spent your time—rocks.

Messed-Up App of the Day

A couple of weeks ago, I returned from the Miracle Oracle Open World 2009 event held in Rødby, Denmark. I go just about every year to this event. This time, I had accepted an invitation from Chris Antognini to speak at his company's TrivadisOPEN event in Zürich. So my travel planning was a little more complicated than it usually is. Instead of finding a simple trip from DFW to Copenhagen and back, this time I had to book a triangle: DFW to Zürich to Copenhagen, and then back to DFW.

I was surprised at how difficult it was to find a good schedule at a good price; it was a lot harder than my normal booking. I ended up finding a suitable enough itinerary at Orbitz. I usually fly American Airlines, but for this trip, the AA flights were much more expensive than I wanted to pay. But I found an itinerary that used British Airways and Air Berlin that I liked. So I booked it.

The trip went just fine. On the morning of the final day of my trip, Dan Norris and I were standing together in Kastrup at the BA ticket counter when the BA agent noticed that I hadn't provided my American Airlines AAdvantage number for the itinerary. (BA and AA are oneworld partners.) So I gave him the number, which he attached to my itinerary, and he informed me that the number would map only to the segments of my flight that I hadn't consumed yet (I had CPH-LHR and then LHR-DFW left to go). I'd need to request credit for my flights prior in the week separately on the web once I got home.

Fine.

So on Monday after I got home, I went to AA.com to register my BA flights for mileage credit. There was a form to fill out. It took a while to type everything into it, because I had to find my ticket numbers, figure out how to isolate the carrier id from the remainder of the ticket number, and then enter date and flight number information for each segment of my trip for which I was requesting credit. It was probably a 10-minute time investment to get the form filled out.

So I reviewed what I had entered, and then I hit the Submit button. Instantly, I got feedback from the page saying that I couldn't request mileage credit right now, that I would have to wait 15 days before requesting mileage credit. So I recoiled a little bit—filling out that form was a bunch of work—and I opened my calendar application to log myself a reminder to go through the process again in 15 days.

There are a couple of problems here:

1. Why did the form force me to enter a bunch of details before telling me that I shouldn't be using this form?
2. Why was it necessary in the first place for me to type in the flight date, flight number, and origin and destination information for each segment of the itinerary? The ticket number should have been enough.

The simple way to fix both these problems is to do what ba.com does: it asks only who I am and what my ticket number is, and it figures out everything else.

Anyway...

The fifteen days passed, and my calendar reminded me to submit my mileage credit form. So I went to work again, gathering my ticket number information, my flight numbers, my dates of travel, and my origin/destination airport codes. I spent another few minutes typing it all in again. Then I clicked Submit. This time: joy. The form said that the submission was complete, and I'd hear back from AA.com via email.

A minute or so later, I got an email in my inbox thanking me for visiting AA.com and confirming that I had issued a mileage credit request. Good: request confirmed.

Then, just a second or two later, I got a second email from AA.com saying this:

Thanks for using AA.com to request mileage credit for your travel on British Airways.

I'm sorry to disappoint you but your transatlantic travel on British Airways (or a BA-ticketed flight) is not eligible for AAdvantage mileage credit. British Airways transatlantic flights to/from the United States are specifically excluded from mileage accrual or redemption in the AAdvantage program.

Grrrr.... I had to work this hard—twice!—just to find out that I wasn't even going to get to do what I wanted to do? Why didn't they tell me this fifteen days ago?!

The first experience was annoying. The second experience took the annoyance to a whole new level.

Here are some application design lessons that this story reinforces:

1. Don't ask for information you can derive.
   
2. Don't separate feedback from its prerequisite input by any more "distance" (measured in either time or user investment) than absolutely necessary. Especially when there's a chance you're going to deny someone his service, don't make him work any harder or wait any longer than absolutely necessary to let him know.

Of these lessons, number one is far and away the most important. If the AA.com site had asked just for the ticket number like ba.com does, the pain of the other problems wouldn't have been that big of a deal.

My Friend Oliver

Oliver Weers has been my friend for several years now. I met him at one of Mogens Nørgaard's fantastic events in Denmark, and it's at these events that I see Oliver once or twice a year.

Oliver is a DBA who works for CSC. Like a lot of people I've met in Denmark, he's a very sharp guy. Like a lot of people I've met who are DBAs, he has hobbies outside of work. I have a lot of fun talking to him, and I enjoy when I get to cross his path.

Oliver is particularly special, because his outside hobby is that he's a rock star.

...Not the "finger quotes" kind of rock star like Don Knuth or Chris Date. I mean Oliver is an actual rock star, like Joe Elliott or David Coverdale. Well, he'd probably be embarrassed by that characterization, but that's where I hope his ship is headed. A lot of people in Europe already know Oliver because of his performance on the TV show "X Factor," which is a Danish show that's similar to "American Idol."

I'm excited for Oliver lately because he has just released his first album called Get Ready. If you like your Whitesnake, I think you'll like Oliver. (He'll be warming up for Whitesnake on December 19th in front of 7,000 people at K. B. Hallen in Copenhagen.)

Here's a fun sample from the X Factor show. Try to hold it together for the first couple of minutes. Remember, it's like American Idol where half the fun is to see how bad it can get. Oliver kicks in at 2:22 to straighten things out.

So if you're interested in the Rock Music, have a look at him. Hit Oliver's MySpace page for a good idea of what he's got. This Calling Out For You video is good, too (including a nice interview på Dansk, with a little operatic demo thrown in there for good measure). Good song, and it's kind of fun to remember that this guy can rock in sqlplus, too.

So, please join me in wishing Oliver the very best of luck. He's worked hard at this for many years—the whole time holding down a pretty tricky job. I hope he'll become a huge overnight success worldwide sometime real soon.