Friday, June 3, 2011

Why KScope?

Early this year, my friend Mike Riley from ODTUG asked me to write a little essay in response to the question, “Why Kscope?” that he could post on the ODTUG blog. He agreed that cross-posting would help the group reach more people, so I’ve reproduced my response to that question here. I’ll hope to see you at Kscope11 in Long Beach June 26–30. If you develop applications for Oracle systems, you need to be there.

MR: Why KScope?

CM: Most people in the Oracle world who know my name probably think of me as a database administrator. In my heart, I am a software designer and developer. Before my career with Oracle, I worked in the semiconductor industry as a language designer. I wrote compilers for a living. Designing and writing software has always been my professional true love. I’ve never strayed too far away from it; I’ve always found a reason to write software, no matter what my job has been. [Ed: Examples include the Oracle*APS suite and a compiler design project he did for Great West Life in the 1990s, the queueing theory models he worked on in the late 1990s, the Method R Profiler software (Cary wrote all the XSLT code), and finally today, he spends about half of his time designing and writing the MR Tools suite.]

My career as an Oracle performance specialist is really a natural extension of my software development background. It is still really weird to me that in the Oracle market, performance is regarded as a job done primarily by operations people instead of by development people. Developers control at least 90% of the leverage over how fast an application will be able to run. I think that performance became a DBA responsibility in the formative years of our Oracle world because so many early Oracle projects had DBA teams but no professional development teams.

Most of those big projects were people implementing big off-the-shelf applications like Oracle Financial and Manufacturing Applications (which grew into the Oracle E-Business Suite). The only developers that most of those implementation teams had were what I would call nonprofessional developers. Now, I don’t mean people who were in any way unprofessional. I mean they were predominantly businesspeople who had never been educated as software developers, but who’d been told that of course anybody could write computer programs in this new “fourth-generation language” called SQL.

Just about any time you implement a vendor’s highly customizable new application with 20,000+ database objects underneath it, you’re going to run into performance problems. Someone had to attend to those problems, and the DBAs and sysadmins were the only technical people anywhere near the project who could do it. Those DBAs and Oracle sysadmins were also the people who organized the early Oracle conferences, and I think this is where the topic of “performance tuning” became embedded into the DBA track.

The resulting problem that I still see today is that the topic became dominated by “tips and techniques”—lists of tricks that operational people could try to maybe make their systems go a little bit faster. The word “tuning” says it all. I almost never use the word except facetiously, because it’s a cheap imitation of what systems really need, which is performance optimization, which is what designers and developers of software are supposed to do. Even the evolution of Oracle tools for the performance analyst mirrors this post-production tips-and-techniques “tuning” mentality. That’s why most performance management tools you see today are predominantly oriented toward viewing performance from a system resource perspective (the DBA’s perspective), rather than the code path perspective (the developer’s perspective).

The whole key to performance is the application design and development team, especially when you realize that the performance of an application is not just its code path speed, but its overall interaction with the person using it. So many of the performance problems that I’ve found are caused by applications that are just stupid in how they’re designed to interact with me. For example, if you’ve seen my “Messed-up apps” presentation before, you might remember the self-service bus ticket kiosk that made me wait for over a minute while the application tallied the more-than-2,000 different bus trips for which I might want to buy a ticket. That’s an app with a broken specification. There’s nothing that a run-time operations team can do to make that application any fun to use (short of sending it back for redesign).

My goal as a software designer is not just to make software that runs quickly. My goal is also to make applications that are delightful to use. It’s the difference between an application that you use because you must and one that feels like it’s a necessary part of who you are. Making software like that is the kind of thing that a designer learns from studying Don Norman, Edward Tufte, Christopher Alexander, and Jonathan Ive. It’s a level of performance that just isn’t on the menu for operational run-time support staff to even think about, because it’s beyond their control.

So: why Kscope? The ODTUG conferences are the best places I can go in the Oracle market where I can be with people who think and talk about these things. …Or for that matter, who understand that these ideas even exist and deserve to be studied. KScope is just the right place for me to be.


Joel Garry said...

I remember being taught Operations Researchincluded performance tuning, 30 years ago. I went on a tour of a large oil company data center, all star-trekky inside huge blast doors, with a little pdp off to the side, which immediately caught my interest as that was what I was working on at the time. When I asked, the fellow said they used it to watch and probe network load between the supercomputer centers.

word: exifier

Darryl Griffiths said...

Nice post, well written.
Would you say that the availability of better performance & monitoring tools (I won't mention tuning ;-) ) to DBAs, seems to reinforce the fact that Oracle DBAs are having to spend more time catering for bad (sloppy) application design and coding?

Would it be right to say that Oracle themselves see operational performance as being part of the DBA role and way after product design (in a life cycle following ITIL principles). I, however, agree with your perception on the matter. Better informed design = better results.

I'd like to mention that Agile design seems like a great idea (I've not used it, yet), but that the "feedback" step may put too much requirement on the DBA for performance measurements, if the DBA is heavily charged with operational support issues.

If more informative tools were available to designers, better, faster "feedback" would be obtainable.

Cary Millsap said...

Thank you Darryl.

Most of the operational performance talent that I know within Oracle Corporation are people who consider themselves DBAs, who work in teams that tend to visit clients in post-production situations. I don't think that's Oracle's intent as much as it's just not until after trying to go live that Oracle customers themselves tend to pay any attention to performance. That's because it's only then that clients have enough information to discover that they have performance problems.

At the best-run projects I know about, the product development teams have their own DBAs. These are DBAs who are focused on the development process and who have no operational responsibilities to other projects (although they may follow the applications they've helped to build into production).

I'm speaking at Kscope 11 this week about my appreciation for Agile software development methods. There's also a recording of a webcast presentation I did last week, which you can find at Red Gate Software's web site. Two other sessions to look for are Ron Crisco's "Data design for developers—a primer," and Dominic Delmolino's "Is your performance feedback loop a racetrack or a noose?"

I agree wholeheartedly with your final paragraph: "If more informative tools were available to designers, better, faster 'feedback' would be obtainable." That sentiment is the motive for my company's focus on bringing feedback about response time and throughput more elegantly and less expensively into the early phases of software development projects.