Friday, July 31, 2009

Computer Programming as Stochastic Art

Apparently the ancient philosophers used the word art to mean a body of knowledge about a subject that enables the artist to achieve a particular result. Art was broadly applicable to most any organized practice that people might undertake—what today we rather call vocations or professions.

But the ancient philosophers had a nagging problem. While they could easily criticize a shoemaker for failing to successfully create a pair of shoes as being insufficiently skilled and knowledgeable about their art, they had a hard time convicting a doctor of being insufficiently knowledgeable of medicine simply because their patient died. Thus a debate ensued about how to judge an artist’s skill based on some other criteria than the results of their practice.

The ancient’s coined the term, stochastic art to differentiate those arts in which the knowledge and skill of the practitioner could not be measured simply by the results of their work. They were particularly interested in this idea as it relates to the art of rhetoric and oratory (a form with which some Greeks had a famous obsession).

Stochastic means to “guess at.” But in this context it means, “to aim.” The point being that the aim of the artist is as important as the results, even if they miss their target. And that significant credit can be given to the methods they use, regardless of those results.

Aristotle himself used a lucid example when he said that the product of medicine is not health, but the practices that move the patient as far towards health as practical for the circumstances. He went on to say “It is possible to give excellent treatment even to those who can never enjoy sound health.” (Rhetoric 1355b)

So now we’ve got a framework in which we can categorize art forms. Anecdotally it appears that productive artists like blacksmiths and cobblers can be more easily judged based on the product of their work, while problem solvers like mechanics, doctors and politicians need some latitude to recognize their skill and knowledge as distinct from immediate results. One can recognize an excellent oncologist, even if the majority of their patients tend to die.

Thus the light bulb that illuminates my dim wit, and says computer programming, and probably other specializations of IT, may fall into the category of stochastic art. Keys to the distinction are complexity, problem solving, creativity, and an artist-subject relationship atop a substrate of deeply technical subject matter. This perspective provides a possible out from the dilemma of why writing software does not fit well into the model of building a bridge or a sky-scraper. It may also be a useful tool to help explain other problems with IT that we break our pick on every day.

Doctors, like programmers, are people with deep technical knowledge who rely on their experience in a search for specific solutions to problems. They’re constantly looking for solutions in an ever-changing environment. To make things more challenging for the programmer, their art is probably less episodic than that of the doctor. The programmer must deal with the code they write during spurts of creativity and action from the day they write that code forward. This perspective, for example, provides insight into the central tenant of refactoring in agile methods. (As an aside, note that we're not talking about Stochastic Programming here--that's a different animal).

Additionally, some people believe that the majority of IT projects are failures. Typically, when they survey the beneficiaries of a project, those beneficiaries report to be dissatisfied with the results. While the IT professional might have used the proper methods, they may have missed the aim for a variety of reasons external to their control. Just as most oncologists’ patients die, most IT professionals' projects fail?

This isn’t a mea culpa for the programming community’s failures. In Scientific Failure, James Allen writes that: “failure..led to revisions of the conception of an art itself and the associated notions of artistic knowledge and expertise.” What can we learn from looking at IT from the perspective of stochastic arts and analogues like medicine rather than traditional engineering arts?

For example consider the problem of project management in software development. Imagine if a project manager developed gantt charts and task lists for a team of medical professionals attempting to diagnose and treat a patient with a degenerative neurological disorder. Like most software projects the problem solving, testing, diagnostic attempts, treatments, and the patient themselves simply may not cooperate with the best laid plan. And in the case of something like Alzheimer’s, the project is probably destined for failure if you use the obvious success criteria.

But medicine isn’t all ad-hoc. Methodological techniques and protocols exist in medicine that attempt to provide some deterministic quality to the practice. For example, how do doctors put a schedule on their treatments? There’s probably a lot to gain from the perspective of computer programming as a stochastic art. We should look for examples of stochastic arts and non-stochastic arts and compare and contrast them with computer programming. We may learn from that exercise.

Wednesday, July 15, 2009

How Integrated Are Your Customer Experiences?

When I worked in banking I learned a lot about the tricks banks use to coordinate the sale of their products to customers across the bank's various contact points with said customer. The part I worked on was to address the ability of the bank to integrate IT systems to promote products to you via a variety of outlets: printed inserts, prompts to tellers, ATM, the bank's web site, etc. The holy grail was a system in which promotions were not sprayed at you shotgun style, but rather if you said "no" to the ATM prompt for a credit card, the teller would know not to bug you about it when you walked in the branch. Banks also want to individualize their promotions to your lifestyle, net worth, etc., again via a coordinated effort across the bank's various contact points with you. Wells Fargo, for example, is getting really good at this.

Peter Merholz has some more on this in his HBR blog post entitled
How Integrated Are Your Customer Experiences?. Of particular interest to me is the discussion around "Channel-specific organizational silos rarely have incentives to coordinate their activities."

Thursday, July 09, 2009

Chrome OS, the user experience, storage, and software prices

This week started with a bang when Google announced they would create an OS-like version of Chrome designed to displace the traditional OS and initially targeted at netbooks. Given the coverage in the press, the world seems to have a short memory. You couldn't swing a dead cat back in the mid 1990's without hitting a similar strategy--specifically from Sun. Granted, the implementation is different here. Sun wanted to use the Java VM on the hardware and run a web browser on top of it, whereas Google wants to run the web browser on the hardware, and rely on apps on top of that. But the limitations and criteria for success are the same. They will need to convince users that everything they need is on the web. This simply wasn't possible in 1996. Will it be possible now?

This is a nice piece about how new and profitable user experiences can be created from existing technologies. I'm interested in other examples of this. One that pops to mind might be the story of the Ford Mustang.

There are some amazing things happening in storage right now. Prices continue to plummet while simultaneously offering more sophisticated technology. Granted, this is a general trend for any technology sector, but seems to be most disruptive to storage today.

I can't read this piece about intellectual property being free without thinking of software prices. If one could measure software complexity in quantifiable terms (like we do transistors on a processor), the complexity increase in software vs cost would put Moore's Law to shame. There is nothing tangible to prop up prices for any intellectual property (including software). Prices will always fall--deal with it. You only have three choices to sustain growth in a software business: high volume, very low prices (think Apple iPhone apps or MS Office), own a platform (sometimes a prerequisite for choice number 1--think Apple and Microsoft), or bundle software with other high margin offerings (like services--think Oracle or IBM). This natural progression of software prices to near zero is going to be extremely disruptive to the video surveillance industry in the near future.

Wednesday, July 01, 2009

Transformations and team

Seth Godin's Learning from Singer is a nice, in-your-face reminder to reinvent yourself or die. The IT space is littered with the bodies of companies that couldn't do it (like DEC), and some surprisingly that could, like IBM. I'm particularly fond of the IBM example, as I think it's one of the miraculous cultural transformation stories in IT.

Speaking of transformation, two examples of how our (video security) industry is transforming. Milestone's release of a generic IP camera driver and the announcement that LILIN will release an ONVIF camera. These developments raise the bar for openness. They're indicative of things to come. Lowest common denominator interoperability is what makes IT work. The days of “one off” integration that require a business relationship to facilitate interoperability are leaving this business very quickly. Look at the histories of DEC and IBM for some parallels (hint: one of them "got it").

Finally, Nick (via Facebook) had some counterpoints to Alok Srivastava's pragmatic advice on lessons learned in software development.
Nicholas Jost at 2:38pm June 29
1.) Bogus. We've known for years that key contributors have out sized impact on software projects. This has been proven again and again and is a key part of the "master" model.
4.) Scaling is one thing, performance another. Premature optimization is the bane of software development.
7.) This, at last, is a very important concept. More important because it is true that within an organization different software groups are willing to compete for "coolness" and any process that hampers this competition (notice that this is different from "directing" it) will reduce the innovativeness of a company. Small, fairly independent groups that can talk to each other is a good thing.
9.) Also important but may be culturally nuanced. Software engineers aren't that way by training, they are that way by _nature_. Take the computer away, take the diploma away, and they would be chipping on rocks to make better wheels.

Pedantic off.
But I think "team" was referring to the larger group outside the core software development team that includes support, marketing, sales--all those weirdos. They always seem to be getting in the way of the master's craft, but are a necessary evil if you want to do this stuff for a living. Thus Alok's advice that we invite them to the party--teamwork and all that jazz.