Arthur C. Clarke famously stated "any sufficiently advanced technology is indistinguishable from magic." It's easy for me to get caught up in the practicalities of the business of software—aligning development resources with strategic plans, executing against a road map, leveraging relevant frameworks, etc. But even perfectly executed software can fall flat and fail to matter. Perfectly executed or haphazardly hacked, the software that really makes an impact is the one that delivers magic.
I remember the first time I got exposed to chromatography data systems. The thing that impressed me most wasn't the instrument control aspects of the system, but rather the way the data from the detector snaked onto the screen and turned into peaks, and magically the system determined where the peaks started and stopped, drew their baselines, and calculated the area under the curve. Even though the method to do so was relatively straightforward, there was an essence of magic in it. It was non-obvious to me.
There is a science behind cool. There are cognitive underpinnings to what makes us react the way I did the first time I saw a chromatogram being drawn in real time. It goes beyond what is aesthetically pleasing or practically functional—it has something to do with the non-obvious nature of it. I get the same reaction when I see an unexpected conclusion emerge from a mass of information, the aha moment you so desperately seek from the promise of big data.
I see popular software and services criticized for not being novel, useful or monetizeable, and yet these services generate a lot of attention. Some of them are successful because they act on the deeply human need for interaction and personal engagement. They facilitate a conversation, perhaps with complete strangers, but with people reacting to what you have to say or share. The closer to real time, the more personal and authentic this interaction, the more it delights us. Some are successful because they facilitate or automate a workflow and let you perform tasks more efficiently than before. But the examples that fascinate me most are the ones that do the unexpected. Maybe they let you apply cool filters to your photos that transform them in a way you didn't anticipate, or they respond to your voice commands with an uncanny accuracy. Visually, through analytics, or in the way data is combined and correlated to achieve an unexpected conclusion, they deliver a magical experience that didn't occur to you a priori.
Over the years, in the many and varied systems I've worked on, there has always been at least one aspect of the system that seemed magical. I admit now that I haven't always credited those magical elements with the value they deserve. Magic can be hard to achieve, and investing in it can be hard to justify. But it can also represent a barrier of entry to your competitors and a delight for your users and customers—it is your purple cow. Another Arther C. Clarke tenet is "the only way of discovering the limits of the possible is to venture a little way past them into the impossible." As technologists, we have the opportunity to build something that ventures into the (seemingly) impossible. Whether or not you consider it a key component of your value proposition, be cognizant of the magic in the systems you build, and know it can be the one thing that makes all the difference in the world.
Saturday, February 11, 2012
You may have read the Inc. article about the best definition of an entrepreneur. Having worked with a number of entrepreneurs, the definition rings true with me: "Entrepreneurship is the pursuit of opportunity without regard to resources currently controlled." Over the course of my career, I frequently find myself acquiring, applying, and "controlling" resources to achieve the goals of the entrepreneur's pursuit. In the most constrained environments, we must gain knowledge, devise capabilities, and figure out a way to succeed while the sands of our investors' money slip through the hourglass. It's this environment where we learn how to experiment, take chances, and challenge our assumptions. Sometimes this leads to significant breakthroughs.
Looking beyond the world of technology and business, constraint appears to be a necessary ingredient in human creativity. We impose constraints as a way to foster creativity, a way to express rich emotion and feelings with a subset of the full language we have at our disposal. Traditional Blues has the AAB lyric constraint and three chords played over a 12 bar scale. Poetic verse is often constrained to a specific meter, form or structure. Haiku is limited to the 5 - 7 - 5. Maybe we impose these constraints to challenge our imagination and express something new while fixing certain fundamental variables.
An article in Psychology Today states that you are more creative if you don't allow your mind to roam free. That's counter-intuitive to say the least, but research indicates this to be the case. After all, Science Fiction writers describe alien life forms as similarly humanoid with remarkable consistency. The article mentions work presented in the book Creativity from Constraints: The Psychology of Breakthrough, suggesting that certain types of constraints promote novel thinking and innovation. Without these constraints, we tend to fall back to past examples, and the variability of our creations are limited. Constraints can rule out specific solutions that have worked before, and foster novel approaches.
This ties back to business, technology and innovation. Without a constrained environment, in a mythical world where all possible solutions were available, you will tend to pick an approach with which you're most comfortable or familiar. While this could result in success (assuming the same conditions for success exist with the new and previous situation), it will not result in a breakthrough. I've seen entrepreneurs fail because they apply the same patterns they used successfully in the past to a new challenge. The presence of constraints is no guarantee for creativity, and creativity is no guarantee for success, but those who support and invest in entrepreneurs often use creativity as an indicator of potential success. They chastise those whose ideas are overly-constrained without realizing that constraint is at the core of creativity. Alas, there are different kinds of constraints.
As innovators, we must learn to embrace constraints. We must be cognizant of the constrained environment in which we operate and explore the full range of possibilities within that field of play. Sometimes you only have 17 syllables to express the wonder of a sunset or the crisp air of a winter morning. Sometimes the limitations you feel are crushing your creativity and suppressing your best work are in fact providing you the opportunity of a lifetime.
Friday, September 02, 2011
Over ten years ago, my colleagues and I at Kenamea had some ideas about lightweight desktop apps and event-driven communication models that would allow them to interact with other apps over the internet. Here are some of the considerations that shaped our thinking.
- To enable truly interactive applications that communicate with other instances of themselves and other applications/services on the internet, the request/response model of the web is insufficient. We need a reliable messaging infrastructure that provides event-driven communication (i.e. apps that can send and receive messages asynchronously), but do so in a way compatible with the protocols and scale of the web.
- One reason web apps succeed is because they are more-or-less client-agnostic (browser compatibility not withstanding), and issues about version control and updates are virtually non-existent.
- Web apps often use a lot of bandwidth re-delivering UI elements that don't change from page to page. I'm not talking about graphical assets that get cached by the browser, I'm talking about web servers resending page markup containing new data when really only the data has changed. A more efficient model for interactive apps is to only send new data to the client, and let client-side logic implement how that data is presented in the UI.
- To be really useful, networked applications should continue to work, albeit in a limited capacity, when there is no network connection. Web applications typically fail in this regard.
- Particularly in conjunction with the above, and to enable store-and-forward messaging, applications need to have non-trivial local persistence. An application-scoped local store of key/value pairs can go a long way to building really interesting client-side apps.
- A facility for installing, registering, and updating apps. Once installed, the app instance is registered with the messaging platform so it can send and receive both point-to-point and topic-based messages. App updates are delivered as system messages, and the client-side framework manages the installation and data migration seamlessly.
I think there's still plenty of room for innovation in this area. Many of the concerns we had in 1999 about interactive, distributed applications are even more valid in the world of connected mobile devices. There is still a need for services such as:
- an easily accessible, internet scalable, bandwidth-friendly messaging infrastructure,
- a robust client-side app framework that provides store-and-forward messaging, app version management, and offline functionality,
- a programming model that promotes bi-directional data exchange between applications and client-side control of presentation (vs. page-oriented request/response).
I can't help but think about how the lessons we learned building a desktop-oriented app framework back then could be applied to the eerily similar mobile app environments of today.