My fundamental question is: what exactly is innovation? We throw this word around all the time - but is there a way to quantify it? Perhaps the number of people who use or rely upon a given technology? Even if something is widely used, is it necessarily innovative (i.e. is that part of the definition)?
I don't think innovation has a final closed-form definition, which might be related to the fact that there is no foolproof closed-form static method or algorithm that can reliably produce it.
Speaking more loosely I'll take a stab. Innovation is a shift in the way things are done that either permits more value to be created with less resources, increases human freedom of choice, or that opens an entirely new avenue of human endeavor.
Innovations can be small or large, but when I think of innovation I usually exclude routine, linear improvements to existing things. For example, I don't consider a car that gets 32mpg instead of 30mpg an "innovation" but a "refinement of an existing innovation." But fuel injection was an innovation, and boosting fuel economy was one result.
One of the characteristics of innovation, I think, is the "leap." An innovation isn't just a purely logical, linear next step derived in a formulaic way from previous steps. There is some new information there, and the fact that it's very hard to say how this new information came into being seems one of the circumstantial characteristics of an innovation.
That is not to say it emerges out of the blue. It does come from study and focus. But it is not plodding and purely logical. There is some "eureka" that's hard to back-trace. I've had innovations come in dreams, or out of the blue in the middle of the day while doing something else.
All that being said, I think it's probably true that overly rigid structures inhibit it. Overwork can also inhibit it, due to its serendipitous and apparently "right brain" nature. Agile and Lean are all about increasing efficiency, and they often work that way, but they also impose a lot of rigidity in certain places.
A lot of rambling and philosophizing I know... trying to say something about innovation does that.
