The problem with software

Software does not wear and tear. It does not break. It does not fade, evaporate or oxidate. Software is not affected by its environment. To update software to its changing environment code needs to be changed, by humans.

Another challenge is we can not proof that the software is programmed correctly. Testing helps, but: program testing can be used to show the presence of bugs, but never to show their absence (Dijkstra, EWD 249).

Software has become complex beyond comprehension. It resembles a dynamic system. Small changes in input can give major changes in output. Code is built on code and turns from structured material into amorphous mass. And we constantly change the input as well as the code.

So we must move to a different model for creating and maintaining software.
What we do today is gather requirements. From the requirements we build the software.
Then we test the software by translating the requirements into test cases (ideally – linking requirements to test cases is not such a generally adopted practice even).

Why not turn requirements in code, automatically? Instead of teaching programmers to create code, we should teach them to generate code. Then we can change the requirements, and regenerate the code. Or it can change/regenerate itself.

A recent article in The Atlantic discusses a coming software crisis. The remediation that is described in this article is to apply formal methods. This basically means we do not code but create a formal, mathematical model of the problem space. This model can then be turned into code. This code can be proven to be correct. The article explains how the existing formal methods, like TLA+, are still immature and there is still a lot of skepticism in the market on its applicability.

Formal methods have been around for a long time, but they never developed beyond an academic context. Translating more complex problems into formal models has proven to be very difficult.

For now, we will have to do with the current discipline. Which could do with more discipline.
Programmers often (must) start coding before understanding the problem, and having a comprehensive idea about the solution. They are caught up in the detail of programming. And modern languages (Java or JavaScript) do not make that easier. These nitty-gritty programming languages leave a lot of detail to be handled by the programmer – making it even more difficult to oversee a program’s complexity.

Also, a higher level of reuse should be encouraged. Every day in software you see programmers solving problems that have been solved for decades. Time pressures, and inaccessibility of existing code make programmers build yet another sort routine, search algorithm or data access mechanism. The building libraries of abstract, generalized libraries should become a common discipline among development teams.

Adopting a microservices approach (not necessarily the technology) will further help separating functionality, making the software system more flexible and therefore change-able.

I am a big proponent of coding for longevity but that is a very controversial opinion these days. But delivery speed is more important than delivery quality. Because things change so fast. Yes, this is true, but we oftentimes can not tell which code will last for decades and which code is quickly obsolete. So we’d better increase the quality of what’s delivered. And invest in methods and technologies that can help improve quality.

Google’s Documentation Style Guide – helping the writer in the developer

Google have published a style guide for their developers.

It’s a nice guide for all writers of non-fiction.

Some cute things:

On semicolons (I never know when to use) not much help (https://developers.google.com/style/semicolons)

Where to use semicolons is often a matter of judgment. Use them judiciously.

In the resources section the style guide refers to Apple’s and Microsoft’s style guides. (https://developers.google.com/style/resources)

Other companies’ style guides, such as the Apple Style Guide or the Microsoft Manual of Style.

Anthropomorphisms, which I like especially in documentation (makes them read like a novel, if done well), however not recommended by Google (https://developers.google.com/style/anthropomorphism):

Examples

Not recommended: A Delimiter object tells the splitter where a string should be broken.

Recommended: A Delimiter object specifies where to split a string.

Just one I did not understand (https://developers.google.com/style/headings):

Things to avoid in headings

Don’t use the following in headings:

Enjoyable.

Should you learn how to code?

Everyone in IT should know the basics of programming. (Also you should have installed some Linux distribution on your laptop. Even if its your laptop, it increases your understanding of the complexities of managing a platform.)

Program something

Program a crappy little program, you will understand more about the challenges of the programmer.

Derek Sivers wrote a nice article about why you should learn programming. https://sivers.org/prog
For him learning to code means self reliance and basic understanding are the main reasons.

Also experience the joy of building you own computer. I helped my son when he built his gaming computer. I learned at least as much as he did.

What language to start with

Derek recommends to start with HTML and CSS. Well, yes, agree of if you are interested in programming web sites. But HTML is such an ugly programming language. It’s like staring programming with BASIC. Before you know you think GOTO is a great flexible feature. While it actually is kitsch. I would rather start with Python, php or Java.

Build something useful in a very short time. Make it ugly. It doesn’t matter. Improve it step by step. Or start something new.

I like exploring programming languages.

 

I recently played with php. Built a small website showing photo’s from google images using random keywords.
Using MAMP.

Then played with python, programming an interface between Kindle and Evernote. Used Pydev.

Did some Javascript and node-RED.

A lot of Rexx programming. 

And Erlang.  That’s a hard one if you are not familiar with functional programming languages.

I learned a ton. I am always amazed what you can learn FOR FREE on the Internet.

Programming: a profession

Programming, a profession not a monkey task

A few years ago an IT manager said to me: for this programming job, I should be able to hire any monkey from the street.

I told him such an attitude would very quickly ruin his application, if not his entire business.

For an organisation that relies on software so heavily, allowing unmaintainable code to enter your applications is like accepting a loan your will never be able to pay off. You are building up an insurmountable technical debt. You can only hope you have some superb programmers around when the bugs hit the fan.

Programming profession

But for us programmers the problem is also about professionalism.

Coding is a profession. Good programming is a skill.

Some organisations want to have code done for 5$ per hour. Or so. Less than you would pay for a plumber. You would trust a plumber for that rate, why leave a programming job to someone for that rate? You are simply not serious about the problem at hand if you hire like that.

Democratization – the amateur and the pro

There is a tendency to underestimate the importance of skill.
Programming is democratized. That is good. Coding is not something mythical either. Amateurs can do it. Do it well. And enjoy it.

But for the problem in my organisation I need a pro.

Photographing is democratized. But for my wedding photos or for my business brochure I hire a professional photographer. Because he has a number of things extra, which I would summarize a craftsmanship and experience.

If you want stuff done you want good craftsmanship. A specialist you can talk to.

And by the way, ideally they should be at your desk and you at their’s. This is where outsourcing often goes off the track. Too little interaction.

Go pro

Cheap programming may work for throw-away apps. But not for high quality solutions that need to work be maintained for a couple of years or more. That stuff is built for the future. For maintainability.

By an expert.

Innovation: getting comfortable with chaos

First impression: this book is either beyond my intelligence.

People in Rainforests are motivated for reasons that defy traditional economic notions of “rational” behavior.

Had to re-read that sentence a couple of times to grasp its meaning. I hit a few more of these texts in The Rainforest, by Victor W. Hwang and Greg Horowitt.

I was a false start. Now and then the writers fall in the trap of academic writing, and they follow the “misguided lessons you learn in academia” as Jason Fried and David Heinemeier Hansson call it in “Rework” (more on that in another post).

The book looks at psychological, neurological context of forming innovation groups, and what to look at. It touches open many other aspects of inactive environments (rainforests).

There’s a sociological aspect to it that very much speaks to my heart.

As veteran Silicon Valley venture capitalist Kevin Fong says, “At a certain point, it’s not about the money anymore. Every engineer wants their product to make a difference.”

This reminds me of The Soul of a New Machine from Tracy Kidder. Excellent book by the way, a must read for (computer) engineers and other Betas. You will get your soldering iron out.
Anyway in this book also, the goal of money is way out of sight, it is the product that counts. Personal issues are set aside, esthetic issues with respect to the new machine prevail. The team is totally dedicated to creating the new machine. They are in the flow, very similar to the psychological flow that psychology professor Mihaly Csikszentmihalyi, has described in “Flow”. The state in which people (typically athletes talk a lot about pushing themselves into a flow) where conscious thinking and acting disappear and a person gets totally submerged in the activity itself.

Back to the Rainforest, where the authors have found that a social context is key for a innovative rainforest to thrive. It’s not just about creating the brain power, but an entire entrepreneurial context that turns this brainpower into a innovative growing organism. The trick is to create a social environment where cross-fertilization takes place.

“Governments are increasingly seeking to spur entrepreneurial activity across the entire system, not just for large companies. Today, countries are ambitiously seeking to create entire innovation economies.”

 

 

“The biggest invisible bottleneck in innovation is not necessarily the economic desirability of a project, the quality of the technology, or the rational willingness of the customer. The real cost frequently boils down to the social distance between two vastly different parties.”

“Serendipitous networking is essential because, in the real world, it is impossible for a central agent to do everything.”

A lot of word and advice are spent on the topic. Tools are presented as guidelines for achieving such an environment.

“Tool #1: Learn by Doing Tool #2: Enhance Diversity Tool #3: Celebrate Role Models and Peer Interaction Tool #4: Build Tribes of Trust Tool #5: Create Social Feedback Loops Tool #6: Make Social Contracts Explicit”

I am not sure if Hwang and Horowitt prove in their work that a central organization (government) can really steer this. An analytical approach to culture change is something different from a (working) prescriptive culture change. I may be skeptical, but with me are the Fried and Heinemeier again in Rework about culture (in context of an organisation):

“Culture is the byproduct of consistent behaviour. 

…

It isn’t a policy. It isn’t the Christmans party or the company picnic. Those are objects and events, not culture. And it’s not a slogan, either. Culture is action, not words.”

The Rainforest continues and brings together Deming’s approach to maximize quality of product procedures by an organization with the entrepreneurial approach towards innovation. This so serve as a model to evolve innovative, informal and entrepreneurial spirited organizations, a kind of primordial soup into mature structured organization.
(In this soup of entrepreneurial elements, a “flow” should be created igniting an entrepreneurial life form.)

“We surmise that one of the major reasons large corporations often fail at innovation―whether they create venture arms, new product divisions, or otherwise―is because they typically create new business divisions in a formal sense without the “cultural walls” separating the Deming and the Rainforest communities.”

Interestingly this is also what Christensen speaks of in “The Innovators Dilemma”. Christensen makes a similar claim. Organizations fail at innovation because they manage innovation the same way as they do there mature business units. This inherently fails. There is a lot of similarity between the thinking of Christensen and Hwang here. These guys should talk. And invite Fried and Heinemeier to the party.

I conclude managing innovation in an existing (large) organizations can only be successful if it is operated in a completely separate entity. With their own culture that is free to grow, and in a social environment that is not constraint by bureaucratic “efficiencies”.