Knowtu

This weekend downunder there’s a conference called Australia 2020. We have a fresh new government and a group of luminaries will meet in Canberra to discuss what Australia’s future should be.

Will this actually result in anything? The cynic in me, says no. With the range of idea’s that come out it becomes an opportunity to cherry pick those ideas that align with the party of the days policies and then flag that it was done with full community consultation. But maybe I’m wrong. It just strikes me that high level visions are meaningless without a vision as to specifics and how to get there. Maybe the conference needs a followup with how this should be achieved and what it looks like in specifics. I’ll be interested to see the outcomes and see if they devolve into how we’re going to achieve that vision.

I think politicians have a hard task nowdays, vision means putting a concrete view of how things should be up for discussion which provides an immediate option to kick in the media machine on how it’s not practical, it’s not shared by the community, it’s too expensive, … . My feeling generally is the contrarian view is often over reported as groups of people vigorously agreeing doesn’t sell well. For parties to have a specific strong vision as to how something should be done and achieved becomes in a lot of cases a political liability.

I think this tends to self select large visions that are non controversial, eg World Peace, meaningless platitudes. Specific visions mean it can be cut down. It’s possible to discuss it in specifics. The previous Howard government had a vision of workplace reform but the specifics are what eventually brought them down. The high level vision was sold but the vision of how this was to be achieved was not, so when it came and was delivered it shocked alot of people who felt that it unfair and cheated.

Shipping products I think actually has a lot to learn from politics. A high level vision doesn’t a shipping product make. It’s the first step, not the end result. Getting to the stage of a shipped product is ultimately all about the people. If the team doesn’t have a vision and someone that can drive the vision then I think the product is doomed to failure.

Specific vision is hard though. Leadership means not only the ability to have a vision but to get people to adopt the vision as their own and deliver on it. This is a lesson I’ve seen in action and been learning from my companies CTO. (He put me onto the Silicon Valley Product Group articles which has good writing on leadership and people).

Vision is pretty much the thing that turns a small startup into a successful company but is also something that can break it. When a company’s small and a couple of people the vision is strongly shared. It has to be, the reduction of income, long hours mean that the vision is what drives the ability to make sacrifices. When the company starts to grow and acquire employees they’re not required to adopt the vision, you can’t put it in the contract. Vision I think is like one of those communication node problems, for each extra person the vision has to communicate through, the less it is shared and adopted . I hypothesise it’s one of the reasons to keep a flat management structure in product shipping companies. Companies that are successful can ultimately get a group of people to adopt a vision. That makes things happen. People are enthused and do what needs to be done to deliver on it. If the management or the founder can’t get people to adopt and indeed grow the vision then I think it’s ultimately going to be an unsuccessful company.

I was previously a consultant, senior J2EE consultant for a large multinational services company. I left after about 2 years. I was onsite, the applications I was team lead for were interesting, had a vision which I helped deliver with some cool technology, but ultimately I felt that the company as a whole didn’t have a vision (apart from make money), or at least I wasn’t part of it. I think this is a common theme of some of the developers we have in the company I work for. Refugees from services/consulting/contracting in search of a vision. You give up the consulting and contracting rates but you gain a vision and the ability to shape and grow it.

At my work at the moment we’re trying a change to the product development process. Basically it’s a quick spec with some high level scenarios framing the problem. When this actually becomes concrete, we’ve decided it’s worth doing and the resources available we get all the developers, testers, analysts, managers in the room and storyboard and brainstorm ideas. Iterating between the solution and prototyping and more brainstorming. It’s all about building a shared vision and team. The people in the room must be the people doing the work. There’s no over the fence with a solution. The solution and innovation comes about as the team develops the vision for solving the problem. Sounds obvious but getting this to work smoothly can be very challenging. Smart, passionate people mean that disagreement is a natural and expected outcome. Harnessing that into a shared vision that can be focussed like a laser can be a tricky problem. The joke at the moment is well at least we don’t have a problem with passion.

The problem was that all previously this was too disjoint, people would come late to the party and didn’t share the same vision or thought it should change. Because they didn’t own and share the original vision it meant a lot of rework and too much of a over the fence culture which meant while it was a team, there was too much of the developers, testers, documenters, analysts subteams rather than a single focussed team solving a shared problem. Rework for what should be done was too high and meant friction over the vision.

It will be interesting to see how this goes and it’s effectiveness building and shaping vision. The same can be said of the 2020 summit.

I have 270 feeds I follow, working my way through the full list about every 2-3 days.

After the great book, there’s a couple of things where there’s too much thinking needed.

• A site having a feed where it isn’t automagically detected by Firefox.  This makes subscribing a task of hide and seek trying to find the rss icon which can be anywhere.
• Also if there’s separate rss feeds for sections, on that section include subscriptions for both the section and the full feed.  I often find I have to click around to move up the section tree so I can find the full feed.
• Non full content feeds.   I understand some sites monetise content but rather than opt for partial content, monetise the feed as well.  RSS is all about giving me the ability to consume the web my way.  I don’t mind adds but if I have to click around and leave my feedreader then it breaks my flow.

There’s a bit of talk around the free meme at the moment. It’s something I have an interest in academically and with some app’s I’d like to eventually launch, so i’ll add my thoughts. At this stage I’m busy just learning so I plan to revisit in 6 months and still see if I think the same.

First a bit of background. It’s quite an old meme, and there’s been a good deal of commentary but here’s the articles that I’ve liked.

• Chris Andersons, ‘Free!, Why $0.00 Is The Future Of Business’ article in wired : http://www.wired.com/techbiz/it/magazine/16-03/ff_free?currentPage=all
• Smooth Span (mandatory reading if you’re into cloud computing and SASS!) ‘Does Free Really Have Value’, http://smoothspan.wordpress.com/2008/02/28/does-free-really-have-value/
• And A VC’s earlier article (another great blogger), ‘In Defense of Free’, http://avc.blogs.com/a_vc/2005/07/in_defense_of_f.html

Economic theory has long understood information goods have very different characteristics of physical goods. There’s been very limited chances to test and see this happening in the real world but with the relatively recent arrival of ubiquitous cheap broadband across most of the developed world and the advent of ‘free’ web applications we’re starting to have to consider ‘free’ as a business model. There’s a future shock where the technology starts to catch up to the theory and the ripples are starting to propagate. I suspect there’s some interesting economics PHD’s and masters going on right now :). I think the more disruptive technology is nano/on demand manufacturing which is just getting started, however I think the lessons of ‘free’ around software will help shape the next century and it’s economy in dealing with non scarce and informational goods. If the growth of manufacturing defined the last century I really see this century being defined by information.

So some general traits of information that stand it apart from physical goods.

• The more it’s used the more valuable it becomes (typically although there’s some interesting implications around signal to noise ratio’s).
• It’s infinitely reproducible for trivial costs, they behave differently, the more it’s used the less scarce it tends to become.
• There’s less exclusive access, diminishing scarcity means more people have access and learn about it (watch that noise level rise, reddit?).

There’s some interesting consumer theory changes to take note of under free. Under traditional models relative price changes in goods means that consumers substitute and adopt different goods until they’re equally happy. What happens under ‘free’? Instead of substitution and good selection occurring on the basis of money it shifts to time. People substitute attention, the one thing that is now limited. Scarcity doesn’t disappear but changes to a different factor. This is why I find the semantic web developing meme a harbinger of disrupted markets, at least in the attention and network space. Technologies that can save the consumer time and direct attention are the new old thing. Google case in point, relevant directed advertisements around strong search are all about minimising time to find things. The one thing consumers cannot extend is the amount of time available to them, the steady metronome. In the long term we’re all dead!

Until now free services have only existed in relative isolation and physical constraints like travel and shipping have prevented a real explosion in adoption and growth. Libraries are the best example, free to borrow but interchange is still in most cases based on the physical items. Large scale digitisation is scaring the bejesus out of libraries and publishers everywhere because it’s hard to understand how ‘free’ affects them. Until now academic theory only really had gedankenexperiments but we’re starting to reach a point where wide scale adoption and usage of free services are having significant effects on the economy and testable models. It’s still early days, so like any disruptive technologies expect some bubbles and crashes!

Information is cheap to produce but can rapidly increase by use and value via network effects. So in combination with now relative cheapness to launch a startup and network effects of the web, you can rapidly increase the value of the information a startup trades in. Previously under the 1st boom, the cost of launching was tied up in expensive infrastructure and the economy of scale wasn’t quite there. We’re starting to hit a relative glut of cheap processing time which means the sunk cost for pure web startups has plummeted. Suck it and see wasn’t a widely available option. There’s also the opportunity cost coming into play, bootstrapping a startup verses relatively easy money probably didn’t encourage it either. I think software business models are undergoing a disruptive shift. Under enterprise or wide scale software rollouts it was a ‘winner wins most’ type of market. Natural monopolies were the norm which meant preventing access and not sharing had a strong economic benefit for the company that was currently on top. I suspect we’re starting to see a move away from this. Companies can now gain a natural monopoly by leveraging and taking account of the change to an informational good, (coopertition maybe?). I suspect Microsoft’s move to be a more open company and trying to change it’s culture with it’s yahoo acquisition isn’t purely about competing with a new batch of upstarts, but a dawning realisation that the next growth area is by leveraging and existing in a market defined by the informational good. When the type of good changes you’re forced to change your model to reflect the new reality. There’s only really so long you can use a business model to drive itself, that only really works if you’re a monopoly. I said before information becomes more valuable the more it is used, so unlocking all the proprietary documents and information held in corporate repositories becomes the next growth enabler for their current markets as well. This is why I like Xobni so much. Outlook inboxes across the world are full of information that has so much untapped value that by unlocking some of that, it’s obvious the value that they can provide so they get Bill Gates demoing their product.

Making it free changes the people who benefit, it’s a shakeup for the market and the market is still figuring out how to even understand the new world order. Previously people and organisations that could extract revenue from restricting or mediating access were the winners, ISP’s, big media, installable software companies. It’s much easier to quantify these kind of models, toll based models are easy to understand and work well for scarcity driven resources. ‘Free’ changes this to content and service authors. Ad networks, publishing tool providers, media providers. Serendipitous discovery and alignment of interests becomes a growth driver of the market itself and for the revenues of the companies that can capitalise on it and quantifying value becomes an exercise in modelling.

The great trouble with the model is how do you quantify the value of attention? Is a service around general attention more valuable than a niche one? Click per action (immediate) more valuable than intention (deferred) and click per add? Is Microsoft onto something with their intention based advertising? How do you possibly value this under traditional models? How much value do you put on the API (opensocial for example) that allows your competitors to leverage your talent and information. By spending money to grow yourself, you grow the market and your competitors. I think boards and traditional company management will struggle with this for quite some time and I don’t see any easy answers. It’s a bit gut feel which is why the risk takers at the moment are the ones who can feel the value rather than quantify it. Typical of paradigm changes, but definitely biased to those who can let go of, ignore or not care about traditional corporate valuation. I think inevitably it will still be winner wins most but I do think there’ll be a bigger market for niche players, maybe more an oligopoly than natural monopolies. Maybe this will change over time and is a symptom of just an immature market. Maybe opensocial and integration API’s would never have even been a consideration if the market was not still growing. That’s not to say the market is likely to start to plateau any time soon. My guess is it’s just scrambled up the other side of the chasm in the adoption curve. As I see it at the moment, the startups that are most likely to succeed are those that can introduce or develop new approaches of extracting value from information, new and existing. This was always the case but it’s now much cheaper to do it with a quicker payoff. Would be interesting to try and quantify this.

Not every company and application benefits from free access to information though. The companies that can leverage attention into quantifiable measures, ad networks, analysts, support will benefit most. Where it’s not possible to quantify or too disruptive to the service I expect these applications to be offered as loss leader ancillary services or left as niche providers. Eg If you use gmail as your webmail clients, what are the chances you use google search primarily. Companies that feed on attention, can achieve economies of scope and maximise time for people naturally gravitate to free . Google, Yahoo, RedHat, Automattic, all are able to convert attention to dollars using strategies such as corporate support, ad networks and ‘professional’ upgrades to generate revenue.

For attention, free is like gravity. Why? If something increases in value the more it’s used then the cost to use it must approach zero to encourage people to build and adopt it. Non free is a barrier to entry that limits the growth and adoption of the service. If wikipedia had cost money or required formal qualifications and expertise (ie) time it would never been as successful as it has (compare it against Britannica). Wikipedia itself is the perfect ancillary service, ie driving attention to relevant and potentially very targeted paid services or goods.

Not all information good producers and providers should make their content free though. The more niche the content, the less the need to be free. If the value that the information accrues as more people use it diminishes rapidly, then the benefit of being free diminishes as well. A niche has a much smaller tolerance for noise than something that is for general consumption. Niches also tend to have non transferable information, specific content and solutions don’t lend themselves to application in another niche. The way of ensuring high signal to noise ratios noise is to boost the strength of the signal or reduce the noise. Where something needs a strong signal, which can be achieved by paid researchers, analysts or specific solutions; the large and non transferable costs are more efficiently recouped by having a paid service. Non free information services reduce the noise by introducing something that responds like an ordinary good and responds to scarcity, money. Perception of value is a very important thing. People are less likely to contribute noise as they devalue their own investment by doing so. Pricing is a very important signal to a niche, too low and noise becomes too high devaluing the channel for all and when priced too high reduces the volume of the signal. That’s not to say a general solution can’t be applied to a niche. Ning is a case in point, niche social networks using a general solution. I would say though that the majority of ning networks are still pretty broad. For really specific niches where the value that can be accrued is highly vulnerable to noise, a paid service may be a better option.

I don’t think you need to defend free. In some cases it’s appropriate in others it’s not. It’s counter intuitive for most people and it’s not unexpected as information goods behave differently than any other good people have experience with. A lack of scarcity is hard to understand and comprehend when nothing physical behaves this way.

Hopefully this hasn’t bounced around too much, but I’m trying to bring together a few different strings in my head and understand if there’s a market for an idea I have.

The last bunch of people working on the code were a bunch of crack smoking monkeys. It’s landed in your lap like a wet cat and it’s your job to pull it all together, make it work and ship it yesterday. The entire documentation for the project is a patchy user manual and the people responsible who know what it’s supposed to do have moved on after being told that maybe they’d be more suited to something like, well, knitting or demolition.

I’m going to detail a strategy that I’ve used a couple of times now successfully to help fill in the cracks. I figured it’s probably worth writing up so I remember it and can point it out anonymously. Look, it’s on the internet, it must be authoritative. It won’t give you pristine documentation but I’ve found at least, it will give you a basis to build on for the next few releases. If you’ve got nothing and you can’t stop and write it all up (who can?) you’re not going to be able to fix it all in a single release. You can plan to fill it in though and see the light at the end of the tunnel. First of all you’ve got a project wiki right? Something you and the team can write informal documentation on? If you haven’t go download and install somewhere, and come back later. Ok so on to the story.

My grandma had a cookbook. It wasn’t a hard cover printed manual with a picture of an attractive (or not so attractive) chef gracing the cover. It was a behemoth. It had escaped the binder that contained it. It had devoured other cookbooks. When my grandmother had particularly liked something she had liberated it from it’s uniformity in a published cookbook, ripped it out and crammed it in there. There were handwritten recipes in an unfamiliar hand that I’m sure she had brought from distant ancestors. It had food stains on it. Maybe she thought if you couldn’t read it any more at least the food stains would give an indication of the ingredients. An early form of scratch and sniff maybe? Here’s the rub though, it was disorganised and in a lot of case incomplete but she knew it back to front and in between. Based on her knowledge of the world she could use it to build on and serve consistently high quality meals. When someone at dinner said ‘I must get the recipe for this’. She would know exactly where the recipe was, she’d take it out, rewrite it filling in all the skipped steps and then someone else could make something great.

So nice story? but how does this relate to code and documentation? I’m going to suggest you start building a cookbook. This is where the wiki comes in. A cookbook isn’t complete. It doesn’t contain the thousands of variations on macaroni and cheese, but it does usually have a single recipe which captures the intent.

So in this context, what’s a recipe. Well the way I’ve thought about it is a recipe in a application context usually covers a simple functional area. Eg employee mangement. It will have a UI usually for creating, editing, deleting and updating. So it has a datasource which in most cases is hooked up to a db and may encompass a database and maybe some stored procedures. There’s probably some data integrity constraints or some interesting behaviour. It may be related to another recipe, eg organisation hierarchy. It may have some gotchas, deleting a employee only sets an additional field indicating they’re no longer active but doesn’t actually delete them. There’s no hard and fast rule but you need to come up with something people can add to and build on so that you can start filling in what the system does. I usually create a really simple template.

• Functional Area Description - what does it do, who uses it, when do they use it
• UI - Basic sequence, create with this form, delete with this form, this validation sequence is important
• Data - These are the tables that are involved, this uses this specific algorithm
• Gotchas - General things that might trip someone up. The worse the code the more important this is.

For a first release 50-80% coverage is good. You start to build an understanding of the system. Future specifications actually have a basis and documenters, testers, developers can start adding and reading it. You get a bit more time on the next release because some of it’s already documented and you’re not starting from scratch. You start organising it into categories and breaking it down a bit more. You start adding things like schema diagrams, system UML sequences and data integrity documentation, glossaries, table of contents. Eventually you end up with pretty good documentation. I ‘ve found that development actually gets faster, when someone asks or someone new joins the answer is, it’s on the wiki in the cookbook.

This is a strategy I’ve found helpful, it’s not perfect but it’s often a good start and a good conversation a team can have with familiar concepts.

You can’t say you’ve climbed it until you’ve reached the peak, it doesn’t count unless you ship it. To get to the point where someone pays something for your software, there’s a lot of leeway and like a chess game many small moves that will affect the outcome. You can cut corners, drop features, play with resources, all of which define the history of the application and start adding technical debt. Behind every line of code is debt; wages, premises, capital equipment that someone will eventually pay for.

Technical debt is one of those intangible things that make or breaks an application, sometimes whole companies. Unfortunately it’s one of those things that I think software engineers as a profession don’t do well in conveying the importance and long term effects of. It’s often impossible for outsiders to understand the difference between a quick hack and rigorously tested application as they can appear to be exactly the same thing. Shipping software isn’t a rigorous scientific process and offers a lot of flexibility in getting to and even defining the end point. The call to arms here is that developers must get better at informing users as to costs and importance of technical debt.

Businesses and accountants are already familiar with the fact that the actions of a business can have an affect on the overall success of that business. When a business is acquired there is an allowance for good will. Favourable things the business has done that values the business more than it’s books report. While it can be haggled over, a business with loyal and repeat customers leading to a steady income stream is much more valuable than the one with occasional and sporadic customers despite the fact they may have the same value of assets and liabilities. So most sales, financial and management staff are familiar with the goodwill concept and I find it’s a good basis to sometimes assist in conveying what technical debt is and the affect it can have on their balance sheets. It’s also reasonably intuitive to a wider audience.

I like to think of technical debt as negative goodwill. Carry too much technical debt and the ability to deliver new features and customer satisfaction diminishes, which for a software company, reduces it’s viability and success potential. Technical debt is not an incidental, it’s a critical factor for the long term success of a software company. If your software cannot address customers needs and cannot be delivered in an acceptable time frame or cost then the business entity will soon cease to exist. This is why quantification of technical debt is so important, it may be vague, but until you’ve attempted to understand the costs of a decision, you can’t trade them off in a knowledgeable manner. I think a lot of poorly performing products are a result of people not fully understanding the technical debt they are accruing. Every piece of code has technical debt, this is expected and fine as long as the debt can be serviced by the organisation.

One pertinent example I like to use is that technical debt can spiral out of control just (well kindof) like the sub prime meltdown. There were lost of small loans (modules) which were very risky. Individually they were comparatively small amounts of debt so were manageable and were an acceptable debt. Agencies then bought up and packaged into larger units (made into an application) all of the smaller debts and mixed them with less risky debts which meant that the ability to see and quantify the real cost of the debt was lost. This tends to be true for applications, internally they may be sawdust, glue and paper, but on the outside it looks the same as every other application, so appears less risky that it really is. When conditions changed aversely in the market the true debt and risk became apparent and it all fell apart very quickly in a cascading manner. A good deal of the ramifications of the sub prime crisis extended beyond the US as outside bodies that had previously had good governance and risk management were unable to quantify and understand the real cost and risk of the debt. If you want to avoid a subprime meltdown in your own codebase it’s important to understand, quantify and actively manage technical debt.

So how do you measure technical debt? The first things is to acknowledge that it’s a nebulous concept and it’s really about estimation. You can live with an untested and poorly implemented piece of code or library but it’s going to be a source of errors for customers until you fix it. If you want to quantify the technical debt, start planning the project in your head to fix it in the next release. You’ll end up with costs like; the cost to write the new software in resources and time, the cost of ongoing support and fixes, the cost of developing patches and hotfixes, … . I like to use the thrown estimate model, pick a task, get a few wise heads, and each come up with an estimate in workdays then take the average.

Basic risk management techniques are also useful. Severity x Probability = Cost. Something terrible like a bug that causes 10% of your customers to lose data might be exceedingly rare, but the cost is monumental. Having code that could cause this because of a lack of investment in testing would be a high technical debt cost very few businesses with customers could afford. The more contingencies and planning you can do, the better you’ll be in a position to understand the real tradeoffs and convey that in terms that business understands. It’s also worth noting that you can go overboard, at some point you do need to ship, and not shipping also has a huge cost.

There’s much harder examples that don’t really lend themselves to quantification, for example; what is the technical debt of a product where getting user feedback is too slow, what is the cost of a hacked together and non intuitive user interface? My favourite at the moment, what is the debt of having distributed development teams in different timezones? The main thing is identifying what, if any, debt is being caused. It may not be able to be quantified specifically, but it’s often intuitive to people that something can have a intolerably high cost and must be addressed. That is, if the UI is just rushed out without proper feedback and testing cycle people won’t buy our software as it’s non intuitive and ugly and we’ll get poor reviews. If our install is terrible because we didn’t have the knowledge and didn’t sit down and plan it, then despite the fact our software is great, people won’t ever be able to install it, hence won’t know that.

I find thinking about choices in terms of debt and trying to quantify the costs often helps also cost the solutions. Eg What is likely to happen if there’s poor communication in distributed teams -> disjointed and poorly performing software with a high ongoing maintenance and rework cost -> what is the cost of outfitting a room in each location so people can do daily face to face meetings? and so on. In debt terms it’s just as important to cost the solution which is easier in some cases as it’s specific actions which usually means someone has to pay for it. There’s always the choice of doing nothing and you don’t want to start something if the future debt will be higher than the foreseeable benefits or that you can manage. What is the debt of the decision you are making when you write that line of code or choose that architecture?

This may sound like it’s a lot of overhead but not every decision needs to be quantified. Every dog has his day and every project has points where a decision which will define the success of the software has to be made. It tends to involve meetings with business owners, senior management and users. Going into the meeting with the options and quick dot points about the future costs and debt associated can really help to focus on getting the optimal decision.

I’ve taken over a legacy product in my current role and my job is with a new team to productise the good bits, drop the bad bits and make it so others can deliver applications and modules on top of the core with lower time and costs. It’s got quite a long history and was never really ‘productised’ and is about to become one of the core products of the company. It was developed by developers who were told to ‘just get it done’ which has meant that it’s devolved into a not so good piece of software.

This has got me thinking about the code I’ve worked on over the years which I think I can narrow down to three categorisations.

Bad code doing bad things, I think this comes from junior or naive programmers, the code is not understandable and it does things in what would be considered worst practice.

Good code doing bad things, the code is commented and understandable but it does something that gives you queasy feelings, eg sql injection, intermingling data access and presentation, no source control, no unit tests, quick fixes where no one’s really tried to understand why ….

Good code doing good things, the optimum, the code is well documented, efficient, semantically grokable and it solves the problem with the minimum of code and communicates and encourages good code to be built on top of it. Interestingly it’s usually less code in my experience.

I think over your career you write all three. It’s sometimes hard to admit, but at some point everyone has written bad code, it might not have reached production or you’ve gotten queasy half way through and taken a different approach, or it’s in production right now and it’s keeping you up knowing what a egregious hack it is. So how do you get a team writing good code to do bad things to be one where they write good code doing good things?

For starters start a review, mentoring and design process. I think often the bad things comes out of lack of time, lack of mentoring, lack of knowledge or sometimes lack of passion.  Someone on the team has to understand technical debt and convey this to the business.  People will in most cases try to do the right thing if given the opportunity. The ability to acquire and grow knowledge is a fundamental tenant of software engineering. For most passionate people in the industry they relish the challenge and if given the opportunity will take it with both hands.

Secondly start writing things down, start an operations manual for the dev team, start a wiki, communicate the how and why, build some organisational knowledge. If people coming into the team, especially graduates, can’t see the how then they’ll be doomed to repeat the process and never understand the why except from the hard lessons they’ll experience on their own. The scientific progress of human kind comes from standing on the shoulders of those who came before. If you’re not writing these things down and communicating them then you’re not part of the cycle of knowledge and people will be doomed to reinvention which is just a sad waste.

Play the devils advocate in reviews, don’t give people the answer but ask them how they would solve a problem with the implementation they’ve chosen, use a scenario to describe an extension to the problem or another view of the problem that their solution doesn’t cover. Ask how they could do this more efficiently, what about performance? This of course requires that you understand what it’s meant to do really well.

Always; positive, negative, positive. People respond better to encouragement than criticism, especially when creativity or passion is required to research and derive a solution. You don’t want new people on the team struck like bambi in the headlights.

These are some of the things I’ve found useful in teams I’ve been part of. I’ll be using some of these over the next 6-9 months to run the new team and reset the old habits in the delivery team so it will be interesting to see what works and what doesn’t.

I got this for Christmas, all in all an interesting read. Ultimately trying to distil the story of googles success into a book that appeals to the wider public is tricky but I think the author did a good job. I personally was more interested in the startup phases and how they got funded and going which the book chronicled and really gave the sense of two founders with a vision trying to get the world to see just how compelling it was.

For the most of the book it was on song but at times I felt the author was going too far to simplify and explain things. I think there were a few chapters in the middle that could be dropped or further expounded. 13 Global Googling and 14 April fools I didn’t think added much and seemed out of place in the middle of the book. 15 Porn Cookie Guy would have been more interesting and relevant if the author expounded in more detail google’s involvement with porn and how it fits in their ethos and just how much it is responsible for googles early success, if at all. I was surprised to see a section on click fraud but considering this is one of the key issues for a company that makes the majority of it’s money from advertising was pertinent and a good run-down on the advertising based challenges google faces at the moment. There was some interesting sections on the triumvirate nature of googles management, how hard it was to recruit a CEO both founders liked and how the split classes of stock works. Also interesting was the fact that the IPO system is so heavily slanted to the big financial players and how google tried to usurp the power with their Dutch auction. After reading the book I admire both the founders for staying true to their vision above pretty much everything else and having faith that they were right.

It was the ‘Fully Updated Edition’ but I think on Internet time things move so fast it’s inevitably behind. That said it’s the early times of google that I found most interesting and the author does really well to convey the attitudes and the feeling of the founding and crazy growth of google.

Interesting points of note:

Ultimately the first 100k they received was by an angel (Andy Bechtolsheim) before they even had a company. According to the book he met with the founders, talked, saw the value of the idea (despite the fact others hadn’t seemed to see it) and then wrote a check there and then.

Google had a great product for a while before they were able to commercialise it. At least from what I can see they were forced to commercialise it after passes from the leaders in the field at the time, yahoo, altavista. I get the sense Larry and Sergey believed in their idea and the way to make it happen meant they had to found google.

It seems at least Larry and Sergey are fans of chaotic spaces. I’ve always somewhat disagreed with Joel’s notion that programmers must have quite spaces to work. I think in environments where there’s lots of smart people together you get a boiling pot and together you get great ideas rising to the top. For all intensive purposes I think the next office is equivalent to the next continent. Having people next to each other and chiming in to conversations just results in brilliant ideas.

It’s worth a read, it’s pretty thin so something you can power through on a lazy weekend.

Things have been a bit quite around here lately as we just pushed out a major release of our software and then went into intensive planning sessions for 2 weeks on the next product roadmap.

So how did we do?

Things that went well:

• Continuous Integration, second major release using it, overall it’s really starting to pay dividends as people get used to the tool.
• Bug fixes by removing code. We have quite a large code base now so we were able to sit down and say what should this really do for some strategic bugs. This allowed us to do some cruft cleaning so the tricky code is much more maintainable and understandable.
• Minimal overtime, only three or four nights when people were here to 10-11.
• Full end to end integration of release and automated UI testing using HP quick test pro. (Basically we can hit publish in CC, do full build and package into each configuration upload to the releases share and then have the installs run and all functional UI tests from the testers test plan run)

Things to fix for next time:

• First time we had a product analyst embedded, the engineers we have are pretty good at satisfying and working with users so this rubbed in a few places.
• We had a lot of switching of resources as fires came up on other projects which meant lots of juggling.
• We need to improve our estimation skills and making a more concerted focus on knocking off features and getting them to our internal users for feedback sooner.

Overall I’m pretty happy with this release and we’re about to start some major new work here with larger and distributed teams so it will be interesting to revisit in 6-9 months and see how we went. My gut feeling is things that are minor niggles , like lagging documentation for example, will become much bigger pain points when there’s three or four teams in different time zones trying to meet an aggressive new product road map.

Just a public service announcement, my guess is this particular circumstance will be pretty rare.

If you’ve been running embedded tomcat under a wrapper class you might have some problems when you go to java 1.6 and you watch stdin and stdout.

We ran an embedded instance in our application with a wrapper that communicates via stdin and stdout to our controlling application, piping if you will. It appears that unless you define a root logger the embedded instance will redirect stdout to a null output handler which means you no longer get output. This will mean if you’re writing status outputs to which another application is listening to it will appear to just stop.

try {
    FileHandler fileHandler = new FileHandler(File.createTempFile("embedded-tomcat", ".log").getAbsolutePath(), true);
    fileHandler.setFormatter(new SimpleFormatter());
    logger.addHandler(fileHandler);
    logger.setLevel(Level.ALL);
} catch (IOException e) {
   e.printStackTrace();
}

Also under java 1.6, make sure you redirect standard in when invoking java from a .net 2 process. This previously worked under 1.5 but probably shouldn’t have. My guess is java 1.6 ‘fixed’ a lot of stdin, out stream which meant coincidental behaviour that previously worked no longer does.

See http://www.mail-archive.com/users@tomcat.apache.org/msg36244.html for a thread where someone else was having a similar problem.

Digital Digressions posted an interesting article about insiders and how technology seems to be displacing them. This is a topic close to my heart and I’m going to disagree and say the era of insiders will never end but will merely be a transformation. Why? well first I think it takes an examination of some fundamentals.

In economics there’s a a concept called information asymmetry. Under this theory insiders can use their knowledge of the market and the cost to outsiders of acquiring this knowledge to either better exploit the market or do something like charge a fee or commission to allow outsiders to act like insiders, ala real estate agents. The less open a market or more complicated it is, the more the insider can exploit this and the higher the premium they can charge to outsiders. Lag measures are a symptom of a closed or opaque market in that it takes time for the true state of the market to become exposed to outsiders. Real estate in many cases is a grossly inefficient market which is why the role of the real estate agent exists, an insider who can theoretically maximise profit via inside information. If the real estate market acted independently and house sales were purely a function of house size, block size, dummy variable of a pool, … then you could expect the insiders to disappear as information becomes more abundant and available to all, but that’s not the case. Perfect information is a concept that helps explain this. Perfect information is where all information is perfectly shared between all participants and sellers know as much as buyers. As long as perfect information cannot be obtained insiders will be able to exploit the cost of acquiring knowledge, so will exist and make money from capitalising on experience and inside knowledge of the market. Real estate is not a market that can be captured or explained purely by the numbers. It helps and can act as a dampener on irrational behaviour, but there are things that are not captured purely in closing prices.

In Canberra we have a local property site called allhomes, it has some ridiculous market penetration, they claim that their Nielson independent monitoring is 99% of people who buy property in Canberra visit the site before purchasing. It has information on all sales in Canberra for the last n years, and things like block size, rates, suburb sales info, pictures, etc. It’s great and it’s how we found our home, so at least in this market, there’s some good experience with online property sales. Anecdotally, I’d have to say though that the number and use of real estate agents is not diminishing. The market in Canberra is pretty solid as it’s got some interesting demographics (large public service with a higher than average income) that isolates it from shocks to some extent, so it may not reflect other markets. One thing you do see is that real estate agents list on the site as well as newspaper advertisements. One of allhomes taglines to sellers is ‘before you list, insist’ on listing on allhomes as well. So while I think commissions for insider information is under pressure, sales volume is increasing over time. Buying a house is one of the most stressful things you can do and is a strong discouragement for people to repeat the process. My guess is as the ease of buying and selling increases so will sales volumes but it’s probably too early to tell. In terms of lag measures though, while these sites are now easier to access they’re still lag measures as they reflect the closing price. It takes time to sell a house and good agents can still outperform outsiders using their knowledge and experience of the market. A month can make a huge difference in the closing price of a home. Getting it right when others get it wrong is the best outcome and this is a skill that insiders, investors, good real estate agents have.

One thing to not forget is that this information previously wasn’t available to real estate insiders either. Insiders with their experience and existing knowledge of the market can also capitalise on the new knowledge which allows them to increase their efficiency as well. This means that they in turn can still outperform outsiders and charge a fee for their knowledge. So I don’t see the era of the insider as over but rather inefficient agents being squeezed out and the way agents behave changing. This should make buying a house easier and more efficient and the flip side is in an efficient market typically there are more sales, so insiders who can capitalise on the information, may charge a lower premium, but make up for it in volume now.

This is why I think the future is more in property sites like redfin, where they still have agents (insiders) but because of changes in the market, the agents act differently. One of their staff outlines why they chose to be a redfin agent and highlights the differences between becoming a traditional agent and joining redfin. The point I think speaks volumes is the ‘work cooperatively with fellow agents’. House characteristics mean different things to different people and one of the skills of a real estate agents is intimate knowledge or a neighbourhood, street, house, people that allows them to act as a broker and map buyers to sellers. Previously the market was, an agent has a specific set of houses they sell to everyone, seller driven really, and it was about the stock of houses an agent had access to. The better agents/agencies had better houses and the cycle would repeat, the more they sold the better their access to house stocks. Compensation had to be price/time based which meant aggressive competition between agents which didn’t necessarily mean the best outcome for buyers or sellers. This is where I think online, information rich clearing houses are changing the behaviour of insiders. Now everyone has the same information and knowledge of the market, it becomes more about the skills in matching buyers with sellers more directly. Sales becomes more cooperative based as the insiders can use their shared knowledge of the market to outperform the individual and maximise volume with more informed buyers and sellers.

When a market is opaque to outsiders, insiders can exploit this and make more money by charging a higher premium per transaction. The more inefficient the market the more likely this is to occur. Once that information is available to all, the insider looses that advantage and has to modify their behavior. The knowledge they have is not worthless though, and by reusing knowledge and acting in groups they can exploit and use each others knowledge to again exploit the market better than outsiders. The insider doesn’t disappear but is rather transformed like a phoenix.

Another article on Wired describing how real estate agents sell their own home provides interesting information as well.