market research mistakes

In SixApart's response to the MT 3.0 feedback fiasco, Mena says:

One of the most valid comments we heard is that the personal licenses do not work well for many people who are currently using Movable Type. This surprised us because in a survey of 2500 people, a whopping 85% of respondents had 5 of fewer weblogs or authors. This help educate our final decisions about the weblog and author limits.

Who was it that thought that surveying 2500 random users of MT would be the best way to gauge user reaction?

You don't just need to know what the random(user) thinks, you need to know what the opinion makers and change agents think--because since Movable Type users are all publishers, with audiences, those people will have an immediate impact on other users with their public reactions. More importantly, they made the mistake of thinking all blogs are the same. They're not. My son's one-author personal blog is qualitatively (not just quantitatively) different from Crooked Timber, which runs on the same software but has fifteen authors. Blogs based on my courseware templates are nothing like journalistic blogs. You need to know the different segments of your audience, and how their response to your ideas varies.

The fact that the response to the new licenses surprised them so much says volumes about how little they understood their users. And what's astonishing about that to me is that in this industry, there's really no excuse for not having ongoing conversations with your market, about all aspects of your product or service. There should be no big surprises in a weblog-enabled company.

What I hate about all of this is that I know the people involved, and I know this wasn't motivated by greed or malice or contempt for their users. I know that. But the whole thing is clearly a consequence of poor communication with users, something that SixApart has been criticized about in the past. (While writing this, I received a trackback ping to my M2M post on the subject from Chuq Von Rospach, who makes some similar points on the communication issue.)

While they may have learned from this (and their quick response yesterday would indicate that they have), it doesn't really matter much at this point. I've been following the ripples from the initial outrage, and the major impact has been for people to be shaken out of the inertia of not wanting to change software packages. The response isn't "I'll never pay a cent for software," it's "if I'm going to pay for software, I'd better shop around a bit and make sure I'm getting the best bang for my buck." Or "I don't like surprises, and I'd rather have a tool where things won't change so unexpectedly."

As a result, people who would never have thought seriously about changing programs (myself included) are now downloading and playing around with alternatives. And with people like Shelley Powers and Mark Pilgrim not only leading the way but also providing tips and tutorials on how to follow them, that genie can't ever be stuffed back into the bottle.

Am I willing to pay for a high-quality software package that does exactly what I want? Of course. But like Jennifer over at ScriptyGoddess, I'm a lot less likely to pay for one that's still going to require me to do a lot of tweaking to get it to do what I want. And in order to get me to feel good about paying for a new version of something when the older version was free, you've really got to make it more, not less attractive. They might have had less backlash if they'd changed the pricing without adding restrictions. Or if they'd added restrictions on commercial licenses and not personal licenses. As it is, they gambled big based on poor research, and lost not only customers, but also good will.

And while I'm grateful for the promise of significant educational discounts, I think the decision not to publish that information publicly is a mistake. If you force people to come after you for the information, you'll lose some of them--particularly when there are other tools that they can explore instead. The most important users for them to target in education right now aren't the institutional purchasers--for them, hundreds of dollars (or even thousands, if the software is important) is not an issue. It's the individual teachers and students who serve as change agents in their organizations. If you put barriers in front of those early adopters, they'll simply go elsewhere. And the timing of the change was awful in that regard, given that so many competitors are emerging right now with viable alternatives.

I really don't want to switch away from MovableType--I've got a huge amount of time and energy invested in learning its ins and outs. But I'm nervous now, and far more aware of the precarious position that dependence on commercial software puts me in. So while I won't jump ship just yet, I'm preparing some lifeboats, and testing the waters in them. I don't want to surprised like this again.

--

Update: Christina Wodtke has an eloquent piece about why she'll probably move her site off MT. I'm collecting a lot of the "why I'm considering a switch" posts over on del.icio.us, as well. It's interesting to me to see how people are thinking out loud about their options.