market research mistakes

| 8 Comments | 7 TrackBacks

In SixApart's response to the MT 3.0 feedback fiasco, Mena says:

One of the most valid comments we heard is that the personal licenses do not work well for many people who are currently using Movable Type. This surprised us because in a survey of 2500 people, a whopping 85% of respondents had 5 of fewer weblogs or authors. This help educate our final decisions about the weblog and author limits.

Who was it that thought that surveying 2500 random users of MT would be the best way to gauge user reaction?

You don't just need to know what the random(user) thinks, you need to know what the opinion makers and change agents think--because since Movable Type users are all publishers, with audiences, those people will have an immediate impact on other users with their public reactions. More importantly, they made the mistake of thinking all blogs are the same. They're not. My son's one-author personal blog is qualitatively (not just quantitatively) different from Crooked Timber, which runs on the same software but has fifteen authors. Blogs based on my courseware templates are nothing like journalistic blogs. You need to know the different segments of your audience, and how their response to your ideas varies.

The fact that the response to the new licenses surprised them so much says volumes about how little they understood their users. And what's astonishing about that to me is that in this industry, there's really no excuse for not having ongoing conversations with your market, about all aspects of your product or service. There should be no big surprises in a weblog-enabled company.

What I hate about all of this is that I know the people involved, and I know this wasn't motivated by greed or malice or contempt for their users. I know that. But the whole thing is clearly a consequence of poor communication with users, something that SixApart has been criticized about in the past. (While writing this, I received a trackback ping to my M2M post on the subject from Chuq Von Rospach, who makes some similar points on the communication issue.)

While they may have learned from this (and their quick response yesterday would indicate that they have), it doesn't really matter much at this point. I've been following the ripples from the initial outrage, and the major impact has been for people to be shaken out of the inertia of not wanting to change software packages. The response isn't "I'll never pay a cent for software," it's "if I'm going to pay for software, I'd better shop around a bit and make sure I'm getting the best bang for my buck." Or "I don't like surprises, and I'd rather have a tool where things won't change so unexpectedly."

As a result, people who would never have thought seriously about changing programs (myself included) are now downloading and playing around with alternatives. And with people like Shelley Powers and Mark Pilgrim not only leading the way but also providing tips and tutorials on how to follow them, that genie can't ever be stuffed back into the bottle.

Am I willing to pay for a high-quality software package that does exactly what I want? Of course. But like Jennifer over at ScriptyGoddess, I'm a lot less likely to pay for one that's still going to require me to do a lot of tweaking to get it to do what I want. And in order to get me to feel good about paying for a new version of something when the older version was free, you've really got to make it more, not less attractive. They might have had less backlash if they'd changed the pricing without adding restrictions. Or if they'd added restrictions on commercial licenses and not personal licenses. As it is, they gambled big based on poor research, and lost not only customers, but also good will.

And while I'm grateful for the promise of significant educational discounts, I think the decision not to publish that information publicly is a mistake. If you force people to come after you for the information, you'll lose some of them--particularly when there are other tools that they can explore instead. The most important users for them to target in education right now aren't the institutional purchasers--for them, hundreds of dollars (or even thousands, if the software is important) is not an issue. It's the individual teachers and students who serve as change agents in their organizations. If you put barriers in front of those early adopters, they'll simply go elsewhere. And the timing of the change was awful in that regard, given that so many competitors are emerging right now with viable alternatives.

I really don't want to switch away from MovableType--I've got a huge amount of time and energy invested in learning its ins and outs. But I'm nervous now, and far more aware of the precarious position that dependence on commercial software puts me in. So while I won't jump ship just yet, I'm preparing some lifeboats, and testing the waters in them. I don't want to surprised like this again.


Update: Christina Wodtke has an eloquent piece about why she'll probably move her site off MT. I'm collecting a lot of the "why I'm considering a switch" posts over on, as well. It's interesting to me to see how people are thinking out loud about their options.

7 TrackBacks

Prompted by one of the bazillions of posts on the new MovableType license, I actually went and looked at the new 3.0 licenses, and I've come to the conclusion that the so-called Personal use license is so restrictive that virtually nobody who writes a ... Read More

Wrong End of the Stick from lago at errant dot org on May 18, 2004 4:26 PM

Yes, many people who create web sites using Movable Type are aflame over the licensing schemes that seem to offer... Read More

Six Apart, the people that created Movable Type, the blogware that runs this here site, released the much anticipated MT 3.0 with a licensing agreement radically different from the ones they've ran up until version MT 2.661 ... and all hell broke loose... Read More

Cluetrainwreck from Unlocking the Air on May 20, 2004 2:50 AM

I've been almost obsessively reading all I can stand to regarding reaction to Six Apart's announcement last week of the new p...

Read More

So everyone is weighing in with an opinion about the new licensing scheme for Movable Type 3.0 . Read More

Six Apart's MovableType blogging software is right now probably the single largest of the standalone packages for setting up a weblog of one's own. While services such as Blogger handle a lot of the details for such as hosting for... Read More

Part 2 of 3 I worked at Colgate-Palmolive for four years as the "Technical Writer" for the Consumer Affairs department. I literally wrote the "book" (actually, intranet reference manual) on how to respond to consumer complaints and praises. All "commun... Read More


I was one of those 2500 bloggers who filled out that survey. Since I filled it out, I've added 4 blogs to my MT install. Because it was so simple, didn't cost me anything extra, and because it is, or rather was fun to do with MT. I've been looking around for other blogware for quite a while, because I felt MT wasn't moving forward much anymore, but right now 6A's biggest problem is that MT isn't fun for me anymore because of the way they handled this whole issue.

Nice to see another blogger who caught the SA error in their surveying methodoloy.

SA badly needs to hire someone with marketing clue.

I think you've hit the nail on the head - out there in the webbyworld it's the opinion makers and early adopters that count.

p.s. I trackback pinged you, but it didn't seem to show up on your site.

I need to add a patch so that trackbacks will cause the page to rebuild; when I upgraded to 2.65 I lost that, and have been too lazy to reimplement.

<quote>Who was it that thought that surveying 2500 random users of MT would be the best way to gauge user reaction?</quote>

Probably the same person who thought that whining about making $0.38 per download was going to make us all hally about the new license scheme.

Add me to the list of people amazed at how badly an organization based on person-to-person communications could do such an abysmal job at it.

hally? p'raps "happy" would make more sense.

You raise good points... it's not just about asking the questions to garner research, it's about asking the right questions to get you the info you really need.

One thing though, you mention: "I���ve got a huge amount of time and energy invested in learning its ins and outs."

While it's obviously good to take those things into account, you need to make sure you're making a rational decision and not being too heavily swayed by what you perceive to be "sunk costs".

I believe I took the survey and so they did have at least some data points from fools like me with 20+- blogs and authors.

About this Entry

This page contains a single entry published on May 16, 2004 2:07 PM.

movable type changes was the previous entry in this blog.

how i'm using movable type is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.


Category Archives