Archive for July, 2008

This is why you should let someone experienced do surveys. Or, well, not.

July 1st, 2008

Surveys are complex. There is a lot that you can do wrong. Actually, looking at many surveys around, there is a lot that is done wrong. Time after time.

Sometime it’s the big stuff. Sometimes small.

Sometimes the surveys are not done to get answers, but to show what you want the answers to be, by skewing the questions. That’s bad for academic research, but very popular in politics.

And sometimes you really do want answers. Which is hard to do right. Ask the wrong question, ask them in the wrong way, or give the wrong options for answers, and the results may not say what you think they do, or may be impossible to analyse properly. That’s why there are those who deal professionally with surveys, know the theory (and, hopefully, statistics), have done it many times before, and should be able to avoid most of the mistakes.

They usually don’t do the really big mistakes.

A very long birth yearThey do, however, often do small, or really incomprehensible, mistakes. I guess finding a professional can be a problem as well.

Take, for example, a survey currently being run by iPerceptions , for InforWorld and ComputerWorld.

Both these clients are one client, belonging to the same company. And they do very similar things. So the surveys are practically identical (I did the ComputerWorld one originally, and just now noticed they also run it for InfoWorld. I progressed a little bit, and they’re the same questions in the same order with the same possible answers. Just the name of the company in the survey changed).

This survey has some strange points.

One main problem was that they apparently forgot that some questions may not apply. There was one (maybe 2-3) question where they did have an option to indicate the question is not relevant, or that I don’t know or can’t judge. For all the rest, and there were many of them, I was asked to rank the sites on many criteria, some of which really didn’t interest me and I didn’t know. But the options were just to rank.

Assuming that I’m not the only person who goes through a site that has many different sections, and doesn’t know (or use) all sections, this means that the answers they receive are worthless. What do you pick when you don’t know, or can’t rank? Do you say that it was excellent, since you don’t know it’s bad, causing a potential problematic part to appear good? Do you rank it as very lousy, since it didn’t do anything for you, thereby causing a potentially excellent service to appear bad? Do you rank it in the middle, trying not to judge either way, but still making anything really good, or really bad, seem more average and undeserving of attention?

This is why these things usually contain an answer to state that this question isn’t relevant for you, and you don’t have a real answer for it. But here, no. Good luck to them in the later analysis.

A second point is much less severe, but far more amusing and baffling.

In the personal details, at some point they asked for year of birth. And provided a field to type the year number in. With a maximum of 500 characters. Yes, you read that right, 500 characters to answer the question “In what year where you born?”. They also made the text box large enough to type a small essay in.

What sort of an answer where they expecting? Hmm… Maybe…

That’s a tough question, there. I don’t know what year I was born in. It was a cold, harsh, dark year for my family. My parents were moving a lot. I don’t have no birth certificate, ’cause they were always running from them cops. Who need a stinking certificate? My mom knew I was born. And I had a tough childhood, so people tell me I look 40, but I bet I’m younger. I don’t remember much from those years, really can’t say. Is this important? If it is, I can try remembering, just let me know. Yes?

That’s not a true story (for me, anyway), but it does have exactly 500 characters. For comparison, writing something like 2008, or 1912, takes 4 characters (as does “NOYB“). They could have even been generous, cover all their bases, and give 5-6 characters (You know, for time travellers, or for really really old people). Maybe 3 digits more, for a space followed by “AD”, in case they’re actually worried? A little longer still, so they can get “year of the dragon”? Why the heck 500 characters?

You want to know what’s even more strange? This is in the third part of the survey. In the first part they already asked most personal questions (gender, business, people working in same company, etc), including one about age. But there they just gave several age groups (e.g. 24-35 or something like that), so I suppose they really needed the birth year too.

Then again, if they ask for birth year, why bother asking for age? Odd, that.

Well, I was in a nice mood (this was more amusing than annoying), so I decided I’ll let them know. At the last page of the survey they had a link to provide feedback. I was actually impressed with that, since sometimes I have comments, and nobody official to tell them to. This was nice. Or so I thought.

It was a mailto style link, that contained an email address, and a prepared subject line with the code/number of this survey (Good idea, so they won’t have to wonder what survey it was, and I won’t have to try and describe it too much to ensure they identify it).

There was just one main problem with it. The email address they provided? It wasn’t correct.

I sent a message. I got back a bounce.

<info1@iperceptions.com>:
208.65.144.12 does not like recipient.
Remote host said: 553 mailbox info1@iperceptions.com is restricted (Mode: normal)
Giving up on 208.65.144.12.

Impressive. This is a company that specializes in running surveys. In getting feedback from people for their clients. Except they can’t seem to arrange to get feedback for themselves.

It seems like a typo. The “1″ in the email address does not belong. I checked later on their site, and this address is listed there, without the “1″, in their contact page. But, well, by that time I was out of the helpful mood, and into the annoyed and unimpressed one. Which I think is perfectly understandable.

If you can’t handle bidirectional text, don’t show bidirectional text

July 1st, 2008

Some ad companies think they can get better results by targeting the ads to the viewers. Some strategies are matching the ad with the viewed page, while others try to target the audience in different ways, based on location or language. Which makes sense.

One problem is finding the country of origin of the viewer. Most companies seem to have solved that by pretty accurate geotargeting. Though some, of course, are still stumbling in the dark. For example, as a Jew living in Israel, I still occasionally get ads for Muslim dating sites. Or for various deals which are only relevant to US residents. But these are becoming more rare.

When they do detect a location, the basic step is only to show ads relevant to people from that location. That’s the basic step, which most have been doing (or trying to do) for a while.

Ad with the Hebrew text going backwardThese ads often don’t only change content, but language as well. If the advertised product is sold internationally, people from different countries may pay more attention to ads in their own language[1].

One way to do it is to have a set of pre-made ads, and show them according to the location.

Another way, for those wanting to be more… efficient? is to have a single ad, with several localized text strings that can change inside this ad according to the source.

In theory, it’s nice. There is a need to keep only one copy of a picture, or interactive program, and yet still someone from the US will see English, and someone from, say, France, will see French. The main needed investment is to get the text lines translated into the relevant languages.

And then you have those that go the extra mile (backwards, usually, though) and pick languages that are harder to handle. They do the whole design with languages that go left to right, like English, and then put in right-to-left text, like Hebrew or Arabic.

In many of those cases that I saw, they then forget that the text has to be added to the pictures a little differently. And they don’t bother to show the finished result (calculated ad with the language) to someone who knows the language. They probably just verify the initial text strings, thinking that nothing can go wrong since the same exact text will go into the image.

The end result? Extremely unprofessional advertising, when all the words in the text, or even the whole sentence, go backward, letter by letter. ( !stoidi diputS )

Like this image taken from an ad I saw on several websites. It was on a page together with at least one more different ad, by the same advertiser, that contained the exact same problem.

Did I mention that it looks extremely unprofessional, silly, and pathetic? Because, well, it does. And it definitely gets you thinking that if they managed to screw the ads so bad, on something so basic, what else didn’t they bother to pay attention to, and was it important?

So, the advertising company (the one putting the ads, I don’t know who designed them) is fastclick.net , which redirects to ValueClick Media. Nice name, not so much value to the advertiser.

I thought I’d be nice, and let them know. So I went to their site, got the Contact page, and looked for an email address, or a form. No email address, but there is a contact form. A contact form where the required fields include things like phone number, company, location, how I heard about them, and so on.

This may be alright (OK, not really) for people who are potential customers. But for someone who just wants to do them a favour by dropping a quick helpful note? Completely unacceptable. I shouldn’t have to work, and provide lots of details, just to try and help them.

Required fields should be the message content, and a quick subject. Maybe not even the quick subject. Asking for email address is also fine, if the message may need a follow-up, but that should be left to the discretion of the person sending the message.

And this company is supposed to make money by selling things to people?! By marketing?! That’s supposed to be their strong side? Funny.

---
  1. Personally it annoys me, and I always feel more comfortable when it’s English, rather than Hebrew or other language, if I read on a computer, but I’m really not representative here[back]

Updates, and getting back

July 1st, 2008

After neglecting this blog for too long, I now updated the software (WordPress) to a new version, updated the several plugins it uses, and am also planning to get back to actually using it.

In the meantime, if anything seems broken, or just strange, in the way the blog behaves, please let me know. It can be because of the large amounts of updates.

Real content to follow soon, in more posts…