Posted by Alan J. Flavell on 05/19/06 11:00

On Fri, 19 May 2006, Jukka K. Korpela wrote:

> The lesson is that using iso-8859-15 instead, in addition to being a
> wrong move in general as Alan explained, would not help against all
> _other_ characters that people may enter, even if it "worked" in
> some circumstances.

I was following-up to a posting which hadn't mentioned that this was a
form submission question, so my initial answer could have been a bit
off-beam.

But, now that I know it's a form submission question, my advice to use
utf-8 is much stronger. Pretty much any currently used browser will
support utf-8 form submission nowadays. The last browser of any
widespread use to cause problems was NN4, and (to the best of my
recollection) that browser would not perform any better with
iso-8859-15 anyway. (Windows-1252 perhaps, but I would not recommend
going that way!).

Worse, NN4 claimed in its Accept-charset to be capable of rendering
utf-8, so the obvious strategem of doing content negotiation on the
browser's Accept-charset is ruled out. In fact, NN4 pretty much
*could* render utf-8 as claimed - but in forms submission it submitted
crap.

Anyhow, the search engines such as google, which in earlier times
supported a wide range of different form submission encodings, have
been using utf-8 as their standard for several years now, so it's
evident that they've concluded this is a viable way to proceed.
I'd be happy to go along with that now.

> You cannot prevent people from entering arbitrary data through your
> form; you can just process it the best you can.

Absolutely...

> If you expect "any characters", then the logical move is to use
> utf-8. Naturally, your form handler then needs to be able to process
> utf-8 encoded data. In practice, you need a suitable library module
> for the job.

Agreed, this is the way to go for all practical purposes - hand
knitted code for this job is sometimes useful for diagnostics, but
for production one should use well-tested libraries/modules.

regards

Oh, http://ppewww.ph.gla.ac.uk/~flavell/charset/form-i18n.html

[Back to original message]