zend mail character set

[Illasera]

*Failed to set zend mail character setI know its easy, I followed the documentation but i still unable to set my email head/subject/body with the right character-set,I see the mail with gibbrish kinda looking character set (The famous squares and weird symbols).I am using Zend mail(After a lot of hot suggestions from people around here).php files are encoded as 8 bit no bomIncluded header code :

header('Content-Type: text/html;charset=UTF-8');

Here is the code, (params has been changed for the sake of security) What am i doing wrong?

$config = array('auth' => 'login',				'username' => 'valid_username',				'password' => 'valid_password'); $transport = new Zend_Mail_Transport_Smtp('valid_mailserver', $config); $mail = new Zend_Mail('ISO-8859-8', 'UTF-8');$mail->setHeaderEncoding(Zend_Mime::ENCODING_BASE64);$mail->setBodyText('non-english characters text', Zend_Mime::ENCODING_8BIT);$mail->setFrom('valid_sender', 'valid_sender');$mail->addBcc('somemail@hotmail.com', 'somemail@hotmail.com');$mail->setSubject('None english characters text');$mail->send($transport);

[Illasera]

Bumping

[Gex]

Bumping

never used this Zend mail thing before.. but if you are wanting to send emails couldn't you just use the PHP mail() function?

[boen_robot]

php files are encoded as 8 bit no bom

You mean UTF-8 without BOM?I think you should be able to just specify UTF-8 in the constructor, like:

$mail = new Zend_Mail('UTF-8');

and then just skip all other settings, and in particular, omit the second argument of setBodyText(), and omit setHeaderEncoding() completely.And of course, if your message is constructed with data from a database or something, that part should also be in UTF-8. But for a trival, real case...

$config = array('auth' => 'login',				'username' => 'valid_username',				'password' => 'valid_password');$transport = new Zend_Mail_Transport_Smtp('valid_mailserver', $config);$mail = new Zend_Mail('UTF-8');$mail->setBodyText('Text with Кирилица (Cyrillic) in it.');$mail->setFrom('valid_sender', 'valid_sender');$mail->addBcc('somemail@hotmail.com', 'somemail@hotmail.com');$mail->setSubject('Кирилица demo');$mail->send($transport);

[Illasera]

You mean UTF-8 without BOM?I think you should be able to just specify UTF-8 in the constructor, like:

$mail = new Zend_Mail('UTF-8');

and then just skip all other settings, and in particular, omit the second argument of setBodyText(), and omit setHeaderEncoding() completely.And of course, if your message is constructed with data from a database or something, that part should also be in UTF-8. But for a trival, real case...

$config = array('auth' => 'login',				'username' => 'valid_username',				'password' => 'valid_password');$transport = new Zend_Mail_Transport_Smtp('valid_mailserver', $config);$mail = new Zend_Mail('UTF-8');$mail->setBodyText('Text with Кирилица (Cyrillic) in it.');$mail->setFrom('valid_sender', 'valid_sender');$mail->addBcc('somemail@hotmail.com', 'somemail@hotmail.com');$mail->setSubject('Кирилица demo');$mail->send($transport);

Thank you mate!However : Some clients who support encoding to the language am using, Still need to menually declare in their mail settings : utf-8,Is there some ways to pass that?Some people get the mail with the right coding but some need to set menually, I want everyone to get the right settings, Any ideas?

[boen_robot]

I hadn't known about this problem before honestly... all mail clients I've tested worked, but now that I see there may be a problem... if it's plain text, the setBodyText() accepts second and third arguments about the charset and the encoding. In your case, the content is UTF-8 encoded, so the charset should be UTF-8 as well.

$mail->setBodyText('Text with Кирилица (Cyrillic) in it.', 'UTF-8', 'UTF-8');

And the same if you want to send HTML emails.While I haven't tried if this works (like I said, I haven't seen clients that have incorrect behaviour), it should work.BTW, you'll probably want to find yourself an editor with some auto complete features... I only found out about those arguments with NetBeans's PHP auto complete feature.

[Illasera]

I hadn't known about this problem before honestly... all mail clients I've tested worked, but now that I see there may be a problem... if it's plain text, the setBodyText() accepts second and third arguments about the charset and the encoding. In your case, the content is UTF-8 encoded, so the charset should be UTF-8 as well.

$mail->setBodyText('Text with Кирилица (Cyrillic) in it.', 'UTF-8', 'UTF-8');

And the same if you want to send HTML emails.While I haven't tried if this works (like I said, I haven't seen clients that have incorrect behaviour), it should work.BTW, you'll probably want to find yourself an editor with some auto complete features... I only found out about those arguments with NetBeans's PHP auto complete feature.

Now the encoded language doesnt apear at all in that mail client...Allthough in my mail account it does apear.The subject encoded name does seems to apear right, just the mail body has issues... weird huh?

[boen_robot]

Strange... try to omit the third argument.If that too doesn't work, try to specify Zend_Mime::ENCODING_8BIT or Zend_Mime::ENCODING_BASE64 as a value.If even that doesn't work, try to specify the headers by the setHeader method, like

$mail->addHeader('Content-Type', 'text/plain;charset=UTF-8');

[Illasera]

Strange... try to omit the third argument.If that too doesn't work, try to specify Zend_Mime::ENCODING_8BIT or Zend_Mime::ENCODING_BASE64 as a value.If even that doesn't work, try to specify the headers by the setHeader method, like

$mail->addHeader('Content-Type', 'text/plain;charset=UTF-8');

NO NO NO :)I love you mate, Thanks but not working :'( Am clueless...

[boen_robot]

Damn... Then perhaps all of the above, plus base64 encoding on the headers, like:

$config = array('auth' => 'login',				'username' => 'valid_username',				'password' => 'valid_password');$transport = new Zend_Mail_Transport_Smtp('valid_mailserver', $config);$mail = new Zend_Mail('UTF-8');$mail->setHeaderEncoding(Zend_Mime::ENCODING_BASE64);$mail->setBodyText('Text with Кирилица (Cyrillic) in it.', 'UTF-8', Zend_Mime::ENCODING_BASE64);$mail->setFrom('valid_sender', 'valid_sender');$mail->addBcc('somemail@hotmail.com', 'somemail@hotmail.com');$mail->setSubject('Кирилица demo');//Try to uncomment this only if even this doesn't work//$mail->addHeader('Content-Type', 'text/plain;charset=UTF-8');$mail->send($transport);

And if even this fails... what's the mail client anyway? It could be a bug on their part if it only fails with them...

[Illasera]

Damn... Then perhaps all of the above, plus base64 encoding on the headers, like:

$config = array('auth' => 'login',				'username' => 'valid_username',				'password' => 'valid_password');$transport = new Zend_Mail_Transport_Smtp('valid_mailserver', $config);$mail = new Zend_Mail('UTF-8');$mail->setHeaderEncoding(Zend_Mime::ENCODING_BASE64);$mail->setBodyText('Text with Кирилица (Cyrillic) in it.', 'UTF-8', Zend_Mime::ENCODING_BASE64);$mail->setFrom('valid_sender', 'valid_sender');$mail->addBcc('somemail@hotmail.com', 'somemail@hotmail.com');$mail->setSubject('Кирилица demo');//Try to uncomment this only if even this doesn't work//$mail->addHeader('Content-Type', 'text/plain;charset=UTF-8');$mail->send($transport);

And if even this fails... what's the mail client anyway? It could be a bug on their part if it only fails with them...

Nope, NothingWhen i set the encoding menually in the client mail to utf-8Then it displays right, Any idea what changes when i change the encoding set menually?I mean, I know the entire encoding paradigm changes but, What could it be in zend_mail that changes it?

[boen_robot]

It should be the character set (the argument at the constructor and/or second argument at the setBodyText())... for some reason, the mail client is disregarding it, and it switches to its default, which is not UTF-8. That's why I'm asking what's the client... can it give you the raw message it received?

[Illasera]

It should be the character set (the argument at the constructor and/or second argument at the setBodyText())... for some reason, the mail client is disregarding it, and it switches to its default, which is not UTF-8. That's why I'm asking what's the client... can it give you the raw message it received?

The raw message got nothing displayed...But am afried these are just whitespace or invisiable characters, but the context is there...

[boen_robot]

No, by "raw message", I mean the thing where not just the content, but the headers are included. If you have a GMail account, you can notice that for every message, there's "Show original" in the options. THAT is the kind of view I'm talking about.

[Illasera]

No, by "raw message", I mean the thing where not just the content, but the headers are included. If you have a GMail account, you can notice that for every message, there's "Show original" in the options. THAT is the kind of view I'm talking about.

This is the correct mail that i have in my GMAIL client mailbox,But like i said, The problematic mail is OUTLOOK 2007 This is GMAIL original message raw context, I cant find it in outlook 2007 though :)Am i right boan robot cool guy?Thanks in advance.

[boen_robot]

The third argument in setBodyText() should've ensured that Content-Transfer-Encoded is base64. If that's not the case, try to explicitly replace this header, with

$mail->setBodyText('Text with Кирилица (Cyrillic) in it.', 'UTF-8', Zend_Mime::ENCODING_BASE64);$mail->addHeader('Content-Transfer-Encoding', 'base64');

Here's for example a test message I sent myself to GMail via GMail, and therefore the kind of mail to go after:

MIME-Version: 1.0Received: by 10.100.168.14 with HTTP; Sun, 11 Jul 2010 06:28:43 -0700 (PDT)Date: Sun, 11 Jul 2010 16:28:43 +0300Delivered-To: example@gmail.comMessage-ID: <AANLkTime0vqmeoKTm1MB08b5VTBeqWe5ZzGLtKOjsW3x@mail.gmail.com>Subject: =?UTF-8?B?0KLQtdGB0YIg0L3QsCDQutC40YDQuNC70LjRhtCw0YLQsCAoQ3lyaWxsaWMpLg==?=From: Vasil Rangelov <example@gmail.com>To: example@gmail.comContent-Type: text/plain; charset=UTF-8Content-Transfer-Encoding: base64 0J7RidC1INC10LTQuNC9INGC0LXRgdGCINC90LAg0LrQuNGA0LjQu9C40YbQsNGC0LAgKEN5cmlsbGljKS4K

(email address faked to avoid spam bots; I suggest you do the same with your samples)I'll try to do the same with Zend_Mail... now just to prepare the mail server (yes, I'm really keen on resolving this, because it is likely to bite me on the arse pretty soon)...

[boen_robot]

Well, at least with Cyrillic, in both GMail and Outlook 2007, the tests I do pass.

$transport = new Zend_Mail_Transport_Smtp();$mail = new Zend_Mail('UTF-8');$mail->setHeaderEncoding(Zend_Mime::ENCODING_BASE64);$mail->setBodyText('Text with Кирилица (Cyrillic) in it.', 'UTF-8', Zend_Mime::ENCODING_BASE64);$mail->setFrom('example@fromDomain.com', 'From Name');$mail->addTo('example@gmail.com');$mail->setSubject('Кирилица demo');$mail->send($transport);

In my case, the SMTP server doesn't require authentication for local scripts, but that's just an authentication detail. It shouldn't be affecting everything else.What characters are you trying to send? Could you please post some, so that I could see if that fails? I mean, it could be that because I've set up stuff for myself and Cyrillic appropriately, Cyrillic works, but if it's something else, it may not unless everything is truly UTF-8.

[Illasera]

Well, at least with Cyrillic, in both GMail and Outlook 2007, the tests I do pass.

$transport = new Zend_Mail_Transport_Smtp();$mail = new Zend_Mail('UTF-8');$mail->setHeaderEncoding(Zend_Mime::ENCODING_BASE64);$mail->setBodyText('Text with Кирилица (Cyrillic) in it.', 'UTF-8', Zend_Mime::ENCODING_BASE64);$mail->setFrom('example@fromDomain.com', 'From Name');$mail->addTo('example@gmail.com');$mail->setSubject('Кирилица demo');$mail->send($transport);

In my case, the SMTP server doesn't require authentication for local scripts, but that's just an authentication detail. It shouldn't be affecting everything else.What characters are you trying to send? Could you please post some, so that I could see if that fails? I mean, it could be that because I've set up stuff for myself and Cyrillic appropriately, Cyrillic works, but if it's something else, it may not unless everything is truly UTF-8.

Sure mate, However , am not at the office today, tommrow ill be able to, ill bump this topic then.Please note that most outlook 2007 clients , my mail is getting decoded right,However, this client isn`t getting the mail right only from me and another person,But i wish to prevent as many errors as i can, Since i send my mails to over 1000 clients,And one error can represent an entire group of clients with the same error,So for my note here, Most outlook 2007 clients beside this one having no problem with encoding...

[boen_robot]

Oh, so it's not simply Outlook 2007 - the mail client program. It's a client (person) using Outlook 2007... and where is his mail at? hotmail I'm guessing based on your earlier posts? And he's looking his hotmail account from Outlook 2007?

[Illasera]

Oh, so it's not simply Outlook 2007 - the mail client program. It's a client (person) using Outlook 2007... and where is his mail at? hotmail I'm guessing based on your earlier posts? And he's looking his hotmail account from Outlook 2007?

Of course its not simply outlook 2007 (The client (The user)) is using outlook 2007, and the CLIENT has the problem, but with outlook 2007 software (Something about the settings am sure), When an indvidiual have the same software, same mail supplier, and same mail box domain inbox like the rest of the company, and only he has the problem, then its a settings issue, My question is, how to make Zend_mail pass it, Not Everyone who has that problem can solve it , We use a mail server provider for our mailboxes (mail@ourcompany_domain), no hotmail Something private, The same mail provider we use to send out mail via our website.

[boen_robot]

When the mail is not UTF-8, the problem is the regional settings, and there's no way to bypass it short of using UTF-8.But when using UTF-8, that should not be a problem, unless UTF-8 is not applied.Here's also the contents from the headers of the message sent in the above fashion from Zend_Mail to GMail via my mail server (seen from Outlook 2007's message properties window):

Delivered-To: example@gmail.comReceived: by 10.100.168.14 with SMTP id q14cs15053ane;		Sun, 11 Jul 2010 07:49:29 -0700 (PDT)Received: by 10.227.152.18 with SMTP id e18mr7989712wbw.1.1278859769334;		Sun, 11 Jul 2010 07:49:29 -0700 (PDT)Return-Path: <example@fromDomain.com>Received: from fromDomain.com ([###.###.###.###])		by mx.google.com with SMTP id c1si3952476wbb.7.2010.07.11.07.49.28;		Sun, 11 Jul 2010 07:49:29 -0700 (PDT)Received-SPF: neutral (google.com: ###.###.###.### is neither permitted nor denied by best guess record for domain of example@fromDomain.com) client-ip=###.###.###.###;Authentication-Results: mx.google.com; spf=neutral (google.com: ###.###.###.### is neither permitted nor denied by best guess record for domain of example@fromDomain.com) smtp.mail=example@fromDomain.comReceived: from localhost ([127.0.0.1])	by fromDomain.com; Sun, 11 Jul 2010 17:49:34 +0300Message-ID: <30483B93-D31B-4D46-AC78-4F88339EAAC6@fromDomain.com>From: From Name <example@fromDomain.com>To: example@gmail.comSubject: =?UTF-8?B?0JrQuNGA0LjQu9C40YbQsCBkZW1v?=Date: Sun, 11 Jul 2010 17:49:34 +0300Content-Type: text/plain; charset=UTF-8Content-Transfer-Encoding: base64Content-Disposition: inlineMIME-Version: 1.0

If you have the headers in this fashion, it should always work. I don't know what kind of settings could be problematic... except maybe registry settings. If this client is using a registry cleaning program, it's time he reinstalls Windows, and forgets about the registry cleaning program - those things do all sorts of anomalities, and this could be one. There's no way you can counter them.

[Illasera]

When the mail is not UTF-8, the problem is the regional settings, and there's no way to bypass it short of using UTF-8.But when using UTF-8, that should not be a problem, unless UTF-8 is not applied.Here's also the contents from the headers of the message sent in the above fashion from Zend_Mail to GMail via my mail server (seen from Outlook 2007's message properties window):

Delivered-To: example@gmail.comReceived: by 10.100.168.14 with SMTP id q14cs15053ane;		Sun, 11 Jul 2010 07:49:29 -0700 (PDT)Received: by 10.227.152.18 with SMTP id e18mr7989712wbw.1.1278859769334;		Sun, 11 Jul 2010 07:49:29 -0700 (PDT)Return-Path: <example@fromDomain.com>Received: from fromDomain.com ([###.###.###.###])		by mx.google.com with SMTP id c1si3952476wbb.7.2010.07.11.07.49.28;		Sun, 11 Jul 2010 07:49:29 -0700 (PDT)Received-SPF: neutral (google.com: ###.###.###.### is neither permitted nor denied by best guess record for domain of example@fromDomain.com) client-ip=###.###.###.###;Authentication-Results: mx.google.com; spf=neutral (google.com: ###.###.###.### is neither permitted nor denied by best guess record for domain of example@fromDomain.com) smtp.mail=example@fromDomain.comReceived: from localhost ([127.0.0.1])	by fromDomain.com; Sun, 11 Jul 2010 17:49:34 +0300Message-ID: <30483B93-D31B-4D46-AC78-4F88339EAAC6@fromDomain.com>From: From Name <example@fromDomain.com>To: example@gmail.comSubject: =?UTF-8?B?0JrQuNGA0LjQu9C40YbQsCBkZW1v?=Date: Sun, 11 Jul 2010 17:49:34 +0300Content-Type: text/plain; charset=UTF-8Content-Transfer-Encoding: base64Content-Disposition: inlineMIME-Version: 1.0

If you have the headers in this fashion, it should always work. I don't know what kind of settings could be problematic... except maybe registry settings. If this client is using a registry cleaning program, it's time he reinstalls Windows, and forgets about the registry cleaning program - those things do all sorts of anomalities, and this could be one. There's no way you can counter them.

The thing is, I am functioning as my company new web builder(webmaster) Whatever you wanna call it,Am building the company website from scratch,Before me, A Web-building company (A Company in charge of builidng websites), Has built the old one,The thing is, The old website, Used to send mails via our website with the same mail provider, STMP authenticated mail, Probably without Zend_mail, And all the other clients (including the problematic client), received the mail properly encoded.Now here is what we are going to do mate,Am going to send mail via the old website (with the mail system that working properly),And with the new one (That am using with zend_mail), And search for gaps in the settings,How do i reach the outlook 2003/7 Settings page like you did?

[boen_robot]

Right click on the message, and select... I'm not sure about the English translation (I'm using a Bulgarian UI)... it should be something like "Message Properties..." or "Message Options...". It should be the last or second last option of the right click. In the newly opened window, you'll see a read only textarea which contains the message headers.As for why the old systems works... it probably didn't used UTF-8, but used a culture specific encoding. Like I said, those messages are vulnerable to the regional settings of Windows. If you have the wrong one, you'll see other culture's characters instead of the culture specifc ones. Still, because the message system is internal, the admin probably relied on the fact that if there are problems, he'll be able to force clients to adjust their regional settings.For you, this means that if you want to use the old method (as opposed to forcing the client to reinstall his Windows; a thing which would be good, even if it doesn't solve the problem), this means you'll have to not only adjust the "UTF-8" in Zend_Mail to whatever other encoding it has to be, but also that you'll have to use something like iconv to convert every piece (subject, body, etc.) from UTF-8 to the other encoding.

[Illasera]

Right click on the message, and select... I'm not sure about the English translation (I'm using a Bulgarian UI)... it should be something like "Message Properties..." or "Message Options...". It should be the last or second last option of the right click. In the newly opened window, you'll see a read only textarea which contains the message headers.As for why the old systems works... it probably didn't used UTF-8, but used a culture specific encoding. Like I said, those messages are vulnerable to the regional settings of Windows. If you have the wrong one, you'll see other culture's characters instead of the culture specifc ones. Still, because the message system is internal, the admin probably relied on the fact that if there are problems, he'll be able to force clients to adjust their regional settings.For you, this means that if you want to use the old method (as opposed to forcing the client to reinstall his Windows; a thing which would be good, even if it doesn't solve the problem), this means you'll have to not only adjust the "UTF-8" in Zend_Mail to whatever other encoding it has to be, but also that you'll have to use something like iconv to convert every piece (subject, body, etc.) from UTF-8 to the other encoding.

Ok first thing first, The old mail sent HTML files, (I can view source while in the new one i cant , since the new one, I send mails as plain php mail)In the old mail header(The html using mail), In the source i have the following header tags

<html><head><meta http-equiv="Content-Type" context="text/html; charset=UTF-8" />...<mail body></End tags>

While in the new mail, am sending plain php mail so no view source options (I am using php since i am including variables like username/password, and other stuff) (If there is a way to include html syntax in php mail or the other way around, I would be happy to get some pointers on how)*Please note that the subject name is getting encoded right in the all mails (Only problem with the body syntax).**Reminder that old mail sending system and new both use the same mail service provider.***The only thing that seems different is the "Content-type:multipart/mixed", How do i set it , and what does it do?****I also noticed that the old mail used to send HTML files with php parameters, I guess as PRE-procssesing, So (How do i set php variables into html syntax)?Now this is the old mail raw headers

Received: from mail_server_provider ([ip address])by mail_server_proider (domain Mail Server) with ESMTP id URR32721for <sender@domain_name>; Mon, 12 Jul 2010 13:31:21 +0300Received: by mail.server.provider.domain (Postfix, from userid 65534)id 83CAEFEC137; Mon, 12 Jul 2010 13:30:33 +0300 (IDT)To: clien@domain_nameSubject: =?UTF-8?B?157Xkdem16Ig15TXl9eV15PXqSAtINeZ15XXnNeZIDIwMTA=?=From: "=?UTF-8?B?157XqNepIC0g15nXmdeg15XXqiDXkNeZ15vXldeqINeQ15XXodeY16jXnNeZ150sINeZ15nXoNeV16og15HXldeY15nXpywg157XmdeY15Eg15TXmdeZ16DXldeqLCDXmdeZ16DXldeqINek16jXnteZ15XXnSw=?=" <sender@domain_name>Reply-To: "=?UTF-8?B?157XqNepIC0g15nXmdeg15XXqiDXkNeZ15vXldeqINeQ15XXodeY16jXnNeZ150sINeZ15nXoNeV16og15HXldeY15nXpywg157XmdeY15Eg15TXmdeZ16DXldeqLCDXmdeZ16DXldeqINek16jXnteZ15XXnSw=?=" <sender@domain_namel>MIME-Version: 1.0Content-Type: multipart/mixed;boundary="4a3b31380c41cfc2bc69ab19aab7f84e"X-Mailer: mail provider Mime MailerMessage-Id: <20100712103033.83CAEFEC137@mail_server_provider@domain>Date: Mon, 12 Jul 2010 13:30:33 +0300 (IDT)X-Spam-Flag: YESX-Spam-Status: Yes, hits=22.75 required=10.00 tests=HTML_MESSAGE=2.00,HTML_MISSING_CTYPE=2.52,BAYES_99=6.00,SARE_FROM_SPAM_WORD3=0.75,NO_RDNS2=2.75,OTHER=12.8version=3.2.1X-Spam-Level: **********************X-Spam-Checker-Version: SpamAssassin 3.2.1 (1.0) on mail_server_provider@domainX-EsetId: 3F25BB28FC157A692023BB7CF3423E

And the new (zend_mail) using raw headers

Received: from localhost ([ip cansored])		by Domain(Mail server) (Domain Mail Server) with ASMTP id URK76519		for <client@domain>; Mon, 12 Jul 2010 13:50:19 +0300Content-Type: text/plain; charset=UTF-8From: sender@domainSubject: =?UTF-8?B?15HXk9eZ16fXlA==?=Date: Mon, 12 Jul 2010 13:50:12 +0300Content-Transfer-Encoding: Content-Transfer-Encoding: base64Content-Disposition: inlineMIME-Version: 1.0Message-ID: ab114098163ab4f6ba1a5d6a1bd79858X-EsetId: 3F25BB28FC157A692023BB7CF3423E

Also, i need to add some parameter called "content-language" , How do i do that?

[Illasera]

Let us not forget my active topic Last bump