Jeśli jesteś właścicielem tej strony, możesz wyłączyć reklamę poniżej zmieniając pakiet na PRO lub VIP w panelu naszego hostingu już od 4zł!
Home
Po polsku
 

MIME-Proxy—adding support for national character encodings

Instructions are based on the current (as of time of writing) MIME-Proxy version 2.x: 2.3-5Y released on 10 November 2006 and 2.3.c.1 released on 4 February 2007.

Contents:

  1. General information
  2. Installation and configuration
    1. Installation
    2. Using MIME-Proxy with Xnews and other programs that do not support MIME
      1. Using MIME-Proxy with Eudora for Windows
    3. Using MIME-Proxy with programs that support Unicode and MIME but do not support selected encodings
    4. One-time settings change using X-Mp-headers
    5. Server-specific vs. global settings
  3. Issues
  4. Links.

1. General information

MIME-Proxy uses an iconv.dll library and thanks to it supports all encodings that are registered with IANA link to external site (and a few that are not). It can be used for the three following purposes:

  1. Adding MIME support to a program that lacks any support for encodings. Examples: Xnews, Eudora for Windows. These programs will send out and display correctly only the messages that happen to use encoding that the system uses (for example in case of MS Windows and Western Europe/Americas: Windows-1252). When using Western settings, MIME-Proxy will recode received message to Windows-1252 from whatever was used (ISO-8859-1, ISO-8859-15, even UTF-8–only the characters that exist in Windows-1252). When using Central European settings (system encoding: Windows-1250), MIME-Proxy will recode received message to Windows-1250 from ISO-8859-2, ISO-8859-16 or UTF-8 (also only a subset that exists in WIndows-1250). On systems set for Baltic regional settings (Windows-1257) MIME-Proxy will recode received message to Windows-1257 from ISO-8859-13 (and subset of UTF-8). And so on—you get the picture. When sending, MIME-Proxy will recode message sent from Windows encoding to appropriate Internet encoding, will add required header fields, will also prevent a common error of sending Windows-1252 mislabelled as ISO-8859-1 (with characters on positions 128 – 159 that do not exist in ISO-8859-1 and do not display correctly in many programs, especially the ones that are not working under Windows and are not written by Microsoft). Although when converting from Unicode to Windows encoding Unicode support is for obvious reasons only limited to the characters present in the Windows code page, MIME-Proxy can transliterate certain characters that do not exist in the target Windows encoding.
    When using MIME-Proxy for this purpose, it is necessary to make sure that the program with which MIME-Proxy is working uses appropriate font. If the language does not match the system’s regional settings, it is necessary to choose right script for the font, and if it is not possible, to install a non-Unicode font using required encoding and configure the program to use this font for display. For example you can use a Courier font that can be found on the Windows installation CD-ROM: coure.fon for Western European languages (English, French, German, Spanish etc.), coue1250.fon for Central European languages (Polish, Czech, Slovak, Hungarian, Slovenian etc.), coue1251.fon for Cyrillic-based languages (Russian, Belarussian, Ukrainian, Bulgarian, Serbian), coue1257.fon for Baltic languages (Latvian, Lithuanian).
  2. Adding support for additional encodings that a program does not support. For example popular MS Outlook Express lacks support for ISO-8859-16 used in Romania and (in versions earlier than 6 SP2) for ISO-8859-13 used in Lithuania. MIME-Proxy will recode incoming mesage to UTF-8, and will recode outgoing message from UTF-8 to appropriate ISO-8859 encoding.
  3. A secondary function of MIME-Proxy may be automatic selection of the lowest possible encoding for programs that lack this capacity (OE, Pegasus, Opera, KNode, Sylpheed Claws and more) and/or changing a message’s content-transfer-encoding between Quoted-Printable or Base64 and 8bit, if a party cannot properly decode this encoding or uses old server that still supports only 7-bit SMTP.

The encoding part can be used independently of the decoding part. It is possible to make configuration server-dependent and change settings dynamically by adding special X-Mp-header fields to outgoing messages.

There is one potential problem with using MIME-Proxy: if a message lacks charset declaration (is not MIME compliant), MIME-Proxy will apply a default charset to it which can result in mojibake. It is difficult to reconvert the message to appropriate encoding.

There is no GUI (clickable interface) and it is configured through editing a *.INI file (normally mproxy.ini, but it can have any other legal name). Changes to configuration will be registered after closing and reopening MIME-Proxy. I strongly advise making a backup of mproxy.ini before editing it!

MIME-Proxy is in French and in the distribution package there is a English language file available. Author supplies precompiled versions for Windows and sources that compile under Linux.

2. Installation and configuration

2.1. Installation

You will need two things. One of them is obviously the MIME-Proxy itself: the latest version (as of writing) is 2.3.c.1 for Windows (4.02.2007 r.) and can be downloaded from its author’s website link to external site. If you are installing the program for the first time, you must also download the libiconv link to external site library. You may put this library in the same directory as MIME-Proxy or in the system directory (c:\windows\system, c:\winnt\system32 etc.) in which case it will be available to all other applications that need it to work properly.

2.2. Using MIME-Proxy with Xnews and other programs that do not support MIME

Xnews totally lacks any MIME support whatsoever and its author has declared numeorus times that it won’t be added. Therefore it can basically be used only in an environment in which only US-ASCII is used. It will neither encode nor decode Quoted-Printable in header fields or body. In order to bring postings sent by Xnews to MIME standard and to enable it to display MIME-compliant postings correctly, the following settings are necessary:

  1. Language-neutral settings (to be used for all encodings and local settings)

    MIME-Proxy should not trust charset declaration in outgoing messages as these programs lack proper support for MIME:


    The headers must be encoded to Quoted-Printable as per MIME:


    The message body should be sent as 8bit, not Quoted-Printable:

    (if sending e-mail to a recipient using old-style SMTP server that still supports only 7 bit, use value of 1)

    The message body should be sent as 8bit, not Base64:



    The headers of the incoming message should be recoded to encoding selected for body:


    The headers the incoming message should not be encoded in Quoted-Printable:


    The headers received using XOVER command should be also decoded (for news clients only, enables scoring/filtration):


    Replacement character to be used when a character cannot be recoded in selected 8-bit encoding:



    For Xnews only, MIME-Proxy should change HELO to EHLO:

    No other programs require it and for them it should be set to 0

  2. Western European languages and local encoding (Windows-1252)

    Encodings allowed to be used in outgoing messages:


    Charset used in the composed message (system encoding):


    Charset used in the outgoing message’s headers (system encoding):



    Incoming message to be recoded to this encoding:


    Default charset to be used for messages that lack required charset declaration:

    you can use other encoding as appropriate for the majority of messages that you receive

  3. Central European languages and local encoding (Windows-1250)

    Encodings allowed to be used in outgoing messages:

    For Romanian you can use:


    Charset used in the composed message (system encoding):


    Charset used in the outgoing message’s headers (system encoding):



    Incoming message to be recoded to this encoding:


    Default charset to be used for messages that lack required charset declaration:

    you can use other encoding as appropriate for the majority of messages that you receive

  4. Cyrillic-based languages and local encoding (Windows-1251)

    Encodings allowed to be used in outgoing messages:

    Or:


    Charset used in the composed message (system encoding):


    Charset used in the outgoing message’s headers (system encoding):



    Incoming message to be recoded to this encoding:


    Default charset to be used for messages that lack required charset declaration:

    you can use other encoding as appropriate for the majority of messages that you receive

  5. Baltic languages and local encoding (Windows-1257)

    Encodings allowed to be used in outgoing messages:


    Charset used in the composed message (system encoding):


    Charset used in the outgoing message’s headers (system encoding):



    Incoming message to be recoded to this encoding:


    Default charset to be used for messages that lack required charset declaration:

    you can use other encoding as appropriate for the majority of messages that you receive

2.2.1. Using MIME-Proxy with Eudora for Windows

Old Eudora for Windows (versions 7.1.0.9 and earlier) only partially supports MIME—it supports the Quoted-Printable content-transfer-encoding but always assumes that what is typed is what is to be sent, and what is in the received message is to be displayed using system encoding. This, together with declaring ISO-8859-1 charset in all messages sent, makes it difficult to use it in an environment in which a language other than a Western is used and/or the system is set for regional settings other than Western. In fact Eudora will send Windows-1252 —e.g. will send € as the character no. 0x80, in QP =80; this character does not exist in ISO-8859-1 and in general no ISO-8859 family encoding contains any prontable characters in the 0x80 – 0x9F range. It also ignores charset declaration in received messages. Eudora will work with the settings described above with the single change:

The headers of the incoming message should be encoded in Quoted-Printable:

There is no setting in Eudora options screen that would allow you to select server’s port. It is however possible to set by editing the eudora.ini file: after configuring servers you have to close Eudora, then open the eudora.ini file and find a [persona-XXX] section (XXX is a persona’s name). In each section, as appropriate, enter the following:

After each equals sign write port number.

2.3. Using MIME-Proxy with programs that support Unicode and MIME but do not support selected encodings, Quoted-Printable/Base64 and/or automatic selection of optimal encoding

There are certain programs on the market that support Unicode but lack support for some 8-bit encodings. As an example one of such programs is Microsoft Outlook Express that supports neither ISO-8859-16 nor ISO-8859-13 (in versions 6 SP1 and earlier). Other programs may support numerous encodings but when sending a message in one of 8-bit encodings that contains characters that cannot be encoded in this encoding, they will not warn sender, but rather replace all these characters with question marks or even make mojibake sending them without any recoding. Since MIME-Proxy can recode an incoming message from any encoding to UTF-8 and an outgoing message from UTF-8 to appropriate 8-bit encoding (if possible), it can be used to correct the described flaws. Below are the recommended settings:

  1. Language-neutral settings (to be used for all encodings)

    MIME-Proxy should trust charset declaration in outgoing messages (remember to configure your program properly):


    The headers must be encoded to Quoted-Printable as per MIME:


    The message body should be sent as 8bit, not Quoted-Printable:

    (if sending e-mail to a recipient using old-style SMTP server that still supports only 7 bit, use value of 1)

    The message body should be sent as 8bit, not Base64:



    The headers of the incoming message should be recoded to encoding selected for body:


    The headers the incoming message should be encoded in Quoted-Printable:


    The headers received using XOVER command don’t need to be decoded (for news clients only, they will decode them themselves):


    Replacement character to be used when a character cannot be recoded in selected 8-bit encoding:

    Not really applicable, as all characters can be recoded to UTF-8

    Charset used in the composed message:

    Not really necessary, as the program will send correct charset declaration

    Charset used in the outgoing message’s headers:

    Not really necessary, as the program will send correct charset declaration


    Incoming message to be recoded to this encoding:



    MIME-Proxy should not change HELO to EHLO:

    unless you are sure that your program sends malformed HELO, in which case use 1

  2. Western European languages

    Encodings allowed to be used in outgoing messages:


    Default charset to be used for messages that lack required charset declaration:

    you can use other encoding as appropriate for the majority of messages that you receive

  3. Central European languages

    Encodings allowed to be used in outgoing messages:

    For Romanian you can use:

    support for ISO-8859-16 is not universal

    Default charset to be used for messages that lack required charset declaration:

    you can use other encoding as appropriate for the majority of messages that you receive

  4. Cyrillic-based languages

    Encodings allowed to be used in outgoing messages:

    Or:


    Default charset to be used for messages that lack required charset declaration:

    you can use other encoding as appropriate for the majority of messages that you receive

  5. Baltic languages

    Encodings allowed to be used in outgoing messages:


    Default charset to be used for messages that lack required charset declaration:

    you can use other encoding as appropriate for the majority of messages that you receive

  6. Multilingual environment

    If user sends out messages in varius languages, it is possible to select a list of charsets allowed for outgoing messages appropriate for these languages and appropriate encoding will be selected according to language used (or exactly according to characters used). Since some languages can be encoded using more than one encoding, and some of these encoding are not usually used for the language in question (like ISO-8859-2 for German or ISO-8859-13 for Polish), it is important to remember that when MIME-Proxy works on an outgoing message, it tries firstly to apply the first encoding, if it does not work it tries the second one and so on until it finds the first fitting one or gets to the end of the list (in the latter case it returns an error and will not send the message). The list of encodings should be ordered in such a way that the encoding that can be used for two languages is after the encoding that can be used for one of these languages only. As an example we will use a situation in which we want to send messages in English and German (both using possibly us-ascii, ISO-8859-1 or ISO-8859-15), Polish (using ISO-8859-2) and Lithuanian (using ISO-8859-13), with UTF-8 for messages using special characters that do not exist in any of the ISO-8859 encodings listed. The configuration line will look as follows:



    Thanks to this:

    It mostly works very well but in certain circumstances it is possible that the message will be encoded in unexpected charset. It may happen when the body contains non-typical characters. It is possible to change the list of allowed encodings for a given message by using a special X-Mp-header. See below.

2.4. One-time settings change using X-Mp-headers

The following X-Mp-headers are supported. They will modify MIME-Proxy setting for a given message only. It’s only a selection. Unless indicated otherwise, their value can be either yes or no.

List of encodings allowed to be used:
X-Mp-Charsets: <encoding1>, <encoding2>, <encoding3>, …, <encodingN>

Should the outgoing message be en-/recoded?
X-Mp-Encoding:

Can you trust charset declaration in the message?
X-Mp-Trust-Client:

Should the headers be encoded (RFC2047)?
X-Mp-Encoded-Word:

Should the body be encoded using Quoted-Printable?
X-Mp-Encode-To-Qp:

Should the body be encoded using Base64?
X-Mp-Encode-To-B64:

Encoding used to encode 8-bit characters in headers:
X-Mp-Header-Charset: <charset>r

2.5. Server-specific vs. global settings

It is possible to specify some or all settings on a per-server basis. Global settings are in the section under [default] heading and server-specific are in the section under the [server_alias] heading. Please remember that for e-mail you will have separate server alias for SMTP server (for sending out) and for POP3 server (for receiving); do not mix up settings—the first one will only process the configuration lines that configure behaviour for outgoing messages, and the second one only thse that configure behaviour for incoming mesages.

3. Issues

1. When the outgoing message contains characters that cannot be encoded in any of the charsets allowed, MIME-Proxy will reject the message with error message saying that it cannot encode the article.

2. Transliteration function can transliterate only a limited range of characters from Western European languages (mostly those present in Windows-1252) and therefore is useful only for messages encoded in UTF-8 that contain characters from ISO-8859-1 and recipient’s local encoding. This is actually a limitation of the libiconv library. It is possible to define user’s list of characters to be transliterated and their transliterated symbols but only as a 1:1 character (i.e. it is not possible to transliterate a character to a string of more than one characters, like € to EUR).

3. MIME-Proxy will recognise and recode only standard header fields: From:, To:, Subject: and Organization:. Any other headers that may have been added by a sender (X-headers) will not be reencoded.

4. MIME-Proxy will also recode attachment names. Those encoded according to RFC 2047 link to external site will be converted to raw format (8-bit in system’s encoding). MIME-Proxy does not support RFC 2231 link to external site encoding and will not allow to send a message with attachment which name is encoded this way.

4. Links

Tested on MIME-Proxy 2.3-5Y working with Xnews, Eudora 7.1.0.9, Xananews 1.17.6.6, MPGravity 2.70b (build 2067), Outlook Express 6 SP1, and Opera M2 8.5 under Windows 2000 SP4 English.

licznik