Skip to main content

PHP String Encoding

Introduction​

When working with strings in PHP, understanding character encoding is crucial. Character encoding is the method computers use to store and represent text. Different encoding schemes represent characters using different byte sequences, which can lead to unexpected behavior if not handled properly.

In this tutorial, we'll explore how PHP handles string encodings, common encoding-related issues, and the functions you can use to work with different character sets effectively.

What is Character Encoding?​

Character encoding is a system that assigns numeric values to characters, including letters, numbers, symbols, and control characters. These numeric values are stored as bytes in a computer's memory.

Some common character encodings include:

  • ASCII: The American Standard Code for Information Interchange, which uses 7 bits to represent 128 characters
  • ISO-8859-1 (Latin-1): An 8-bit encoding that represents Western European characters
  • UTF-8: A variable-width encoding that can represent any Unicode character
  • UTF-16: Another Unicode encoding that uses 16 bits for most common characters

PHP's Default Encoding​

PHP doesn't enforce a specific character encoding for strings. Instead, it treats strings as byte arrays, leaving the interpretation of those bytes up to the programmer. However, most modern PHP applications use UTF-8 encoding by default.

Working with Encodings in PHP​

Detecting Encoding​

PHP provides the mb_detect_encoding() function to attempt to determine the encoding of a string:

php
<?php
$string = "Hello, 世界!"; // A string with both ASCII and non-ASCII characters
$encoding = mb_detect_encoding($string, ['ASCII', 'UTF-8', 'ISO-8859-1']);
echo "The string encoding is: $encoding";
?>

Output:

The string encoding is: UTF-8

Converting Between Encodings​

To convert a string from one encoding to another, use the mb_convert_encoding() function:

php
<?php
$string = "こんにちは"; // "Hello" in Japanese
$utf8_string = $string; // Already in UTF-8
$iso_string = mb_convert_encoding($string, 'ISO-8859-1', 'UTF-8');
$utf16_string = mb_convert_encoding($string, 'UTF-16', 'UTF-8');

// Convert back to UTF-8 for display
$back_to_utf8 = mb_convert_encoding($iso_string, 'UTF-8', 'ISO-8859-1');

echo "Original (UTF-8): " . bin2hex($utf8_string) . "
";
echo "ISO-8859-1: " . bin2hex($iso_string) . "
";
echo "UTF-16: " . bin2hex($utf16_string) . "
";
echo "Back to UTF-8: " . $back_to_utf8 . "
";
?>

Output:

Original (UTF-8): e38193e38293e381abe381a1e381af
ISO-8859-1: 3f3f3f3f3f
UTF-16: feff30533093306b3061306f
Back to UTF-8: ?????

Notice that the conversion to ISO-8859-1 resulted in question marks because ISO-8859-1 cannot represent Japanese characters. This is a common issue when converting between encodings with different character support.

Setting the Internal Encoding​

The mb_internal_encoding() function allows you to set the default encoding that PHP's multibyte string functions will use:

php
<?php
// Set the internal encoding to UTF-8
mb_internal_encoding('UTF-8');
echo "Current internal encoding: " . mb_internal_encoding() . "
";

// This affects how mb_* functions behave
$string = "Hello, 世界!";
echo "String length: " . mb_strlen($string) . "
";
echo "Byte length: " . strlen($string) . "
";
?>

Output:

Current internal encoding: UTF-8
String length: 9
Byte length: 13

The difference between mb_strlen() and strlen() highlights why encoding matters. mb_strlen() counts characters, while strlen() counts bytes. In UTF-8, non-ASCII characters like "世" and "界" require multiple bytes.

Common Encoding Issues and Solutions​

Mojibake (Garbled Text)​

Mojibake occurs when text is decoded using a different encoding than the one it was encoded with.

php
<?php
$utf8_string = "こんにちは"; // UTF-8 encoded Japanese text

// Incorrectly treating UTF-8 as ISO-8859-1
$misinterpreted = mb_convert_encoding($utf8_string, 'UTF-8', 'ISO-8859-1');
echo "Original: $utf8_string
";
echo "Mojibake: $misinterpreted
";

// Fix the mojibake
$fixed = mb_convert_encoding($misinterpreted, 'ISO-8859-1', 'UTF-8');
echo "Fixed: $fixed
";
?>

Output:

Original: こんにちは
Mojibake: ã"ã‚"ã«ã¡ã¯
Fixed: こんにちは

Handling Form Input​

When receiving data from forms, it's important to ensure the encoding is handled correctly:

php
<?php
// Assume form data arrives in ISO-8859-1 but we want to use UTF-8
$name = $_POST['name'] ?? "José Martínez"; // Example data

// Convert the input to UTF-8
$name_utf8 = mb_convert_encoding($name, 'UTF-8', 'ISO-8859-1');

// Store or display the properly encoded string
echo "Name: $name_utf8";
?>

Database Interactions​

When working with databases, ensure that both the database connection and tables are configured with the same encoding:

php
<?php
$dbh = new PDO('mysql:host=localhost;dbname=test', 'user', 'password');
// Set the connection encoding to UTF-8
$dbh->exec("SET NAMES utf8mb4");

// Now all data sent to and retrieved from the database will use UTF-8
$stmt = $dbh->prepare("INSERT INTO users (name) VALUES (?)");
$name = "张伟"; // Chinese name
$stmt->execute([$name]);
?>

HTML and Output Encoding​

When outputting HTML, always specify the character encoding in your HTTP headers or meta tags:

php
<?php
// Set HTTP header
header('Content-Type: text/html; charset=UTF-8');

// Or in HTML
echo '<!DOCTYPE html>';
echo '<html>';
echo '<head>';
echo '<meta charset="UTF-8">';
echo '<title>PHP Encoding Example</title>';
echo '</head>';
echo '<body>';
echo '<h1>Hello, 世界!</h1>';
echo '</body>';
echo '</html>';
?>

Best Practices for Handling Encodings​

  1. Use UTF-8 Throughout Your Application: UTF-8 is the most versatile encoding and can handle characters from virtually all languages.

  2. Set Explicit Encodings: Always specify the encoding in HTTP headers, HTML meta tags, and database connections.

  3. Use the mb_ Functions*: Instead of PHP's standard string functions, use the Multibyte String extension functions (mb_*) when working with non-ASCII text.

  4. Validate Input Encoding: When receiving user input, validate or convert it to your application's standard encoding.

  5. Test with Various Languages: Test your application with strings in different languages to ensure proper handling.

Practical Real-World Example: Multilingual Blog System​

Let's create a simple example of a blog system that handles posts in multiple languages:

php
<?php
// Set internal encoding
mb_internal_encoding('UTF-8');

class BlogPost {
    private $title;
    private $content;
    private $language;
    
    public function __construct($title, $content, $language) {
        // Ensure all data is in UTF-8
        $this->title = mb_convert_encoding($title, 'UTF-8');
        $this->content = mb_convert_encoding($content, 'UTF-8');
        $this->language = $language;
    }
    
    public function getTitle() {
        return $this->title;
    }
    
    public function getContent() {
        return $this->content;
    }
    
    public function getLanguage() {
        return $this->language;
    }
    
    public function getSnippet($length = 100) {
        // Safely truncate multibyte string
        if (mb_strlen($this->content) <= $length) {
            return $this->content;
        }
        
        // Find a natural break point
        $breakpoint = mb_strpos($this->content, '. ', $length - 20);
        if ($breakpoint !== false && $breakpoint <= $length) {
            return mb_substr($this->content, 0, $breakpoint + 1);
        }
        
        // Just cut at the specified length
        return mb_substr($this->content, 0, $length) . '...';
    }
    
    public function display() {
        echo '<article lang="' . htmlspecialchars($this->language) . '">';
        echo '<h2>' . htmlspecialchars($this->title) . '</h2>';
        echo '<p>' . htmlspecialchars($this->content) . '</p>';
        echo '</article>';
    }
}

// Example usage with different languages
$posts = [
    new BlogPost(
        'The Power of PHP',
        'PHP is a versatile scripting language that powers many websites...',
        'en'
    ),
    new BlogPost(
        'La Puissance de PHP',
        'PHP est un langage de script polyvalent qui alimente de nombreux sites Web...',
        'fr'
    ),
    new BlogPost(
        '日本語のブログ投稿',
        'これはPHPで作られた多言語ブログシステムのテストです。文字エンコーディングが正しく機能することを確認しています。',
        'ja'
    )
];

// Display the posts
foreach ($posts as $post) {
    $post->display();
    echo '<p>Snippet: ' . htmlspecialchars($post->getSnippet(50)) . '</p>';
    echo '<hr>';
}
?>

This example demonstrates:

  • Converting input to UTF-8
  • Safely handling multilingual text
  • Proper HTML output with language attributes
  • String truncation that respects multibyte characters

Summary​

Understanding character encoding is essential for PHP developers, especially when creating applications that handle international or multilingual content. Key points to remember:

  • Character encoding translates characters to byte sequences for computer storage and transmission
  • UTF-8 is the recommended encoding for modern PHP applications
  • The Multibyte String extension (mb_* functions) provides tools for working with different encodings
  • Always be explicit about the encodings you use in your application
  • Test your application with a variety of languages and character sets

By following these guidelines, you can avoid common encoding-related issues and create robust PHP applications that work correctly with strings in any language.

Additional Resources​

Exercises​

  1. Create a function that safely truncates a UTF-8 string to a specified number of characters (not bytes).
  2. Write a script that detects if a string contains characters from multiple languages.
  3. Build a form that accepts input in multiple languages and properly stores it in a database.
  4. Create a function that normalizes different encodings of the same text to a standard form.
  5. Write a script that compares the performance of standard string functions versus their mb_* counterparts for different types of text.

If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)