Mojibake

Mojibake[1] is a low-level Unicode 17 text-processing library written in C11 and compatible with C++17. It is released under the MIT License.

It aims to be:

  1. Small
  2. Easy to use
  3. Fast
  4. Self-contained

Mojibake do:

  1. Run in all modern OSes (Linux, macOS, FreeBSD, OpenBSD, NetBSD, Windows 10/11)
  2. Pass the official Unicode test suites for supported algorithms
  3. Implement all Unicode standard algorithms
  4. Satisfy all Unicode Conformance Requirements

Feature highlights

All the C files, together with the Unicode data tables, are concatenated into a single large file and header: mojibake.c and mojibake.h. Zero dependencies.

Text transformation

Text analysis

Sorting and comparison

Security

Integration

Usage

You don't need to install anything. Add the C source and header to your build.

  1. Download it here mojibake-amalgamation-024.zip
  2. Unzip it
  3. Add mojibake.c and mojibake.h to your project

Example:

#include <stdio.h>
#include <string.h>
#include "mojibake.h"

// This is a simple example of how to use the Mojibake library.
int main(int argc, char * const argv[]) {
    printf("This is an example of Mojibake v%s\n", mjb_version());
    printf("Unicode version: %s\n", mjb_unicode_version());

    const char *input = "Cafe\xCC\x81";
    mjb_result result;

    if(mjb_normalize(input, strlen(input), MJB_NORMALIZATION_NFC, MJB_ENC_UTF_8, MJB_ENC_UTF_8,
        &result) != MJB_STATUS_OK) {
        return 1;
    }

    // Output NFC: Café
    // e + ◌́ into é
    printf("NFC: %.*s\n", (int)result.output_size, result.output);

    if(result.transformed) {
        mjb_free(result.output);
    }

    return 0;
}

This library works only in little-endian systems to avoid adding too much overhead. This means that it works on all modern general-purpose CPUs today (x86, x86-64, ARMv8, RISC-V, etc.) It has been tested on:

  1. Linux
  2. macOS
  3. FreeBSD
  4. OpenBSD
  5. NetBSD
  6. Windows 10/11

Build-time features

Mojibake can compile out optional feature tables to reduce binary size. Feature macros default to enabled.

With CMake:

cmake -S . -B build-no-name -DMJB_FEATURE_CHARACTER_NAMES=OFF
cmake --build build-no-name

With the provided Makefile:

make build BUILD_DIR=build-no-name FEATURE_CHARACTER_NAMES=OFF
make test-no-names

API documentation

See API.md for the detailed documentation.

CLI

The src/shell directory builds the mojibake CLI used to test the library. Example usage:

mojibake nfc $'Cafe\u0301'

Plain text output:

Café

Emoji sequence analysis:

mojibake emoji "☺️"
mojibake -c emoji 263A FE0F

Coverage

Mojibake run a total of 1,560,857 tests, including all the official tests included in the standard:

  1. auxiliary/GraphemeBreakTest.txt
  2. auxiliary/LineBreakTest.txt
  3. auxiliary/SentenceBreakTest.txt
  4. auxiliary/WordBreakTest.txt
  5. BidiCharacterTest.txt
  6. BidiTest.txt
  7. CaseFolding.txt
  8. CollationTest/CollationTest_NON_IGNORABLE.txt
  9. CollationTest/CollationTest_SHIFTED.txt
  10. emoji-test.txt
  11. intentional.txt
  12. NormalizationTest.txt
  13. SpecialCasing.txt

Fuzzing

The public API is fuzzed with libFuzzer over untrusted byte input.

make fuzz # 60 seconds by default
make fuzz FUZZ_TIME=300

Building from source

See CONTRIBUTING.md for instructions.

Licenses

Mojibake is released under the MIT License (see LICENSE).

Unicode references

Mojibake's Unicode data and algorithm references are scoped to The Unicode Standard, Version 17.0.0 and the Unicode Character Database 17.0.0. Normative algorithm references here and in https://github.com/zaerl/mojibake/blob/main/API.md use the archived Unicode 17.0.0 versions of the applicable annexes and synchronized technical standards:

Generic Unicode links, when present, are informational or download links rather than normative conformance references.

Unicode Conformance Requirements

Mojibake satisfy the Unicode Conformance Requirements. See CONFORMANCE-REQUIREMENTS.md for details.

Unicode tailoring

Unless listed here, Mojibake applies the referenced Unicode 17.0.0 algorithms without higher-level protocol tailoring.

Unicode conformance inventory

Mojibake interprets Unicode text only through the public APIs and supported UTF encodings listed in this documentation. It does not implement rendering, font shaping, locale collation tailoring, or higher-level protocol behavior beyond the documented locale-sensitive casing and display-width policy. The table below maps the advertised Unicode algorithm and data claims to their Unicode 17.0.0 reference and test evidence.

Claim Public surface Unicode reference Evidence
Unicode Character Database data and derived properties mjb_codepoint_character, mjb_codepoint_property_value, script/block/category/numeric helpers UAX #44, UCD 17.0.0 Generated from UCD data files including UnicodeData.txt, Blocks.txt, Scripts.txt, PropList.txt, DerivedCoreProperties.txt, PropertyAliases.txt, and PropertyValueAliases.txt; covered by local UCD/property tests.
Unicode Normalization Forms and quick check mjb_normalize, mjb_string_is_normalized UAX #15 NormalizationTest.txt, DerivedNormalizationProps.txt, tests/normalization.c, and tests/quick-check.c.
Default case conversion and caseless matching mjb_case, simple codepoint case helpers Unicode Core Section 3.13, UAX #29 for titlecase word boundaries SpecialCasing.txt, CaseFolding.txt, WordBreakTest.txt, tests/special-case.c, tests/case.c, and tests/break-word.c.
Grapheme, word, and sentence boundaries mjb_break_grapheme_cluster, mjb_break_word, mjb_break_sentence, related truncation helpers UAX #29 GraphemeBreakTest.txt, WordBreakTest.txt, SentenceBreakTest.txt, tests/segmentation.c, tests/break-word.c, and tests/break-sentence.c.
Line breaking mjb_break_line UAX #14 LineBreakTest.txt and tests/break-line.c.
Bidirectional Algorithm mjb_bidi_resolve, mjb_bidi_reorder_line, mjb_bidi_line_runs UAX #9 BidiCharacterTest.txt, BidiTest.txt, tests/bidi.c, and tests/bidi-class.c.
Unicode Collation Algorithm, DUCET mjb_string_compare, mjb_collation_key UTS #10 CollationTest_NON_IGNORABLE.txt, CollationTest_SHIFTED.txt, and tests/collation.c; surrogate-code-point rows are filtered because public string input rejects ill-formed surrogate code points.
Unicode identifiers and pattern syntax data ID/XID/pattern predicates and mjb_string_is_identifier UAX #31 UCD ID/XID and pattern properties from DerivedCoreProperties.txt and PropList.txt; covered by tests/identifier.c.
Confusable skeleton matching mjb_string_is_confusable UTS #39 confusables.txt, intentional.txt, and tests/security.c.
Emoji properties and sequence data Emoji property predicates, mjb_string_emoji_sequence, RGI checks UTS #51 emoji-data.txt, emoji-sequences.txt, emoji-zwj-sequences.txt, emoji-variation-sequences.txt, emoji-test.txt, and tests/emoji.c.
East Asian Width property mjb_codepoint_east_asian_width; consumed by mjb_display_width UAX #11 EastAsianWidth.txt, tests/east-asian-width.c, and property tests; display column counts are a documented local policy over that property.

Thanks

Mojibake is built using the work of extraordinary individuals and teams.

  1. Unicode Character Database - Copyright © 1991-2026 Unicode, Inc. (see license.txt)
  2. Unicode CLDR Project - Copyright © 2004-2026 Unicode, Inc. (see LICENSE)

  1. Mojibake (Japanese: 文字化け 'character transformation') is the garbled text that is the result of text being decoded using an unintended character encoding. I created this library because I don't like any of the existing one. ↩︎

Try it online

You can try the library online by using the WASM-compiled version mojibake-wasm-024.zip. It is a live demo that you can use to preview the API.

mjb_codepoint_character

Return the codepoint character.

mjb_status mjb_codepoint_character(
    mjb_codepoint codepoint,
    mjb_character *character
);

Fill character with the Unicode Character Database record of a codepoint: name, category, combining class, bidirectional category, decomposition, numeric values, mirrored flag, and simple case mappings. When the library is compiled with MJB_FEATURE_CHARACTER_NAMES=OFF the name field is reported as Codepoint U+XXXX.

Returns

  • MJB_STATUS_OK — The character was found and filled
  • MJB_STATUS_INVALID_ARGUMENT — character is NULL or the codepoint is not valid
  • MJB_STATUS_NOT_FOUND — The codepoint is not assigned

Example

mjb_character character;

if(mjb_codepoint_character(0x022A, &character) != MJB_STATUS_OK) {
    return 1;
}

// U+022A lowercase: U+022B
printf("U+%04X lowercase: U+%04X", character.codepoint, character.lowercase);

Related

Specifications

mjb_normalize

Normalize a string to NFC/NFKC/NFD/NFKD form.

mjb_status mjb_normalize(
    const char *buffer,
    size_t byte_length,
    mjb_normalization form,
    mjb_encoding encoding,
    mjb_encoding output_encoding,
    mjb_result *result
);

Normalize a string to the requested Unicode normalization form. If the input is already normalized and no encoding conversion is needed, the input buffer is returned as-is in result->output with result->transformed set to false, without allocating.

Returns

  • MJB_STATUS_OK — The string was normalized (or already normal)
  • MJB_STATUS_INVALID_ARGUMENT — result is NULL, or buffer is NULL with a non-zero size
  • MJB_STATUS_INVALID_FORM — form is not NFC, NFD, NFKC, or NFKD
  • MJB_STATUS_OVERFLOW — The output size would overflow
  • MJB_STATUS_NO_MEMORY — Allocation failed

Example

const char *input = "Cafe\xCC\x81"; // "Cafe" + U+0301 COMBINING ACUTE ACCENT
mjb_result result;

if(mjb_normalize(input, strlen(input), MJB_NORMALIZATION_NFC, MJB_ENC_UTF_8, MJB_ENC_UTF_8,
    &result) != MJB_STATUS_OK) {
    return 1;
}

// NFC: Café
printf("NFC: %.*s", (int)result.output_size, result.output);

if(result.transformed) {
    mjb_free(result.output);
}

Related

Specifications

mjb_string_is_normalized

Check if a string is normalized to NFC/NFKC/NFD/NFKD form.

mjb_quick_check_result mjb_string_is_normalized(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_normalization form
);

Run the normalization quick-check on a string without allocating. MJB_QC_MAYBE means the string may still be normalized, and only a full normalization pass with mjb_normalize can decide.

Returns

  • MJB_QC_YES — The string is normalized to the requested form
  • MJB_QC_NO — The string is not normalized
  • MJB_QC_MAYBE — Inconclusive: a full normalization is needed to decide

Related

Specifications

mjb_string_filter

Filter a string with the selected mjb_filter flags.

mjb_status mjb_string_filter(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_encoding output_encoding,
    mjb_filter filters,
    mjb_result *result
);

MJB_FILTER_LIMIT_COMBINING removes combining marks after the first MJB_FILTER_MAX_COMBINING_MARKS consecutive marks in an emitted run. This is useful for reducing Zalgo-style text while keeping ordinary accents and stacked marks.

Example

const char *mixed_whitespace = "Hello\t\t\n\nworld";
mjb_result result;

if(mjb_string_filter(mixed_whitespace, strlen(mixed_whitespace), MJB_ENC_UTF_8, MJB_ENC_UTF_8,
    MJB_FILTER_COLLAPSE_SPACES, &result) != MJB_STATUS_OK) {
    return 1;
}

// Filtered: Hello world
printf("Filtered: %.*s", (int)result.output_size, result.output);

if(result.transformed) {
    mjb_free(result.output);
}

const char *controls = "\x1\x2\t\n\v\f\r\x1f";

if(mjb_string_filter(controls, strlen(controls), MJB_ENC_UTF_8, MJB_ENC_UTF_8,
    MJB_FILTER_CONTROLS, &result) != MJB_STATUS_OK) {
    return 1;
}

// Filtered: \t\n\v\f\r
printf("Filtered: %.*s", (int)result.output_size, result.output);

if(result.transformed) {
    mjb_free(result.output);
}

Related

mjb_string_each_character

Run a callback for each character of a string.

mjb_status mjb_string_each_character(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_string_each_character_fn callback
);

mjb_codepoint_property_value

Return if a codepoint has a property.

mjb_status mjb_codepoint_property_value(
    mjb_codepoint codepoint,
    mjb_property property,
    uint8_t *value
);

Specifications

mjb_codepoint_script

Return the script of a codepoint.

mjb_script mjb_codepoint_script(
    mjb_codepoint codepoint
);

Specifications

mjb_string_encoding

Return the string encoding (the most probable).

mjb_encoding mjb_string_encoding(
    const char *buffer,
    size_t byte_length
);

mjb_string_encoding reports BOM-derived UTF-16/UTF-32 schemes with the generic family bit plus the resolved endian bit. Passing that detected value consumes the leading BOM as a signature. Passing an explicit-endian encoding such as MJB_ENC_UTF_16BE preserves an initial U+FEFF as text. When flags overlap, as with a UTF-32LE BOM that also has the UTF-16LE BOM prefix, decoding gives UTF-32 precedence.

mjb_string_is_utf8

Return true if the string is encoded in UTF-8.

bool mjb_string_is_utf8(
    const char *buffer,
    size_t byte_length
);

mjb_string_is_utf16

Return true if the string is encoded in UTF-16BE or UTF-16LE.

bool mjb_string_is_utf16(
    const char *buffer,
    size_t byte_length
);

mjb_string_is_ascii

Return true if the string is encoded in ASCII.

bool mjb_string_is_ascii(
    const char *buffer,
    size_t byte_length
);

mjb_codepoint_encode

Encode a codepoint to a string.

unsigned int mjb_codepoint_encode(
    mjb_codepoint codepoint,
    char *buffer,
    size_t byte_length,
    mjb_encoding encoding
);

mjb_string_convert_encoding

Convert from one encoding to another.

mjb_status mjb_string_convert_encoding(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_encoding output_encoding,
    mjb_result *result
);

Convert a string between the supported encodings (UTF-8, UTF-16LE/BE, UTF-32LE/BE). Generic UTF-16/UTF-32 input consumes a leading BOM as the encoding scheme signature and uses it to resolve byte order. Explicit-endian input preserves an initial U+FEFF as text. Generic UTF-16/UTF-32 without a BOM, and generic UTF-16/UTF-32 output, are rejected because the byte order is not specified.

Returns

  • MJB_STATUS_OK — The string was converted
  • MJB_STATUS_INVALID_ARGUMENT — result is NULL, buffer is NULL with a non-zero size, or the input is not valid in the source encoding
  • MJB_STATUS_INVALID_ENCODING — A generic UTF-16/UTF-32 encoding did not provide enough byte order information
  • MJB_STATUS_UNSUPPORTED — The requested encoding conversion is not supported
  • MJB_STATUS_OVERFLOW — The output size would overflow
  • MJB_STATUS_NO_MEMORY — Allocation failed

Related

mjb_string_length

Return the length of a string.

size_t mjb_string_length(
    const char *buffer,
    size_t max_length,
    mjb_encoding encoding
);

mjb_string_compare

Compare two strings using UCA.

int mjb_string_compare(
    const char *s1,
    size_t s1_byte_length,
    mjb_encoding s1_encoding,
    const char *s2,
    size_t s2_byte_length,
    mjb_encoding s2_encoding,
    mjb_collation_mode mode
);

Compare two strings using the Unicode Collation Algorithm and the default collation element table (DUCET), with strcmp-style semantics.

Returns

  • < 0 — The first string collates before the second
  • 0 — The strings are equal under UCA
  • > 0 — The first string collates after the second

Related

Specifications

mjb_collation_key

Generate a UCA sort key for a string.

mjb_status mjb_collation_key(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_collation_mode mode,
    mjb_result *result
);

Generate a binary sort key for a string. Sort keys of different strings can be compared with memcmp and yield the same order as mjb_string_compare. Useful when the same strings are compared many times, such as sorting or database indexing.

Returns

  • MJB_STATUS_OK — The sort key was generated
  • MJB_STATUS_INVALID_ARGUMENT — result is NULL, or buffer is NULL with a non-zero size
  • MJB_STATUS_OVERFLOW — The sort key size would overflow
  • MJB_STATUS_NO_MEMORY — Allocation failed

Related

Specifications

mjb_case

Change string case.

mjb_status mjb_case(
    const char *buffer,
    size_t byte_length,
    mjb_case_type type,
    mjb_encoding encoding,
    mjb_encoding output_encoding,
    mjb_result *result
);

Convert a string to uppercase, lowercase, titlecase, or its case-folded form. Full case mappings are applied, including special casing and conditional mappings, so the output may have a different length than the input. Titlecase uses UAX #29 word boundaries: the first cased character in each word segment is titlecased, and subsequent characters in that segment are lowercased. Casing is tailored by the process-global locale set with mjb_locale_set: the default MJB_LOCALE_EN uses default non-Turkic mappings. MJB_LOCALE_TR and MJB_LOCALE_AZ apply Turkish/Azerbaijani dotted-I casing and Turkic T case-folding mappings. MJB_LOCALE_LT applies Lithuanian dot-above casing rules, while case folding remains the default non-Turkic mapping.

Returns

  • MJB_STATUS_OK — The case conversion succeeded
  • MJB_STATUS_INVALID_ARGUMENT — result is NULL, buffer is NULL with a non-zero size, or type is not a valid case type
  • MJB_STATUS_NO_MEMORY — Allocation failed

Example

const char *input = "Stra\xC3\x9F""e"; // "Straße"
mjb_result result;

if(mjb_case(input, strlen(input), MJB_CASE_UPPER, MJB_ENC_UTF_8, MJB_ENC_UTF_8,
    &result) != MJB_STATUS_OK) {
    return 1;
}

// Upper: STRASSE
printf("Upper: %.*s", (int)result.output_size, result.output);

if(result.transformed) {
    mjb_free(result.output);
}

Related

Specifications

mjb_codepoint_is_valid

Return true if the codepoint is valid.

bool mjb_codepoint_is_valid(
    mjb_codepoint codepoint
);

mjb_codepoint_is_graphic

Return true if the codepoint is graphic.

bool mjb_codepoint_is_graphic(
    mjb_codepoint codepoint
);

mjb_codepoint_is_combining

Return true if the codepoint is combining.

bool mjb_codepoint_is_combining(
    mjb_codepoint codepoint
);

mjb_codepoint_is_hangul_syllable

Return if the codepoint is a hangul syllable.

bool mjb_codepoint_is_hangul_syllable(
    mjb_codepoint codepoint
);

mjb_codepoint_is_cjk_ideograph

Return if the codepoint is CJK ideograph.

bool mjb_codepoint_is_cjk_ideograph(
    mjb_codepoint codepoint
);

mjb_codepoint_is_cjk_ext

Return if the codepoint is CJK extension.

bool mjb_codepoint_is_cjk_ext(
    mjb_codepoint codepoint
);

mjb_category_is_graphic

Return true if the category is graphic.

bool mjb_category_is_graphic(
    mjb_category category
);

mjb_category_is_combining

Return true if the category is combining.

bool mjb_category_is_combining(
    mjb_category category
);

mjb_codepoint_numeric_value

Return the numeric value of a codepoint.

mjb_status mjb_codepoint_numeric_value(
    mjb_codepoint codepoint,
    mjb_numeric_value *value
);

Return the numeric value of a codepoint, if any. If the codepoint has no numeric value, value->decimal and value->digit are set to MJB_NUMBER_NOT_VALID (-1).

Returns

  • MJB_STATUS_OK — The character was found and filled
  • MJB_STATUS_INVALID_ARGUMENT — value is NULL or the codepoint is not valid

Example

mjb_numeric_value num;

if(mjb_codepoint_numeric_value(0x0031, &num) != MJB_STATUS_OK) { // U+0031 = 1
    return 1;
}

// decimal=1, digit=1, numeric=1
printf("decimal=%d, digit=%d, numeric=%s", num.decimal, num.digit, num.numeric);

if(mjb_codepoint_numeric_value(0x00BD, &num) != MJB_STATUS_OK) { // U+00BD = '½'
    return 1;
}

// decimal=-1, digit=-1, numeric=1/2
printf("decimal=%d, digit=%d, numeric=%s", num.decimal, num.digit, num.numeric);

Specifications

mjb_codepoint_block

Return the character block.

mjb_status mjb_codepoint_block(
    mjb_codepoint codepoint,
    mjb_block_info *block
);

Specifications

mjb_codepoint_to_lowercase

Return the codepoint lowercase codepoint.

mjb_codepoint mjb_codepoint_to_lowercase(
    mjb_codepoint codepoint
);

Return the lowercase codepoint of a codepoint. If the codepoint has no lowercase equivalent, the original codepoint is returned.

Returns

  • codepoint — The lowercase codepoint, or the original codepoint

Example

mjb_codepoint codepoint;

codepoint = mjb_codepoint_to_lowercase(0x0041); // U+0041 = 'A'

// A > a
printf("%c > %c", 'A', codepoint);

codepoint = mjb_codepoint_to_lowercase(0x03A3); // U+03A3 = 'Σ'

// U+03A3 > U+03C3, Σ > σ
printf("U+%04X > U+%04X, %s > %s",  0x03A3, codepoint, "Σ", "σ");

Related

mjb_codepoint_to_uppercase

Return the codepoint uppercase codepoint.

mjb_codepoint mjb_codepoint_to_uppercase(
    mjb_codepoint codepoint
);

Return the uppercase codepoint of a codepoint. If the codepoint has no uppercase equivalent, the original codepoint is returned.

Returns

  • codepoint — The uppercase codepoint, or the original codepoint

Related

mjb_codepoint_to_titlecase

Return the codepoint titlecase codepoint.

mjb_codepoint mjb_codepoint_to_titlecase(
    mjb_codepoint codepoint
);

Return the titlecase codepoint of a codepoint. If the codepoint has no titlecase equivalent, the original codepoint is returned.

Returns

  • codepoint — The titlecase codepoint, or the original codepoint

Related

mjb_break_line

Unicode line break algorithm.

mjb_break_type mjb_break_line(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_next_line_state *state
);

Related

Specifications

mjb_break_word

Word cluster breaking.

mjb_break_type mjb_break_word(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_next_word_state *state
);

Related

Specifications

mjb_truncate_word

Return the number of bytes that form the first max_segments word-break segments.

size_t mjb_truncate_word(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    size_t max_segments
);

mjb_truncate_word_width

Return the number of bytes whose word-break segments fit within max_columns display columns.

size_t mjb_truncate_word_width(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_width_context context,
    size_t max_columns
);

mjb_break_sentence

Sentence boundaries breaking.

mjb_break_type mjb_break_sentence(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_next_sentence_state *state
);

Related

Specifications

mjb_break_grapheme_cluster

Grapheme cluster breaking.

mjb_break_type mjb_break_grapheme_cluster(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_next_state *state
);

Iterate the grapheme cluster (user-perceived character) boundaries of a string. Call repeatedly with the same state until it reports the end of the string.

Related

Specifications

mjb_truncate

Return the number of bytes that form the first `max_graphemes` grapheme cluster segments.

size_t mjb_truncate(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    size_t max_graphemes
);

mjb_truncate_width

Return the number of bytes whose grapheme clusters fit within max_columns display columns.

size_t mjb_truncate_width(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_width_context context,
    size_t max_columns
);

mjb_bidi_resolve

Resolve bidirectional text (TR9) for a paragraph.

mjb_status mjb_bidi_resolve(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_direction direction,
    mjb_bidi_paragraph *result
);

Resolve the embedding levels of a paragraph following the Unicode Bidirectional Algorithm. The resolved paragraph can then be split into lines and reordered visually with mjb_bidi_reorder_line and mjb_bidi_line_runs.

Returns

  • MJB_STATUS_OK — The paragraph was resolved
  • MJB_STATUS_INVALID_ARGUMENT — result is NULL, or buffer is NULL with a non-zero size
  • MJB_STATUS_OVERFLOW — The paragraph size would overflow
  • MJB_STATUS_NO_MEMORY — Allocation failed

Related

  • mjb_bidi_free
  • mjb_bidi_reorder_line
  • mjb_bidi_line_runs

Specifications

mjb_codepoint_plane

Return the plane of the codepoint.

mjb_plane mjb_codepoint_plane(
    mjb_codepoint codepoint
);

mjb_plane_is_valid

Return true if the plane is valid.

bool mjb_plane_is_valid(
    mjb_plane plane
);

mjb_plane_name

Return the name of a plane, NULL if the plane specified is not valid.

const char *mjb_plane_name(
    mjb_plane plane,
    bool abbreviation
);

mjb_codepoint_is_id_start

Return true if the codepoint is a valid Unicode identifier start (Unicode 17.0.0 UAX #31 ID_Start).

bool mjb_codepoint_is_id_start(
    mjb_codepoint codepoint
);

Specifications

mjb_codepoint_is_id_continue

Return true if the codepoint is a valid Unicode identifier continuation (Unicode 17.0.0 UAX #31 ID_Continue).

bool mjb_codepoint_is_id_continue(
    mjb_codepoint codepoint
);

Specifications

mjb_codepoint_is_xid_start

Return true if the codepoint is a valid NFKC identifier start (Unicode 17.0.0 UAX #31 XID_Start).

bool mjb_codepoint_is_xid_start(
    mjb_codepoint codepoint
);

Specifications

mjb_codepoint_is_xid_continue

Return true if the codepoint is a valid NFKC identifier continuation (Unicode 17.0.0 UAX #31 XID_Continue).

bool mjb_codepoint_is_xid_continue(
    mjb_codepoint codepoint
);

Specifications

mjb_codepoint_is_pattern_syntax

Return true if the codepoint is reserved for use in patterns (Unicode 17.0.0 UAX #31 Pattern_Syntax).

bool mjb_codepoint_is_pattern_syntax(
    mjb_codepoint codepoint
);

Specifications

mjb_codepoint_is_pattern_white_space

Return true if the codepoint is pattern whitespace (Unicode 17.0.0 UAX #31 Pattern_White_Space).

bool mjb_codepoint_is_pattern_white_space(
    mjb_codepoint codepoint
);

Specifications

mjb_string_is_identifier

Return true if the string is a valid Unicode identifier (Unicode 17.0.0 UAX #31).

bool mjb_string_is_identifier(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_identifier_profile profile
);

Validate a string as a Unicode identifier: the first character must be a valid identifier start and the following ones valid identifier continuations, using ID_Start/ID_Continue for the DEFAULT profile or XID_Start/XID_Continue for the NFKC profile.

Related

Specifications

mjb_property_name

Return the name of a property, NULL if the property specified is not valid.

const char *mjb_property_name(
    mjb_property property
);

Specifications

mjb_string_is_confusable

Return true if two strings are visually confusable (Unicode 17.0.0 UTS #39 Section 4): skeleton(s1) == skeleton(s2).

bool mjb_string_is_confusable(
    const char *s1,
    size_t s1_byte_length,
    mjb_encoding s1_encoding,
    const char *s2,
    size_t s2_byte_length,
    mjb_encoding s2_encoding
);

Compute the confusable skeleton of both strings and return true when the skeletons are equal, meaning the two strings are visually confusable, such as "good" and "gооd" with Cyrillic о.

Related

Specifications

mjb_codepoint_emoji

Return the emoji properties.

mjb_status mjb_codepoint_emoji(
    mjb_codepoint codepoint,
    mjb_emoji_properties *emoji
);

Related

Specifications

mjb_codepoint_is_emoji

Return true if the codepoint has the Unicode Emoji property.

bool mjb_codepoint_is_emoji(
    mjb_codepoint codepoint
);

Specifications

mjb_codepoint_is_emoji_presentation

Return true if the codepoint has the Unicode Emoji_Presentation property.

bool mjb_codepoint_is_emoji_presentation(
    mjb_codepoint codepoint
);

Specifications

mjb_codepoint_is_emoji_modifier

Return true if the codepoint has the Unicode Emoji_Modifier property.

bool mjb_codepoint_is_emoji_modifier(
    mjb_codepoint codepoint
);

Specifications

mjb_codepoint_is_emoji_modifier_base

Return true if the codepoint has the Unicode Emoji_Modifier_Base property.

bool mjb_codepoint_is_emoji_modifier_base(
    mjb_codepoint codepoint
);

Specifications

mjb_codepoint_is_emoji_component

Return true if the codepoint has the Unicode Emoji_Component property.

bool mjb_codepoint_is_emoji_component(
    mjb_codepoint codepoint
);

Specifications

mjb_codepoint_is_extended_pictographic

Return true if the codepoint has the Unicode Extended_Pictographic property.

bool mjb_codepoint_is_extended_pictographic(
    mjb_codepoint codepoint
);

Specifications

mjb_string_emoji_sequence

Return emoji sequence metadata for a complete string.

mjb_status mjb_string_emoji_sequence(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_emoji_sequence *emoji
);

Related

Specifications

mjb_string_is_emoji_sequence

Return true if the complete string is an emoji sequence listed by Unicode, including standardized emoji variation sequences.

bool mjb_string_is_emoji_sequence(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding
);

Related

Specifications

mjb_string_is_rgi_emoji

Return true if the complete string is an RGI emoji sequence, excluding plain standardized variation sequences.

bool mjb_string_is_rgi_emoji(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding
);

Related

Specifications

mjb_codepoint_east_asian_width

Return the east asian width of a codepoint.

mjb_status mjb_codepoint_east_asian_width(
    mjb_codepoint codepoint,
    mjb_east_asian_width *width
);

Related

Specifications

mjb_display_width

Return the display width of a string.

mjb_status mjb_display_width(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_width_context context,
    size_t *width
);

Compute the number of display columns a string occupies in a terminal, accounting for wide and ambiguous East Asian characters, combining marks, and emoji sequences.

Returns

  • MJB_STATUS_OK — The width was computed
  • MJB_STATUS_INVALID_ARGUMENT — width is NULL, or buffer is NULL with a non-zero size
  • MJB_STATUS_OVERFLOW — The width would overflow

Related

Specifications

mjb_locale_parse

Parse a BCP 47 language tag.

mjb_status mjb_locale_parse(
    const char *id,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_locale_id *locale,
    mjb_error *error
);

Parse a BCP 47 language tag, such as sr-Latn-RS, into its components: language, extended language, script, region, variant, extensions, private use, and grandfathered tags. Parsing is strict: malformed tags are rejected and error is filled with the failure reason.

Returns

  • MJB_STATUS_OK — The tag was parsed and locale filled
  • MJB_STATUS_INVALID_ARGUMENT — An argument is NULL or the tag is not a valid BCP 47 language tag
  • MJB_STATUS_NO_MEMORY — Allocation failed

Related

Specifications

mjb_locale_set

Set current locale used by locale-sensitive casing.

mjb_status mjb_locale_set(
    unsigned int locale
);

Set the process-global locale used by mjb_case. The default locale is MJB_LOCALE_EN, and mjb_shutdown resets it to MJB_LOCALE_EN. Only MJB_LOCALE_TR, MJB_LOCALE_AZ, and MJB_LOCALE_LT currently tailor casing. Other valid locale values are accepted but do not change Unicode algorithm behavior.

Returns

  • MJB_STATUS_OK — The locale was set
  • MJB_STATUS_INVALID_ARGUMENT — locale is not a valid mjb_locale value

Related

mjb_version

Output the current library version (MJB_VERSION).

const char *mjb_version(void);

mjb_version_number

Output the current library version number (MJB_VERSION_NUMBER).

unsigned int mjb_version_number(void);

mjb_unicode_version

Output the current supported Unicode version (MJB_UNICODE_VERSION).

const char *mjb_unicode_version(void);