Mojibake v0.2.5

Unicode 17 - C11 - zero dependencies

Unicode text processing,
without the baggage.

Mojibake is a small, fast, self-contained Unicode library for C11 and C++17. Ship standards-compliant text handling without a runtime or dependency tree.

Download v0.2.5 Explore the API
Standard
Unicode 17.0
Runtime
None
License
MIT

Start here

A practical Unicode toolkit

Mojibake is a low-level Unicode 17 text-processing library written in C11 and compatible with C++17. It is released under the MIT License.

It aims to be:

  1. Small
  2. Easy to use
  3. Fast
  4. Self-contained

Mojibake do:

  1. Run in all modern OSes (Linux, macOS, FreeBSD, OpenBSD, NetBSD, Windows 10/11)
  2. Pass the official Unicode test suites for supported algorithms
  3. Implement all Unicode standard algorithms
  4. Satisfy all Unicode Conformance Requirements

Feature highlights

All the C files, together with the Unicode data tables, are concatenated into a single large file and header: mojibake.c and mojibake.h. Zero dependencies.

Text transformation

  • Normalization: NFC/NFD/NFKC/NFKD (mjb_normalize), plus a fast quick-check (mjb_string_is_normalized) (UAX #15, Unicode 17.0.0)
  • Case conversion: uppercase, lowercase, titlecase, and case folding with full special-casing and conditional mappings (mjb_case)
  • Filtering: strip controls, spaces, or numeric characters while normalizing (mjb_string_filter)

Text analysis

  • Character database: every Unicode Character Database property: category, script, block, plane, numeric value, name (mjb_codepoint_character)
  • Segmentation: grapheme clusters, words, sentences, and line-break opportunities (UAX #29, Unicode 17.0.0, UAX #14, Unicode 17.0.0)
  • Bidirectional text: full Unicode Bidirectional Algorithm: paragraph resolution, line reordering, runs (UAX #9, Unicode 17.0.0)
  • Emoji: codepoint properties, sequence analysis, RGI emoji detection
  • Display width: East Asian width and terminal display width, with width-aware truncation (mjb_display_width, mjb_truncate_width)

Sorting and comparison

  • Collation: Unicode Collation Algorithm string comparison and sort keys, in shifted and non-ignorable modes (mjb_string_compare, mjb_collation_key, UTS #10, Unicode 17.0.0)

Security

  • Confusable detection: check if a string is visually confusable with another (mjb_string_is_confusable, UTS #39, Unicode 17.0.0)
  • Identifier validation: XID/ID checks for parser and compiler authors (mjb_string_is_identifier, UAX #31, Unicode 17.0.0)

Integration

  • Encodings: the API accepts and outputs UTF-8, UTF-16LE, UTF-16BE, UTF-32LE, UTF-32BE strings, with encoding detection and conversion (mjb_string_encoding, mjb_string_convert_encoding)
  • Parsing and string functions: character-by-character iteration (mjb_next_character) and standard C string.h-style helpers (mjb_string_length, and others)
  • Locales: strict BCP 47 language tag parsing (mjb_locale_parse)
  • Embeddable: custom allocators (mjb_set_memory_functions), build-time feature flags to trim table size, a C++17 wrapper (src/cpp/mojibake.hpp), a CLI tool (src/shell), and a WASM + TypeScript API (src/api)
  • Tested: Mojibake uses Attractor as test suite and run 1.5M+ assertions including the official Unicode conformance suites for supported algorithms
  • Fuzz It's fuzzed with libFuzzer
  • AddressSanitizer and UBSan clean

Usage

You don't need to install anything. Add the C source and header to your build.

  1. Download it here mojibake-amalgamation-025.zip
  2. Unzip it
  3. Add mojibake.c and mojibake.h to your project

Example:

#include <stdio.h>
#include <string.h>
#include "mojibake.h"

// This is a simple example of how to use the Mojibake library.
int main(int argc, char * const argv[]) {
    printf("This is an example of Mojibake v%s\n", mjb_version());
    printf("Unicode version: %s\n", mjb_unicode_version());

    const char *input = "Cafe\xCC\x81";
    mjb_result result;

    if(mjb_normalize(input, strlen(input), MJB_NORMALIZATION_NFC, MJB_ENC_UTF_8, MJB_ENC_UTF_8,
        &result) != MJB_STATUS_OK) {
        return 1;
    }

    // This outputs "NFC: Café", e + ◌́ -> é
    printf("NFC: %.*s\n", (int)result.output_size, result.output);

    if(result.transformed) {
        mjb_free(result.output);
    }

    return 0;
}

This library works only in little-endian systems to avoid adding too much overhead. This means that it works on all modern general-purpose CPUs today (x86, x86-64, ARMv8, RISC-V, etc.) It has been tested on:

  1. Linux
  2. macOS
  3. FreeBSD
  4. OpenBSD
  5. NetBSD
  6. Windows 10/11

Build-time features

Mojibake can compile out optional feature tables to reduce binary size. Feature macros default to enabled.

  • MJB_FEATURE_CHARACTER_NAMES controls the Unicode character-name tables used by mjb_codepoint_character(...) to fill mjb_character.name. When disabled, the tables are not compiled and mjb_character.name is reported as Codepoint U+XXXX.

With CMake:

cmake -S . -B build-no-name -DMJB_FEATURE_CHARACTER_NAMES=OFF
cmake --build build-no-name

With the provided Makefile:

make build BUILD_DIR=build-no-name FEATURE_CHARACTER_NAMES=OFF
make test-no-names

API documentation

See API.md for the detailed documentation.

CLI

The src/shell directory builds the mojibake CLI used to test the library. Example usage:

mojibake nfc $'Cafe\u0301'

Plain text output:

Café

Emoji sequence analysis:

mojibake emoji "☺️"
mojibake -c emoji 263A FE0F

Coverage

Mojibake run a total of 1,560,861 tests, including all the official tests included in the standard:

  1. auxiliary/GraphemeBreakTest.txt
  2. auxiliary/LineBreakTest.txt
  3. auxiliary/SentenceBreakTest.txt
  4. auxiliary/WordBreakTest.txt
  5. BidiCharacterTest.txt
  6. BidiTest.txt
  7. CaseFolding.txt
  8. CollationTest/CollationTest_NON_IGNORABLE.txt
  9. CollationTest/CollationTest_SHIFTED.txt
  10. emoji-test.txt
  11. intentional.txt
  12. NormalizationTest.txt
  13. SpecialCasing.txt

Fuzzing

The public API is fuzzed with libFuzzer over untrusted byte input.

make fuzz # 60 seconds by default
make fuzz FUZZ_TIME=300

Building from source

See CONTRIBUTING.md for instructions.

Licenses

Mojibake is released under the MIT License (see LICENSE).

Unicode references

Mojibake's Unicode data and algorithm references are scoped to The Unicode Standard, Version 17.0.0 and the Unicode Character Database 17.0.0. Normative algorithm references here and in https://github.com/zaerl/mojibake/blob/main/API.md use the archived Unicode 17.0.0 versions of the applicable annexes and synchronized technical standards:

Generic Unicode links, when present, are informational or download links rather than normative conformance references.

Unicode Conformance Requirements

Mojibake satisfy the Unicode Conformance Requirements. See CONFORMANCE_REQUIREMENTS.md for details.

Unicode tailoring

Unless listed here, Mojibake applies the referenced Unicode 17.0.0 algorithms without higher-level protocol tailoring.

  • Case conversion and case folding: mjb_case is locale-sensitive through the process-global locale set by mjb_locale_set. The default locale is MJB_LOCALE_EN. MJB_LOCALE_TR and MJB_LOCALE_AZ apply the Turkish/Azerbaijani dotted-I rules from SpecialCasing.txt for uppercase, lowercase, and titlecase, and the Turkic T mappings from CaseFolding.txt for full and simple case folding. MJB_LOCALE_LT applies Lithuanian dot-above rules from SpecialCasing.txt for uppercase, lowercase, and titlecase; case folding remains the default non-Turkic mapping.
  • Collation: mjb_string_compare and mjb_collation_key use DUCET without locale collation tailoring. The mjb_collation_mode argument only selects the UCA variable weighting strategy.
  • Display width: mjb_display_width has an explicit mjb_width_context policy for East Asian Width Ambiguous characters. mjb_codepoint_east_asian_width itself reports the Unicode 17.0.0 property value without tailoring.
  • Other Unicode algorithms: normalization, bidirectional processing, grapheme/word/sentence/line breaking, identifier validation, confusable skeletons, and emoji sequence checks are not locale-tailored by Mojibake.

Unicode conformance inventory

Mojibake interprets Unicode text only through the public APIs and supported UTF encodings listed in this documentation. It does not implement rendering, font shaping, locale collation tailoring, or higher-level protocol behavior beyond the documented locale-sensitive casing and display-width policy. The table below maps the advertised Unicode algorithm and data claims to their Unicode 17.0.0 reference and test evidence.

Claim Public surface Unicode reference Evidence
Unicode Character Database data and derived properties mjb_codepoint_character, mjb_codepoint_property_value, script/block/category/numeric helpers UAX #44, UCD 17.0.0 Generated from UCD data files including UnicodeData.txt, Blocks.txt, Scripts.txt, PropList.txt, DerivedCoreProperties.txt, PropertyAliases.txt, and PropertyValueAliases.txt; covered by local UCD/property tests.
Unicode Normalization Forms and quick check mjb_normalize, mjb_string_is_normalized UAX #15 NormalizationTest.txt, DerivedNormalizationProps.txt, tests/normalization.c, and tests/quick-check.c.
Default case conversion and caseless matching mjb_case, simple codepoint case helpers Unicode Core Section 3.13, UAX #29 for titlecase word boundaries SpecialCasing.txt, CaseFolding.txt, WordBreakTest.txt, tests/special-case.c, tests/case.c, and tests/break-word.c.
Grapheme, word, and sentence boundaries mjb_break_grapheme_cluster, mjb_break_word, mjb_break_sentence, related truncation helpers UAX #29 GraphemeBreakTest.txt, WordBreakTest.txt, SentenceBreakTest.txt, tests/segmentation.c, tests/break-word.c, and tests/break-sentence.c.
Line breaking mjb_break_line UAX #14 LineBreakTest.txt and tests/break-line.c.
Bidirectional Algorithm mjb_bidi_resolve, mjb_bidi_reorder_line, mjb_bidi_line_runs UAX #9 BidiCharacterTest.txt, BidiTest.txt, tests/bidi.c, and tests/bidi-class.c.
Unicode Collation Algorithm, DUCET mjb_string_compare, mjb_collation_key UTS #10 CollationTest_NON_IGNORABLE.txt, CollationTest_SHIFTED.txt, and tests/collation.c; surrogate-code-point rows are filtered because public string input rejects ill-formed surrogate code points.
Unicode identifiers and pattern syntax data ID/XID/pattern predicates and mjb_string_is_identifier UAX #31 UCD ID/XID and pattern properties from DerivedCoreProperties.txt and PropList.txt; covered by tests/identifier.c.
Confusable skeleton matching mjb_string_is_confusable UTS #39 confusables.txt, intentional.txt, and tests/security.c.
Emoji properties and sequence data Emoji property predicates, mjb_string_emoji_sequence, RGI checks UTS #51 emoji-data.txt, emoji-sequences.txt, emoji-zwj-sequences.txt, emoji-variation-sequences.txt, emoji-test.txt, and tests/emoji.c.
East Asian Width property mjb_codepoint_east_asian_width; consumed by mjb_display_width UAX #11 EastAsianWidth.txt, tests/east-asian-width.c, and property tests; display column counts are a documented local policy over that property.

Thanks

Mojibake is built using the work of extraordinary individuals and teams.

  1. Unicode Character Database - Copyright © 1991-2026 Unicode, Inc. (see license.txt)
  2. Unicode CLDR Project - Copyright © 2004-2026 Unicode, Inc. (see LICENSE)

No installation required

Try every function in your browser

Each API reference below includes a live form backed by the WASM build. Expand a function, enter its arguments, and inspect the result immediately.

Download mojibake-wasm-025.zip

Reference

C API

Functions are organized by their metadata section. Select the plus button to see detailed behavior, examples, specifications, and the live WASM form.

API section

Text transformation

Normalize, case-convert, filter, and convert Unicode text.

8 functions

mjb_normalize

Normalize a string to NFC/NFKC/NFD/NFKD form.

mjb_status mjb_normalize(
    const char *buffer,
    size_t byte_length,
    mjb_normalization form,
    mjb_encoding encoding,
    mjb_encoding output_encoding,
    mjb_result *result
);

Normalize a string to the requested Unicode normalization form. If the input is already normalized and no encoding conversion is needed, the input buffer is returned as-is in result->output with result->transformed set to false, without allocating.

Returns

  • MJB_STATUS_OK — The string was normalized (or already normal)
  • MJB_STATUS_INVALID_ARGUMENT — result is NULL, or buffer is NULL with a non-zero size
  • MJB_STATUS_INVALID_FORM — form is not NFC, NFD, NFKC, or NFKD
  • MJB_STATUS_OVERFLOW — The output size would overflow
  • MJB_STATUS_NO_MEMORY — Allocation failed

Example

const char *input = "Cafe\xCC\x81"; // "Cafe" + U+0301 COMBINING ACUTE ACCENT
mjb_result result;

if(mjb_normalize(input, strlen(input), MJB_NORMALIZATION_NFC, MJB_ENC_UTF_8, MJB_ENC_UTF_8,
    &result) != MJB_STATUS_OK) {
    return 1;
}

// NFC: Café
printf("NFC: %.*s", (int)result.output_size, result.output);

if(result.transformed) {
    mjb_free(result.output);
}

Related

Specifications

mjb_string_filter

Filter a string with the selected mjb_filter flags.

mjb_status mjb_string_filter(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_encoding output_encoding,
    mjb_filter filters,
    mjb_result *result
);

MJB_FILTER_LIMIT_COMBINING removes combining marks after the first MJB_FILTER_MAX_COMBINING_MARKS consecutive marks in an emitted run. This is useful for reducing Zalgo-style text while keeping ordinary accents and stacked marks.

Example

const char *mixed_whitespace = "Hello\t\t\n\nworld";
mjb_result result;

if(mjb_string_filter(mixed_whitespace, strlen(mixed_whitespace), MJB_ENC_UTF_8, MJB_ENC_UTF_8,
    MJB_FILTER_COLLAPSE_SPACES, &result) != MJB_STATUS_OK) {
    return 1;
}

// Filtered: Hello world
printf("Filtered: %.*s", (int)result.output_size, result.output);

if(result.transformed) {
    mjb_free(result.output);
}

const char *controls = "\x1\x2\t\n\v\f\r\x1f";

if(mjb_string_filter(controls, strlen(controls), MJB_ENC_UTF_8, MJB_ENC_UTF_8,
    MJB_FILTER_CONTROLS, &result) != MJB_STATUS_OK) {
    return 1;
}

// Filtered: \t\n\v\f\r
printf("Filtered: %.*s", (int)result.output_size, result.output);

if(result.transformed) {
    mjb_free(result.output);
}

Related

filtersThe filters to use. Leave all unchecked for MJB_FILTER_NONE.

mjb_codepoint_encode

Encode a codepoint to a string.

unsigned int mjb_codepoint_encode(
    mjb_codepoint codepoint,
    char *buffer,
    size_t byte_length,
    mjb_encoding encoding
);

mjb_string_convert_encoding

Convert from one encoding to another.

mjb_status mjb_string_convert_encoding(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_encoding output_encoding,
    mjb_result *result
);

Convert a string between the supported encodings (UTF-8, UTF-16LE/BE, UTF-32LE/BE). Generic UTF-16/UTF-32 input consumes a leading BOM as the encoding scheme signature and uses it to resolve byte order. Explicit-endian input preserves an initial U+FEFF as text. Generic UTF-16/UTF-32 without a BOM, and generic UTF-16/UTF-32 output, are rejected because the byte order is not specified.

Returns

  • MJB_STATUS_OK — The string was converted
  • MJB_STATUS_INVALID_ARGUMENT — result is NULL, buffer is NULL with a non-zero size, or the input is not valid in the source encoding
  • MJB_STATUS_INVALID_ENCODING — A generic UTF-16/UTF-32 encoding did not provide enough byte order information
  • MJB_STATUS_UNSUPPORTED — The requested encoding conversion is not supported
  • MJB_STATUS_OVERFLOW — The output size would overflow
  • MJB_STATUS_NO_MEMORY — Allocation failed

Related

mjb_case

Change string case.

mjb_status mjb_case(
    const char *buffer,
    size_t byte_length,
    mjb_case_type type,
    mjb_encoding encoding,
    mjb_encoding output_encoding,
    mjb_result *result
);

Convert a string to uppercase, lowercase, titlecase, or its case-folded form. Full case mappings are applied, including special casing and conditional mappings, so the output may have a different length than the input. Titlecase uses UAX #29 word boundaries: the first cased character in each word segment is titlecased, and subsequent characters in that segment are lowercased. Casing is tailored by the process-global locale set with mjb_locale_set: the default MJB_LOCALE_EN uses default non-Turkic mappings. MJB_LOCALE_TR and MJB_LOCALE_AZ apply Turkish/Azerbaijani dotted-I casing and Turkic T case-folding mappings. MJB_LOCALE_LT applies Lithuanian dot-above casing rules, while case folding remains the default non-Turkic mapping.

Returns

  • MJB_STATUS_OK — The case conversion succeeded
  • MJB_STATUS_INVALID_ARGUMENT — result is NULL, buffer is NULL with a non-zero size, or type is not a valid case type
  • MJB_STATUS_NO_MEMORY — Allocation failed

Example

const char *input = "Stra\xC3\x9F""e"; // "Straße"
mjb_result result;

if(mjb_case(input, strlen(input), MJB_CASE_UPPER, MJB_ENC_UTF_8, MJB_ENC_UTF_8,
    &result) != MJB_STATUS_OK) {
    return 1;
}

// Upper: STRASSE
printf("Upper: %.*s", (int)result.output_size, result.output);

if(result.transformed) {
    mjb_free(result.output);
}

Related

Specifications

mjb_codepoint_to_lowercase

Return the codepoint lowercase codepoint.

mjb_codepoint mjb_codepoint_to_lowercase(
    mjb_codepoint codepoint
);

Return the lowercase codepoint of a codepoint. If the codepoint has no lowercase equivalent, the original codepoint is returned.

Returns

  • codepoint — The lowercase codepoint, or the original codepoint

Example

mjb_codepoint codepoint;

codepoint = mjb_codepoint_to_lowercase(0x0041); // U+0041 = 'A'

// A > a
printf("%c > %c", 'A', codepoint);

codepoint = mjb_codepoint_to_lowercase(0x03A3); // U+03A3 = 'Σ'

// U+03A3 > U+03C3, Σ > σ
printf("U+%04X > U+%04X, %s > %s",  0x03A3, codepoint, "Σ", "σ");

Related

mjb_codepoint_to_uppercase

Return the codepoint uppercase codepoint.

mjb_codepoint mjb_codepoint_to_uppercase(
    mjb_codepoint codepoint
);

Return the uppercase codepoint of a codepoint. If the codepoint has no uppercase equivalent, the original codepoint is returned.

Returns

  • codepoint — The uppercase codepoint, or the original codepoint

Related

mjb_codepoint_to_titlecase

Return the codepoint titlecase codepoint.

mjb_codepoint mjb_codepoint_to_titlecase(
    mjb_codepoint codepoint
);

Return the titlecase codepoint of a codepoint. If the codepoint has no titlecase equivalent, the original codepoint is returned.

Returns

  • codepoint — The titlecase codepoint, or the original codepoint

Related

API section

Text analysis

Inspect codepoints, properties, boundaries, width, emoji, and bidirectional text.

29 functions

mjb_codepoint_character

Return the codepoint character.

mjb_status mjb_codepoint_character(
    mjb_codepoint codepoint,
    mjb_character *character
);

Fill character with the Unicode Character Database record of a codepoint: name, category, combining class, bidirectional category, decomposition, numeric values, mirrored flag, and simple case mappings. When the library is compiled with MJB_FEATURE_CHARACTER_NAMES=OFF the name field is reported as Codepoint U+XXXX.

Returns

  • MJB_STATUS_OK — The character was found and filled
  • MJB_STATUS_INVALID_ARGUMENT — character is NULL or the codepoint is not valid
  • MJB_STATUS_NOT_FOUND — The codepoint is not assigned

Example

mjb_character character;

if(mjb_codepoint_character(0x022A, &character) != MJB_STATUS_OK) {
    return 1;
}

// U+022A lowercase: U+022B
printf("U+%04X lowercase: U+%04X", character.codepoint, character.lowercase);

Related

Specifications

mjb_string_is_normalized

Check if a string is normalized to NFC/NFKC/NFD/NFKD form.

mjb_quick_check_result mjb_string_is_normalized(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_normalization form
);

Run the normalization quick-check on a string without allocating. MJB_QC_MAYBE means the string may still be normalized, and only a full normalization pass with mjb_normalize can decide.

Returns

  • MJB_QC_YES — The string is normalized to the requested form
  • MJB_QC_NO — The string is not normalized
  • MJB_QC_MAYBE — Inconclusive: a full normalization is needed to decide

Related

Specifications

mjb_string_encoding

Return the string encoding (the most probable).

mjb_encoding mjb_string_encoding(
    const char *buffer,
    size_t byte_length
);

mjb_string_encoding reports BOM-derived UTF-16/UTF-32 schemes with the generic family bit plus the resolved endian bit. Passing that detected value consumes the leading BOM as a signature. Passing an explicit-endian encoding such as MJB_ENC_UTF_16BE preserves an initial U+FEFF as text. When flags overlap, as with a UTF-32LE BOM that also has the UTF-16LE BOM prefix, decoding gives UTF-32 precedence.

mjb_string_is_ascii

Return true if the string is encoded in ASCII.

bool mjb_string_is_ascii(
    const char *buffer,
    size_t byte_length
);

mjb_string_is_utf8

Return true if the string is encoded in UTF-8.

bool mjb_string_is_utf8(
    const char *buffer,
    size_t byte_length
);

mjb_string_is_utf16

Return true if the string is encoded in UTF-16BE or UTF-16LE.

bool mjb_string_is_utf16(
    const char *buffer,
    size_t byte_length
);

mjb_string_length

Return the length of a string.

size_t mjb_string_length(
    const char *buffer,
    size_t max_length,
    mjb_encoding encoding
);

mjb_string_each_character

Run a callback for each character of a string.

mjb_status mjb_string_each_character(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_string_each_character_fn callback
);

mjb_codepoint_numeric_value

Return the numeric value of a codepoint.

mjb_status mjb_codepoint_numeric_value(
    mjb_codepoint codepoint,
    mjb_numeric_value *value
);

Return the numeric value of a codepoint, if any. If the codepoint has no numeric value, value->decimal and value->digit are set to MJB_NUMBER_NOT_VALID (-1).

Returns

  • MJB_STATUS_OK — The character was found and filled
  • MJB_STATUS_INVALID_ARGUMENT — value is NULL or the codepoint is not valid

Example

mjb_numeric_value num;

if(mjb_codepoint_numeric_value(0x0031, &num) != MJB_STATUS_OK) { // U+0031 = 1
    return 1;
}

// decimal=1, digit=1, numeric=1
printf("decimal=%d, digit=%d, numeric=%s", num.decimal, num.digit, num.numeric);

if(mjb_codepoint_numeric_value(0x00BD, &num) != MJB_STATUS_OK) { // U+00BD = '½'
    return 1;
}

// decimal=-1, digit=-1, numeric=1/2
printf("decimal=%d, digit=%d, numeric=%s", num.decimal, num.digit, num.numeric);

Specifications

mjb_codepoint_is_valid

Return true if the codepoint is valid.

bool mjb_codepoint_is_valid(
    mjb_codepoint codepoint
);

mjb_codepoint_is_graphic

Return true if the codepoint is graphic.

bool mjb_codepoint_is_graphic(
    mjb_codepoint codepoint
);

mjb_codepoint_is_combining

Return true if the codepoint is combining.

bool mjb_codepoint_is_combining(
    mjb_codepoint codepoint
);

mjb_codepoint_is_cjk_ideograph

Return if the codepoint is CJK ideograph.

bool mjb_codepoint_is_cjk_ideograph(
    mjb_codepoint codepoint
);

mjb_codepoint_is_cjk_ext

Return if the codepoint is CJK extension.

bool mjb_codepoint_is_cjk_ext(
    mjb_codepoint codepoint
);

mjb_category_is_graphic

Return true if the category is graphic.

bool mjb_category_is_graphic(
    mjb_category category
);

mjb_category_is_combining

Return true if the category is combining.

bool mjb_category_is_combining(
    mjb_category category
);

mjb_codepoint_plane

Return the plane of the codepoint.

mjb_plane mjb_codepoint_plane(
    mjb_codepoint codepoint
);

mjb_plane_is_valid

Return true if the plane is valid.

bool mjb_plane_is_valid(
    mjb_plane plane
);

mjb_plane_name

Return the name of a plane, NULL if the plane specified is not valid.

const char *mjb_plane_name(
    mjb_plane plane,
    bool abbreviation
);

API section

Sorting and comparison

Compare and sort strings with the Unicode Collation Algorithm.

2 functions

mjb_string_compare

Compare two strings using UCA.

int mjb_string_compare(
    const char *s1,
    size_t s1_byte_length,
    mjb_encoding s1_encoding,
    const char *s2,
    size_t s2_byte_length,
    mjb_encoding s2_encoding,
    mjb_collation_mode mode
);

Compare two strings using the Unicode Collation Algorithm and the default collation element table (DUCET), with strcmp-style semantics.

Returns

  • < 0 — The first string collates before the second
  • 0 — The strings are equal under UCA
  • > 0 — The first string collates after the second

Related

Specifications

mjb_collation_key

Generate a UCA sort key for a string.

mjb_status mjb_collation_key(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_collation_mode mode,
    mjb_result *result
);

Generate a binary sort key for a string. Sort keys of different strings can be compared with memcmp and yield the same order as mjb_string_compare. Useful when the same strings are compared many times, such as sorting or database indexing.

Returns

  • MJB_STATUS_OK — The sort key was generated
  • MJB_STATUS_INVALID_ARGUMENT — result is NULL, or buffer is NULL with a non-zero size
  • MJB_STATUS_OVERFLOW — The sort key size would overflow
  • MJB_STATUS_NO_MEMORY — Allocation failed

Related

Specifications

API section

Security

Detect visually confusable Unicode text.

2 functions

mjb_string_is_identifier

Return true if the string is a valid Unicode identifier (Unicode 17.0.0 UAX #31).

bool mjb_string_is_identifier(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_identifier_profile profile
);

Validate a string as a Unicode identifier: the first character must be a valid identifier start and the following ones valid identifier continuations, using ID_Start/ID_Continue for the DEFAULT profile or XID_Start/XID_Continue for the NFKC profile.

Related

Specifications

mjb_string_is_confusable

Return true if two strings are visually confusable (Unicode 17.0.0 UTS #39 Section 4): skeleton(s1) == skeleton(s2).

bool mjb_string_is_confusable(
    const char *s1,
    size_t s1_byte_length,
    mjb_encoding s1_encoding,
    const char *s2,
    size_t s2_byte_length,
    mjb_encoding s2_encoding
);

Compute the confusable skeleton of both strings and return true when the skeletons are equal, meaning the two strings are visually confusable, such as "good" and "gооd" with Cyrillic о.

Related

Specifications

API section

Segmentation

Segment text into grapheme clusters, words, and sentences.

7 functions

mjb_break_grapheme_cluster

Grapheme cluster breaking.

mjb_break_type mjb_break_grapheme_cluster(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_next_state *state
);

Iterate the grapheme cluster (user-perceived character) boundaries of a string. Call repeatedly with the same state until it reports the end of the string.

Related

Specifications

mjb_truncate

Return the number of bytes that form the first `max_graphemes` grapheme cluster segments.

size_t mjb_truncate(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    size_t max_graphemes
);

mjb_truncate_word

Return the number of bytes that form the first max_segments word-break segments.

size_t mjb_truncate_word(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    size_t max_segments
);

mjb_truncate_word_width

Return the number of bytes whose word-break segments fit within max_columns display columns.

size_t mjb_truncate_word_width(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_width_context context,
    size_t max_columns
);

API section

Bidirectional text

Resolve embedding levels, reorder lines, and get line runs for bidirectional text.

4 functions

mjb_bidi_resolve

Resolve bidirectional text (TR9) for a paragraph.

mjb_status mjb_bidi_resolve(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_direction direction,
    mjb_bidi_paragraph *result
);

Resolve the embedding levels of a paragraph following the Unicode Bidirectional Algorithm. The resolved paragraph can then be split into lines and reordered visually with mjb_bidi_reorder_line and mjb_bidi_line_runs.

Returns

  • MJB_STATUS_OK — The paragraph was resolved
  • MJB_STATUS_INVALID_ARGUMENT — result is NULL, or buffer is NULL with a non-zero size
  • MJB_STATUS_OVERFLOW — The paragraph size would overflow
  • MJB_STATUS_NO_MEMORY — Allocation failed

Related

Specifications

mjb_bidi_free

Free a bidi paragraph allocated by mjb_bidi_resolve.

void mjb_bidi_free(
    mjb_bidi_paragraph *paragraph
);

API section

Emoji

Inspect emoji sequences and properties.

9 functions

API section

Display width

Compute the number of display columns a string occupies in a terminal.

3 functions

mjb_truncate_width

Return the number of bytes whose grapheme clusters fit within max_columns display columns.

size_t mjb_truncate_width(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_width_context context,
    size_t max_columns
);

mjb_display_width

Return the display width of a string.

mjb_status mjb_display_width(
    const char *buffer,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_width_context context,
    size_t *width
);

Compute the number of display columns a string occupies in a terminal, accounting for wide and ambiguous East Asian characters, combining marks, and emoji sequences.

Returns

  • MJB_STATUS_OK — The width was computed
  • MJB_STATUS_INVALID_ARGUMENT — width is NULL, or buffer is NULL with a non-zero size
  • MJB_STATUS_OVERFLOW — The width would overflow

Related

Specifications

API section

Hangul language

Inspect Hangul syllables and perform Hangul syllable decomposition and composition.

8 functions

mjb_codepoint_is_hangul_l

Return if the codepoint is a hangul L.

bool mjb_codepoint_is_hangul_l(
    mjb_codepoint codepoint
);

mjb_codepoint_is_hangul_v

Return if the codepoint is a hangul V.

bool mjb_codepoint_is_hangul_v(
    mjb_codepoint codepoint
);

mjb_codepoint_is_hangul_t

Return if the codepoint is a hangul T.

bool mjb_codepoint_is_hangul_t(
    mjb_codepoint codepoint
);

mjb_codepoint_is_hangul_syllable

Return if the codepoint is a hangul syllable.

bool mjb_codepoint_is_hangul_syllable(
    mjb_codepoint codepoint
);

mjb_hangul_syllable_name

Return hangul syllable name.

mjb_status mjb_hangul_syllable_name(
    mjb_codepoint codepoint,
    char *buffer,
    size_t byte_length
);

mjb_hangul_syllable_decomposition

Hangul syllable decomposition.

mjb_status mjb_hangul_syllable_decomposition(
    mjb_codepoint codepoint,
    mjb_codepoint *codepoints
);

mjb_hangul_syllable_composition

Hangul syllable composition.

size_t mjb_hangul_syllable_composition(
    mjb_buffer_character *characters,
    size_t characters_len
);

API section

Utilities

Use locales, memory hooks, version details, and low-level string helpers.

12 functions

mjb_locale_parse

Parse a BCP 47 language tag.

mjb_status mjb_locale_parse(
    const char *id,
    size_t byte_length,
    mjb_encoding encoding,
    mjb_locale_id *locale,
    mjb_error *error
);

Parse a BCP 47 language tag, such as sr-Latn-RS, into its components: language, extended language, script, region, variant, extensions, private use, and grandfathered tags. Parsing is strict: malformed tags are rejected and error is filled with the failure reason.

Returns

  • MJB_STATUS_OK — The tag was parsed and locale filled
  • MJB_STATUS_INVALID_ARGUMENT — An argument is NULL or the tag is not a valid BCP 47 language tag
  • MJB_STATUS_NO_MEMORY — Allocation failed

Related

Specifications

mjb_locale_set

Set current locale used by locale-sensitive casing.

mjb_status mjb_locale_set(
    unsigned int locale
);

Set the process-global locale used by mjb_case. The default locale is MJB_LOCALE_EN, and mjb_shutdown resets it to MJB_LOCALE_EN. Only MJB_LOCALE_TR, MJB_LOCALE_AZ, and MJB_LOCALE_LT currently tailor casing. Other valid locale values are accepted but do not change Unicode algorithm behavior.

Returns

  • MJB_STATUS_OK — The locale was set
  • MJB_STATUS_INVALID_ARGUMENT — locale is not a valid mjb_locale value

Related

mjb_result_free

Free a mjb_result.

mjb_status mjb_result_free(
    mjb_result *result
);

Free the memory allocated for a mjb_result. The result pointer is set to NULL.

Returns

  • MJB_STATUS_OK — The result was freed
  • MJB_STATUS_INVALID_ARGUMENT — result is NULL

mjb_version_number

Output the current library version number (MJB_VERSION_NUMBER).

unsigned int mjb_version_number(void);

Output the current library version number as an unsigned integer.

Related

mjb_unicode_version

Output the current supported Unicode version (MJB_UNICODE_VERSION).

const char *mjb_unicode_version(void);

Output the current supported Unicode version as a string, such as "15.0.0".

Related

mjb_set_memory_functions

Set the library memory functions.

mjb_status mjb_set_memory_functions(
    mjb_alloc_fn alloc_fn,
    mjb_realloc_fn realloc_fn,
    mjb_free_fn free_fn
);

Replace the allocator used by the library for all internal allocations and for the buffers returned in mjb_result. Must be called before any other library call.

Related

mjb_shutdown

Shutdown the library. Not needed to be called.

void mjb_shutdown(void);

mjb_alloc

Allocate memory.

void *mjb_alloc(
    size_t byte_length
);

Allocate memory using the allocator set by mjb_set_memory_functions. If no allocator is set, the default allocator is used.

Related

mjb_realloc

Reallocate memory.

void *mjb_realloc(
    void *ptr,
    size_t new_size
);

Reallocate memory using the allocator set by mjb_set_memory_functions. If no allocator is set, the default allocator is used.

Related

mjb_free

Free memory.

void mjb_free(
    void *ptr
);

Free memory using the allocator set by mjb_set_memory_functions. If no allocator is set, the default allocator is used.

Related