Skip to content
FutureBlitz

    Install

    Import std.text.unicode

    std.text.unicode

    std.text.unicode: what Unicode's data says about characters and text, through the runtime's fb_unicode_ functions over utf8proc: each character's general category, normal forms, case folding, and the columns text takes in a terminal.

    std/text/unicode.fb

    Types

    UnicodeCategory enum 

    Public Enum UnicodeCategory

    A character's general category, in Unicode's order, as Category gives it.

    std/text/unicode.fb, line 16

    • UNASSIGNED enum member #

      Cn, a code point with no character.

    • UPPERCASE_LETTER enum member #

      Lu, an uppercase letter.

    • LOWERCASE_LETTER enum member #

      Ll, a lowercase letter.

    • TITLECASE_LETTER enum member #

      Lt, a digraph whose first part is uppercase.

    • MODIFIER_LETTER enum member #

      Lm, a modifier letter.

    • OTHER_LETTER enum member #

      Lo, another letter, like an ideograph.

    • NONSPACING_MARK enum member #

      Mn, a mark that takes no space.

    • SPACING_MARK enum member #

      Mc, a mark that takes space.

    • ENCLOSING_MARK enum member #

      Me, a mark that encloses its base character.

    • DECIMAL_NUMBER enum member #

      Nd, a decimal digit.

    • LETTER_NUMBER enum member #

      Nl, a number of letters, like a Roman numeral.

    • OTHER_NUMBER enum member #

      No, another number, like a fraction or a superscript.

    • CONNECTOR_PUNCTUATION enum member #

      Pc, a connector, like '_'.

    • DASH_PUNCTUATION enum member #

      Pd, a dash or a hyphen.

    • OPEN_PUNCTUATION enum member #

      Ps, an opening bracket.

    • CLOSE_PUNCTUATION enum member #

      Pe, a closing bracket.

    • INITIAL_PUNCTUATION enum member #

      Pi, an opening quotation mark.

    • FINAL_PUNCTUATION enum member #

      Pf, a closing quotation mark.

    • OTHER_PUNCTUATION enum member #

      Po, other punctuation.

    • MATH_SYMBOL enum member #

      Sm, a math symbol.

    • CURRENCY_SYMBOL enum member #

      Sc, a currency symbol.

    • MODIFIER_SYMBOL enum member #

      Sk, a modifier symbol, like an accent on its own.

    • OTHER_SYMBOL enum member #

      So, another symbol.

    • SPACE_SEPARATOR enum member #

      Zs, a space.

    • LINE_SEPARATOR enum member #

      Zl, U+2028, the line separator.

    • PARAGRAPH_SEPARATOR enum member #

      Zp, U+2029, the paragraph separator.

    • CONTROL enum member #

      Cc, a control character.

    • FORMAT enum member #

      Cf, a format character, like a zero-width joiner.

    • SURROGATE enum member #

      Cs, a UTF-16 surrogate.

    • PRIVATE_USE enum member #

      Co, a private use character.

    NormalForm enum 

    Public Enum NormalForm

    The ways Normalize can write the same text.

    std/text/unicode.fb, line 80

    • NFC enum member #

      Composed.

    • NFD enum member #

      Decomposed.

    • NFKC enum member #

      Composed, with compatibility characters, like "fi" or "²", as their plain ones.

    • NFKD enum member #

      Decomposed, with compatibility characters, like "fi" or "²", as their plain ones.

    Functions

    Category function 

    Public Function Category(c: Char): UnicodeCategory

    c's general category: a letter of some case, a mark, a number, punctuation, a symbol, a separator, or another kind.

    std/text/unicode.fb, line 93

    Normalize function 

    @PanicsAtCaller Public Function Normalize(text: String, form: NormalForm = NormalForm.NFC): String

    text in the normal form form, so text that looks the same is the same: "é" as one character in NFC, and as "e" and its accent in NFD. Panics past the most a String holds.

    std/text/unicode.fb, line 99

    IsNormalized function 

    Public Function IsNormalized(text: String, form: NormalForm = NormalForm.NFC): Bool

    Whether text is in the normal form form already. One past the most a String holds isn't.

    std/text/unicode.fb, line 107

    CaseFold function 

    @PanicsAtCaller Public Function CaseFold(text: String): String

    text case-folded, as Unicode has it, so two texts that differ only by case fold to the same: "Straße" and "STRASSE" both to "strasse". Panics past the most a String holds.

    std/text/unicode.fb, line 113

    DisplayWidth function 

    Public Function DisplayWidth(c: Char): Int

    The columns c takes in a terminal: 2 for a wide one, like "中" or most emoji, 0 for a mark or a control character, and 1 for the rest.

    std/text/unicode.fb, line 122

    DisplayWidth function 

    Public Function DisplayWidth(text: String): Int

    The columns text takes in a terminal, each grapheme cluster as many as its widest character, so "e" with its accent is 1, and an emoji made of several is 2. A flag is 2, and so is a character that U+FE0F asks to be drawn as emoji, like "\u{2764}\u{FE0F}", though it's 1 alone.

    std/text/unicode.fb, line 127

    Report an issue with this page or with FutureBlitz on GitHub.