The Unicode Bidirectional Algorithm (UAX #9), which lays out text that mixes left-to-right scripts, such as Latin, with right-to-left ones, such as Hebrew and Arabic.
Text is stored in logical order, the order it is typed and read in. resolve/2 computes the
embedding level of every character of a paragraph: even levels are left to right, odd levels
right to left. A line of the paragraph is then reordered for display with reorder/2, split
into directional runs with runs/2, or rendered in visual order with visual/2:
iex> paragraph = Mojibake.Bidi.resolve!("abc אבג")
iex> paragraph.direction
:ltr
iex> Mojibake.Bidi.levels(paragraph)
[0, 0, 0, 0, 1, 1, 1]
iex> Mojibake.Bidi.visual(paragraph)
"abc גבא"Positions and lines
Positions are indices into the characters of the paragraph, 0 to paragraph.length - 1,
not codepoint indices of the text: rule X9 removes the embedding and override controls and the
Boundary Neutrals, such as U+200D ZERO WIDTH JOINER, from the paragraph. chars/1 gives the
byte offset of each character in the text.
The functions that take a line take a range of positions with step 1, such as 0..9, and
default to the whole paragraph. Break a paragraph into lines yourself, with
Mojibake.line_breaks/1 and the width of your output, and lay out each line separately.
A paragraph is one paragraph: split text at paragraph separators, such as "\n", first.
Resolving it is the expensive step; reordering a line of it is cheap, and the resolved
paragraph can be shared between processes.
Summary
Types
A character of a resolved paragraph
The base direction of a paragraph. :auto takes it from the first strong character (rules P2
and P3), and is left to right if there is none.
A run of characters with the same embedding level on a reordered line
Functions
Return the characters of paragraph, in logical order. See char_info/0.
Return the paragraph direction of text: the direction of its first strong character, or
:ltr if it has none (rules P2 and P3). Raises Mojibake.Error if text is not well-formed
UTF-8.
Return the resolved embedding level of each character of paragraph, in logical order.
Reorder a line of paragraph for display (rules L1 and L2). Returns the positions of the line
in visual order, from left to right.
Resolve the embedding levels of text, one paragraph, with base direction direction.
Like resolve/2, but raises Mojibake.Error.
Split a reordered line of paragraph into runs of the same embedding level, from left to
right. See run/0.
Return a line in visual order, with mirrored characters, such as parentheses in right-to-left text, replaced by their mirroring glyphs (rule L4). The controls that rule X9 removes are left out.
Types
@type char_info() :: %{ codepoint: non_neg_integer(), offset: non_neg_integer(), level: non_neg_integer(), class: Mojibake.Char.bidi_class(), mirror: non_neg_integer() | nil }
A character of a resolved paragraph:
codepoint- the characteroffset- its byte offset in the paragraph textlevel- its resolved embedding level; odd levels are right to leftclass- its class after the weak and neutral rules (W1-W7, N1-N2) resolved it, an atom ofMojibake.Char.bidi_class/0.Mojibake.Char.bidi_class/1gives its original class.mirror- the Bidi_Mirroring_Glyph to display instead of the character (rule L4), when it is mirrored and its level is odd, elsenil
@type direction() :: :ltr | :rtl | :auto
The base direction of a paragraph. :auto takes it from the first strong character (rules P2
and P3), and is left to right if there is none.
@type run() :: %{ visual: Range.t(), logical: Range.t(), level: non_neg_integer(), direction: :ltr | :rtl }
A run of characters with the same embedding level on a reordered line:
visual- the positions of the run in the visual order, indices into the list thatreorder/2returns for the same linelogical- the paragraph positions the run covers, in logical orderlevel- the embedding level of the rundirection-:rtlfor odd levels,:ltrotherwise
Functions
@spec chars(Mojibake.Bidi.Paragraph.t()) :: [char_info()]
Return the characters of paragraph, in logical order. See char_info/0.
Examples
iex> "(א)" |> Mojibake.Bidi.resolve!() |> Mojibake.Bidi.chars() |> hd()
%{codepoint: ?(, offset: 0, level: 1, class: :right_to_left, mirror: ?)}
@spec direction(String.t()) :: :ltr | :rtl
Return the paragraph direction of text: the direction of its first strong character, or
:ltr if it has none (rules P2 and P3). Raises Mojibake.Error if text is not well-formed
UTF-8.
Examples
iex> Mojibake.Bidi.direction("Hello")
:ltr
iex> Mojibake.Bidi.direction("123 שלום")
:rtl
iex> Mojibake.Bidi.direction("123")
:ltr
@spec levels(Mojibake.Bidi.Paragraph.t()) :: [non_neg_integer()]
Return the resolved embedding level of each character of paragraph, in logical order.
Examples
iex> "ab אב 12" |> Mojibake.Bidi.resolve!() |> Mojibake.Bidi.levels()
[0, 0, 0, 1, 1, 1, 2, 2]
@spec reorder(Mojibake.Bidi.Paragraph.t(), Range.t() | nil) :: [non_neg_integer()]
Reorder a line of paragraph for display (rules L1 and L2). Returns the positions of the line
in visual order, from left to right.
Raises ArgumentError if line is not a range of positions of the paragraph.
Examples
iex> paragraph = Mojibake.Bidi.resolve!("abc אבג")
iex> Mojibake.Bidi.reorder(paragraph)
[0, 1, 2, 3, 6, 5, 4]
iex> Mojibake.Bidi.reorder(paragraph, 4..6)
[6, 5, 4]
@spec resolve(String.t(), direction()) :: {:ok, Mojibake.Bidi.Paragraph.t()} | {:error, Mojibake.Error.t()}
Resolve the embedding levels of text, one paragraph, with base direction direction.
Returns {:error, %Mojibake.Error{}} if text is not well-formed UTF-8.
Examples
iex> {:ok, paragraph} = Mojibake.Bidi.resolve("שלום world")
iex> {paragraph.direction, paragraph.level, paragraph.length}
{:rtl, 1, 10}
iex> {:ok, paragraph} = Mojibake.Bidi.resolve("שלום world", :ltr)
iex> paragraph.direction
:ltr
iex> Mojibake.Bidi.resolve(<<?a, 0xFF>>)
{:error, %Mojibake.Error{reason: :malformed_input, offset: 1, length: 1, detail: :invalid_leading_byte}}
@spec resolve!(String.t(), direction()) :: Mojibake.Bidi.Paragraph.t()
Like resolve/2, but raises Mojibake.Error.
Examples
iex> Mojibake.Bidi.resolve!("مرحبا").direction
:rtl
@spec runs(Mojibake.Bidi.Paragraph.t(), Range.t() | nil) :: [run()]
Split a reordered line of paragraph into runs of the same embedding level, from left to
right. See run/0.
Raises ArgumentError if line is not a range of positions of the paragraph.
Examples
iex> "abc אבג" |> Mojibake.Bidi.resolve!() |> Mojibake.Bidi.runs()
[
%{visual: 0..3, logical: 0..3, level: 0, direction: :ltr},
%{visual: 4..6, logical: 4..6, level: 1, direction: :rtl}
]
@spec visual(Mojibake.Bidi.Paragraph.t(), Range.t() | nil) :: String.t()
@spec visual(String.t(), direction()) :: String.t()
Return a line in visual order, with mirrored characters, such as parentheses in right-to-left text, replaced by their mirroring glyphs (rule L4). The controls that rule X9 removes are left out.
Takes a resolved paragraph and an optional line, or a text and an optional base direction.
Visual order is for display on a device that does not do bidirectional layout itself, such as
a terminal; do not store or process text in it.
Raises ArgumentError if line is not a range of positions of the paragraph, and
Mojibake.Error if text is not well-formed UTF-8.
Examples
iex> Mojibake.Bidi.visual("abc אבג")
"abc גבא"
iex> Mojibake.Bidi.visual("abc (אבג)", :rtl)
"(גבא) abc"
iex> paragraph = Mojibake.Bidi.resolve!("abc אבג")
iex> Mojibake.Bidi.visual(paragraph, 2..5)
"c בא"