ae.utils.text

Utility code related to string and text processing.

Modules

ascii
module ae.utils.text.ascii

Simple (ASCII-only) text-processing functions, for speed and CTFE.

functor
module ae.utils.text.functor

Functor-powered lazy @nogc text formatting.

html
module ae.utils.text.html

ae.utils.text.html

parsefp
module ae.utils.text.parsefp

Pure D code to parse floating-point values. Adapted to nothrow/@nogc from std.conv.

Members

Aliases

CIAsciiString
alias CIAsciiString = NormalizedArray!(immutable(char), s => s.byCodeUnit.map!(std.ascii.toLower))

Case-insensitive ASCII string.

CIUniString
alias CIUniString = NormalizedArray!(immutable(char), s => s.map!(toLower))

Case-insensitive Unicode string.

doubleToString
alias doubleToString = fpToString!double
Undocumented in source.
toLowerHex
alias toLowerHex = toHex!lowerHexDigits
Undocumented in source.

Enums

fpCFormatString
eponymoustemplate fpCFormatString(T)

C format string to exactly format a floating-point type T.

fpCScanString
eponymoustemplate fpCScanString(T)

C format string to scan a floating-point type T.

fpFormatString
eponymoustemplate fpFormatString(T)

Format string for a FP type which includes all necessary significant digits

significantDigits
eponymoustemplate significantDigits(T : real)

How many significant decimal digits does a FP type have (determined empirically - valid for all D FP types on x86/64)

Functions

UTF8ToRaw
ascii UTF8ToRaw(char[] r)

Undo rawToUTF8.

arrayFromHex
ubyte[] arrayFromHex(char[] hex)

Parses hex into an array of bytes. hex.length should be even.

arrayFromHex
void arrayFromHex(char[] hex, ubyte[] buf)

Parses hex into the given array buf.

asText
inout(char)[] asText(inout(ubyte)[] bytes)

Like readText, but with in-memory data. Reverse of ae.utils.array.bytes (for strings).

asciiSplit
T[][] asciiSplit(T[] text)

Like std.string.split (one argument version, which splits by whitespace), but only splits by ASCII and does not autodecode.

asciiStrip
T[] asciiStrip(T[] s)

Like strip, but only removes ASCII whitespace.

camelCaseJoin
string camelCaseJoin(string[] arr)

Join an array of words into a camel-cased string.

contains (from ae.utils.array)
bool contains(T[] str, U[] what) via public import ae.utils.array : contains;

Ditto, for substrings

eatLine
T[] eatLine(T[] s, bool eatIncompleteLines)

Consume a LF or CRLF terminated line from s. Sets s to null and returns the remainder if there is no line terminator in s.

fastReplace
T[] fastReplace(T[] what, T[] from, T[] to)

An implementation of replace optimized for common cases (short strings).

fastSplit
T[][] fastSplit(T[] s, U d)

An implementation of split optimized for common cases. Allocates only once.

findBestMatch
sizediff_t findBestMatch(string[] items, string target, float threshold)

Select best match from a list of items. Returns -1 if none are above the threshold.

forceValidUTF8
string forceValidUTF8(ascii s)

Lossily convert arbitrary data into a valid UTF-8 string.

formatAs
string formatAs(T obj, string fmt)

UFCS helper

fpAsString
FPAsString!T fpAsString(T f)
Undocumented in source. Be warned that the author may not have intended to support it.
fpParse
F fpParse(const(C)[] s)

Parse a floating-point number using the C standard library. Note: might produce slightly different results than e.g. to!double.

fpTryParse
bool fpTryParse(const(C)[] s, F f)

Parse a floating-point number using the C standard library. Note: might produce slightly different results than e.g. to!double.

fromHex
T fromHex(const(C)[] s)

Parses s as a hexadecimal number into an integer of type T.

fromZArray
C[] fromZArray(C[n] arr)
C[] fromZArray(C[] arr)

Return the slice up to the first NUL character, or of the whole array if none is found.

hexDump
string hexDump(const(void)[] b)

Formats binary data as a hex dump (three-column layout consisting of hex offset, byte values in hex, and printable low-ASCII characters).

newlinesToSpaces
T[] newlinesToSpaces(T[] s)

Replaces runs of ASCII whitespace which contain a newline ('\n') into a single space.

normalizeWhitespace
ascii normalizeWhitespace(ascii s)

Replaces all runs of ASCII whitespace with a single space.

nullStringTransform
string nullStringTransform(char[] s)

Where a delegate with this signature is required.

numberToString
string numberToString(T v)

Get shortest string representation of a numeric type that still converts to exactly the same number.

parseHexDigit
ubyte parseHexDigit(char c)

Parse a single hexadecimal digit according to the policy in config.

putFP
void putFP(Writer writer, F v)

Like fpToString, but writes the result to a sink.

randomString
string randomString(int length, string chars)

Generate a random string with the given parameters. std.random is used as the source of randomness. Not cryptographically secure.

rawToUTF8
string rawToUTF8(char[] s)

Convert any data to a valid UTF-8 bytestream, so D's string functions can properly work on it.

sarrayFromHex
void sarrayFromHex(Hex hex, ubyte[N] buf)

Parses hex into the given array buf. Fast version for static arrays of known length.

segmentByWhitespace
T[][] segmentByWhitespace(T[] s)

Covering slice-list of s with interleaved whitespace.

selectBestFrom
string selectBestFrom(string[] items, string target, float threshold)

Select best match from a list of items. Returns null if none are above the threshold.

splitAsciiLines
T[][] splitAsciiLines(T[] text)

Like splitLines, but does not attempt to split on Unicode line endings. Only splits on "\r", "\n", and "\r\n".

splitByCamelCase
string[] splitByCamelCase(string s)

Splits out words from a camel-cased string. All-uppercase words are returned as a single word.

stringDistance
int stringDistance(string s, string t)

Simpler implementation of Levenshtein string distance

stringSimilarity
float stringSimilarity(string string1, string string2)

Return a number between 0.0 and 1.0 indicating how similar two strings are (1.0 if identical)

verbatimWrap
string verbatimWrap(string s, size_t columns, string firstIndent, string indent, size_t tabWidth)

Like std.string.wrap, but preserves whitespace at line start and between (non-wrapped) words.

Imports

DecimalSize (from ae.utils.text.ascii)
deprecated public import ae.utils.text.ascii : DecimalSize;
Undocumented in source.
ascii (from ae.utils.text.ascii)
public import ae.utils.text.ascii : ascii, decimalSize, toDec, toDecFixed, asciiToLower, asciiToUpper;
Undocumented in source.
asciiToLower (from ae.utils.text.ascii)
public import ae.utils.text.ascii : ascii, decimalSize, toDec, toDecFixed, asciiToLower, asciiToUpper;
Undocumented in source.
asciiToUpper (from ae.utils.text.ascii)
public import ae.utils.text.ascii : ascii, decimalSize, toDec, toDecFixed, asciiToLower, asciiToUpper;
Undocumented in source.
decimalSize (from ae.utils.text.ascii)
public import ae.utils.text.ascii : ascii, decimalSize, toDec, toDecFixed, asciiToLower, asciiToUpper;
Undocumented in source.
formatted (from ae.utils.text.functor)
public import ae.utils.text.functor : formatted;
Undocumented in source.
toDec (from ae.utils.text.ascii)
public import ae.utils.text.ascii : ascii, decimalSize, toDec, toDecFixed, asciiToLower, asciiToUpper;
Undocumented in source.
toDecFixed (from ae.utils.text.ascii)
public import ae.utils.text.ascii : ascii, decimalSize, toDec, toDecFixed, asciiToLower, asciiToUpper;
Undocumented in source.

Structs

FPAsString
struct FPAsString(T)

Wraps the result of fpToString in a non-allocating stringifiable struct.

HexParseConfig
struct HexParseConfig

Policy for parseHexDigit.

Templates

eatLine
deprecated template eatLine(OnEof onEof)
Undocumented in source.
fpToString
template fpToString(F)

Get shortest string representation of a FP type that still converts to exactly the same number.

toHex
template toHex(alias digits = hexDigits)

Conversion from bytes to hexadecimal strings.

Meta

License

This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was not distributed with this file, You can obtain one at http://mozilla.org/MPL/2.0/.

Authors

Vladimir Panteleev <ae@cy.md>