ae.sys.data

Reference-counted objects for handling large amounts of raw data.

Using the Data type will only place a small object in managed memory, keeping the actual bytes in unmanaged memory.

A proxy class (Memory) is used to safely allow multiple references to the same block of unmanaged memory.

When the Memory object is destroyed, the unmanaged memory is deallocated.

This has the following advantage over using managed memory (regular D arrays):

- Faster allocation and deallocation, since memory is requested from the OS directly as whole pages.

- Greatly reduced chance of memory leaks (on 32-bit platforms) due to stray pointers.

- Overall improved GC performance due to reduced size of managed heap.

- Memory is immediately returned to the OS when no more references remain.

- Unlike D arrays, Data objects know their reference count, enabling things like copy-on-write or safely casting away constness.

Members

Aliases

CMemory alias CMemory = DynamicMemory!CAllocator: Wrapper for data in RAM, allocated from the C standard library. Used for small objects.
Data alias Data = TData!ubyte: The most common use case of manipulating unmanaged memory is working with raw bytes, whether they're received from the network, read from a file, or elsewhere.
DataVec (from ae.sys.dataset) deprecated alias DataVec = Vec!Data via public import ae.sys.dataset : copyTo, joinData, joinToHeap, DataVec, shift, bytes, DataSetBytes;: A vector of Data with deterministic lifetime.
DataWrapper deprecated alias DataWrapper = Memory: Undocumented in source.
OSMemory alias OSMemory = DynamicMemory!OSAllocator: Wrapper for data in RAM, allocated from the OS.

Classes

DynamicMemory class DynamicMemory(Allocator): Some form of dynamically-allocated memory. Implementation is provided by the Allocator parameter.
Memory class Memory: Base abstract class which owns a block of memory.

Functions

as T as(Data data): Undocumented in source. Be warned that the author may not have intended to support it.
copyTo (from ae.sys.dataset) deprecated void[] copyTo(R data, void[] buffer) via public import ae.sys.dataset : copyTo, joinData, joinToHeap, DataVec, shift, bytes, DataSetBytes;: Undocumented in source.
debugLog void debugLog(char* s, Args args): Undocumented in source. Be warned that the author may not have intended to support it.
debugStackTrace void debugStackTrace(): Undocumented in source. Be warned that the author may not have intended to support it.
joinData (from ae.sys.dataset) deprecated TData!(DataElementType!(ElementType!R)) joinData(R data) via public import ae.sys.dataset : copyTo, joinData, joinToHeap, DataVec, shift, bytes, DataSetBytes;: Join an array of Data to a single Data.
pop T pop(Data data): Undocumented in source. Be warned that the author may not have intended to support it.
setGCThreshold void setGCThreshold(size_t value): Set threshold of allocated memory to trigger a garbage collection.
shift (from ae.sys.dataset) deprecated DataVec shift(DataVec data, size_t amount) via public import ae.sys.dataset : copyTo, joinData, joinToHeap, DataVec, shift, bytes, DataSetBytes;: Remove and return the specified number of bytes from the given Data array.
unmanagedAlloc void* unmanagedAlloc(size_t sz): Undocumented in source. Be warned that the author may not have intended to support it.
unmanagedDelete void unmanagedDelete(C c): Delete a class instance created with unmanagedNew.
unmanagedFree void unmanagedFree(void* p): Undocumented in source. Be warned that the author may not have intended to support it.
unmanagedNew C unmanagedNew(Args args): Allocate and construct a new class in malloc'd memory.

Properties

bytes (from ae.sys.dataset) deprecated DataSetBytes bytes [@property getter] via public import ae.sys.dataset : copyTo, joinData, joinToHeap, DataVec, shift, bytes, DataSetBytes;: Undocumented in source.
joinToHeap (from ae.sys.dataset) deprecated void[] joinToHeap [@property getter] via public import ae.sys.dataset : copyTo, joinData, joinToHeap, DataVec, shift, bytes, DataSetBytes;: Undocumented in source.

Static variables

allocCount uint allocCount;: How many allocations have been done so far.
collectThreshold size_t collectThreshold;: Threshold of allocated memory to trigger a collect.
dataCount uint dataCount;: How many Memory instances there are live currently.
dataMemory size_t dataMemory;
dataMemoryPeak size_t dataMemoryPeak;: How many bytes are currently in Data-owned memory.

Structs

DataSetBytes (from ae.sys.dataset) deprecated struct DataSetBytes via public import ae.sys.dataset : copyTo, joinData, joinToHeap, DataVec, shift, bytes, DataSetBytes;: Return a type that's indexable to access individual bytes, and sliceable to get an array of Data over the specified byte range. No actual Data concatenation is done.
TData struct TData(T): A reference to a reference-counted block of memory. Represents a slice of data, which may be backed by managed memory, unmanaged memory, memory-mapped files, etc.

Templates

DataElementType template DataElementType(D): Get the underlying type of a TData. (For Data, this will be ubyte.)

Variables

allocatedThreshold size_t allocatedThreshold;: Counter towards the threshold.