Class DbnConstants

Namespace

DatabentoDotNet.Dbn

Assembly

DatabentoDotNet.Dbn.dll

Wire-format constants for Databento Binary Encoding (DBN).

public static class DbnConstants

Inheritance

object

DbnConstants

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

These mirror the normative definitions in the databento/dbn Rust crate. Changing any value here is a wire-format change, not an implementation detail.

Fields

FixedPriceScale

Fixed-precision price scale: every 1 unit is 1e-9.

public const long FixedPriceScale = 1000000000

Field Value

long

MagicLength

Length of Magic in bytes.

public const int MagicLength = 3

Field Value

int

MaxMetadataLength

The largest metadata block this decoder will make room for: 512 MiB, excluding the prelude. A stream declaring more is rejected as corrupt or hostile.

public const int MaxMetadataLength = 536870912

Field Value

int

Remarks

The prelude's length field is four bytes wide and arrives before a single byte of it has been validated, so without a ceiling eight attacker-chosen bytes on a socket turn straight into a multi-gigabyte allocation. That is the first byte of an unauthenticated live stream, so it must fail as a DbnDecodeException like every other malformed input, not as an OutOfMemoryException.

Why not lower. Metadata is not small in the general case. An ALL_SYMBOLS definition query over a large venue emits roughly 71·symbols + Σ(75 + 79·intervals) bytes of variable section, which for a query on the order of a million instruments over a real date range is tens to hundreds of megabytes. A 16 MiB cap — the obvious first guess — rejects ordinary production data.

Why not higher. 2 GiB is where int arithmetic stops working and where a single forged length field is worth mounting as a denial of service. 512 MiB sits comfortably above the largest legitimate block anyone has a reason to send and comfortably below the point where honouring it costs the process.

This is a bound, not a fix, and as of issue #31 it is fully retired as one: nothing in this codec sizes an allocation from the declared length any more. The structural answer was always to allocate as the bytes arrive rather than to buffer a whole block on the strength of a field nothing has validated. Decode(Stream, VersionUpgradePolicy) reads that way as of issue #12; DbnFsm — the path both DbnDecoder and the live client read through — grows its read buffer that way as of issue #31.

What the ceiling still does is worth keeping. It rejects an absurd length before the arithmetic that derives a capacity from it can overflow, it turns a forged prelude into a DbnDecodeException at the first eight bytes rather than at some later symptom, and it caps how much memory a peer can make this process hold by actually sending the bytes. A limit on what will be buffered is a different thing from a limit on what will be believed; only the second one has been removed.

MaxRecordLength

Largest possible record: InstrumentDefMsg (520) plus an 8-byte ts_out. The read buffer must never be smaller than this.

public const int MaxRecordLength = 528

Field Value

int

MetadataDatasetCstrLength

Width of the metadata header's dataset C-string field, NUL padding included.

public const int MetadataDatasetCstrLength = 16

Field Value

int

MetadataFixedLength

Length of the fixed portion of the metadata header, following the prelude.

public const int MetadataFixedLength = 100

Field Value

int

Remarks

The same 100 bytes in every DBN version, even though the versions lay out different fields inside it: v1 spends 8 bytes on a deprecated record_count and 47 on reserved padding, while v2 and v3 spend 2 bytes on symbol_cstr_len and 53 on padding. Both add up to 100, so the total is version-independent and the offsets are not.

MetadataPreludeLength

Magic (3) + version (1) + metadata length (4). Read this before anything else.

public const int MetadataPreludeLength = 8

Field Value

int

MetadataReservedLength

Length of the reserved run in the DBN v2 and v3 metadata header.

public const int MetadataReservedLength = 53

Field Value

int

MetadataReservedLengthV1

Length of the reserved run in the DBN v1 metadata header.

public const int MetadataReservedLengthV1 = 47

Field Value

int

Remarks

Six bytes shorter than MetadataReservedLength: v1 spends 8 bytes on a deprecated record_count where v2 spends 2 on symbol_cstr_len, and the reserved run absorbs the 6-byte difference so the fixed section still totals MetadataFixedLength.

NullLimit

Sentinel written to the metadata header when the query had no record limit.

public const ulong NullLimit = 0

Field Value

ulong

Remarks

Zero, not MaxValue: the metadata header's limit and end fields are both 64-bit and both nullable, but they use opposite sentinels. Confusing them turns "no limit" into a limit of 18 quintillion records, or an open-ended query end into the UNIX epoch.

NullRecordCount

Sentinel written to the metadata header when the record count is unknown.

public const ulong NullRecordCount = 18446744073709551615

Field Value

ulong

NullSchema

Sentinel written to the metadata header when no schema applies (mixed-schema streams).

public const ushort NullSchema = 65535

Field Value

ushort

NullStype

Sentinel written to the metadata header when no symbol type applies.

public const byte NullStype = 255

Field Value

byte

RecordLengthMultiplier

Length is expressed in 32-bit words; multiply by this for bytes.

public const int RecordLengthMultiplier = 4

Field Value

int

SymbolCstrLength

Length of a symbol C-string in DBN v3.

public const int SymbolCstrLength = 71

Field Value

int

SymbolCstrLengthV1

Length of a symbol C-string in DBN v1. Retained for decoding older streams.

public const int SymbolCstrLengthV1 = 22

Field Value

int

UndefOrderSize

Sentinel for an absent order size.

public const uint UndefOrderSize = 4294967295

Field Value

uint

UndefPrice

Sentinel for an absent price.

public const long UndefPrice = 9223372036854775807

Field Value

long

UndefStatQuantity

Sentinel for an absent StatMsg quantity in DBN v3.

public const long UndefStatQuantity = 9223372036854775807

Field Value

long

UndefStatQuantityV1

Sentinel for an absent StatMsg quantity in DBN v1 and v2, where the field is 32-bit.

public const int UndefStatQuantityV1 = 2147483647

Field Value

int

Remarks

Upgrading a v1 or v2 record must translate this value to UndefStatQuantity rather than widening it. A plain widening turns "no quantity" into the literal quantity 2,147,483,647, which looks entirely plausible in a market-data feed and no round-trip test would catch it.

UndefTimestamp

Sentinel for an absent timestamp.

public const ulong UndefTimestamp = 18446744073709551615

Field Value

ulong

Version

The highest DBN version this library encodes. Versions 1 and 2 remain decodable.

public const byte Version = 3

Field Value

byte

Properties

Magic

Magic prefix at the start of every DBN stream.

public static ReadOnlySpan<byte> Magic { get; }

Property Value

ReadOnlySpan<byte>