Table of Contents

Class DbnDecoder

Namespace
DatabentoDotNet.Dbn
Assembly
DatabentoDotNet.Dbn.dll

Decodes a DBN stream — a file, a memory buffer, a socket — into metadata and records, handling Zstandard framing transparently.

public sealed class DbnDecoder : IDisposable
Inheritance
DbnDecoder
Implements
Inherited Members

Examples

using DatabentoDotNet.Dbn;

// .dbn or .dbn.zst — the Zstandard frame magic is detected, not declared.
using var decoder = new DbnDecoder(File.OpenRead("data.dbn.zst"));

Metadata metadata = decoder.Metadata!;
Console.WriteLine($"DBN v{metadata.Version} {metadata.Dataset}, compressed: {decoder.IsCompressed}");

var trades = 0;
while (decoder.TryNextRecord(out RecordRef record))
{
    // `record` points into the decoder's own buffer and is valid only until the next call on it.
    if (record.TryGet(out TradeMsg trade))
    {
        Console.WriteLine($"{DbnTime.ToInstant(trade.IndexTs)} {trade.Price} x {trade.Size}");
        trades++;
    }
}

Console.WriteLine($"{trades} trade(s)");

A DBN fragment — a bare run of records with no magic prelude and no metadata block — has to say so, because there is nothing in the bytes to detect: 

using var fragment = new DbnDecoder(
    File.OpenRead("data.dbn.frag"), skipMetadata: true, inputDbnVersion: 3);

Remarks

The thin I/O layer over DbnFsm: it reads from a Stream into the state machine's own buffer and drives it. All the decoding lives in the state machine, which knows nothing about streams; this type knows nothing about DBN beyond "read more when asked".

Compression is detected, not declared. The first four bytes are compared against the Zstandard frame magic and then handed straight back to whichever reader is chosen, so nothing is consumed by the test itself — see DatabentoDotNet.Dbn.Internal.PrefixedStream. A stream that starts with the magic is read through the Zstandard seam; anything else is read as raw DBN.

Synchronous by design. RecordRef is a ref struct and cannot cross an await, which is what keeps records pointing at live buffer bytes rather than at a copy. The asynchronous client sits above this and drives DbnFsm directly.

Constructors

DbnDecoder(Stream, VersionUpgradePolicy, bool, byte?, bool, bool, int)

Opens a DBN stream, decoding its metadata block immediately unless skipMetadata says there is none.

public DbnDecoder(Stream source, VersionUpgradePolicy upgradePolicy = VersionUpgradePolicy.UpgradeToV3, bool skipMetadata = false, byte? inputDbnVersion = null, bool tsOut = false, bool leaveOpen = false, int bufferSize = 65536)

Parameters

source Stream

The stream to read, positioned at its first byte. Read forward only; never seeked.

upgradePolicy VersionUpgradePolicy

How to present records from an older DBN version. The default converts v1 and v2 records to v3 as they are decoded.

skipMetadata bool

true when source is a DBN fragment: a bare run of records with no magic prelude and no metadata block.

inputDbnVersion byte?

The fragment's DBN version, when known. Ignored unless skipMetadata is set, since a metadata block states the version itself.

tsOut bool

Whether every record carries an appended 8-byte ts_out. Ignored unless skipMetadata is set.

leaveOpen bool

true to leave source open when this decoder is disposed.

bufferSize int

The read buffer's size in bytes.

Exceptions

ArgumentNullException

source is null.

DbnDecodeException

The stream does not begin with valid DBN metadata, or ends part-way through it. A stream that ends between records is not an error — see TryNextRecord(out RecordRef).

Properties

IsCompressed

true when the source stream is Zstandard-compressed.

public bool IsCompressed { get; }

Property Value

bool

Metadata

The stream's metadata, or null for a fragment. Already presented according to the upgrade policy.

public Metadata? Metadata { get; }

Property Value

Metadata

Methods

Dispose()

Disposes the decompressor, if any, and the source stream unless it was left open.

public void Dispose()

TryNextRecord(out RecordRef)

Decodes the next record, reading from the source stream as needed.

public bool TryNextRecord(out RecordRef record)

Parameters

record RecordRef

Receives the decoded record. Valid only until the next call on this decoder — the bytes it points at live in the decoder's own buffer, which the next read may move.

Returns

bool

true when a record was decoded; false at the end of the stream.

Remarks

A stream ending is not an error, including one that ends part-way through a record: the trailing partial bytes are simply never yielded. That is what makes this the Try-shaped member and not a throwing one.

Exceptions

DbnDecodeException

The stream's bytes are not valid DBN.

Edit this page