Table of Contents

Class Metadata

Namespace
DatabentoDotNet.Dbn
Assembly
DatabentoDotNet.Dbn.dll

The header that opens every DBN file and every live DBN stream: what the data is, what range it covers, and how its symbols resolve.

public sealed class Metadata
Inheritance
Metadata
Inherited Members

Examples

using var decoder = new DbnDecoder(File.OpenRead("data.dbn.zst"));

// Null only for a fragment opened with skipMetadata — a stream that has no metadata block.
Metadata metadata = decoder.Metadata!;

Console.WriteLine($"DBN v{metadata.Version} {metadata.Dataset}");
Console.WriteLine($"schema {(metadata.Schema is { } schema ? schema.ToWireString() : "mixed")}");
Console.WriteLine($"from {DbnTime.ToInstant(metadata.Start)}");

// End and Limit are both absent as null here, though the wire spells their two sentinels
// differently — UndefTimestamp for one, NullLimit for the other.
if (metadata.End is { } end)
{
    Console.WriteLine($"to {DbnTime.ToInstant(end)}");
}

// The symbol mappings are what a symbol map is built from.
TsSymbolMap symbols = TsSymbolMap.FromMetadata(metadata);

Remarks

A class, not a struct, because it owns heap data — four lists whose contents are strings — so there is no value-type win to be had, and copying one by value would copy references anyway. This is also the one place in the codec where allocation is fine: metadata is decoded once per stream, ahead of the first record, and never on the per-record path the zero-copy rule is about.

Construction is by object initializer with required properties, not by a builder. Upstream needs a generic type-state builder to make "you must set dataset, schema, start, stype_in and stype_out" a compile-time error; C# has required for exactly that, and it costs no extra type.

Two 64-bit "unset" sentinels that are not the same. End is absent when the wire holds UndefTimestamp; Limit is absent when the wire holds NullLimit, which is zero. Both surface here as null, so callers never have to remember which is which.

Properties

Dataset

The dataset code, for example GLBX.MDP3.

public required string Dataset { get; init; }

Property Value

string

End

UNIX nanoseconds: the query end, or the last record's timestamp when the file was split. null for an open-ended query.

public ulong? End { get; init; }

Property Value

ulong?

Remarks

A raw zero on the wire also decodes to null, matching upstream, which treats both zero and UndefTimestamp as "no end". This is the one metadata field whose round-trip is not byte-identical, because re-encoding null always writes UndefTimestamp; upstream has the same behaviour, no stream in the conformance corpus carries a zero here, and both spellings mean the same thing to a reader.

Limit

The maximum number of records the query asked for, or null when it was unlimited.

public ulong? Limit { get; init; }

Property Value

ulong?

Remarks

Zero and null are the same thing on the wire — 0 is the "no limit" sentinel — so a zero set here encodes as unlimited and decodes back as null.

Mappings

Each requested symbol paired with the dated intervals it resolved over. Never null.

public IReadOnlyList<SymbolMapping> Mappings { get; init; }

Property Value

IReadOnlyList<SymbolMapping>

NotFound

Symbols that failed to resolve on every day of the query range. Never null.

public IReadOnlyList<string> NotFound { get; init; }

Property Value

IReadOnlyList<string>

Partial

Symbols that failed to resolve on at least one day of the query range. Never null.

public IReadOnlyList<string> Partial { get; init; }

Property Value

IReadOnlyList<string>

Schema

The record schema every record in the stream conforms to, or null when the stream may mix record types — which is the normal case for live data.

public Schema? Schema { get; init; }

Property Value

Schema?

Start

UNIX nanoseconds: the query start, or the first record's timestamp when the file was split.

public required ulong Start { get; init; }

Property Value

ulong

Remarks

Raw nanoseconds, matching the wire and the record structs. Convert with ToInstant(ulong), which handles UndefTimestamp; NodaTime's Instant represents a DBN timestamp exactly, where a BCL tick of 100 ns cannot.

StypeIn

The input symbology the query's symbols were expressed in, or null when the stream mixes several — again the normal case for live data.

public SType? StypeIn { get; init; }

Property Value

SType?

StypeOut

The output symbology symbols resolve to. Never absent.

public required SType StypeOut { get; init; }

Property Value

SType

SymbolCstrLength

The width in bytes of every fixed-length symbol field in this stream, NUL terminator included.

public required int SymbolCstrLength { get; init; }

Property Value

int

Remarks

Read from the wire in DBN v2 and later. DBN v1 has no such field, so a v1 stream is SymbolCstrLengthV1 by definition — see SymbolCstrLengthForVersion(byte). It is kept as decoded rather than re-derived from Version so that re-encoding reproduces the original bytes exactly.

Symbols

The symbols the original query asked for. Never null.

public IReadOnlyList<string> Symbols { get; init; }

Property Value

IReadOnlyList<string>

TsOut

true when every record in the stream carries an appended gateway send timestamp.

public bool TsOut { get; init; }

Property Value

bool

Version

The DBN version this metadata describes: 1, 2, or 3.

public required byte Version { get; init; }

Property Value

byte

Remarks

After decoding, this is the version the data is presented as, which is the input version only under AsIs. Under the default UpgradeToV3 a v1 or v2 stream reports 3 here.

Methods

SymbolCstrLengthForVersion(byte)

The fixed symbol-field width a DBN stream of the given version uses when the wire does not state one.

public static int SymbolCstrLengthForVersion(byte version)

Parameters

version byte

A DBN version.

Returns

int

SymbolCstrLengthV1 below version 2, otherwise SymbolCstrLength — v2 and v3 share the same width.

Edit this page