> For the complete documentation index, see [llms.txt](https://hub.bsvblockchain.org/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://hub.bsvblockchain.org/brc/transactions/0139.md).

# Multicast Shard Manifest Announcement Protocol

Jeff Harris (<jeff@lightweb.net>)

## Abstract

This BRC specifies the shard manifest announcement protocol for the BSV multicast transaction distribution pipeline. It defines a single UDP datagram format — ShardManifest (`MsgType 0x40`) — by which any participant periodically declares its `shard_bits` configuration and the set of shard groups it has joined. Manifests are emitted directly to the control-plane beacon group (`0xFFFD`, see [BRC-129](/brc/transactions/0129.md)), the same group used by [BRC-126](/brc/transactions/0126.md) ADVERT beacons, and are distinguished by their `MsgType` byte. The protocol provides operator visibility into network-wide sharding configuration, enables cross-peer divergence detection, and defines a normative consumer profile for automated, rate-limited shard-bit coordination. It is observation-only by default: there is no retransmission and no acknowledgment.

## Copyright

This BRC is licensed under the Open BSV License.

## Motivation

Every component of the multicast pipeline — ingress proxy, listener, retry endpoint, and transaction producer — must agree on a single `shard_bits` value to interoperate, because the shard group index for a transaction is derived as `groupIndex = binary.BigEndian.Uint32(txid[0:4]) >> (32 − shardBits)` ([BRC-129](/brc/transactions/0129.md)). Today this value is configured manually on each component and there is no on-network signal to verify agreement. A silent mismatch causes senders and receivers to use disjoint group spaces, dropping traffic without an obvious cause.

BRC-139 introduces a small, dedicated announcement that lets every participant:

1. Periodically declare its current `shard_bits` and the set of shard group indices it claims to have joined.
2. Let observers detect inconsistent configuration across peers.
3. Carry identity, timestamp, TTL, and a `GenerationID` so a normative consumer profile can implement automated, rate-limited `shard_bits` shifts safely.

The protocol reuses the existing beacon group rather than allocating a new control-plane index, and requires no proxy involvement: manifests do not transit the BRC-124 ingress path.

## Specification

### Common Message Preamble

ShardManifest datagrams share the [BRC-126](/brc/transactions/0126.md) control-message preamble: the `MsgType` byte at offset 6 occupies the same position as `FrameVersion` in BRC-124 data frames. Value `0x40` identifies a ShardManifest and is distinct from both the data-frame version codes (`0x01`–`0x07`) and the BRC-126 control messages (`0x10`–`0x2F`). All multi-byte integers are big-endian.

### ShardManifest Wire Format (`MsgType 0x40`) — 64-byte header + variable payload

| Offset | Size | Field            | Description                                                                    |
| ------ | ---- | ---------------- | ------------------------------------------------------------------------------ |
| 0      | 4    | Magic            | `0xE3E1F3E8` (BSV mainnet P2P magic)                                           |
| 4      | 2    | ProtoVer         | `0x02BF`                                                                       |
| 6      | 1    | MsgType          | `0x40` (ShardManifest)                                                         |
| 7      | 1    | Flags            | See [Flags](#flags)                                                            |
| 8      | 16   | SrcIPv6          | Announcer's primary IPv6 (informational; datagram source is authoritative)     |
| 24     | 4    | InstanceID       | CRC32c of hostname; stable across restarts                                     |
| 28     | 4    | Epoch            | Unix seconds when the announcement was generated                               |
| 32     | 2    | TTL              | Validity in seconds; `0` = use consumer default                                |
| 34     | 2    | AnnounceInterval | Seconds between sends; consumers compute TTL hint = `3 × this`                 |
| 36     | 1    | ShardBits        | `0`–`12` (MUST be ≤ 12 per [BRC-129](/brc/transactions/0129.md))               |
| 37     | 1    | RoleHint         | Informational role (see [RoleHint](#rolehint))                                 |
| 38     | 2    | GroupCount       | `N`: number of `groupIndex` entries in list form (`0` if bitmap or no claim)   |
| 40     | 2    | BitmapBytes      | `M`: length of trailing bitmap; `0` ⇒ list form                                |
| 42     | 2    | SourceCount      | `K`: number of trailing 16-byte source-IPv6 entries; `0` unless `SourcesValid` |
| 44     | 4    | ManifestCRC      | CRC32c over the whole datagram with these 4 bytes treated as zero              |
| 48     | 16   | GenerationID     | Operator-supplied 128-bit value; bumped whenever `ShardBits` changes           |
| 64     | \*   | Payload          | Groups, then Sources, then Successor block (see below)                         |

The payload sections appear in this order after the 64-byte header:

1. **Groups** — when `GroupsValid=1`: either `GroupCount × 2` bytes of big-endian `groupIndex` (sorted ascending, no duplicates), or exactly `BitmapBytes` bytes of bitmap (LSB-first, bit `i` = group index `i`). Empty when `GroupsValid=0`.
2. **Sources** — when `SourcesValid=1`: `SourceCount × 16` bytes of source IPv6 addresses in network byte order.
3. **Successor** — when `SuccessorValid=1`: a 24-byte Successor block (see [Successor block](#successor-block-when-successorvalid1)).

Total datagram size = `64 + max(N×2, M) + K×16 + (24 if SuccessorValid else 0)`. Implementations SHOULD keep the total size ≤ 1232 B to avoid IPv6 fragmentation on typical paths. With `ShardBits=12` (4,096 groups) the bitmap form is exactly 512 B; the list form is 2 B per joined group. Operators with large source lists SHOULD spread them across multiple announcers (each advertising its own source as the lone entry) so individual datagrams stay within the recommended MTU.

### Encoding-form rules

| `GroupsValid` | `BitmapBytes` | `GroupCount` | Groups payload form                                                       |
| ------------- | ------------- | ------------ | ------------------------------------------------------------------------- |
| `0`           | `0`           | `0`          | identity-only (no group claim)                                            |
| `1`           | `> 0`         | `0`          | bitmap, exactly `BitmapBytes` bytes; bit positions `0..(BitmapBytes×8)-1` |
| `1`           | `0`           | `> 0`        | list, exactly `GroupCount × 2` bytes, sorted ascending, no duplicates     |

A datagram with `GroupsValid=1` and both `BitmapBytes=0` and `GroupCount=0` is malformed and MUST be rejected. A datagram with `GroupsValid=1` and both `BitmapBytes>0` and `GroupCount>0` is also malformed. For bitmap form, bits at positions ≥ `2^ShardBits` MUST be zero and MUST be ignored by consumers.

### Sources payload (when `SourcesValid=1`)

Each 16-byte entry is a publisher source IPv6 in network byte order, contributed by this announcer. Consumers MUST reject the datagram when `SourcesValid=1 && SourceCount=0` or when `SourcesValid=0 && SourceCount>0`, treat entries as set-valued (order not significant), and deduplicate across the union of all currently-valid manifests. When `SourceModeSSM=1`, consumers feed the union into the Source-Specific Multicast `(S,G)` join calls for data-plane groups derived from the announced parameters. Each sender SHOULD announce only its own data-plane source (typically a single entry) rather than the operator-curated full fleet list, so per-datagram size stays small and the source set tracks publisher lifecycle.

### Successor block (when `SuccessorValid=1`)

The Successor block signals an in-flight generation transition: the announcer commits to a future `ShardBits` (and optionally `SourceModeSSM`) value that becomes the sole active generation at `TransitionEpoch`. It enables live re-sharding consumers to enter a bridging window before cutover. When present, a 24-byte block is appended immediately after the Sources payload:

| Offset (in block) | Size | Field                 | Description                                                     |
| ----------------- | ---- | --------------------- | --------------------------------------------------------------- |
| 0                 | 16   | SuccessorGenerationID | The incoming generation's 128-bit ID                            |
| 16                | 1    | SuccessorShardBits    | `1`–`12`; MUST satisfy `\|Successor − ShardBits\| ≤ 1`          |
| 17                | 1    | SuccessorFlags        | Bit 0 = `SuccessorSourceModeSSM`; bits 1–7 reserved (`0`)       |
| 18                | 2    | Reserved              | MUST be `0`                                                     |
| 20                | 4    | TransitionEpoch       | Unix seconds at which the successor becomes the sole generation |

Consumer rules (normative when auto-configuration is enabled):

* Reject the datagram when `|SuccessorShardBits − ShardBits| > 1`.
* Reject the datagram when `SuccessorValid=1 && Authoritative=0` (live re-sharding signals require operator authority).
* Apply the existing adoption gates (quorum, hysteresis) to the Successor block as a unit; the tuple `(SuccessorGenerationID, SuccessorShardBits, SuccessorFlags, TransitionEpoch)` is the candidate value.
* Consumers that implement live re-sharding MAY enter a bridging window between the moment the Successor block first satisfies quorum and `local_clock ≥ TransitionEpoch`. Consumers that do not implement live re-sharding MUST treat Successor-block adoption as a divergence event and wait for the pilot to roll `GenerationID` (promoting the successor to the active generation) before reacting on the new value.

Pilots MUST choose `TransitionEpoch ≥ now + 2 × AnnounceInterval` (RECOMMENDED `≥ now + 4 × AnnounceInterval`) and MUST reject configurations below that floor. Clock skew between pilot and consumers MUST stay below `AnnounceInterval / 2`; operators SHOULD run NTP.

### Flags

| Bit | Name           | Meaning                                                                                                |
| --- | -------------- | ------------------------------------------------------------------------------------------------------ |
| 0   | GroupsValid    | The trailing payload carries a valid joined-groups encoding.                                           |
| 1   | Authoritative  | Operator-curated authoritative announcer; counts toward adoption quorum.                               |
| 2   | Shutdown       | Final announcement before graceful shutdown; consumers MAY evict immediately.                          |
| 3   | SourceModeSSM  | Data plane uses Source-Specific Multicast (`FF3x::/32`, RFC 4607); consumers MUST use the SSM prefix.  |
| 4   | SourcesValid   | The trailing payload includes `SourceCount × 16` bytes of publisher source IPv6 addresses. `0` ⇒ none. |
| 5   | PilotOnly      | Groups describe desired fleet state, not the announcer's own joins; implies `Authoritative=1`.         |
| 6   | SuccessorValid | The trailing payload includes a 24-byte Successor block; requires `Authoritative=1`.                   |

Bit 7 is reserved and MUST be `0`. Consumers MUST reject `PilotOnly=1 && Authoritative=0` and `SuccessorValid=1 && Authoritative=0` as malformed.

### RoleHint

`RoleHint` is informational; consumers SHOULD NOT make filtering decisions on it.

| Value | Role           |
| ----- | -------------- |
| `0`   | generic        |
| `1`   | proxy          |
| `2`   | listener       |
| `3`   | retry-endpoint |
| `4`   | producer       |
| `5`   | manifest-only  |

Values ≥ `6` are reserved.

### ManifestCRC

`ManifestCRC` is a CRC32c (Castagnoli polynomial) computed over the entire datagram with the four CRC bytes themselves treated as zero. Consumers MUST verify the CRC before acting on the manifest.

### Multicast Group and Scope

Manifests are sent **directly** to the beacon group defined in [BRC-129](/brc/transactions/0129.md) (index `0xFFFD`), the same group used by BRC-126 ADVERT:

| Index    | Scope  | Compressed Address |
| -------- | ------ | ------------------ |
| `0xFFFD` | `FF05` | `FF05::B:FFFD`     |
| `0xFFFD` | `FF08` | `FF08::B:FFFD`     |
| `0xFFFD` | `FF0E` | `FF0E::B:FFFD`     |

The announcer chooses one or more scopes; when multiple are configured the same datagram is sent to each. The proxy is **not** involved. Because BRC-126 ADVERT (`MsgType 0x20`) shares this group, listeners MUST dispatch on the `MsgType` byte (offset 6) before parsing: `0x40` ⇒ ShardManifest; `0x20` ⇒ BRC-126 ADVERT.

### Cadence and Freshness

| Parameter        | Default | Notes                                        |
| ---------------- | ------- | -------------------------------------------- |
| AnnounceInterval | `300 s` | Every 5 minutes                              |
| Default TTL      | `900 s` | `3 × AnnounceInterval` (used when `TTL = 0`) |
| Send jitter      | ±10 %   | RECOMMENDED, to avoid global synchronisation |

Consumers MUST treat entries older than `Epoch + TTL` (or `Epoch + 3 × AnnounceInterval` when `TTL=0`) as expired. There is no retransmission and no NACK semantics; loss is tolerated by re-announcement, and an empty registry is valid.

### Identity and State

Consumers SHOULD key registry entries on `(SrcIPv6, InstanceID)`. The authoritative `SrcIPv6` is taken from the IPv6 datagram header; the in-frame `SrcIPv6` field is informational only. `InstanceID` is the CRC32c of the announcer's hostname, matching BRC-126 ADVERT semantics, and is stable across restarts. `GenerationID` is an operator-supplied 128-bit value (typically a UUID) that operators MUST bump whenever `ShardBits` changes; it is opaque to the protocol and consumers compare it for change detection.

### Consumer Behaviour — Observation (informative)

A consumer MAY join the beacon group(s) of interest and:

1. Dispatch incoming datagrams on the `MsgType` byte. `0x40` ⇒ ShardManifest.
2. Verify `ManifestCRC`; reject on mismatch.
3. Upsert into a registry keyed on `(SrcIPv6, InstanceID)`.
4. Evict on `Epoch + TTL` (or `Epoch + 3 × AnnounceInterval` when `TTL=0`).
5. Surface metrics: per-peer `ShardBits`, joined-group count, last-seen, and the count of distinct `ShardBits` values currently observed (divergence).

### Consumer Behaviour — Auto-configuration (normative when opted in)

A consumer that opts in to automatic configuration MUST implement the observation requirements above and additionally MUST satisfy the following. Components that do not opt in are unaffected.

1. **Authoritative-only adoption.** Manifests with `Authoritative=0` MUST NOT contribute to any adopted value (they MAY still be indexed for visibility and contribute to the source-set union below).
2. **Quorum.** A candidate value is eligible for adoption only when reported by at least `pilot-quorum` distinct authoritative announcers (keyed on `(SrcIPv6, InstanceID)`) within their TTL window. `pilot-quorum` MUST be configurable; default `2`.
3. **Hysteresis.** A candidate that satisfies quorum MUST hold quorum continuously for `≥ 2 × AnnounceInterval` before adoption. A change in adopted value resets the timer.
4. **`ShardBits` shift bound.** A consumer MUST NOT adopt a `ShardBits` value differing from the currently adopted value by more than ±1 within any rolling `AnnounceInterval` window.
5. **Manual pin precedence.** An operator-pinned value is the local authority and MUST NOT be overridden by adoption; the consumer MUST still evaluate quorum and emit divergence telemetry when the adopted candidate differs from the pin.

Fields subject to adoption: `ShardBits`, `SourceModeSSM`, and the deduplicated union of `SourcesValid` payloads (the source set is **not** gated by quorum — it is the union of every currently-valid manifest's sources, irrespective of `Authoritative`). `MCGroupID` is not carried in the payload; consumers MUST derive it from the destination address of the beacon socket on which the manifest was received. Consumers SHOULD rate-limit source-set additions and removals before feeding them into kernel join calls to avoid thrashing the multicast forwarding information base, and MUST NOT label any metric with raw source IPv6 addresses.

### Safety Guidance

1. Bump `GenerationID` whenever `ShardBits` changes.
2. Keep authoritative announcers to a small operator-curated set (RECOMMENDED: three instances across failure domains).
3. Deploy authoritative announcers with `PilotOnly=1` so consumers can distinguish operator intent from an announcer's own joins.
4. Treat non-authoritative manifests as observational (and, when `SourcesValid=1`, as per-publisher source contributions).
5. Warn — do not auto-shift — on observed `ShardBits` divergence; the adoption gates already prevent unsafe shifts, but operator visibility is where misconfiguration is caught.

## Interactions With Other BRCs

* [**BRC-126**](/brc/transactions/0126.md) (Retransmission / ADVERT) — shares the beacon group `0xFFFD` and listen port. Distinguished by the `MsgType` byte (`0x20` ADVERT vs `0x40` ShardManifest). BRC-139 is neither retransmitted nor a retransmitter.
* [**BRC-127**](/brc/transactions/0127.md) (Subtree group announcements) — orthogonal: BRC-127 announces SubtreeID→GroupID bindings on `0xFFFC` via the proxy; BRC-139 announces participant configuration directly on `0xFFFD`.
* [**BRC-129**](/brc/transactions/0129.md) (Multicast addressing) — no new index allocated; manifests reuse the existing beacon group, and `ShardBits ≤ 12` per BRC-129.

## References

* [BRC-82: Defining a Scalable IPv6 Multicast Protocol for Blockchain Transaction Broadcast and Update Delivery](/brc/peer-to-peer/0082.md) — Overall multicast protocol architecture
* [BRC-124: Multicast Transaction Frame Format](/brc/transactions/0124.md) — Data-plane frame format and `HashKey`/`SeqNum` derivation
* [BRC-126: Multicast Transaction NACK Retransmission Protocol](/brc/transactions/0126.md) — ADVERT beacon sharing the same beacon group
* [BRC-129: IPv6 Multicast Group Address Assignments](/brc/transactions/0129.md) — Beacon group and `shard_bits` bound

## Constants Reference

| Name                      | Value      | Hex          | Description                                      |
| ------------------------- | ---------- | ------------ | ------------------------------------------------ |
| `MagicBSV`                | 3823236072 | `0xE3E1F3E8` | BSV mainnet P2P magic                            |
| `ProtoVer`                | 703        | `0x02BF`     | Protocol version                                 |
| `MsgTypeShardManifest`    | 64         | `0x40`       | ShardManifest datagram type                      |
| `ShardManifestHeaderSize` | 64         | `0x40`       | Fixed header size in bytes                       |
| `GroupBeacon`             | 65533      | `0xFFFD`     | Control-plane beacon group index                 |
| `FlagGroupsValid`         | 1          | `0x01`       | Flags bit 0: groups payload present              |
| `FlagAuthoritative`       | 2          | `0x02`       | Flags bit 1: operator-curated announcer          |
| `FlagShutdown`            | 4          | `0x04`       | Flags bit 2: final pre-shutdown announcement     |
| `FlagSourceModeSSM`       | 8          | `0x08`       | Flags bit 3: data plane uses SSM addressing      |
| `FlagSourcesValid`        | 16         | `0x10`       | Flags bit 4: sources payload present             |
| `FlagPilotOnly`           | 32         | `0x20`       | Flags bit 5: groups describe desired fleet state |
| `FlagSuccessorValid`      | 64         | `0x40`       | Flags bit 6: Successor block present             |
| `DefaultAnnounceInterval` | 300        | —            | Default re-announce period (s)                   |
| `DefaultTTLMultiplier`    | 3          | —            | TTL = 3 × AnnounceInterval when `TTL=0`          |


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://hub.bsvblockchain.org/brc/transactions/0139.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.