# Bug Report: Coinbase-Credited Accounts Become Permanently Unable to Send Native Transfers on BDAG Mainnet

## Summary (updated finding — read this first)

We have identified a specific, reproducible trigger for a stuck-transaction
condition on BDAG mainnet (chainId `1404`) that has affected pool payout
wallets across multiple independent nodes over several days of
investigation: **any account that has ever received a direct coinbase
(mining reward) credit becomes permanently unable to get outgoing native
transfers included in a block** — regardless of gas price (tested up to the
node's own hard fee cap), transaction type (legacy and EIP-1559), or the
timing of when the transaction is attempted relative to the account's
coinbase-recipient status.

Accounts that have never received a coinbase credit remain completely
healthy, including one observed with over 1.5 million confirmed
transactions and zero pending backlog, on the same node, at the same time
as multiple coinbase-credited accounts sat permanently stuck.

We do not have source code access, but binary analysis of the (unstripped)
`blockdag-node` binary surfaced specific internal mechanisms — a
governance-based transaction admissibility check wired directly into the
core transaction-application path, a separately-tracked nonce validation
path, and a coinbase-transaction watcher — any of which could plausibly
produce this exact behavior. Details below.

## Environment

- Network: BDAG mainnet, chainId `1404`
- Node software: community-rescue build `2.0.0-community-rescue-rc.62-linux-amd64`
  (also reproduced on `rc.44` earlier in this investigation)
- Multiple independently operated nodes tested over several days across
  cloud and home-network environments — all showing the same behavior

## The Core Finding: Coinbase Exposure, Not Backlog Depth or Timing

Earlier phases of this investigation (see "Prior Findings" below) had
narrowed the trigger to "something about deep pending-transaction queues,"
since that was the common factor in every case observed at the time. On
further testing, a sharper and more specific pattern emerged.

### Test 1: Fresh wallet, coinbase target while sending

A brand-new wallet (zero prior history) was made the `MINING_POOL_ADDRESS`
(coinbase recipient) on a live node. After receiving real coinbase reward
(~497 BDAG), a plain, correctly-signed, baseline-gas transaction at nonce 0
(a 1-wei self-send) was submitted **while the wallet was still the active
coinbase target**. Result: accepted into the mempool, correctly reflected
in `txpool_content` as `pending`, never confirmed.

### Test 2: Fresh wallet, coinbase target switched away before sending

A second brand-new wallet was made the coinbase target, allowed to receive
exactly one block's reward (~331 BDAG), and **mining reward was switched
away from it immediately afterward, before any transaction was ever
attempted**. Only after the switch was its first-ever transaction (nonce 0,
identical shape to Test 1) submitted. Result: identical outcome — accepted,
never confirmed.

This rules out "currently active coinbase target" as the specific
condition; **having ever received a coinbase credit is sufficient**, with
no dependency on continued or contemporaneous coinbase status.

### Test 3: Direct correlation check across all recently-active accounts

Every distinct address observed receiving a `bdagEngine.updateAutoStake`
event (a real, frequently-firing consensus-level operation affecting most
active accounts on the chain, confirmed unrelated to this bug — see "Ruled
Out" below) on a live node was checked for nonce gap
(`pending - latest` via `eth_getTransactionCount`):

| Address (truncated) | Confirmed txs | Gap |
|---|---|---|
| 0x0B4cb5d4... | 205,483 | 0 |
| 0x12d8377d... | 514,982 | 0 |
| 0x2e91Ce66... | 179,305 | 0 |
| 0x4BE5B1a9... | 499,014 | 0 |
| 0x6387C32c... | 1,569,797 | 0 |
| ...12 more addresses, all | — | **0** |
| **0x2fCaFcb6...** (current coinbase target) | 0 | **1 (stuck)** |
| **0xC9b4808a...** (former coinbase target, this and other nodes, over many days) | 5,567 | **8,315 (stuck)** |

Of 18 addresses checked, the only two showing any nonce gap at all were the
only two that had ever been a `MINING_POOL_ADDRESS` coinbase target. Every
other address — including several with hundreds of thousands to over a
million confirmed transactions — showed a perfectly healthy `gap=0`.

## Prior Findings (from earlier in this investigation, still valid context)

These were established before the coinbase-specific pattern was isolated,
and remain accurate as ruled-out variables:

- **Gas price is not the blocking factor.** A stuck transaction was
  re-submitted at escalating gas price from baseline up to
  **~400,000× baseline** (legacy `gasPrice`) on one node, and separately up
  to the **node's own hard 1 BDAG transaction-fee cap** (~45,000 Gwei
  `maxFeePerGas`, EIP-1559) on another. All escalated attempts were
  correctly accepted into the mempool as valid, non-underpriced
  transactions. None confirmed.
- **Mempool mechanics are correct.** Direct inspection via `txpool_content`
  (after discovering this RPC method exists in the binary but is not
  enabled by default — see "Recommendation" below) confirmed stuck
  transactions are classified as `pending` (immediately executable per the
  node's own logic), not `queued`. Replacement transactions correctly
  evict the prior entry at the same nonce. A stuck transaction was observed
  stable in `pending` — no confirmation, no eviction — across a clean
  10-minute continuous observation window.
- **Transaction type does not matter.** Both legacy (`type: 0`) and
  EIP-1559 (`type: 2`) transactions exhibit identical behavior.
- **A specific candidate root cause was tested and ruled out.**
  `BDAG_NODE_MINING_NO_PENDING_TX=1` (`--miningnopendingtx`) was
  hypothesized (based on a third-party report) to cause reward-only block
  templates that exclude all mempool transactions. This was directly
  disproven: the flag was confirmed active in the live process arguments
  on a node that, in the same session, successfully confirmed ten
  consecutive real payout transactions.

## Binary-Level Evidence

The `blockdag-node` binary is not stripped of Go symbol information. The
following were extracted via `go tool nm` and `strings` and appear
directly relevant:

### A governance-based native-transfer authorization check, unconditionally wired into transaction application

```
github.com/BlockdagNetworkLabs/bdag/bevm/bdag.(*GovernanceAdmissibility).CheckNativeTransfer
github.com/BlockdagNetworkLabs/bdag/bevm/bdag.(*GovernanceAdmissibility).NativeTransferAuthorizer
github.com/BlockdagNetworkLabs/bdag/bevm/bdag.TemplateGovernanceAdmissibility
github.com/BlockdagNetworkLabs/bdag/bevm/bdag.(*BdagPool).installGovernanceAdmissibilityHook
github.com/BlockdagNetworkLabs/bdag/bevm/bdag.(*BdagPool).applyTransaction.GovernanceAdmissibility.applyToVMConfig.GovernanceAdmissibility.NativeTransferAuthorizer.func1
github.com/ethereum/go-ethereum/core/txpool.ErrGovernanceRejected
github.com/ethereum/go-ethereum/core/txpool/legacypool.(*LegacyPool).pruneGovernanceRejectedLocked
github.com/ethereum/go-ethereum/core/txpool/blobpool.(*BlobPool).pruneGovernanceRejectedLocked
```

The `NativeTransferAuthorizer` hook is embedded directly in the closure
chain of `BdagPool.applyTransaction` — the core transaction-application
function — not behind an optional or separately-invoked code path.
Associated literal strings found in the binary:

```
"transaction rejected by governance policy"
"Skipping governance-inadmissible transaction in template"
"governance_rejected"
```

The second string in particular describes exactly the observed behavior:
a transaction being deliberately skipped from block templates for
governance reasons, independent of mempool validity or gas price.

### The governance subsystem's sync mechanism appears disabled

Every node tested logs the following at startup, at all verbosity levels
including `trace`:

```
[INFO ] governance upgrade watcher disabled
```

We were unable to capture the `"Skipping governance-inadmissible..."` log
line firing live, even at `trace` verbosity over a 90-second window with
~32,000 log lines and multiple real blocks mined. One plausible
reconciliation: the admissibility *check* is unconditionally compiled into
the transaction-application path (confirmed by the symbol table), but the
*data* it evaluates against depends on a watcher that keeps governance
parameters synced from on-chain state — and that watcher is disabled. If
so, native-transfer authorization could be evaluating against stale,
default, or empty parameters rather than genuinely absent input, which
would not necessarily produce a per-attempt log line but could still
produce a consistent fail-closed result for any account matching whatever
condition the (unsynced) rule keys on — plausibly something related to
coinbase-recency, given the reproducible trigger we've isolated.

### A separate, custom nonce-validation path

```
github.com/BlockdagNetworkLabs/bdag/bevm/bdag.(*BdagPool).ValidateTxDirect
```
with the associated literal string:
```
"Nonce too low in validateTxDirect in bdagpool.go"
```

This confirms `BdagPool` performs its own nonce validation, in a function
named distinctly from (and presumably in addition to) standard txpool
nonce checks. We do not have visibility into whether this maintains
separate account-nonce state that could desync from canonical EVM state
under coinbase-crediting specifically, but the existence of a second,
custom nonce-check path is itself notable given the specificity of the
observed trigger condition.

### A coinbase-specific transaction watcher

```
"coinbase tx long time not confirm ,will remove for watch"
```

We located this string but were not able to trace it to a specific named
function via symbol search in the time available. Given its direct,
literal reference to coinbase transactions specifically and to a
confirmation-time-based watch/removal mechanism, this seems likely to be
directly relevant and worth investigating first by anyone with source
access.

## Live Governance Contract: Located and Partially Verified On-Chain

The `governanceProxyAddr` / `governanceImplAddr` symbols identified above
are stored in the binary's `.noptrbss` section (confirmed via `readelf -S`),
meaning they are zero-initialized at load and populated at runtime rather
than being fixed compile-time constants — they could not be read directly
from the binary file on disk.

Since the binary is compiled as a non-PIE `EXEC` (confirmed via
`readelf -h`), the addresses reported in the symbol table are the actual
runtime virtual addresses with no ASLR offset to account for. This made it
possible to read the live values directly from a running node's process
memory (via `gdb -p <pid> -batch -ex "x/20xb <addr>" -ex detach -ex quit`,
attach-read-detach, read-only, no process modification):

```
governanceProxyAddr = 0x1000000000000000000000000000000000000005
governanceImplAddr  = 0x1000000000000000000000000000000000000015
```

Both are clean, low-numbered addresses, consistent with a deliberately
reserved system-contract address range rather than a normally-deployed
contract address.

**Both addresses have real, substantial deployed bytecode** (confirmed via
`eth_getCode` against a live node), ruling out the possibility that these
are simply unused/placeholder addresses. The proxy contract's bytecode
contains the literal 32-byte topic
`0xbc7cd75a20ee27fd9adebab32041f755214dbc6bffa90cc0225b39da...`, which
matches the standard EIP-1967 `Upgraded(address)` event signature,
suggesting this proxy's lineage derives from (or was inspired by) a
standard upgradeable-proxy pattern.

However, direct on-chain testing shows this is **not** a fully
standard-layout proxy:

- **EIP-1967 implementation slot**
  (`0x360894a13ba1a3210667c828492db98dca3e2076cc3735a920a3ca505d382bb`)
  read via `eth_getStorageAt` against the proxy address returns empty
  (all zeros) — the implementation address is stored somewhere other than
  the standard EIP-1967 slot.
- **EIP-1967 admin slot**
  (`0xb53127684a568b3173ae13b9f8a6016e243e63b6e8ee1178d6a717850b5d6103`)
  **does** return a real, populated address:
  ```
  0x3655a92f2db839a9ef30216c8843b310c65ce298
  ```
  This is a genuine, concrete admin address with real control over this
  proxy, and may be a useful starting point for anyone with source access
  investigating what this contract is configured to allow or restrict.
- A direct `eth_call` to the proxy for the standard OpenZeppelin
  `AccessControl.DEFAULT_ADMIN_ROLE()` selector (`0xa217fddf`) reverts,
  confirming the implementation is custom rather than a stock
  `AccessControl`-based contract. We do not have the actual Solidity
  interface/ABI, and were unable to locate embedded function-signature
  strings in the binary to construct further calls with confidence — this
  is the practical limit of what black-box on-chain inspection could
  establish.

**Summary of what this narrows down for anyone with source access:** there
is a real, live, custom-implemented upgradeable governance contract at
`0x1000...0005` (proxy) / `0x1000...0015` (implementation), with a known
admin address (`0x3655a92f...ce298`), that the node's `applyTransaction`
path consults via `NativeTransferAuthorizer` on every transaction. Reading
this contract's actual state (with source/ABI access) for whatever rule
governs native transfers — and specifically whether that rule keys on
coinbase-recipient history — seems like the most direct path to confirming
or ruling out our leading hypothesis.

## Ruled Out

- **`bdagEngine.updateAutoStake`**, a real, frequently-firing (roughly
  once every 5-6 seconds per active account) consensus-level auto-staking
  mechanism, was investigated as a candidate cause given its consensus-layer
  nature and the fact that it touched the same accounts. Directly
  disproven: it fires for the large majority of active accounts on the
  chain, the great majority of which remain perfectly healthy senders.
- **Wallet age/history** (ruled out by Test 2 above, and by the presence of
  perfectly healthy multi-million-transaction accounts alongside stuck
  brand-new ones).
- **Gas price, at any tested magnitude** (see "Prior Findings").
- **Transaction type** (legacy vs. EIP-1559).
- **`BDAG_NODE_MINING_NO_PENDING_TX`** (directly tested and disproven).

## Requested Information

1. What does the governance contract at `0x1000...0005` (proxy, admin
   `0x3655a92f...ce298`) / `0x1000...0015` (implementation) actually
   enforce regarding native transfers, and does that rule key on an
   account's coinbase-recipient history (recency, cumulative amount, or
   similar)? The `NativeTransferAuthorizer` / `CheckNativeTransfer` code
   path in `applyTransaction` seems the most direct candidate given the
   evidence above.
2. Is the "governance upgrade watcher" intended to be enabled in a
   community-rescue / pool-operator deployment? If it is disabled by
   design in this release, what governs the default/fallback state that
   `NativeTransferAuthorizer` evaluates against when unsynced — and could
   that default be fail-closed specifically for coinbase-touched accounts?
3. What does `BdagPool.ValidateTxDirect`'s nonce check evaluate against,
   and could it desync from canonical EVM nonce state specifically as a
   result of coinbase crediting (which, per our testing, changes an
   account's balance without going through the same transaction-processing
   path as a normal transfer)?
4. What triggers `"coinbase tx long time not confirm ,will remove for
   watch"`, and could whatever it does to a coinbase transaction under
   that condition leave persistent state on the account that interferes
   with subsequent unrelated transactions from the same address?
5. Is there a supported way to enable `txpool` in `evm.http.api` by
   default? It is implemented in the binary but not enabled in the shipped
   `node.conf`, and was essential to this investigation. We'd suggest
   enabling it by default in future releases for any node intended for
   pool operation.

## Impact

This affects the core, intended function of any pool node on this chain:
the account that receives mining rewards (`MINING_POOL_ADDRESS`) is, by
definition, coinbase-credited, and per this finding becomes permanently
unable to send outgoing transactions — meaning it cannot pay miners
directly, cannot fund payout infrastructure, and cannot be topped up and
reused. This not a misconfiguration or an edge case; it appears to be a
structural conflict between how mining rewards are credited and how
outgoing transfers are authorized, affecting the base use case of "operate
a pool that both mines and pays out."

We are aware of other independent pool operators on this chain reporting
similar symptoms, described elsewhere as "stuck nonces," which this
investigation suggests may share this same root cause.

## What Has NOT Worked as a Fix or Workaround

For the benefit of anyone else investigating this: gas-price escalation
(any magnitude, any transaction type), switching a wallet's coinbase status
before attempting to send, and waiting (up to several hours observed on
some affected wallets) have not resolved a stuck condition once triggered.
The only mitigation identified so far is prevention — routing payouts
through wallets that have never been, and will never be, a coinbase
recipient, funded instead via internal transfers from a dedicated
coinbase-collection wallet that is never itself used to send to end
recipients. Even this workaround has limits: the collection wallet itself,
once coinbase-credited, becomes unable to make even those internal transfer
transactions, which is itself a significant open problem for any pool
architecture on this chain.