# Diagnosing stalled pool payouts RC65.6

## The symptom

You're mining, the pool is finding blocks, and the logs even show your worker getting credited — but nothing is showing up in your wallet. Before you assume something's broken, there's one normal thing to rule out first, and then one real thing worth checking.

## Rule this out first: block maturity isn't instant

Every block credit needs to sit for a number of confirmations before the pool is even allowed to pay it out — that's the `POOL_PAYOUT_MATURITY` setting (4096 blocks on a stock config). You can see this on a per-block basis in the pool's own log. Every time it finds and credits a block, it prints a line like:

```
💰 CREDIT SUMMARY block=... height=14321811 ... matureHeight=14321821 payoutUnlockHeight=14325907 ...
```

`payoutUnlockHeight` is the chain height that has to be reached before that specific block's reward is even eligible to be paid. Compare it to your node's current height (the pool's live status stream shows this too, as `last_template_height`). If the chain hasn't reached `payoutUnlockHeight` yet, that credit simply isn't due — that's expected behavior, not a bug, and no amount of restarting anything will make it pay out early.

## The real thing to check: has a payout ever actually been sent?

Log grepping for words like "payout" or "error" is unreliable here for two reasons: the pool's Docker log rotates, so on a busy pool you may only have the last few hours retained (check with `docker inspect <pool container> --format '{{.State.StartedAt}}'` versus the oldest timestamp in `docker logs <pool container> | head -1` — if there's a big gap, you're not seeing the real startup log anymore), and a stalled payout loop in this pool software fails *silently*. It doesn't log an error each time it fails to run — it just doesn't run.

The reliable way to check is to go straight to the pool's own database, which is the actual source of truth regardless of what the logs show:

```bash
docker exec <postgres container> psql -U bdag_pool -d bdagpool -c \
  "SELECT count(*) AS total_credits, count(*) FILTER (WHERE is_paid) AS paid, count(*) FILTER (WHERE NOT is_paid) AS unpaid FROM credits;"

docker exec <postgres container> psql -U bdag_pool -d bdagpool -c \
  "SELECT count(*), min(created_at), max(created_at) FROM credits WHERE NOT is_paid;"
```

The second query is the important one. If the oldest unpaid credit is way older than the maturity window (4096 blocks is roughly an hour or so of chain time at typical block rates, so anything unpaid from many hours or a day+ ago is not a timing issue), the payout loop is genuinely stuck, not just running behind.

You can also confirm whether payouts have ever fired at all by checking the `payouts` table directly:

```bash
docker exec <postgres container> psql -U bdag_pool -d bdagpool -c \
  "SELECT tx_hash, amount, created_at FROM payouts ORDER BY created_at DESC LIMIT 5;"
```

If the most recent row here is from well before your pool container's current start time, that confirms the *current* running instance has never successfully paid anyone, even though the database shows a long history of real payouts from before.

## The fix

Simply restarting the pool container clears this — something in the payout goroutine gets stuck (we don't know the exact root cause, but it doesn't log anything when it happens) and a fresh process restarts it cleanly. A couple of things to know before you do this on a production pool:

- **Don't use a bare `docker restart`.** This pool stack gates container startup behind a watchdog-issued "lease." If you just restart the container directly, the entrypoint will come back up refusing to start with `refusing pool start: watchdog lease is stale, future-dated, or expired`, and your pool will stay down. Instead, trigger it through the watchdog itself, which issues a fresh lease and brings the pool (and its companion services) up properly:

  ```bash
  docker exec <watchdog container> python3 ops/watchdog.py --activate-pool-after-convergence --reason "restart to clear stalled payouts"
  ```

- **Expect a brief mining interruption.** Your ASIC(s) will disconnect and reconnect automatically within seconds — same as their normal reconnect cycle — and any already-earned or already-paid balance is untouched.

- **Check the underlying node too.** In our case, the watchdog's activation sequence also briefly stopped and restarted the actual blockchain node container as part of bringing everything back into a converged state. If your pool comes back up but logs something like `Failed to get chain tip: connect: connection refused`, check whether your node container is still running (`docker ps -a --filter name=<node container>`) and start it if it's stopped.

## Confirming it worked

Once the pool is back up and connected to a healthy node, watch the log — you should see it immediately start working through the backlog:

```
💸 Processing Payouts for Block ... (N credits)
[Payout Debug] Address=0x... Amount=... Wei, GasPrice=..., Limit=21000, ...
  Sent ... to 0x... (Tx: 0x...)
💰 Block ... marked as PAID
```

And you can watch the backlog actually shrink in real time:

```bash
docker exec <postgres container> psql -U bdag_pool -d bdagpool -c \
  "SELECT count(*) FILTER (WHERE NOT is_paid) FROM credits;"
```

Run that a couple of times a minute or two apart — the number should be dropping. Once it stabilizes near zero (minus whatever's still inside the maturity window), you're caught up, and your wallet should reflect it within minutes.