> ## Documentation Index
> Fetch the complete documentation index at: https://docs.istravex.com/llms.txt
> Use this file to discover all available pages before exploring further.

# stellar-etl pilot

> SDF's data-warehouse pipeline over the corpus, through its own transforms, and the two open issues it reproduces.

[stellar/stellar-etl](https://github.com/stellar/stellar-etl) turns ledgers into the rows behind SDF's BigQuery dataset (Hubble). It's the second SDF pipeline the kit runs, after [wallet-backend](https://docs.istravex.com/pilots/wallet-backend.md).

## How it works

stellar-etl's exports read Galexie files from a GCS datastore, and that's the only datastore it supports. So the pilot does what the wallet-backend pilot does: a small adapter, [`kit/pilot/stellar-etl/adapter.go`](https://github.com/hackcat29/istravex/blob/main/kit/pilot/stellar-etl/adapter.go), is copied into a checkout of the latest release (v2.8.34) and runs the same transforms each export runs, on each fixture's canonical ledger read from `corpus/ledgers`:

| Export | Transform |
| - | - |
| `export_transactions` | `TransformTransaction` |
| `export_operations` | `TransformOperation` |
| `export_effects` | `TransformEffect` |
| `export_ledger_entry_changes` | `TransformSigners`, `TransformTrustline` |

The rows are scored against each fixture's `expected_indexed` by [`kit/pilot/score-etl.mjs`](https://github.com/hackcat29/istravex/blob/main/kit/pilot/score-etl.mjs). stellar-etl writes 64-bit integers as JSON numbers (`to_muxed_id` can be 2⁶⁴ − 1), so the driver parses them with a reviver that keeps each number's source text: nothing is rounded on our side.

```bash theme={null}
npm run pilot:stellar-etl   # → corpus/results/pilot-stellar-etl.json
```

## Results

| Fixture | Verdict | Checks |
| - | - | - |
| `base-max-amount-local-0001` | incorrect | ✗ operation amount: expected `9223372036854775807`, got `9223372036854775000` ✓ account\_credited amount ✗ trustline balance: expected `9223372036854775807`, got `9223372036854775000` |
| `cap15-fee-bump-pubnet-0001` | correct | ✓ fee\_charged ✓ fee\_account |
| `cap27-muxed-max-id-local-0001` | correct | ✓ to ✓ to\_muxed\_id ✓ accounts |
| `cap27-muxed-payment-pubnet-0001` | correct | ✓ to ✓ to\_muxed\_id |
| `cap27-muxed-payment-pubnet-0002` | correct | ✓ to ✓ to\_muxed\_id |
| `cap40-signed-payload-signer-local-0001` | correct | ✓ signer ✓ weight |
| `cap40-signed-payload-signer-testnet-0001` | correct | ✓ signer ✓ weight |
| `cap46-muxed-soroban-source-local-0001` | correct | ✓ account |
| `cap73-sac-trust-local-0001` | incorrect | ✓ trustline row ✓ trust\_line\_limit ✗ trustline\_created effect: expected `GD45B6QQ2FUCP5RH6ESMVYVB37N3WC63TR7RVE6E3MQJ525QZYMROHJX`, got `—` |
| `cap73-sac-trust-pubnet-0001` | incorrect | ✓ trustline row ✓ trust\_line\_limit ✗ trustline\_created effect: expected `GBR6F2WIA5ORKAWHE7MPVOQJOYVRU7Y2E64TQCZK7R5KNU6ITIVNJFJZ`, got `—` |
| `cap73-sac-trust-testnet-0001` | incorrect | ✓ trustline row ✓ trust\_line\_limit ✗ trustline\_created effect: expected `GBHD5DZGDWWGLYBAS4Q2U2N4LAXZCTSNN2VU4NAOWGKQ6M46WMIEZBPA`, got `—` |

stellar-etl gets the fee-bump, every muxed ID (exactly, up to 2⁶⁴ − 1), the full `P…` signer and the muxed Soroban source right. It gets two things wrong, both already open upstream:

* **[#454](https://github.com/stellar/stellar-etl/issues/454), amounts as float64.** A payment of i64::MAX stroops comes out as `922337203685.4775` in the operation row and the trustline balance, 807 stroops short. The `account_credited` effect keeps the exact string. Any amount above 900,719,925.4740992 units is rounded the same way. ([F-0016](https://docs.istravex.com/project/findings.md))
* **[#460](https://github.com/stellar/stellar-etl/issues/460), no effect for SAC `trust()`.** The trustline row is there, but no `trustline_created` effect, on all three CAP-0073 fixtures, including a real mainnet USDC `trust()`, for which Horizon does record the effect. ([F-0017](https://docs.istravex.com/project/findings.md))

Every failing check maps to a finding, and `kit/check-results.mjs` fails if a new one doesn't, or if a finding claims a failure that no longer happens. When stellar-etl fixes either issue, the pilot shows it on the next run.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.