SIGNODEX
← Doğruluk kaydı

Metodoloji · sabitlenmiş

Metodoloji v1

Sabit

Halka açık YZ sinyal doğruluk kaydının methodology_version = 1 altında skorlanan her satır.

Bağlayıcı spesifikasyon İngilizce. Yukarıdaki yerelleştirilmiş sayfa çerçevesi onu çerçeveler; aşağıdaki gövde kanonik versiyondur.

1. Scope

Every row in ai_signal_outcomes with methodology_version = 1 was scored under the rules below. Currently this is all rows. Rows from a future v2 will be excluded from this page's claims and surfaced separately on /track-record.

2. Direction extraction

The AI's directional call is parsed from the "Signal: BULLISH / BEARISH / NEUTRAL" line in the streamed response (server-side, by_shared/signal_parsing.ts). If the model emits none of the three words at all, the row is dropped — never silently coerced.

3. Resolution sources (v1)

  • Crypto: price at +24h and +7d is fetched from CoinGecko's top-250 markets endpoint, USD denomination.
  • Stocks / metals / forex: the v1 cron does not score non-crypto signals. They are recorded in the ledger but theirwas_accurate_* columns remain null until v2 ships.
  • No third-party aggregator is used. Symbols that CoinGecko cannot return inside the resolution window are skipped, not estimated.

4. Accuracy rules (v1)

A signal is "accurate" when the directional call matches the actual price move by the magnitudes below:

  • Bullish is accurate when the resolution price is at least +0.5% above the signal price.
  • Bearish is accurate when the resolution price is at least -0.5% below the signal price.
  • Neutral is accurate when the absolute price move stayed below 2.0%.

These thresholds are intentionally narrow. Wider thresholds would inflate the accuracy figure because crypto crosses ±0.5% on noise alone. We pre-committed to the narrow band before publishing to avoid the appearance of tuning for outcomes.

5. Anti-spoofing

The price_at_signal is fetched server-side by ai-proxy after the AI response streams, NOT supplied by the client. A bad actor passing a fake priceAtSignalparameter cannot skew the published statistic. The server fetch path is documented at supabase/functions/ai-proxy/index.ts (search fetchTrustedPrice).

6. Survivorship bias

A small fraction (~12%, recomputed daily on /track-record) of issued signals are dropped because their resolution price cannot be fetched within the cron window — long-tail / low-liquidity crypto. This introduces a survivorship bias toward higher-liquidity assets. Dropped rows are NOT counted in either numerator or denominator.

6.5 Origin neutrality

The ledger is bootstrapped by a daily cron that issues signals on a fixed list of ~24 high-liquidity assets, plus signals issued ad-hoc by paying users. Both paths go through the same model, same system prompt, same compliance filter and the same resolution rules. The ledger does not segregate them. The aggregate stats above include both. The cron list is defined in supabase/functions/seed-ai-signals/index.ts.

7. Aggregation rules

  • Each row contributes equally — there is no weighting by user, by asset, by direction, or by sample. The figure published is a raw hit rate.
  • Wilson 95% confidence intervals are computed and displayed alongside the point estimate. The CI is the canonical figure; the central percentage is just its midpoint.
  • Only rows with methodology_version = 1 contribute to v1 statistics, even after v2 ships. The two are never mixed.

8. Audit hash

The thresholds in §4, the resolution sources in §3 and the aggregation rules in §7 jointly produce this version's identity. A change to any of them requires a new /methodology/v{n} page and an increment of methodology_version on new rows. The migration that introduced v1 is 008_methodology_version.sql. Git history of that file is the audit trail.

9. Out of scope (v1)

  • Stock, metal and forex signals are recorded but not scored. Coming in v2.
  • Confidence-weighted scoring (where the model's self-reported confidence affects whether the row counts) is not used. Direction is binary; confidence ignored.
  • Position-sizing or hypothetical PnL is not computed. The ledger measures pattern recognition, not investment performance.

Past performance does not predict future performance. See Disclaimer. For the live numbers see track record.