Will this Turkish bill become law? I tested my own model on 2026

In short

The problem. New bills reach the Turkish parliament (TBMM) every week, and most never become law. Anyone who needs to know which ones will pass, and when, still follows the parliament's pages by hand.

The answer. ULAK-1 puts a probability on a dated question for each bill: will this bill be adopted by the General Assembly by this date? On 2026 questions it never saw in training, its error came out 45% lower than the historical average.

The proof. Live forecasts are written to a public ledger before the event and scored once the outcome is known. Misses stay there too.

What is the problem?

In the current (28th) term, none of the 3,541 bills led by opposition MPs became law. Of the 410 bills led by MPs of the governing AK Party and MHP bloc, 134 did. That is a count from parliament's own records, not a political judgement.

For a law firm, a company's public affairs team or a trade association, the real question is which of hundreds of pending bills will pass, and when. Spending hours on the wrong bill is expensive. Noticing the right one late is too.

General-purpose AI models do not help much here. An open 27-billion-parameter language model I tested expected 82% of bills to be adopted. In reality about one in twenty is. The same model believed the current year was 2024.

How did I ask the question?

"Is this bill important?" is not a forecasting question. For each bill I ask a dated yes-or-no question instead: "Will bill X be adopted by the TBMM General Assembly by 16.01.2027?" When the deadline arrives, the answer falls out of the parliamentary record and leaves no room for interpretation.

I ask at three points in a bill's life: after submission (S0, within 90 or 180 days), after the committee report (S1, within 30 or 60 days) and after the first plenary debate (S2, within 14 or 30 days).

Every forecast is made at a specific moment, and the model only sees records that were public by then. Nothing learned later is put in front of the model, not even in a backtest. This is exactly where most backtests get inflated.

What is ULAK-1?

ULAK-1 is a small probability model I trained from scratch on a TBMM dataset I built myself. It does not generate text like a chat model. It reads process, sponsor and agenda signals and returns a single number. It was trained on 7,030 questions.

I do not publish how it computes its forecasts. I do publish, in full, how I measure them.

The result: error cut roughly in half

The yardstick is the Brier score. Think of it as an error score: the further a probability lands from what actually happened, the larger it gets. 0 is perfect. Someone who says 50% to every question scores 0.25.

The reference point is the historical average: a forecaster that reads nothing and simply says how often bills at this stage were adopted within this window in the past.

The test set is 396 questions asked on or after 1 March 2026, covering 226 bills, 70 of which were adopted. The model never saw this period in training.

Against two frontier language models, with the same information

I did not want an easy opponent. I asked a current frontier language model all 396 questions in Turkish, giving it exactly the information ULAK-1 sees, plus the historical average for that stage. Its knowledge ends around mid-2025, so it cannot have memorised 2026 outcomes.

With all of that in hand, the model added almost nothing over the historical average (+1%). The gap to ULAK-1 is −0.046, with a 95% confidence interval of [−0.081, −0.014]. The interval excludes zero, so the gap is not chance.

I also tried a second, larger frontier language model with reasoning turned on, on 100 random questions. ULAK-1 scored 0.070 and that model 0.129, a gap of −0.059 [−0.115, −0.010]. Blending the two forecasts 50/50 did not help either: the blend scored 0.071, worse than ULAK-1 alone.

So access to facts did not make the difference. Turkey-specific, time-stamped data and a model trained on it did.

Where does it work, and where does it not?

ULAK-1 is strong in the first two stages. It cuts the error by 68% after submission and by 53% after the committee report. That is where the value is, because a bill's fate is mostly settled in those two stages.

At the plenary stage the model fell behind the historical average (−8%). The reason is easy to see. In the test period every bill at this stage came from the government bloc, and between 62% and 86% were adopted within 14 or 30 days. There is almost no signal left to separate them. I removed this stage from live forecasts.

Are the probabilities honest?

For a probability to be useful, the events I call at 30% should happen about 30% of the time. On the 2026 test the average gap (ECE) is 0.034, about 3.4 percentage points. When ULAK-1 gives a bill 30%, read it as "about three in ten bills like this are adopted by that date", not as a verdict.

The live ledger: sealing forecasts before the event

However carefully a backtest is built, it remains a claim. The real proof will come from live results.

Since 17 October 2026 I have been writing forecasts for open bills to a public ledger before the event. The first two files hold 203 forecasts each. Files are written once and never edited. Each file's SHA-256 hash stops matching if a single digit changes.

You can see today's open forecasts, resolved results and each file's hash on the Turkey Probability Index page (in Turkish). I will publish results there as they resolve, misses included, and I will never put backtest and live scores in the same table.

Limits

• -No live results yet. Every performance number in this piece is a backtest.
• -One legislative term. The model was trained and tested on the 28th term. If the balance or the procedure in parliament changes, it needs to be tested again.
• -A small test set. 396 questions, 226 bills, 70 adoptions, so the confidence intervals are wide.
• -I found and fixed two bugs. Bill numbers restart every term, so the first run mixed in agenda records from other terms. Bills renewed from earlier terms also kept their old dates. Every number here is after both fixes.
• -The comparison used the same information. I have not yet compared against frontier models using their own web search.
• -I forecast procedure, not content. I take no position on whether a bill is good or bad.

What comes next?

• -The first deadlines fall in November 2026. From then on I will post the results every week, misses included.
• -I will rebuild the plenary stage with agenda timing, and keep it out of live forecasts until it works.
• -I will compare against frontier language models with their own search, on the same questions and deadlines.

If you follow a Turkish bill, send me its number and I will show you where it stands in the ledger.

Method

• -Data. TBMM 28th-term bills: process, signature and agenda records, each stamped with the moment it became public.
• -Questions. Three stages, two windows each: S0 (90 or 180 days), S1 (30 or 60 days), S2 (14 or 30 days). 7,030 training questions.
• -Test. 396 questions asked on or after 1 March 2026, 226 bills, 70 adoptions.
• -Metric. Brier score. Skill is the reduction in error against the historical average (the training-period adoption rate by stage and window). ECE for calibration.
• -Uncertainty. 95% confidence intervals from a 2,000-sample bootstrap clustered by bill.
• -Comparison. Two current frontier language models. One with reasoning off, on all 396 questions. One with reasoning on, on 100 random questions. Turkish prompts, the same information ULAK-1 sees, plus the historical average.
• -Live record. github.com/cdliai/ulak-ledger. Files are written once, and their SHA-256 hashes are listed in the ledger's HASHES.md.

Citation

Karagöz, F. B. (2026). Will this Turkish bill become law? I tested my own model on 2026. CDLI Publications. https://cdli.ai/yayinlar/kanun-teklifi-tahmini/en

The probabilities in this piece are event forecasts, not investment or legal advice. I take no position on the content of any law. I estimate the probability of procedural events from public records.