Lightweight Latent Reasoning for Narrative Tasks

Alexander Gurung¹, Esmeralda S. Whitammer^1,2, Mirella Lapata¹

¹ University of Edinburgh ² CIFAR Fellow

TACL 2026 (Presented at EMNLP 2026)

TL;DR

LiteReason interleaves latent reasoning with standard token sampling. We initialize a lightweight Reasoning Projector from a small dataset of the model’s own reasoning traces, then use RL to improve task performance and learn when to use the projector. On narrative tasks, it reaches 69-96% of the gains from non-latent RL while cutting final reasoning traces by 70-73% and RL training tokens by about half. In contrast to other latent reasoning methods, LiteReason’s savings do not come at the cost of general capabilities (GSM-Hard, AIME25, and MMLU-Redux). Furthermore, LiteReason can be combined with length-penalty reward shaping, producing the shortest and most performant model on Flawed Fictions.

Figure 1. LiteReason alternates between the model's normal discrete sampling and latent reasoning. In discrete mode, the LM Head samples a token and its embedding is fed back as the next input. If the sampled token is an implicit-thought tag such as <bot> (shorthand for our <implicit_thought> tag) with a thought budget, generation switches to latent mode: the Reasoning Projector maps the last hidden state to a continuous embedding, feeds that embedding back for the budgeted number of steps, and then returns to discrete sampling. The model can switch between these modes multiple times before producing the final answer.

The LiteReason Framework

LiteReason keeps ordinary LM Head sampling but adds a Reasoning Projector. When the model emits an implicit-thought tag with a step budget, the projector predicts continuous token embeddings directly from the final hidden state before generation returns to text.

We first sample reasoning traces from the model and retain those with positive reward. We replace selected reasoning sentences with implicit-thought tags and train the Reasoning Projector on the remaining text, keeping the rest of the LLM frozen. We then train the LLM with GRPO, treating only discrete token sampling steps as policy actions. After each RL epoch, we update the projector with SFT on reasoning traces generated during that epoch.

Example Generation

Illustrative example of LiteReason inference on a Flawed Fictions-style input. Discrete tokens are sampled normally; an <implicit_thought>n</implicit_thought> tag switches the model into latent mode for n steps (the pulsing dots) before discrete generation resumes.

Narrative Reasoning Requires Longer, More Varied Steps

Compared with the benchmarks used in previous latent-reasoning work, our narrative tasks involve more varied and complex reasoning steps. For both Flawed Fictions and Next Chapter Prediction, the model has to cite and reason over story information in varied and diverse ways depending on the specific story. We hypothesize that these tasks provide a more difficult and realistic setting for latent reasoning.

Example reasoning steps. Steps from common latent-reasoning benchmarks (top) are short and formulaic; steps from our narrative tasks (bottom) are longer, reference long-range context, and vary in structure.

Dataset	Example Reasoning Step
GSM8K-Aug	The helmet costs $15 × 2 = $30.
ProsQA	Every bompus is a wumpus.
ProntoQA	Each vumpus is mean.
Flawed Fictions	The continuity error occurs because the story earlier establishes that the little girl was very poor and had no room to live in or bed to sleep in, but later it states that she returned to her small bed in the shelter with her newfound wealth.
Next Chapter Prediction	`<citation>`Source A (Character Sheet: Rose) says Rose is rebellious, disobedient, and has a sarcastic sense of humor.`</citation>`, therefore `<reasoning>`Rose will likely continue to challenge authority figures and express her opinions, possibly provoking Miss Wellwood and leading to a confrontation.`</reasoning>`

During SFT, selected reasoning sentences are replaced with sequences of latent tokens. The projector is trained to produce embeddings that help the model generate the remaining reasoning text.

LiteReason Approaches RL Performance with Fewer Tokens

We evaluate on two narrative tasks: Flawed Fictions, a 414-example plot-hole detection benchmark, and Next Chapter Prediction (NCP), a 1,347-example book-planning task. The plots below show the main performance-compute tradeoff: better methods move right, and cheaper methods move down. LiteReason is the only latent-reasoning method that moves close to non-latent RL performance while staying far below it in generated-token cost.

69-96%

of non-latent RL gains

77-92%

fewer inference tokens vs. base

49-53%

fewer RL training tokens

~3×

faster single-example inference vs. RL

Flawed Fictions

Flawed Fictions asks whether a story contains a plot hole. RL improves Qwen2.5-7B from 57.26% to 88.71% accuracy, but still generates 114.53 tokens on average. LiteReason reaches 87.42% accuracy with 34.01 tokens, while prior latent baselines remain much closer to the base model.

Figure 2. Accuracy vs. generated tokens on Flawed Fictions. Aside from RL-Trained, LiteReason is separated from the next-best method by roughly 30 accuracy points and about 190 generated tokens.

Next Chapter Prediction

NCP evaluates a generated plan for the next chapter in a book. We define a contrastive version of the VR-CLI objective that measures how much a plan increases the likelihood of the true next chapter while decreasing the likelihood of other chapters, which we call Contrastive Improvement. The RL-trained model obtains the highest score (0.666) but uses 721.33 tokens on average. LiteReason obtains 0.478 with 193.11 tokens, landing on the same performance-cost frontier; CoLaR is similarly short but much weaker at 0.118.

Figure 3. Contrastive Improvement vs. generated tokens on NCP. LiteReason is far cheaper than RL-Trained while substantially outperforming the other latent-reasoning baselines.

LiteReason Improves RL Efficiency

With the same RL steps and samples, LiteReason uses about half as many generated training tokens and is faster in wall-clock inference.

RL training tokens. The RL and RL + LiteReason runs use the same RL epochs, steps, and samples; LiteReason generates 52.8% fewer tokens on Flawed Fictions and 49.5% fewer on NCP during training.

Method	FF Training Tokens ↓	NCP Training Tokens ↓
RL	61.4M	108.8M
RL + LiteReason	29.0M(-52.8%)	54.9M(-49.5%)

Inference wall-clock time. Mean seconds per example, averaged over three sequential repetitions, for Qwen2.5-7B models using vLLM on one H100. b=1 runs one example at a time; b=all passes the full test set for parallel computation. The Latent column indicates whether LiteReason uses the latent-reasoning prompt at inference.

Task	Model	Latent	b=1 (s) ↓	b=all (s) ↓
FF	Base	×	3.10	0.13
	RL-Trained	×	0.97	0.06
	LiteReason	×	0.83	0.20
	LiteReason	✓	0.31	0.03
NCP	Base	×	8.02	0.43
	RL-Trained	×	6.46	0.42
	LiteReason	×	2.21	0.26
	LiteReason	✓	1.84	0.23

Compared with RL-Trained, LiteReason produces traces 70% smaller on Flawed Fictions and 73% smaller on NCP while still achieving 96% and 69% of the respective RL performance gains.

LiteReason Combines with Length-Based Reward Shaping

We apply DAPO’s overlong reward shaping to both standard RL and LiteReason on Flawed Fictions. The length penalty improves performance and efficiency for both methods. On LiteReason, adding the length penalty produces the highest accuracy, 93.55%, and the fewest generated tokens, 6.00 on average.

Method	Accuracy (%) ↑	Tokens ↓
RL-Trained	88.71	114.53
RL-Trained + LP	91.94	16.65
LiteReason	87.42	34.01
LiteReason + LP	93.55	6.00

LiteReason Retains General Model Capabilities

On GSM-Hard, AIME25, and MMLU-Redux, LiteReason and the RL-Trained baseline largely retain the abilities of Qwen2.5-7B. LiteReason models also reason more concisely, consistently producing fewer tokens than the base model and RL-Trained variants.

Cross-task generalization for Qwen2.5-7B (Table 10). ✓ uses each method’s latent-inference pipeline; × uses standard token inference. Values are mean accuracy (%) and generated tokens.

Task	Method	Latent	GSM-Hard		AIME25		MMLU-Redux
Task	Method	Latent	Acc. (%) ↑	Tokens ↓	Acc. (%) ↑	Tokens ↓	Acc. (%) ↑	Tokens ↓
No finetuning		×	27.3	371.4	11.3	891.1	78.7	317.0
FF	RL-Trained	×	27.2	359.1	10.0	857.3	79.0	300.5
	LiteReason	×	27.3	349.7	10.7	834.7	78.8	283.4
	LiteReason	✓	26.9	316.3	4.7	857.4	78.6	273.4
	CoLaR	×	27.1	372.2	8.0	876.1	78.9	328.4
	CoLaR	✓	26.9	338.8	8.0	865.8	78.2	208.6
	COCONUT	×	7.9	461.1	0.0	489.4	29.9	404.6
	COCONUT	✓	12.2	393.4	0.7	660.0	35.7	344.2
NCP	RL-Trained	×	27.3	365.0	10.0	912.7	78.8	297.9
	LiteReason	×	26.5	334.4	4.0	737.2	78.2	250.0
	LiteReason	✓	26.5	301.1	7.3	617.1	76.7	186.9
	CoLaR	×	5.9	716.0	0.7	655.0	26.9	424.0
	CoLaR	✓	1.3	674.9	0.7	516.9	29.4	660.3
	COCONUT	×	19.4	349.1	2.0	783.5	43.1	274.8
	COCONUT	✓	15.2	422.2	0.0	829.6	38.9	376.5

LiteReason Works Across Model Sizes and Families

On Flawed Fictions with Qwen3-4B-Instruct-2507, LiteReason improves accuracy from 33.23% to 57.42% while reducing output length from 1709.86 to 995.38 tokens. On GSM-Hard with Gemma-3-1B-IT, LiteReason largely matches the base model while producing about 9% fewer tokens.

Flawed FictionsQwen3-4B
Method	Accuracy (%) ↑	Tokens ↓
Base	33.23	1709.86
RL-Trained	64.84	1078.57
LiteReason	57.42	995.38
CoLaR	29.35	336.49
COCONUT	28.55	20.64

GSM-HardGemma3-1B
Method	Accuracy (%) ↑	Tokens ↓
Base	14.70	932.82
RL-Trained	15.91	941.13
LiteReason	15.76	849.92
CoLaR	1.14	978.24
COCONUT	1.52	25.12

Takeaways

LiteReason lets the model learn when to interleave latent reasoning with discrete token sampling during RL. On Flawed Fictions and Next Chapter Prediction, it comes closest to non-latent RL performance among the latent-reasoning methods tested, while generating substantially fewer tokens during training and inference. It also preserves general model capabilities and can be combined with length-based reward shaping.

Citation

@article{gurung2026lightweightlatentreasoning,
  title     = {Lightweight Latent Reasoning for Narrative Tasks},
  author    = {Alexander Gurung and Esmeralda S. Whitammer and Mirella Lapata},
  journal   = {Transactions of the Association for Computational Linguistics},
  year      = {2026},
  url       = {https://arxiv.org/abs/2512.02240}
}