Surgically making neural networks safe
Detecting misalignment from model internals, even when models know they are being watched.
I am an independent researcher. I study how to catch unsafe behaviour from a model's internals rather than its outputs: backdoors, evaluation awareness, and reasoning that never reaches the chain of thought. I also mentor research teams at Algoverse. Previously: M.Sc. in AI at NTU Singapore.
What a model says it is doing and what it computes internally can differ, so checking outputs alone cannot confirm that a model is aligned.
Models can tell when they are being evaluated and change how they behave, and information can move to places interpretability tools do not look.
I build white-box monitors for agents and test them against models that are trying to hide: backdoors, obfuscation, and reasoning that never reaches the chain of thought.
Current focus: white-box monitoring for AI agents. A side interest: Awareness Jailbreaking: Revealing True Alignment in Evaluation-Aware Models
LLMs can fool a monitor by moving information between linear and non-linear representations and by altering feature covariance.
As model size grows, evaluation awareness grows with it.
Open-source sparse autoencoders disentangle facts worse than raw neurons, while distributed alignment search beats them all.
A nearly 50-page primer connecting trustworthy machine learning and causality, extended to large language models.
A common formal language for interpretability methods, built on causal abstraction.
Authors at FAR.AI, Mila, UK AI Security Institute, Tsinghua University, University of Cambridge and Harvard University
cited Evaluation Awareness Scales Predictably in Open-Weights Large Language Models
in Preparing for AI Systems That Deceive Developers, EvalDetectBench and Behavioural Analysis of Alignment Faking
I mentor research teams at Algoverse AI Research.
I am open to collaborations on white-box monitoring and agent safety. The fastest way to reach me is email.