Alisher Yeginbay

Writing in Public

Wed, 12 Aug 2026 00:00:00 +0000

It's much easier to think about writing than actually writing. It's been a while since I wrote the post on starting this blog. I've written a few personal write-ups and technical posts on interpretability. But I believe I could've done much more.

It's unsustainable in the long-term–at least for me–to constantly learn and do any intellectual work solely by having technical curiousity. There are other crucial components for being motivated as well: social belonging, seeing progress, and maintaining basics (physical and mental health).

Operation K #2: Habits

Thu, 09 Jul 2026 00:00:00 +0000

안녕하세요 여러분!

It's been about 3 weeks since the Operation K #1 blog post. This part isn't about Korean itself, but rather about learning. I decided to write about the thing that keeps language learning moving—habit.

I noticed that I struggle with consistency a lot. I review my Anki cards daily, but I struggle to review the grammar lessons and do immersion. I do progress vocabulary-wise, but that's not the engine of the language learning process.

Operation K #1: Beginning

Thu, 18 Jun 2026 00:00:00 +0000

This is the beginning of my Korean learning blog post series. I'll share my progress, updates on the learning approach and other Korean-related updates.

Introduction

Learning a language is a long-term process. It's important to choose a plan that will yield good results in the future.

What I can say for sure is that most of the internet guides are ambiguous or low quality. Choosing the right way to learn is really hard. Often while starting you might stumble upon many issues that will make you lose the motivation to learn. It's especially notable for Korean, since the community of Korean learners is not that developed compared to, e.g., Japanese learners' community.

Susceptibility Limits Faithfulness Measurement

Fri, 12 Jun 2026 00:00:00 +0000

Chain-of-thought (CoT) lets us look at reasoning steps behind LLM's thinking processes. Monitoring CoT provides a way to catch unaligned behaviour in order to prevent catastrophic events. The safety benefits of interpreting CoT depend on how faithfully models produce them. However, in order to evaluate CoT faithfulness, it's important to factor in the model's susceptibility—how readily an LLM is swayed by the hint.

We find that Opus seldom changes its answer under biasing hint. Since it exceedingly rarely gets persuaded by the bias, it becomes hard to evaluate the true faithfulness of the model.

Seeing Superposition

Sat, 23 May 2026 00:00:00 +0000

I've been doing mech-interp for some time and "superposition" kept appearing in everything I read. I had a basic sense of the term and wrote a brief explanation post. Still, the best way to comprehend something is to see the thing with your own eyes. So I decided to replicate the experiment from Elhage et al., 2022 (notebook). It's not the whole paper, but specifically $W^T W$ heatmap. I wanted to see superposition emerge from training and address a few questions I had while replicating the experiment — specifically why non-linearity matters and what off-diagonal patterns are actually showing.

Building AI Agent Memory

Thu, 07 May 2026 00:00:00 +0000

I recently built a memory service for an AI agent as part of a coding challenge. This post walks through the design space such a system operates in.

The Problem

LLMs are stateless. They start from a blank context every time you call them, i.e. they forget everything between sessions. The way to address this problem is to provide a memory solution for an agent.

On the higher level, a memory layer is pretty simple. It sits between your app and the LLM, and automatically extracts, stores and pulls (if needed) various facts from conversations.

Superposition, Briefly

Sun, 26 Apr 2026 00:00:00 +0000

It would be great for interpretability purposes if neurons of a neural network could cleanly represent features. Practically, this might be true for some cases, but not always. The problem is that inputs might have a very wide range of features — this is especially true for neural networks like large language models. In response to this, models end up representing more features than they have dimensions. This phenomenon is called superposition.

Transformer Circuits

Sat, 11 Apr 2026 00:00:00 +0000

Mechanistic interpretability (often shortened to mechinterp) tries to understand what's actually happening inside transformers, rather than treating them as black boxes.

The standard implementation of the transformer from the original transformers paper is not the most interpretable one. They use a computationally efficient form that obscures the structure. As described in more detail in "A Mathematical Framework for Transformer Circuits", there's another way of doing the mathematically equivalent computations but in a more comprehensible way.

The Blank Page and Me

Mon, 30 Mar 2026 00:00:00 +0000

I love reading interesting people and their work — especially essays from Paul Graham, Patrick Collison, Alexey Guzey, Gwern, Derek Sivers and many other cool people. So I decided to start my own journey as a writer to improve my skills and create more!

I was putting off writing because of "absence of ideas" and "awkwardness" of publishing rough posts, especially as a perfectionist. Obviously, these reasons are more of an excuse rather than being rational.

Contact

Mon, 01 Jan 0001 00:00:00 +0000

Feel free to email me at alisher at yeginbay dot com or DM on Twitter!