Mohamed Ayman Mohamed

I want to understand how reinforcement learning algorithms actually work, and what it takes to make them work in something closer to the real world.

Most RL research trains an agent until it converges, freezes the policy, and scores it. The real world doesn't sit still for that. Continual learning takes the other view: the agent should keep learning for as long as it runs. The trouble was that there was nowhere good to study it. The usual trick is to bolt task changes onto an episodic benchmark, which really only captures abrupt switches and still leans on episode boundaries. The purpose-built alternatives were too simple to be interesting.

So at Marlos C. Machado's lab at Alberta, for my MSc, I built AgarCL, a research platform based on the game Agar.io. It's non-episodic and partially observable, with high-dimensional observations, continuous actions, and dynamics that keep evolving as the agent grows. There's no point where the problem settles down, which is the whole idea.

Then we measured. DQN, PPO, and SAC all struggle on it. The result I found more interesting was what happened when we tried the standard continual-learning fixes: Shrink and Perturb, ReDo, and Continual Backpropagation each barely improved on plain RL. Those methods are built to treat loss of plasticity, so if they don't help, the difficulty here isn't only the stability-plasticity trade-off. Something else is going on, and working out what is the part I care about.

A realistic environment also contains other agents, and the same kind of convenient assumption turns up there. Psychology splits empathy in two: working out how someone else is doing, and caring about it. Multi-agent RL has borrowed the caring part heavily, calling it social preferences, and skipped the working-out part by simply telling each agent what its neighbours earned. No real agent gets that, because a reward is private. We figure out how someone is doing by watching what they do and picturing ourselves in their place.

So in Self-Referenced Social Preferences we gave agents that same move. Each one learns a model of its own reward, then applies it to what it sees others do, asking what it would have earned in their place. That alone keeps cooperation going in social dilemmas, and it often splits the group's earnings more fairly than agents that can read everyone's real rewards.

Both projects come from the same instinct, and it's the thing I'd most like to be known for: find the assumption a field is quietly leaning on because it's convenient, take it away, and see what still works.

Before all this, at KAUST, I worked on connecting language to 3D scenes, which is where the ICLR and TPAMI papers below come from. And before research I spent years close to the hardware: a pipelined RISC-V processor in Verilog, an x86-64 bootloader that brings a machine up into 64-bit mode, a distributed system in Rust with its own leader election. Day to day I'm now a software engineer at AWS, building AI tools for Kubernetes and region infrastructure. Those hardware years still shape how I work: when everyone assumes a number, I'd rather go and measure it.