Researchers at Harvard and Georgia Tech have created RLE-Bench, a 48-task test of whether general-purpose AI agents can do the work of robotics engineers. Current systems can write controllers, train robot policies, interpret sensor data and produce mechanical designs; the results also expose a persistent weakness when apparently sensible designs meet basic physics.
Artificial intelligence can already write software for robots. A more difficult question is beginning to matter: can an AI actually engineer the robot itself?
Researchers at Harvard University and the Georgia Institute of Technology have developed a benchmark intended to find out.
RLE-Bench, released in September 2026, tests general-purpose AI coding agents on 48 robotics engineering tasks. Rather than asking a model to answer questions about robotics or generate snippets of code, the benchmark gives AI agents engineering problems that require them to observe simulated physical environments, develop solutions, test them, inspect failures and revise their work.
The results suggest that current AI systems can already perform substantial parts of a robotics engineer’s job. They can develop control systems, train movement policies, build perception software and create mechanical designs.
They can also design robots that fall over.
That failure is one of the central findings of the project.
What does it mean for AI to “make a robot”?
Building a robot is considerably broader than generating a 3D model or writing instructions for a robotic arm.
A working robot requires mechanical engineering, control systems, sensing, software and an understanding of how all those systems interact.
A robot arm may be able to reach an object mathematically while causing its base to tip over. A controller may work under ideal conditions but fail once the robot carries additional weight. A perception system may work when an object is unobstructed and then lose track of it as the robot itself passes in front of the camera.
RLE-Bench was designed around this wider engineering process.
The benchmark divides robotics work into four areas: interactive control, policy development, perception and estimation, and mechanical design. The current release contains nine task families and 48 individual tasks.
The agents are given development time and computational resources, then allowed to experiment.
They can write code, run simulations, look at the results and change their approach. Their finished work is subsequently evaluated under hidden conditions that were unavailable during development.
That structure is intended to make the test closer to engineering than a conventional question-and-answer benchmark.
AI can already control robots
The first group of tasks examines whether an AI agent can operate a robot in an environment.
One RLE-Bench task places the agent in simulated kitchens and asks it to carry out physical jobs. The AI receives visual information about the environment and access to controls for the robot.
Depending on the difficulty level, it may receive only basic action interfaces or additional robotics tools such as camera calibration, perception systems and information about object locations.
The agent must work out how to complete tasks using the available information.
RLE-Bench then tests the resulting system in kitchen scenes the agent did not encounter during development.
Another task asks the AI to construct its own robotics “harness”, a collection of perception tools, controllers and instructions. That package is then given to a separate agent and tested on a new task.
The distinction is important. Writing a controller that solves one problem can amount to highly specialised programming. Producing reusable tools that allow another system to solve an unseen problem is closer to general engineering.
AI can train robot behaviour
RLE-Bench also examines whether an AI can develop learned control policies.
In one task, the agent has to train a simulated humanoid robot to track reference movements while maintaining its balance.
The system is allowed four hours of development on a single high-end GPU. Its final product is a trained motion-control policy that is tested under conditions that differ from those used during development.
The benchmark’s researchers documented one development run in which the AI repeatedly trained, evaluated and modified its policy. Early versions learned simply to remain upright. Later versions improved movement accuracy and robustness.
A final development candidate completed 545 randomly varied 20-second evaluation episodes without falling. The benchmark’s authors stress that this development result is separate from its hidden-test score, but it demonstrates that an AI coding agent can carry out a substantial reinforcement-learning workflow rather than merely propose one.
Another task asks agents to develop training recipes for vision-language-action models, systems designed to translate visual observations and instructions into robotic actions.
The submitted training procedure is used to train a new model, and that model is then evaluated under different conditions.
These experiments move the AI’s role beyond writing individual robot commands. The agent is making decisions about how another artificial system should learn to control a machine.
Perception remains part of the engineering problem
A robot cannot respond correctly to a world it cannot measure.
RLE-Bench therefore includes perception tasks in which AI agents have to build systems capable of locating and identifying objects under difficult conditions.
One task requires the resulting estimator to determine the position and orientation of an asymmetric object while it moves and becomes partially obscured by a robot arm.
The final system is tested against 100 hidden static images and ten movement sequences. Slow inference is penalised, and incorrect object identification can invalidate an entire group of results.
Another problem combines vision and force sensing. The agent must develop a policy for a robot that removes steel brackets from a cluttered bin and places them on a conveyor.
Performance is judged on hidden arrangements of objects and includes penalties for dropping objects, damaging them or causing excessive impacts.
The test therefore evaluates more than whether an AI can produce code that runs. The code must interact successfully with an uncertain simulated physical environment.
Then comes the awkward part: physics
Mechanical design exposes a different problem.
Software can often be corrected after an error. A badly designed physical machine may never have been capable of doing its intended job in the first place.
One RLE-Bench task asks the AI to design a common mobile base capable of supporting three different robot arms: Panda, UR5e and xArm7.
The robot must move its arm to targets at different heights and depths while carrying payloads. The agent produces both a mechanical model and the controller required to operate it.
Its design is then evaluated using different arms, shelf targets, payloads and static and dynamic stability tests.
This is where some apparently successful AI engineering begins to unravel.
The researchers highlight one submission that could reach the required shelves and carry the required payload. On those criteria, it worked.
Under stability testing, however, the same design failed both static and dynamic checks. The complete machine could tip over.
Another AI-produced design contained disconnected structural components despite the task explicitly requiring a connected frame.
These are not failures caused by obscure philosophical questions about intelligence. They are straightforward engineering errors.
A machine has mass. Its centre of gravity changes when an arm moves. Acceleration produces forces. A payload increases torque. A design that satisfies each obvious requirement individually may still fail when those requirements interact.
The benchmark’s researchers describe anticipating these combined physical consequences as one of the persistent difficulties exposed by current agents.
Why can an AI write good code and still design a bad robot?
Software development and physical engineering share many skills, but physical systems impose additional constraints.
A program can often be tested cheaply and repeatedly. If it crashes, the software can be changed and run again.
Physical systems contain consequences that cannot always be reversed so easily.
A motor can be undersized. A structure can buckle. A robot can collide with an object. A battery can run out. A machine can become unstable as its arm extends. Real sensors introduce noise and blind spots.
RLE-Bench attempts to reproduce some of these problems inside physics simulators.
The agents are allowed to follow a loop of building a solution, acting, observing what happens and revising their design. After development ends, the solution is tested under hidden scenes, robot configurations or physical conditions.
That hidden evaluation matters because an AI could otherwise optimise a design around the exact simulation it repeatedly sees.
A robot intended for the real world has to survive conditions its designer did not enumerate individually.
So can AI actually build a robot?
There is an important limitation to RLE-Bench: the robots in the benchmark are simulated.
The benchmark does not demonstrate an AI ordering components, machining parts, assembling hardware, wiring electronics and placing a completely self-designed physical robot on a laboratory floor.
Its mechanical-design tasks produce simulated models and software. The resulting designs are evaluated using physics simulations and hidden test conditions.
RLE-Bench therefore does not establish that today’s AI systems can autonomously manufacture a functioning physical robot from beginning to end.
It does show that the boundary is becoming less straightforward.
Modern coding agents can perform parts of the engineering workflow that would previously have required specialised human work. They can operate simulated robots, build controllers, create perception systems, train movement policies and produce mechanical designs.
The strongest agents can also use the results of failed experiments to modify their approach.
What they do not yet demonstrate consistently is the ability to integrate every part of the system while anticipating all of the physical consequences.
That integration is much of what engineering actually is.
Robotics knowledge still helps
The experiments also reveal that many AI systems perform substantially better when they are given specialised robotics tools.
In its interactive-control tasks, RLE-Bench compares agents working with several levels of assistance.
At the lowest level, the agent receives observations, robot state information and low-level controls. More advanced levels supply perception models, coordinate transformations, geometry tools and motion primitives; the highest level additionally provides privileged information about the simulated scene.
Most of the tested systems improved when given richer robotics infrastructure, although the strongest performer in the study was less dependent on it and in some comparisons performed worse when additional scaffolding was introduced.
The broader implication is that a general language or coding model does not automatically acquire every capability of a robotics engineer simply because it can write Python.
Specialised tools, simulations, perception systems and engineering interfaces remain important.
The benchmark itself is still incomplete
The researchers describe RLE-Bench 1.0 as a starting point.
Important parts of real robotics engineering remain outside the current benchmark, including broader hardware design, complete system integration, debugging, safety engineering and deployment. Harvard professor Na Li, one of the project’s leaders, said the team wants future versions to incorporate engineering problems contributed by robotics researchers and practitioners.
The benchmark is open source, allowing other researchers to run new AI systems through the same tasks and compare their results.
That could become increasingly important as coding agents become more autonomous.
Traditional AI coding benchmarks ask whether a model can repair a repository, implement a function or pass a suite of software tests. Robotics adds another question: does the thing produced by the AI continue to work once physics becomes part of the test?
RLE-Bench’s early answer is mixed.
AI systems are already capable of meaningful robotics engineering work. Some can iterate through failed experiments, build functioning control systems and train robust policies.
They can also produce a robot base that reaches every requested shelf and then falls over.
For now, that gap between satisfying a specification and understanding the physical consequences of the complete machine may be one of the clearest boundaries between impressive AI-generated engineering and an autonomous robotics engineer.
Sources
Harvard John A. Paulson School of Engineering and Applied Sciences, “Can Your AI Engineer a Robot?”, 17 September 2026.
RLE-Bench research team, “Introducing RLE-Bench: A Qualifying Exam for Coding Agents as Robot Learning Engineers,” updated 9 September 2026.
RLE-Bench, official benchmark and current results.
RLE-Bench GitHub repository, benchmark implementation and task documentation.