Vision, Language, Action

The combination of a vision, language, and action model that translates into a physical action through a robotic system marks a fundamental shift in how humans and machines interact. Instead of following fixed paths and depending on rigid, structured environments, VLA models give robots a new way to approach real-world problems. Coming from mechatronics and autonomous robots, this is the space I follow most closely — and the more I look into it, the more it comes down to one bottleneck: data.

Training a capable VLA policy takes an enormous amount of interaction data, and there are really only two ways to get it. Simulation can produce it at scale, but every simulated environment differs from the real world in physics, materials, and sensor noise — the sim-to-real gap — so a policy trained purely in simulation often falls apart the moment it meets reality. The alternative, teleoperation on real hardware, avoids that gap but is bound by real time: you can't parallelize a human operating a robot arm the way you can parallelize a simulation running on a GPU cluster. Every trajectory costs real human time on a real, usually expensive robotic system, which caps the amount of data you can realistically collect.

That's why I think the most promising lever right now is on the simulation side: building environments realistic and varied enough — better physics, better rendering, deliberate domain randomization — that a policy trained or finetuned on them generalizes to the real world with only a fraction of the real interaction data it would otherwise need.

This is also why I'm genuinely excited about what NVIDIA is doing with Omniverse and Cosmos: Omniverse provides the physically accurate simulated environment, and Cosmos turns that into photorealistic, physically plausible synthetic video data that can post-train robot policies directly — effectively attacking the sim-to-real gap from the data-generation side. I'm following Genesis just as closely: an open-source, generative physics engine that runs 10–80x faster than tools like Isaac Gym or MuJoCo and lets you generate entire scenes, tasks, and trajectories from language prompts, which makes environment variation something you can scale almost for free.

To get hands-on with this myself, I built a small simulation in Genesis of a Franka Emika Panda arm picking up a cube and placing it elsewhere — inverse kinematics for the approach and lift, position control for the gripper. Small in scope, but a good way to feel how quickly you can iterate on a manipulation task in simulation.

None of these fully close the gap yet, but they're moving in exactly the direction I find most interesting — and it's a big part of why I want to keep working at the intersection of simulation, AI, and physical systems.