Dongguk University
← Back to Research
01

Physical AI

We study how robots can connect visual observations and language instructions to actions in the physical world. Our research brings together predictive world models, Vision-Language-Action models, and spatially grounded planning to make robot behavior more adaptable to changing tasks and environments.

Conceptual world-model and VLA pipeline alongside robot tasks in several indoor environments

Vision-Language-Action & Robot World Models

Ongoing research

Robots need more than object recognition to perform a task: they must relate an instruction to the current scene and anticipate how their actions may change it. We investigate Vision-Language-Action (VLA) models alongside robot world models to connect perception, prediction, and action selection.

  • Visual and language grounding: Relate task instructions to objects, the robot, and its end effector using visual observations.
  • Predictive scene modeling: Study how learned models can generate plausible future observations and represent changes associated with robot actions.
  • Action selection and feedback: Explore how predictions can inform robot actions and how new observations can guide subsequent decisions.

The demonstration below illustrates object/end-effector detection and generated observations compared with reference views. Our continuing work focuses on connecting these representations to robot control and evaluating prediction quality and task execution.

▶ Watch demo
Robot-arm and object detections beside reference and generated observations from three viewpoints
Robot and object detection, with reference and generated observations shown from multiple viewpoints.

3D Vision-Language-Planning

Ongoing research project, 2026-2029

Moving an object between locations requires a robot team to understand where objects are, which robot can perform each step, and how those steps depend on one another. Our 3D Vision-Language-Planning (3D VLP) project investigates a shared spatial representation that supports this reasoning across heterogeneous robots.

  • Semantic 3D representation: Integrate visual information from multiple robots into a map that connects scene geometry with object and place semantics.
  • Spatially grounded task planning: Interpret language instructions in the context of the map and decompose them into actions suited to different robot capabilities.
  • Collaborative execution and validation: Investigate how plans can be coordinated, updated during execution, and evaluated through simulation and real-robot experiments.

This project connects our work in Spatial AI with language-guided robot planning. The roadmap below presents the intended development path from shared scene understanding to coordinated operation and simulation-to-real-world validation.

Related project: 3D VLP for heterogeneous robot teams

Conceptual illustration of shared 3D maps, heterogeneous robot task planning, and validation in real environments
Conceptual research roadmap: shared semantic 3D maps, heterogeneous robot task planning, and simulation-to-real-world validation.
← Back to Research