Value Proposition
The technology consists of cooperative high-level planner and low-level controller components, which ensure that natural language requests are transformed into a plan executable by the robotic system. The high-level planner uses a local language model to perform natural language processing (NLP), allowing a user of any skill level to create valid robotic plans. The low-level controller consists of modular robot functions, which allow the robot to execute plans without any order limitations. This technology is intented for static and mobile robotic automization tasks with access to limited compute resources or highly qualified specialists.
Business and Innovation Perspective
Challenge
The effective introduction of robots into new environments requires personal training and programming, both of which are time consuming endeavors. Human-robot collaboration using natural language requires large computational resources or man-hours.
Existing Alternatives
Current alternatives that utilize LLMs generally rely on cloud services that do not ensure privacy and, in the case of a loss of connection, the physical platform loses the ability to perform work. Alternatives for high-level planning are based on creating symbolic descriptions of the specific work environments, which is time consuming and makes the system less portable between different locations.
Solution and Uniqueness
The system is developed for working with limited compute resources and is not reliant on third party services, ensuring maximum availability and the ability to adapt to changes in resource availability or novel work environments.
TRL: 4
Intelectual Property Status:
know-how
Projects:
Edge AI
Technical Specification
Operating principle
High-Level Planner
A user provides the system with a request expressed in natural language. This incoming query is categorized based on it’s semantic contents. If a request for a robotic task is detected, it is passed onto the planning sub-system which expands the request and performs plan generation for the specific robot that could complete the designated task.
Low-Level Planner
The LLP consists of different platform-based skills, such as ‘Pick’, ‘Place’, ‘Go’, which are available to the HLP. The skills consist of primitives such as ‘move_manipulator’, ‘navigate_to_location’, ‘get_grasp_pose’, etc. The skill set can be extended indefinitely, as the individual skills are modular and consist of lower complexity primitives, which can be combined to form new skills. The LLP receives a structured plan, which is a sequence of skills with semantic goals. It then iterates through each step of the plan, giving feedback of the execution result. If all the steps are executed successfully, the goal has been reached, otherwise, if any of the steps fail, the whole plan is considered failed.
Parameters
| Parameters | Description |
| Inference Hardware | Commercial GPU (8-32GB VRAM) |
| Language Processing | Local LLM (<15B parameters) |
| Model Framework | Open-source software (llama.cpp) |
| Control Framework | ROS/ROS2 |
| Robot Platform | Various |
