Applications
The research in robotics are typically focusing on specific problems such as design, planning and control. The study of integrated robotic system is usually not so appreciated and is taken for granted most often. However, to make robots useful for real-world problems, it is extremely important to integrate robotic components from different levels in terms of hardware design, task and motion planning, perception, execution etc. To facilitate this process, we are developing an intelligent industrial robot operation system - CobotSys at COBOT Technology which allows the end-users to check the validity or the optimality for the design, planning, perception and execution processes without really building most of the hardware setups beforehand. We will report many of our recent progress about this work in the future.
On this page, some real-world applications from my past research will be presented ranging from robust grasping to compliant manipulation, as well as deep learning based object defects inspection. From my personal experience, it is always important to bear in mind that robotics is great for the world, however, it is challenging to make robots work for the real world applications.
Current Applications and Industry Translation
Our applied work follows a simple path: collect richer physical-interaction data, learn robust manipulation skills, and deploy them in real environments. Recent systems span the Tianwen/Yuanyou humanoid platforms, HandX tactile-data infrastructure, and healthcare-oriented robotic systems.
Tianwen / Yuanyou Humanoid Robots
The Tianwen family provides a real platform for studying human–robot interaction, dexterous grasping, tool use and embodied foundation models. Public demonstrations have progressed from handshaking and microphone delivery, to grasping cups and laboratory tools, coffee service, brush calligraphy, and long-horizon autonomous manipulation. These systems connect our academic work in grasping, tactile sensing, VLA models and policy learning to real service environments.
HandX Tactile Data Infrastructure
A central HandX direction is to turn human physical interaction into scalable robot-training data. The latest fabric tactile glove integrates tactile sensing directly into a lightweight wearable form factor so that ordinary human demonstrations can capture not only motion and vision, but also the contact information that is otherwise difficult to recover from video alone. Official HandX product information and public reports describe the following configuration. Visit the HandX official website →
- 45 tactile points as standard, expandable to 800 points, including dense sensing around the fingertips.
- Approx. 120 g wearable glove, with BLE 5.0 and wired data transmission and at least 8 hours of stated battery life.
- Multimodal synchronization with head/ego cameras, wrist cameras and motion trackers, enabling aligned touch–vision–motion demonstrations.
- Ego + UMI-style collection interfaces for natural task demonstrations outside expensive motion-capture studios.
- Software toolchain covering data annotation, API/SDK access, visualization and device management, with stated compatibility with ROS 2, OpenXR, Unity, Unreal Engine and NVIDIA Isaac Sim.
- Data-service direction: collected tactile experience can be organized as reusable datasets rather than remaining tied to a single robot experiment.
This direction is closely connected to our research on learning from human demonstrations: the goal is to preserve the hidden contact information in actions such as squeezing, sliding, inserting, rotating and handling fragile objects. HandX has also explored extending the same tactile platform toward pulse sensing and other physical-interaction measurements. HandX official website. [Detailed HandX product article] [Wuhan manufacturing and deployment report]
HandX Intelligent Moxibustion Robot
The moxibustion robot is an example of using embodied intelligence to standardize a contact-adjacent human service task. According to official HandX product information and reports from Optics Valley and Hubei media, the system is available in several embodiments, including a fixed-base robotic arm, a wheeled platform and a humanoid configuration. The engineering workflow combines demonstrations from experienced practitioners with perception and robot control:
- Learning from standardized practitioner demonstrations: expert moxibustion videos are converted into reusable motion/knowledge representations.
- 3D body perception and target localization: the system scans the body contour and automatically matches target regions/acupoints.
- Multiple traditional motion patterns: hovering, circular motion and pecking-style moxibustion are reproduced through controlled robot trajectories.
- Closed-loop temperature and distance control: the robot adjusts height as the body moves, with preset safety thresholds.
- Operational automation: automatic ignition, timed operation, one-button stop and smoke-management functions reduce the need for continuous manual handling.
- Digital service record: reported deployments can generate a session report with operation data and thermal comparison information.
The system has been demonstrated in a dedicated Optics Valley experience store and reported in multiple hospital and healthcare-related deployments. These examples show engineering deployment and user interaction; they should not be interpreted as claims of clinical efficacy. [Optics Valley report] [On-site engineering report]
Explore HandX products and company updates on the official website →
Autonomous Robotic Ultrasound
Autonomous robotic ultrasound has been a long-running research and translation direction in our work: the core question is how a robot can acquire the scanning skills of an experienced sonographer and then adapt them safely across patients. Our research combines ultrasound image perception, probe pose/orientation, contact force, imitation learning and online skill adaptation so that scanning can move from teleoperation toward increasingly autonomous execution.
- Learning from sonographers: multimodal demonstrations combine ultrasound images, probe motion and contact force.
- Autonomous skill execution: learned scanning policies predict probe motion and force while searching for clinically useful views.
- Adaptation across patients: online optimization and guided exploration compensate for anatomical and contact-condition differences.
- Image–motion–force closed loop: visual feedback and force control are integrated rather than treating ultrasound as a purely kinematic task.
Representative Publications
- Learning Freehand Ultrasound Through Multimodal Representation and Skill Adaptation — IEEE T-ASE, 2025, 22: 5117–5130.
- Learning Robotic Ultrasound Scanning Skills via Human Demonstrations and Guided Explorations — ROBIO, 2021, 372–378; T.J. Tarn Best Paper Award Finalist.
- Learning Ultrasound Scanning Skills from Human Demonstrations — Science China Information Sciences, 2021.
- Learning to Predict Action Based on B-ultrasound Image Information — ICARM, 2021.
- Learning Robotic Ultrasound Skills from Human Demonstrations — book chapter, 2022.
- Recent advances in robot-assisted echography: combining perception, control and cognition — Cognitive Computation and Systems, 2020, 2(3): 85–92.
Translation at COBOT Technology
The academic work on robotic ultrasound also connects to industrial deployment at COBOT Technology. COBOT's autonomous ultrasound robot uses real-time ultrasound-image perception and robotic servo control to adjust probe pose and contact force, emulate sonographer scanning maneuvers, automatically tune ultrasound parameters, select standard views, perform measurements and generate structured scanning records. COBOT's public company history states that its intelligent ultrasound robot obtained an NMPA Class III medical-device registration certificate in 2022. The product family has since expanded to autonomous, remote and breast-ultrasound robotic systems. [Autonomous ultrasound robot] [Remote ultrasound robot] [Breast ultrasound robot]
Other Application Directions
- Industrial grasping and manufacturing. Vision-guided picking, robust grasping, compliant manipulation, machine-vision inspection and robotized composite manufacturing.
- AI for Science. Embodied manipulation for scientific experiments, especially repeatable wet-lab operations and multimodal scientific-operation data collection.
- Medical and physical-interaction robotics. Other work includes compliant physical interaction and medical robotic manipulation beyond ultrasound.
Earlier application demonstrations remain below as an archive of the progression from industrial grasping to today's embodied-intelligence systems.
Applications of Robust Grasping
- Grasping for Warehouse Logistics (YouTube Video)
- Grasping for Retailing Shop (YouTube Video)
Applications of Compliant Manipulation
- Tactile Exploration (YouTube Video) and Polishing (YouTube Video)
- Medical Application Most of these applications are still in stealth mode. We will try to release some videos in the future.
Applications of Machine Vision
- Deep Learning based Food Inspection
- 3D Object Localization and Reconstruction This is useful for grasping and many other possible applications. We will update this part later.
