Revolutionizing Child Rehabilitation with Real-Time AI
- * latency under 100ms in gaming applications is considered real-time
At a glance
Jakub conducted the process of data collection & processing, as well as creating and testing ML models used in the Phibox product.
IMPACT
Jakub’s work on the project enabled the implementation of key functionality in the developed medical product: real-time control of video games using ML models.
KEY SERVICES
INDUSTRY
Healthcare
KEY TECHNOLOGIES & PLATFORMS
Philabs Introduction
Philabs start-up is leading a new era of healthcare innovation with Phibox, a state-of-the-art product designed to transform rehabilitation and neurotherapy through interactive, AI-driven experiences. Phibox harnesses the power of Machine Learning, Computer Vision, and Edge Computing to provide real-time, data-driven control of therapeutic games, offering a seamless integration of medical science and technology.
To bring this vision to life, Philabs entrusted me with the crucial tasks of data collection, processing, and the development of sophisticated ML models. These models are at the heart of Phibox, enabling it to deliver personalized, adaptive therapies that respond to the unique needs of each patient in real time.
It was a great pleasure to work with Kuba. He is one of the best specialists in his field. His extensive knowledge greatly benefited the entire team in developing efficient solutions and advancing the Phibox product.
Gamifying Children’s Posture Correction
According to data from the Institute of Mother and Child, the prevalence of posture defects in children is as high as 90%. Scientific studies and observations by Phibox physiotherapists reveal that the primary causes are low levels of physical activity, falling below the recommended 6-15 hours, and incorrect posture during daily activities.
To address this issue, Philabs decided to create a child-friendly solution by incorporating gamification. They developed video games that children can control by performing rehabilitation exercises in front of a TV screen.
Need for my Services
The task was entrusted to me: to implement a compact image processing model capable of running directly on the device, while generating real-time control signals for the game. Given the tight deployment timeline and the start-up environment, it was essential to train the model on a small dataset and keep costs low. The model needed to not only recognize which rehabilitation exercises the child was performing but also assess if they were being executed correctly.
Project Execution
The project followed a start-up model: with short cycles, continuous improvement, and frequent releases. I prioritized the most effective solutions aligned with the goals Philabs aimed to achieve.
| Philabs challenge | 1wayticket.tech solution |
|---|---|
| Given the startup nature of Philabs, the project required an agile approach and a rapid product launch. The short implementation timeline and limited budget meant that working with a large dataset was not feasible. Additionally, building a fully customized model from scratch would take too much time and resources, making it difficult to meet the tight deadlines. | To address these constraints, I decided to implement a transfer learning method using models with pre-trained weights. This approach directly mitigated the challenge of needing a large dataset, as transfer learning allows the use of smaller datasets by leveraging the knowledge captured in pre-trained models. By fine-tuning these models on our limited data, we could efficiently build a functional solution without the need for extensive data collection or expensive computational resources. Furthermore, we developed initial models on partial datasets, enabling us to quickly test the solution’s functionality and playability, ensuring rapid iteration and feedback within the agile framework. |
| Philabs aimed to create an affordable home device using a standard 2D camera, like the one in a phone, capable of analyzing the complex 3D kinematic chain of a child’s body. | To address this, I implemented models that processed 2D images for faster data analysis, while incorporating spatial data during training. This approach allowed the models to capture the complexity of the body’s movements while maintaining efficiency and affordability, ensuring the device could operate effectively in a home environment without the need for high-end hardware. |
| Phibox aimed not only to recognize typical rehabilitation movements like jumps, squats, and swings, as other rehabilitation games on the market do, but also to serve as a fully medical tool that identifies mistakes in a child’s movements and provides corrective guidance. | To address this challenge, I designed an architecture of 10 interconnected models, each dedicated to analyzing a specific type of movement. Together, these models not only classified the exercise but also worked collaboratively to detect errors in the performance. This approach ensured that Phibox could provide both accurate exercise tracking and feedback on mistakes, meeting the medical requirements of the tool. |
| In medical projects, one of the biggest challenges is mapping the expert knowledge of rehabilitation specialists onto the functionalities of an AI model. Excellent communication between development and medical teams was crucial in Phibox project. | To meet this challenge, I led regular meetings and daily in-office collaboration with the physiotherapists. By breaking down complex AI and Neural Networks concepts into simple, understandable terms, I facilitated productive communication, ensuring their input was accurately reflected in both the data collection and model validation processes. |
| Although data was collected in controlled laboratory conditions, Phibox needed to function accurately in unpredictable home environments, where factors like device position, perspective, lighting, and surroundings could vary significantly. | To bridge this gap, I developed advanced image augmentation techniques that simulated the variability of home environments. These methods accounted for changes in the device’s position, perspective, and lighting conditions, ensuring the model could adapt to and perform reliably in real-world, unpredictable settings. This allowed the device to maintain accuracy outside of controlled lab conditions, meeting the requirement for home use. |
Jacob is an accomplished, well organised engineer. His knowledge of data science and ML is not only impressive, but also helpful in co-creating efficient and well-planned pipelines and development solutions.
Outcomes and benefits
The outcome of the project is a functional medical device that supports the rehabilitation process and encourages children’s engagement in physical exercises. Given the scale of posture problems among children, this is a groundbreaking approach, packed into a compact device. It reflects the synergy between medicine and ML. The collaboration of many creative, passionate individuals led to the development of a device that sheds a completely new light on children’s rehabilitation.
Jakub is a great teammate and skilled engineer, constantly looking for ways to improve. I very much value his proactivity and people skills, it enables smooth cooperation without communication errors