On the Use of Natural User Interfaces in Physical Rehabilitation: A Web-based Application for Patients with Hip Prosthesis

Yves Rybarczyk, Clément Cointe, Tiago Gonçalves, Vitor Minhoto, Jan Kleine Deters, Santiago Villarreal, Arián Aladro Gonzalo, Jonathan Baldeón, Danilo Esparza



This study aims to develop a telemedicine platform for self-motor rehabilitation and remote monitoring by health professionals, in order to enhance recovery in patients after hip replacement. The implementation of such a technology is justified by medical (improvement of the recovery process by the possibility to perform rehabilitation exercises more frequently), economic (reduction of the number of medical appointments and the time patients spend at the hospital), mobility (diminution of the transportation to and from the hospital) and ethics (healthcare democratization and increased empowerment of the patient) purposes. The Kinect camera is used as a Natural User Interface to capture the physical exercises performed at home by the patients. The quality of the movement is evaluated in real-time by an assessment module implemented according to a Hidden-Markov Model approach. The results show a high accuracy in the evaluation of the movements (92% of correct classification). Finally, the usability of the platform is tested through the System Usability Scale (SUS). The overall SUS score is 81 out of 100, which suggests a good usability of the Web application. Further work will focus on the development of additional functionalities and an evaluation of the impact of the platform on the recovery process.


Web development; Artificial Intelligence; Usability; Therapeutic Application

Full Text:



Antón, D., Go-i, A., & Illarramendi, A. (2015). Exercise recognition for Kinect-based telerehabilitation. Methods of Information in Medicine, (54), 145-155.

Brook, G., Barry, G., Jackson, D., Mhiripiri, D., Olivier, P., & Rochester, L. (2014). Accuracy of the Microsoft Kinect sensor for measuring movement in people with Parkinson's disease. Gait & Posture, (39), 1062-1068.

Brooke, J. (1996). SUS – A quick and dirty usability scale. Usability Evaluation in Industry. London: Taylor & Francis, pp. 189-194.

Carrasco, G., Rybarczyk, Y., Cardoso, T., & Martins, I.P. (2013). A serious game for multimodal training of physician novices. Proceedings of the 6th International Conference of Education, Research and Innovation. Seville, Spain.

Coelho, T., De Oliveira, R., Cardoso, T., Rosas, J., & Rybarczyk, Y. (2014). Body ownership of virtual avatars: an affordance approach of telepresence. Innovative and Creative Development in Multimodal Interaction Systems. Heidelberg: Springer, pp. 3-19.


Gameiro, J., Cardoso, T., & Rybarczyk, Y. (2014). Kinect-Sign: teaching sign language to listeners through a game. Innovative and Creative Development in Multimodal Interaction Systems. Heidelberg: Springer, pp. 141-159.


Heiberg, K., & Figved, W. (2016). Physical functioning and prediction of physical activity after total hip arthroplasty: Five-year follow up of a randomized controlled trial. Arthritis Care & Research, (68), 454-462.


Holden, M.K., Dyar, T.A., & Dayan-Cimadoro, L. (2007). Telerehabilitation using a virtual environment improves upper extremity function in patients with stroke. IEEE Transactions on Neural Systems and Rehabilitation Engineering, (15), 36-42.


Jäppinen, A.M., Hämäläinen, H., Kettunen, T., & Piirainen, A. (2016). Postoperative patient education in physiotherapy after hip arthroplasty: Patients' perspective. Musculoskeletal Care. DOI: 10.1002/msc.1153.


Jogi, P., Zecevic, A., Overend, T., Spaulding, S., & Kramer, J. (2016). Force-plate analyses of balance following a balance exercise program during acute post-operative phase in individuals with total hip and knee arthroplasty: a randomized clinical trial. SAGE Open Medicine, (4), 1-9.


Kleine Deters, J., & Rybarczyk, Y. (2018). Hidden Markov Model approach for the assessment of tele-rehabilitation exercises. International Journal of Artificial Intelligence, (16), 1-19.

Mahapatra, A., Tusar, K.M., Pankaj, K.S., & Banshidhar, M. (2014). Human recognition system for outdoor videos using Hidden Markov Model. International Journal of Electronics and Communications, (68), 227-236.


Ni, S., Jiang, W., Guo, L., Jin, Y., Jiang, T., Zhao, Y., & Zhao, J. (2015). Cryotherapy on postoperative rehabilitation of joint arthroplasty. Knee Surgery, Sports Traumatology, Arthroscopy, (23), 3354-3361.


Papadopoulos, G.T., Axenopoulos, A., & Daras, P. (2014). Real-time skeleton-tracking-based human action recognition using Kinect data. Lecture Notes in Computer Science: MultiMedia Modeling, (8325). Heidelberg: Springer.


Peter, W., Nelissen, R., & Vlieland, T. (2014). Guideline recommendations for post-acute postoperative physiotherapy in total hip and knee arthroplasty: are they used in daily clinical practice? Musculoskeletal Care, (12), 125-131.


Plantard, P., Shum, H., Le Pierres, A.S., & Multon, F. (2016). Validation of an ergonomic assessment method using Kinect data in real workplace conditions. Applied Ergonomics, (65), 562-569.

Rybarczyk, Y., Kleine Deters, J., Cointe, C., & Esparza, D. (2018). Smart Web-based platform to support physical rehabilitation. Sensors, (18), 1344.


Rybarczyk, Y., & Gonçalves, M.J. (2016). WebLisling: a web-based therapeutic platform for the rehabilitation of aphasic patients. IEEE Latin America Transactions, (14), 3921-3927.


Rybarczyk, Y., & Vernay, D. (2016). Educative therapeutic tool to promote the empowerment of disabled people. IEEE Latin America Transactions, (14), 3410-3417.


Sauro, J. (2011). A Practical Guide to the System Usability Scale: Background, Benchmarks and Practices. Denver: CreateSpace.

Schega, L., Bertram, D., Fölsch, C., Hamacher, D., & Hamacher, D. (2014). The influence of visual feedback on the mental representation of gait in patients with THR: a new approach for an experimental rehabilitation strategy. Applied Psychophysiology and Biofeedback, (39), 37-43.


Tsukagoshi, R., Tateuchi, H., Fukumoto, Y., Ibuki, S., Akiyama, H., So, K., Kuroda, Y., Okumura, H., & Ichihashi, N. (2014). Functional performance of female patients more than 6 months after total hip arthroplasty shows greater improvement with weight-bearing exercise than with non-weight-bearing exercise. Randomized controlled trial. European Journal of Physical and Rehabilitation Medicine, (50), 665-675.

DOI: http://dx.doi.org/10.7559/citarj.v10i2.402

Cited by


  • There are currently no refbacks.

Journal of Science and Technology of the Arts
Revista de Ciência e Tecnologia das Artes
ISSN: 1646-9798
e-ISSN: 2183-0088
Portuguese Catholic University | Porto


Esta revista científica é financiada por Fundos Nacionais através da FCT – Fundação para a Ciência e a Tecnologia

 Governo da República Portuguesa