Design and Implementation of an Intelligent System for ... · cameras, and mattress pressure...

Computación y Sistemas, Vol. 19, No. 3, 2015, pp. 467–474doi: 10.13053/CyS-19-3-2013

ISSN 2007-9737

Design and Implementation of an Intelligent Systemfor Controlling a Robotic Hospital Bed for Patient Care Assistance

Eduardo Vazquez-Santacruz, William Cruz-Santos, Mariano Gamboa-Zuniga

CGSTIC Cinvestav IPN, Mexico DF,Mexico

[email protected], [email protected], [email protected]

Abstract. In this article we propose an intelligent sys-tem (IS) for automatic movements of a robotic-assistedhospital bed, it is based on posture classification andrecognition using mattress pressure sensors. The pro-posed IS allows to program a sequence of movements ofthe robotic bed that are executed automatically throughelectric actuators in response to the pressure distributionof a patient on the bed. The experimental results showthat programmed movements are useful in preventingbed-sores in patients who stay in bed for extended pe-riods of time.

Keywords. Pattern classification, support vector ma-chines, pressure sensors, intelligent systems, assistiverobotics.

1 Introduction

In recent years, artificial intelligent systems (IS)have been used in several applications such asindustrial control, robot control, traffic surveillance,remote sensing, and speech recognition, amongothers. In particular, IS applied to medical environ-ments have been a challenging task due to highrisk decisions in diagnosis, monitoring, and careof patients. However, in rehabilitation of patientswith limited or restricted mobility, IS have beenused to control the positioning of robotic hospitalbeds to prevent the appearance of bed-pressureulcers, also in activity monitoring and bed-rails con-trol [5, 1, 12, 13]. Most of the systems for automaticcontrol of hospital bed positions are based in thedetection of the posture of patients on the bedusing presence sensors, digital cameras, thermalcameras, and mattress pressure sensors [6, 3, 10].Those systems based on digital and thermal cam-eras have the disadvantages that they are affected

by the environment illumination, temperature con-ditions, and occlusions. On the other hand, mat-tress pressure sensors can be used to detect theposture of patients in any illumination and tem-perature conditions; this is done by performing ananalysis of the pressure distribution obtained fromthe contact of the patient over the pressure sensorarray.

Fig. 1. Rendering sketch of the developed robotic bed

In this research we propose an IS to control thepositions of a robotic hospital bed. The purposeof the proposed system is to prevent accidentswhen the bed is moving and provide a mechanicsto programming a sequence of movements of therobotic bed that are executed automatically. Werecognize the posture of patients using a mattresspressure sensor and performing an analysis andclassification of the pressure distributions. Theproposed IS can be used to assist medical staffand also provides a mechanics for an automaticand non-assisted control of bed positioning.


Eduardo Vázquez-Santacruz, William Cruz-Santos, Mariano Gamboa-Zúñiga468

ISSN 2007-9737

2 Bed System Description

A robotic hospital bed is able to change its positiondepending on the needs of a patient. Figure 1presents a diagram sketch of the developed hos-pital bed at the Hospital Juarez de Mexico, andTable 1 shows all possible positions and transitionswhich it can adopt. The base position is the homeposition, to which all the others can transit, withexception of the sit-to-stand position. The transi-tions of the robotic bed are performed by means ofelectric actuators, and the time it takes to go fromone position to another depends on the weight ofthe patient and the current position of the bed.

Table 1. Positions and transitions of the developedhospital bed that are most used by medical specialists

Number Symbol Name position Transitions

0 Home 1-7

1 Trendelemburg 0, 2-6

2 Inverted 0,1, 3-6Trendelemburg

3 Fowler 0-2, 4-7

4 Lifting legs 0-3, 5-7

5 Orthopedic 0-4, 6,7

6 Cardiac 0-5, 7

7 Sitting 0, 3-6

8 Sit-to-stand 7

9 Lateral rotation 0

2.1 Pressure Sensor Array

The robotic bed uses a flexible pressure sensorarray consisting of 512 units distributed in an areaof 1108×544 mm [11]. Figure 2 shows the pressuresensor system that is composed of two indepen-dent units of 16 × 16 sensors. The pressure sen-sors are based on variable resistive devices withpressure levels ranging between 0 and 4096 andnormal resistance from 1 to 4 kiloohm. All pres-sure measurements were normalized to a range of

Fig. 2. Pressure sensor array of 32× 16 units in an areaof 1108× 554 mm

values from 2500 to 4096 which correspond to themean weight of an adult which ranges between 40and 150 kg. A computer program was developedin order to synchronize the two independent pres-sure array units and obtain pressure maps of size32× 16.

3 The Proposed Intelligent System

The proposed IS was designed to be part of amedical application, performing the classificationof human postures. The recognition of humanpostures is an important task in the monitoring ofpatients and in the control of robotic hospital beds.The IS used in hospital environments must satisfyhigh quality norms and standards, only in such wayan intelligent device is used to assist medical staff.However, there are applications in which low-riskdecisions can be made by an IS. Examples of suchapplications are the IS used to prevent bed-soresin patients with restricted mobility and automaticpositioning of robotic beds.

3.1 Technical Specifications

Figure 3 shows the main blocks of the proposed ISfor posture recognition, based on a low-resolutionpressure sensor array. The description of eachstage is given as follows:

a) Data acquisition: pressure maps are obtainedfrom an array of 31 × 16 pressure sensors.The measured pressure levels range between0 and 4096, where 0 is the maximum pres-sure value and 4096 is the minimum pressurevalue.


Design and Implementation of an Intelligent System for Controlling a Robotic Hospital Bed for Patient Care Assistance 469

ISSN 2007-9737

Fig. 3. Main blocks of the IS for posture recognition

b) Data normalization: in this block, raw data(matrices of size 32 × 16) are normalized ac-cording to three basic considerations. The firstconsideration is based on the mean weight ofa person, which ranges between 40 and 150kg and corresponds to the pressure valuesfrom 2500 to 4096. Pressure values out ofthis range are omitted. The second considera-tion is normalization from the range 2500-4096to the range 0-255 considered as gray-scalevalues. The last consideration is to apply animage interpolation algorithm to raw pressuremaps. After this block, two kinds of pres-sure maps are obtained, raw- and interpolatedpressure maps. A two factor scale was usedfor the interpolation algorithm, that is, the pres-sure maps are scaled to 64× 32 in size.

c) Image descriptors: in this stage, image de-scriptors are computed. An image descriptoris a vector that represents the most relevantfeatures in an image. HOG [4] and SIFT [9] de-scriptors were extracted from the interpolatedpressure maps, which are considered as grayscale images.

d) SVM training: in this block (delimited withdashed lines), a classification model is con-structed using an SVM. A database of pres-sure maps is constructed with four differentrepresentations of the postures, that is, raw-and interpolated pressure maps, HOG andSIFT vector descriptors. Then, for each set

of postures, a classification model is con-structed.

e) SVM classification: the classification is per-formed using the already constructed classi-fication models. In this stage, we make a com-parison of the performance of the classificationresults using each model.

f) Prediction: we consider a majority votingscheme in order to output a prediction ac-cording to the classification results, for eachclassification model.

The output prediction can be used for anothersystem in order to make a decision. In the followingsection, posture classification and recognition aredescribed.

4 Posture Classification

The IS will be able to detect if the patient is in acorrect position in order to perform the requestedtransition and will send a visible alert to preventpossible downfalls. In order to simplify posturerecognition, we consider the sleeping positions ac-cording to the famous study by professor ChrisIdzikowski [7], who classified the sleeping posi-tions into six types: foetus, log, yearner, soldier,freefaller, and starfish. A simplified set of positionswas considered by grouping them into four basicclasses: left lateral decubitus (foetus), right lateraldecubitus (log and yearner), supine (soldier andstarfish), and prone (freefaller). In Figure 4 someexamples of these classes can be seen.

Posture classification was performed using asupport vector machine (SVM) [2], and a classifi-cation model was constructed for posture recog-nition. Four different databases were constructedfor training; this was done in order to comparethe performance of different representations of thepressure maps. The considered databases are thefollowing ones:

1. Raw pressure maps

2. Interpolated pressure maps

3. HOG descriptors



ISSN 2007-9737

Fig. 4. The four basic sleeping positions (image sourceat www.paramedicine.com/)

4. SIFT descriptors

Each considered database has characteristicvector features depending on the size of the pres-sure matrices. The raw pressure map databaseconsists of the pressure matrices obtained fromthe sensor array. The interpolated pressure mapdatabase is an oversampling of the raw database.The HOG and SIFT databases consist of the ex-tracted feature vectors in the interpolated pressurematrices. The motivation of considering four differ-ent databases is to find an efficient representationof pressure maps in the sense that vectors of fea-tures are small in size and representative.

5 Automatic Control of Robotic BedPositions

Sensor DAQ CPU

Fig. 5. Principal elements of a DAQ system

Since robots interpret the real world through de-vices called sensors, it is important to ensure thecommunication status of these devices. Thesesensors are responsible for effectively getting allthe information of the surrounding environment,and the data acquisition system is responsible for

collecting, conditioning, and transferring such in-formation to the control unit. Figure 5 shows ageneral description of this system. These sensorscontact pressure distribution in their applications;the sensor technology enables a two-dimensionalarray of pressure sensing elements in a thin, con-tinuous sheet. By placing sensors on a flat surface,users measure and visualize pressure informationin the interface using this simple but powerful resis-tive sensing approach.

Our development implies an IS which controlsthe robot in order to reduce the risks of operatinga hospital bed with multiple positions. The mainobjective of our system is to prevent accidentswhen the bed is moving, this is done by detectingthe posture of patients using a mattress pressuresensor and by performing an analysis and clas-sification of pressure distributions using an initialtraining set of correct postures for all bed positions.The proposed IS can be used to assist medicalstaff and also provides a mechanics for an au-tomatic and non-assisted control of bed position-ing. Monitoring of leaning people is often usedin a variety of hospital process such as geriatrics,rehabilitation, orthopedics, and is employed evenin psychology and sleep studies [8]. Althoughpatient monitoring is usually an activity of doctorsand nurses, this can be automated to have bettercontrol of patients in any time with human interven-tion only to rate different positions through whichthe patient has passed, the time spent in eachposition, and incurred transitions. Given a highcost of fabrication of array pressure sensors, it isnecessary to develop a method that uses few datato solve the classification issue.

We choose four basic positions according to thefamous study by professor Chris Idzikowski [8],who classified the most common sleeping posi-tions in six classes, that is, foetus, log, yearner,soldier, freefaller, and starfish. These positionscan be grouped in four basic classes: the leftlateral decubitus position or foetus, the right lateraldecubitus position including the log and yearnerpositions, the supine position including the soldierand starfish positions, and the prone position orfreefaller. We use a flexible array pressure sensorwith 448 units distributed in an area of 1860x886mm. We construct a database with pressure levels



ISSN 2007-9737

(scaled to 0-255 range) and apply it in the HOGalgorithm [4]. The database contains samples offour postures. With this database, we can con-struct a classification model using a support vec-tor machine (SVM) classifier. This bed is able toadopt several positions depending on the needs ofa particular patient; it can also be programmed toperform a series of movements within a period oftime. The IS will be able to detect if the patient is ina correct position in order to perform the requestedtransition, and will send a visible alert to preventpossible downfalls. The main stages of the IS forthe posture recognition are as follows: in the initialstage the pressure distributions are obtained fromthe pressure sensor array, in the second and thirdstages an analysis and pre-processing are per-formed, also a feature extraction using HOG andSHIFT descriptors is applied over the pressure dis-tributions considered as gray scale images; in thefourth stage a database of features is constructed,and in the last two stages we construct a model forfeature classification and prediction. Also, we com-pare the results of three classifiers such as Sup-port Vector Machines, Decision Trees, and NaıveBayes Networks. To apply posture recognition, weconsider the basic postures; posture detection isachieved by an analysis of pressure distribution.Thus, a database of features is constructed suchthat it contains the sets of equal size of the ba-sic positions and an appropriate number of theirvariants. Then, the model of the classifier is con-structed in order to make predictions of pressuredistributions that do not belong to the database.

Also, the IS can be used to control the actuatorsof the robotic hospital bed in a semi- or automaticway and to prevent accidents when the bed is mov-ing slowly. Thus, when a bad posture is detected,the IS sends a signal to the actuators either to stopthe transition movement or to return to the previousposition. We use a HOG descriptor to characterizeeach pressure image. Then, for each pressuremap, a vector of gradient directions is constructed.We consider that a histogram of gradient directionscan be used for image classification since smallchanges in pressure images are easily detected aschanges in gradient directions.

6 Experiments and Results

We must consider that the pressure sensor arrayconsists of two independent modules with a reso-lution of 16 × 14 units in an area of 930 × 886 mm.Each pressure sensor has a response between 0and 25 kg/cm2. We develop an interface to syn-chronize the two modules at a frequency of 100measurements per second.

6.1 Pressure Map Databases

Also, we have constructed four different pressuremap data sets for classification: each data setconsists of four classes, for every posture position.For each class, we generated 501 instances. Eachdata set has 2004 instances and is stored as anindividual database.

The data sets are described as follows:

1. Data set interpolated to 640 x 640:



4. Data set in raw format (32 x 14):

Figure 7 shows some examples of interpolatedpressure maps. We normalize the range of valuesof each attribute vector; this is done in order toimprove the performance of the classifier.

Then, we construct four different databases forposture classification: raw data, interpolated rawdata, HOG descriptors, and SIFT descriptors. Thedata sets are described as follows:

1. Raw data: consists of the 32 × 16 pressurematrices, that are represented as vectors with512 entries.

2. Interpolated raw data: consists of interpolatedpressure data using bilinear interpolation. Weuse a factor of 2x for sub-sampling, generatingpressure matrices of 64 × 32 which are repre-sented as vectors with 2048 entries.



ISSN 2007-9737

3. HOG descriptors: we use the 64 × 32 inter-polated pressure matrices in order to com-pute the HOG descriptors using 9 bins and 3windows per box. The descriptors are repre-sented as vectors with 81 entries (see [4] for adetailed description of HOG descriptors).

4. SIFT descriptors: we use the 64 × 32 interpo-lated pressure matrices in order to computethe SIFT descriptors. The size of the his-togram of occurrences was 500. Thus, foreach interpolated pressure matrix, we com-pute a sparse attribute vector with 500 entries(see [9] for a detailed description of SIFT de-scriptors).

The structure of each database consists in atable with n columns or the number of personsand four rows or the positions (see Figure 4), sothat for each person and each position there are501 vectors or variants of the considered position.The number of vectors for each position and for allpersons is 501 · n, and the total number of vectorsfor each database is k = 501 · 4 · n. Table 2shows a comparison of the size of each databasewith respect to the kind of descriptor vectors. Itcan be seen that the most efficient representationis through the HOG descriptors, while the interpo-lated representation is the most inefficient.

Table 2. Database size with respect to the vector de-scriptor and the number k of vectors in each database

Vectors Total size databaseRaw-data 512 · kInterpolated 2048 · kHOG 81 · kSIFT 500 · k

6.2 Software System Interface

A software system interface was developed in or-der to interact with the IS and display informationof each patient and the current state of the roboticbed (see Figure 6). This interface is able to showvaluable information of the patient condition suchas name, sex, age, temperature, weight, date ofadmission, specialty, allergies, patient risk, and the

name of the medical specialist. We have decidedto show this information based on a survey appliedto nurses, residents, and specialists at the HospitalJuarez of Mexico. The interface also shows thecurrent position of the robotic bed and providesa set of commands to manually operate the bedpositions. Additionally, the patient position andpressure distribution are displayed.

Fig. 6. Software system interface of the IS

6.3 Pressure Map Classification

We use the LibLINEAR toolbox to perform theposture classification, and we compare the resultsusing the proposed databases. We found that thepressure maps for persons with the same height donot change significantly in their distribution. There-fore, the number of persons considered in orderto perform our experiments was n = 1. Figure 7shows an example of interpolated pressure mapsfor the four basic positions.

Table 3 shows the results of the classificationusing 10-fold cross-validation. As it can be seen,the raw and interpolated databases have the sameperformance, but with different time to build theclassification model. This is a consequence ofincreasing the number of attributes in the interpo-lation. The HOG and SIFT databases have almostthe same performance, but with a time noticeablydifferent to build the models. It is important tonote that the size of the HOG database is onlyabout 0.158% of the raw database. This can beconsidered as a reduction of dimensionality in theattribute space. An important feature of using im-age descriptors is that the time to construct the



ISSN 2007-9737

Fig. 7. Interpolated pressure maps corresponding to thefour basic posture positions

model of the classifier is slower than when usingraw and interpolated data.

Table 3. Percentage of correctly classified instances andthe time to build the classification model

Database LibLinear Time to build modelRaw data 99.7005% 1.46 secInterpolated 99.7005% 4.3 secHOG 98.7026% 0.14 secSIFT 98.9521% 0.05 sec

Table 4 shows the confusion matrices for theclassification results given in Table 3. It can beobserved that the confusion matrices for the raw-and interpolated data are the same. This meansthat in terms of classifying pressure maps, the in-terpolation does not contribute to the description ofthe input data. The HOG and SIFT matrices havesimilar values and do not coincide in any entry. Itcan be seen that the Prone position was classifiedincorrectly as the Right position in most cases.

These experiments show that the classificationof posture pressure maps can be done with a per-formance of 99.7% using a linear kernel for SVM.The use of image descriptors, so helpful in con-structing and efficiently representing the pressuremaps, reduces the time complexity to build theclassification model.

Table 4. Confusion matrices for the raw- and inter-polated data (top), HOG descriptor (middle), and SIFTdescriptor (bottom)

Supine Prone Right LeftRaw- andinterpolatedSupine 501 0 0 0Prone 0 498 3 0Right 0 1 500 0Left 0 0 0 501HOG-descriptorSupine 495 0 6 0Prone 8 484 7 2Right 1 0 500 0Left 1 0 1 499SIFT-descriptorSupine 497 1 1 2Prone 1 491 9 0Right 3 3 495 0Left 0 1 0 500

7 Conclusions

The intelligent system proposed in this paper rep-resents a strategy to endow robotic assistant abilityto detect risk scenarios for patients. In this case,when the robotic bed is moving, situations of riskcan be generated if patients perform any bodilymovement that is inappropriate to the robotic bedmovement configuration. Our system representsan advance in medical care since through suchkind of intelligent devices patients can be treatedwithout participation of medical personnel who mayrun a risk of an injury when moving patients man-ually. In this case, the robotic bed can be pro-grammed in such a way that the appropriate med-ical personnel could attend other patients whilethe bed automatically applies a certain movementtherapy to patients with only supervised medicalmonitoring by hospital experts.

Acknowledgements

We would like to thank the CGSTIC-Cinvestav IPNteam for making this development possible. We arealso grateful to the medical staff of Hospital Juarezof Mexico for their help.



ISSN 2007-9737

References

1. Arcelus, A., Herry, C. L., Goubran, R. A., Knoefel,F., Sveistrup, H., & Bilodeau, M. (2009). Deter-mination of sit-to-stand transfer duration using bedand floor pressure sequences. IEEE Trans. Biomed.Engineering, Vol. 56, No. 10, pp. 2485–2492.

2. Byun, H. & Lee, S.-W. (2003). A survey on patternrecognition applications of support vector machines.International Journal of Pattern Recognition and Ar-tificial Intelligence, Vol. 17, No. 3, pp. 459–486.

3. Chica, M., Campoy, P., Perez, M. A., Rodrıguez,T., Rodrıguez, R., & Valdemoros, O. (2013). Cor-rigendum to ”real-time recognition of patient inten-tions from sequences of pressure maps using ar-tificial neural networks” [computers in biology andmedicine 42 (2012) 364-375]. Comp. in Bio. andMed., Vol. 43, No. 9, pp. 1302.

4. Dalal, N. & Triggs, B. (2005). Histograms oforiented gradients for human detection. Proceed-ings of the 2005 IEEE Computer Society Confer-ence on Computer Vision and Pattern Recogni-tion (CVPR’05), volume 1, IEEE Computer Society,Washington, DC, USA, pp. 886–893.

5. DeVocht, J. W., Wilder, D. G., Bandstra, E. R.,& Spratt, K. F. (2006). Biomechanical evaluationof four different mattresses. Applied Ergonomics,Vol. 37, No. 3, pp. 297 – 304.

6. Grimm, R., Bauer, S., Sukkau, J., Hornegger, J.,& Greiner, G. (2012). Markerless estimation ofpatient orientation, posture and pose using rangeand pressure imaging. Int. J. Computer AssistedRadiology and Surgery, Vol. 7, No. 6, pp. 921–929.

7. Hao, J., Jayachandran, M., Kng, P., Foo, S.,Aung Aung, P., & Cai, Z. (2010). Fbg-based smartbed system for healthcare applications. Frontiers ofOptoelectronics in China, Vol. 3, No. 1, pp. 78–83.

8. Idzikowski, C. (2010). Learn to Sleep Well.

9. Lowe, D. G. (1999). Object recognition from localscale-invariant features. ICCV, pp. 1150–1157.

10. Seo, K.-H., Choi, T.-Y., & Oh, C. (2011). De-velopment of a robotic system for the bed-ridden.Mechatronics, Vol. 21, No. 1, pp. 227 – 238.

11. Textiles, S. T. S. (2013). Webpage:.http://www.sensingtex.com/. [Online; accessed12-Dicember-2013].

12. Townsend, D. I., Holtzman, M., Goubran, R. A.,Frize, M., & Knoefel, F. (2011). Relative threshold-ing with under-mattress pressure sensors to detectcentral apnea. IEEE T. Instrumentation and Mea-surement, Vol. 60, No. 10, pp. 3281–3289.

13. Vazquez-Santacruz, E. & Gamboa-Zuniga, M.(2013). A diagnosis methodology for assistive tech-nology development. 10th International Conferenceon Electrical Engineering, Computing Science andAutomatic Control (CCE), pp. 163–169.

Eduardo Vazquez-Santacruz holds Ph.D. in Engi-neering Science; he is Coordinator of Automationof the CGSTIC, Cinvestav-IPN.

William Cruz-Santos holds Ph.D. in EngineeringScience; currently he is Professor of Computer En-gineering, CU-UAEM Zumpango, Zumpango, Mex-ico and cooperates with CGSTIC, Cinvestav-IPN.

Mariano Gamboa-Zuniga has Ph.D. in Engineer-ing Science; he is General Services Coordinatorof Information Technology and Communications atCGSTIC, Cinvestav-IPN.

Article received on 02/12/2014; accepted on 21/04/2015.Corresponding author is Eduardo Vazquez-Santacruz.

Design and Implementation of an Intelligent System for ... · cameras, and mattress pressure...

Documents

Transcript of Design and Implementation of an Intelligent System for ... · cameras, and mattress pressure...