Digital Handwriting with a Finger or a Stylus: a ... Handwriting with a Finger or a Stylus: a...

Click here to load reader

download Digital Handwriting with a Finger or a Stylus: a ...  Handwriting with a Finger or a Stylus: a Biomechanical Comparison ... chanical model of the hand,

of 16

  • date post

    22-Apr-2018
  • Category

    Documents

  • view

    216
  • download

    4

Embed Size (px)

Transcript of Digital Handwriting with a Finger or a Stylus: a ... Handwriting with a Finger or a Stylus: a...

  • 1

    Digital Handwriting with a Finger or a Stylus:a Biomechanical Comparison

    Domenico Prattichizzo, Senior Member, IEEE, Leonardo Meli, Student Member, IEEE,and Monica Malvezzi, Member, IEEE

    AbstractIn this paper we present a study concerning the human hand during digital handwriting on a tablet. Two different cases areconsidered: writing with the finger, and writing with the stylus. We chose an approach based on the biomechanics of the human handto compare the two different input methods. Performance is evaluated using metrics originally introduced and developed in robotics,such as the manipulability indexes. Analytical results assess that writing with the finger is more suitable for performing large, but notvery accurate motions, while writing with the stylus leads to a higher precision and more isotropic motion performance. We then carriedout two experiments of digital handwriting to support the approach and contextualize the results.

    Index TermsH.5.2 User Interfaces: Evaluation/methodology; H.5.2 User Interfaces: Input devices and strategies; handwriting; tablet;biomechanics; kinematics; hand model; stylus; touch.

    F

    1 INTRODUCTIONThe hand is the main tool adopted by humans to physi-cally interact with the external environment. The humanhand is highly versatile and easily adaptable to a varietyof manipulation tasks, exposing flexible solutions to theneeds of control [1]. In daily life, humans beings are,apparently without effort, able to generate complex andelegant movements of the hand and fingers, such astyping on keyboards, playing a musical instrument, orwriting.

    In this paper we focus on the analysis of human handmovements during handwriting tasks, a subject whichhas been studied for many decades.

    In 1918 M. Freeman published a book [2] about thefactors affecting performance in handwriting, using thesocalled motion pictures of the act of handwriting. Therole of hand biomechanical parameters in the handwrit-ing process has been introduced by Schomaker and Pla-mondon in [3], in which they investigated the correlationbetween the force exerted with the pen and differentkinematic parameters. In [4] a handwriting trajectorygeneration model, based on kinematics and dynamicoptimization, was formulated and tested.

    Hermsdorfer et al. [5] exploited information on theforces applied by the fingers on a pen, and the rela-

    The research leading to these results has received funding from the Euro-pean Union Seventh Framework Programme FP7/2007-2013 under grantagreement 601165 of the project WEARHAP, and by the Italian Ministry ofEducation, Universities and Research, Futuro in Ricerca 2012 Programme,with the MODELACT project (Code RBFR12C608).

    D. Prattichizzo, L. Meli, and M. Malvezzi are with the Departmentof Information Engineering and Mathematics, University of Siena, ViaRoma 56, 53100 Siena, Italy.

    D. Prattichizzo, and L. Meli are also with the Department of AdvancedRobotics, Istituto Italiano di Tecnologia, via Morego 30, 16163 Genova,Italy.

    (a) (b)

    Fig. 1. The benchmark: writing on a tablet with the fingerand with the stylus. Which solution is better?

    tive grip kinetics, to inform diagnosis and treatment ofhandwriting disorders. In [6] a device called the KineticPen was presented. It is capable of measuring the six-component force and torque that each of the individualcontacts applies to the pen while writing. Shim et al.[7] adopted this device to investigate synergistic actionsof hand-pen contact forces during circle drawing tasks.Kushki et al. [8] analyzed how some handwriting ki-netic and kinematic parameters vary during a prolongedwriting task, in children with and without handwritingdeficiency. Falk et al. [9] developed an instrumented writ-ing tool to measure kinematic and temporal informationduring a writing task, as well as forces exerted on thewriting surface and on the barrel of the pen. They usedthis experimental setup to quantitatively measure thegrip force variability, looking for a correlation with hand-writing legibility, form, and strokes. In all these studiesan important role is played by the main biomechanicalproperties of the hand, and by the grip force between thehand and the pen. This motivates the analysis approachproposed in this paper.

    Due to the rapid development and diffusion of smart-

  • 2

    phones and tablets, the handwriting problem, and inparticular the interaction with the touchscreen throughthe finger or the stylus, is becoming more and moreimportant (see Fig. 1) [10]. The direct use of the fingeris more practical and typically allows one to save time,especially for shortterm interactions, e.g., verifying ameeting time, navigating a map, answering a call, orcontrolling a media player. On the other hand, stylusbased input tools allow one to perform more complexoperations and operate with smaller targets. In general,the finger interaction is less accurate than the stylus one[11].

    One of the issues in writing with the finger is theambiguous selection point created on the surface bythe fingers contact area, together with the occlusion ofthe target [12]. Although there are several algorithmsapplied in gesture/finger detection allowing one to ob-tain a stable center with purely continuous movement,the lack of visual feedback can only be overcome withpractice.

    Using a stylus mitigates such problems: the contactpatch is typically smaller and the vertical offset betweenthe users hand and the screen reduces occlusion issues.The selection points and the target trajectory can then bedefined more clearly.

    As a particular combination of the two interactionmethods, a bimanual graphical editor application waspresented in [13]. The cooperation between touch andpen interaction was also the basic idea of the interactiveplatform developed in [14].

    One of the problems in the assessment of handwritingis the availability of objective indexes to evaluate hand-writing performance. In [15] the problem of defining ahandwriting legibility index was investigated. In partic-ular, an experiment was performed on texts produced bychildren. The subjective rating of handwriting legibility,evaluated by some teachers, was related to an objectiveassessment based on a check list including 20 items.

    In this work we contribute to the studies on hand-writing presenting for the first time, at the best ofour knowledge, a methodological approach based onthe biomechanics of the human hand to compare twodifferent input methods, i.e., the finger and the stylus,in digital handwriting tasks. Our analysis differs fromother research presented in the literature. For instance,in the work by Zabramski [16], the author comparesthe accuracy and efficiency of different input methodsin a line-tracing task without taking into account thehuman hand structure and kinematics. Although data-driven approaches are still crucial to address this typeof problem, we believe that the kinematic of the handcan improve the comprehension of digital handwriting.

    Some studies in the literature investigate the humanrobot analogy in dexterity tasks, such as handwriting.In [17] the motion analysis of a redundant anthropo-morphic arm was investigated during a writing task. In[18] the authors focused on the humanmachine analogyin handwriting, including an evaluation of legibility.

    In [19] the humanrobot analogy was further investi-gated. A mathematical model of fatigue progress and analgorithm for humanlike redundancy resolution werepresented.

    In this paper we present the analysis of handwritingon a touchscreen with numerical and analytic toolswidespread in robotics; in particular we investigate ma-nipulability properties. Generally speaking, a manipula-bility index is a number expressing the ratio betweena measure of performance, e.g., displacement, velocity,force in the task space, and a measure of effort in theinput/joint space [20], [21]. The manipulability analysismainly consists in defining the directions in the taskspace, where the index is maximized or minimized. Inthis paper performance of two input methods in digitalhandwriting, i.e., finger and stylus, is evaluated andcompared in terms of manipulability indexes in the taskspace [22], [23].

    Beside the mathematical analysis based on a biome-chanical model of the hand, two experiments are pre-sented, in which subjects were asked to write on atouchscreen using either their index finger, or a stylus.The first experiment introduces the problem we modeledin this manuscript: some participants were asked towrite a short sentence on a tablet and, for each sample,we measured the writing bounding box size, the pathlength, and the wrist motion. In the second experimentdigital handwriting task performance was evaluated interms of completion time and accuracy in retracing agiven shape.

    The paper is organized as follows: Section 2 introducesthe problem presenting an experiment in which a hand-writing task is performed on a tablet. Section 3 describesthe kinematic model of the human hand, the joints, andthe link geometries and parameters. Section 4 summa-rizes the main mathematical relationships necessary toevaluate hand performance, and specifically Section 4.4covers the manipulability indexes. Section 5 presentsthe numerical simulations carried out to analyze thehand parameters during handwriting tasks. In Section 6another experiment is explained and commented, inwhich subjects were asked to retrace a given shape ona touchscreen. Finally, Section 7 addresses concludingremarks.

    2 EXPERIMENT: FREE HANDWRITINGIn order to introduce the problem, we present the firstexperiment in which participants were asked to write ashort sentence on a tablet using both the fingertip anda stylus. Fig. 2 shows the experimental setup, composedof an iPad (Apple Inc., Cupertino, California) tablet witha screen of 9.7 diagonal and a resolution of 1024 by