Mechanical and civil engineering students learnfundamental engineering concepts through acourse in statics. Statics problems usually require

the student to draw planar truss and other free-body dia-grams (FBDs).

In civil and mechanical engineering, visual and spatialthoughts are essential for the correct absorption of funda-mental concepts. By relying on visual aids as opposed to onlyusing the verbal channel of acquiring knowledge, the cog-nitive load becomes lighter and learning becomes moreeffective (Sweller 1994). Furthermore, learners should active-ly engage in the process in order to reach the highest levelsof comprehension. The task of freehand sketching encour-ages and demands that learners are actively constructingtheir knowledge. In the civil and mechanical engineeringdomain, sketched planar truss and other free-body dia-grams are particularly helpful to the understanding ofstatics concepts. This type of “forced active processing”ensures attention to visual information and helps learnersattend to key elements of the visual system (Kozma 1994).

A planar truss diagram is simply a two-dimensional rep-resentation of a structure that is constructed from physicalbeams and joints. Joints, also referred to as nodes, are locat-ed at the intersection of two or more beams and are thelocation where external forces may act upon the object.

Articles

WINTER 2012 55Copyright © 2013, Association for the Advancement of Artificial Intelligence. All rights reserved. ISSN 0738-4602

Mechanix:A Sketch-Based Tutoringand Grading System for

Free-Body Diagrams

Stephanie Valentine, Francisco Vides, George Lucchese, David Turner,Hong-hoe Kim, Wenzhe Li, Julie Linsey, Tracy Hammond

n Introductory engineering courses withinlarge universities often have annual enroll-ments that can reach up to a thousand stu-dents. In this article, we introduceMechanix, a sketch-based deployed tutoringsystem for engineering students enrolled instatics courses. Our system not only allowsstudents to enter planar truss and free- bodydiagrams into the system, just as theywould with pencil and paper, but our sys-tem also checks the student’s work against ahand-drawn answer entered by theinstructor, and then returns immediateand detailed feedback to the student. Stu-dents are allowed to correct any errors intheir work and resubmit until the entirecontent is correct and thus all of the objec-tives are learned. Since Mechanix facilitatesthe grading and feedback processes,instructors are now able to assign morefree- response questions, increasing teacher’sknowledge of student comprehension. Fur-thermore, the iterative correction processallows students to learn during a test,rather than simply display memorizedinformation.

Furthermore, these external forces create mem-ber forces within each individual beam by ten-sion or compression of the beam. Trusses areused as supports in many structures such asbridges, houses, and other buildings. An excel-lent foundation of how to construct a truss iscritical for a student’s success as an engineer inthe future. While a truss is a type of free-bodydiagram, other nontruss free-body diagramscan be used to analyze all of the internal andexternal forces acting on a general object of anyshape.

In the task of learning how to construct acorrectly structured truss the learner mustreceive appropriate feedback on what he or sheis doing in order to correctly assess the effec-tiveness of the learning process. Feedback helpslearners identify misconceptions and guidesthe learner to a more accurate conception ofthe knowledge (Bangert-Drowns et al. 1991). Itis fundamental that this feedback is both con-cordant with the student’s learning stage andalso that it is given in a correctly timed manner.If feedback is delayed for too long, the overalllearning process becomes degraded, which givesus a preference for immediate feedback.

While immediate feedback is ideal, the largeclass sizes of introductory engineering courses(excess of 100 students per instructor)preventdetailed feedback on hand-drawn solutionsfrom being given often because of the timecommitment involved. In these courses, stu-dents are likely to be assigned only one or twohand-drawn assignments a semester. To com-bat these time constraints in testing environ-ments, multiple choice questions are the pri-mary evaluation method.

To stimulate the educational value of thesecourses, the need for a better method of grad-ing hand-drawn truss diagrams is necessary.Hand-drawn homework problems, such astruss diagrams, afford themselves the use ofsketch recognition as a solution.

Here, we introduce Mechanix, a sketch-recognition system that can recognize, correct,and provide real-time feedback on a student’shand-drawn truss diagram that is checkedagainst a single instructor-entered key sketch.The aim of our deployed system is to advancethe artificial intelligence of automatedmechanical and civil engineering instruction,such that the automated instruction emulatesthe expert performance achieved by humaninstructors.

Prior WorkSketch Worksheets (Yin et al. 2010) is a sketch-based system that allows instructors to create non-

domain-specific sketched worksheets for studentsto complete. The system generates “facts” based ona spatial analysis of the sketch. Instructors choosewhich facts are important and define what adviceshould be given to the student. In order to do this,instructors must understand a somewhat compli-cated language based on spatial relationships. BothSketch Worksheets and Mechanix allow instructor-sketched solutions and provide incremental feed-back, but Mechanix requires much less of a learn-ing curve. Additionally, because Mechanix is aspecialized system, it understands the diagrams itrecognizes and can even solve students’ trusseswithout an instructor-provided sketch.

Some educational software packages specializedto the statics domain already act as tutors for stu-dents that are learning statics. These packagesinclude WinTruss (Callahan et al. 1988), McGrawHill Connect Engineering, and Bridge Architect,which are respectively stand-alone, web-based, andmobile phone engineering tutoring applications.Similarly to our deployed system they providestep-by-step instructions and provide some form offeedback on the input. However, none of themoffer an opportunity for students to solve the com-plete problem by themselves; they all provide apartially completed solution and give overall feed-back as to whether the student solution is corrector not. Additionally, none of these allow for hand-drawn input.

Two other related systems are the Andes physicstutoring system (Vanlehn et al. 2005) and the FreeBody Diagram Assistant (Roselli et al. 2003), whichallow students to create electronic solutions forhomework assignments. Both systems weredesigned as alternatives to pen-and-paper home-work assignments to make classroom adoptioneasy for professors. Additionally, both consist of adesign palette where users can pick pieces and usea mouse to drag them on the workspace in order tobuild their solution. While this is an improvementto providing partially completed solutions to thestudent (as in the methods from the previous para-graph), the deployed system described in this arti-cle further improves on this by allowing studentsto use a stylus to hand-sketch their input. We pro-vide a unique opportunity to use a more physicaland traditional method of solving the problems,while assessing the correctness of the free-bodydiagram. In this way, students can acquire impor-tant skills that might prove to be valuable in theirfuture careers even when they are away from acomputer.

Some other systems also tackle the truss andfree-body diagram teaching problem, allowing forfreehand sketching as input. Newton’s Pen (Lee etal. 2008) is a pen-based tutoring system for staticsthat runs on the FLY pentop computer (based onthe Anoto digital pen-and-paper technology). The

Articles

56 AI MAGAZINE

recognition capability of Newton’s Pen is limitedby the hardware in the FLY pentop computer.Additionally, in order to facilitate recognition,Newton’s Pen constrains the user to draw free-bodydiagram components in a very specific order.Unlike Newton’s Pen, Mechanix offers truly freesketching in that the recognition is not dependenton the order in which the student draws the com-ponents of the solution.

To achieve this goal we rely on state-of-the-arttechniques of sketch recognition. The most promi-nent research in this field can be categorized intothree categories: gesture recognition (Rubine 1991;Wobbrock, Wilson, and Li 2007), vision-basedrecognition (Kara and Stahovich 2005; Miller, Mat-sakis, and Viola 2000), and geometric recognition(Hammond and Davis 2005). Geometric recogni-tion has been explored and researched in variousdistinct domains. In this kind of recognition thereis usually a bottom-up approach, and after prepro-cessing, there is a low-level recognizer that canidentify primitive shapes such as lines, circles, orarcs. On top of primitive recognition there is ahigh-level recognizer that can use a set of con-straints to determine whether the basic shapes andthe relationships between them compose a morecomplex shape. This approach has been used suc-cessfully in domains such as military course ofaction diagrams (Johnston and Hammond 2010)and circuit diagrams (Alvarado and Davis 2004,Gennari et al. 2005). In all cases, a combination ofprimitive shapes under a set of known constraintsresults in the production of higher-level shapesthat comply with certain standards, yet allow freesketching.

In our case, we rely on a powerful low-level rec-ognizer called PaleoSketch (Paulson and Ham-mond 2008), or Paleo, which identifies primitiveshapes such as lines, arcs, ellipses, or spirals. Paleointegrates several techniques such as corner find-ing and geometric perception to perform a series ofprerecognitions over the supported shapes. It thenuses a novel ranking algorithm to determine whichof these shapes has a better fit. Although the cur-rent version of Paleo supports more than 10 basicshapes, we mostly rely on the recognition of lines,polylines, and dots for Mechanix. Paleo has areported accuracy of more than 98 percent (Paul-son and Hammond 2008).

ImplementationThe Mechanix system provides a clean and intu-itive interface in which students and instructorscan interact. Figure 1 shows the Mechanix inter-face.

The text at the top center of the interface pro-vides the description of the problem to be solved.

The problem can be accompanied by an image thatallows clicking to zoom in for more detail. The toolpanel lies just below the problem description andcontains useful functions to help edit the sketches.The buttons (from left to right) are undo, redo,clear, open, save, and erase. The checklist on theleft edge of the interface provides step-by-stepguides to assist students to finish the problem. Apullout notepad exists on the tab behind thechecklist. Students can use this panel to makenotes when they are working on any problem. Thedrawing panel is the center of the interface onwhich students draw their diagrams. Each pen-down motion is captured and processed by oursoftware. Students use the equation panel (locatedbelow the drawing panel) to enter the requiredequations and values of reaction forces. Then thesystem compares these inputs with correctanswers.

The system gives feedback by showing a helpfuland informative message in the orange bar belowthe tool panel. In figure 1, the message shows thatwe have forgotten to draw an axis, which is neces-sary to process the force directions. These messagesare an intuitive guide for students. The submit but-ton is the green checkmark in the top right cornerof the interface. Whenever students want to checktheir solutions, they can simply click the submitbutton to see whether their answers are correct.

Interface ModesThere are two distinct modes of interaction inMechanix, the student mode and the instruc-tor mode. Both modes provide a workspace todraw truss diagrams and enter metadata rele-vant to the problem. The student mode is theinterface that a student sees when working ona homework assignment and was described indetail above.

The instructor mode allows instructors to cre-ate new assignments and corresponding setsof questions in a simple fashion. Instructors canadd the problem statement in the form of textand explanatory images, and by using a similarinterface to the one provided for students,they can sketch a solution diagram to eachproblem. This sketched solution is interpretedby the system and will be used to check thestudent-drawn sketches. Instructors areresponsible for labeling nodes and forces.Mechanix then generates the appropriate sys-tem of equations and values for reaction forces.Certain additional required values can be inputby the instructor, such as the factor of safety.After the instructor has finished creating aproblem he or she can save it to the centralserver so students can access it. This same inter-face is also used to review detailed informationabout each student submission.

Articles

WINTER 2012 57

Interaction MethodsOne great advantage of a sketch-based systemis that it allows users to continually modify oredit their drawings as they would on pen andpaper. The current Mechanix system providessuch functionality through our round menu,buttons, and freehand erasure.

When we hover the mouse or pen on top ofa stroke or recognized shape for a small timeinterval, the round menu will automaticallyappear. In figure 1, the round menu is shownproviding options for an axis. Using this roundmenu, we can change the color of a shape,delete the highlighted shape, or label theshape.

There are several buttons on the tool panel,as we have shown in figure 1. Users can usethese buttons to explicitly undo or redo or clearor erase their drawings.

We allow erasure by means of scribblingstrokes, which can be faster and more naturalfor interaction than explicitly using buttons or

menus. Keeping this purpose in mind, we inte-grated scribble gesture into the Mechanix sys-tem. Figure 2 shows how we can use the scrib-ble gesture to remove either a complete shape orpart of a shape. We recognize scribble shapes ascombinations of strokes in which time intervalsare within 400 milliseconds. If a scribble strokeintersects most of a shape, the scribble erases theentire shape. On the other hand, if the strokeintersects only one line of the shape, then thescribble deletes that single line. In the case ofamorphous closed shapes, if the scribble islocalized on the stroke, it deletes only thatpart.

We also allow certain interactions defined bythe tapping of the pen or clicking of themouse. Users can move the drawn shapes byclicking and holding until the cursor changesto the move cursor, then the user can drag theshapes around to the desired position. Itemssuch as arrows or nodes can be labeled by thestudent either using the round menu asdescribed above or by double clicking on the

Articles

58 AI MAGAZINE

File Edit Tools Instructor HelpMechanix – Assignment 1

of 3

6440 lbs

Equations: ΣFxΣFyΣM

Find the reactions and internal forces for the truss shown when P=6440 lbs (lbf)

D

E

FC

P2

2

2

2 2 2 2

1

T

Set up the Problem

Input Forces

Reaction Forces

Member Forces

You’re missing a reaction force at node C.

Question 1

Figure 1. Mechanix Interface.

shape and entering the text.

Visual FeedbackPrimitive recognition is used not only as thebasis for higher-level recognition but also forcoloring drawn shapes as a feedback method.For example, a shape recognized as a force (anarrow) is colored lime green to indicate to thestudent that Mechanix recognized the forceand that they may enter relevant metadata forthat force, such as the magnitude.

Geometric RecognitionThe hierarchical building blocks of our recogni-tion are points, strokes, and shapes. The programgenerates a point for each movement of themouse. It records the x and y coordinates and thecurrent time. Strokes contain the group of pointscollected in the time between when the pen touch-es down on the tablet and when it loses contactwith it. For example, a y character written in cur-sive will be one stroke, but a printed y will likely betwo strokes. Primitive shapes contain at most a sin-gle stroke. Strokes are segmented using a cusp

detector (Wolin 2010) and the primitive shapes arerecognized by PaleoSketch (Paulson and Ham-mond 2008). Examples of primitive shapes are linesegments, circles, arcs, curves, polylines (severalline segments drawn in a single stroke), triangles,and others. Complex shapes, such as roller sup-ports (described in figure 3), are built first of prim-itives and composed hierarchically to allow formore and more complex shapes. Mechanix createscomplex shapes only after the member shapes passour geometric-constraint-based recognizers.

Steps to RecognitionOur geometric shape recognition happens in sixsimple steps. First, we record points as the pentraces on the screen and add points to a newstroke. Second, Mechanix sends each new stroke toPaleoSketch to find its primitive shapes (line, cir-cle, arc, polyline, and others.). Third, we add eachnew shape to the collection of all shapes. Fourth,we send various groupings of shapes to complexshape recognizers. Then, we apply a system of geo-metric constraints to recognize high-level shapes.Finally, we replace low-level shapes with new high-

Articles

WINTER 2012 59

a. b.

c. d.

Figure 2. Examples of the Capabilities of the Scribble Erasure Feature.

level shapes, return to step 3, and cycle until nomore complex shapes can be found. As an exampleof a system of geometric constraints, the recogniz-er for the roller support given in figure 3 requires atriangle and two circles as components. The rollersupport recognizer checks each of the conditionsgiven in figure 3.

If the component shapes meet all of the con-straints for a specific configuration, the recognizercombines them into a new complex shape. Uponpositive recognition, the recognizer assigns thenew shape a label that acts as a name tag to othershapes. The recognizers test new groupings madewith that new shape to ensure the most completeand thorough recognition possible before the pro-gram returns to the first step (gathering points andmaking strokes). All recognition executes in realtime.

Truss RecognitionTrusses are one of the main structures or symbolsfor the Mechanix system to recognize. Trusses arebasic structures used in applications such asbridges, airplane wings, and buildings. Truss dia-grams allow students to learn the way forces reacton beams. Rather than attempt to define each validtruss individually, we use a general definition toidentify any trusses the instructors or students maywant to draw. We collected 589 data sets andachieved an accuracy of 94.6 percent.

We define a truss as a collection of convex poly-gons, each of which shares at least one side withanother polygon in the truss. If we can find two

polygons that share an edge, then we have founda truss.

Figure 4 shows two examples of shared edges.Once we have built the connected graph, we con-sider each edge AB as potentially a shared edge. Tofind out if the edge AB is a shared edge, we removethe edge from the graph. Then the system tries tofind another path to go from A to B using abreadth-first algorithm. If we can find anotherpath, then the edge AB is recognized as a sharededge. As shown in figure 4, the breadth-first search(BFS) algorithm will first find the blue path andremove all of its edges from the graph. At thatpoint, the red path tries to find its way from A toB. In figure 4 (1), the red path can reach from A toB, and our system identifies the edge AB as a sharededge. However, in figure 4 (2), the red line cannotreach from A to B, so our algorithm will not detectthe edge AB as a shared edge. The algorithm can befound in Field et al. (2011).

Students receive subtle feedback that leads themto draw correctly recognized trusses or bodies. Forexample, when any shape is recognized, hoveringover any part of that shape will highlight all of thecomponent strokes in unison. If a shape is not rec-ognized, the strokes will highlight individually.Additionally, when the truss is recognized correct-ly in student mode, the nodes that the instructordefined will appear. With little to no training, stu-dents know that in the rare case that the truss orbody was misrecognized and the nodes do notappear, they need to redraw the truss.

Articles

60 AI MAGAZINE

top intersectionpoint and

bottom midpointmust form

perpendicularline

bottom-mostline must behorizontal

circles mustbe smaller than

the triangle

circles mustbe similar insize to one

another

circles mustbe below and

reasonably nearthe triangle

midpoints ofcircles must

form horizontalline

Figure 3. Example Geometric Constraints for a Roller Support.

Answer CheckingMechanix automatically checks the students’assignments as they are submitted. If students sub-mit incorrect answers, Mechanix provides benefi-cial feedback to help students reach the correctanswers. Mechanix initializes the answer checkerwhenever a student clicks the green check buttonin the upper right corner of the interface. In orderto grade the student’s submission, Mechanix com-pares the student’s sketch with the instructor’ssketch. Mechanix checks the assignments by firstensuring that the student’s truss has the correctconfiguration. Then, it checks that the student’ssketch contains an axis. Then it ensures all forcesare present and in the correct direction. Finally,Mechanix checks that the student’s force values areequivalent to the instructor’s force values.

To check whether the student’s truss is similar tothe instructor’s hand-drawn truss, we first create agraph data structure composed of connected nodesand beams for both trusses. Students have differ-ent styles of drawing trusses. For example, when anassignment requires a truss such as the ones givenin figure 1, some students draw a big triangle firstand then draw a line down the middle, while oth-er students draw the two triangles individually. Tomake the number of beams equal between all cas-es, we implemented a beam intersect mechanism.If a beam intersects another, both beams are seg-mented at the intersection point and a node isformed between them. After all intersections(nodes) have been found, we use basic graph iso-morphism to determine whether the graphs aresimilar.

After the answer checker determines that thetruss is correct, the system checks to see if the

sketch includes an axis. Axes are necessary to spec-ify what is considered positive.

To check the force values, we first check the typeof force, which can be either a reaction force or aninput force. An input force means that it has a val-ue or constant for its label, and a reaction force hasa label that starts with F or R and its direction X orY at the end of the label (for example “Fay”). AfterMechanix checks the types of forces, the systemchecks whether the forces are attached to appro-priate nodes and have been given the correct val-ues. Finally, if the submitted sketch has any miss-ing or incorrect answers, Mechanix gives beneficialfeedback such as “You are missing an input force atNode B” or “You have not drawn an axis”.

Other Problem TypesNontruss free-body diagrams are a second type ofproblem in the statics domain. These free-bodydiagrams depict the forces acting on an arbitraryphysical body, such as a table or an escalator stair.The body given in the problem statement could beany bubblelike shape, such as those seen in figure5. Because we as programmers cannot predict andwrite geometric recognizers for all of the possiblebodies, a generic closed-shape comparison tech-nique is necessary. An instructor simply sketchesthe desired closed-shape body in the answer key.All student closed shapes are compared with theinstructor’s and a binary similarity decision ismade.

First, we recognize the shape bodies. We take acollection of primitive line shapes drawn in one ormore strokes and attempt to traverse them andreturn to the first point of the first line. Two lines

Articles

WINTER 2012 61

B

A

B

A

1) 2)

Figure 4. Examples of Shared Edges in Trusses.

are traversible if their end points are within 9 per-cent of the total path length of all the strokes com-bined. This gap allowance allows for bodies to bemade of multiple strokes, without excessive care toperfectly line up end points.

To begin comparing the two shapes (the stu-dent’s shape and the instructor’s shape), we use themethod defined by Wobbrock, Wilson, and Li(2007). We resample both shapes to contain 64evenly spaced points. We then scale the shapesdown to a 40 by 40 window so we can measure dis-tances accurately.

We use three of the similarity metrics proposedby Kara and Stahovich (2005), the Hausdorff dis-tance, a modified Hausdorff distance, and the Tan-imoto coefficient. To find the Hausdorff distancesbetween the two bodies A and B, we initialize twodistance vectors DA and DB , each of size 64 (thenumber of points in each shape) such that eachentry represents the closest distance from eachpoint to the closest point in the other shape. Themaximum value in both DA and DB is the Haus-dorff distance. The modified Hausdorff distance isthe average of the DA and DB values combined.Because the Hausdorff distance (the maximumminimum distance between two points in A and B)and the modified Hausdorff distance (the averageminimum distance between two points in A and B)are distance measures, we convert them to a simi-larity measure by dividing the distance by 20 andsubtracting that value from 1.

We chose the value 20 because it represents halfof the width of the bounding window. Any twoshapes that contain points whose nearest neighborpoints are more than half the width of the bound-ing window apart cannot be deemed similar. Thefinal measure used to determine body similarity isthe Tanimoto coefficent. This is simply the ratio ofpoints that “overlap” (number of points that havedistance values in DA and DB less than or equal to4.0) to the total number of points in DA and DB.

Finally, we take the three measures (Hausdorffdistance similarity measure, modified Hausdorff

similarity measure, and Tanimoto coefficient) andaverage their values (Kara and Stahovich 2005). Ifthe resulting average similarity measure is greaterthan 0.65, we consider the student’s and theinstructor’s shapes to match. A full description ofthe algorithm can be found in Field et al. (2011).

As an indication to the student that a match hasbeen found, the nodes from the instructor’s sketchare automatically revealed to the student. (Instruc-tors add nodes by drawing small dots and labelingthem with the desired letter in instructor mode.)

Creative ResponseA creative response problem is a problem that isopen ended for the student to solve. Instead of thenormal problem where the student has to draw atruss to match the teacher’s truss, the student hasto draw a truss to meet a certain set of design con-straints. A typical problem is:

A village needs a bridge to connect it to the citymarketplace across a chasm. The bridge should spanbetween 7 and 8 feet as measured from the end sup-ports and should be able to carry a load of 56pounds applied to the center top of the span. Themaximum load for each member is 70 Newtons.

This allows the students to design any bridge theydesire to accomplish the task at hand. There is noexample truss that is used for comparison. Instead,the Mechanix system uses artificial intelligence tocheck the student-created truss (figure 6).

It creates a linear system of equations with threeparts. The first part of the system is the summationof all forces along the x-axis in figure 6 (Rax and 10lbs). It uses the arrow recognizer and compares theslope with the axis to determine the direction,which gives the equation

After adding all the forces that reside along the x-axis, it does the same process along the y-axis,which ends up with

Fx! = Rax +10

Articles

62 AI MAGAZINE

Figure 5. Example Closed-Shape Bodies.

The forces in the same direction have the samesign. For the final equation, Mechanix chooses anode with the most reaction forces around it. Thenit does a summation of the moments iteratingthrough all of the other forces computing the crossproduct between the forces and the distance to thenode. In figure 6, node A is chosen to compute thecross product and results in the following equa-tion:

After finding all the external forces, a system ofequations is formed using the method of joints foreach beam in the truss. An example of this is withnode A:

M! = Rey"2#10"1#10"1sin($3)

= M! = Rey"2#18.660

Fy! = Rey + Ray "10

After creating all values from the student-drawntruss, Mechanix will use the values to compare toa set of constraints that the instructor previouslyentered, such as the length of the bridge or maxi-mum force. The student will receive helpful feed-back on whether the truss he or she drew adheredto the constraints.

Distributed ArchitectureStudents begin assignments by downloading andstarting a client application. The client communi-cates to the server, which stores not only the stu-dent’s final solution but also a log of mistakes thatthe student made while solving the problem. Theselogs may be used for assigning grades or, moreimportantly, determining problem areas where stu-

Ax! =1Fac +0.5Fab"10.0= 0

Ay! = Fab sin(#3)+ Ray = 0

Articles

WINTER 2012 63

=

of 3

6440 lbsRay

Rax

10 lbs

Rey

2

ΣFx =

ΣFy =

ΣM

A village needs a bridge to connect it to the city market place across a chasm. The bridge should span between 7 and 8 feet as measured from the end supports and should be able to carry a load of 56 pounds applied to the center top of the span. The maximum load for each member is 70 Newtons.

Rax -10.0 Lbs

Fab -.773 (units) Fac 10.368 (units) Fbc .773 (units) Fbd -10.773 (units) Fcd 10.773 (units) Fce 5.386 (units)

Ray .6698 Lbs Rey 9.33 Lbs

A -rey * 2.0 - 18.660254

rey + ray - 10

rax + 10.0

DDDDDDBBBBBBB

AAAAAAAAA EEEEECCCCC

File Edit Tools Instructor HelpMechanix – Assignment 1

Figure 6. A Creative Response Truss.

dents struggle with understanding the material. Ifmany students make the same mistake, the instruc-tor knows he or she should probably spend moreclass time explaining that particular concept.

To prevent the possibility of cheating, theanswer sketches drawn by the instructor are neversent to the student’s Mechanix client application.This means that all answer checking must be per-formed in a secure server application before feed-back can be sent back to the Mechanix client. Ini-tially this was handled by a single server, but theload of recognition for one problem submission isnontrivial, and when 30 plus students would sub-mit sample problems simultaneously our serverwould become inoperable.

To overcome this limitation, we use web appli-cation load-balancing techniques. All data is trans-ferred between the clients and servers as XML overHTTP. We use an HTTPS proxy server to encrypt allincoming and outgoing data to protect authenti-cation information and student confidentiality. Allincoming HTTP requests are then routed using anHTTP load balancer, HAProxy, to several machines,each running the Mechanix server software. EachMechanix server runs an embedded instance of Jet-ty, which we use to handle HTTP communicationand user session information. Finally the XMLrequest body is parsed, we perform answer check-ing on the resultant sketch object, and return thenecessary feedback as an XML HTTP response. Thisarchitecture allows us to use off-the-shelf softwareto achieve simple and practical scalability to sup-port larger groups of students.

Deployed System ResultsWe have deployed Mechanix in the classroom forthree consecutive semesters. The participatingcourse, Engineering 111 (Foundations of Engi-neering), covers basic statics, visualization andCAD tools, Newton’s laws, unit conversions, andothers. Thus far, a total of 111 students (interest-ing coincidence given the course number) haveused Mechanix to submit homework assignmentsthat otherwise would be submitted on paper.

In the fall semester of 2010, Mechanix was firstdeployed in an honors section of ENG 111. Stu-dents in the course were given the option to useMechanix or pencil and paper for two homeworkassignments. For the first assignment, all 33 stu-dents chose to use Mechanix. For the secondassignment, 22 students chose to continue usingMechanix. In the first semester, our purpose wasmainly debugging and refinement; we found itpromising that 22 students chose to use the soft-ware again. Note that students were required to useMechanix in a computer lab and could not accessthe software from home. This may have affectedparticipation.

In the second semester of deployment (spring2011), 20 of 64 student volunteers from a regularsection of ENG 111 used Mechanix, and theremaining used traditional pencil and paper forcomparison. Mechanix was used for three home-work assignments. For the first of the Mechanixassignments, the grades were similar between theMechanix and control groups. On the second andthird assignments, however, the Mechanix groupscored an average of more than 40 percent higherthan the control group, as detailed in figure 7.More details regarding this deployment can befound in Atilola et al. (2011).

Note that in the spring 2011 semester, the stu-dent pool was made up significantly of at-risk stu-dents who either did not have the necessary mathskills their first semester or were transfer students.Much of the increase in scores was due to the stu-dents in the Mechanix group having higherhomework completion rates. We believe this to bea good measure of engagement. The honors sec-tions from the other two semesters did not showthis increase, which is most likely due to the factthat the honors students were already highlyengaged, and thus already turning in completehomework.

For the third semester of deployment, 122 stu-dent volunteers from both honors and regularsections participated in the study. In total, 58 stu-dents were assigned randomly to the Mechanixgroup, and the remaining used pencil and paper.Again, three homework assignments were given.Partway through the semester, we found our serv-er could not handle the increased student load (ajump from 20 to 58). Not all test students contin-

Articles

64 AI MAGAZINE

0

2

4

6

8

10

Scor

e (o

ut o

f 10)

Scores on Exam Free Body Diagram Question

HW 1 HW 2* HW 3* HW 4* HW 5

* Homework assignments where Mechanix was used

Control GroupMechanix Group

Figure 7. Students Using Mechanix Score 40 Percent Higher on Homework Assignments.

ued using Mechanix after the first assignment.These students were removed from our data.

A force concept inventory quiz was given beforeand after the lectures associated with free-body dia-grams and forces. The students that used Mechanixfor all three assignments showed substantiallymore improvement than the other students, asseen in figure 8. Students that used Mechanixscored higher on the free-body diagram questionon the exam, as seen in figure 9. These findingsindicate that Mechanix fosters learning of staticsconcepts better than traditional pen and paper.More detailed descriptions of the third semester’sdeployment can be found in Atilola et al. (2012).

In postexperiment focus groups each semester,students offered many constructive comments onMechanix. Students found the instant feedbackfeature very helpful, which encouraged them tochoose to use Mechanix over pencil and paper. Stu-dents were also impressed that the program couldrecognize even badly drawn trusses. This madethem think more about the problem and less ontrying to draw a perfect truss. One other featurethat students mentioned was the checklist thattold them the order in which to solve the problem.They thought it helped ensure they were solvingthe problem correctly. In the focus groups somestudents requested the use of Mechanix on examsin addition to homework. This implies confidencein the software and a willingness to continue touse it. Some students also mentioned that usingMechanix encouraged them to move on to anoth-er problem after finishing the first. Many studentsexpressed that they thought of Mechanix more asa learning tool that helps teach the process of solv-ing these problems than just another way to turnin homework.

ConclusionIn this article we described Mechanix, a deployedsystem, and its use of artificial intelligence to aidteachers and students with the learning process.Mechanix is built with a number of recognitiontechniques that give it many features to help stu-dents become successful in the classroom. It hasbeen able to interpret students’ answers in realtime to provide instant feedback in a transparentenvironment. The goal of Mechanix is to be a flu-id system that can provide instant feedback whilestill allowing students to hand-draw their solu-tions.

AcknowledgementsThe authors thank the many other contributors toMechanix, specifically Martin Field and ChrisAikens. This research is funded in part by Googleand the National Science Foundation under GrantNos. 0935219 and 0942400.

ReferencesAlvarado, C., and Davis, R. 2004. Sketchread: A Multi-Domain Sketch Recognition Engine. In Proceedings of the17th Annual ACM Cymposium on User Interface Softwareand Technology, 23–32. New York: Association for Com-puting Machinery.

Atilola, O.; Field, M.; McTigue, E.; Hammond, T.; and Lin-sey, J. 2011. Evaluation of a Natural Sketch Interface forTruss FBDs and Analysis. In Proceedings of the IEEE 2011Frontiers in Education Conference (FIE). Piscataway, NJ:Institute of Electrical and Electronics Engineers.

Atilola, O.; Osterman, C.; Hammond, T.; and Linsey, J.2012. Mechanix: The Development of a Sketch Recogni-tion Truss Tutoring System. In Proceedings of the American

Articles

WINTER 2012 65

Percent Increase - Pre to PostForce Concept Inventory

0%

10%

20%

30%

40%

50%

Perc

ent

Incr

ease

HonorsMechanixStudents

HonorsControlStudents

RegularMechanixStudents

RegularControlStudents

Figure 8. Students Using Mechanix Proved a Better Improvement in Knowledge of Forces than Nonusers

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

16.00

18.00

20.00

Scor

e (o

ut o

f 20)

HonorsMechanixStudents

HonorsControlStudents

RegularMechanixStudents

RegularControlStudents

Scores on Exam Free Body Diagram Question

Figure 9. Students Using Mechanix Scored Higher on the Free-Body Diagram Exam Question.

tant in the Sketch Recognition Lab underthe guidance of Tracy Hammond. Hisresearch primary interests are in computerhuman interaction, sketch recognition, pat-tern analysis, and distributed systems.

David Turner is a junior undergraduate stu-dent at Texas A&M University. He is aresearch assistant in the Sketch RecognitionLab under Tracy Hammond. He designedand implemented the creative responsefunctionality of Mechanix over the past twosummers.

Hong-hoe Kim is a Ph.D. student in theSketch Recognition Lab at Texas A&M Uni-versity under the supervision of Tracy Ham-mond. His research is in the area of human-computer interaction and artificialintelligence, including the design of perva-sive computing applications, sketch-basededucational applications, and pattern analy-sis of human’s drawings. He holds a Bache-lor’s degree in computer science fromSoongsil University in Korea.

Wenzhe Li is currently working as a soft-ware development engineer atAmazon.com. His research interests are inmachine learning and computer humaninteraction. He received an MS in computerscience at Texas A&M University, and a BSin computer science at Nankai University.

Julie Linsey is an assistant professor in theMechanical Engineering Department atTexas A&M University. Her research focus ison design methods, theory, and engineeringeducation with a particular focus on inno-vation and conceptual design.

Tracy Hammond is the director of theSketch Recognition Lab and an associateprofessor in the Department of ComputerScience and Engineering at Texas A&M Uni-versity. Hammond earned a BA in mathe-matics, a BS in applied mathematics, an MSin computer science, and an MA in anthro-pology as well as an FTO (finance technolo-gy option) and a Ph.D. in computer sciencefrom MIT under Randy Davis.

Society for Engineering Education 2012 AnnualConference. Washington, DC: American Soci-ety for Engineering Education.

Bangert-Drowns, R. L.; Kulik, C. L. C.; Kulik,J. A.; and Morgan, M. 1991. The Instruc-tional Effect of Feedback in Test-Like Events.Review of Educational Research 61(2): 213–238.

Callahan, J.; Hopkins, D.; Weiser, M.; andShneiderman, B. 1988. An Empirical Com-parison of Pie Versus Linear Menus. In Pro-ceedings of the SIGCHI Conference on HumanFactors in Computing Systems, 95–100. NewYork: Association for Computing Machin-ery.

Field, M.; Valentine, S.; Linsey, J.; and Ham-mond, T. 2011. Sketch Recognition Algo-rithms for Comparing Complex and Unpre-dictable Shapes. In Proceedings of theTwenty-Third International Joint Conference onArtificial Intelligence. Menlo Park, CA: AAAIPress.

Gennari, L.; Kara, L. B.; Stahovich, T. F.; andShimada, K. 2005. Combining Geometryand Domain Knowledge to Interpret Hand-Drawn Diagrams. Computers and Graphics29(4): 547–562.

Hammond, T., and Davis, R. 2005. LADDER,A Sketching Language for User InterfaceDevelopers. Computers & Graphics 29(4):518–532.

Johnston, J., and Hammond, T. 2010. Com-puting Confidence Values for GeometricConstraints for Use in Sketch Recognition.In Proceedings of the Seventh Eurographics /ACM Sketch-Based Interfaces and ModelingSymposium, 71–78. Aire-la-Ville, Switzer-land: Eurographics Association.

Kara, L. B., and Stahovich, T. F. 2005. AnImage-Based, Trainable Symbol Recognizerfor Hand-Drawn Sketches. Computers &Graphics 29(4): 501–517.

Kozma, R. B. 1994. Will Media InfluenceLearning Reframing the Debate? EducationalTechnology 42: 7–19.

Lee, W.; de Silva, R.; Peterson, E. J.; Calfee, R.C.; and Stahovich, T. F. 2008. Newton’s Pen:A Pen-Based Tutoring System for Statics.Computers & Graphics 32(5): 511–524.

Miller, E. G.; Matsakis, N. E.; and Viola, P. A.2000. Learning from One Example ThroughShared Densities on Transforms. In Proceed-ings of the 2000 IEEE Conference on ComputerVision and Pattern Recognition, 464–471. Pis-cataway, NJ: Institute of Electrical and Elec-tronics Engineers.

Paulson, B., and Hammond, T. 2008. Pale-oSketch: Accurate Primitive Sketch Recogni-tion and Beautification. In Proceedings of the13th International Cnference on Intelligent UserInterfaces, 1–10. New York: Association forComputing Machinery.

Roselli, R. J.; Howard, L.; Cinnamon, B.; Bro-

phy, S.; Norris, P.; Rothney, M.; and Eggers,D. 2003. Integration of an Interactive FreeBody Diagram Assistant with a CoursewareAuthoring Package and an ExperimentalLearning Management System. In Proceed-ings of the American Society for EngineeringEducation. Washington, DC: American Soci-ety for Engineering Education.

Rubine, D. 1991. Specifying Gestures byExample. In SIGGRAPH ’91: Proceedings of the18th Annual Conference on Computer Graph-ics and Interactive Techniques, 329–337. NewYork: Association for Computing Machin-ery.

Sweller, J. 1994. Cognitive Load Theory,Learning Difficulty, and InstructionalDesign. Learning and Instruction 4(4): 295–312.

Vanlehn, K.; Lynch, C.; Schulze, K.; Shapiro,J. A.; Shelby, R.; Taylor, L.; Treacy, D.; Wein-stein, A.; and Wintersgill, M. 2005. TheAndes Physics Tutoring System: LessonsLearned. International Journal of ArtificialIntelligence Education 15(3): 147–204.

Wobbrock, J. O.; Wilson, A. D.; and Li, Y.2007. Gestures Without Libraries, Toolkits,or Training: A $1 Recognizer for User Inter-face Prototypes. In Proceedings of the 20thAnnual ACM Symposium on User InterfaceSoftware and Technology, 159–168. New York:Association for Computing Machinery.

Wolin, A. 2010. Segmenting Hand-DrawnStrokes. Master’s thesis, Texas A&M Univer-sity, College Station, TX.

Yin, P.; Forbus, K. D.; Usher, J.; Sageman, B.;and Jee, B. D. 2010. Sketch Worksheets: ASketch-Based Educational Software System.In Proceedings of the 22nd AAAI Conference onInnovative Applications of Artificial Intelli-gence. Menlo Park, CA: AAAI Press.

Stephanie Valentine is a Ph.D. student atTexas A&M University. She works as aresearch assistant in the Sketch RecognitionLab under the guidance of Tracy Ham-mond. Her research interests include geo-metric sketch recognition, educational soft-ware, child-computer interaction, andcyber-safety for preteens and early adoles-cents.

Francisco Vides is currently working as asoftware engineer at PayPal. He received aMaster’s degree in computer science at TexasA&M University where he also worked as agraduate researcher in the Sketch Recogni-tion Lab under the direction of Tracy Ham-mond. His research interests are in comput-er-human interaction, artificial intelligence,and intelligent tutoring systems.

George Lucchese is a Master’s student atTexas A&M University and a software devel-oper with IBM. He works as a research assis-

Articles

66 AI MAGAZINE