0001 Meng

8/11/2019 0001 Meng

1/10

Recent Patents on Mechanical Engineering2009,2,169-178 16

1874-477X/09 $100.00+.00 2009 Bentham Science Publishers Ltd.

Recent Innovations in Off-Road Intelligent Vehicles: In-Field AutomaticNavigation

Francisco Rovira-Ms*

School of Agricultural Engineering, Polytechnic University of Valencia, Valencia, Spain

Received: March 25, 2009; Accepted: May 27, 2009; Revised: May 29, 2009

Abstract: The fast evolution of computing power together with the improvement of electronics and sensors in the last

decade has resulted in the introduction of vehicle automation at a consumer level; therefore, every day, an increasing

amount of commercial solutions are reaching the global market. The special shape in which agricultural fields are

structured, on the other hand, makes off-road vehicles one of the best candidates for driving assistance. Very often, tractor

or harvester operators are required to follow predefined rows for long periods of time, leading to physical and mental

fatigue that can provoke dangerous situations or even fatal accidents. In such conditions, the aid brought by intelligent

systems can represent a positive asset to farm production systems. The recent popularization of satellite navigation

systems, such as GPS, has meant the final thrust to the deployment of automatic guidance systems. However, not only

global positioning sensors are key in vehicle automation but a complex architecture comprising imaging, laser and

ultrasonic devices can be found in recent models as well. This article presents an overview of patents issued in this field of

automotive engineering over the last twenty years.

Keywords: Automatic guidance, off-road vehicles, GPS auto-steering, farm machinery automation, autonomous navigationfield robotics, agricultural robotics, crop-tracking vehicles, tractor guidance & intelligent vehicles.

1. BACKGROUND AND SCOPE

Automatic guidance of vehicles is one of those themeswhere imagination and popular imagery has long precededtechnology. As early as 1924, Willrodt described a system toguide a machine following furrows across a field, andinterestingly enough, it was published as a patent [1]. Sincethe time of that pioneering work, engineers have been lucu-brating and imagining ways to make vehicles move without adriver. However, the solutions found were quite limited andspecifically adapted to particular environments. The siliconrevolution and digital era of the sixties and seventies wasgoing to change the scene; the fast growth of technology incomputers and electronics soon hinted that the old dream ofautonomous robots might be feasible. Yet there was a longway to go, but the special know-how that automatic navi-gation requires was being developed at a high pace. Never-theless, not until the eighties did ideas start to take form withthe first prototypes beginning the desired trip from conceptdevelopment to product development. The definite stroke hasbeen banged by the popularization of the Global PositioningSystem (GPS), which allows a real-time localization of amobile platform in a global frame. With all the ingredientsfor automation obtainable, it makes sense to wonder if there

is still a strong interest in the automation of off-road vehi-cles. The answer is affirmative, and the increasing number ofpatents and scientific articles focusing on these topicsdemonstrates that they are of intense and immediate interest.Spatial variability is a typical phenomenon in agriculturewhich traditional farming systems find difficult to cope with.

*Address Correspondence to this author at the School of AgriculturalEngineering, Polytechnic University of Valencia, Camino de Vera, s/n,46022 Valencia, Spain; Tel: +34-963877291; Fax: +34-963877299;E-mail: [email protected]

The possibility of knowing the accurate position of a mobilereceiver at any time using GPS technology has revolutionized farming production, and in fact has led to thedevelopment of a new concept: Precision Agriculture. Thebenefits of this technological approach are numerous, andbesides the observable improvements in the quality oregular tasks, ergonomics and safety are certainly enhancedas intelligence is being incorporated to off-road equipmentLaboratory models are giving way to actual applications, andCalifornia law, for example, allows tractors in furrow

traveling less than two miles per hour to travel without adriver, provided that the throttle, clutch, and brakes can becontrolled remotely, a practice that is wide-spread withharvest crews [2].

There exist many channels to transmit scientific andtechnological innovations, the most frequent being journaarticles, technical conferences, industry shows and patents. Iis evident that they are not mutually exclusive, and thereforit must be an overlap among them. Consequently, eventhough this paper presents a review based upon patents, animportant amount of the material covered here can also befound in regular scientific literature. The majority of ideaprotected through patents that have been disclosed byacademic personnel, for instance, have also been publishedas journal papers or conference proceedings. However, thecorrespondence is not direct, and many companies onlyreveal their discoveries as a need to protect their intellectuaproperty. For the layman, and even for engineers, thelanguage and format in which patents are structured maybecome hermetic and difficult to grasp because experattorneys intend to protect the maximum while divulging thminimum. As a result, an innovation very often results in afamily of patents where differences among individual textare hard to identify, and yet there are slight details making

8/11/2019 0001 Meng

2/10

170 Recent Patents on Mechanical Engineering 2009, Vol. 2, No. 3 Francisco Rovira-M

worth the multiplication of documents. There is a long, notnecessarily straight, route from a patent application to thecorresponding commercial product. As a matter of fact,many patents do not f inalize in concrete physical realizationsand they, generally speaking, serve as a shield for conceptsupon which further developments can be constructed withoutthe need of incurring in costly intellectual property expensesdue to other assignees. In spite of the scanty circulation and

trade of intellectual property documentation, since they tendto remain within the institution holding legal rights overthem, patents stimulate innovation and technical advances.Prog-ress spurred by industry in this field is essential andshould never be overlooked, as the successful outcomes ofthe Grand Challenge competition clearly prove [3].

The objective of this article is to provide an overview ofrecent innovations in automatic guidance of off-road vehiclesthrough the revision of key patents issued in this field.Obviously, it is impractical to analyze every single patentrelated to this novel application, but a selection of thosetransmitting the fundamental knowledge involved have beencarefully examined and discussed in the following sections.A total of sixty patent documents have been studied. Figure(1) shows a list of the most important features included in theset of patents considered, and the percentage of documents inwhich every subject matter listed is treated. If a threshold ofpopularity can arbitrarily be traced at 20%, the bar chart ofFig. (1) indicates that the most recurrent features are: 1)actuation through electrohydraulics; 2) safeguarding andsecurity mechanisms; 3) navigation via satellite localization;

4) computer vision applications; 5) control systems; and 6trajectory planning.

2. GENERAL SOLUTIONS FOR THE EARLYCHALLENGES

Even though there are patents dealing with driverlessvehicles since the 1920s, substantial inventions for thecurrent state of the art are mainly dated from the 80onward. This patent review considers two different periodof study: what took place during the 80s and 90s of the 20

t

Century; and all the innovations developed after the year2000. The main reason for setting the division line at the turnof the century is the boost given to this technology by a seof circumstances such as the state of maturity reached byvision sensors, the development of compact binoculacameras, the cancellation of the selective availability of GPSthe dominance of the digital imagery, etc.

Once computer power reached significant levels operformance and sensors became available and affordablegeneral solutions for autoguidance emerged. It is commonfor these first inventions to cover most of the key areas in

vehicle automation. In the robot tractor envisioned by Pryoin 1988 [4], automatic steering is achieved when a solid stateTV camera detects target boards placed in the field. Amicroprocessor is able to control both steering and throttlegenerate pre-programmed 180 turns at the end of thefurrowed area, and even manipulate an arm for pickingfruits. Logically, safety has been a concern from the initiaapplications, and this invention proposes a toy-like bumper

Fig. (1).Important features included in patents on automatic guidance of off-road vehicles.

8/11/2019 0001 Meng

3/10

Recent Innovations in Off-road Intelligent Vehicles Recent Patents on Mechanical Engineering2009, Vol. 2, No. 3 171

system around the tractor or a cable buried along theboundaries of the field detectable by a magnetic sensor. Interms of localization, the pre-GPS era could be consideredthe beacons reign, as this was the preferred positioningtechnique. The 1987 steering automata described in [5] wascapable of following a preprogrammed course calculatingangles and positions between the vehicle and referencepoints marked by fixed beacons or rotating lasers. Steering

actuation was achieved through en electrically controlledhydraulic valve governing the steering cylinder, method thathas remained in practice until now. When optical sensorswere not used, the alternative to global positioning was localsensing with mechanical feelers. The mechanism of [6] wasable to follow a furrow previously traced with a gauge wheelattached to the guided vehicle. Actuation was executed byturning the steering wheel with an electric motor and atransmission commanded by a rotating shaft inserted in theleading furrow.

Before the turn of the century, general robotic systemsfor assisting off-road vehicles in a wide selection of agricul-tural operations were envisioned. The dawn of precisionagriculture was stimulating complex, and very oftenimaginative, solutions to transform the traditional farm into aproduction factory. The self-navigating robot proposed byRocks [7], for instance, claims accuracies of one inch whenperforming farming tasks as tillage, planting and harvesting,even in undulated irregular terrain. Such level of accuracy isobtained by means of seven halogen bulb beacons whichallow for dead reckoning calculations comparable to a pre-stored scenario loaded in the on-board computer. If anemergency arises, the vehicle can be halted remotely. Thescouting vehicle described in [8], was developed around thesame time, but it already incorporates satellite localization toget rid of on-site references. Both techniques coexistedbefore the US Department of Defense canceled the selectiveavailability.

3. THE ADVENT OF SATELLITE LOCALIZATION

The real-time knowledge of a mobile machine preciseposition unchained the fast development of automation in thefarm field with all kind of solutions and numberless projectsbeing tackled in both academy and industry. A significantmilestone in the progress of satellite navigation applicationsto agriculture, where high accuracy to move between crops isnecessary, has been the deactivation of the selectiveavailability by a Presidential Order on 2 May, 2000 [9]. Theselective availability (SA) was an on-purpose GPS signaldegradation introduced by the Department of Defense tocontrol accuracy of GPS for civil use. At present, GPS is

being used extensively in rural areas where large fields allowfor efficient vehicle automation, and the debate is beingshifted from signal accuracy to in-field performance accu-racy [10]. Even though the only completely functionalnavigation satellite system is NAVSTAR GPS, several othersystems are being implemented, such as Galileo orGLONASS, and therefore it is more appropriate to group allof them under the common denominator GNSS (GlobalNavigation Satellite System).

The applicability of GNSS to agriculture is, a priori,boundless. All kinds of platforms can benefit from global

localization. The global positioning base station described in[11], corrects and sends differential signals to either groundor airborne rovers to accurately dispense agriculturaproducts. The ground clearing machine of [12] uses rotatingtracks with a flat-surfaced tread to better distribute weighand improve stability. In general, GPS-guided vehiclerequire the establishment of a reference, or initial line, whichmany times is recalled as A-B line. The difference will be, o

course, on the steering algorithms, but the localization data iidentical. The raw data, basically time and geodesic coordinates, captured in the receiver at 1 Hz or 5 Hz can beprocessed in different ways in order to deduce as muchnavigation information as possible. Four GPS antennas weremounted on a tractor cabin by a team at Stanford University[13] to determine the vehicle attitude due to roll and pitchThis information was used to minimize the lateral positionerror of the tractor. The definition of the reference linetypically associated to the desired offset for parallel trackingis one of the features present in most of the satellite navigation systems. This line does not need to be straight, andcurved segments are eligible as well. The trajectory to befollowed by the vehicle is usually composed of a series o

points, technically called waypoints, which draw the desiredpath. The organizers of the Grand Challenge competition [3]for instance, provide the contenders with a complete set owaypoints from the beginning to the end line of the race. Theguidance assisting apparatus for agricultural vehicledescribed in [14] is driven from one waypoint to the nexalong the wayline. The wayline is a closed polygon with upto 1000 waypoints per kilometer. When the waypoints arenot in the same plane, difficulties arise. As mentioned in[13], a plurality of antennas can be used to estimate pitch androll, but this solution tends to be complex and bulky. Analternative [15] is to integrate the GPS with a tilt measuringdevice which, in addition to an electronic compass for robusheading, forms a terrain compensation unit. In these

circumstances, common in hilly terrains, the wayline isupgraded to a 3D line where 3D coordinates and 3D slopeangles need to be known. Generally speaking, the terraincompensation module can comprise gyros, accelerometerstilt sensors and the like. The terrain compensation module isessential to navigate inside the field, but not always thGNSS-based guidance systems provide full autonomy to thevehicle. An intermediate solution, known as semi-autonomysituation, provides assistance to the operator who is actuallydriving the vehicle but following the indications of thautomatic navigator [11, 16-18]. This is a fairly commonsolution, mainly deployed in the first commercial units, andis typically realized through the lightbar displays. The lateradisplacements of the vehicle are indicated by lighting an

equivalent number of lights on the proper half of a LED barLightbars are often replaced by more sophisticated monitorwhere a Graphic User Interface (GUI) provides the drivewith the important data for a safe operation. The greaaccuracy demands of precision agriculture applications haveresulted in the development of high performance GPSsolutions capable of localizing a receiver with sub-inchexactitude. The method described in [18] integrates a RealTime-Kinematics (RTK) GPS with precision farming toprovide accurate seeding, cultivating, planting or harvestingDue to its accuracy, this system can make use or assist in thconstruction of Geographic Information Systems (GIS) with

8/11/2019 0001 Meng

4/10


the overall goal of generating a digital map including theprecise position required by each operation.

4. OPTICAL AND LOCAL PERCEPTION

After the initial euphoria brought by the easy and reason-ably priced availability of GPS data, significant challengeswith potential severe consequences became visible andresulted in the necessity of local perception for vehiclesoperating with some degree of autonomy. Bias, drift andguess-row errors led Rovira-Ms et al. [19] to postulate amodel for sensor fusion of machine vision and GPS to guidea tractor at various speeds and under diverse field and sensorsignal quality conditions.

Optical range finders are universal solutions forestimating distances as a way to perceive locally, that is, inthe vicinity of the vehicle. These sensors emit electro-magnetic radiation and receive a reflecting beam from thedetectable object; a timer determines the elapsed time thatallows the calculation of distances. The electromagneticradiation can be infrared (IR), near infrared (NIR), ultra-violet (UV), red light, the visual spectrum, etc. In a general

embodiment [20, 21], the outbound optical path consists ofthe electromagnetic source, the focusing optical member(lens), and a micro-mirror array and transmission opticalmember. The inbound optical path includes the receptionoptical member and a detector. A processor is in charge ofconverting elapsed time into distance, and to do so it mayconsider the first return, the last return, the strongest return,the average of returns, or the median of returns. Sensing oneobject can be helpful for obstacle avoidance, but guiding avehicle requires multiple distance measurements. Theapplication patented in [22], for example, utilizes a pluralityof markers located on a predetermined road; a method coinedcontinuous/discrete marker system. The markers arereflective stripes painted along the path. The separation bet-

ween reflectors depends on the design and vehicle speed, butthe inventors suggest in one of the prototypes described aspacing of 50 feet. The signal processing algorithm, calcu-lates the heading angle with respect to the approximated pathas a mean to determine the desired steering angle of thevehicle. The previous descriptions set the common frame-work for local sensing, but the specific applications can bedivided into two groups: computer vision and the remainderof range finders.

4.1. Computer Vision

The tremendous potential of machine vision as aperception-navigation sensor for off-road vehicles, due in

part to its similitude to human visual perception, has resultedin the coalescence of agricultural and non-agriculturalinstitutions pursuing similar goals regarding vehicle auto-mation. This is the case of the self-propelled harvesterrobotized by Carnegie Mellon University [23, 24]. Theharvester was endowed with a crop-tracking apparatus forsteering the vehicle along the crop without operator assis-tance, although, for safety principles, the presence of theoperator was required in the cabin when navigating underautomatic mode. A video camera was set on a protrudingarm at 4 m height to detect the cut-uncut edge (crop line) ofthe yield being harvested. A color-based discriminator func-

tion running on a microprocessor found the best position forthe crop line. The harvester executed the turns by poweringthe front wheels at different speeds with hydraulic motorsThe kind of scene captured by the vision sensor shapes anddefines the image processing algorithms from whichdecisions will be made. In the case just explained, thesearched feature was the crop line delimited by the cut-uncufront, one of the most common scenarios found in agricul

tural fields.Another familiar in-field situation is the important case

of crops arranged in rows. For this condition, the systemproposed in [25, 26] features four regions of interest (ROI) toidentify two crop rows robustly and guide a tractor. A regionof interest is a window selected within the image whereimage processing techniques are applied to extract meaningful information, in our particular case about navigation. Theseries of images depicted in Fig. (2) illustrates a typicaimage analysis for computer vision navigation of a tractor ina soybean field [27]. The distinction between crop and noncrop (mainly soil) implemented in [25, 26] was based on thek-means algorithm applied to the four regions of interestThe k-means algorithm is a clustering technique whichminimizes the within group variance to partition the clusterwhere krepresents the number of groups or classes. Once thepartitions have been realized, the algorithm applies momenoperations to determine the location and orientation of croprows. Linear regression is then used to define the trajectorypath. An alternative processing technique to identify thecrops and find the optimum path is based on the Houghtransform [27], applied to the image shown in the example oFig. (2). Machine vision techniques have been combinedwith other algorithms more representative of artificiaintelligence, but really effective to cope with the usuadifficulties for an imaging device operating in open fields. Atwo-class k-means algorithm fed a fuzzy logic controller todecide the best estimate for the slope and intercept of a linerepresenting the cut-uncut edge detected by a video cameramounted on a harvester [28]. Due to the fact that the cameracreated a perspective shift growing with the width of thecombine head, and contrary to the case reported in [23, 24]the camera was eventually mounted on the combine headmaking the edge an almost vertical line. In the headlandsthough, there are no cut-uncut boundaries and the farmeneeds to steer the vehicle in a semicircular path until it facethe boundary from the opposite direction. Since direction isreversed, the camera is too far to capture an adequate viewand therefore a second camera was installed in the oppositeside of the corn head so that the vehicle could harvest fromboth sides [29].

A third situation, in addition to the detection of harvesting cut-uncut lines and crop rows, is represented by theautomation of a self-propelled picker following a cut swathof crop lying on the ground [30]. Given that the only sourceof guidance features is the swath, the ROI needs to beprecisely defined to avoid the swath falling outside theprocessing window. A successful technique to deal with theswath detection challenges classified swath pixels accordingto their texture information in relation to the texture of thesurroundings; the vertical stalks of grass remain standingwhile the swath includes longer stalks in multiple directionshence the difference in texture. A grayscale dependency

8/11/2019 0001 Meng

5/10


matrix based on 8-connectivity was the preferred operationto find the swath in the input images. When the color discri-minator of the harvester described in [23] found problemswith the Sudan grass, a texture discriminator based on theGabor filter was implemented instead. The localizationinformation deduced from image analysis can be compli-mented with other navigation parameters such as theodometry-based traction control on farm vehicles disclosed

in [31]. Odometry measures travel over a surface and isnormally used in dead-reckoning navigation. The accuracyof this distance estimation may be improved with theassistance of GPS position information. The participation ofGPS technology enhances the reliability of the measurementsbecause the optical odometer is a differential instrument andaccumulates errors over distance.

4.2. Other Range Finders

One of the classic alternatives to computer vision forvehicle local perception is represented by lasers rangefinders. The laser radar for lateral guidance reported in [32],produced a pulse modulated beam and received reflectedlight beams from mounted retroreflectors, covering theportion of the road ahead of the vehicle. The distance andangle to each reflector were calculated to determine theposition of the vehicle and to est imate the road curvature forcommanding course correction information. The roadreferences may be standard roadway reflectors mounted atregular intervals of, say, 15 meters apart. A conventionallaser transmitter may be a medium power IR (100 W)centered at a wavelength of 820-830 nm.

The other broad-spectrum local sensors for mobilerobotics are ultrasonic range finders. An ultrasonic guidanceapparatus envisioned by Coker [33] is capable of steering avehicle on a desired track between adjacent crop rows orsides of a marked furrow. Onboard sensors are setup for

transmitting ultrasonic ranging signals and receiving thereflected signals from the rows or furrows. Two ultrasonicemitters need to point downwardly and outwardly inopposite directions towards adjacent crop rows or plant stalkbottoms; as the vehicle moves laterally, the distance fromone sensor will increase while the distance to the other willdecrease. A transducer measures the elapsed time of theultrasonic pulse to travel to the object and back to the sensor,computing the distance based on the speed of sound. If bothtransducers sense an increase of distance, the data isdisregarded because it indicates a vertical motion (bounce)rather than a lateral displacement. This approach is, to a

certain extent, reactive and in some way reminds of mechanical feelers.

5. GUIDANCE ALGORITHMS, PATH-PLANNINGAND VEHICLE CONTROLS

The birth and development of the concepts key toartificial intelligence has preceded the availability of fastefficient, and cost-effective robotic hardware, yet it has beenthe fast growth of electronics and sensors what boostedautomation in off-road environments. Whatever the case maybe, any vehicle bestowed with some degree of autonomyrequires (artificial)

intelligence, and the design of guidancealgorithms and control strategy is probably the central stagein automatic navigation. This section reviews the most usedapproaches to process sensor data and elaborate an actuationplan to drive the vehicle as safe and as accurate as possible.

Different sensing techniques might make use of the samecontrol stratagem to steer a vehicle and vice versa; the samelocalization and perception data can be interpreted in manydifferent ways to guide an autonomous vehicle. Beingcomputer vision one of the preferred techniques for vehicle

local perception, there is a rich body of vision-based algorithms to resolve a vehicles trajectory. The self-propelledharvester described in [23] couples a best fitting stepfunction to the harvesting frontline using the least-squarederror criterion and a vote counter. The position of the cropline directly establishes the turning angle of the wheelswhich is executed by rotating the front wheels at differenrate. The multiple ROI guidance system for rows trackingcovered in [25, 26, 34-36] is able to trace a trajectoryexpressed in a matrix with a discrete number of points, thefirst five being extracted from the input image. A fixed lookahead time is employed to estimate look-ahead distancesThe trajectory path is determined via linear regression, andthe planner eventually finds the desired wheel angle and the

look-ahead distance along the y-axis (forward direction). Thesame k-means algorithm but with k = 2 [28] was used totrack the cut-uncut edge of corn being harvested. Theseparation line was determined by regression analysis andthe acceptability of the solution evaluated through fuzzylogic membership functions: a perfect vertical line (infinitslope) indicates that the camera is located over the cut-uncuedge [29]. A PID steering controller governs the hydrauliccylinders of the steering mechanism in the rear wheels of thecombine. Commands from the controller excite a solenoidthat actuates a proportional control valve.

(a) (b) (c) (d)

Fig. (2). Typical image analysis for navigation of off-road vehicles: (a) Region of interest selection; (b) Segmentation; (c) Mid-poin

encoding; and (d) Row identification and optimal path [27].

8/11/2019 0001 Meng

6/10


Vehicle navigation entails the definition of the desireddestination to be reached by the automatic machine, and thiscan range from a simplistic reactive system such asmechanical feelers to a sophisticated trajectory planner. Anexample of an intermediate case is represented by the lateralguidance control system described in [37], where thecalculation of the target point is based on the length of thepreview distance obtained as the intersection of the radius of

half circle with road geometry. Steering angles are computedwith linear equations as functions of marker gains andheading angles. The multisensory harvester equipped withGPS, INS, and two video cameras of [38] followed a fieldcoverage plan generated by a trajectory planner. The targetedspeeds are about 4 or 5 miles per hour (6-8 km/h) and themain controller follows a behavior-based approach. Thiscontrol strategy was conceived by Rodney Brooks [39] andestablishes a direct link between perception and actuation,circumventing cognition by high-level reasoning entities.

Conventional wisdom establishes that before reaching thestate of full autonomy, there will be many ideas,applications, prototypes, and solutions regarding semi-autonomy. The majority of the works reported so far requestthe presence of drivers at the extremes of the fields toperform turns between passes, and a good policy forengaging and disengaging automatic control becomes amatter of efficiency. The method developed in [40]initializes automatic guidance after turning at the end of thepath. When the operator moves the steering mechanism,autoguidance deactivates to allow the driver perform theturning maneuver; after it, the position of the steeringmechanism is monitored, and if it falls within apredetermined zone, autoguidance remains activated. Other-wise, it is disconnected. This system lets automatic controlremain connected when the machine is turning and thesteering mechanism is not in the straight travel position,regarded as the deadband zone. The deadband zone encom-passes several degrees on either side to account for smallvariations in steering. Once automatic control is engaged,most of the systems rely on fluid power to control thevehicle. The harsh and force-demanding nature of off-roadequipment duties places hydraulics as the preferred actuationmode. The vehicle mentioned in [14] features a PIDcontroller to move from one waypoint to the next, gettingfeedback from a wheel angle sensor. The robotized tractor of[41] possesses various sensors whose data is fused for real-time guidance. It combines map-based guidance and sensor-based guidance because no individual sensing technology isideally suited for vehicle automation. Data fusion is carriedout with the Kalman filter. The actuator is, again, an electro-

hydraulic valve commanded with Pulse Width Modulation(PWM) signals. The wheel angle sensor is a rotarypotentiometer aligned with the king pin on the right wheel .The basic difference between semi-autonomous and auto-nomous guidance rests on actuation; the rest of the processremains the same. Therefore, both systems may includesimilar localization and perception systems, identical controlalgorithms and trajectory planners, but only the latter isexpected to implement actuating mechanisms to guide thevehicle, typically through hydraulic cylinders. A popularsemi-automatic solution is represented by the so-calledassistive systems. The embodiment of one of such systems

[42, 43] utilizes a feedback control system to compare theactual position of the vehicle with the desired path, providingfeedback to the human operator through a visual display. Theenhancement in efficiency conducting regular farming taskis considerable with assistive systems. The complexity of thecontrol algorithm is comparable to those implemented inautonomous navigators, and the assistive system mentionedabove, for instance, proposes a third order scheme (three

poles) for under-damped response with variable (velocitydependent) gains kp, ks, and kh.

Very often, the output data from localization or perception engines directly feed the vehicle controller to executthe essential functions of steering, braking, or acceleratingHowever, some situations and complex solutions insert atrajectory planner in-between. One non-trivial scenario ifound when a vehicle is guided along adjacent swaths acrosterrain with slope [44]. The solution can be outlined in threeconsecutive steps: first, estimate the tilt of the vehicle due tothe terrain slope; second, use GPS to find the position of afixed point on earth; and third, from tilt and positionmeasurements, calculate guiding commands to prevent gapand overlaps. A complete solution can be understood as theone comprising curved and straight segments, headlandturns, and a plurality of modes. The vehicle foreseen in [45accepts three operation modes (manual, semi-autonomousand autonomous), projects a plurality of trajectories (paralletracking or several row skipping patterns), and adapts thforward velocity to the position of the vehicle in the field(turn-entry velocity, turn velocity, and turn-exit velocity)The region-filling module (path planner) weaves the pathpattern within the perimeter defined by the user. The Aalgorithm is the internal engine in charge of producing theinterconnected segments that form the preferred path [46]. Inthe embodiment of this complete solution, the path planneactually comprises a perimeter training module whichcollects location data and a contour region-filling module[47]. The starting input data for the planner is the vehicleminimum turning radius, the task width (note that vehiculawidth may vary from implement width), and the referencerow location. The interference module establishes geometricconstraints between adjacent arcs to avoid interferences, thais to say, when two contiguous components converge andthere is no room to fit both. The reference contour must bechosen in such a way that the energy consumption focompletion of coverage is minimized. The reference path idefined by the coordinates and heading of the starting andfinal points; once known, the planner identifies candidatepaths. An estimator calculates an economic cost for traversing each candidate path. The economic cost depends on

distance and duration. The estimator may eliminate a path iit is not drivable, collides with obstacles, or the economiccost is above a determined threshold. By applying theserules, the estimator is capable of eliminating unnecessarylooping [48]. For certain regions, spiral paths may be moreefficient than linear or other paths in terms of time, fuel, orthe like. In order to cope with these situations, a spiraregion-filling module is designed [49] with the potential tomap out contour rows following a spiral, plan a pathcomprising nested loops, or generate a curve of variableradius. Spiral shapes might have problems with some workareas, such as those with pinch spots, and require a partia

8/11/2019 0001 Meng

7/10


application. If the work area is mostly circular, an Archi-medean spiral may be used. When contours run adjacent to along side of the region, the spiral tends to minimize flip turnsto reduce time. When the desired path is defined by a set ofwaypoints, and the turning radius of the vehicle can imperilthe completion of the mission, sophisticated courses need tobe taken such as the bi-variate periodic bi-cubic -splinefitting curve, which is known to yield a smooth curve

between waypoints [50]. With complex geometry, such asspirals and loops, it might be confusing to separate theregular tracking course from headland turns. The methoddeveloped in [16] specifically defines turns as changes inheading 30 over a distance of 100 m.

6. PRECISION AGRICULTURE AND IMPLEMENT

CONTROL

It seems somewhat contradictory to increase accuracy ina vehicles localization system, by means of RTK-GPS forexample, while incurring in gross errors controlling theimplements as a consequence of key sensors being mountedon the vehicle and not on the tool executing the task. Thisproblem becomes more severe as machinery grows.Incidentally, outsized terrains are the most benefitted fromprecision farming applications, but on the other hand, theyusually require large-scale equipment. For this reason,implement control is becoming a recurrent topic in this novelfarming technique. Precision agriculture is, from and by itsgenesis, inseparable from global satellite localization.Furthermore, very frequently it combines real-time GNSSinformation with GIS and aerial photographs of farms tocreate rich digital maps. A generic example of a site-specificfarming system is given in [51], where the boundaries ofagricultural fields are identified to execute functions depen-ding on the position of the vehicle: variable-rate applicationand implement height depends on boundary information. The

speed of the vehicle can be adjusted according to thedistance of the vehicle to the boundaries. Implement ele-vation, as it will be in most of the applications describedbelow, is controlled with relays, solenoids, and PWM gene-rators.

Information technology (IT) has enriched the corpus ofdata handled by intelligent machines, and consequentlyautoguidance is, far from an isolated and curious feature asin the beginning, another characteristic of a site model of thefield, where a path planning system is adapted to the model,the implement, and the operation [52]. The site modelmanages parameters such as yield, rainfall, prescriptions(rate, implement elevation, and application speed), boundaryinformation, and obstacles data. During normal operation,

the vehicles also stores data for the site model of the field,which is allocated in the site database, the operationsdatabase, or the implement database. Nevertheless, precise orsite-specific applications can manage without satellitepositioning in especially controlled atmospheres. Let us con-sider the case of a self-navigating robotic vehicle enclosed ina garden with undulated irregular terrain [53]. The vehiclecan follow a specified course by dead-reckoning if multiplenavigation beacons are placed all around the perimeter of thegarden. The navigating module can estimate position andorientation with accuracies of one inch by just processing theimages captured with a camera detecting the beams from the

beacons. The robot controls work implements withouhuman intervention according to a pre-stored scenario calledthe work order. The work order can include route, speedturning radii, implement control points, etc. This mode oestablishing a tasking protocol for the vehicle appliesindustrialization approaches to agricultural production. Thinvention proposed by Hanley [54] applies flexiblmanufacturing software, robotics, and sensing techniques to

agricultural operations. The idea is to move programmablworkstations typically used in manufacturing operations tofarm fields through mobile robots that can sense theienvironment and respond with the typical flexibility omanufacturing processes. The specific tasks considered arepruning, harvesting, weeding, trimming, planting, sprayingpollinating, or any operation on plants comprising a mobilerobotic machine with an arm and a decision making unibased on artificial intelligence. In this approach, a humanoperator is essential either locally or remotely, with someautonomy also available. Captivatingly enough, it also pointat the convenience of this application to specialty crops, aproduction niche that every day is demanding more robotisolutions to lower labor costs while augmenting selectivity

and adaptability, or in other words, intelligence.Aerial photographs and GIS-generated maps cannot catch

up with the dynamism involved in the active production ofarms run by precision agriculture principles. The ideasituation is mapping while working in such a way that the seof spatial variability maps are always being updated in reatime. This is the case of [55], where a supporting membewith a plurality of sensors is set to gather perceptioninformation in combination with the vehicles position tocreate a map of the field, as the presence or absence of weedat specific locations to build a weed-map. The mapping uniis connected to the control unit and a display via CANprotocol, so that speed and optical sensors are able to openand close spray nozzles at the right time. In addition toregulating chemicals or irrigation water, height control inimplements is crucial for automating farming operationsThe control circuit designed in [56] generates signals to raisand lower agricultural tools such as plows and combineheads. A height sensor is coupled to the control circuit tosense the implements height and therefore provide thefeedback signals required by a closed loop control. Thiautomatic control decreases operator workload while increasing accuracy in comparison to manual control. Obviouslythe higher precision in controlling the implement, the higheaccuracy in the operation, therefore it makes sense to invesin improving sensing and actuating technology as a means toenhance productivity. One way of reinforcing localization

quality is by coupling two positioning receivers with apower-actuated hitch to maintain an implement on course[57]. The hydraulic cylinder of the hitch is equipped with potentiometer for position feedback. Another way to meliorate the performance of automated tasks is by upgrading thpositioning technology. The system reported in [58integrates RTK-GPS with precision farming to provideaccurate seeding, cultivating, planting, or harvesting. Aseeder can be configured as a boom supporting one or moreGPS antennas. If the antennas are placed at either end of theboom, the elevation of each end can be estimated. Precisseeding with RTK-GPS may provide means for micro-crop

8/11/2019 0001 Meng

8/10


rotation: rotation of the position of crop plants within a smallarea of the field, considering cm-level accuracy. Theultimate step in reaching high accuracy with the implementactuation is by mounting the GPS antenna (RTK if feasible)directly on the farm implement, and coupling the antenna tothe receiver [59, 60]. The implement has the tendency todisplace to the side opposite the direction that the tractor issteered. This implement guidance system provides a fast

response for dynamically adjusting side-to-side positioningof the implement. The automatic hitch includes an actuatorand a position sensor. The actuators are dual hydraulicdevices for pushing and pulling the tongue of the hitch. Thecontrol computer reads the geographic information of thepath from memory and determines the cross track error ofthe implement rather than the vehicle. The authors claim thatthis farming system corrects the location of the implement inless than one half the time or distance required with ordinaryon-vehicle guidance. The computer handles two paralleloperations: vehicle guidance and implement guidance. Analternative, non-GPS related, attention-grabbing control of aguided implement with extending arms for spraying theground is illustrated in [61]. This technique uses fluorescent

chemicals to mark regions, to be exact 4-6 inches widebands, and determine the position of the implement to keep iton course. The objective is to avoid overlapping andunderlapping of chemicals. The spraying boom requires anozzle for depositing a band of fluorescent chemical, whichin turns will be excited by a light source. A hydraulic motorgoverned by a proportional valve brings the implement ontrack. This system can control large implements with boomsover 120 feet.

7. SYSTEM RELIABILITY AND SAFEGUARDING

Safeguarding is probably the factor that has delayed thedesired transformation from concept to product the most.

Rural environments, where heavy and powerful machinesregularly operate, are potential candidates for mortalaccidents. Leading manufacturers do not want to have theirreputation damaged and consequently tend to be conser-vative in the implementation and commercialization of newtechnology, mainly with regards to vehicle automation.Errors accepted to drivers are not accepted to machineswhich usually need to outperform skilled operators.

The wide scope of onboard safety systems ranges frombasic semi-autonomous applications to sophisticated roboticsolutions. The invention covered in [62] is designed to aidhumans in the safe use of equipment by deploying a messag-ing network and monitoring remote control operations. Thesystem includes a sensory device for monitoring and

reporting data, a memory device, at least one processor, anda plurality of external devices such as pagers and cell phonesto communicate via security protocol with encryptiontechnology. A probable hazard for an automatically guidedvehicle performing a task in the field is the presence of anunexpected obstacle interfering with its trajectory. Thenatural reaction for the vehicle would be to detect theobstacle and move around it if there is enough space. Inorder to do so, the controller must contain an obstacleavoidance algorithm. The point-to-point path planning of[48] is capable of detecting obstacles and defining aboundary zone about each obstacle. The obstacle clearance

zone is defined by modeling a perimeter as a percentageaugmented periphery and is based on such vehicle constraints as width, wheelbase, and turning radius. The planneidentifies candidate paths and eliminates those colliding withobstacles. The number of touches of the obstacle zones ilimited. Another method for getting information abouobjects in an environment around a vehicle uses infrared (IRlight to create a 3D representation of the surroundings based

on the measured distances to the objects [63]. The securitysystem can be amplified with the addition of an ultrasonicsystem in an optical-acoustical combination. Dangerousituations are warned by turning on the hazard lights, sounding the horn, or similar actions. A remarkable advantage othis system is that even in the presence of strong sunlight, anarrow beam of IR laser can be observed if using a narrownotch frequency filter on the received light. This embodiment is recommended for air seeding systems, precisionplanters, and sprayers. When the vehicle approximates toomuch to an interfering obstacle, stronger warning systemneed to be activated. The anticipatory sensor system of [64informs that an impact involving the vehicle is about tooccur unless a corrective action is taken. This action is based

on data obtained prior to the impact. Potential applications othe system include protection to pedestrians, lane departurewarning signaling, and night driving assistance. Laserscameras, GPS, and IMU are some of the sensors employed topresage an impact. In its adaptation to agricultural vehicle[65], a ground speed sensor is mounted onboard to calculatewheel slip of the tractor or sprayer. The obstacle detectionalgorithm utilizes pattern recognition techniques to identifythe dangerous objects [66], and some of the habituaprocessing methods are neural networks, Fourier analysis, ofuzzy logic. In addition to reactive trajectory re-planninginduced by local perception sensors, when global positioningis part of the navigation system, reliability can also beimproved by tracking a diagnostic function to ensure that th

navigation system is working properly [14]. This functioncan include the number of satellites, quality indices such asde dilution of precision, CPU load, or even the solenoidresponse.

8. CURRENT & FUTURE DEVELOPMENTS

In spite of more than two decades of intense and fertiledevelopment of automatic guidance technology, we canconsider this technology a young technology from thecustomer acceptance standpoint; if it is true that some earlyadopters are regularly using some of the few commerciasystems available in the market, the majority of professionaproducers still remain stuck to conventional systems and

procedures. There is still a long way to go, although usergrow year after year and the trend seems to be favorable. Thincorporation of new satellite systems to the arena of GNSSsuch as GLONASS and Galileo, will probably lead to awider use, perhaps universal, in agriculture, forestry, andconstruction. The side effect will certainly be the increase incomplexity of receivers which will need to accept differensources of signals, but on the other hand, reliability will alsobe enhanced as redundant signals result in safer outcomesThis GNSS augmentation will have direct repercussions onprecision agriculture future paths as spatial variability will bbetter tracked with the new localization data abundance

8/11/2019 0001 Meng

9/10


Western countries seem to find troubles with conventionalagriculture while developing countries need technology for asustainable production; precision farming appears as afactual solution to assist both needs. The incorporation ofthese technological advances, however, will require severalsteps. Hardware and software reliability needs to beguaranteed for the harsh environments found in agriculturalfields. Extension personnel should smooth the transfer from

research laboratories to farming sites. Manufacturers, inconjunction with dealerships, have to be prepared to instructand assist their customers in the operating techniques anddata handling.

The analysis of the technological evolution aroundintelligent vehicles can bring some insights to predict whatmight occur in the next decade. The bar chart of Fig. (3)represents the weight of key features found in automaticguidance patents according to their period of publication:1985-2000 and 2001-2008. It is apparent in the graphic thatbeacons, pseudolites, and non-vision local sensors haveexperienced a recession in favor of computer vision systemsand algorithms, probably motivated by the significant growthin computer power. Inventions related to precision agri-culture were more numerous in the first period, which issurprising because selective availability disappeared after2000. It may be possible, though, that the loss in GPS signalsrestrictions has weakened research devoted to compensatefor signal degradation. Nevertheless,the applications of GPSsystems, as expected, have increased significantly, with 80%of the patents citing this technology being published after

2000, and 90% of the inventions dealing with paralletracking also belonging to the period 2001-2008. It is noteworthy the fact that most of the semi-autonomous applications reported in the reviewed patents have been developedin the second period, and lightbars guiding assistance idominant in the second period as well. After the oneiric starof vehicle automation where some people envisaged farms ocrowded machines multitasking on their own, roboticists and

engineers found a modest commencement based on thedeployment of semi-autonomous equipment a more reasonable approach for consolidating this technology. Guidancevisual assistance through lightbar displays and straight-alonautopilots turned out to be grounded solutions to begin withHeadland turns and field-filling complete trajectories are inthe agenda but still need the support of highly safe softwareand hardware. In light of these evidences, the coming yearswill probably see a continuation of GNSS applications withbetter signals and filters, a perfection of imaging sensors, andthe expected enhancement of reliability and safety. Sensofusion of global and local information might also step up.

The Grand Challenge competition demonstrated that thepossibility of endowing a vehicle with advanced intelligenceis realistic; the required technology is available and reasonably reachable. Yet the competition was a giant laboratorywhere high-tech prototypes were defied, and it will takesome time for manufacturers to incorporate automatic guidance kits in their mass-produced models. Some of the initiaproblems such as sensor accuracy, processing speed, or cosare being fixed with time; but reliability is still an unsolved

Fig. (3).Technological evolution of auto-guidance for the periods 1985-2000 and 2001-2008.

8/11/2019 0001 Meng

10/10


problem. For this reason, perceptual devices will expandtheir horizons, and more sophisticated sensors as stereo-scopic vision heads or 3D laser scanners, perhaps borrowedfrom space technology, will join the already numerousfamily of hardware integrated in intelligent vehicles. In anycase, the essence of a vehicles behavior, its intelligence afterall, will be dictated by the onboard algorithms and program-med conducts, and from that stand-point, significant advan-

ces are expected in the philosophical conception of thevehicles brain.

ACKNOWLEDGEMENTS

I would like to express my most earnest gratitude to JohnReid, Qin Zhang and Shufeng Han for being the individualswho introduced me into this fascinating field, taught me theessentials, and motivated me to continue within its boun-daries and even transcend them.

CONFLICT OF INTEREST

None known.

REFERENCES

[1] Reid JF, Zhang Q, Noguchi N, Dickson M. Agricultural automaticguidance research in North America. Comput Electron Agr 2000;25: 155-167.

[2] Shirley R. Ventura Countys Changing Agriculture. Resource;February 2008: 5-8.

[3] Gibbs WW. Innovations from a robot rally. Sci Am 2006; 294(1):64-71.

[4] Pryor, T.R.: US4769700 (1988).[5] Dyke, H.L.: US4700301 (1987).[6] Barnes, R.L., Gary, D.A.: US5121799 (1992).[7]

Rocks, J.K.: US5974348 (1999).

[8] Hale, G.H., Wendte, K.W.: US5870689 (1999).[9] Misra P, Enge P. Global Positioning System. Signals, Measure-

ments, and Performance. Ganga-Jamuna Press: Lincoln, Mass, USA2006.

[10]

Rovira-Ms F, Han S, Reid JF. Trajectory-based algorithm for thequantitative evaluation of automatically steered vehicles. Trans

ASABE 2008; 51(5): 1545-1552.[11] Goodwin, J.L., Hartt, J.R., Vetter, A.A., Walton, J.T., Bletzacker,

F.R., Forgette, T.M., Hughes, A., Neuperger, F.H., Falkenberg,W.H.: WO9515499 (1995).

[12] Mullins, D. B.: US6377881 (2002).[13] Parkinson, B.W., O'Connor, M.L., Elkaim, G.H., Bell, T.:

US20006052647 (2000).[14] Mailer, R.L.: US6876920 (2005).[15] Nichols, M.E.: US6501422 (2002).[16] Lindores, R.J.: US7363154 (2008).[17] Lindores, R.J.: US20070083299 (2007).[18] Keller, R.J., Nichols, M.E., Lange, A.F.: US6199000 (2001).[19]

Rovira-Ms F, Han S, Wei J, Reid JF. Fuzzy logic model for sensorfusion of machine vision and GPS in autonomous navigation.ASABE Annual Meeting 2005: Paper No. 051156.

[20] Anderson, N.W., Thackray, R.G.: US7064810 (2006).[21] Anderson, N.W., Thackray, R.G.: US7206063 (2007).[22]

Margolis, D.L., Yasui, Y.: US5357432 (1994).

[23] Stentz, A., Hoffman, R., Ollis, M., Whittaker, W., Fitzpatrick, KEP0801885 (2002).

[24] Stentz, A., Ollis, M., Fitzpatrick, K., Hoffman, R., Whittaker, WUS5911669 (1999).

[25] Dickson, M.A., Ni, B., Han, S.: US6285930 (2001).[26] Dickson, M.A., Ni, B., Han, S., Reid, J.F.: US6385515 (2002).

[27] Rovira-Ms F, Zhang Q, Reid JF, Will JD. Hough-transform-based

vision algorithm for crop row detection of an automated agriculturavehicle. J Automobile Eng 2005; 219: 999-1010.

[28] Benson, E.R., Zhang, Q., Reid, J.F.: US6721453 (2004).[29] Benson, E.R., Zhang, Q., Reid, J.F.: US6819780 (2004).[30] Hofer, J., Bohner, M., Freichel, T.: US7400957 (2008).

[31] Weinstein, L., Sinclair, K., Gainsboro, J., Willison, PWO2005084155 (2005).

[32] Frank, A.A., Nakamura, M.: US5202742 (1993).[33] Coker, W.B.: US5410479 (1995).[34] Dickson, M.A., Ni, B., Han, S.: US6686951 (2004).[35] Dickson, M.A., Ni, B., Han, S.: US6278918 (2001).[36]

Dickson, M.A., Ni, B., Han, S.: US6490539 (2002).[37] Margolis, D., Yasui, Y.: US5390118 (1995).[38] Pangels, H., Pilarski, T., Fitzpatrick, K., Happold, M., Ollis, M

Whittaker, W., Stentz, A.: US6336051 (2002).[39] Brooks RA. Cambrian Intelligence. The Early History of the New

AI. The MIT Press: Cambridge 1999.

[40]

Motz, D.S., Saxsma, A.L., Staub, M.D.: US6738695 (2004).[41] Dickson, M.A., Noguchi, N., Zhang, Q., Reid, J.F., Will, J.D.

US6445983 (2002).[42]

Harrison, Jr. J.H., Harrison, H.J.: US6925380 (2005).

[43] Harrison, Jr. J.H., Harrison, H.J.: US20060089796 (2006).[44] Nichols, M.E.: US6703973 (2004).[45] Gray, S.A., Hansen, S.L., Flann, N.S.: US6907336 (2005).[46] Gray, S.A., Hansen, S.L., Flann, N.S.: US7110881 (2006).[47] Flann, N.S., Hansen, S.L., Gray, S.A.: US7216033 (2007).[48]

Flann, N.S., Hansen, S.L., Gray, S.A.: US7079943 (2006).

[49] Flann, N.S., Hansen, S.L., Gray, S.A.: US7228214 (2007).[50] Overschie, J.: US7277792 (2007).[51] Hale, G.H., Wendte, K.W.: US5978723 (1999).[52] Motz, D.S., Staub, M.D.: US6236924 (2001).[53] Rocks, J.K.: US5974348 (1999).[54] Hanley, B.P.: US6671582 (2003).

[55] Beck, J.L., Kinter, M.L.: US5809440 (1998).

[56]

Hale, G.H., Wendte, K.W., Orbach, A.: US5961573 (1999).[57] Mcclure, J.A., Heiniger, R.W., Timm, J.T.E., Funk, K.D., Wong

R.B.: US7373231 (2008).[58] Keller, R.J., Nichols, M.E., Lange, A.F.: US6199000 (2001).[59] Lange, A.F., Funk, T.J.: US7054731 (2006).

[60] Lange, A.F., Funk, T.J.: US7383114 (2008).[61] Sauter, G.F., Nelson, G.F., Andersen, D.P., Paulinski, D.W

US5636792 (1997).[62] Walker, R.C.: WO01054044 (2001).[63] Breed, D.S., DuVall, W.E., Johnson, W.C.: US7209221 (2007).[64] Breed, D.S.: US20080040004 (2008).[65]

Breed, D.S.: US20050278098 (2005).[66] Breed, D.S., DuVall, W.E., Johnson, W.C.: US2007018252

(2007).

0001 Meng

Documents

Transcript of 0001 Meng