Automatic Generation of Artistic Chinese Calligraphyfcmlau/papers/autocalli.pdfappealing Chinese...

8
32 1541-1672/05/$20.00 © 2005 IEEE IEEE INTELLIGENT SYSTEMS Published by the IEEE Computer Society C o n s t r a i n t - B a s e d R e a s o n i n g Automatic Generation of Artistic Chinese Calligraphy Songhua Xu, Zhejiang University Francis C.M. Lau, University of Hong Kong William K. Cheung, Hong Kong Baptist University Yunhe Pan, Zhejiang University C hinese calligraphy is among the finest and most important of all Chinese art forms and an inseparable part of Chinese history. Its delicate aesthetic effects are gen- erally considered to be unique among all calligraphic arts. Its subtle power is integral to traditional Chinese painting, where—as figure 1a shows—calligraphy is not just an A novel intelligent system uses a constraint-based analogous-reasoning process to automatically generate original Chinese calligraphy that meets visually aesthetic requirements. annotation but also a stylized visual component affecting the viewer’s emotional response to a paint- ing. This emotional effect also explains why, as fig- ure 1b shows, calligraphy is often preferred to printed type in Asian banners, signage, newspaper mast- heads, and other promotional contexts. The 2008 Beijing Olympics Games logo (http://en.beijing- 2008.org) is a recent example. Chinese calligraphers predominantly use a soft hair brush. Generating artistically appealing callig- raphy with the brush can be highly challenging. The brush-stroke shapes as well as the topology over mul- tiple strokes are often very complex. The Chinese language’s large character set—more than 3,000 commonly used characters—presents a problem all its own. Being able to master some of the characters doesn’t mean that you can also write the other char- acters as satisfactorily. Similarly, mastery of one or more styles doesn’t necessarily indicate mastery in other styles, let alone creativity in generating new styles. This is where computers can help. In the digital world, calligraphic art is most often applied to creating typographic or artistic fonts for printing or display; Donald Knuth has done pioneer- ing work in this area. 1 We propose an intelligent sys- tem that can automatically create novel, aesthetically appealing Chinese calligraphy from a few training examples of existing calligraphic styles. To demon- strate the proposed methodology’s feasibility, we have implemented a prototype system that automatically generates new Chinese calligraphic art from a small training set—typically, fewer than 10 samples for each character. To the best of our knowledge, no other pub- lished work uses our approach. One remotely related project uses analogous reasoning to simulate the cre- ativity in jazz performance and to model other artis- tic activities from the simulation. 2 Overall approach and system architecture Let P denote a model with a parameterization E that is flexible enough to represent a class of highly deformable shapes—different Chinese character styles, in our case. Normally, constructing a flexible model requires significant effort. At the same time, arbitrary instantiations from such models can easily

Transcript of Automatic Generation of Artistic Chinese Calligraphyfcmlau/papers/autocalli.pdfappealing Chinese...

Page 1: Automatic Generation of Artistic Chinese Calligraphyfcmlau/papers/autocalli.pdfappealing Chinese calligraphy from a few training examples of existing calligraphic styles. To demon-strate

32 1541-1672/05/$20.00 © 2005 IEEE IEEE INTELLIGENT SYSTEMSPublished by the IEEE Computer Society

C o n s t r a i n t - B a s e d R e a s o n i n g

Automatic Generationof Artistic ChineseCalligraphySonghua Xu, Zhejiang University

Francis C.M. Lau, University of Hong Kong

William K. Cheung, Hong Kong Baptist University

Yunhe Pan, Zhejiang University

Chinese calligraphy is among the finest and most important of all Chinese art forms

and an inseparable part of Chinese history. Its delicate aesthetic effects are gen-

erally considered to be unique among all calligraphic arts. Its subtle power is integral to

traditional Chinese painting, where—as figure 1a shows—calligraphy is not just an

A novel intelligent

system uses a

constraint-based

analogous-reasoning

process to

automatically generate

original Chinese

calligraphy that meets

visually aesthetic

requirements.

annotation but also a stylized visual componentaffecting the viewer’s emotional response to a paint-ing. This emotional effect also explains why, as fig-ure 1b shows, calligraphy is often preferred to printedtype in Asian banners, signage, newspaper mast-heads, and other promotional contexts. The 2008Beijing Olympics Games logo (http://en.beijing-2008.org) is a recent example.

Chinese calligraphers predominantly use a softhair brush. Generating artistically appealing callig-raphy with the brush can be highly challenging. Thebrush-stroke shapes as well as the topology over mul-tiple strokes are often very complex. The Chineselanguage’s large character set—more than 3,000commonly used characters—presents a problem allits own. Being able to master some of the charactersdoesn’t mean that you can also write the other char-acters as satisfactorily. Similarly, mastery of one ormore styles doesn’t necessarily indicate mastery inother styles, let alone creativity in generating newstyles. This is where computers can help.

In the digital world, calligraphic art is most oftenapplied to creating typographic or artistic fonts for

printing or display; Donald Knuth has done pioneer-ing work in this area.1 We propose an intelligent sys-tem that can automatically create novel, aestheticallyappealing Chinese calligraphy from a few trainingexamples of existing calligraphic styles. To demon-strate the proposed methodology’s feasibility, we haveimplemented a prototype system that automaticallygenerates new Chinese calligraphic art from a smalltraining set—typically, fewer than 10 samples for eachcharacter. To the best of our knowledge, no other pub-lished work uses our approach. One remotely relatedproject uses analogous reasoning to simulate the cre-ativity in jazz performance and to model other artis-tic activities from the simulation.2

Overall approach and system architecture

Let P denote a model with a parameterization Ethat is flexible enough to represent a class of highlydeformable shapes—different Chinese characterstyles, in our case. Normally, constructing a flexiblemodel requires significant effort. At the same time,arbitrary instantiations from such models can easily

Page 2: Automatic Generation of Artistic Chinese Calligraphyfcmlau/papers/autocalli.pdfappealing Chinese calligraphy from a few training examples of existing calligraphic styles. To demon-strate

generate unacceptable results. Thus, a model-based approach to generating novel and yetaesthetically appealing calligraphy is by nomeans straightforward. Our approach uses aconstraint-based analogous-reasoning process(ARP), which we apply to a given set of train-ing examples. Analogous reasoning basicallyfuses knowledge from multiple sources to sup-port a restricted form of reasoning.3 In ourcase, the knowledge sources are trainingexamples, which are in the form of images.

In our experiments using the prototypesystem, the training examples come fromprinted “copybooks” that present multiplecalligraphic styles. Because Chinese charac-ters derived from pictographs, which evolvedover time into symbols, many basic featuresrecur in different Chinese characters. To takeadvantage of this redundancy, we devised ahierarchical representation for Chinese char-acters as the basis for our process. The pro-posed ARP consists of three major phases:

• Shape decomposition. Decomposing (orrecovering) the calligraphic shapes of agiven training example is equivalent to theproblem of extracting structural featuresfor constructing a reference model P. Thereference model is an instance of themodel P that best represents the inputexample. The underlying mechanism inour approach is character stroke segmen-tation and extraction.

• Calligraphy model creation from exam-ples. Given n reference models {Pi},where i is the index of the reference modelconstructed from a set of training exam-ples, we can define a family of novelshapes P(�) by blending the referencemodels: first, we identify the correspond-ing structural features among the reference

models; then, we combine the alignedmodels by interpolating or extrapolatingthe parameterizations {E}. The newlyderived shape family is essentially a “re-parameterization” via the blending para-meters, �, which control the contributionof each training example.

• Artistic calligraphy generation. GivenP(�) and a set of aesthetics-related geo-metrical constraints, we identify some �that satisfies the given constraints.

Figure 2 shows the overall architecture of theproposed intelligent calligraphy generationsystem.

Character representationOur proposed system decomposes Chi-

nese calligraphy into the six levels shown infigure 3a: constructive ellipse, primitivestroke, compound stroke, radical, singlecharacter, and complete artwork. We adoptedparametric representations at all levels. Taken

Figure 1. Artistic Chinese calligraphy in Asian societies: (a) Chinese painting with calligraphy; (b) top—ceiling of a Kong Zi (Confucius) temple; bottom—masthead ofthe China Daily newspaper.

(a) (b)

Input images ofsample calligraphy

from copybook

Images ofautomatically

generated artisticcalligraphy

Shape recoverythrough vision

techniques

Hierarchicalrepresentation ofsource calligraphy Calligraphy model

generation throughanalogous reasoning

Hierarchicalrepresentation of newcalligraphy candidates

Rendering

Hierarchicalrepresentation ofgenerated artistic

calligraphy

Appreciation throughconstraint

satisfaction

Structural strokedatabase

Calligraphy constraints

Figure 2. System architecture for intelligent calligraphy generation.

MAY/JUNE 2005 www.computer.org/intelligent 33

Page 3: Automatic Generation of Artistic Chinese Calligraphyfcmlau/papers/autocalli.pdfappealing Chinese calligraphy from a few training examples of existing calligraphic styles. To demon-strate

together, these representations form the para-meter space E for modeling Chinese callig-raphy artwork.

Prototype implementationIn the prototype system, we’ve imple-

mented five typical, frequently occurringprimitive strokes: points, horizontals, verti-cals, left slants, and right slants. Figure 4shows these strokes as well as 24 typical, fre-quently occurring compound strokes and 36radicals. Figure 3b shows the hierarchical rep-resentation of the Chinese character “zhe,” asin “Zhejiang,” a scenic coastal province andthe home of Zhejiang University.

Level 0 of the hierarchical representationviews an artwork as a collection of con-structive ellipses (see figure 3a). The systemwill render the artwork’s “image” as theimage space regions that the constructiveellipses cover. This representation is inspiredby the Blum model,4 which defines a zonaryarea by an ellipse moving along a predefinedcurve. A 4 � 1 matrix parameterizes eachconstructive ellipse: two rows store the coor-dinates of the constructive ellipse’s center,

and the other two rows store the lengths ofits major and minor axes.

Traversing the hierarchy from the bottomup, the system first “lines up” the construc-tive ellipses to form primitive strokes (level1). Then, using shape grammar rules, it com-bines primitive strokes to form compoundstrokes (level 2), which are subsequentlycombined to form radicals (level 3). Bygrouping radicals according to their spatialproximity, the system forms single charac-ters (level 4). It blends learned examples ofthe same character in different styles into aflexible character model.

Finally, the top-level constructive elementis calligraphy artwork (level 5), which mightcombine more than one character.

Advantages of the hierarchicalparametric approach

Because our approach generates new cal-ligraphic styles by reasoning from a set ofexisting styles, it belongs to the hard domainof qualitative reasoning. Our parametricrepresentation offers a tool to attack thechallenging qualitative-reasoning problem

through quantitative means—analogousreasoning together with aesthetic constraintsatisfaction.

Hierarchical representation enables effi-cient local learning of constructive elementsand reduces the huge global-knowledge representation space to only local shape-variation characterizations. It also supportsefficient retrieval (and thus reuse) of pastcalligraphy artwork reasoning results.

The hierarchical parametric approachcan represent all calligraphy styles—including cursive styles that are heavilydeformed and distorted—in a uniform six-level hierarchy, and it can process the char-acters using the same reasoning pipeline.This increases our system’s capability tolearn and generate cursive calligraphy,which is an important aspect of calligraphicaesthetics.

Shape decompositionIn the first of its three phases, our system

extracts hierarchical and parametric represen-tations from the knowledge sources, which arestatic images of calligraphy artwork.

P2, 2 P2, 8P2, 3 P2, 9P2, 1 P2, 7P2, 4 P2, 6

P1, 2 P1, 3 P1, 10 P1, 11P1, 1 P1, 4

P2, 5

P1, 5 P1, 6 P1, 7 P1, 8 P1, 9

(a) (b)

Single-character level4th Single characters

3rd Radicals

5th Calligraphy artwork

2nd Compound strokes

0th Constructive ellipses

1st Primitive strokes

Radicallevel

Compound-stroke level

Primitive-stroke level

P4, 1

P3, 1 P3, 2 P3, 3

Figure 3. Chinese calligraphy representation: (a) six-level representation hierarchy; (b) four levels in the representation for the Chinese character “zhe.”

C O N S T R A I N T - B A S E D R E A S O N I N G

34 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

Page 4: Automatic Generation of Artistic Chinese Calligraphyfcmlau/papers/autocalli.pdfappealing Chinese calligraphy from a few training examples of existing calligraphic styles. To demon-strate

Extracting levels 0–1 elementsTo extract primitive strokes and thus the

corresponding constructive ellipses from atraining example, we first compute the inputimage’s skeleton—that is, a close approxi-mation to the brush’s actual trajectory whenthe calligrapher created the artwork.

Various approaches exist for skeletoniz-ing binary character images. We employedthe algorithm proposed by Rong He andHong Yan,5 which composes the extractedskeleton from segmented primitive strokes.Figure 5 gives a step-by-step illustration.Such a stroke decomposition is by no meansoptimal, and our system can benefit from anyimproved decomposition algorithm.

To further enhance the robustness of thestroke identification step, the system usesseveral structural variants of the five prim-itive strokes in figure 4. Once it identifiesthe primitive stroke skeletons, the systemuses Bresenham’s ellipse rasterization algo-rithm6 to compute all the constructiveellipses.

Extracting levels 2–3 elementsWe identify compound strokes and radi-

cals by analyzing the spatial relation betweenthe primitive and compound strokes, respec-tively. The analysis uses carefully designedshape grammar production rules. The syn-tactic description of any constructive elementis represented using the production system’ssyntax and generated using rule deduction.

As an example, the shape production rulefor the compound stroke in the upper left-most corner of figure 4b, denoted as CS1, isas follows:

IF horizontal(a) AND vertical(b) AND ontop(a,b) AND onleft(a,b) ANDtouch(a,b) THEN CS1 := {a,b}

where horizontal(a), vertical(b), ontop(a,b), onleft(a,b),and touch(a,b) are the predicates indicating therelationships of horizontal primitive stroke,vertical primitive stroke, a on top of b, a onleft side of b, and {a,b} touching each other,respectively.

We use fuzzy set theory to increase theextraction process’s reliability.7 This approachlets us associate a confidence value with eachshape production rule via the deductionprocess. The system derives an overall confi-dence value for the shape grammar produc-tion from the confidence values of all its state-ments. Then it applies the rule that yields the

highest overall confidence for the corre-sponding stroke composition.

Extracting level 4 elementsTo extract the level 4 constructive ele-

ments, we must determine which radicals canbe grouped to form a character. This is equiv-alent to the well-known “character segmen-tation” problem in pattern recognition. In oursystem, we use standard projection analysis8

to segment the individual characters in a cal-ligraphy artwork.

Calligraphy model creation from examples

To generate new calligraphic art, we applyanalogous reasoning to a training set of dif-

ferent calligraphic styles. Our approach hasroots in Herbert A. Simon’s notion of artisticdesign generation and synthesis, discussed in1975,3 and in Mark Keane’s later applicationof analogical mechanisms to problem solv-ing.9 In general, we can view ARP as aprocess that synthesizes new knowledge—shapes, in our case—by fusing or blendingwith existing, independent knowledgesources—that is, the training examples. Tosupport the fusion, the process must establishfeature correspondence between the knowl-edge sources.

In principle, we can apply ARP at differ-ent hierarchical levels, resulting in differentartistic effects. Assume that we apply it toPk,m, the mth constructive element at the kth

Figure 4. (a) Five primitive strokes, (b) 24 compound strokes, and (c) 36 radicals.

(a) (b) (c)

Figure 5. Extracting levels 0–1 elements of a character from its image: (a) the inputimage, (b) the raw skeleton computed using a thinning algorithm, (c) the plausibleshort branches detected and color marked, (d) the short branches identified andremoved, (e) the skeleton segmented into different strokes in the character and colorcoded, and (f) the reconstructed character using the extracted levels 0–1 elements.Note that the reconstructed image (f) and the original image (a) have slightdifferences at the ends of some strokes, as shown in the red rectangles.

(a) (b) (c)

(d) (e) (f)

MAY/JUNE 2005 www.computer.org/intelligent 35

Page 5: Automatic Generation of Artistic Chinese Calligraphyfcmlau/papers/autocalli.pdfappealing Chinese calligraphy from a few training examples of existing calligraphic styles. To demon-strate

level of the hierarchical representation, andthat n different versions of Pk,m exist, P1

k,m, …,Pn

k,m, derived from n training examples—theindependent ARP knowledge sources. TheARP result is denoted as Pr

k,m. We can thenstate the general mathematical principle:

where �i(i = 1, …, n) is defined as the anal-ogous reasoning intensity for Pi

k,m with theconstraint that �n

i=1�i = 1.

Obviously, a Pik,m with a higher �i value

would contribute more to the overall reason-ing result.

We can interpret the proposed ARP aseither an interpolation or an extrapolationprocess. We’re assuming a one-to-one corre-spondence among different versions of Pk,m.In reality, a constructive element, such as aconstructive ellipse or a primitive stroke,derived from a training example can corre-spond to one element in another trainingexample in more than one way. The processtherefore requires a feature correspondencestep before it can blend together featuresextracted from the different examples. In ourintelligent calligraphy generation system, theuser can either adjust all the analogous rea-soning intensities manually through a graph-ical interface or generate them randomly. Theintensities are fed to a subsequent phase thatautomatically filters out those that violateaesthetic constraints.

Fusing knowledge sources in ARPTo establish the feature correspondence

between training examples, we first derive adiscrete planar curve for each constructiveelement Pi

k,m using the centers of all the con-structive ellipses associated with it. Thecurve forms the element’s skeleton, and weuse Pengfei Zhu and Paul Chirlian’s algo-rithm10 for detecting the critical points on theplanar curves as the key points for setting upthe correspondence. Details of this processare available online at http://pantheon.yale.edu/~sx25/ca.

In our application, we first assume theshape of a constructive element in the fontstyle “Kai” (the GB2312 character set usedin the recent version of Microsoft Word) to bethe standard reference, which we denote asPstd

k,m. Because the shape decompositionphase has already extracted the Pk,m’s shape,we can easily compute the deviation Fi

k,m bywhich the shape of the ith source Pi

k,m differs

from that of Pstdk,m, as follows:

Fik,m � Pi

k,m � Pstdk,m

After computing all the deviations of thereasoning sources F1

k,m, �, Fnk,m, we can then

derive the overall deviation Frk,m as follows:

Frk,m = ∅(F1

k,m, �, Fnk,m, �)

where ∅ is defined as the analogous reason-ing mechanism simulation operator, whichour prototype system implements as an inter-polation or extrapolation process, and � isthe aesthetic viewpoint sequence dictatingthe weights and contribution order for dif-ferent sources. The ordered set of intensitiesis ARP’s viewpoint sequence. So not only dif-ferent reasoning intensities affect the finaloutput, but also different orders of the train-ing examples.

Finally, by adding back the Pstdk,m shape

(that is, the standard constructive elementassociated with ARP’s reasoning result Pr

k,m),we obtain

Prk,m = Fr

k,m ⊕ Pstdk,m

where the operator ⊕ is a reverse function ofthe operator �.

Note that the system can apply ARP notonly to the constructive elements from all thereasoning sources but also to some topolog-ical constructors in the form of geometrictransformation matrices for the correspond-ing constructive elements. The topologicalconstructors further increase the reasoningpower. Simple ARP simulation operators forthe topological constructors include arith-metic mean, geometric mean, and harmonicmean.

A computational-psychology perspective

If all the knowledge sources’ intensitiesfall within (0, 1), ARP is simulated using aninterpolation process; otherwise, it’s simu-lated using an extrapolation process. From apsychological point of view, the negative val-ues for the reasoning intensities reflect thebrain’s inverse reasoning activities, whichtreat certain input source knowledge as neg-ative examples.

By contrast, positive values correspond toexaggeration of brain activities, in which thelarger an input example’s reasoning inten-sity, the more heavily the generated resultwill follow the input source knowledge’s

style. When the number of knowledgesources is greater than two, ARP mimicscombined thinking activity, which will makeuse of several knowledge reference casesduring the reasoning process.

Artistic calligraphy generationWith the mechanisms of calligraphic

shape decomposition and model creationfrom examples in place, we can easily gen-erate candidates for original calligraphy art-work by randomly perturbing the reasoningintensities. The analogous reasoning stepsapply equally as well to a single parameterof a constructive ellipse as they do to all theparameters of a character. This provides ahighly flexible system for varying shapes togenerate many possible candidates.

A filtering step ensures that only candi-dates meeting aesthetic requirements areoutput.

Extracting aesthetic constraintsfrom training examples

Aesthetic constraints are criteria that quan-tify the aesthetic quality of a candidate or itsparts. There are two categories:

• constraints for the visual appearance of aconstructive element, and

• constraints for the spatial relationshipbetween neighboring constructive ele-ments.

Fortunately, the proposed ARP can auto-matically guarantee satisfaction of the firstconstraint—at least under most circum-stances—because the constructive elementsare parametric. Therefore, we need focusonly on deriving and applying constraints ofthe second type to guarantee the generationof visually pleasing calligraphy.

An important consideration for a quantifi-able constraint on aesthetics is the degree ofoverlap between two constructive elements.Our system uses three types of overlappingbetween a pair of elements, a and b: the x-dimensional overlapping ϑx(a, b), the y-dimensional overlapping ϑy(a, b), and thearea overlapping ϑs(a, b). All three measuresare based on the constructive element’sbounding boxes. After computing the over-lapping measures for all element pairs of thetraining examples, the system records theirupper and lower bounds. The upper boundkeeps two neighboring elements in a new cal-ligraphy artwork from being squeezed tooclosely together while the lower bound keeps

P Pk mr i

k mi

i

n

, ,==∑ω

1

C O N S T R A I N T - B A S E D R E A S O N I N G

36 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

Page 6: Automatic Generation of Artistic Chinese Calligraphyfcmlau/papers/autocalli.pdfappealing Chinese calligraphy from a few training examples of existing calligraphic styles. To demon-strate

them from being too far apart. These over-lapping measures then direct the process ofgenerating the upper-level constructive ele-ments from the lower-level ones.

The overall effect is to constrain ARPfrom perturbing the spatial relationships ofeach constructive element’s subcomponentstoo much beyond what they are in the train-ing examples. Thus, for example, if ARPgenerates a calligraphy candidate that con-tains subconstructive elements whose x-dimensional overlapping is smaller than thederived lower bound, the prototype systemwill reject the candidate.

If needed, end users can relax the upper-and lower-bound constraints to allow for newstyles that cannot be easily imagined. In oursystem, users can adjust the two boundsinteractively according to their preferences.Thus, choosing the best values for the twobounds becomes a matter of personal aes-thetic taste. According to our experience withthe prototype system’s behaviors, relaxing orignoring the ARP constraints seems to cre-ate a more cursive and running-style writing.

Using past results for efficient reasoning

You could choose a random process to sim-ulate ARP, but this might be computationallyintensive. Reusing past results can make thereasoning process more efficient. In addition,the hierarchical representation lets users reusewhole or partial results. Our proposed systemtherefore offers a high degree of reusabilityof past reasoning results.

Experimental resultsFigure 6 shows the results our prototype

system obtained using six training examplesas the input knowledge sources and linearinterpolation to simulate the ARP generationstep. Figure 7 shows the results of using fivetraining examples and a nonlinear interpola-tion process. In the latter case, you can easilysee the consistency in style among characterswithin the same newly generated calligraphicpiece. Space limitations restrict the resultsshown to a small set; more results are avail-able at http://pantheon.yale.edu/~sx25/ca.

The results show that our approach canyield novel calligraphy styles given someexisting ones. To verify that the system wasgenerating quality outputs, we asked practic-ing artists, art school professors, and calligra-phy enthusiasts to examine the outputs; mostof them considered the generated calligraphyto be novel with regard to their writing style.

In addition, we analyzed the system’s sen-sitivity in terms of its “creativity” by vary-ing the number of training examples. In thisexperiment, we used training examples intraining examples of different calligraphystyles, including Kai, Li, Xingshu, Weibei,

Xingkai, Xingchao, and Kuangchao. Weinvited six calligraphic fans with at least twoyears’ writing experience and four profes-sional calligraphists to form a review com-mittee. They cast votes on the system-gener-ated calligraphic art. If a result received more

Figure 6. A single character in many styles; the first row is the training examples, andthe other rows are automatically generated by our system.

Figure 7. A calligraphic couplet, or epigram, in many styles: (a) the training examples;(b)–(l) selected computer-generated results.

(i) (j) (k) (l)(e) (f) (g) (h)

(a) (b) (c) (d)

MAY/JUNE 2005 www.computer.org/intelligent 37

Page 7: Automatic Generation of Artistic Chinese Calligraphyfcmlau/papers/autocalli.pdfappealing Chinese calligraphy from a few training examples of existing calligraphic styles. To demon-strate

than seven votes, it was considered a new cal-ligraphic work. With more training examplesof different styles, the chance of generatinga creative and yet aesthetically acceptablecalligraphic art increased. Also, nonlinearARP generated more creative calligraphic artthan linear APR. Using from six to seventraining examples, the linear process gener-ated about 30 acceptable calligraphic worksand the nonlinear process generated morethan 50.

Figure 8 shows an interesting example. Anartist generated the horse using paint-brushsoftware.11 Our prototype system automati-cally generated the calligraphy (the charac-ter “forever” in Chinese).

ApplicationsThis system for computer-generated orig-

inal Chinese calligraphy has several poten-tial applications.

For example, users could generate per-sonalized fonts with it. First, they wouldimport computer-installed fonts. Then oursystem could compute a flexible model ofeach character by aligning all correspondingcharacters of the different fonts. Next, thesystem would ask for a small set of charac-ters in the user’s handwriting as additional

knowledge. On the basis of these knowledgesources, the system could simulate the pro-posed ARP but now with the additional cri-terion to best match the user inputs. The uni-formity of the hierarchical representationmeans that the resulting set of intensitiesapplies directly to the full character set of allthe existing fonts, generating a font cus-tomized to the user’s handwriting.

Figure 9 shows the results from a simpletest. The top row shows handwritten inputcharacters from different users, and the rowsbelow show the characters generated bymimicking the input style. The results areimpressive. Of course, if the user’s hand-writing is so peculiar that it lies outside thefeasible space composed from all the existingfonts, the system might fail.

An application along the same lines couldmake an individual’s handwriting style morebeautiful, which could improve readers’impressions of the writer. With our system,users can input their handwriting as one ARPsource and then specify an appealing existingstyle as another source. By manually settingthe reasoning intensities, users could choosethe extent to which they rectified their per-sonal handwriting while still preserving adegree of its unique qualities.

C O N S T R A I N T - B A S E D R E A S O N I N G

38 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

Figure 8. “Forever running.” The calligra-phy is styled to match a horse createdwith paint-brush software.

Figure 9. Personalized font generation from a single character. The first row shows a single character written by different users intheir respective handwriting styles, and the second row shows automatically generated characters that mimic the captured handwriting style.

(a) (b) (c) (d) (e) (f) (g) (h)

Page 8: Automatic Generation of Artistic Chinese Calligraphyfcmlau/papers/autocalli.pdfappealing Chinese calligraphy from a few training examples of existing calligraphic styles. To demon-strate

The proposed system is essentially a specificversion of a deformable model for modelingChinese characters of different calligraphicstyles. The model’s representation powermakes the system suitable for deformablemodel-based handwriting recognition. Inaddition, the unified ARP modeling approachis an excellent candidate for writer adapta-tion in existing or emerging handwritingrecognition systems.

Carnegie Mellon University’s CAPTCHA

project (www.captcha.net) presents anotherpossible application of our approach.CAPTCHA stands for “Completely AutomatedPublic Turing Test to Tell Computers andHumans Apart.” A CAPTCHA is any programthat can generate and grade tests that com-puters cannot pass, and one kind—popularfor protecting Web sites from massive auto-mated registrations—uses words written indistorted letters that humans can read butcomputers cannot. Our system could be usedto develop a Chinese CAPTCHA agent thatwould generate similarly distorted Chinesewriting that humans but no machine-com-putable algorithms could read.

Parametric hierarchical knowledge rep-resentation enables computer-gener-

ated Chinese calligraphy artwork in a vari-ety of styles—fully automated in real timefrom a compact set of printed calligraphicinputs. The creative capability of the pro-posed intelligent system lies mainly in thehuge feasible space available for the simu-lated analogous reasoning process. Our pro-totype system has generated artwork that canstand among existing instances, whether real-istic or completely inventive in appearance.

We are considering several extensions toour proposed system. We could extend theapproach to other languages. We might alsoadd a feedback component to fine-tune theaesthetic constraints through reinforcementlearning. Finally, we could capture andtranslate the “psychological” states of othermedia and then link them to the corre-sponding states in calligraphy—for exam-ple, allowing music to direct the generationof calligraphy.

AcknowledgmentsWe are grateful to the reviewers whose com-

ments have been most useful and encouraging.

References

1. D.E. Knuth, Tex and Metafont: New Direc-tions in Typesetting, Am. Mathematical Soc.,1979.

2. G. Ramalho and J.-G. Ganascia, “SimulatingCreativity in Jazz Performance,” Proc.12thNat’l Conf. Artificial Intelligence (AAAI 94),1994, pp. 108–113.

3. H.A. Simon, “Style in Design,” Spatial Syn-thesis in Computer Aided Building Design,C.M. Eastman, ed., Applied Science, 1975.

4. H. Blum, “A Transformation for ExtractingNew Descriptors of Shape,” Models for thePerception of Speech and Visual Form, W.Wathen-Dunn, ed., MIT Press, 1967, pp.362–380.

5. R. He and H. Yan, “Stroke Extraction as Pre-processing Step to Improve Thinning Resultsof Chinese Characters,” Pattern RecognitionLetters, vol. 21, no. 9, 2000, pp. 817–825.

6. J.E. Bresenham, “A Linear Algorithm forIncremental Digital Display of Circular

Arcs,” Comm. ACM, vol. 20, no. 2, 1977, pp.100–106.

7. H.M. Lee, C.W. Huang, and C.C. Sheu, “AFuzzy Rule-Based System for HandwrittenChinese Characters Recognition Based onRadical Extraction,” Fuzzy Sets and Systems,vol. 100, nos. 1–3, 1998, pp. 59–70.

8. B.A. Yanikoglu and P.A. Sandon, “Recogniz-ing Offline Cursive Handwriting,” Proc.Computer Vision and Pattern Recognition,IEEE CS Press, 1994, pp. 397–403.

9. M. Keane, “Analogical Mechanisms,” Artifi-cial Intelligence Rev., vol. 2, no. 4, 1988, pp.229–250.

10. P. Zhu and P.M. Chirlian, “On Critical PointDetection of Digital Shapes,” IEEE Trans.Pattern Analysis and Machine Intelligence,vol. 17, no. 8, 1995, pp. 737–748.

11. S. Xu et al., “Virtual Hairy Brush for PainterlyRendering,” Graphical Models, vol. 66, no.5, 2004, pp. 263–302.

MAY/JUNE 2005 www.computer.org/intelligent 39

T h e A u t h o r sSonghua Xu is a member of the CAD and CG State Key Lab of China atZhejiang University and an honorary researcher in the Department of Com-puter Science at the University of Hong Kong. He is also pursuing computergraphics research with the Computer Science Department at Yale University.His research interests include artificial intelligence, computer graphics, andhuman-computer interaction. He received his bachelor of engineering summacum laude in computer science from Zhejiang University. He is a member ofthe AAAI and Eurographics. Contact him at the CAD and CG State Key Labof China, Zhejiang Univ., Hangzhou, China, 310027; [email protected].

Francis C.M. Lau is head of the Department of Computer Science at the Uni-versity of Hong Kong. His research interests are in parallel and distributedcomputing, object-oriented programming, operating systems, Web and Inter-net computing, computer graphics, and AI. He received his PhD in computerscience from University of Waterloo. He is a senior member of the IEEE.Contact him at the Dept. of Computer Science, University of Hong Kong,Hong Kong; [email protected].

William K. Cheung is an assistant professor in the Department of ComputerScience at Hong Kong Baptist University. His research interests includemodel-based pattern recognition, machine learning, and artificial intelligencewith applications to data mining, information extraction, recommender sys-tems, and Web and Grid service management. He received his PhD in com-puter science from the Hong Kong University of Science and Technology.Contact him at the Dept. of Computer Science, Hong Kong Baptist Univ.,Hong Kong; [email protected].

Yunhe Pan is a full professor in the Department of Computer Science andpresident of Zhejiang University, as well as an honorary professor of the Uni-versity of Hong Kong. His research interests include artificial intelligence,computer graphics, and multimedia computing. He received his MS in com-puter science and engineering from Zhejiang University. Contact him at thePresident’s Office, Zhejiang Univ., Hangzhou, China, 310027; [email protected].