Data-driven Enhancement of Chinese Calligraphy Aesthetic StyleChinese calligraphy [17]. This method...

14
Journal of Information & Computational Science 10:12 (2013) 3645–3658 August 10, 2013 Available at http://www.joics.com Data-driven Enhancement of Chinese Calligraphy Aesthetic Style Wei Li a,b, , Changle Zhou a,b a Cognitive Science Department, Xiamen University, Xiamen 361005, China b Fujian Key Laboratory of the Brain-like Intelligent Systems, Xiamen University Xiamen 361005, China Abstract The generating of a large-scale Chinese character set from a small one has been notoriously challenging due to the complexity of character topology and the inherently elusive sophistication in glyphic aesthetics. This paper proposes an innovative approach to synthesize character-glyph, via which a large-scale character glyph set was generated modeling on a small-scale one of a desired style. The approach initiated from the sampling of designated calligraphic works and proceeded to build a character stroke database, followed by the proposal of F-histogram-based character topology. Drawing on aesthetic intuition, we abstract glyphic aesthetics to establish several evaluative rules, and we further designed an algorithm for the evaluation of the character topology with the Support Vector Machine (SVM) algorithm. At last, we adopted simulated annealing algorithm to optimize character glyphs with the desired style(s). Comparatively, this approach deserves credit in that the representation via F-histogram character topology accommodates more types of character topology and the synthesizing of glyphs integrates the stroke shape and the character topology. Keywords : Glyphic Synthesizing; Chinese Character Topology; Machine Learning; Optimization 1 Introduction Calligraphy art and beauty have fascinated human beings from the emergence of text, inspiring countless artists and philosophers. However, an absolute definition of aesthetic style remains difficult. For example, when a group of human raters is presented with a collection of callig- raphy characters and asked to classify them according to their aesthetic style, the results often indicate that there is a statistical consensus among the raters. Yet it might be hard to define a succinct set of rules that capture the aesthetic perceptions of raters. Furthermore, such percep- tions vary among different classes of shapes, and sometimes differ significantly from culture to This work was supported in part by National Natural Science Foundation of China (No. 61273338) and the Open Project Foundation of Chinese Font Design and Research Center (No. CCF2012-01-06). * Corresponding author. Email address: [email protected] (Wei Li). 1548–7741 / Copyright © 2013 Binary Information Press DOI: 10.12733/jics20102052

Transcript of Data-driven Enhancement of Chinese Calligraphy Aesthetic StyleChinese calligraphy [17]. This method...

Page 1: Data-driven Enhancement of Chinese Calligraphy Aesthetic StyleChinese calligraphy [17]. This method greatly depends on the user’s initial trajectories input. How to represent the

Journal of Information & Computational Science 10:12 (2013) 3645–3658 August 10, 2013Available at http://www.joics.com

Data-driven Enhancement of Chinese Calligraphy

Aesthetic Style ⋆

Wei Li a,b,∗, Changle Zhou a,b

aCognitive Science Department, Xiamen University, Xiamen 361005, ChinabFujian Key Laboratory of the Brain-like Intelligent Systems, Xiamen University

Xiamen 361005, China

Abstract

The generating of a large-scale Chinese character set from a small one has been notoriously challengingdue to the complexity of character topology and the inherently elusive sophistication in glyphic aesthetics.This paper proposes an innovative approach to synthesize character-glyph, via which a large-scalecharacter glyph set was generated modeling on a small-scale one of a desired style. The approachinitiated from the sampling of designated calligraphic works and proceeded to build a character strokedatabase, followed by the proposal of F-histogram-based character topology. Drawing on aestheticintuition, we abstract glyphic aesthetics to establish several evaluative rules, and we further designedan algorithm for the evaluation of the character topology with the Support Vector Machine (SVM)algorithm. At last, we adopted simulated annealing algorithm to optimize character glyphs with thedesired style(s). Comparatively, this approach deserves credit in that the representation via F-histogramcharacter topology accommodates more types of character topology and the synthesizing of glyphsintegrates the stroke shape and the character topology.

Keywords: Glyphic Synthesizing; Chinese Character Topology; Machine Learning; Optimization

1 Introduction

Calligraphy art and beauty have fascinated human beings from the emergence of text, inspiringcountless artists and philosophers. However, an absolute definition of aesthetic style remainsdifficult. For example, when a group of human raters is presented with a collection of callig-raphy characters and asked to classify them according to their aesthetic style, the results oftenindicate that there is a statistical consensus among the raters. Yet it might be hard to define asuccinct set of rules that capture the aesthetic perceptions of raters. Furthermore, such percep-tions vary among different classes of shapes, and sometimes differ significantly from culture to

⋆This work was supported in part by National Natural Science Foundation of China (No. 61273338) and theOpen Project Foundation of Chinese Font Design and Research Center (No. CCF2012-01-06).

∗Corresponding author.Email address: [email protected] (Wei Li).

1548–7741 / Copyright © 2013 Binary Information PressDOI: 10.12733/jics20102052

Page 2: Data-driven Enhancement of Chinese Calligraphy Aesthetic StyleChinese calligraphy [17]. This method greatly depends on the user’s initial trajectories input. How to represent the

3646 W. Li et al. / Journal of Information & Computational Science 10:12 (2013) 3645–3658

culture. Therefore, in this work, we explore the feasibility of a Data-driven approach to aestheticstyle enhancement. Chinese characters belongs to hieroglyphs and consists of strokes. Chinesecalligraphy styles are, to a large extent, embodied by the spatial relations of strokes (i.e. Chinesecharacter topology). On the one hand, user’s trajectories simulated calligrapher as input is moreconvenient than keyboard, and on the other hand, these trajectories can also distinguish variousstroke types. Specifically, we focus on the challenging problem of enhancing the aesthetic styleof character topology in user’s trajectories input, while maintaining the correctness of character.Data-driven means that the properties of a particular set of character topology features are thesame irrespective of the perceiver. The universality of the notion of calligraphy style along withthe ability to reliably and automatically predict the style of calligraphy has motivated this work.Specifically, we present a novel tool capable of automatically enhancing the aesthetic style ofChinese calligraphy character in given trajectories. Although for brevity we often refer to thisprocess as optimization, it should be understood that we merely claim that Chinese calligraphycharacter generated by our tool are more likely to receive a higher rating, when presented to agroup human observers.

• Applications

Although ancient Chinese calligraphy works spread are rarely, by learning their styles, an po-tential application of our techniques is to generate some new characters to rich font library andrepair the eroded characters to protect cultural heritage. Another interesting application is towrite documents with personal style and design a logo or an advertisement. For instance, thecomputer can generate a whole email in handwritten style as if it were manually written charac-ter by character by the human author. Emails in handwriting style produced this way are more“personal”and can draw the reader closer to the author than “typed”emails.

• Overview

The key component in our approach is an optimization engine trained using datasets of calligraphyworks with certain style. The entire optimization process is depicted in Fig. 1. Giving trajectoriesas input we first recognize character strokes. And using F-histogram [9] representing the spatialrelations of strokes, we extract a vector of character topology in the graph. This vector is thenfed into the optimization engine, which yields a modified vector of topology, processing a higherpredicted score than that of the original vector. Next, the strokes are readjusted in the glyphattempting to make the new character topology as close as possible to the modified charactertopology. The resulting new spatial relations of strokes define a character with certain style.Our results indicate that the proposed method is capable of effectively increasing the perceivedcharacter topology style for most trajectories of the user.

• Background

Much of the research work in computerized calligraphic handwriting synthesis has focused onEnglish or Japanese characters [2, 3, 7, 11, 12]. These works are mainly used to imitate the user’strajectory via samples. A Chinese calligraphy character consists of strokes, which are 2D areaobject, so the relations of stokes are the relative position between areal objects. By contrast,Chinese calligraphy synthesis has more difficulties. Existing methods for Chinese calligraphysynthesis can be roughly divided into two categories. The first one is based on interpolation idea [4,

Page 3: Data-driven Enhancement of Chinese Calligraphy Aesthetic StyleChinese calligraphy [17]. This method greatly depends on the user’s initial trajectories input. How to represent the

W. Li et al. / Journal of Information & Computational Science 10:12 (2013) 3645–3658 3647

Fig. 1: The overview of Chinese calligraphy synthesis with personal style

13, 14, 16], in which the corresponding point or topology between samples is found and a characterwith new style is generated by weighted average in these corresponding components. This methoddepends heavily on many samples to the same character and non-rigid point matching is alsothorny issue for pattern recognition. The second category involves rule-based method [6], wherea new style character is reconstructed by some rules. For the complexity of Chinese character,only more than ten rules are incompletely to capture the style of Chinese character. In addition,some researchers employ the method that replace the trajectories with strokes directly to generateChinese calligraphy [17]. This method greatly depends on the user’s initial trajectories input. Howto represent the topology of a Chinese character is key to synthesize calligraphy character. As theelement of Chinese calligraphy character strokes are 2D areal object due to width variation andtheir relations are also viewed as the relative position of areal objects. Freeman [5] proposed thatthe fuzzy set theory be applied because “all-or-nothing”standard mathematical relations areclearly not suited to models of spatial relations. Therefore, in contrast to all previous methods oncharacter topology representation [13, 14, 6, 8], we employ fuzzy-based method in this paper, torepresent the spatial relations between strokes. The main challenge in this work is as the follows:1) How to represent and optimize the Chinese character topology. 2) With analogous style, howto generate a large-scale Chinese character set from a small one.

2 Topological Representation of Chinese Characters

Chinese characters consist of radicals which in turn are composed with basic strokes (see Fig. 2).We favor the proposal by Lai et al. [6] to define the relations among radicals. Lai postulatesthat Chinese characters are topologically featured with horizontal, vertical or bounding patterns.The arrays of these patterns to represent characters enjoy logic and hierarchical clarity but theytend not to cover all complex relations among strokes. For example, characters of the horizontaltopology may vary in horizontal proximities, vertical deviations and stroke shapes. The one-side-fits-all treatment will be doomed to oversimplification. Calligraphic strokes are a kind ofa planar area and the relations among strokes are the spatial relationships of planar objects.Relationships among planar strokes involve such complex factors as distances, directions, areas

Page 4: Data-driven Enhancement of Chinese Calligraphy Aesthetic StyleChinese calligraphy [17]. This method greatly depends on the user’s initial trajectories input. How to represent the

3648 W. Li et al. / Journal of Information & Computational Science 10:12 (2013) 3645–3658

LEVEL 0

LEVEL 1

LEVEL 2

LEVEL 3

Fig. 2: The primitive strokes and Chinese character topology

and shapes, being elusively inexhaustible. The approach by Matsakis et al. [9] accommodatesthese factors and will be adopted in this paper to analyze the relationships among strokes. Itrelies on the intersections of the objects with lines having the desired direction.

2.1 Hierarchical Representation of Chinese Character Topology

With all complexities, the shapes of Chinese characters are perceptively regular: they may behorizontal, vertical or bounding (labeled by H, V and E respectively) in cases of non-singlecharacters. In this fashion, the character 伟 can be represented by H(亻, 韦), 李 by V(木,子) and 框 by H(木, E(匚, 王)), so on and so forth. Single characters can be treated as singleradicals. This representation is merited for hierarchy, contributing to the synthesis of characterswith coarser granularity.

2.2 F-histogram-based Representation of Chinese Character Topolo-gy

The method introduced in Sec. 2.1 specifies only the rough character topologies and the hierarchi-cal approach is failing to the exposition of identical graphs of different styles and is not supportivefor single characters. Therefore, we combined F-histogram into the method introduced in Sec. 2.1and built a high dimensional matrix to further describe the complex Chinese character topology.Cognitive experiments reveal that distance is important to spatial relationship. The statisticsof the distances between points of strokes to different directions were then employed to expressthe support of strokes to the directions and the support of all directions constitutes the vectorexpressing the relationships among strokes. This method accommodates angles, distances andthe distribution of stroke points and defines stroke relationships with vectors, describing Chinesecharacter topology in a more comprehensive and accurate way. The following display manifestsit in detail. Our treatment of spatial relations builds up from points, to regions. Let T be atriple unit (θ, sθi (υ), s

θj(υ)), θ, υ be real representing direction and displacement, Si and Sj be the

respective areas of stroke describing graph C. And let sθi (υ) be a particular longitude of Si, i.e.sθi (υ) = Si

∩θ(υ). Likewise, s

θj(υ) = Sj

∩θ(υ), θ(υ), as Fig. 3 demonstrates, represents the

Page 5: Data-driven Enhancement of Chinese Calligraphy Aesthetic StyleChinese calligraphy [17]. This method greatly depends on the user’s initial trajectories input. How to represent the

W. Li et al. / Journal of Information & Computational Science 10:12 (2013) 3645–3658 3649

Strokeextraction

Relationrepresentation

M

Graphic symbol

N

∆θ(ν)

∆ θ(ν

)

Stroke S1 Stroke S2

s1(ν)θ s2(ν)

O

j

ν

θ

θ

Strokeextraction

Relationrepresentation

MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM

Graphic symbol

NN

∆ θ(ν

)

Stroke S1SS Stroke S2SS

s1(ν)θ s2(ν)

O

j

ν

θ

θ

Fig. 3: Representation stroke relationship using F-histogram

vector obtained from (O, i, j)’s horizontal movement along j-axis after anti-clockwise θ-anglerotation.

Definition 1: The spatial relations between points are considered first. Given two pointsM ∈ sθi (υ), N ∈ sθj(υ), the reciprocal of the square distance between M and N (i.e. φ(M,N, θ, υ))represents the argument put forward to support the proposition “stroke Sj in the direction θ ofSi”.

φ(M,N, θ, υ) =

1

|M−N |3/2 ,−−→MN · θ(υ) = 1

0, other cases(1)

Definition 2: Given two longitudes sθi (υ) and sθj(υ), the value f(sθi (υ), sθj(υ)) represents the

argument put forward to support the proposition “stroke Sj in the direction θ of Si”.

f(sθi (υ), sθj(υ)) =

∑M∈sθi (υ)

∑N∈sθj (υ)

φ(M,N, θ, υ) (2)

Definition 3. Given two longitude sets Sθi (υ) = sθi (υ) : Si

∩θ(υ) and Sθ

j (υ) = sθj(υ) :Sj

∩θ(υ), the value h(Sθ

i (υ), Sθj (υ)) represents the argument put forward to support the propo-

sition “stroke Sj in the direction θ of Si”.

h(Sθi (υ), S

θj (υ)) =

∑sθi (υ)∈Sθ

i (υ)

∑sθj (υ)∈Sθ

j (υ)

f(sθi (υ), sθj(υ)) (3)

Definition 4. The argument put forward to support the proposition “stroke Sj in the directionθ of Si” is:

F (θ, Si, Sj) =∑

υ∈edge(C)

h(Sθi (υ), S

θj (υ)) (4)

where edge(C) is the edge of the Chinese character C’s bounding box.

Definition 5. The spatial relation between stroke Si and Sj is:

R(Si, Sj) = F (θ, Si, Sj) : θ ∈ [0, 2π) (5)

Page 6: Data-driven Enhancement of Chinese Calligraphy Aesthetic StyleChinese calligraphy [17]. This method greatly depends on the user’s initial trajectories input. How to represent the

3650 W. Li et al. / Journal of Information & Computational Science 10:12 (2013) 3645–3658

Definition 6. The character topology of glyph C consisting of n strokes S1, S2, ..., Sn is definedas:

Topology(C) =

∣∣∣∣∣∣∣∣R(S1, S1) · · · R(S1, Sn)

.... . .

...

R(Sn, S1) · · · R(Sn, Sn)

∣∣∣∣∣∣∣∣ (6)

Demonstrably, with sufficient division of spatial directions F-histogram will exhaust the re-lations among strokes. What follows is the demonstration, with the example of coincidencerelationship, of the advantages attributed to the approach proposed in this paper. coincidencerelationship is taken in mathematics as one case but for human vision, coincidence by differentobjects impresses differently. As revealed by Table 1 below, strokes S1 and S2 are identical but atthe same time discriminative according to F-histogram-based calculation (see Eq. (1)-(6)) for co-incidence, the latter conforming to human vision. Table 2 shows that the gradualness foregroundswhen stroke S1 approaches S2, the subtle trend being captured by F-histogram. It is consistentwith intuitive understanding.

3 Constructing Stroke Database

Stroke glyphs scanned from ancient China’s calligraphic works are marginally incomplete due tocorrosion. In view of this, the controlling points along stroke contours are obtained interactively

Table 1: The F-histogram from different elements with coincidence relationship

Angle

Stroke

0 0.068829 0.104107 0.342571 0.108933 0.038300

2π/16 0.000205 0.004593 0.000155 0.005733 0

2π ∗ 2/16 0.060345 0.069603 0.047110 0.206856 0.037783

2π ∗ 3/16 0.003320 0.007013 0 0.008268 0.001281

2π ∗ 4/16 0.261773 0.263889 0.051068 0.109434 0.385210

2π ∗ 5/16 0.017720 0.002309 0 0.000474 0.001114

2π ∗ 6/16 0.087808 0.047596 0.055458 0.059665 0.036313

2π ∗ 7/16 0 0.000890 0.003639 0.000638 0

2π ∗ 8/16 0.068829 0.104107 0.342571 0.108933

2π ∗ 9/16 0.000205 0.004593 0.000155 0.005733 0

2π ∗ 10/16 0.060345 0.069603 0.047110 0.206856

2π ∗ 11/16 0.003320 0.007013 0 0.008268 0.001281

2π ∗ 12/16 0.261773 0.263889 0.051068 0.109434 0.385210

2π ∗ 13/16 0.017720 0.002309 0 0.000474 0.001114

2π ∗ 14/16 0.068829 0.104107 0.342571 0.108933 0.038300

2π ∗ 15/16 0 0.000890 0.003639 0.000638 0

Page 7: Data-driven Enhancement of Chinese Calligraphy Aesthetic StyleChinese calligraphy [17]. This method greatly depends on the user’s initial trajectories input. How to represent the

W. Li et al. / Journal of Information & Computational Science 10:12 (2013) 3645–3658 3651

Table 2: The F-histogram reflects different relationships

Angle

Stroke

0 0 0.000653 0.032382 0.053446

2π/16 0 0 0.000189 0.002534

2π ∗ 2/16 0 0.000431 0.023480 0.037383

2π ∗ 3/16 0 0 0.001213 0.005677

2π ∗ 4/16 0 0.002047 0.077520 0.088406

2π ∗ 5/16 0 0.003043 0.014514 0.011512

2π ∗ 6/16 0.042699 0.131216 0.107971 0.117545

2π ∗ 7/16 0.123006 0.076413 0.047133 0.031729

2π ∗ 8/16 0.563346 0.419938 0.294517 0.234185

2π ∗ 9/16 0.153791 0.110594 0.059988 0.037974

2π ∗ 10/16 0.117157 0.225363 0.215874 0.221455

2π ∗ 11/16 0 0.007419 0.011170 0.053497

2π ∗ 12/16 0 0.021793 0.069557 0.045608

2π ∗ 13/16 0 0 0.004089 0.004177

2π ∗ 14/16 0 0.001091 0.039637 0.049832

2π ∗ 15/16 0 0 0.000766 0.005040

to fit with Bezier curves (see Fig. 4). Automatic extraction of strokes remains undesirable. We gotthe trajectories of glyph manually, rather than computing the skeleton automatically. Thus, wecan distinguish various stroke styles with this method. It is reasonable that we use the controllingpoints of the stroke contour and the sequence points of trajectory to represent the stroke. Asshown in Fig. 4, the contours of strokes are represented by continuous sections of 3-order Beziercurves. Adjustable controlling points (circles) will adapt to desired bending degree. And then wecomputed the bounding box of a stroke and mapped out the point coordinates to its inner. Thecoordinate in the bounding box is a stroke’s parametric representation, ranging from zero to one.When redrawing the stroke, we can map the parameters of the strokes to a new canvas. Then,we fit the contour using curve and fill the contour’s inner area by scanning line filling algorithm.

Fig. 4: The presentation of strokes using Bezier curve and trajectory

Page 8: Data-driven Enhancement of Chinese Calligraphy Aesthetic StyleChinese calligraphy [17]. This method greatly depends on the user’s initial trajectories input. How to represent the

3652 W. Li et al. / Journal of Information & Computational Science 10:12 (2013) 3645–3658

4 Calculating Similarity of Trajectory

The method proposed by Belongie et al. [1] was employed to describe the property of every pointby Eq. (7). For a point pi on the shape, computed was a coarse histogram hi of the relativecoordinates of the remaining n− 1 points,

hi(k) = #q = pi : (q − pi) ∈ bin(k) (7)

Sim(pi, qj) =1

2

K∑k=1

(hi(k)/|S| − hj(k)/|T |)2/(hi(k)/|S|+ hj(k)/|T |) (8)

match(S, T ) =1

|S|+ |T |∑pi∈S

minqj∈T

Sim(pi, qj) +∑qj∈T

minpi∈S

Sim(qj, pi) (9)

S = argminT∈Ωmatch(S, T ) (10)

This histogram is defined to be the shape context pi. The difference between the two points wasobtained by Eq. (8) and the average of similarity between S and T by Eq. (9), where S denotes apoint set (the user’s input trajectory) and T is also a point set (the trajectory of stroke database).Locating of trajectory in stroke databases most similar to a user’s was calculated with Eq. (10).

5 Encoding and Identification of Chinese Characters

A table (Table 3) was then created, encoding Chinese characters by stroke order, stroke nameand topological type (see Sec. 2.1). With input trajectory matching, targeted characters andtopology can be retrieved from encoding of character.

Table 3: The code of some Chinese characters

Character name Code Structure type

大 S(横&撇&捺&) Single

仁 H(S(撇&竖&), S(横&横&)) Horizontal

李 V(S(横&竖&), S(撇&钩&横&)) Vertical

问 E(S(点&竖&钩&), S(竖&折&横&)) Surrounding

梨 V(H(S(撇&横&竖&撇&捺&), S(竖&钩&)), S(横&竖&撇&捺&)) Compound

6 Modeling Glyphic Styles

Pictographic Chinese characters feature formal complexity. radicals, strokes and pixels are crit-ically pertinent granularities for the establishment of aesthetic rules, which, coupled with F-histogram, are serviceable to the extraction of styles.

Rule 1: radical alignment. The writing of Chinese characters is known for symmetryand harmony. Take the horizontal topology for an instance: there are many varieties consist-ing of radical r1 and radical r2 Three possibilities are listed in Fig. 5, of which (c) is calli-graphically and aesthetically more desirable. (c) can be formulated quantitatively as follows:

Page 9: Data-driven Enhancement of Chinese Calligraphy Aesthetic StyleChinese calligraphy [17]. This method greatly depends on the user’s initial trajectories input. How to represent the

W. Li et al. / Journal of Information & Computational Science 10:12 (2013) 3645–3658 3653

O

(a) (b) (c)

O Oy1

y4

y1

y4

y1

y4

y2

y3

y2

y3

y2

y3

r1 r2

Fig. 5: Alignment of two sibling radicals, where (a), (b) and (c) are different relationships between thetwo radicals

R1 =α∗|(y2−y1)−(y4−y3))|+|y2−y1|+|y3−y4|

maxbh(r1),bh(r2), the parameter α > 1, Specifically, bh(x) signifies the height

and width of the bounding of radical x. y1 and y4 signify respectively the y-coordinates of the topand the bottom of the bounding of radical r2. Similar treatment also applies to the vertical topol-ogy. The smaller R1 is, the higher alignment of radicals is, and calligraphically and aestheticallythe more desirable.

Rule 2: balancing the shortest distance among radicals. The gaps among radicals have agreat influence on character topology and appropriate width is desirable. As shown in Fig. 6, theaverage gap distance R2 between a pair of radicals r1 and r2 is defined as: R2 =

∑y2i=y1

|x2i−x1i||y2−y1| ,

where x1i denotes x-axis of the rightmost black pixel of radical r1 on scan line i, and x2i is x-axisof the leftmost black pixel of radical r2 on the same scan line. Similar treatment also applies tothe vertical and bounding topologies.

Ox

y

y1

y2

yi

x1i x2i

r2r1

O

y1

y2

y

x1

r2

Fig. 6: Average gap distance between radicals

Rule 3: balancing of stroke width. Calligraphy postulates uniformed style, namely, balancedand consistent stroke width. Let the point p(x, y) ∈ Gξ(C), Gξ(C) ⊂ B(C), where Gξ(C)denotes the image of the Chinese character C with ξ style and B(C) is the bounding box ofC. The calculation of the width of a single stroke Cj can be defined as follows: we have thatSkeleton(Gξ(Cj)) = st1, st2, ..., stk, Contour(Gξ(Cj)) = ct1, ct2, ..., cts. ∀sti, ∃ctl, ctm, s.t.cos (

−−−−−→ctl − sti,

−−−−−→sti − ctm) = 1 and swi = min |−−−−−→ctl − sti|+ |−−−−−→sti − ctm| : 1 ≤ l < m ≤ s, 1 ≤ i ≤ k.

So the width of the stroke Cj, SW (Gξ(Cj)) =∑k

m=1 swm/k. The average width of C, µ3 =∑Nj=1 SW (Gξ(Cj))/N . The standard deviation, R3 =

√1N

∑Nj=1 (SW (Gξ(Cj))− µ3)2, where N

represents the number of stroke. The method proposed by Neusius at el. [10] was employed to

Page 10: Data-driven Enhancement of Chinese Calligraphy Aesthetic StyleChinese calligraphy [17]. This method greatly depends on the user’s initial trajectories input. How to represent the

3654 W. Li et al. / Journal of Information & Computational Science 10:12 (2013) 3645–3658

extract the stroke skeleton.

Rule 4: holistic symmetry. Calligraphy desires stable gravity, that is, straightness andbalance, which can be aesthetically realized somehow by achieving possibly large overlappingbetween centroid and geometrical center. To calculate the centroid of a Chinese character, eachblack pixel is treated as a unit mass. Suppose the glyph consists of n black pixels denoted by(xi, yi), 1 ≤ i ≤ n, the centroid (Gx(C), Gy(C)) is defined as, Gx(C) =

∑Ni=1 xi/n,Gy(C) =∑N

i=1 yi/n. Then, R4 = |Gx(C)W

− 12|+ |Gy(C)

H− 1

2|, where W , H are respectively the width and the

height of the bounding box of C.

Rule 5: even white space. A sliding window is defined to scan the glyph and to count the num-ber of white spacing points. The average value and standard deviation of white space points in allwindows are then calculated to represent the white spacing of characters. Let the size of the slidingwindow be (2ω+1)× (2ω+1), where wk(x, y) donates the number of white spacing points center-

ing on point p(x, y). p(x, y) =

0, (x, y) ∈ Gξ(C)

1, other cases, wk(x, y) =

∑ωi=−ω

∑ωj=−ω p(x+ i, y + j),

µ5 =∑N

k=1 wk

N, where wk denotes wk(x, y), R5 =

√1N

∑Nk=1 (wk − µ5)2, where N denotes the

number of acquired windows and the smaller N is, the evener the white spacing.

Glyphs of different styles are recognizable to human beings and are appreciated stylistically toveteran calligraphers. The myth lies in people’s mastery of the stylistic aura of the glyphs. Toreduce the number of training sampling, the description of particular style(s) is realized via thecalculation of differences between regular scripts and targeted scripts, which supposedly avoids thesemantic content of concrete characters. It is as follows: Let the regular script of glyph C be C1

and its idiosyncratic glyph script be C2 the Rule 1-Rule 5-based calculation will be: di(C1, C2) =

Ri(C1)−Ri(C

2), i = 1, 2, 3, 4, 5 The space was divided into N bins. According to Eq. (6) and thenumber of the bins, the matrix Topology(C1) is divided into N 2D-matrixes. These 2D-matrixeswere defined as M1(C

1),M2(C1), ...,MN(C

1). Likely M1(C2),M2(C

2), ...,MN(C2) represent F-

histogram of C2 Difference in topological matrices for C1 and C2 in all directions was computed,namely, Di(C

1, C2) = Mi(C1) − Mi(C

2), i = 1, 2, ..., N. (simplified into Di for convenience). Asymmetric matrix Ai was acquired by Ai = DiD

Ti , i = 1, 2, ..., N. Obtainable were then the

maximum element value, the minimum element value, the average element value, the mean value,the maximum absolute value and the first two eigenvalues about the matrix Ai, i = 1, 2, 3, ..., N.Thus, a (7 ∗N +5)-dimensional vector t(C2) was taken to denote the style of C2. In the learningprocess, SVM was adopted to train samples. By using various kernels, SVM can fit highlynon-linear functions. The (7 ∗ N + 5)-dimensional feature vectors and their corresponding stylesimilarity scores (on a scale in [0, 5], 0-best and 5-worst) are used as training samples to constructa SVM model.

7 Optimizing Character Topology

Given a particular Chinese character, our optimization goal is to minimize the energy E(·),where depth(·) is the depth of the search tree, and evaluate(·) is the evaluation value by theSVM, and w1, w2 are weights respectively. We employ a simulated annealing strategy during thediscrete optimization. In each iteration, we randomly select one stroke and randomly displaceits position with a distance of at most d. (Here, d=10) or scale its size with a measure of at

Page 11: Data-driven Enhancement of Chinese Calligraphy Aesthetic StyleChinese calligraphy [17]. This method greatly depends on the user’s initial trajectories input. How to represent the

W. Li et al. / Journal of Information & Computational Science 10:12 (2013) 3645–3658 3655

most δ. (Here, δ = 0.3). Then, we employ SVM to evaluate the changed Chinese character.If E is smaller than the last, this modification is accepted; otherwise, a transition probabilityPr = exp(δ/T (t)) is used to make the decision, where δ is the energy difference between twoiterations; T (t) = T0/log(1 + t) is the temperature; t is the iteration index; T0 is the standardvariation. If Pr is smaller than a random number in [0, 1], this modification is accepted; otherwise,it is rejected. The optimization is terminated whenever evaluate(·) is 0 or not reduced for t0consecutive iterations, where t0 = 1000 in our implementation. Fig. 8 shows the intermediateresults along with their energies. As the energy reduces, the visual quality of the characterimproves accordingly.

E(X) = w1 ∗ depth(X) + w2 ∗ evaluate(X) (11)

8 Experimental Analysis

The programming environment of Visual Studio 2010 and OpenCV2.3.1 were employed under theWindow 7 system to realize the algorithm proposed in this paper. The hardware includes Intel?Core(TM)2 Duo CPU [email protected], 2.50G memories. Trajectories were recorded by tablets ofbamboo one CTE-631. Trajectories were sampled before recognizing corresponding stroke typesfrom stroke databases. The ultimate calligraphic glyphs of intended styles were acquired viacomputing with the optimizing method introduced in Sec. 7. The following is a verification ofthe algorithm with regular scripts of Liu Gongquan (simplified into Liu style for convenience), anancient Chinese artist in the Tang dynasty, as examples. Forty-six types of basic strokes from Liu’sscripts were extracted to construct a stroke database with every type of strokes accommodatingdifferent variations, as shown in the case of the stroke dot: Standardtemplates of strokes and trajectories were constructed with tablets ready for users’ scribing in therecognition stage. The method introduced in Sec. 4 was employed to acquire trajectories mostsimilar to those in templates before locating the ultimate corresponding contours. Testing of basicstrokes with 100 trajectories as a testing set showed an accuracy rate of 97% and one of 95% forsecondary classified strokes like chestnut point, sunflower seed point, etc. Fig. 7 demonstrated thetrajectories by different scribers in the left and their optimized glyphs in the right. Topologically,a outperforms b but generally, both are desirable. Unlike Zhang et al. [17], the approach in thispaper does not rely on the initial topology so desirable glyphs are obtainable even with mediocrescribing.

(a)

(b)

Fig. 7: The generated result contrast from different author and writing

Page 12: Data-driven Enhancement of Chinese Calligraphy Aesthetic StyleChinese calligraphy [17]. This method greatly depends on the user’s initial trajectories input. How to represent the

3656 W. Li et al. / Journal of Information & Computational Science 10:12 (2013) 3645–3658

200 characters in Liu’s copybook were extracted and another 300 samples are combined byscribers using strokes of Liu style. Fig. 8 shows that the optimization process of the Chinesecharacter 心(meaning heart in Chinese). When calculating each level in the search space, trans-formations were needed for these strokes, including moving (left, right, up, down) and scaling(left, right, up, down).

i=0 i=10 i=50 i=70 i=80 i=100 i=120

Fig. 8: The optimization process of the character “heart”

The minor differences among the different topologies of the glyph力(meaning power in Chinese)were calculated to demonstrate the advantages of the F-histogram-based approach. The topologyof 力consists of a horizon-break-hook stroke and a left-falling one, a case unidentifiable in thetopological representation in Xu [13, 14] due to the unchanged relative positions in their respectivebounding boxes. The relationship between the two strokes, however, has been changed de factodue to the thickened left-falling stroke. The subtle change was captured in F-histogram-based(Fig. 9), conforming to human vision. The experiment validates again the higher desirability of

0

F-h

isto

gram

(X

, Y

, Θ

)

0.35

0.30

0.25

0.20

0.15

0.10

0.05

0π/2 3π/2π

0≤Θ<2π

Plot of F-histogram

Case (a)

A

B

Case (b)

A

B

The case a and the case b is the sameto F-histogram (A, A, Θ)

0

F-h

isto

gram

(X

, Y

, Θ

)

0.35

0.30

0.25

0.20

0.15

0.10

0.05

0π/2 3π/2π

0≤Θ<2π

Plot of F-histogram

Case a: F-histogram (A, B, Θ)Case b: F-histogram (A, B, Θ)

0

F-h

isto

gram

(X

, Y

, Θ

)

0.25

0.20

0.15

0.10

0.05

0π/2 3π/2π

0≤Θ<2π

Plot of F-histogram

0

F-h

isto

gram

(X

, Y

, Θ

)

0.35

0.30

0.25

0.20

0.15

0.10

0.05

0π/2 3π/2π

0≤Θ<2π

Plot of F-histogram

Case a: F-histogram (B, A, Θ)Case b: F-histogram (B, A, Θ)

Case a: F-histogram (B, B, Θ)Case b: F-histogram (B, B, Θ)

Fig. 9: The difference of F-histogram to the same Chinese character “power”

Page 13: Data-driven Enhancement of Chinese Calligraphy Aesthetic StyleChinese calligraphy [17]. This method greatly depends on the user’s initial trajectories input. How to represent the

W. Li et al. / Journal of Information & Computational Science 10:12 (2013) 3645–3658 3657

the approach proposed in this paper.

At last, Fig. 10 (a)-(f) contrasted the glyphs in Liu’s copybook and those synthesized by theapproach proposed in this paper. The exemplifications encompassed horizontal, vertical andbounding topologies. Finally, as shown in Fig. 10 (g), the prototype system generated a poemcreated by Zhihuan Wang (668 A.D. -742 A.D.), who is a great poet in ancient China, using thestrokes of Liu style.

(a) (b) (c) (d) (e) (f) (g)

Fig. 10: The result imitating Liu’s style. (a) The user’s Trajectories. (b) Replace the Trajectories withstrokes. (c) Retain stroke contour and remove the trajectories. (d) Filling the strokes. (e) The synthesizeworks. (f) Characters from calligraphy tablet image. (g) Generate a poem (“On the Stork Tower”) usingour prototype system

9 Conclusion

The representation of Chinese characters topology constitutes the key in glyphic synthesis. Thispaper introduced an innovative approach to the topological representation of Chinese character-s with an unprecedented merit found in the capability of differentiating the subtleties amongChinese character topologies including coincidence. A synthesis system for calligraphic glyphs ofparticular styles was developed to verifying the proposed algorithm. The process started from therecognition of trajectories, proceeded to the learning of writing style through the replacement oftrajectories with strokes of original style, before ultimately optimized synthesis of intended newglyphs with some strokes of particular styles. This is an exploration into the challenging taskof generating large-scale glyph sets with small-scale ones. Admittedly, the styles of calligraph-ic glyphs are subject to the ‘hollow-strokes’ effect, which deserves more consideration in futureprobes.

References

[1] Serge Belongie, Jitendra Malik, Jan Puzicha, Shaping matching and object recognition using shapecontexts, IEEE Transactions on Pattern Analysis and Machine Intelligence, (24) 2002, 509-522

Page 14: Data-driven Enhancement of Chinese Calligraphy Aesthetic StyleChinese calligraphy [17]. This method greatly depends on the user’s initial trajectories input. How to represent the

3658 W. Li et al. / Journal of Information & Computational Science 10:12 (2013) 3645–3658

[2] Hyunil Choi, Sung-Jung Cho, Jin H. Kim, Generation of handwritten characters with Bayesiannetwork based on-line handwriting recognizers, Seventh International Conference on DocumentAnalysis and Recognition, 2003, 995-999

[3] Jan Dolinsky, Hideyuki Takagi, Analysis and modeling of naturalness in handwritten characters,IEEE Transactions on Neural Networks, (20) 2009, 1540-1553

[4] Jun Dong, Miao Xu, Xianjun Zhang, Yanqing Gao, Yunhe Pan, The creation process of Chinesecalligraphy and emulation of imagery thinking, IEEE Intelligent Systems, (23) 2008, 56-62

[5] J. Freeman, The modelling of spatial relations, Computer Graphics and Image Processing, (4)1975, 156-171

[6] Pak Keung Lai, Dit Yan Yeung, Man Chi Pong, A heuristic search approach to Chinese glyphgeneration using hierarchical character composition, Computer Processing of Oriental Languages,(10) 1996, 307-323

[7] Zhouchen Lin, Liang Wan, Style-preserving English handwriting synthesis, Pattern Recognition,(40) 2007, 2097-2109

[8] Jing Hong Low, Chee Onn Wong, Eunjung Han, Hwangkyu Yang, Using skeletonization andshortest skeleton path approach for Chinese character representation, Proceedings of Digital ImageComputing: Techniques and Applications, 2008, 472-479

[9] Pascal Matsakis, Laurent Wendling, A new way to represent the relative position between arealobjects, IEEE Transactions on Pattern Analysis and Machine Intelligence, (21) 1999, 634-643

[10] Christian Neusius, Jan Olszewski, Noniterative thinning algorithm, ACM Transactions on Math-ematical Software, (20) 1994, 5-20

[11] Rapee Suveeranont, Takeo Igarashi, Example-based automatic font generation, Smart Graphics,Lecture Notes in Computer Science, (6133) 2010, 127-138

[12] Jue Wang, ChenyuWu, Ying Qing Xu, Heung Yeung Shum, Liang Ji, Learning-based cursive hand-writing synthesis, Proceedings of the Eighth International Workshop on Frontiers in HandwritingRecognition, 2002, 157-162

[13] Songhua Xu, Francis C. M. Lau, William K. Cheung, Yunhe Pan, Automatic generation of artisticChinese calligraphy, IEEE Intelligent Systems, (20) 2005, 32-39

[14] Songhua Xu, Hao Jiang, Tao Jin, Francis C. M. Lau, Yunhe Pan, Automatic generation of Chinesecalligraphic writings with style imitation, IEEE Intelligent Systems, (24) 2009, 44-53

[15] Fenghui Yao, Guifeng Shao, Jianqiang Yi, Extracting the trajectory of writing brush in Chinesecharacter calligraphy, Engineering Applications of Artificial Intelligence, (17) 2004, 631-644

[16] Xiafen Zhang, Guangzhong Liu, Chinese calligraphy character image synthesis based on retrieval,Lecture Notes in Computer Science, (5879) 2009, 167-178

[17] Zhenting Zhang, Jiagnqin Wu, Kai Yu, Chinese calligraphy specific style rendering system, Pro-ceedings of the ACM International Conference on Digital Libraries, 2010, 99-108