QR Code Denoising using Parallel Hopfield Networksare QR codes with some part of them being highly...
Transcript of QR Code Denoising using Parallel Hopfield Networksare QR codes with some part of them being highly...
-
1
Abstract— We propose a novel algorithm for using Hopfield
networks to denoise QR codes. Hopfield networks have mostly been
used as a noise tolerant memory or to solve difficult combinatorial
problems. One of the major drawbacks in their use in noise tolerant
associative memory is their low capacity of storage, scaling only
linearly with the number of nodes in the network. A larger capacity
therefore requires a larger number of nodes, thereby reducing the
speed of convergence of the network in addition to increasing
hardware costs for acquiring more precise data to be fed to a larger
number of nodes. Our paper proposes a new algorithm to allow the
use of several Hopfield networks in parallel thereby increasing the
cumulative storage capacity of the system many times as compared
to a single Hopfield network. Our algorithm would also be much
faster than a larger single Hopfield network with the same total
capacity. This enables their use in applications like denoising QR
codes, which we have demonstrated in our paper. We then test our
network on a large set of QR code images with different types of
noise and demonstrate that such a system of Hopfield networks can
be used to denoise and recognize QR code’s in real time.
I. INTRODUCTION
In this paper we consider the problem of de-noising QR codes
which have recently grown popular due to the explosion of
mobile applications. QR codes which stand for “Quick
response” codes were invented in Japan by Denso Wave in the
year 1994, QR bar codes are a type of matrix 2D bar codes
which were originally created to track vehicles during their
manufacturing process (see Figure 1). They were specifically
designed to allow its contents to be decoded at high speed, they
have now become the most popular type of matrix 2D bar codes
and are easily read by most smartphones as they allow for data
capture at high speeds [1].
There are various versions of QR codes available depending on
their information capacity [2]. We have used Version 10 which
has a size of 57 x 57 for all further research. Currently these QR
codes use the “Reed-Solomon” error correction codes which
occupy significant space of the QR code itself [3]. Hence, we
propose a novel way of denoising these codes without use of
error correcting codes. The proposed algorithm uses Hopfield
networks as an associative and content addressable memory
(CAM) which are then used to denoise the codes. Hopfield and
his colleagues have already shown that a symmetric
interconnected neural network (a Hopfield neural network) can
perform error corrections in associative retrieval [4].
Ishan Bhatnagar is with the Thadomal Shahani Engineering College, Bandra West, Mumbai(e-mail:[email protected]). Shubhang Bhatanagar is with
Indian Institute of Technology, Bombay, Powai ,Mumbai(e-
mail:[email protected])
The Content Addressable memory (CAM) is by definition a
memory in which the stored patterns are reconstructed by
presenting the network with a partial form of the pattern which
in our case are noisy images of QR codes [5]. We also test our
model on various forms of noise and how our model deals with
these various forms of noisy QR codes. The Hopfield network
model is a single layer of neurons which are fully connected and
recurrent i.e. every neuron is connected to each other neuron
including themselves as shown in Figure 2. In the figure, a
system of 3 neurons is taken. In this notation xi are the input
vectors whereas yi are the output vectors for the given system.
The model can be extended for any general system with any n
number of neurons. Although we have described a recurrent
example of Hopfield network sometimes the self-loops are not
used such that a unit does not influence itself.
Figure 1-A sample QR code
The Hopfield network we use has a set of interconnected
neurons whose values are updated asynchronously. The
activation values used are bipolar i.e. {+1, -1} and work like a
human brain to store and retrieve memory. The general
Hopfield model assumes usage of Hebbian learning in which
two neurons which fire together simultaneously have their
synaptic connections (in our case the weight matrix
coefficients) strengthened. We use a modified form of Hebbian
learning in our model which is described in the algorithm
section of the paper. The term Hebbian learning was used as it
was derived from the work of Donald Hebb [6] who
hypothesized in 1949 how the neurons are connected and work
in the brain.
QR Code Denoising using Parallel Hopfield
Networks Ishan Bhatnagar and Shubhang Bhatnagar
-
2
II. RELATED WORK
Recently due to the growth of hardware, computational power
and simultaneous increase of memory being available has
increased the feasibility of the use of Hopfield networks in a
real time scenario. Hopfield networks have been recently used
as a bidirectional associative memory for semantic neural
networks [7]. They have also been used as a content addressable
memory to find code words in VLSI hardware structures since
code words are essentially stored as a pattern of binary data [8].
We similarly use Hopfield network as an associative and
content addressable memory (CAM) to retrieve the correct QR
codes given a noisy version of the same codes.
Hopfield networks have also been used to restore various kinds
of images recently combined with other algorithms and the
basic Hebbian rule of learning [9]. There have also been
attempts to increase the capacity of Hopfield networks using
various types of genetic algorithms [10]. However, we propose
a novel method to increase the capacity of the Hopfield network
by distributing the load of one Hopfield network into several
parallel Hopfield networks. This also brings about the problem
of having multiple energy functions and choosing between
them when reconstructing a specific QR code which is
discussed in detail in the next section of the paper. Also, several
methods have been tried to improve the de-noising of QR codes
using traditional error codes and pre-processing techniques [11]
but these waste precious space on the QR code which might
otherwise have been precious data.
III. ALGORITHM
Our algorithm for denoising the QR code involves training the
network with the QR codes to be recognized. The steps
followed for training the network to denoise the QR codes are-
A. Generation of data We generate data using an open source QR code generator
[12] based on the ZXing library for java. A total of 4,000 QR
57×57 (version 10) QR codes were generated using this QR
code generator. The QR codes were generated for a random
200-character string as input. The blank space padding was
taken as 0.
B. Training Algorithm- We have divided the 4,000 QR codes into 10 sets of 400 each
(done randomly). This is required because the number of nodes
in the Hopfield network which we use = number of
pixels/binary units in the QR code=57*57=3249. The 57×57
QR codes are converted to 1 dimensional 3249×1 vectors before
using them for training the network. Also, the maximum
capacity of associative memory for such a network, when using
the Hebbian rule of learning has an upper bound of≈ 0.14 n [13].
So, to have a larger capacity, we used the pseudo-inverse rule
for training the network [14].
The pseudo-inverse rule involves first training the network
using Hebbian rule[9] to obtain a weights matrix W. Then we
make the diagonal elements of the matrix equal to zero (to
prevent self-connections for extra stability). Then for getting
the pseudo-inverse rule’s weight matrix, we take the
pseudoinverse of the zero diagonal matrix W. Then we again
make the diagonal elements of the weight matrix zero (to
prevent self-connections for extra stability). For getting
Wpseudoinverse we do-
𝑊𝑑,ℎ𝑒𝑏𝑏𝑖𝑎𝑛 = 𝑊ℎ𝑒𝑏𝑏𝑖𝑎𝑛 𝑤𝑖𝑡ℎ 𝑊𝑖𝑖 = 0
𝑊𝑔,𝑝𝑠𝑒𝑢𝑑𝑜𝑖𝑛𝑣𝑒𝑟𝑠𝑒 = 𝑝𝑠𝑒𝑢𝑑𝑜𝑖𝑛𝑣𝑒𝑟𝑠𝑒(𝑊𝑑,ℎ𝑒𝑏𝑏𝑖𝑎𝑛)
𝑊𝑝𝑠𝑒𝑢𝑑𝑜𝑖𝑛𝑣𝑒𝑟𝑠𝑒 = 𝑊𝑔,𝑝𝑠𝑒𝑢𝑑𝑜𝑖𝑛𝑣𝑒𝑟𝑠𝑒 𝑤𝑖𝑡ℎ 𝑊𝑖𝑖 = 0
Figure 2 A general Hopfield network
-
3
But even this capacity(0.5 added to them.
Also, pixels=1 are only flipped if a noise X>0.5 is added to
them, while pixels=0 are flipped if X
-
4
𝐼 = [ 𝑥1,1 𝑥1,2 ⋯ ⋯ 𝑥1,56 𝑥1,57
⋮ ⋮]
57×57
This is converted to a 3249×1 vector-
𝑉 =
𝑥1,1⋮
𝑥57,57
And then we convert it to a bipolar vector {1, -1} form from the
{0,1} form a usual binary image is in, by multiplying it by 2
and subtracting 1 from each entry-
𝑆 = (2 × 𝑉) −1⋮
Then, we calculated the energy of this noisy input in relation
with each of the Hopfield network’s (and their weight matrices)
using the formula-
𝐸𝑘 = −1
2∑ 𝑊𝑘,𝑖𝑗
𝑖,𝑗× 𝑠𝑖 × 𝑠𝑗
Where Wk,ij represents the (I,j)th element of the kth networks weight matrix. The si and sj represent the ith and the jth
element/node of the Hopfield network (their value) which is
equal to the corresponding element of the noisy input vector’s
(S) value.
This energy of a state in a Hopfield network is a measure of
how ‘stable’ a state of the network is. It indicates how close to
convergence a network is. The trained networks energy
functions have a minima at the points which are the trained
patterns in that network. Also, each iteration of the update of
the network involves updating one node (asynchronously) at a
time using the rule-
𝑠𝑖 = 𝑠𝑔𝑛 (∑ 𝑊𝑖,𝑗 × 𝑠𝑗𝑗
)
sgn is the signum function
Each such iteration leads to a decrease in energy (or at least its
energy remaining the same) of the network, as a node is updated
to be of the same sign as the above summation, leading to a
negative contribution to the overall energy of the network [16].
A Hopfield network converges by eventually settling to a
minima in its energy function. Trained patterns also correspond
to energy minima’s in the network and are hence stable points.
Then, for selecting which network this noisy input belongs to
(it belongs to only one of these Hopfield networks as each QR
code was only trained in 1 network), we run all the networks
with the same noisy input for a small number of iterations in
comparison to the number of iterations the network takes for
convergence to a stable minima(say 100
-
5
` Figure 6The original QR code
Figure 8 The denoised QR code
We also tested denoising QR codes with salt and pepper noise
in our network. Such noise is routinely found in binary images
due to random thermal fluctuation in cameras-
Figure 9 The original QR code
Figure 11 The denoised QR code
Salt and pepper noise affecting the whole QR code was
effectively removed and the original QR code was completely
recovered, as seen in figures 9,10,11 .
Figure 7 The noisy QR code with gaussian noise of variance=0.3
Figure 10 The noisy QR code with salt and pepper noise throughout the QR code
-
6
Figure 12 The original QR code
Figure 14 The denoised QR code
We also tested our network on QR codes which in which a
region is very noisy or completely destroyed. This type of
situation does happen in real world scenarios(especially when
the QR code is kept outdoors and exposed to wear and tear) and
our network should perform well in such situations too.
Such QR codes with a corner affected by high amounts of salt
and pepper noise for also completely denoised by our network,
as seen in figure 12,13,14.
Figure 15 The original QR code
Figure 17 The denoised QR code
QR codes with 1 corner being blackened/ whitened out should
also be denoised and recovered with our network. This is
important as at many times, in outdoor use, parts of the QR code
may get destroyed (torn off/ blackened away). They can still be
recognized, without requiring any repair/ replacement by our
network -
Figure 13 The noisy QR code with its left corner being affected by a large amount of salt pepper noise
Figure 16 The noisy QR code with its left corner blacked out
-
7
Figure 18 The original QR code
Figure 20 The denoised QR code
The Hopfield network accurately denoised QR codes with
varying types of noise. It also correctly denoises and recognizes
QR codes with high amounts of noise localized in a few regions.
The storage capacity of QR codes for a system of n networks in
our case was-
400 × n where n is the number of networks used in parallel. Also, it is
much faster than a single Hopfield network having the same
capacity, as if our network has 3249 nodes and a capacity of
400 × n, a single Hopfield network would have 3249 × n nodes
as capacity of a Hopfield network scales linearly. Such a
network with 3249 × n nodes would be a lot slower to converge,
as it has a much larger weight matrix- of the order (3249×n)2
terms while each of our network would only be only of the order
of (3249)2 terms. This would make the update much quicker.
Also, our algorithm would use up less memory than a single
Hopfield network of the same storage capacity. A single
network with the same storage capacity would require a storage
of the order (3249×n)2 terms. Our algorithm would require
much lesser storage of the order of n × (3249)2 .This means that
our algorithm leads to huge gains in speed and storage
requirements over usual Hopfield networks, making them
feasible for a wider variety of purposes.
V. CONCLUSIONS
Denoising using Hopfield networks provides a new method for
denoising QR codes. Large amounts of noise can be tolerated
by our denoising method as seen in the results. Our algorithm
for distributing the QR codes in several Hopfield networks and
selecting the correct one helps increase the storage capacity of
QR codes by the networks thus making it possible to use it in
large systems for QR code recognition. It makes use of an
abundance of spurious minima’s and the energy function of the
network to select the network containing the denoised QR code.
Also, our method is speedier than storing all the networks in a
much larger method as we have multiple smaller networks,
which are only run for a very few iteration’s (only around 100
in our case). Only one of them (which is also a small network)
is run until convergence, thus providing a huge speed benefit. It
also reduces the storage cost of the network, decreasing it by n
times as compared to a single Hopfield network. This method
can be used in various uses of Hopfield network’s for increasing
the capacity.
REFERENCES
[1] https://en.wikipedia.org/wiki/QR_code
[2] http://www.qrcode.com/en/about/version.html
[3] http://www.qrcode.com/en/about/error_correction.html [4] J. J. Hopfield,” Neural networks and physical systems with emergent
collective computational abilities,” Proc. Nat. Acad. Sci. U.S., vol.79
(1982). [5] Aiyer, S.V.B. & Niranjan, Mahesan & Fallside, Frank. (1990).
Theoretical investigation into the performance of the Hopfield model.
Neural Networks, IEEE Transactions on. 1. 204 - 215. 10.1109/72.80232.
[6] Hebb D. 1949. The organization of behavior. New York, NY:
John Wiley and Sons. [7] Kimani Mungai, Peter & Huang, Runhe & Chen, Zhong & Zhou,
Xiaokang. (2017). Semantic Neuron Networks Based Associative
Memory Model. 1-8. 10.1109/DASC-PICom-DataCom-
CyberSciTec.2017.18. [8] Ionescu, L., Mazare, A.G., & Serban, G. (2010). VLSI implementation
of an associative content addressable memory based on Hopfield
network model. CAS 2010 Proceedings (International
Semiconductor Conference), 02, 499-502. [9] Tan, Jian-Hui & Pan, Bao-Chang & Liang, Jian & Huang, Yong-Hui
& Fan, Xiao-Yan. (2010). A new algorithm for infrared image restoration based on multi-scale morphological wavelet and Hopfield
neural network. 10.1109/ICWAPR.2010.5576349.
[10] Singh, Thipendra & Jabin, Suraiya & Singh, Manisha. (2010). Evolving weight matrices to increase the capacity of Hopfield neural
network associative memory using hybrid evolutionary algorithm.
Figure 19 The noisy QR code with its left corner whited out
-
8
2010 IEEE International Conference on Computational Intelligence and Computing Research, ICCIC 2010.
10.1109/ICCIC.2010.5705809.
[11] Jeng-An Lin and Chiou-Shann Fuh, “2D Barcode Image Decoding,” Mathematical Problems in Engineering, vol. 2013, Article ID 848276
[12] Richter, J. (n.d.).
https://in.mathworks.com/matlabcentral/fileexchange/49808-
qr-code-generator-1-1-based-on-
zxing?focused=7172243&tab=example [13] R. McEliece, E. Posner, E. Rodemich and S. Venkatesh, "The capacity
of the Hopfield associative memory," in IEEE Transactions on
Information Theory, vol. 33, no. 4, pp. 461-482, July 1987. [14] S. S. Venkatesh and D. Psaltis, "Linear and logarithmic capacities in
associative neural networks," in IEEE Transactions on
Information Theory, vol. 35, no. 3, pp. 558-568, May 1989. [15] https://in.mathworks.com/help/images/ref/imnoise.html [16] Neurons with graded response have collective computational
properties like those of two-state neurons, J J Hopfield Proceedings of
the National Academy of Sciences May 1984, 81 (10) 3088-3092 [17] J. Bruck and V. P. Roychowdhury, "On the number of spurious
memories in the Hopfield model (neural network)," in IEEE
Transactions on Information Theory, vol. 36, no. 2, pp. 393-397, March 1990.
[18] Anthony V. Robins, Simon J.R. McCallum, A robust method for distinguishing between learned and spurious attractors, Neural Networks, Volume 17, Issue 3, 2004, Pages 313-326, ISSN 0893-6080