Neurocomputing 208 (2016) 117126

Contents lists available at ScienceDirect

Neurocomputing
journal homepage: www.elsevier.com/locate/neucom

Research on the natural image super-resolution reconstruction

algorithm based on compressive perception theory and deep
learning model
Ganglong Duan n, Wenxiu Hu, Jianren Wang
Xi'an University of Technology, Shaanxi 710054, China

art ic l e i nf o a b s t r a c t

Article history: With the bursting development of machine learning and articial intelligence, the pattern recognition
Received 9 October 2015 based image processing techniques are growing faster than ever before. In this paper, we conduct the-
Received in revised form oretical analysis on the natural image super-resolution reconstruction algorithm based on compressive
16 December 2015
perception theory and deep learning model. The image restoration is the purpose of the degraded image
Accepted 16 December 2015
Available online 1 June 2016
processing which make its recovery as it had been before the degradation of ideal image. According to
the views of Fourier optics, optical imaging system is a low pass lter, due to the general inuence of
Keywords: optical diffraction. The deep neural network with hierarchical unsupervised training method stratied
Image super-resolution greed training beforehand matter will be the result of the training as the novel learning supervision
Compressive perception and sensing
probability model of the initial value to make good use of the optical imaging system. The adopted
Deep learning model
compressed sensing theory points out that as long as signal is compressible or sparse, so, if there is a
Neural network structure
Optimization transformation matrix is not related observation matrix on signal can directly obtain compressed form of
Natural images the original signal. Our research adopts the advances of the mentioned technique, in the training step, we
Image restoration use deep neural network to automatically capture the features and in the reconstruction procedure we
use the compressive sensing and dictionary learning theory to reconstruct the high resolution image. By
enhancing both of the steps, our experimental result indicates the feasibility of the novel algorithm. The
prospect is also discussed in the nal part.
& 2016 Elsevier B.V. All rights reserved.

1. Introduction kind of methodology that holds the characteristics of signicantly

Areas such as remote sensing, military reconnaissance and
medical imaging needs to target high resolution image, but due to
restrictions on the size of CCD pixels or high resolution CCD
camera are expensive, volume and weight is very big which can't
meet the requirements of high resolution image can be obtained
directly. The necessary to make further data processing the
acquired image to fully tap the image contains abundant infor-
mation [13]. In the case of the camera imaging system para-
meters are known, the former and prior super resolution image
reconstruction technology of solving nonlinear system of equa-
tions by iteration the method to obtain high resolution images of
the optimal solution and resolution image reconstruction of sur-
plus. Super-resolution image refers to the use of one or a few
pieces of low resolution images and reconstruct the high pixel
density and includes more details of high resolution images. As a
n
Corresponding author.
E-mail address: gl-duan@xaut.edu.cn (G. Duan).
improving the quality of image without extra hardware enhance-
ment, the super-resolution technique has captured sufcient
applications on the video surveillance, medical imaging, remote
sensing images, and other elds [49]. From the mathematical
point of view, the image super-resolution problem is a patholo-
gical problem and to solve this problem there must be reasonable
prior assumptions. The reconstruction based approach is to use the
information got from the high resolution image to the low-
resolution image to generate the priori assumptions of the
model to solve the problem of super-resolution.
Image restoration is the purpose of the degraded image pro-
cessing which make its recovery as it had been before the degra-
dation of ideal image. According to the views of Fourier optics,
optical imaging system is a low pass lter, due to the general
inuence of optical diffraction, the transfer function in determined
by the diffraction limited resolution of a cut-off frequency value
above is zero. Obviously, ordinary image restoration techniques
such as deconvolution technique such as the frequency of the
object can only be recovered to the diffraction limit corresponding

http://dx.doi.org/10.1016/j.neucom.2015.12.125
0925-2312/& 2016 Elsevier B.V. All rights reserved.
118 G. Duan et al. / Neurocomputing 208 (2016) 117126

cutoff frequency, and cannot go beyond it, such as frequency of

energy and information was lost helpless. Super-resolution image
restoration is to recover outside of the cut-off frequency infor-
mation, so that the image for more details and information.
Recently, deep learning as a kind of complex learning hierarchical
probabilistic model method is widely studied and applied. The
deep learning can be understood as the development of neural
network and the neural network is the basic characteristic of the
brain or biological neural network abstraction and modeling
which can learn from the external environment and adapt to the
environment in similar to biological interactions. Neural network
is an important part of intelligent science, to solve the problem of
complex and intelligent control provides a novel efcient way [10
15]. Neural network was once become a high-prole direction in
the eld of machine learning.
The deep neural network is with hierarchical unsupervised
training method stratied greed training beforehand matter which
will be the result of training as the novel learning supervision
probability model of the initial value and the study performance is
greatly improved. Characteristics of unsupervised learning are the
complex hierarchical structure with huge amounts of data set in
the matter between statistical modeling. By the unsupervised Fig. 1. The architecture of deep learning and compressive sensing theory.

training network for advanced abstraction features, and to provide

the good initial weights with the weight limit on the global
training benet within the scope of the use of the local informa-
tion between layer and layer step by step training we could pay
attention to the nature of the training data and can reduce the risk
of learning goals through tting to avoid delivered long deep
neural network error accumulation problem which will be t for
solving the image reconstruction issues.
Image fusion is to integrate information collected by multiple
source channel of the same target image data through the image
processing and computer technology, such as maximum extraction
favorable information in each channel, and nally integrated into
high quality images in order to improve the basic utilization rate of
the general image information, and improve the spatial resolution
and spectral resolution of the original image to facilitate the
monitoring. Image fusion technology refers to the source channel
more of the same target collected by the image after image pro-
cessing and the extraction of each channel of the information and
nally integrated into the same image for observation or further
processing. By the basic theory of compression perception, com-
pressing the original signal is sampled data acquired in the ran-
dom sampling of the adaptive linear projection matrix. Part as a
result of Fourier transform in Fourier matrix under the special
structure, the partial Fourier matrix as a random sampling matrix
can greatly expand the application range of the compressive sen-
sing theory [1621]. Compressed sensing is the main idea of a class
of signals with sparse prior, by a small number of nonlinear
sampling, it contains enough information to close to the original
signal, and then by means of a linear or nonlinear decoding
mechanism to reconstruct the original signal can be high prob-
ability. With the development of compressed sensing theory, there
are many good reconstruction algorithms, the algorithms could be
generally separated into two kinds: one kind is convex optimiza-
tion algorithms and the other kind is the greedy algorithm. Convex
optimization algorithm is the main idea by solving the convex
optimization problem by the high probability close to the objective
function of minimization algorithm to reconstruct the original
signal. In the gure one, we show the primary architecture of deep
learning and compressive sensing theory. Fig. 1 shows the archi-
tecture of deep learning and compressive sensing theory.
To combine the advances of the deep neural network and
compressive sensing theory with the image reconstruction, in this
paper, we research on the natural image super-resolution recon-
struction algorithm based on compressive perception theory and
deep learning model. In the next sections, we will analyze the
issues theoretically and practically.
2. The literature review and basic concepts
2.1. The previous research highlight on image super-resolution
In this sub-section, we will theoretically review on the recent
state-of-the-art algorithms for the task of the image super-
resolution. In [8], the Chens group proposed the novel low-rank
neighbor embedding for single image super-resolution. They rst
extract input low resolution to match images to SIFT key points
and the feature vector. Then select candidate matching points to,
through a robust method to remove singular values, and
assumptions of translation based on geometric constraint model,
get the image of the translational motion of registration para-
meters, then select the corresponding eld center or the specied
image frame as the initial reference frame, and then use the tra-
ditional super resolution reconstruction of the reconstruction of
the framework to obtain the nal results. In [9], Xuelongs group
conducted research on the unied learning framework for single
image super-resolution. The proposed a super-resolution image
restoration method based on the learning, minimizing total var-
iation regularization is introduced into model of image restoration
based on sparse representation. By introducing a new variable, the
model can be converted to two easy to solve optimization pro-
blem, and then uses the alternating iterative method to solve the
model. At the same time, considering the characteristics of image
information, using nonlocal similarity of images and iterative back
projection algorithm to deal with restored image. In [10], the
Pelegs group researched on the statistical prediction model based
on sparse representations for single image super-resolution [11].
Based tracking method is a new method in the eld of signal
sparse representation. It sought from the base in collection of
super complete to get representation of signal, which is the base as
accurately as well as little as the possible according to the original
signal, thereby gaining the intrinsic nature of the signal features.
The base of tracking method is applied to image super resolution.
In the rst place in the phase history domain according to target
attribute scattering model structure for a class of rming the
dictionary, so as to the greatly reduce the solution by the dimen-
sions of the problem, then we design a new iterative algorithm for
 G. Duan et al. / Neurocomputing 208 (2016) 117126 119

quick, get the image of each scattering center position and

amplitude in the high precision estimate, nal phase history
domain target attribute scattering model and generate the more
large-scale phase history data and to generate the phase history
data imaging that is to get higher resolution image. In experi-
mental part, we will simulate our proposed algorithm compared
with these popular algorithms with numerical analysis.

2.2. The theoretical basis of super-resolution reconstruction

As the same for the general linear space imaging system, the
available imaging process type is described as the following
expression one.
g x hx  f x 1

Where theg xdenotes the phase space and thef xrepresents the
object space. So if we want to recover the cut-off frequency of
information, no matter in theory or practical concept it is impos-
sible. The above conclusion is equivalent to the imaging system as
a Fourier lter which has carried on the limits to solution [22,23].
But in fact there are many methods to estimate, this method does
not have the nature of the Fourier lter by using the method that
can be successfully implemented cut-off frequency information
Fig. 2. The interactive projection for obtaining the solution.
recovery which is expressed below [24].
(
F u Gu=H u following solution [2931].
2
Gu H u UF u " #2
  XP X
N X
M qM X
X qN
b
F f g i; j  b
Rk i; j; m; nf m; n f
Analytic function is nature of a well-known, if in nite interval
k1j1i1 m1n1
is known, it will be known everywhere. This means that if the two
6
analytic functions on any given interval are identical, they must be
completely consistent on the whole and is the same function. For fringe area, due to its variance is larger and the human
According to the given analytical function on a range of values to visual masking effect, make the human eye is not sensitive to
the function of the overall reconstruction is called analytical con- parasitic corrugated the image edge area, negligible parasitic rip-
ple effect on the edge and at area due to its own variance is small
tinuation [25,26]. For incoherent imaging, the actual image should
and the human parasitic ripple of the area is most sensitive to the
have the following constraints and properties: the negative and
increase of the variance can be thought of purely caused by
boundedness shown as the follows.
( parasitic corrugated. Therefore, when the image get restoration,
f x 4 0; x A X the edge area of parasitic interference is relatively small with the
3 condence the recovery results. In the gure two, we show the
f x 0; x2=X
interactive projection for obtaining the solution [32].
With the noise condition, the imaging process type is described as Fig. 2 illustrates the interactive projection for obtaining the
the follows. solution.

g x hx  f x nx 4

As a result of the existence of noise, image super-resolution
reconstruction is an ill-posed problem, and we must introduce the
corresponding constraints during recovery. First of all, according to
the observation image type looking for a clear image as the
transcendental characteristics, make its have the maximum simi-
larity with the super resolution image gray level distribution.
Secondly, using the principle of minimum identication informa-
tion construct a restored image and prior image information items
as constraint minimum identications and thus construct a new
model of the super-resolution restoration, ensure that the restored
image grayscale distribution maximum close to the transcendental
characteristics on the image. The regularization restoration after
the introduction of constraint model is denoted as formula 5.
  X
p  
J b
f g k DH k Ek b
f 2 b
f 5
k1
To solve the model, it involves the general large matrix calcu-
lation and the calculation workload is amazing [27,28]. In general,
fuzzy drop point spread function and under-sampling function is
far less than the size of the image size. Therefore, we use the
3. The proposed image super-resolution algorithm
3.1. The deep learning model
Every time faced with a large number of the human perception
data, the brain can easily capture the important information with
low time-consuming. The core problem of articial intelligence is
to copy the brain's ability to accurately said information can do
this effectively. Through the research in recent years, we know
already had some brain mechanism, which promotes the devel-
opment of articial intelligence. The deep learning as opposed to a
shallow learning and now a lot of learning methods are shallow
structure algorithm, and they exist some limitations, such as in the
case of limited samples of complex function ability is limited, its
generalization ability for complex classication problems under
certain constraints. Deep architecture and the deep learning by
learning a kind of nonlinear network structure and the complex
function approximation to achieve the characterization of the
input data distributed said, and in the case of sample set is seldom
study the nature of the data set. Although shallow learning
application is very extensive, it only effective for simple calculation
120 G. Duan et al. / Neurocomputing 208 (2016) 117126

and can't reach the effect of the reaction of the human brain,
which requires the depth of machine learning. All of these show
that shallow learning network has signicant limitations which
inspired us to depth of network modeling research. Convolutional
neural network was the rst real successfully sample that uses the
multi-layer hierarchical structure has the robustness of the net-
work of deep learning approach, through research, the correlation
of data on the space to reduce number of training parameters. At
present, in eld of image recognition, convolutional neural net-
work has become an effective identication method [3336].
Generally, the basic layers of neural networks are for the fea-
ture extraction, each neuron in the input and the previous layer of
local receptive eld is linked together, and extracts the local
characteristics, according to local characteristics to determine the
location of the relationship between it and other feature space. To
analyze the structure of the deep neural network model, we rstly
show the energy and propagation functions as the follows [37].
  X   
p v; 1 h
exp  E v; h; =Z 7
Fig. 3. The proposed deep neural network structure for super-resolution.
     
p v; h; 2 exp E v; h; =Z 8 shown as the follows [3841].

  XI XJ
1XI  2 XJ

  X
I X
J X
I X
J p v; h; 4  wij vi hj vi  bj  aj h j 10
p v; h; 3  wij vi hj  bi vi  aj hj 9 i1j1
2i1 j1
i1j1 i1 j1
It has the deep structure of the neural networks tend to have
The traditional algorithm is widely used in the classic network very strong characteristics of the abstract expression ability, rather
structure, but for deep learning training has met with many dif- than shallow structure or the traditional features better char-
culties: rst, the BP algorithm is supervised learning, training acterization of unknown images, but the problem of deep struc-
need a label sample set, but it can get the data are no labels; ture is training hard which need a lot of labeled training samples,
Second, the BP algorithm in the learning structures of many hid- and gradient diffusion problem. This learning method can be
den layer, the learning process slower; Third, the improper para- understood as before the traditional BP algorithm study adopts the
meter choice will lead to local optimal solution. Therefore, to solve method of unsupervised training beforehand. Through preliminary
it, we use the following steps to generally initialize the parameters training, it can be more complicated for subsequent learning
for the neural network system. provides a good initial weight which makes all the right value of
vector is good for global training area. In the gure three, we use
Fig. 3 to show the proposed deep neural network structure for the
later analysis.

3.2. The compressive perception theory

The compressed sensing theory points out that as long as signal

Step by step a multilayer perceptron signals from left to right.
In turn, according to the calculation of neurons for each layer and
nally produce the output. In general, for multilayer perceptron,
hope for a given input, produce reliable close to the target output.
The weights of different network can make a difference in treat-
ment effect and hope that the response of the network output
generated in statistical sense close to the target response. Usually
adopts the back propagation algorithm to adjust the network
weights and then achieve this goal. In many practical applications,
it was found that this algorithm is very effective. Through the
study of successful, the complex mapping relation between input
and output can be obtained and in the top two oors, the weights
are connected together so as to lower levels of output will provide
clues to a reference or link to the top, so the top will be linked to
the memory of it. After the preliminary training, DBN can take
advantage of the labeled data and BP algorithm to adjust the
structure of network performance. The revised energy function is
is compressible or sparse and if there is a transformation matrix is
not related observation matrix on signal, we can directly obtain
compressed form of the original signal. These are as few objective
observation signals carry the original of all information, at the
receiving end by solving the optimization problem to reconstruct
the original signal with high probability. Due to the sensor part of
the observation involves only the high-dimensional signals into
low dimension space of simple arithmetic, so we can completely
meet the requirements of sensor low energy consumption low
computational complexity. Block compression perception is more
than these advantages, make its have good application prospect in
terms of image acquisition. Traditional BCS there are dis-
advantages, however, when it is in the image reconstruction
although a small amount of calculation, but the quality of recon-
structed images, this is BCS is the problem to be solved urgently. In
addition, in order to get a better quality of the reconstructed
images, it often require sensors to provide more related observa-
tions and this increases the cost of sensors is unfavorable to data
compression. How to solve this contradiction is to study forward
the improvement methods to improve the quality of image
reconstruction [4143]. Sparse matrix can choose the wavelet
transform matrix and the Fourier transform matrix and discrete
cosine transform matrix, in this article use the discrete cosine
 G. Duan et al. / Neurocomputing 208 (2016) 117126
transform matrix which is denoted as the follows.
X S
At this time, observations of how many only decided by the
structure of the signal, without being limited by the Nyquist fre-
quency, so compression perception is very suitable for use in the
larger bandwidth of the signal sampling. Therefore, it could be
denoted as:
Y X 12
In the process of compression perception, the observation
matrix plays an important role, and the matrix is suitable to
determine whether the X in the general process of observation
information is destroyed it directly relates to whether signal can
be accurately reconstructed features. The restriction condition for
the representation is shown below.
   
1  K v22 r t v22 r 1 K v22 13
In traditional perception algorithm, the observation matrix of
each image block with the same features, such as department with
Gaussian random matrix. According to characteristics of the visual
system, the human eye for the low frequency components in
image data with greater sensitivity. The quality of reconstructed
image quality and low frequency component of the reconstruction
error is directly related to the size. In order to improve the quality
of the reconstructed image, some can use similar to the JPEG
quantization matrix of the various elements of the observation
matrix different weighted as new observation matrix, so that we
can get a better effect of image restoration. The method of parti-
cipating in the low position the elements of the observation
matrix of data operation, larger weighted and to the participation
in the high frequency location data computation of matrix ele-
ment, the weighted value is smaller. Because of the reconstruction
in the low frequency part of the picture, when weight is much
bigger, so the low frequency part of the image data has higher
Fig. 4. The optimization of the traditional compressive sensing algorithm.
121
precision of refactoring and the high frequency part due to less
11 weight. Reconstruction error is relatively large. But due to the
inuence to the quality of the images in the low frequency com-
ponent is given priority to, so to build the image of the overall
quality is better. To modify the prior approaches we conduct
optimization in the following procedures [44,45]. Fig. 4 shows the
optimization process of the traditional compressive sensing
algorithm.
In the code, it will be the same size of the image that is divided
into macro block, using the same random matrix respectively on
the macro block measurement and at the decoding end, to
reconstruct the rst each macro block and then these macros can
merge into a whole image. This kind of image reconstruction
model of random measurement matrix storage capacity is small
with simple structure, and the measurement method based on
block is very suitable for real-time systems. When space related or
coherent source, subspace algorithm performance fell sharply
when the failure. However in the practical application, due to the
multipath effect, the signal phenomena often exist. The empty
asked smoothing pretreatment which can to certain extent, solve
the problem of the estimation of coherent sources, but the
operation is limited to a specic geometric conguration, such as
uniform linear array at the same time can also lead to loss of array
aperture effectively and the rise of computational complexity.
3.3. The modied image super-resolution algorithm
Total compressive and network structured model is bounded
the variation function space theory of typical application in the
eld of the image noise removal, etc. It's not smooth constraints
imposed by the solution which can remain in image edge infor-
mation. However, many super-resolution reconstruction practice
shows that this approach to the distinction between the edge
region and plain region ability insufcient, especially in low
122 G. Duan et al. / Neurocomputing 208 (2016) 117126

Fig. 5. The owchart of the overall proposed SR algorithm.

resolution image sequence is affected by the strong noise [4651]. stronger. To optimize the reconstruction result, we should follow
The common characteristic of the variational method is to use the the next restriction condition and optimization issues [5557].
simple mathematical expressions to distinguish between the (
min 0
images features in different areas. The Fig. 5 shows the overall 16
s:t:Dl a y22 r
process.
However, due to nature image composition is very complicated,
Due to the strong correlation between image pixels, gradient in
such as images of edge and gray scale changes in the both the gray
most areas of the image is close to zero, so the image of gradient is
level jump at areas, there is also a complex texture and the
the sparse space. Minimize applied to image reconstruction which
existence of various noise. So this kind of method is still not fully
can better keep the image edges, consider joining to the total
describes the complex and changeable content in natural images.
variation regularization method, and the vectorization, based on
This model combines with the advantages of both sides, in process
the study of a full variational super-resolution restoration model
of image super resolution reconstruction, we can not only inhibit
denoted as the follows.
the reconstruction process of all kinds of noise, and can reduce
the inuence on the at area and texture structure and the pur- X
vv
min Dl a  y22 a1 T i x2 17
pose of making super-resolution reconstruction method has
i1
better performance as well [52,53]. The next formula describes
the model [54]. Set about the terminating condition, this article adopts method
of commonly used, namely when the iteration times meet pre-set
X X  1 2  2 2  
  3 2  4 2 1=2


J f f i;j  f i;j  1  wi;j f i;j  f i;j  value or restored image between residual meet terminated when
i j setting the threshold value of iteration [58]. In order to get the
14 recovery image with better effect through the method of the
restored image, this paper uses the nonlocal average algorithm
Identication of fuzzy set theory is the analysis, judgment,
and iterative back projection algorithm to get the high resolution
fuzzy phenomenon of a kind of effective tool. Due to the uncer-
image of further processing and the next section will illustrate the
tainty of image information is usually are the fuzzy and not ran-
experimental result.
dom, fuzzy set theory to identify different features in the image
area has a good effect. To use the compressive sensing theory, the
following formula describes transformation matrix.
2      3 4. The experimental simulation
u f 11 u f 12 u f 1N
6      7
6 u f 21 u f 22 u f 2N 7 In this section, we simulate our proposed algorithm compared
6
F Compress 6 7 15
7 with other state-of-the-art methodologies. First, interpolation
4   





5
u f M1 u f M2 u f MN algorithm are used to get the initial high resolution image; Sec-
ondly using adaptive adjusting the weight of shock ltering image
According to the Nyquist sampling signal detection, under the gradient, reduce the edge of the image spread, through the for-
condition of compressed sampling is deduced under the condition ward diffusion process to eliminate noise. Before the nal impact
of signal detection. It does not consider the effect of rake merger will lter and diffusion to effectively combine resulting high
and this article will rake the combined signal compression test, resolution image. The experimental results, compared with the
due to the compression related to enhance the signal-to-noise image super-resolution method based on the general interpola-
ratio of core received signal, so its anti-disturbance ability is tion, impact traditional ltering model and the other several
 G. Duan et al. / Neurocomputing 208 (2016) 117126 123

Fig. 6. The dictionary learned by the compressive sensing model.

Fig. 7. The simulation of the algorithms on the gray scale natural images.

Lena Elaine Baboon Peppers

Bicubic 26.95 28.91 20.53 25.71
WZP 26.68 28.15 18.12 23.79
Bilinear 26.29 25.53 20.49 25.11
NEDI 28.79 29.93 21.09 28.39
HMM 28.86 30.49 21.44 29.55
HMT SR 28.98 30.49 21.44 29.59
DASR 27.01 27.99 17.92 23.79
WZP-CS 29.22 30.59 21.49 30.01
Carey 28.83 30.41 21.57 29.53
Proposed 34.87 32.69 23.22 32.31
Fig. 8. The numerical statistical result of the experiments (PSNR/dB).
124 G. Duan et al. / Neurocomputing 208 (2016) 117126

Fig. 9. The additional visuable simulation on the proposed algorithm.

representative other super-resolution methods, the method 5. Summary and conclusion

improves the image resolution, subjective effect is improved
obviously, the objective effect must be improved as a whole. In the
Fig. 6, we show the dictionary learned by the compressive sensing
model, we could see that feature extracted is much more sufcient
compared with the traditional methods. In the Fig. 7, we illustrate
the simulation of the algorithms on the gray scale natural images.
We could easily conclude that our method outperforms the other
methods. In the Fig. 8, we show the numerical statistical result of
the experiments. Finally, in Fig. 9, we show the additional visuable
simulation on the proposed algorithm [59,60]. From the simula-
tion result indicated below, we could conclude that through the
deep learning and compressive sensing based method is robust
and effective. In addition, the PSNR is dened as the follows.
  between images, and thus can recover on the visual effect than the
2
PSNR 10  log 10 2n  1 =MSE 18
In this paper, we research on the natural image super-
resolution reconstruction algorithm based on compressive per-
ception theory and deep learning model. Theoretically, any natural
images are unlimited bandwidth and, in fact, in the process of
image acquisition, image and many factors such as atmospheric
disturbances, relative motion, focusing on the bad, the CCD sam-
pling rate lower which will cause the image resolution. Improve
the image resolution method are many, but most of the methods
are for single image, its effect is very limited. With the emergence
of multiple video frame camera system, image super-resolution
reconstruction, more and more get the attention of people.
Because it can make use of similar complementary information
original image of low resolution image. Super resolution recon-
struction abstract images as numerical function optimization
 G. Duan et al. / Neurocomputing 208 (2016) 117126
problem, using deep learning good parallelism and the char-
acteristics of global optimization using genetic algorithm and
gradient descent of hybrid genetic algorithm combining the ori-
ginal high resolution images of the optimal estimation, the
experiment got the signal-to-noise ratio is better than that of
super resolution results of conjugate gradient method. Deep
learning is a focus research eld in recent years which plays an
important role in machine learning by establishing, simulation,
hierarchical structure of the brain to realize the external input data
from low level to high level feature extraction which can explain
external data. Therefore, the combination of deep structure and
super resolution analysis will obtain better performance. Our
experimental result proves the robustness and feasibility of the
proposed algorithm. In the future, we will enhance the current
method through the more analysis on the network structure
enhancement.
Acknowledgements
This research is supported by the project of Xi'an Science and
Technology fund: Technology Research of Urban Intelligent
Transportation System Based on big data (No. cxy1509(14)) and
the project supported by the Natural Science Foundation of the
Shaanxi Province education department: exible recommendation
system based on hybrid strategy research and development (No.
14JK1522).
References
[1] Jiwen Lu, Venice Erin Liong, Xiuzhuang Zhou, Jie Zhou, Learning compact
binary face descriptor for face recognition, IEEE Trans. Pattern Anal. Mach.
Intell. vol. 37 (2015).
[2] Y. Yan, E. Ricci, G. Liu, N. Sebe, Egocentric daily activity recognition via mul-
titask clustering, IEEE Trans. Image Process. 24 (10) (2015) 29842995.
[3] Sven Proppert, Steve Wolter, Thorge Holm, Teresa Klein, Sebastian van de
Linde, Markus Sauer, Cubic B-spline calibration for 3D super-resolution mea-
surements using astigmatic imaging, Opt. Express 22 (9) (2014) 1030410316.
[4] Marissa K. Lee, Prabin Rai, Jarrod Williams, Robert J. Twieg, William
E. Moerner, Small-molecule labeling of live cell surfaces for three-dimensional
super-resolution microscopy, J. Am. Chem. Soc. 136 (40) (2014) 1400314006.
[5] Yongqiang Zhao, Jingxiang Yang, Cheung.-Wai. Chan, Hyperspectral imagery
super-resolution by spatialspectral joint nonlocal similarity, Sel. Top. Appl.
Earth Obs. Rem. Sens., IEEE J. 7 (6) (2014) 26712679.
[6] Hui Yang, Norman Moullan, Johan Auwerx, Martin A.M. Gijs, Superresolution
biological microscopy using virtual imaging by a microsphere nanoscope,
Small 10 (9) (2014) 17121718.
[7] Y. Yan, E. Ricci, R. Subramanian, G. Liu, N. Sebe, Multi-task linear discriminant
analysis for multi-view action recognition, IEEE Trans. Image Process. 23 (12)
(2014) 55995611.
[8] Xiaoxuan Chen, Chun Qi, Low-rank neighbor embedding for single image
super-resolution, Signal Process. Lett., IEEE 21 (1) (2014) 7982.
[9] Jifei Yu, Xinbo Gao, Dacheng Tao, Xuelong Li, Kaibing Zhang, A unied learning
framework for single image super-resolution, Neural Netw. Learn. Syst., IEEE
Trans. 25 (4) (2014) 780792.
[10] Tomer Peleg, Michael Elad, A statistical prediction model based on sparse
representations for single image super-resolution, Image Process., IEEE Trans.
23 (6) (2014) 25692582.
[11] Baraka Jacob Maiseli, Ogada Achieng Elisha, Huijun Gao, A multi-frame super-
resolution method based on the variable-exponent nonlinear diffusion reg-
ularizer, EURASIP J. Image Video Process. 2015 (1) (2015) 116.
[12] Jiwen Lu, Yap.-Peng. Tan, Gang Wang, Discriminative multimanifold analysis
for face recognition from a single training sample per person, IEEE Trans.
Pattern Anal. Mach. Intell. 35 (1) (2013) 3951.
[13] Xiaodong Li, Yun Du, Feng Ling, Super-resolution mapping of forests with
bitemporal different spatial resolution images based on the spatial-temporal
Markov random eld, Sel. Top. Appl. Earth Obs. Remote Sens., IEEE J. 7 (1)
(2014).
[14] Jing Gao, Ye Wang, Mingjun Cai, Yangang Pan, Haijiao Xu, Junguang Jiang,
Hongbin Ji, Hongda Wang, Mechanistic insights into EGFR membrane clus-
tering revealed by super-resolution imaging, Nanoscale 7 (6) (2015)
25112519.
[15] ShangGuo Hou, Le Liang, Su.Hui Deng, JianFang Chen, Qing Huang, Ya Cheng,
ChunHai Fan, Nanoprobes for super-resolution uorescence imaging at the
nanoscale, Sci. China Chem. 57 (1) (2014) 100106.
125
[16] Julien Caron, Clment Fallet, Jean-Yves Tinevez, Lionel Moisan, L.
Philippe Braitbart, Gabriel Y. Sirat, Spencer L. Shorte, Conical diffraction illu-
mination opens the way for low phototoxicity super-resolution imaging, Cell
Adhes. Migr. 8 (5) (2014) 430439.
[17] Natalia H. Revelo, Dirk Kamin, Sven Truckenbrodt, Aaron B. Wong,
Kirsten Reuter-Jessen, Ellen Reisinger, Tobias Moser, Silvio O. Rizzoli, A new
probe for super-resolution imaging of membranes elucidates trafcking
pathways, J. Cell Biol. 205 (4) (2014) 591606.
[18] Chujian Bi, Qiushi Zhang, Rui Bao, and Haoxiang Wang, SAR image restoration
and change detection based on game theory, in: Proceedings of the 2014
International Conference on Intelligent Computing and Internet of Things
(ICIT), IEEE, 2015, pp. 5558.
[19] Uwe Schmidt, Jeremy Jancsary, Sebastian Nowozin, Stefan Roth, and Carsten
Rother. Cascades of Regression Tree Fields for Image Restoration, 2014, arXiv
preprint arXiv:1404.2086.
[20] Y. Yan, E. Ricci, R. Subramanian, O. Lanz, N. Sebe, No matter where you are:
exible graph-guided multi-task learning for multi-view head pose classi-
cation under target motion, in: IEEE International Conference on Computer
Vision, 2013.
[21] Jiwen Lu, Gang Wang, Pierre Moulin, Human identity and gender recognition
from gait sequences with arbitrary walking directions, IEEE Trans. Inf. For-
ensics Secur. 9 (n1) (2014) 4151.
[22] Ernie Esser, Xiaoqun Zhang, Nonlocal patch-based image inpainting through
minimization of a sparsity promoting nonconvex functional, Dept. Math., Univ.
California Irvine, Irvine, CA, USA, Tech, Rep, 2014.
[23] W.K. Chooi, S. Matthews, M.J. Bull, S.K. Morcos, Multislice helical CT: the value
of multiplanar image reconstruction in assessment of the bronchi and small
airways disease, Br. J. Radiol. (2014).
[24] Emil Y. Sidky, Rick Chartrand, John M. Boone, Xiaochuan Pan, Constrained
minimization for enhanced exploitation of gradient sparsity: application to CT
image reconstruction, Transl. Eng. Health Med., IEEE J. 2 (2014) 118.
[25] O. Honda, M. Yanagawa, A. Inoue, A. Kikuyama, S. Yoshida, H. Sumikawa,
K. Tobino, M. Koyama, N. Tomiyama, Image quality of multiplanar recon-
struction of pulmonary CT scans using adaptive statistical iterative recon-
struction, Br. J. Radiol. (2014).
[26] Peng , Thomas Page, Guojie Luo, Wentai Zhang, Pei Wang, Peng Zhang, Peter
Maass, Ming Jiang, Jason Cong, FPGA Acceleration for Simultaneous Medical
Image Reconstruction and Segmentation, in: Proceedings of the 2014 IEEE
22nd Annual International Symposium on Field-Programmable Custom
Computing Machines (FCCM), IEEE, 2014, pp. 172172.
[27] Ding Lu, Xose Lus Den-Ben, Christian Lutzweiler, Daniel Razansky, Vasilis
Ntziachristos, Image reconstruction in cross-sectional optoacoustic tomo-
graphy based on non-negative constrained model-based inversion, i:n Eur-
opean Conferences on Biomedical Optics. International Society for Optics and
Photonics, 2015, p. 953919.
[28] Q. Zhao, D. Meng, Z. Xu, W. Zuo, Y. Yan, L1-norm low-rank matrix factorization
by variational bayes, IEEE Trans. Neural Netw. Learn. Syst. 26 (4) (2015)
825839.
[29] Yulin Xiao, Shangping Zhong, An improved online multiple kernel classica-
tion algorithm based on double updating online learning, in: Proceedings of
the 2014 International Conference on Cloud Computing and Internet of Things
(CCIOT), IEEE, 2014, pp. 109113.
[30] Michael R.H. Hill, Itzhak Fried, Christof Koch, Quantication and classication
of neuronal responses in kernel-smoothed peristimulus time histograms, J.
Neurophysiol. 113 (4) (2015) 12601274.
[31] Haoxiang Wang, Jingbin Wang, An effective image representation method
using kernel classication, in: Proceedings of the 2014 IEEE 26th International
Conference on Tools with Articial Intelligence (ICTAI), IEEE, 2014, p. 853.
[32] Yudong Zhang, Shuihua Wang, Zhengchao Dong, Classication of Alzheimer
disease based on structural magnetic resonance imaging by kernel support
vector machine decision tree, Prog. Electromagn. Res. 144 (2014) 171184.
[33] Jiwen Lu, Xiuzhuang Zhou, Yap.-Peng. Tan, Yuanyuan Shang, Jie Zhou,
Neighborhood repulsed metric learning for kinship verication, IEEE Trans.
Pattern Anal. Mach. Intell. 36 (2) (2014) 331345.
[34] Qi Yuan, Weidong Zhou, Shasha Yuan, Xueli Li, Jiwen Wang, Guijuan Jia, Epi-
leptic EEG classication based on kernel sparse representation, Int. J. neural
Syst. 24 (04) (2014) 1450015.
[35] Shenghua Gao, Liang.-Tien. Chia, Wai.-Hung. Tsang, Zhixiang Ren, Concurrent
single-label image classication and annotation via efcient multi-layer group
sparse coding, Multimed., IEEE Trans. 16 (no. 3) (2014) 762771.
[36] Y. Yan, H. Shen, G. Liu, Z. Ma, C. Gao, N. Sebe, GLocal tells you more: coupling
glocal structural for feature selection with sparsity for image and video clas-
sication, Comput. Vision. Image Underst. volume 124 (7) (2014) 99109.
[37] Yongqiang Cao, Yang Chen, Deepak Khosla, Spiking deep convolutional neural
networks for energy-efcient object recognition, Int. J. Comput. Vision. 113 (1)
(2015) 5466.
[38] Hou Le, Dimitris Samaras, Tahsin M. Kurc, Yi Gao, James E. Davis, Joel H. Saltz,
Efcient Multiple Instance Convolutional Neural Networks for Gigapixel
Resolution Image Classication, 2015, arXiv preprint arXiv:1504.07947.
[39] Caglar Gulcehre, Kyunghyun Cho, Razvan Pascanu, Yoshua Bengio, Learned-
norm pooling for deep feedforward and recurrent neural networks, inL
Machine Learning and Knowledge Discovery in Databases, Springer Berlin
Heidelberg (2014), p. 530546.
[40] Y. Yan, Y. Yang, D. Meng, G. Liu, W. Tong, A. Hauptmann, N. Sebe, Event
oriented dictionary learning for complex event detection, IEEE Trans. Image
Process. 24 (6) (2015) 18671878.
126 G. Duan et al. / Neurocomputing 208 (2016) 117126

[41] Deng Li, Jianshu Chen, Sequence classication using the high-level features [59] Zhangyang Wang, Nasser M. Nasrabadi, Thomas S. Huang, Semisupervised
extracted from deep neural networks, in: Prceedings of the 2014 IEEE Inter- hyperspectral classication using task-driven dictionary learning with Lapla-
national Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, cian regularization, Geosci. Remote Sens., IEEE Trans. 53 (3) (2015) 11611173.
2014. [60] Xixuan Han, Line Clemmensen, Regularized generalized eigen-decomposition
[42] William R. Foland Jr, James H. Martin, Dependency-based semantic role with applications to sparse supervised feature extraction and sparse dis-
labeling using convolutional neural network, in: Proceedings of the Fourth criminant analysis, Pattern Recognit. (2015).
Joint Conference on Lexical and Computational Semantics, 2015, pp. 279288.
[43] Tobias Domhan, Tobias Springenberg, Frank Hutter, Extrapolating learning
curves of deep neural networks, in: ICML 2014 AutoML Workshop, June 2014.
[44] Earnest Ijjina, Paul, C. Krishna Mohan, Human Action Recognition Using
Ganglong Duan received the B.S. and M.S. degrees in
Action Bank Features and Convolutional Neural Networks Computer Vision-
Department of Information Management from Xian
ACCV 2014 Workshops, Springer International Publishing,, 2014. University of Technology, Xian, China, in 2002 and
[45] Yuming Li, Lai-Man Po, Chun-Ho Cheung, Xuyuan Xu, Litong Feng, Fang Yuan,
2005, respectively. He is currently a lecturer with the
Kwok-Wai Cheung, No-Reference Video Quality Assessment with 3D Shearlet Department of Information Management, Xian Uni-
Transform and Convolutional Neural Networks, 2015. versity of Technology, Xian, China. Currently, he is
[46] Dong Wang, Jing Yang, Jiankang Deng, Qingshan Liu, Hierarchical convolu- working towards his Ph.D. degree in School of Eco-
tional neural network for face detection, in: Image and Graphics, Springer nomics and Management at Xian University of Tech-
International Publishing (2015), p. 373384. nology, Xian, China. His current research interests
[47] Jiwen Lu, Yap.-Peng. Tan, Cost-sensitive subspace analysis and extensions for include intelligence information processing and digital
face recognition, IEEE Trans. Inf. Forensics Secur. 8 (3) (2013) 510519. image processing.
[48] Soren Goyal, Paul Benjamin, Object Recognition Using Deep Neural Networks:
A Survey, 2014. arXiv preprint arXiv:1412.3684.
[49] Hongsheng Li, Rui Zhao, Xiaogang Wang. Highly eFcient Forward and Back-
ward Propagation of Convolutional Neural Networks for Pixelwise Classica-
tion, 2014. arXiv preprint arXiv:1412.4526.
[50] Lszl Tth, Combining time-and frequency-domain convolution in convolu- Wenxiu Hu is a Doctor of Management, he is currently
tional neural network-based phone recognition, In: Proceedings of the 2014 the dean of school of Economics and Management,
IEEE International Conference on Acoustics, Speech and Signal Processing Xi'an University of Technology, Xi'an, China. He is also a
(ICASSP), IEEE, 2014, pp. 190194. Professor, a doctoral tutor, the academic leaders of
[51] Toni Hirvonen, Classication of spatial audio location and content using Business Administration. His main research areas
convolutional neural networks, in: Audio Engineering Society Convention, including nancial engineering and risk management,
138, Audio Engineering Society, 2015. corporate mergers and acquisitions, investment man-
[52] Biao Leng, Yu Liu, Kai Yu, Xiangyang Zhang, Zhang Xiong, 3D object under- agement and capital operation.
standing with 3D Convolutional Neural Networks, Inf. Sci. (2015).
[53] Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, Jun Zhao, Relation classi-
cation via convolutional deep neural network, in: Proceedings of COLING,
2014, pp. 23352344.
[54] Jiwen Lu, Yap.-Peng. Tan, Regularized locality preserving projections and its
extensions for face recognition, IEEE Trans. Syst., Man., Cybern.Part B:
Cybern. 40 (3) (2010) 958963.
[55] Mingsheng Long, Jianmin Wang, Guiguang Ding, Sinno Jialin Pan, Philip S. Yu,
Adaptation regularization: A general framework for transfer learning, Knowl. Jianren Wang is currently an associate professor with
Data Eng., IEEE Trans. 26 (5) (2014) 10761089. the Department of Management Science and Engi-
[56] Ahmad Mozaffari, L. Azad Nasser, Optimally pruned extreme learning machine neering, Xi'an University of Technology, Xi'an, China.
He is the director of the Department of Management
with ensemble of regularization techniques and negative correlation penalty
Science and Engineering at Xi'an University of Tech-
applied to automotive engine coldstart hydrocarbon emission identication,
nology. His current research interests include intelli-
Neurocomputing 131 (2014) 143156.
gence information processing and data mining.
[57] Chujian Bi, Haoxiang Wang, Rui Bao, SAR image change detection using reg-
ularized dictionary learning and fuzzy clustering, in: Proceedings of the 2014
IEEE 3rd International Conference on Cloud Computing and Intelligence Sys-
tems (CCIS), IEEE, 2014, pp. 327330.
[58] Gang Niu, Bo Dai, Makoto Yamada, Masashi Sugiyama, Information-theoretic
semi-supervised metric learning via entropy regularization, Neural Comput.
26 (8) (2014) 17171762.