Ising Model with Hopfield Network
Hemanta Bhattarai
University of NotreDame
hbhattar@nd.edu
ABSTRACT
Hopfield network along with the Glauber form of activation
energy is used to study the Ising model. 1D Ising model
dont show phase transition however, 2D Ising model shows
phase transition. 1D lattice can be considered as a cycle
graph with edge weighted by interaction strength. The 2D
lattice can be transformed to cycle graph and extra edges
corresponding to non-neighbour connection. So, the cause
of phase transition is due to the non-neighbour interactions
in a 1D cycle graph. Considering this, the transition tem-
perature of the random graph was studied.
Keywords
Ising model, Hopfield Network,
1. INTRODUCTION:
Optimization is always hot topic in physics, mathematics
and computer science. Many methods both theoretical and
simulation has been introduced to find the optimum solu-
tion for a given problem. Travelling salesmen problem and
8 queen problem are some of the NP hard optimization prob-
lems that has been solved using networks and optimization
algorithms. Most important network, that is used to solve
many optimization problem is Hopefield networks. Hope-
filed network is used to solve Travelling salesmen problem
and N-queen problems by various researchers by introduc-
ing the optimization or error function[20, 13] .A nonlinear
neural framework, called the Generalized Hopfield network,
is proposed, which is able to solve in a parallel distributed
manner systems of nonlinear equations. The method is ap-
plied to the general nonlinear optimization problem.[19]
Hopefiled network is one of the most important network in
physics as most of the problems in physics involves the opti-
mization of a function with some constraints on it. Hopefiled
network structure of Artificial Neural Network (ANN) has
been used in different research areas.ANN is used in devel-
opment of force field[5, 4, 7], estimation of parameters in
Density Functional theory[18, 2] and simulation in molecu-
lar dynamics[14, 3, 1]. Hopfield network is used to study
the simulation of the dynamcis of Cyclohexane. The depen-
dence in the dynamics in the temperature of the system is
governed by Glabular dynamcis[10]. The strucutre of the
Globular dyanamics and Boltzman machine in neural net-
work is similar.
N-vector model is general model in mathematics where N
gives the dimensionlaity of hte problem. N=0 is the ran-
dom walk, N=1 is ising model, N=2 is XY model, N=3 is
Heisenberg model and N=4 gives a toy model that expalins
the Higgs bosons of the Standard model. The Hamiltonian
of the N-vector model is similar to Ising model in ferromag-
netism so the simulation used for Ising model can be used to
simulate N-vector model with some modifications. So, mim-
micry of Ising model can be used to expalin most complex
N=4 model i.e to expalin the Higgs bosons.
Ising model has been used to study different network struc-
ture and find the transition temperature of the netwroks.[8,
11]
This paper has five sections, the recent works are mentioned
in section 1, the section 2 includes the theory, section 3
describes the simulaiton method and section 4 and 5 has
results and its discussion.
2. THEORY:
One of the most interesting phenomenon in physics is the
ferromagnetism where the some metals get a finite fraction
of spin of atoms to be spontaneously polarized in same di-
rection and give macroscopic magnetic field.
Ising model is mathematical model to explain observed fer-
romagnetism [9]. And, taking help of statistical mechanics,
the model is exactly solvable for 1D and 2D. But, some ap-
proximations is required to solve beyond 1D. 2D[12].
Use of networks to simulate the Ising model could be used
to study different netwrok structures. Also, Ising model be-
yond 1D predicts the phase transition and could be easily
studied by simulating using ANN. As, ANN converges to the
local minima in the energy landscape.
2.1 Ising model:
The N-vector model with N=1 is Ising model. The Hamil- In above expression, when B = 0, M=0 at all the temper-
tonian for an Ising lattice can be written as[16] atures. So, the order parameter (that is non zero at the
N
phase transition) i.e magnetization is 0 at all the tempera-
H=
X
ij si sj B
X
si (1) ture. Hence, no phase transition.
<i,j> i=1
P 2.1.3 Interaction not 0 for 2D lattice:
where <i,j> denotes sum over nearest-neighbour pairs ij, Onsager gave the closed solution for the 2D ising model ex-
ij is the magnetic interactions energy between nearest-neighbour plaining the phase transition.[15] The order-disorder transi-
i and j, si is z component of spin at its lattice site and B tion i.e phase transition temperature T=Tc is given by con-
is the external magnetic field. For spin 21 objects, si =+1(- dition
1) if the spin of site is oriented in the positive (negative)
z-direction. sinh(2J) = 1
The solution for this equation is
For B=0, if ij < 0, then the lattice will have lowest energy
1 2
when all the lattice sites have spin up or all the lattice sites = 2.269
have spin down (ferromagnetism). And, if ij > 0, B=0, the J ln(1 + 2)
configuration in which neighbouring spins are oriented op- The magnetization for temperature T<Tc (Transition tem-
posite to one another will be favored (anti-ferromagnetism). perature)
1
M = 1 sinh4 (2J) 8

The configuration probability is given by Boltzmann-distribution
with = k T 0
eH()
P () = (2)
Z
where H() is the energy at the spin state where is the
n-tuple (s1 , s2 ...si , ....sn ) representing the spin state of the
lattice. And Z = eH() .
P

onszar.png
2.1.1 Case I: No-interaction and B:
The magnetization when the interaction is 0.
M = tanh (B/)

Figure 2: Phase transition in 2D predicted by Onszar.

2.2 Hopfield Network:

j0-eps-converted-to.pdf The mathematical structure of the Ising model at 0K and
Hopfield model of the Hopfield Networks is isomorphic[17].

Let W denote the weight matrix of Hopfield network of n

units and let be n-dimensional row vector of unit threshold.
The energy E(x) of state x of the network is given by:
1
E(x) = xW xT + xT (3)
2
Figure 1: Magnetization when interaction is 0. i.e
1X X
E(x) = wij xi xj + i xi (4)
2 i,j i
There is no magnetization when B=0 and has some magne-
tization for non-zero B. Greater the value of B greater will
be the magnetization. As B , M 1. Magnetization And, Hopfield networks finds a local minimum of the energy
is defined as the average of total spins in the system. function. And, all the physical system including Ising model
tries to attain the minimum energy (local or global)
2.1.2 Case II: Interactions not 0 1D lattice
The ising model in 1D is exactly solvable. The result ob- So, implementing Hopfield network to simulate Ising model
tained from Transfer matrix method shows there occurs no is useful to study the ferromagnetism (anti-ferromagnetism)
phase transition in 1D[16]. The magnetization is in 1D or more dimension.

sinh(B) The problem we have in Hopfield network is, its isomorphism

M= p
sinh2 (B) + exp(4J) with Ising model at T=0K but we are interested at T>0K.
One of the possible solution is implemented by Liebovitch If Tij < R, dont change the state else change the
using Glauber dynamics et.al[10]. state

Iterate from second step for other nodes as well.

2.3 Algorithm:
A Hopfield network with n units and asynchronous dynam- After several iteration the network will have minimum
ics, which starts from any given network state, eventually local energy state. i.e the equilibrium state with the
reaches a stable state at a local minimum of the energy different orientation of spin will be found.
function[17]. And the dynamics of the network is given by
following algorithm. let us consider each node has binary
states i.e 1 or 0. 2.4 Random Graphs:
1. Assign random states to each nodes The periodic 1D lattice is a cyclic graph (Cn ) with n-nodes.
2. Choose a node in random and find the energy difference Each neighbouring nodes connected with edge interacts with
(E) to change the state at that network. each other. Erdos-Renyi graph is a graph where nodes are
3. If (E) is negative i.e energy of final configuration is less connected to each other with a probablity p. The non-
change the state else dont change the state. neighbour connected with a edge with some probability in
4. choose other node at random and iterate the process from cyclic graph forms a small world network. The modified
3. Ising model with some non-neighbouring interactions are
5. After some iterations the network will be in a state of studied.[6] The Hamiltonian for the modified Ising model
local minimum. is:
X X X
H= Ji,j si sj Iij si sj B si
In the above case, the activation function is (E) defined i,j i,j i
as
(E) = 1E where Ji,j = J for nearest neighbours else 0, Iij = I for the
= 0E > 0 nodes that have connecting edge, B is the applied magnetic
field and si is the spin in the ith node. The aympotic be-
haviour is studied.[6]
However, at finite temperature the activation function can For TIc  1
be replaced by the transition probability for the change in
state at a node. For two state system, the transition prob- 2J
Tc =  
ability is derived using the master equation and principle of ln 1
pln3
detailed balance usually have two forms[10].
1. Metropolis form
I
Tij = 1, f or E < 0 Also, for Tc
1
 
E 2J
= exp , f or E 0 Tc =
T
 
J
ln 
J

pIln pI
2. Glauber form
1
Tij =
1 + exp E

3. SIMULATION:
T The above algorithm was implemented to study the equi-
where E = Ej Ei and kB = 1 (Boltzmanns Constant) librium state of different systems. At the end of simulation
in both the forms. we get the spins in each nodes (si =+1 or si =-1). So, total
average magnetization is given by
In the neural network, the activation function has the form PN
of the sigmoid function i si
M=
N
1
g(x) =
(1 + exp(x))
Simulatin was done in one dimensional Ising model with pe-
riodic boundary condition which is basically a cycle graph
Glauber forms transition probability resembles the sigmoid (Cn ) with n-nodes that has different spin states. The inter-
function, so Glauber form was used as activation function action between the nearest neighbour is given by the weight
during the dynamics of Hopfield network. of the edges in Cn , each weight being equal to J (1 in our
case). The Hamiltonian of the Hopfield network in case of
The algorithm for the dynamics of Hopfield network at finite Ising model is (3), where x is the vector that represents the
temperature is spin state of system, W the adjacency matrix of Cn and is
vector with each elements equal to external magnetic field.
Simulation was done with B=0 and B6=0.
Assign random state to each node i.e +1 or -1 (Spin
state) The simulation was also done for the periodic 2D lattice
Choose a node randomly and find E with J equal to 1. The simulation was done with 10000
nodes in the grid. The periodic boundary condition makes
Generate a random number 0<R<1. the grid as a torus analog to ring in case of 1D. Simulation
was performed at different temperatures. Also, B=0 and
B6=0 was tuned during the simualtion.

Theoretically, there exist the phase transition in 2D Ising

model and not in 1D. The grid of the nodes in 2D can be
transformed into a graph where nodes forms a ring. But in
2D grid each node has edge with 4 nodes, so we can represent j11Dm-eps-converted-to.pdf
the 2D grid as 1D nodes placed on ring with edges between
non neighbouring nodes, i.e the adjacency matrix for the 2D
grid transformed to a ring is not a tri-diagonal matrix. But
the 1D Ising model has the adjacency matrix as tri-diagonal.
So, we can infer the existence of elements in matrix beside
the tri-diagonal is main cause for phase transition.

Introducing a non-nearest neighbouring interaction in the

Figure 4: No phase transition in 1D Ising model
1D ising model will lead to phase transition. So, defining a
graph, which has in addition to a cycle graph, non-nearest
edges with some probability p will result in the phase tran-
sition. Simulation was done to study the phase transition
with various probability p and different values of interaction
strength for non-nearest neighbour.

4. RESULTS:
The simulation results for 1D Ising with J=0 and non zero
B is consistent with theory. The graphical pattern in figure
3 matches the theoretical prediction. The 1D ising model
doesnt predict phase transition which is seen in figure 4.
The order parameter is 0 for all the temperatures except at
T=0. So, the transition temperature is at T=0, however
this doesnt count for phase transition.
The simulation was also carried out for 2-D lattice with pe-
riodic boundary condition and the critical temperature be-
tween 2.0 and 2.5 was seen which matches the theoretical
2
prediction Tc = ln(1+ 2)
= 2.26 The specific heat capacity,
magnetic susceptibility were analyzed. The specific heat and
magnetic susceptibility has sudden jump at the transition
temperature in the simulation which captures the theoret-
ical behaviour i.e the specific heat and magnetic suscepti-
bility is infinite at transition temperature. The transition
temperature is not precisely 2.26 in the simulation plot, but
when the three plots of specific heat, energy, magnetization,
susceptibility, the transition temperature is between 2.0 and
2.5. The fall of the magnetization starts nearly about 2.1
and matches the theoretical prediction given by Onszar.

j01DBm-eps-converted-to.pdf phaseTransition2D-eps-converted-to.pdf

Figure 3: Magnetization when interaction is 0. Figure 5: Phase transition in 2D Ising model.

 phaseTransition2chi-eps-converted-to.pdf realworld_Ji1_diffp-eps-converted-to.pdf

Figure 6: Magnetic Susceptiblity in 2D Ising model. Figure 9: Phase transition in real world network with dif-
ferent values of p and for J=I=1.

phaseTransition2Denergy-eps-converted-to.pdf

realworld_ineg-eps-converted-to.pdf

Figure 7: Energy in 2D Ising model.

Figure 10: No Phase transition in real world network when

the non-neighbour interaction is negative.

phaseTransition2specificheat-eps-converted-to.pdf

Figure 8: Phase transition in 2D Ising model.

realworld_i2-eps-converted-to.pdf
When simulation was performed in small world networks as
described in theory section. The transition temperature was
found between 1 and 3 when J=I=1. And, the transition
temperature was independent of the p as shown in figure 9.
When the interaction I was negative, no phase transition was
seen. The transitions temperature for various values of p for
I=2 was studied and the dependence of transition tempera-
ture on p was found. The larger the value of p, the greater
is the transition temperature. The direct dependence of the Figure 11: Phase transition in real world network with dif-
transition temperature in the value of interaction I was seen ferent values of p and for J=1, I=2.
 designed to solve the n-queens problem. n Biological
Cybernetics, pages 375379.
[14] S. K. Natarajan and J. Behler. Neural network
molecular dynamics simulations of solid-liquid
interfaces: water at low-index copper surfaces. Phys.
Chem. Chem. Phys., 18:2870428725, 2016.
realworld_ichangeP04-eps-converted-to.pdf [15] L. Onsager. Crystal statistics. i. a two-dimensional
model with an order-disorder transition. Phys. Rev.,
65:117149, Feb 1944.
[16] L. E. Reichl. A Modern Course in Statistical Physics.
Arnold, London, 1980.
[17] R. Rojas. Neural Networks. Springer-Verlag, Berlin,
1996.
[18] M. Szaleniec, M. Witko, R. Tadeusiewicz, and
Figure 12: Phase transition in real world network with dif- J. Goclon. Application of artificial neural networks
ferent values of I and for J=1 and p=0.4. and dft-based parameters for prediction of reaction
kinetics of ethylbenzene dehydrogenase. Journal of
Computer-Aided Molecular Design, 20(3):145157,
5. CONCLUSION: 2006.
6. REFERENCES [19] A. G. Tsirukis, G. V. Reklaitis, and M. F. Tenorio.
[1] P. M. Agrawal, A. N. A. Samadh, L. M. Raff, M. T. Nonlinear optimization using generalized hopfield
Hagan, S. T. Bukkapatnam, and R. Komanduri. networks. Neural Computation, 1(4):511521, Dec
Prediction of molecular-dynamics simulation results 1989.
using feedforward neural networks: Reaction of a c2 [20] a. D. R. V. P. Yogeesha C. B. An approach to design
dimer with an activated diamond (100) surface. The and implementation of dynamic geometric travelling
Journal of Chemical Physics, 123(22):224711, 2005. salesperson problem using hopefield neural network.
[2] R. M. Balabin1 and E. I. Lomakina. Neural network International Journal of Technology and Engineering
approach to quantum-chemistry data: Accurate System (IJTES), pages 5864.
prediction of density functional theory energies. The
Journal of Chemical Physics, 131(7), 2009.
[3] M. Benedict and J. F. Maguire. Molecular dynamics
simulation of nanomaterials using an artificial neural
net. Phys. Rev. B, 70:174112, Nov 2004.
[4] K.-H. Cho, K. T. No, and H. A. Scheraga. A
polarizable force field for water using an artificial
neural network. Journal of Molecular Structure,
641(1):77 91, 2002.
[5] R. O. Day and G. B. Lamont. Force field
approximation using artificial neural networks. In
Proceedings of the 2004 Congress on Evolutionary
Computation (IEEE Cat. No.04TH8753), volume 1,
pages 10201027 Vol.1, June 2004.
[6] M. Gitterman. Small-world phenomena in physics: the
ising model. Journal of Physics A: Mathematical and
General, 33(47):8373, 2000.
[7] C. M. Handley and P. L. A. Popelier. Potential energy
surfaces fitted by artificial neural networks. The
Journal of Physical Chemistry A, 114(10):33713383,
2010. PMID: 20131763.
[8] C. P. Herrero. Ising model in clustered scale-free
networks. Phys. Rev. E, 91:052812, May 2015.
[9] K. Huang. Statistical Mechanics. Wiley,UK, 2 edition,
2003.
[10] S. Liebovitch, L, D. Arnold, N, and Y. Selector, L.
Neural networks to compute molecular dynamcis.
Journal of Biological Systems, pages 193228.
[11] A. N. Malmi-Kakkada, O. T. Valls, and C. Dasgupta.
Ising model on a random network with annealed or
quenched disorder. Phys. Rev. B, 90:024202, Jul 2014.
[12] J. Maris, H and P. Kadanoff, L. Teaching the
renormalization group. Am. J. Phys, 46(6), 1978.
[13] J. Mandziuk and B. Macukow. A neural network