GEO PROCESAMIENTO EN WEB

Introduccin
Escenario A
publicacion

Informacin Geogrfica

Internet

Download

Upload

Changing

Processing Desktop applications

publicacion

Cloud computing

Processing

Escenario B

Sources images: alkoholikuin.blogspot.com nro.nao.ac.jp

Evolucin Geopro
Geoprocessing

Web Processing Services (WPS) Improved WPS

(Brauner et al., 2009)

Parallelized WPS On Cloud Computing

Llevar los servicios donde se encuentra la informacin (moving code paradigm) Utilizar la tecnologia Grid Computing Incorporar Service Oriented Architecture (SOA) caracteristicas

Web Services Framework Of OGC Geoprocessing Standards

Web Processing Services (WPS)

Web Processing Services (WPS)

Interface estndar que permite enmascarar procesos, algoritmos y operaciones en la Web de una forma definida y estructurada para ser encontrados y usados por clientes y otros procesos
WPS

GetCapabilities
GetDescription Execute

Disponible servicios Informacin acerca del servicio Ejecuta

Current version 1.0.0

WPS Frameworks

Deegree
Extensions Grid Capabilities

52North PyWPS

GridGain UNICORE

Zoo

Cloud Computing

Definicin

Cloud computing is a technology that uses the Internet and

central remote servers to maintain data and applications

WikiInvest

Caractersticas de Cloud Computing

Amigable Vitalizacin Centrada en internet Adaptacin automtica Escalable Optimizable

Software as a Service (SaaS)

Google Docs

Platform as a Service (PaaS)

Google App Engine, Azure

Se paga por uso

Alta confiabilidad Versatilidad Crecimiento dinmico

Infrastructure as a Service (IaaS)

Amazon Web Services (AWS)

Barata

(Zhang et al., 2010a; Gong et al., 2010; Vaquero et al., 2009)

Proveedores de Cloud Computing

Amazon Web Service (AWS) Google App Engine Microsoft Azure Amazon Elastic Compute Cloud (EC2) GoGrid Elastic Block Store (EBS) Multiple Locations Rackspace
o
o o o

o
o o

o o

Elastic IP Address Auto Scaling Elastic Load Balancing Amazon Simple Storage Service (S3) Amazon CloudFront Amazon Relational Database Service

Aplicaciones Geospaciales en la Web

Publicaciones de Web Map Services (WMS) on Google App Engine (GAE)

Implementacin de indices utilizando Hadoop technology sobre 110millones de parcelas en una nube privada
Google maps utiliza cloud computing para soportar miles de usuarios WPS service en AWS.. =>

WPS en Grid

Geoestadistica
Estudio de la estadistica enfocada en analisis de datos espaciales o espaciotemporales
Wikipedia

El principio de los mtodos

oDependencia de los vecinos oProceso estacionario

Cmo es obtiene s depende del metodo

Kriging => Ordinary Kriging Universal Kriging

Radial Basis Function (RBF)

Inverse Distance Weight (IDW)

Methods
Preprocessing Getting s
k power of interpolation Inverse Distance Weight (IDW)
None

Calculating

Least Squares

Semivariogram
Ordinary and Universal Kriging

Fitting the empirical to Theoretical model. Linear Exponential Spherical ..,

Ordinary Kriging

Universal Kriging

and its variance s2

Least Squares Radial Basis Function (RBF)

Multiquadric:

1 r r 2 c 2

Inverse multiquadric: Thin plate spline: Multilog: Natural cubic spline: Spline with tension:

2 r

1 r c
2 2

(ri) is the radial basis function

z is the evaluation

of all points in the function used

Caso de estudio
Web Processing Services en Grid Interpolacin

Cross Validation
IDW

Number of neighborhoods from 5 to 12, and the power value change from 0.1 to 10 at steps of size 0.1. Number of neighborhoods from 5 to 12, number of lags (7-15), models: Linear, Spherical and Exponential Number of neighborhoods from 5 to 12, the smooth factor from 0.1 to 0.5 at steps of size 0.1 and sub models (Multiquadric, Inverse multiquadric, Thin plane spline, Multilog, Natural cubic spline)

Kriging (Ordinary and Universal)

RBF

Distribucin del proceso de interpolacin

Pixels to be interpolated by each node

Points to be sent to each node

Libreria para paralelizar procesos

Classes with parallelization features

Adicionar servicios en paralelo

Profile by default in the 52North Framework

Estructura de usada
Master Node

Tomcat configuration Master Node (GridGain Node)

Nodes

WPS services parallelized

Validacin cruzada

Interpolacin

Entradas:

Data: WFS with GML and SHP-ZIP format Field: Contain the attribute to do the interpolation Method: Ordinary Kriging, Universal Kriging, IDW, RBF RMS: Error of the best method in the cross validation StdRMS: Standardized Error Correlation coefficient Parameters: Parameters found. Cross Validation Graph: URL with cross validation graph Fitting graph: Show the error behavior according to the method selected Iteration: URL with a log file with the iterations summary.

Entrada:

Data: WFS with GML and SHPZIP format Field: Contain the attribute to do interpolation Method: This input receives a string with the method and parameters for executing the interpolation Resolution: Spatial Resolution
Result: WMS with reference to coverage on Geoserver Duration: process duration

Salida:

Salida:

Instalando a WPS en Amazon AWS (Linux)

Crear una cuenta enAmazon AWS

http://aws.amazon.com/account/

Instalar AWS service API Obtener credenciales variables (Linux) Crear una instancia

Consola de Amazon AWS

Amazon AWS tiene 2433 instancias publicas

Utilizando 10 instancias

Evaluacin

Informacin usada

Meteorological Agency of Spain (AEMET).

[1]

http://www.aemet.es/es/servidor-datos/acceso-datos/listadocontenidos/detalles/datos_observacion

Datos de elevacin

The DEM is published by OpenTopo, created in 2008 with LIDAR, resolution of 0.5 meters, Universal Traversal Mercator (UTM) region 12 North, datum WGS84

Evaluacin con 1000 y 10000 puntos

Software and hardware

Software Java SDK 1.6.0.20. Eclipse Galileo 3.5.2. GridGain 2.1.1 52North WPS Framework RC6 Tomcat 6 Geoserver 2.2 WPS clients 52North WPS OpenLayer client, and OpenJump 1.4.0. Operating System Ubuntu Linux 10.04 LTS Lucid Lynx ArcGIS 9.3 Hardware 1 laptop AMD Athlon X2 Dual-Core QL-64, memory 3.0 GiB, hardisk 250 GiB. 1 laptop Intel core 2 Duo, memory 3.0 GiB

Software y hardware AWS

Master Node Software

Nine Nodes Software

AMI Operating system fedora Tomcat 6.

AMI Operating system fedora Tomcat 6. Java SDK 1.6.0.20. GridGain 2.1.1

Java SDK 1.6.0.20.

GridGain 2.1.1 Geoserver 2.2 52North WPS Framework RC6 52North WPS OpenLayer client Micro Instance 613 MB of memory 32-bit platform High-CPU Medium Instance 1.7 GB of memory, 5 EC2 Compute Units (2 virtual cores with 2.5 EC2 Compute Units each), 350 GB of local instance storage, 32-bit platform

Hardware simulated (1 micro Instance )

Hardware simulated (1 micro Instance )

Micro Instance 613 MB of memory 32bit platform

Hardware simulated

Evaluacin de la libreria
Comparison between Geos. library and ArcGIS Method Kriging
Value Interpolated by WPS (Geostatistcal library

14 12 10

Method

y = 1.0075x
IDW

Geostatistical Library Standardized RMS RMS 1.607 -

ArcGIS (Geostatistical Extension) Standardized RMS RMS 1.607 -

Power=2
Ordinary Kriging Exponential(R:261138;S:5.4;N: 1.97) Universal Kriging Exponential(R:29138;S:4.4;N:1. 67)
0 5 10 15
Value Interpolated by WPS (Geostatistcal library)

8
6 4 2 0

1.587

0.881

1.605

0.829

1.593

0.873

1.623

0.820

RBF Model: Multiquadratic; Factor=0

1.648

1.648

Value Interpolated by WPS (Geostatistcal library

Value Interpolated by WPS (Geostatistcal library

Value Interpolated by WPS (Geostatistcal library

14 12 10 8

Comparison between Geos. library and ArcGIS Kriging Univeral Method

Comparison between Geos. library and ArcGIS RBF Multiquadratic Method

Comparison between Geos. library and ArcGIS IDW method

y = 0.9856x

14
12 10 8 6 4 2 0 0 5 10 Value Interpolated by Geostatistical extension ArcGIS) 15

14 12 10 8 6 4 2 0 0 5 10 15
Value Interpolated by Geostatistical extension ArcGIS

y = 0.9994x

y = 0.9994x

6
4 2 0 0 5 10 15 Value Interpolated by Geostatistical
extension ArcGIS

Comparasion interpolacin

Small differences

RBF

Kriging Ordinario

Evaluation in WPS OpenLayer client

Interpolation WPS service

CrossValidation WPS service

Utilizando intranet 4 nucleos (4 nodos, 1.000 points)

1.000.000 pixels Duration Interpolation Milliseconds 40000 35000 30000 25000 20000 160.000 pixels 40.000 pixels

15000
10000 5000 0
1
34686 26132 35908

2
Res: 2 28323 23232 28482 26000

3
18286 18487 21773 16980

4
17968 14955 18639 19056

1
7152 4464 7116 7389

2
Res: 5 5833 4676 5742 5590

3
4497 4289 4511 4290

4
4460 3748 4593 4609

1
3688 3414 3284 3760

2
4037 2401 2856 4298

3
4391 2602 2592 4391

4
2820 2088 2586 2920

Res: 10

IDW Kriging Spline

KrigingUniversal 34588

Resultados en Amazon AWS

Datos a evaluar

Relacin numero de nodos y tiempo de interpolacin

Entorno de evaluacin:

Evaluacion: 2011-02-22 T 9:00:00 Z - 2011-02-22 T 22:00:00 Z Numero de puntos: 10000 Resolucin: 2 meters (1000000 pixels), 5 m (160.000 pixels) and10 m (40.000 pixels). Numero de nodos(micro instances): 1-10 Repeticiones: 10 Solicitudes secuenciales 15 segundos

Resultados de la evaluacin

Maximas diferencias

Method

Resolution

Number of Pixels

Max Duration

Number of Nodes

Max Difference

Percentage Difference

Kriging Kriging

2 5

1000000 160000

265467 14295

10 7

-221610(a) -10628(a)

-83% -74%

Kriging
IDW IDW IDW

10
2 5 10

40000
1000000 160000 40000

3311
227569 11435 2848

8
10 10 9

-1136(a)
-187858(a) -8024(a) -713(a)

-34%
-83% -70% -25%

Tukeys Honest Significance Difference

Number of Pixels After this node (Not Max find signif. Difference Duration pvalue<0.05) Percentage Difference

Method

Resolution

ANOVA analysis
Nodes F-value p-value Kriging - 2 85.766 p<0.001 Kriging - 5 Kriging - 10 22.186 p<0.001 23.265 p<0.001 IDW - 2 99.45 p<0.001 IDW - 5 18.061 p<0.001 IDW - 10 56.449 p<0.001

Kriging
Kriging Kriging IDW IDW IDW

2
5 10 2 5 10

1000000
160000 40000 1000000 160000 40000

265467
14295 3311 227569 11435 2848

5
3 3 5 3 4

-190891(a)
-8382(a) -791(a) -160542(a) -5874(a) -528(a)

-72%
-59% -24% -71% -51% -19%