Beruflich Dokumente
Kultur Dokumente
MDS Issues
Data Management
Transaction Management
Database recovery
DBS DBS
Insurance companies Emergencies services (Police, medical, etc.) Traffic control Taxi dispatch E-commerce Etc.
Limited wireless bandwidth Wireless communication speed Limited energy source (battery power) Less secured Vulnerable to physical activities
Can physically move around without affecting data availability Can reach to the place data is stored Can process special types of data efficiently Not subjected to connection restrictions
Starting Page
1 8 9
...
depth level 1
depth level 2
To allow any time, anywhere access to the learning materials to support offline access of learning content Often the memory available on the device is not large enough to contain all material of a system a decision should be made what to put on the device To free the user from annoying procedures of pre-fetching content decide automatically what the user will need
9
4.
5.
Pre-fetch, starting from the beginning of the ordered list, putting on the device those objects with bigger priority, until available memory is filled in.
10
12
Caching is an important technique in the computing world. Most of us are familiar with cache used for a single processor computer, as illustrated in Figure
In distributed systems and network computing environments, cache management schemes need to handle two different scenarios.
13
In the first scenario, the data in the shared memory or the server can be read/written by different processors or clients concurrently, as shown in Figure 15.2
In the second scenario the data is read-only for the clients, as shown in Figure 15.3.
14
In these environments (distributed systems and network computing ), when a client accesses a cached data item, the server or another client could have just modified that data item. Hence, in distributed and network computing environments, the most crucial problem for caching is how to maintain data consistency among the clients and the servers?
15
To maintain cache consistency in distributed or network computing environment, different approaches have been developed. These approaches include polling-every-time, adaptive time-to-live (TTL) , invalidation based mechanisms.
16
For polling-every-time and TTL-based caching strategies, the client side initiates the consistency verification: that is, the client is responsible for verifying the data consistency before using it. For the TTL-based caching strategy, every cached data item is assigned a TTL value, which can be estimated based on the data items update history. For example, the adaptive TTL approach in estimates TTL based on the age of a data item. When the user request arrives for a data item x, if data item xs residence time has exceeded its TTL value, the client sends a message to the server to ask if x has changed. Based on the servers response, the client may get a new copy of x from the server (if the data item x has changed since the last time the client cached x) or just use the cached copy to answer the user's request (if the data item x has not been modified since the last time the client received a copy of x).
17
For the polling-every-time approach, every time the data is requested, the clients need to send requests to the server to verify if the cached data has changed. The polling-every-time caching strategy can be thought of as a special type of TTL-based scheme, with the TTL field equal to zero for every data item. Polling-everytime and TTL-based approaches are used in many existing Web caches.
18
On the other hand, for the invalidation-based strategies, the cache consistency verification is initiated by the server. Invalidation-based cache strategies are further classified into stateless and stateful approaches
19
In a stateless approach, the server does not maintain information about the cache contents of the clients (i.e., the server does not know what data is cached or how long it has been cached by a particular client). The stateless-based approach can be further categorized into synchronous and asynchronous approach. In the asynchronous approach, invalidation reports are sent out upon data modification. In the synchronous approach, the server periodically sends out invalidation reports, which may overlap with the previous invalidation.
EX: State less synchronous approaches are (namely TS (time stamps), AT (amnesic terminals) ,and SIG (signatures)
20
In the TS (time stamps) approach, the server periodically broadcasts the invalidation reports, which consist of data item IDs and their last update times during the last w (where w is the invalidation window size) seconds. Then, based on the invalidation reports, the MH will purge the data item from its cache or update the data items time stamp. The MH also maintains a variable to record the last time it received a report. If the difference between the current report time stamp and this variable is greater than w, the MH should drop the entire cache.
The AT (amnesic terminals) approach is similar to the TS approach, but the invalidation reports are the data item IDs that have changed since the last invalidation report. Similarly in the SIG (signatures) method, the difference is that the server periodically broadcasts the signatures of the data items.
21
In case of the stateful approach, the server keeps track of the information of the cache contents. These approaches can also be categorized into synchronous and asynchronous approaches. There are hardly any schemes in the stateful synchronous category. One example for stateful asynchronous approach is proposed is AS algorithm. In that approach, a Home Location Cache (HLC) is maintained for every client and is used to record the data items cached by that client and their last modification time. Based on the information, the server can generate invalidation reports dedicated to a particular client cache.
22
TS/AT Server is stateless (no information about client cache is maintained) Invalidation reports sent regardless of whether clients have any data in cache. Invalidation reports sent periodically at rate L (synchronous). Average cache latency is L/2 plus network queuing delay. Traffic on the network is bursty as queries are aggregated for a period of time L. Mobility is supported by assuming a replication of data across all stationary nodes (not scalable)
AS Server is stateful (HLC maintained) Invalidation report broadcast only if any client has valid data in cache. Invalidation reports sent as and when data changes (asynchronous)
24
25
26
Lack of hardware? No! We have what we need. Lack of applications? Nope - we have those too. Need a system capable of coping with the problems of mobility
need to rely on resources of remote servers, but may not be able to reach them!
28
Adaptation is Key
Highly dynamic environment adaptation key to good performance Who adapts?
System?
take advantage of good times Behave ok during bad times CODA
Client Adaptation
Make mobile clients more robust by offering adaptation rely on servers when possible function autonomously if needed monitor and adjust to current conditions Change application expectations
30
Adaptive Applications
resources are variable so applications adapt use of resources as resource quality changes
31
Application-Aware Adaptation
32
Application-Aware Adaptation
Application only (laissez faire) What if different applications compete for the resources? OS only (application-transparent) Does not differentiate between applications (student viewing a video of a lecture vs. a video teleconference) Joint responsibility in Odyssey (application-aware) Several ways to divide the functionality odyssey only one
33
Odyssey
A Platform for adaptive mobile data access
Built a prototype for Un*x as OS extension Provides a small API to the application
Implementation: Need a central component for resource monitoring and management (Viceroy) Need data aware components that offer fidelity choices (Wardens)
34
Odyssey Architecture
Application
Odyssey runtime
Upcalls
Sys calls
Odyssey calls
kernel
Interceptor
Tsop, request
35
Operational Model
Control loop: Select fidelity (application) Place request (system call)
Detect change
Notify application
36
Agility
An Odyssey client must estimate the quality of network paths used by various applications. Odyssey records:
Round-trip time Throughput
Odyssey updates its estimates of latency and bandwidth once every half second. Aside, Nobles group followed up with agility estimation work for ad hoc networks
37
Agility (cont.)
38
Agility (cont.)
39
Stability
Pursuing agility while completely sacrificing stability can be counterproductive. Rapidly switching Low-fidelity Variable latency Stability is properly incorporated by individual application. When notifying an application , the viceroy can include information about the expected 40
41
Context-aware Computing
Beyond application-aware adaptation Instead of adapting only to resource levels, adapt to contexts Context:
Enumeration-based (categories) Role-based (roles of context in building mobile applications)
42
Types of Context
Computing context includes network connectivity, communication costs, communication bandwidth, and local resources, such as printers, displays, and workstations User context includes user profiles, location, and people in the vicinity of the user Physical context includes lighting and noise levels, traffic conditions, and temperature Temporal context includes time of day, week, month, and season of the year Context history is the recording of computing, user, and physical context over time
43
The 5 Ws
Who is the user? Who are the people with which the user is interacting, or who is nearby? What is the user doing? Where is the user? Home? Work? Bathroom? Familiar coffee shop? When? What time is it? Why? Why is the user performing a certain task? What is the tasks priority in the grand scheme? Low-level vs. High-level details
44
Context Overview
45
Context-aware Requirements
Contextual sensing
detection of environmental states
Contextual adaptation
capability of the system to adapt its behavior by using contextual information
Contextual augmentation
capability to associate contextual information with some digital data Example: association of a particular meeting place and attendees with a set of minutes Example: association of a digital photo with a specific location
46
Designing CA Applications
Build list of relevant contexts
e.g., home, office, traveling, sleeping,
History
Recording user actions and previous contexts
49
Example: Location
Indoor locating systems
e.g., infrared or ultrasound
GPS Associations with nearby computers Motion sensors and cameras, computer vision Ask the user!
50
Service-oriented Architecture
Provide services to context-aware applications Context subscription and delivery service
Delivers contextual information as available
Service discovery
Discovery of nearby services Well examine service discovery protocols next!
51
52
consistent state. Normally a flat transaction follows the basic properties named as ACID (Atomicity, Consistence, Isolation, Durability) properties. In the case of Mobile Transactions, to cope with the problems of mobile environment like handoffs, disconnections etc, ACID properties are generally relaxed. A basic way to relax the ACID property is to divide the transaction into various sub transactions which can be further nested.
53
Execution scenario: User issues transactions from his/her MU and the final results comes back to the same MU. The user transaction may not be completely executed at the MU so it is fragmented and distributed among database servers for execution. This creates a Distributed mobile execution.
54
Definition: A mobile transaction is a transaction of nondeterministic lifetime submitted from a mobile capable node in a mobile environment. The mobile transaction, in general, can be considered to consist of two components a MU component and a fixed network component. The fixed network component of the mobile transaction may have to be partially or completely relocated as the MU moves.
Mobile environments can be considered to be similar to highly distributed environments in many respects. But unlike in distributed environments, locations of some hosts are not permanent in mobile environments. This along with the low communication bandwidth, frequent disconnections, and high vulnerability throws up many challenges to researchers
55
Ti is a triple <F, L, FLM>; where F = {e1, e2, , en} is a set of execution fragments, L = {l1, l2, , ln} is a set of locations, and
FLM = {flm1, flm2, , flmn} is a set of fragment location mapping where j, flmi (ei) = li
56
Applicable Transaction Models I. Open Nested Transactions II. Split transactions III. Saga compensating transactions I. Open Nested Transactions Open Nested Transaction Model is designed for long duration activities. These transactions typically consist of a set of sub transactions, which can be structured as a transaction tree. The Open Nested Transaction Model provides better support for long duration activities. These are also ideally suited for conversational transactions. Ex: a. Reporting and Co-Transactions approach b. The Clustering Model etc..
57
a). Reporting and Co-Transactions: The parent transaction (workflow) is represented in terms of reporting and cotransactions which can execute anywhere. A reporting transaction can share its partial results with the parent transaction anytime and can commit independently. A co-transaction is a special class of reporting transaction, which can be forced to wait by other transaction.
b). Clustering: A mobile transaction is decomposed into a set of weak and strict transactions. The decomposition is done based on the consistency requirement. The read and write operations are also classified as weak and strict.
58
Split transactions or dynamically restructured transactions split a transaction into two independent serializable transactions. The transactions could be committed or aborted independent of each other. The operation split-transaction can be used to split a transaction into two. The split transactions are combined together by an inverse operation termed Join Transaction. Split transactions are mainly designed for user controlled open-ended transactions. Ex: Kangaroo Transaction model
59
ex: Kangaroo Transaction: It is requested at a MU but processed at DBMS on the fixed network. The management of the transaction moves with MU. Each transaction is divided into subtransactions. Two types of processing modes are allowed, one ensuring overall atomicity by requiring compensating transactions at the subtransaction level.
61
III. Sagas Sagas are defined as a set of relatively independent transactions. The transactions in a saga are termed as component transactions. Each component transaction has a dual termed compensating transaction. A predefined order can be defined for the execution of the saga. The transactions belonging to different sagas can be interleaved in any fashion.
62
63
65
66
67
Continuous Queries
Answer Query Query Data
68 Data
69
Query Translation Steps in LDQ Processing 1. Determine the validity (and type) of the query and request the location service to provide a location to which the LDQ is to be bound. 2. Find the appropriate level of the location granularity, which will be used by the target content provider, and convert the query into the correct format. 3. If needed, decompose the query for different servers and send each to the target server for processing.
4. Receive the query results and perform any needed filtering to reduce the result set size.
5. Combine results from multiple servers and put into a format 72 desired by user.
73
Agenda
Introducing fundamentals of database recovery and conventional recovery protocols Identifying those aspects of a mobile database system which affect recovery process Discussing recovery approaches which have appeared in the literature
74
Agenda (Cont.)
Similar to other areas such as transaction modeling, concurrency control, etc., database recovery is also in the development stage, so the coverage here is mostly limited to state-of-the art research and little on commercial products
75
Introduction
Database recovery protocols recover a database from transaction or system failures
They store the database to a consistent state from where transaction processing resumes
Introduction (Cont.)
In a concurrent execution environment when a failure occurs then a transaction may be Active Blocked Being rolled back In the middle of a commit The task of a recovery protocol is to identify the right operation for each transaction 1. Roll forward or Redo 2. Roll backward or Undo
77
Introduction (Cont.)
To implement these operations, Transaction log is required This log file is generated and maintained by the system The log contains Committed values of data items (Before Image - BFIM) Modified values of data items (After Image - AFIM) The log is a crucial document for recovery It is generated and maintained by a protocol called Write Ahead Logging WAL The protocol guarantees that the content of a log is reliable and can be used for Undo and Redo operations
78
Introduction (Cont.)
After a failure the database system reboots and, by using log, applies Redo and Undo operations on transactions which where in the system when it failed
A Redo completes the commit operation for a transaction, and an Undo rolls back a transaction to maintain atomicity
79
Recovery is a time-consuming and resource-intensive operation. The most expensive operation is managing the log This operation is essential for recovery A Mobile Database System (MDS) is a distributed system based on client server paradigm, but it functions differently than conventional centralized or distributed systems.
82
In any database management system, distributed or centralized, the database is recovered in a similar manner and the recovery module is as an integral part of the database system. Database recovery protocols, are not tampered with user level applications. A system which executes applications, in addition to database recovery protocol, requires efficient schemes for Application recovery.
83
This gets more complex in MDS because of Unique processing demands of mobile units The existence of random handoffs the presence of operations in connected, disconnected, and intermittent connected modes Location-dependent logging The presence of different types of failure. There are types of failures Hard failure (Can not be easily repaired) Soft failure (Recoverable)
85
An application can be in any execution state blocked, executing, receiving data slowly, and so on The application may be under execution on stationary units (base station or database server) or on mobile units or on both These processing units(especially the mobile unit), may be Going through a handoff Disconnected In a doze mode Turned off completely
86
The application may be processing a mobilaction or reading some data or committing a fragment, and so on
If a failure occurs during any of these tasks, the recovery system must bring the application execution back to the point of resumption 87
In application recovery, unlike data consistency, the question of application consistency does not arise because the application cannot execute correctly in the presence of any error The most important task for facilitating application recovery is the management of log What is needed is an efficient logging scheme
88
89
The mobile units and the servers must build a log of the events that change the execution states of mobilaction
91
92
The reliability and availability of mobile units, make it a less desirable place to save the log 94 MSC and BS are suitable places
96
Logging schemes
Centralized logging-Saving of log at a designated site A base station is designated as logging site where all mobile units from all data regions save their log Since the logging location is fixed and known in advance, and the entire log is stored at one place, its management (access, deletion, etc.) becomes easier This scheme works, but it has the following
99
102
b) frequency-based scheme
log unification is performed when the number of handoffs suffered by the MU increases above a predefined value
104
106
109
110
115
117
121
123
126
The VLR first changes the status of the mobile unit in its database from normal to failed The VLR then issues a message containing its own identity The adjacent VLRs store these messages until explicit denotify messages are received 127
Case 1-The mobile unit reboots in the same base station where it crashed :
In this scenario, the HoAg informs the VLR that the mobile unit has recovered The VLR then issues a denotify message to all the adjacent VLRs indicating that the forward notification information is no longer valid The status of the mobile unit is changed back to normal from failed
128
Case 2-The mobile unit reboots in a different base station but in the same VLR: First the mobile unit registers with the base station and the registration message is logged on to the corresponding VLR This VLR identifies the status of the mobile unit as failed, and then it proceeds as in case 1 and sends denotify messages to the adjacent VLRs The status of the mobile unit is changed back to normal from failed The new base station then proceeds to perform log 129 unification from the previous base station
Case 3-The mobile unit reboots in a different base station and a different VLR: The mobile unit requests for registration The corresponding VLR identifies the mobile unit as a forward notified mobile unit and returns the identity of the previous base station and the identity of the VLR to the HoAg of the mobile unit in the recovered base station The base station then proceeds to perform log unification from the previous base station Simultaneously, the new VLR sends a recovered message to the previous VLR regarding the recovered status of the mobile unit and also sends a registration message to the HLR regarding the registration of the mobile unit in the new location
130
The status of the mobile unit is changed to normal from forwarded in the new VLR Upon receiving the recovered message, the previous VLR sends a denotify message to all adjacent VLRs except the one in which the mobile unit recovered and removes the registration of the mobile unit from itself as well In the situation where the mobile unit recovers in a nonadjacent VLR that has not received the forward notifications, the new base station has to get the previous base station information from the HLR and then send the previous VLR a recovered message Upon receiving this message, the previous VLR acts similar to the previous VLR of case 2 The forward notification scheme is unsuitable if the mobile unit suffers failures with a very small EFT
131
SUMMARY
the entire process of chekpointing and logging for recovery are comparatively complex than conventional database recovery There are still quite a few research problems need innovative solutions Until they are resolved the mobile database system will continue to remain in research domain
132
133