1.

INTRODUCTION

1.1ABOUT PROJECT: A data warehouse is the main repository of an organizations historical data, its corporate memory. It contains the raw material for managements decision support system. The critical factor leading to the use of a data warehouse is that a data analyst can perform complex queries and analysis, such as data mining, on the information without slowing down the operational systems.

ETL is important, as it is the way data actually gets loaded into the warehouse. This article assumes that data is always loaded into a data warehouse, whereas the term ETL can in fact refer to a process that loads any database. ETL can also be used for the integration with legacy systems. Usually ETL implementations stores an audit trail opn positive and negative process runs. In almost all designs, this audit trail is not at the level of granularity which would allow to reproduce the ETLs results if the raw data were not available.

Extract:

The first part of an ETL process is to extract the data from the source systems. Most data warehousing projects consolidate data from different source systems. Each separate systems may also use a different data organization / format. Common data source formats are relational database and flat files, but may include non-relational database structures such as IMS or other data structures such as VSAM or ISAM.

Transform:

The transform stage applies a series of rules or functions to the extracted data from the source to derive the data to be loaded to the end target. Some data sources will require very little or even no manipulation of data. In other cases, one or more of the following transformations

types to meet the business and technical needs of the end target may be required: Selecting only certain columns to load, translated coded values, this is called automated data cleansing .

Load: The load phase loads the data into the target usually being the data warehouse. Depending on the requirements of the organization, this process ranges widely. Some data warehouses might weakly overwrite existing information with cumulative, updated data, while other DW might add new data in a histories form example hourly. The timing and scope to replace or append are strategic design strategies dependent on the time available and the business needs. ETL is a key process to bring a heteroecism and asynchronous source attracts to a homogeneous environment. The scalability of an ETL system across the life time of its usage needs to be established during analysis. This includes understanding the volumes of data that will have to be processed with in Service Level Agreements. The time available to extract from source system may change which may mean the same amount of data.

2. SYSTEM ANALYSIS

2.1 Existing System:

In the existing system there are some of the disadvantages as described below Data is scattered over the different databases. Many versions, subtle differences. Available data poorly documented. Results are unexpected. Data needs to be transformed from one form to other. Analyzing of the business is not possible as historical data is not present. Transaction is very slow as data is continuously loading to centralized database referred as OLTP.

y y y y y y y

2.2 Proposed System:

There are many benefits of the proposed system, Improve business with informative data analysis. The system helps Business Analyst to do their jobs more effectively and efficiently. With a single, enterprise wide view of real time information it allow decision makers to access and analyze the correlations between:

y y y y y y y y y

Targeted and direct marketing. Customer retention; Management accounting. Market and credit risk. Fraud detection. Historical business trends. Product gaps and opportunities. Activity and performance. Market segmentation.

y y

Competitor products. Actual pricing.

Based on the information collected from data anyalysis, BA can be easily analyzed business, for example y y y y y y y Focused marketing campaigns. Product customization. Product packing. Performance tracking Promotional pricing Competitor alliances. Estimation of wallet share.

Problem Domain:
ETL process was done manually by using SQL code created by programmers. This was tedious and cumbersome in many cases since it involved manyu resources, complex coding and more work hours. On top of it, maintaining the code placed a great challenge among the programmers.

Description:
The total project can be best explained with an example about how it is solved and how it fits the business needs

Sample ETL Process Flow:

3. FEASIBILITY STUDY
The feasibility study/Analysis proposes one or more conceptual solution to the problem set for project. The conceptual solution gives an idea of what the new system will look like. They define what will be done on the computer and what will remain manual. They also indicate what input will be needed by the systems and what outputs will be produced. The solution must be proven feasible and a preferred solution must be accepted. Three key considerations involved in the feasibility analysis are

I. II. III.

TECHNICAL FEASIBILITY OPERATIONAL FEASIBILITY ECONOMICAL FEASIBILITY

3.1TECHI NCAL FE ASI BI LITY: This evaluation determines whether the technology needed for the proposed system is available and how this technology can be integrated with in the organization.

3.2OPE RATIONAL FE ASI BI LTY: Operational feasibility covers two aspects, Technical performance and acceptance with the organization. Technical performance includes issues such as determining whether the system can provide the right information for the organizations personnel, and whether the system can be organized so that it always delivers the information at the right place and in time. Acceptance revolves around the current system and its applications. MS.NET is efficient and understandable with simplicity. The technical performance of the proposed system is good as it is built on technology of Webservices which is platform independent and can be used on any existing system. In proposed system we are also implementing high security levels so that the data flow will be authenticated. The system works if windows operating system, ms.net is installed. These are easily available and economical. Hence the system is operational feasible

3.3ECONOMI CAL FE ASI BI LITY:

Economic analysis is the most frequently used technique for evaluating the effectiveness of a proposed system. More commonly known as Cost-Benefit analysis, this procedure is to determine the benefits and savings that are accepted from a proposed system and compare them with existing system

4. REQUIREMENT SPECIFICATION
The software requirement specification is a document that completely describes what the proposed software should do without describing how the software will do it. A software requirement specification provides a reference for validation of the final product. It establishes the basic agreement between the client and the supplier.

2.3 Requirements
Earlier the file transfer along the network would not be more secure. The unauthorized users can also have the information regarding the files during the file transfer. And the information in the file may be malpractice or forgered. Hence in order to attain the security issues of the information, these files should be authenticated i.e. , even the non repudiation service should also be provided along with these security issues.

2.3.1 Functional Requirements

Extract, Transform, and Load (ETL) is a process in data warehousing that involves
y y y

extracting data from outside sources, transforming it to fit business needs (which can include quality levels), and ultimately loading it into the end target, i.e. the data warehouse.

Extract The first part of an ETL process is to extract the data from the source systems. Most data warehousing projects consolidate data from different source systems. Each separate system may also use a different data organization / format. Extraction converts the data into a format for transformation processing.. Transform The transform stage applies a series of rules or functions to the extracted data from the source to derive the data to be loaded to the end target. Some data sources will require very little

or even no manipulation of data. The transformations types to meet the business and technical needs of the end target may be required:Load The load phase loads the data into the end target, usually being the data warehouse (DW). Depending on the requirements of the organization, this process ranges widely. Some data warehouses might weekly overwrite existing information..

Input Requirements
Input: Primarily it requires the connection strings of all the OLTP to get connected for transferring data from OLTP to OLAP. It then requires the total details of the tables i.e. entities and attributed for mapping process present in various databases. . Output: Data is present in different data bases in a different manner. For ex: Time stamps are stores in different ways in different databases like mm/dd/yy or dd/mm/yy or dd/mon/yy format. So manipulation of this data takes place in transformation phase and details are loaded into the data warehouse according to the user needs.

2.3.2 Non-Functional Requirements

Nonfunctional requirements describe user visible aspects of the system that are not directly related to the functional behavior of the system. Nonfunctional requirements span a

number of issues from user interface look and feel to response time requirements to security issues.

Usability:
ETL Process using is ease, it needs the mapping for the fields which are to be converted. After this just with a click of button large amount of records are modified and loaded into data warehouse according to the user needs. Performance Consideration: Tool can load transactions with in limited time tin OLAP. Multiple dimension reports can be generated according to the needs that fit Business Intelligence effectively.

Supportability:
This process supports all types of databases and can be used by multiple departments simultaneously. It fits in all the business requirements.

Reliability:
In terms of reliability i.e. Mean Time Between Failures (MTBF) is very high because no third party can access the process and also it works fro any type of database. So regard with time improvement of data bases takes place even then it works for transferring data.

SYSTEM REQUIREMENT SPECIFICATIONS: Hardware Requirements: SYSTEM HARD DISK MONITOR MOUSE RAM KEYBOARD Software Requirements: OPERATING SYSTEM : windows xp professional FRONT END : .net BACK END : ms-access DATABASES : Oracle10g, sqlserver : Pentium IV 2.79GHz : 320 GB : 21 SAMSUNG COLOR : iball : 2GB : Multimedia keyboard

1.3 ABOUT ENVIRONMENT: ABOUT Microsoft Visual Studio.NET 2005: Programming a computer is a lot like teaching a child to tie their shoes. Until you find the correct way of giving the instructions, not much gets accomplished. Visual Basic .NET is a language in which you can tell your computer how to do things. But like a child, the computer

will only understand if you explain things very clearly. If you have never programmed before, this sounds like an arduous task, and sometimes it is. However, Visual Basic .NET gives you a simple language to explain some complex things. Although it never hurts to have an understanding of what is happening at the lowest levels, Visual Basic .NET frees the programmer from having to deal with the mundane complexities of writing Windows programs. You are free to concentrate on solving problems. Windows versus DOS Programming A Windows program is quite different from its ancient relative, the MS-DOS program. A DOS program follows a relatively strict path from beginning to end. Although this does not necessarily limit the functionality of the program, it does limit the road the user has to take to get to it. A DOS program is like walking down a hallway; to get to the end you have to walk down the hallway, passing any obstacles that you may encounter. A DOS program would only let you open certain doors along your stroll. Windows on the other hand, opened up the world of event-driven programming. Events in this context include, for example, clicking on a button, resizing a window, or changing an entry in a text box. The code that you write responds to these events. To go back to the hallway analogy: in a Windows program to get to the end of the hall, you just click on the end of the hall. The hallway can be ignored. If you get to the end and realize that is not where you wanted to be, you can just set off for the new destination without returning to your starting point. The program reacts to your movements and takes the necessary actions to complete your desired tasks (Visual Basic .NET). Another big advantage in a Windows program is the abstraction of the hardware; which means that Windows takes care of communicating with the hardware for you. You do not need to know the inner workings of every laser printer on the market, just to create output. You do not need to study the schematics for graphics cards to write your game. Windows wraps up this functionality by providing generic routines that communicate with the drivers written by hardware manufacturers. This is probably the main reason that

Windows has been so successful. The generic routines are referred to as the Windows Application Programming Interface (API). Before Visual Basic 1.0 was introduced to the world in 1991, developers had to be well versed in C++ programming, as well as the rudimentary building blocks (Windows API) of the Windows system itself. This complexity meant that only the dedicated and properly trained individuals were capable of turning out software that could run on Windows. Visual Basic changed all of that, and it has been estimated that there are now as many lines of production code written in Visual Basic as in any other language. Visual Basic changed the face of Windows programming by removing the complex burden of writing code for the user interface (UI). By allowing programmers to draw their own UI, it freed them to concentrate on the business problems they were trying to solve. Once the UI is drawn, the programmer can then add the code to react to events. Visual Basic has also been extensible from the very beginning. Third-party vendors quickly saw the market for reusable modules to aid developers. These modules, or controls, were originally referred to as VBXs (named after their file extension). If you did not like the way a button behaved you could either buy or create your own. However, these controls had to be written in C or C++. Database access utilities were some of the first controls available. Version 5 of Visual Basic introduced the concept of ActiveX that allowed developers to create their own ActiveX controls. When Microsoft introduced Visual Basic 3.0, the programming world changed again. Now you could build database applications directly accessible to users (so-called front-end applications) completely with Visual Basic. There was no need to rely on third-party controls. Microsoft accomplished this task with the introduction of the Data Access Objects (DAO), which allowed programmers to manipulate data with the same ease as manipulating the user interface. Versions 4.0 and 5.0 extended the capabilities of version 3.0 in order to allow developers to target the new Windows 95 platform. Crucially they also made it easier for developers to write code, which could then be manipulated to make it usable to other language developers. Version 6.0 provided a new way to access databases with the integration of ActiveX Data Objects

(ADO). ADO was developed by Microsoft to aid Web developers using Active Server Pages to access databases. With all of the improvements to Visual Basic over the years, it ensured its dominant place in the programming world. It helps developers write robust and maintainable applications in record time. With the release of Visual Basic .NET in February 2002, many of the restrictions that used to exist have been obliterated. In the past, Visual Basic has been criticized and maligned as a "toy" language, as it did not provide all of the features of more sophisticated languages such as C++ and Java. Now, Microsoft has removed these restrictions and made Visual Basic .NET a very powerful development tool. This trend continues with Visual Basic .NET 2003. Although not as drastic a change as from Visual Basic 6 to Visual Basic .NET, there are enough improvements in the language (including support for the .NET Framework 1.1) that Visual Basic .NET 2003 is a welcome upgrade and is a great choice for programmers of all levels. Writing Software for Windows To understand how .NET works, look at how developers used to write software for Windows. The general principle is the same, only they had to do things in different ways to work with different technologies (COM, WIN32 API). Any software that you write has to interact with various parts of the operating system in order to do its job. If the software needs a block of memory to store some data in, it interacts with the memory manager. To read a file from disk, you use the disk subsystem. To request a file from the network, you use the network subsystem. To draw a window on the screen, you use the graphics subsystem, and so on. Where the system breaks down, as far as .NET is concerned, are that there is no commonality between the ways you use the subsystems on different platforms, despite the fact that platforms tend to have things in common. For example, even if you are writing an application for Linux, you may still need to use the network, disk, and screen subsystems. However, because different organizations developed these platforms, the way you open a file

using the Linux platform may be different from the way you do it on Windows. If you want to move code dependent on one platform to another, you will probably have to rewrite portions of the code. You will also have to test the code to ensure it still works as intended. Windows software communicates with the operating system and various subsystems using something called the Windows 32-bit Application Programming Interface, or Win32 API. Although object orientation was around at the time, this API was designed to be an evolution of the original Windows API, which predates the massive adoption of object-oriented techniques It is not easy to port API to other platforms, which is why (despite the fact that Linux has been around for 10 years) there is no version of the Win32 API for Linux. There is a cut-down version of the Win32 API for the Mac, but this has never received much of an industry following. The Win32 API provides all basic stuff, but now and again, Microsoft extends the functionality of Windows with a new API. A classic example is the Windows Internet API, also known as the WinInit API. This API allows an application to download resources from a Web server, upload files to an FTP server, discover proxy settings, and so on. Again, it is not object oriented, but it does work. A large factor in the success of early versions of Visual Basic is that it took the tricky-tounderstand Win32 API calls and packaged them in a way that could be easily understood. Using the native Win32 API, it takes about a hundred lines of code to draw a window on the screen. The same effect can be achieved in Visual Basic with a few gestures of the mouse. Visual Basic represents an abstraction layer on top of the Win32 API that makes it easier for developers to use. A long-time frustration for C++ developers was that a lot of the things that were very easy to do in Visual Basic remained not so much hard as laborious in C++. Conversely, developers like C++ because it gives them an amazing amount of control over how a program works, but at the cost that their programs take longer to write. Microsoft introduced the Microsoft Foundation Classes (MFC) because of this overhead, which, along with the IDE of Visual Studio, brought the ease of Visual C++ development a little towards that of Visual Basic.

The .NET Framework Classes

Unlike the Win32 API, .NET is totally object oriented. Anything you want to do in .NET, you are going to be doing with an object. If you want to open a file, you create an object that knows how to do this. If you want to draw a window on the screen, you create an object that knows how to do this. You will discover that this is called encapsulation; the functionality is encapsulated in an object and you don't really care how it's done behind the scenes. Although there is still the concept of subsystems in .NET, these subsystems are never accessed directlyinstead they are abstracted away by the Framework classes. Either way, your .NET application will never talk directly to the subsystem (although you can do this if you really need or want to). Rather, you talk to objects that then talk to the subsystem. In Figure-2-1, the box marked System.IO.Fil is a class defined in the. NET Framework

Your Application

System.IO.File

Windows File System

POCKET PC System

Linux File System

If you are talking to objects that talk to subsystems, do you really care what the subsystem looks like? Thankfully the answer is "no," and this is how Microsoft removes your reliance on Windows. If you know the name of a file, you use the same objects to open it whether you are running on Windows XP, a Pocket PC or, once the required Framework is

released, Linux. Likewise, if you need to display a window on the screen, you do not care if it is on a Windows operating system or on a Mac. The .NET Framework is actually a set of classes, called base classes. Although it is rather extensive, there are, at the time of writing, things not supported by .NET. This is because Microsoft needs to port everything from the old Win32 model to the new .NET model and some things are deemed more important than others. Microsoft has developed the first release of .NET so that the vast majority of the functionality is in place. Developers moving from the old Win32 approach have to retrain great swathes of their knowledge to move to .NET as certain things are vastly different. Luckily, for you, as a reader of this book, you are likely to be a newcomer and so you will not have to go through the relearning process. You only have to learn it, without forgetting what you used to know! The class library itself is vast. There are several thousand objects available to developers, although in your day-to-day development you will only need to understand a handful of these to create some powerful applications. The other wrinkle to this is that the classes are the same irrespective of the language used. So, if you are writing a Visual Basic .NET application, you use the same object as you would from within a C# application. That object will have the same methods, properties, and events, meaning that there is very little difference in capabilities between the two languages, since they both rely on the framework. Programming languages are somewhere in-between the languages that you and I speak every day and the language that the computer itself understands. The language that a computer uses is the machine code .

5. SYSTEM DESIGN
System design is the transformation of the analysis model into a system design model. During system design, developers define the design goals of the project and decompose the system into smaller subsystems that can be realized by individual teams.

Identifying Design Goals: In this step of system design, the qualities of our system that should focus on are identified. Many of the goals are interred from non-functional requirements or from the application domain. COSTS: Costs are not mentioned, so we cant say anything about cost criteria. USABILITY: User should be able to use it easily with out much strain. READABILITY: Code should be easily understandable. EXTENSIBILITY: It should be possible to add new functionality to this system. ROBUSTNESS: Users are not having good computer knowledge. So they may misuse it unknowingly. So, it should withstand in all such cases. RELIABILITY: It should match the requirement of users. Otherwise they cant cope up with the new system. AVAILABILITY: The system should be available all the time, whenever user wants to work with it. FAULT TOLERANCE: It should work even if there are some errors. It should enable user to correct those errors. Class Design Type, Signature and Visibility During analysis, we identify attributes and operations without specifying their types or their parameters. During object design, we refine the analysis and system design models by adding type and visibility information. The type of an attribute specifies the range of values the attribute can take and the operations that can be applied to the attribute.

Operation parameters and return values are typed in the same way as attributes are. The type constraints the range of values the parameter or the return value can take. The type of the return value is called the signature of the operation. The visibility of an attribute or an operation specifies whether it can be used by other classes or not. UML defines three levels of visibility.

Private: A private attribute can be accessed only by the class in which it is defined. Similarly, a private operation can be invoked only by the class in which it is defined.

y y

Public: A public attribute or operation can be accessed by any class. Protected: A protected attribute or operation can be accesses by the class in which it is defined and any descendent of the class

USE CASE DIAGRAM Actors: Actors represent external entities that interact with the system. An actor can be human or an external system. During this activity, developers identify the Actors involved in this system are:

Administrator Administrator is an actor who starts the service and who control the unnecessary deletions in the file system.

User User is an actor who uses the files and who may deletes the files in the client systems.

Use Cases:

Use cases are used during requirement elicitation and analysis to represent the functionality of the system. Use cases focus on the behavior of the system from an external point of view. A use case describes a function provided by the system that yields a visible result for an actor. An actor describes any entity that interacts with the system. The identification of actors and use cases results in the definition of the boundary of the system, which is, in differentiating the tasks accomplished by the system and the tasks accomplished by its environment. The actors are outside the boundary of the system, where as the use cases are inside the boundary of the system.

Actors are external entities that interact with the system. Use cases describe the behavior of the system as seen from an actors point of view. Actors initiate a use case to access the system functionality. The use case then initiates other use cases and gathers more information from the actors. When actors and use cases exchange information, they are said to be communicate.

For Source System

For Destination System

Scenarios

A Use Case is an abstraction that describes all possible Scenarios involving the described functionality. A scenario is an instance of a use case describing a concrete set of actions. Scenarios are used as examples for illustrating common cases. Their focus is on understandability. Use cases are used as examples for illustrating common cases. Their focus is on completeness. We describe a scenario using a template with three

 The name of the scenario enables us to refer to it ambiguously. The name of a scenario is underlined to indicate it is an instance.  The participating actor instances field indicates which actor instances are involved in this scenario. Actor instances also have underlined names  The flow of events of a scenario describes the sequence of events step by step.

Scenario #1
Scenario Name Participating Actors Flow of Events : Extracting data : User : (1) User enters details (2) User enters the name and password Required for login to the source.

Scenario #2

Scenario Name Participating Actors Flow of Events

: Transform : User :(1) User selects a file to the list (2) Transform the Selected file using content

Scenario #3

Scenario Name Participating Actors Flow of Events

: Load : User : The user specifies the target and the file is loaded into the target.

Class Diagram
Class diagrams describe the structure of the system in terms of classes and objects. Classes are abstraction that specifies the attributes and behavior of a set of objects. A class is a collection of objects that share a set of attributes that distinguish the objects as members of the collection.

BEHAVIORAL MODEL
STATE CHART DIAGRAM

At any given time, an object is in a particular state. It shows the states of a single object in a system which change their states in response to events and to time, state diagram capture these state changes. It is also referred to as a State Machine. The icon for a state is a rounded rectangle and the symbol solid circle stands for the starting point of a sequence of states and a bulls eye represents the end point.

SEQUENCE DIAGRAM

Interaction diagrams describe patterns of communication among a set of interacting objects, One of its two forms, the Sequence diagrams represents the objects participating in the interaction horizontally and time vertically

For Source System

For Destination

ACTIVITY DIAGRAM: The UML activity diagram is much like flow charts of old. It shows steps as well as decision points and branches. Its useful for showing what happens in a business process or an operation.

COMPONENT DIAGRAM Component models the software component of a system. A component is accessible through its inheritance a collection of operations. The relationship between a component and its inerface is called realization. One component can access the services of other using an important interface. The component icon is a rectangle with two small rectangles over laid on item left side.

COLLABORATION DIAGRAM:

Collaboration diagram depicts the same information as sequence diagrams. Collaboration diagrams represents the sequence of messages by numbering the interactions. This removes the need for geometrical constraints on objects and results in a more compact diagram.

For Source

For Destination

6. SAMPLE CODE
Samplecode:ta
Imports System.Data.OleDb Imports System.IO Public Class frmconnect Dim con As New OleDbConnection Dim cmd As New OleDbCommand Dim ds As New DataSet Dim da As New OleDbDataAdapter Dim cb As New OleDbCommandBuilder(da) Dim srccon As New OleDbConnection Dim srccmd As New OleDbCommand Private Sub btnselect_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnselect.Click Dim ofdlg As New OpenFileDialog If srctype = "Text File" Then ofdlg.Filter = "Text Files|*.txt" ElseIf srctype = "XML File" Then ofdlg.Filter = "XML Files|*.xml" ElseIf srctype = "Excel" Then ofdlg.Filter = "Excel Files|*.xls" ElseIf srctype = "Oracle" Then ofdlg.Filter = "Oracle Export Files|*.dmp" ElseIf srctype = "Ms Access" Then ofdlg.Filter = "Access database Files|*.mdb" End If Try ofdlg.ShowDialog() txtfilename.Text = ofdlg.FileName If srctype = "Ms Access" Then lstsrctbl.Visible = True srccon = New OleDbConnection("provider=microsoft.jet.oledb.4.0;data source=" & txtfilename.Text) Dim tbl As DataTable If srccon.State = ConnectionState.Closed Then srccon.Open() tbl = srccon.GetOleDbSchemaTable(OleDb.OleDbSchemaGuid.Tables, New Object() {Nothing, Nothing, Nothing, "TABLE"}) lstsrctbl.DataSource = tbl lstsrctbl.DisplayMember = tbl.Columns(2).ColumnName End If Catch ex As Exception MessageBox.Show(ex.Message) End Try End Sub Private Sub btndestverify_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btndestverify.Click

If btndestverify.Text = "Verify Source" Then If srctype = "Oracle" And desttype = "Ms Access" Then con = New OleDbConnection() con.ConnectionString = "provider=msdaora;user id=" + txtduname.Text + ";password=" + txtdpassword.Text If con.State = ConnectionState.Closed Then con.Open() Dim tbl As DataTable tbl = con.GetOleDbSchemaTable(OleDb.OleDbSchemaGuid.Tables, New Object() {Nothing, txtduname.Text, Nothing, "TABLE"}) lstsrctbl.DataSource = tbl lstsrctbl.DisplayMember = tbl.Columns(2).ColumnName End If Else If desttype = "Oracle" Then Try If con.State = ConnectionState.Closed Then con.ConnectionString = "provider=msdaora;user id=" + txtduname.Text + ";password=" + txtdpassword.Text con.Open() End If Catch ex As Exception MsgBox(ex.Message) End Try Try cmd.Connection = con cmd.CommandText = "select * from " + txtdtblname.Text cmd.ExecuteNonQuery() btncontinue.Enabled = True Catch ex As Exception MsgBox(ex.Message) End Try ElseIf desttype = "Ms Access" Then Try If con.State = ConnectionState.Closed Then con.ConnectionString = "provider=microsoft.jet.oledb.4.0;data source=" & txtdestfilename.Text con.Open() End If Catch ex As Exception MsgBox(ex.Message) End Try Try cmd.Connection = con cmd.CommandText = "select * from " + txtdtblname.Text cmd.ExecuteNonQuery() btncontinue.Enabled = True Catch ex As Exception MsgBox(ex.Message) End Try End If End If End Sub Private Sub btncontinue_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btncontinue.Click

Try If desttype = "Oracle" Then destoracle() ElseIf desttype = "Ms Access" Then destaccess() ElseIf desttype = "Sql Server" Then destsqlserver() End If MessageBox.Show("Process Compleated Successfully") Catch ex As Exception MessageBox.Show(ex.Message) End Try End Sub

For Loading data Into Ms Access:

Private Sub destaccess() con = New OleDbConnection con.ConnectionString = "provider=microsoft.jet.oledb.4.0;data source=" & txtdestfilename.Text If con.State = ConnectionState.Closed Then con.Open() cmd.Connection = con If srctype = "Text File" And desttype = "Ms Access" Then Dim sr As StreamReader = File.OpenText(txtfilename.Text) cmd.CommandText = "select * from " + txtdtblname.Text da.SelectCommand = cmd da.Fill(ds, "destbl") While Not sr.EndOfStream Dim r As String = sr.ReadLine Dim col() As String = r.Split(New Char() {","}) If ds.Tables("destbl").Columns.Count <> col.Length Then MsgBox("Num of columns in source and dest is not matching !Please verify ") Exit Sub End If Dim dr As DataRow = ds.Tables("destbl").NewRow Dim i As Integer For i = 0 To col.Length - 1 dr.Item(i) = col(i) Next ds.Tables("destbl").Rows.Add(dr) da.Update(ds, "destbl") End While ElseIf srctype = "XML File" And desttype = "Ms Access" Then Dim ds As New DataSet ds.ReadXml(txtfilename.Text) cmd.CommandText = "select * from " + txtdtblname.Text da.SelectCommand = cmd da.Fill(ds, "destbl") If ds.Tables(0).Columns.Count <> ds.Tables("destbl").Columns.Count Then MsgBox("Num of columns in source and dest is not matching !Please verify ") Exit Sub

End If Dim i, j As Integer For i = 0 To ds.Tables(0).Rows.Count - 1 Dim dr As DataRow = ds.Tables("destbl").NewRow For j = 0 To ds.Tables("destbl").Columns.Count - 1 dr.Item(j) = ds.Tables(0).Rows(i).Item(j) Next ds.Tables("destbl").Rows.Add(dr) Next da.Update(ds, "destbl") ElseIf srctype = "Excel" And desttype = "Ms Access" Then Dim con1 As New OleDbConnection("provider=microsoft.jet.oledb.4.0;data source=" & txtfilename.Text & ";Extended Properties=Excel 8.0;") con1.Open() Dim ds As New DataSet Dim cmdsrc As New OleDbCommand cmdsrc.Connection = con1 cmdsrc.CommandText = "select * from [sheet1$]" da.SelectCommand = cmdsrc da.Fill(ds, "src") cmd.CommandText = "select * from " + txtdtblname.Text da.SelectCommand = cmd da.Fill(ds, "destbl") If ds.Tables(0).Columns.Count <> ds.Tables("destbl").Columns.Count Then MsgBox("Num of columns in source and dest is not matching !Please verify ") Exit Sub End If Dim i, j As Integer For i = 0 To ds.Tables(0).Rows.Count - 1 Dim dr As DataRow = ds.Tables("destbl").NewRow For j = 0 To ds.Tables("destbl").Columns.Count - 1 dr.Item(j) = ds.Tables(0).Rows(i).Item(j) Next ds.Tables("destbl").Rows.Add(dr) Next da.Update(ds, "destbl") ElseIf srctype = "Oracle" And desttype = "Ms Access" Then Dim destcon As New OleDbConnection destcon.ConnectionString = "provider=microsoft.jet.oledb.4.0;data source=" & txtdestfilename.Text If destcon.State = ConnectionState.Closed Then destcon.Open() Dim destcmd As New OleDbCommand destcmd.Connection = destcon Dim drv As DataRowView Dim srccon As New OleDbConnection

srccon.ConnectionString = "provider=msdaora;user id=" + txtduname.Text + ";password=" + txtdpassword.Text Dim srccmd As New OleDbCommand If srccon.State = ConnectionState.Closed Then srccon.Open() srccmd.Connection = srccon For Each drv In lstsrctbl.SelectedItems ds = New DataSet srccmd.CommandText = "select * from " & drv.Item(2) da.SelectCommand = srccmd da.Fill(ds, "srctbl") Dim j As Integer Dim str As String = "create table " & drv.Item(2) & "(" For j = 0 To ds.Tables("srctbl").Columns.Count - 1 With ds.Tables("srctbl").Columns(j) If .DataType.ToString.ToUpper = System.Type.GetType("System.DateTime").ToString.ToUpper Then str += .ColumnName & " datetime," ElseIf .DataType.ToString.ToUpper = System.Type.GetType("System.String").ToString.ToUpper Then str += .ColumnName & " text," Else str += .ColumnName & " number," End If End With Next str = str.TrimEnd(",") str += ")" Try destcmd.CommandText = str destcmd.ExecuteNonQuery() destcmd.CommandText = "select * from " & drv.Item(2) da.SelectCommand = destcmd da.Fill(ds, "destbl") Dim r As Integer For r = 0 To ds.Tables("srctbl").Rows.Count - 1 Dim dr As DataRow = ds.Tables("destbl").NewRow Dim c As Integer For c = 0 To ds.Tables("srctbl").Columns.Count - 1 dr.Item(c) = ds.Tables("srctbl").Rows(r).Item(c) Next ds.Tables("destbl").Rows.Add(dr) Next da.SelectCommand = destcmd da.Update(ds, "destbl") Catch ex As Exception MessageBox.Show("problem of creating " & drv.Item(2) & "Error:" & ex.Message) End Try ds = Nothing Next ElseIf srctype = "Ms Access" And desttype = "Ms Access" Then

Dim srccon As New OleDbConnection srccon.ConnectionString = "provider=microsoft.jet.oledb.4.0;data source=" & txtfilename.Text Dim srccmd As New OleDbCommand If srccon.State = ConnectionState.Closed Then srccon.Open() srccmd.Connection = srccon Dim destcon As New OleDbConnection destcon.ConnectionString = "provider=microsoft.jet.oledb.4.0;data source=" & txtdestfilename.Text If destcon.State = ConnectionState.Closed Then destcon.Open() Dim destcmd As New OleDbCommand destcmd.Connection = destcon Dim drv As DataRowView For Each drv In lstsrctbl.SelectedItems ds = New DataSet srccmd.CommandText = "select * from " & drv.Item(2) da.SelectCommand = srccmd da.Fill(ds, "srctbl") Dim j As Integer Dim str As String = "create table " & drv.Item(2) & "(" For j = 0 To ds.Tables("srctbl").Columns.Count - 1 With ds.Tables("srctbl").Columns(j) If .DataType.ToString.ToUpper = System.Type.GetType("System.DateTime").ToString.ToUpper Then str += .ColumnName & " datetime," ElseIf .DataType.ToString.ToUpper = System.Type.GetType("System.String").ToString.ToUpper Then str += .ColumnName & " text," Else str += .ColumnName & " number," End If End With Next str = str.TrimEnd(",") str += ")" Try destcmd.CommandText = str destcmd.ExecuteNonQuery() destcmd.CommandText = "select * from " & drv.Item(2) da.SelectCommand = destcmd da.Fill(ds, "destbl") Dim r As Integer For r = 0 To ds.Tables("srctbl").Rows.Count - 1 Dim dr As DataRow = ds.Tables("destbl").NewRow Dim c As Integer For c = 0 To ds.Tables("srctbl").Columns.Count - 1 dr.Item(c) = ds.Tables("srctbl").Rows(r).Item(c) Next ds.Tables("destbl").Rows.Add(dr) Next da.SelectCommand = destcmd da.Update(ds, "destbl") Catch ex As Exception MessageBox.Show("problem of creating " & drv.Item(2) & "Error:" & ex.Message)

End Try ds = Nothing Next End If End Sub

For Loading data Into Oracle:

Private Sub destoracle() If srctype = "Text File" And desttype = "Oracle" Then Dim sr As StreamReader = File.OpenText(txtfilename.Text) cmd.CommandText = "select * from " + txtdtblname.Text da.SelectCommand = cmd da.Fill(ds, "destbl") While Not sr.EndOfStream Dim r As String = sr.ReadLine Dim col() As String = r.Split(New Char() {","}) If ds.Tables("destbl").Columns.Count <> col.Length Then MsgBox("Num of columns in source and dest is not matching !Please verify ") Exit Sub End If Dim dr As DataRow = ds.Tables("destbl").NewRow Dim i As Integer For i = 0 To col.Length - 1 dr.Item(i) = col(i) Next ds.Tables("destbl").Rows.Add(dr) da.Update(ds, "destbl") End While ElseIf srctype = "XML File" And desttype = "Oracle" Then Dim ds As New DataSet ds.ReadXml(txtfilename.Text) cmd.CommandText = "select * from " + txtdtblname.Text da.SelectCommand = cmd da.Fill(ds, "destbl") If ds.Tables(0).Columns.Count <> ds.Tables("destbl").Columns.Count Then MsgBox("Num of columns in source and dest is not matching !Please verify ") Exit Sub End If Dim i, j As Integer For i = 0 To ds.Tables(0).Rows.Count - 1 Dim dr As DataRow = ds.Tables("destbl").NewRow For j = 0 To ds.Tables("destbl").Columns.Count - 1 dr.Item(j) = ds.Tables(0).Rows(i).Item(j) Next ds.Tables("destbl").Rows.Add(dr) Next

da.Update(ds, "destbl") ElseIf srctype = "Excel" And desttype = "Oracle" Then Dim con1 As New OleDbConnection("provider=microsoft.jet.oledb.4.0;data source=" & txtfilename.Text & ";Extended Properties=Excel 8.0;") con1.Open() Dim ds As New DataSet Dim cmdsrc As New OleDbCommand cmdsrc.Connection = con1 cmdsrc.CommandText = "select * from [sheet1$]" da.SelectCommand = cmdsrc da.Fill(ds, "src") cmd.CommandText = "select * from " + txtdtblname.Text da.SelectCommand = cmd da.Fill(ds, "destbl") If ds.Tables(0).Columns.Count <> ds.Tables("destbl").Columns.Count Then MsgBox("Num of columns in source and dest is not matching !Please verify ") Exit Sub End If Dim i, j As Integer For i = 0 To ds.Tables(0).Rows.Count - 1 Dim dr As DataRow = ds.Tables("destbl").NewRow For j = 0 To ds.Tables("destbl").Columns.Count - 1 dr.Item(j) = ds.Tables(0).Rows(i).Item(j) Next ds.Tables("destbl").Rows.Add(dr) Next da.Update(ds, "destbl") ElseIf srctype = "Oracle" And desttype = "Oracle" Then Shell("imp " & txtduname.Text & "/" & txtdpassword.Text & " file=" & txtfilename.Text & " tables=(" & txtdtblname.Text & ")") ElseIf srctype = "Ms Access" And desttype = "Oracle" Then Dim i As Integer For i = 0 To lstsrctbl.Items.Count - 1 ds = New DataSet srccmd.Connection = srccon Dim drv As DataRowView = lstsrctbl.Items(i) srccmd.CommandText = "select * from " & drv.Item(2) da.SelectCommand = srccmd da.Fill(ds, "srctbl") Dim j As Integer Dim str As String = "create table " & drv.Item(2) & "(" For j = 0 To ds.Tables("srctbl").Columns.Count - 1 With ds.Tables("srctbl").Columns(j) If .DataType.ToString.ToUpper = System.Type.GetType("System.DateTime").ToString.ToUpper Then str += .ColumnName & " date," ElseIf .DataType.ToString.ToUpper = System.Type.GetType("System.String").ToString.ToUpper Then

str += .ColumnName & " varchar2(40)," Else str += .ColumnName & " number," End If End With Next str = str.TrimEnd(",") str += ")" Try cmd.Connection = con cmd.CommandText = str cmd.ExecuteNonQuery() cmd.CommandText = "select * from " & drv.Item(2) da.SelectCommand = cmd da.Fill(ds, "destbl") Dim r As Integer For r = 0 To ds.Tables("srctbl").Rows.Count - 1 Dim dr As DataRow = ds.Tables("destbl").NewRow Dim c As Integer For c = 0 To ds.Tables("srctbl").Columns.Count - 1 dr.Item(c) = ds.Tables("srctbl").Rows(i).Item(c) Next ds.Tables("destbl").Rows.Add(dr) Next da.SelectCommand = cmd da.Update(ds, "destbl") Catch ex As Exception MessageBox.Show("problem of creating " & drv.Item(2) & "Error:" & ex.Message) End Try ds = Nothing Next End If End Sub

For Loading data Into Sql Server:

Private Sub destsqlserver() If srctype = "Text File" And desttype = "Sql Server" Then Dim sr As StreamReader = File.OpenText(txtfilename.Text) cmd.CommandText = "select * from " + txtdtblname.Text da.SelectCommand = cmd da.Fill(ds, "destbl") While Not sr.EndOfStream Dim r As String = sr.ReadLine Dim col() As String = r.Split(New Char() {","}) If ds.Tables("destbl").Columns.Count <> col.Length Then MsgBox("Num of columns in source and dest is not matching !Please verify ") Exit Sub End If Dim dr As DataRow = ds.Tables("destbl").NewRow

Dim i As Integer For i = 0 To col.Length - 1 dr.Item(i) = col(i) Next ds.Tables("destbl").Rows.Add(dr) da.Update(ds, "destbl") End While ElseIf srctype = "XML File" And desttype = "Sql Server" Then Dim ds As New DataSet ds.ReadXml(txtfilename.Text) cmd.CommandText = "select * from " + txtdtblname.Text da.SelectCommand = cmd da.Fill(ds, "destbl") If ds.Tables(0).Columns.Count <> ds.Tables("destbl").Columns.Count Then MsgBox("Num of columns in source and dest is not matching !Please verify ") Exit Sub End If Dim i, j As Integer For i = 0 To ds.Tables(0).Rows.Count - 1 Dim dr As DataRow = ds.Tables("destbl").NewRow For j = 0 To ds.Tables("destbl").Columns.Count - 1 dr.Item(j) = ds.Tables(0).Rows(i).Item(j) Next ds.Tables("destbl").Rows.Add(dr) Next da.Update(ds, "destbl") ElseIf srctype = "Excel" And desttype = "Sql Server" Then Dim con1 As New OleDbConnection("provider=microsoft.jet.oledb.4.0;data source=" & txtfilename.Text & ";Extended Properties=Excel 8.0;") con1.Open() Dim ds As New DataSet Dim cmdsrc As New OleDbCommand cmdsrc.Connection = con1 cmdsrc.CommandText = "select * from [sheet1$]" da.SelectCommand = cmdsrc da.Fill(ds, "src") cmd.CommandText = "select * from " + txtdtblname.Text da.SelectCommand = cmd da.Fill(ds, "destbl") If ds.Tables(0).Columns.Count <> ds.Tables("destbl").Columns.Count Then MsgBox("Num of columns in source and dest is not matching !Please verify ") Exit Sub End If Dim i, j As Integer For i = 0 To ds.Tables(0).Rows.Count - 1 Dim dr As DataRow = ds.Tables("destbl").NewRow

For j = 0 To ds.Tables("destbl").Columns.Count - 1 dr.Item(j) = ds.Tables(0).Rows(i).Item(j) Next ds.Tables("destbl").Rows.Add(dr) Next da.Update(ds, "destbl") ElseIf srctype = "Oracle" And desttype = "Sql Server" Then Shell("imp " & txtduname.Text & "/" & txtdpassword.Text & " file=" & txtfilename.Text & " tables=(" & txtdtblname.Text & ")") ElseIf srctype = "Ms Access" And desttype = "Oracle" Then Dim i As Integer For i = 0 To lstsrctbl.Items.Count - 1 ds = New DataSet srccmd.Connection = srccon Dim drv As DataRowView = lstsrctbl.Items(i) srccmd.CommandText = "select * from " & drv.Item(2) da.SelectCommand = srccmd da.Fill(ds, "srctbl") Dim j As Integer Dim str As String = "create table " & drv.Item(2) & "(" For j = 0 To ds.Tables("srctbl").Columns.Count - 1 With ds.Tables("srctbl").Columns(j) If .DataType.ToString.ToUpper = System.Type.GetType("System.DateTime").ToString.ToUpper Then str += .ColumnName & " date," ElseIf .DataType.ToString.ToUpper = System.Type.GetType("System.String").ToString.ToUpper Then str += .ColumnName & " varchar2(40)," Else str += .ColumnName & " number," End If End With Next str = str.TrimEnd(",") str += ")" Try cmd.Connection = con cmd.CommandText = str cmd.ExecuteNonQuery() cmd.CommandText = "select * from " & drv.Item(2) da.SelectCommand = cmd da.Fill(ds, "destbl") Dim r As Integer For r = 0 To ds.Tables("srctbl").Rows.Count - 1 Dim dr As DataRow = ds.Tables("destbl").NewRow Dim c As Integer For c = 0 To ds.Tables("srctbl").Columns.Count - 1 dr.Item(c) = ds.Tables("srctbl").Rows(i).Item(c) Next ds.Tables("destbl").Rows.Add(dr) Next da.SelectCommand = cmd da.Update(ds, "destbl") Catch ex As Exception MessageBox.Show("problem of creating " & drv.Item(2) & "Error:" & ex.Message)

End Try ds = Nothing Next End If End Sub Private Sub frmconnect_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load If srctype = "Oracle" And desttype = "Ms Access" Then grpdestaccess.Location = grpdestlogin.Location grpdestlogin.Location = grpsourcefile.Location grpdestlogin.BringToFront() grpdestlogin.Text = "Source Login" btndestverify.Text = "Verify Source" Me.lbldtblname.Visible = False Me.txtdtblname.Visible = False grpdestaccess.Visible = True btncontinue.Enabled = True lstsrctbl.Visible = True Dim pt As Point = grpdestaccess.Location pt.Y += 100 lstsrctbl.Location = pt btncontinue.Enabled = False ElseIf desttype = "Ms Access" Then grpdestaccess.Visible = True grpdestlogin.Visible = False End If End Sub Private Sub btndestaccessselect_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btndestaccessselect.Click Dim ofdlg As New OpenFileDialog ofdlg.Filter = "Access database Files|*.mdb" ofdlg.ShowDialog() txtdestfilename.Text = ofdlg.FileName btncontinue.Enabled = True End Sub End Class

7. OUTPUT SCREENS

WELCOME SCREEN:

Description:

1) It displays the welcome screen with options.

FORM SOURCE TYPE:

Description:
1) The screen used to select the source among the available.

FORMS CONNECT:

Description 1) It is used to connect with the source(oracle).

IMPORTING THE FILE:

Description 1) It is used to select the source file.

VERIFY DESTINATION:

Description: 1) It is used to verifies the destination.

FORM SOURCE TYPE:

Description: 1) After verifies the destination select continue button.

PROCESS COMPLETION SCREEN:

Description: 1) It indicates the completion of process.

8. SYSTEM TESTING

The process of executing a program with the intent of finding errors in them is called testing. During testing, the program to be tested is executed with a set of test cases, and the output of the program for the test cases is evaluated to determine if the program is performing as expected. Testing forms the first step in determining the errors in the program. The success of testing in revealing errors in program depends critically on test cases.

TESTING ACTIVITIES:

Different levels of testing are used in the testing process, each level of testing aims to test different aspects of the system. The basic levels are Unit testing Integration testing System testing Acceptance testing

Unit Testing: In unit testing different modules are tested against the specifications produced. During design for the modules, Unit testing is essentially for verification of the code produced. During the coding phase and hence the goal is to test the internal logic of the module.

Integration Testing: In the integration testing, many test modules are combined into sub systems, which are then tested. The goal here is to see if the modules can be integrated properly, the emphasis being on testing module interaction.

After structural testing and functional testing we get error free modules. These modules are to be integrated to get the required results of the system. After checking a module, another module is tested and is integrated with the previous module. After the integration the test cases are generated and the results are tested.

System Testing: In system testing the entire software is tested. The reference document for this process is the requirement document and the goal is to see whether the software meets its requirements. The system was tested for various test cases with various inputs. Acceptance Testing: Acceptance testing is sometimes performed with realistic data of the client to demonstrate that the software is working satisfactorily. Testing here focus on the external behavior of the system, the internal logic of the program is not emphasized. In acceptance test the system is tested for various inputs.

BLACK BOX AND WHITE BOX TESTING STRATEGIES

Types of Testing: 1. Black box or functional testing 2. White box testing or structural testing BLACK BOX TESTING: This method is used when knowledge of the specified function that a product has been designed to perform is known. The concept of black box is used to represent a system whose inside workings are not available to inspection. In a black box the test item is treated as Black, since its logic is unknown, all that is known is what goes in and what comes out, or the input and output. In black box testing, we try various inputs and examine the resulting outputs; we can learn what the box does but nothing about How this conversion is implemented. Black box testing works very nicely in testing objects in an object oriented environment. The black box testing technique can also be used for scenario based test, where the systems inside cannot be available for inspection but the input and output are defined through use cases or other analysis information. As shown in the following figure black box testing of our Image Processing System we are not thinking of the internal workings, just we think about

What is the input to our system? What is the output for given input to our system?

Input

?
Universal Data Access Tool

output

Fig. The Black box is an imaginary box that hides its Internal workings WHITE BOX TESTING: White box testing is concerned with testing the implementation of the program. The intent of structural testing is not to exercise all the inputs or outputs but to exercise the different programming and data structures used in the program. Thus structural testing aims to achieve test cases that will force the desire coverage of different structures. Two types of path testing are statement testing coverage and branch testing coverage. Statement Testing Coverage: The main idea of statement testing coverage is to test every statement in the objects method by executing it at least once. However, realistically, it is impossible to test program on every single input, so you never can be sure that a program will not fail on some input.

Branch Testing Coverage: The main idea behind branch testing coverage is to perform enough tests to ensure that every branch alternative has been executed at least once under some test. As in statement testing coverage, it is unfeasible to fully test any program of considerable size

input

INTERNAL WORKING

output

Fig The White Box testing strategy, the internal workings

TEST PLAN: Testing process starts with a test plan. This plan identifies all the testing related activities that must be performed and specifies the schedules, allocates the resources, and specified guidelines for testing. During the testing of the unit the specified test cases are executed and the actual result compared with the expected output. The final output of the testing phase is the test report and the error report.

Test Data: Here all test cases that are used for the system testing are specified. The goal is to test the different functional requirements specified in Software Requirements Specification (SRS) document.

Testing for Time calculation of encryption algorithms The testing for Time calculation of encryption algorithms is carried out in the following way: Unit Testing: Each individual module has been tested against the requirement with some test data.

Test Report: The module is working properly provided the user has to enter correct information. All data entry forms have tested with specified test cases and all data entry forms are working properly.

Error Report: If the user does not enter data in specified (valid data) order then the use will be prompted with error messages. Error handling was done to handle the expected and unexpected errors.

TEST CODE REPORT TEST CASE NO TEST CASE NAME INPUT OUTPUT

Verify destination

With out entering the source and destination-continue Enter the wrong user name and password in oracle Inserting the correct text file in source

Cannot verify the destination Invalid login and password access denied success

Login

Text file

9. CONCLUSION

Even though tool are available for ETL in data warehousing each tool is having its the developed tool main objective is to overcome those problems. a. The project has been appreciated by all the users in the organization.

pit falls

b. It is easy to use, since it uses the GUI provided in the user dialog.

c. User friendly screens are provided.

d. The usage of software increases the efficiency, decreases the effort.

e. It also provides the user with variable options in customizing the packet capture.

f. It has been thoroughly tested and implemented.

FUTURE ENHANCEMENTS:

10. BIBILOGRAPHY

[1] Software Engineering by Pressman.

[2] Software Engineering by Pankaj Jalote.

[3] Object Oriented Software Engineering, Using UML, Patterns and Java by Bernd Bruegge and Allen H. Dutoit (Pearson Education).

[4] Beginning c# .NET 2005 by Thearon Willis, Jonathan Crossland and Richard Blair (Wrox Publications).

[5] Beginning c# .NET 2005 Database Programming by Denise Gosnell, Matthew Reynolds, Bill Forgey (Wrox Publications).