Beruflich Dokumente
Kultur Dokumente
Technical Solution
Revision History
21/09/200 Document Created Shmulik Kachlon
9
25/09/200 Staging database diagram Shmulik Kachlon
9 was added
28/09/200 Defining the solution and Shmulik Kachlon
9 describing the prison
responsibilities
29/09/200 Solution design diagram was Shmulik Kachlon
9 added
20/10/200 Stipulating the action should Shmulik Kachlon
9 be taken when offline upload
is being carried out
26/11/200 Changing the staging Shmulik Kachlon
9 database schema and
elaborating on the download
process
1.1 Acronym
1.2 Audience
The audience of this document is every third party system which needs to upload
and download users in an offline mode to and from Spobt to cater for bulk upload
and download.
1.3 Purpose
The purpose of this document is to outline the technical solution for an offline bulk
upload of users from a CC to the Spobt and the creation of the downloadable
users’ storage for the new prison in order to update its local data storage.
The reason for having a separate method for uploading CC users through a
different interface but the Spobt service is the volume of the data that will be
uploaded into the Spobt. The Spobt service is not designed to support such a
massive upload operation hence AS had to design another solution for uploading
bulk of users’ information through a software which will run in the Spobt server
directly for saving the huge traffic on the DCS internal network.
This solution will also serve as an offline tool for synching users with the Spobt
server in case of disconnection of prisons from the Spobt server for awhile for
whatever reason.
The following diagram shows the software solution design by which the CC users’
information is captured at the CC level and then copied across to the Spobt:
The section describes the technical implementation for uploading users for the very
first time to the Spobt by a CC.
The outcome of the process is described as well so that a CC could take it and
applies its local rules on it and persists the Spobt’s users it might not have locally on
its own database.
The main reason for designing another method for uploading the initial bunch of users
from a CC to the Spobt is the potential large amount of data which should be replicated
from the local CC database to the Spobt database.
If the Spobt’s service would have used for this purpose, there was no guarantee it would
be able to cope with this amount of users and the DCS network would have throttled if we
did so.
Therefore, Argus has to define another way of uploading bulks of users from a CC to the
Spobt server through a staging process to make it works faster and more efficient and
more important, feasible.
2.2 Solution’s concept
AS explored few possible technical solutions and tested few of them and made a decision
to put forward the proposed solution for it has the better test outcomes.
The concept of the solution is to enable prisons to collect users from their own local
databases, store them in a well known format by the Spobt server offline sync software
utility so that the Spobt will store it on the central database and replicate them to any CC
which is subscribed to the Spobt services.
Collecting the user is the process of getting all the users which are currently stored
in the local CC database. It is the responsibility of the CC to collect its users and
have them stored in the staging database in the right format.
Once collected, the users will have to be stored in the staging database (defined in
point 2.3.5) and populate the six tables with all the data which is relevant to the
Spobt central database.
The storage in the staging database is the responsibility of the workflow system
provider .
The AS offline upload and download software utility (point 2.3.4) will then connect
to this staging database , extracts the users from it, persists them to the Spobt
central server and update the UploadResult table (see diagram in 2.3.5) with the
external system user id (i.e. user ids provided by either Cornerstone or
DoorKeeper) alongside the corresponding data centre user id for each external
user.
The prison should then update its local database with the data centre user id for
each local user for further linking between the Spobt’s users and each prison’s
users.
If an exception has occurred during an upload of a user, the UploadResult table will
have the error information for this specific user attached to external user system id
(i.e. Cornerstone user id or DoorKeeper user id).
Once the upload operation has finished, the users are now in the Spobt central
database available for other prisons.
The AS offline sync utility tool for the Spobt will then start the download operation
for each prison which is subscribed to the Spobt.
The AS Spotb’s offline sync tool will create a staging database for each prison with
all the relevant users to this prison. E.g., if there are existing 4 prisons subscribed
to the Spobt and a new prison has just subscribed to it, the output of running the
AS Spotb’s offline sync tool will be as follow:
• An updated upload staging database with the data centre user ids
correspond to the uploaded users as well as errors in the event they occur
(this data is stored in the UploadResult table)
• Five new staging databases with the download users for each prison in the
Spobt (the four existing ones and the just added one)
The AS offline upload and download tool is a .net win form application which is
divided into two sections:
11 The upload section for the new prison’s users
11 The download section for creating a staging download database for the new
added prison and the existing prisons in the Spobt
After this has finished, a download operation can start. The download will populate
a new clean staging database for the new prison and for each existing prison in the
Spobt.
There is one difference between the download staging databases which are created
for the new prison and for the existing prisons in the Spobt:
11 For the new prison, the UploadResult table will be populated with the
ExternalSystemUserKey alongside the DataCentreUserId. For the existing
prisons the UploadResult table will be empty.
Exception handling will be done exactly the same way it is done through a normal
upload user operation using the Spobt service.
In case of exception\error while uploading a user through the AS offline upload and
download tool, the exception will be captured in the UploadResult table using the
Error column and the IsSuccessful column will be set to false.
That means, when the new prison gets back a staging database created by the AS
offline upload and download tool, when extracting the user information for updating
the data centre user id locally in the prison’s database, the prison should check the
IsSuccessful columg on the UploadResult table. If True, then a data centre user id
will be provided for the uploaded user. If False, the Error column will specify the
error which has occurred and the data centre user id will be empty.
The prison will then have to send the failed to upload users again through the
normal upload notification received by the Spobt service.
The staging database schema consists of six tables. This schema is the same for both
upload and download.
1. Providing the upload staging database to Argus so that Argus can run its offline
synchronization software tool in order to upload the new users locally on the Spobt
server
2. Extracting the data centre user ids from the UploadResult table and set it locally in
the prison’s database next to the local user id. The data centre id for each user will
be set through the offline upload process.
3. Extract the users’ information from the download staging database and store it
locally in the prison database. The download staging database has exactly the
same database schema of the upload staging database. The purpose of this
database is to provide the new prison all the users that were uploaded by the other
prisons in the Spobt before the new prison has subscribed to the Spobt.
This database is likely to be very big and will have to be extracted locally at the
prison (potential number of users can reach hundreds of thousands).
The above mentioned are the equivalent operations for synching users with the Spobt
service but in an offline mode running locally on the Spobt server without having to send
big amount of data across the DCS network.
As the offline upload is a very time consuming operation, the following stipulations have to
be taken into account:
1. A notice should be sent to all the prisons that are active against the Spobt few
days in advanced regarding the impending offline upload of the newly introduced
prison
2. The Spobt Broadcaster service should be stopped for not propagating any Spobt’s
availability messages to any prison
3. All the prisons which are subscribed to the Spobt should become inactive through
the Spobt Web Admin Tool
4. Upon completion of the offline upload, all the prisons should be reactivated through
the Spobt Web Admin Tool
5. All the prisons should be notified they are back in business
6. The Spobt Broadcaster Service has to be restarted for sending auto notification to
the subscribed prisons for normal operation to be continued