Sie sind auf Seite 1von 19
Los Angeles Unified School District Office of the Inspector General Contract Audit Unit Contract Audit Report eae ee ae ee Special Review of the My Integrated Student Information System (MiSiS) Project CA 15-996 November 17, 2014 Los Angeles Unified School District Richard; Vadovie, Ed.D. President Office of the Inspector General Mase Garcia snnett Kayser Dr. George J. McKenna Il Ménica Ratliff Steve Zimmer Members of the Board Ramon C. Cortines Superintendent of Schools Ken Bramlett, MPA, CIG Inspector General ‘November 17, 2014 Members, Board of Education Los Angeles Unified School District 333 S, Beaudry Avenue, 24th Floor Los Angeles, CA 90017 Dear Board Members: ‘This is our report on the Special Review of the My Integrated Student Information System (MiSiS) project. We will be available to respond to your questions, Respectfully, KR Ken Bramlett Inspector General c: Ramon C. Cortines, Superintendent of Schools Attachment 4333 South Beaudry Avenue, 12" Floor, Los Angeles, California 90017 Telephone: (213) 241-7700. Fax: (213) 241-6826 Inspector generol@lausd.net We conducted a special review of the District’s My Integrated Student Information System (MiSi8) project. The objectives of the review were to (i) evaluate the District's implementation of the MiSiS project including project management and deployment, adequacy of staff training and support, adequacy of equipment and infrastructure and timeliness of meeting project timelines, (ii) determine the types of problems experienced by District schools, (iii) evaluate the District’s single source selection of the firm providing the services of the MiSiS Project Director, and (iv) determine the project expenditures. Implementation of the MiSiS Project Overall we found the District’s project management of the MiSiS development and implementation to be grossly inadequate, resulting in the software being rolled out before the data integrity issues were resolved, end-to-end testing done, user acceptance testing completed, load testing was properly carried out, and interfaces with other systems determined to be functional. Furthermore, ITD did not allocate adequate resources for program oversight, and management relied heavily on information provided by the Project Director without the benefit of independent verification and validation. The following is a summary of our findings on the implementation of the MiSiS project: The requirements gathering process was inadequate. MISiS project team did not focus on data integrity ‘There was inadequate quality assurance testing during the project life cycle. ‘The required MiSiS hardware and related IT infrastructure was poorly coordinated. Load testing was not adequately performed. End-to-end testing and user acceptance testing were not performed before rollout. MISiS user training was ineffective and/or inadequate. No all-hands meeting was held resulting in poor communication. Project resource management was inadequate. Program oversight was inadequate. Recommendations: 1, We recommend that the District should develop a new project plan for MiSiS and allocate additional resources to resolve the remaining programming and data integrity issues as well as the needs for additional infrastructure, training and support. 2. We recommend that the District should procure an independent third party to provide Independent Verification and Validation (IV&V) services for the remainder of the MiSiS, project. My Integrated Student Information System Page 1 of 16 CA 15-996 (Misis) 3. We recommend that the District should create a strong PMO staff dedicated full-time to the MiSiS project to help ensure proper project oversight. 4. We recommend that the District should proactively engage all users to determine their requirements for the system and ensure that training is provided by staff that is familiar with schoo! operations. Problems Experienced by Schools Our interviews and survey of school staff indicated that schools had experienced numerous problems with MiSiS since the launch. Those problems included: (i) the system was too slow, (i) the programmed functionalities did not work, (iii) many needed functionalities/reports were not included, (iv) data in the system was incorrect, (v) the training/support offered was inadequate, and (vi) the schools’ existing equipment was inadequate for use with the new system. The schools indicated that additional resources are needed to operate MiSiS, which include new computers, more hands-on training of all users, more responsive Help Desk, additional clerical resources and compensation for overtime incurred by staff. The survey results also indicated that the continuing efforts from the MiSiS team to address the various issues have made some improvements to the system, though many problems still remain to be resolved gle Source Contract Selection of Dr. Bria Enterprises, LLC Our review found that ITD and the Contract Administration Branch followed the required procedures for a single source contract by documenting the justifications, obtaining the required approvals for the request, requesting for a pre-award rate review by the OIG, and implementing the OIG’s recommendation to negotiate a lower rate. While ITD may not have selected the right candidate in hindsight, the requirements for a single source contract were followed. Project Expenditures A spreadsheet provided by ITD indicated that as of October 2014, ITD’s expenditures for the MiSiS project totaled approximately $13.3 million. ‘My Integrated Student Information System Page 2 of 16 CA 15-996 (isis) The Modified Consent Decree (MCD) In 1993, a group of plaintiffs filed a class action lawsuit in federal court on behalf of students with disabilities in the LAUSD. The lawsuit alleged that the District's special education practices in the areas of child find, identification, tracking of student records, and placement were in violation of the requirements of the Individuals with Disabilities Education Act (IDEA). As a result of this lawsuit, the MCD required all District schools, including charter schools authorized by the District, to utilize one common data system that is connected to all school sites enabling instant access to students’ records throughout the District. My Integrated Student Information System (MiSiS) MISiS is the District’s new system for entering, managing, analyzing, and reporting student data. Itis a single system that manages all aspects of student data, including enrollment, attendance, scheduling, grades, counseling, discipline, health, and A-G course completion. MiSiS was launched at the beginning of the 2014-2015 school year and replaced the District’s legacy student data systems. SCOPE AND OBJECTIVE The objectives of the review were to (i) evaluate the District’s implementation of the MiSiS project including project management and deployment, adequacy of staff training and support, adequacy of equipment and infrastructure, and timeliness of meeting project timelines, (ii) determine the types of problems experienced by District schools, (iii) evaluate the District's single source selection of the firm providing the services of the MiSiS Project Director, and (iv) determine the project expenditures. Our review was conducted in accordance with Statements on Standards for Consulting Services established by the American Institute of Certified Public Accountants. We conducted our review from August 29, 2014 to November 7, 2014. METHODOLOGY To accomplish our objectives, we (i) made inquiries of management and corroborated responses with appropriate operations personnel, (ii) interviewed and surveyed school personnel, (iii) reviewed relevant and existing MiSiS system development and implementation documentation, (iv) observed, where possible, activities related to the MiSiS remediation efforts, including Help Desk and problem resolution, (v) compared the MiSiS development process against established industry standards (ie. the Project Management Institute's (PMI) Project Management Body of Knowledge (PMBOK) and IEEE Related Standards) and industry best practices. ‘My Integrated Student Information System Page 3 of 16 CA 15.996 (isis) I. MiSiS PROJECT IMPLEMENTATION Overall we found the District's project management of the MiSiS development and implementation to be grossly inadequate, resulting in the software being rolled out before the data integrity issues were resolved, end-to-end testing done, user acceptance testing completed, load testing was properly carried out, and interfaces with other systems determined to be functional. Furthermore, ITD did not allocate adequate resources for program oversight, and management relied heavily on information provided by the Project Director without the benefit of independent verification and validation that was used on other large ITD projects. After the rollout of the scheduling module in spring 2014, users began reporting problems. However, assurances from the Project Director and the positive nature of the status reports did not give any indications of an impending disaster. For example, when concerns were raised by the Chief Strategy Officer (CSO), about problems reported to him regarding the scheduling module in May 2014, an email response from the Project Director attributed the “negative chatter” to “a fear of change” and recommended more training. It was not until early July 2014 when the CSO became sufficiently alarmed by the problems that surfaced from the rollout at the summer schools and Bell High School that he called a meeting with the stakeholders to discuss the possibility of not proceeding with the rollout for the 2014-2015 school opening. During the meeting held on July 11, 2014, there were strong objections from some stakeholders about moving forward, however, it was concluded that by then it was too late to switch back to the legacy system for the August school opening. According to the Project Director, there was not enough time to write the conversion scripts to convert back to the legacy system and to work with the schools to redo their master schedules. Days after that meeting, the CSO sent an email to the former C1O inquiring about the confidence level for a quick resolution of the issues/bugs and about the possibility of more problems that had not yet surfaced. The CIO responded that he “asked those same questions” and indicated that the team was “pretty confident.” The CSO, the former CIO, and the Deputy CIO admitted during our interviews that although problems were anticipated upon school opening, they did not foresee the magnitude of the problems they faced. The status reports from the Project Director from mid-June through end of July showed an overall status of “yellow.” They admitted that, in hindsight, not moving forward would have been a better decision In the end, despite the District's efforts, the Office of the Independent Monitor (OTM) deemed that the District still did not meet all the requirements of the MiSiS Implementation Plan’s Milestone No. 8 by the August 1, 2014 deadline due to (1) the District's decision to delay full implementation of the Gradebook module until the 2015-2016 school year, and (2) the Distriet’s failure to have all charter schools using MiSiS by August 2014. As of the date of this report, only affiliated charters that were using the legacy system have implemented MiSiS. The independent ‘My Integrated Student Information System Page 4 of 16 CA 15-996 (Misis) charter schools are still using their own systems and the data files from those systems need to be validated and uploaded into MiSiS before MiSiS can be used at those schools, The following are our detailed findings on the MiSiS project implementation: Finding No. 1: The Requirements Gathering Process was Inadequate A cctitical process in a system’s development life cycle is the requirements gathering process. It is the process of determining, managing and documenting stakeholder needs and requirements to meet objectives, This process provides the basis for defining and managing the project scope, including product scope. A successful requirements gathering process further ensures a successful system rollout. ‘The requirements gathering efforts were led by the Project Director. Her team of subject matter experts (SMEs) worked with the “business owners” for each functional area to gather the system requirements. Business owners are the owners of the various District policy bulletins and, therefore, consisted of division/department heads and/or their staff. However, working only with the policy holders for each function was inadequate to identify all user requirements since there could be multiple departments using or requiring data from each function. For example, the Migrant Education Program was identified as the business owner for migrant student data However, the Office of Data and Accountability (ODA) is responsible for reporting migrant student data to the State Department of Education and needs the data from the system to meet state reporting requirements. According to ODA’s Director of Student Information Branch, the migrant student data screen did not meet state reporting requirements upon rollout. The same was said of homeless student data, Furthermore, feedback and involvement from end users from the school sites was not actively sought, since school site personnel were not considered “business owners.” It was only when concems were raised by the Associated Administrators of Los Angeles (AALA) regarding the system's functionality that the former CIO invited certain school personnel to attend the “summer institutes” during the summer of 2013 and 2014 to provide feedback on the system to the MiSi$ team. Four school personnel attended the 2013 Summer Institute. According to one of the institute’s participants, a lot of feedback was provided during the 2013 Summer Institute, but he did not see their feedback implemented in the system upon rollout. His impression from the summer institute was that the developers were asked to focus on the MCD requirements and school requests were considered “scope creep.” He also indicated that during the summer institute, any reference to ISIS, the legacy system, was discouraged by the Project Director. ‘Whenever such references were made, the participants were asked to “feed the pig” which meant to put a quarter into a piggy bank as a penalty for mentioning ISIS functionalities. During the 2014 Summer Institute, he indicated that the participants were asked to write user stories (which are used for requirements definition purposes) but were instructed not to speak to the developers My Integrated Student Information System Page 5 of 16 CA 15-996 (visis) per the Project Director's instructions. The speculation was that by that time the development team was busy fixing bugs reported by the summer schools and Bell High School. Furthermore, although a requirements gap analysis between the legacy systems (ESIS, SSIS and LAUSDMax) and MiSiS was performed, there were some favorable functionalities from the legacy system that were not made available in MiSiS. All of this resulted in a software program that was significantly lacking in certain required/desired functionalities or reporting capabilities upon rollout. According to the Project Director, the lack of certain required/desired functionalities or reporting capabilities was due to the fact that (i) her focus was to meet the Independent Monitor’s (IM) requirements, and (ii) the business owners were not cooperating and did not send knowledgeable people to work with her SMEs until late in the development process. We interviewed the Chief Strategy Officer (CSO), the former CIO, and the Deputy CIO about the Project Director’s claim that her focus was on meeting the IM requirements. The CIO indicated that there may have been mismanaged expectations with the Project Director, stating that since he had asked her to meet the MiSiS project milestones, in her mind she might have believed that the focus should be on the IM requirements. The Deputy CIO, who had direct oversight of the MiSiS project, stated that the Project Director was never instructed to only satisfy the IM requirements. The CSO expressed astonishment that a system rollout would not involve consideration of the needs of all the users. According to him, the IM requirements were only a subset of the overall system requirements. Our interview with the team lead for the SMEs found no evidence to support the Project Director's claim that business owners were not cooperating. The lead SME indicated that the business owners were very cooperative and sent knowledgeable people to work with her team. We also spoke with the business owner from the Student Testing Unit of the Office of Data and Accountability regarding the testing functionality who stated that he was only contacted by the SMEs a few times regarding requirements prior to the system rollout. He indicated that he attended an initial meeting where a list of requirements was handed out to the various business owners to confirm or update. He noticed that the list was incomplete and had wanted to meet further for a more comprehensive discussion of his requirements. However, he indicated that subsequent to that meeting, he had only received a few sporadic phone calls from the SMEs about questions that needed clarification, and he had thus assumed that the SMEs had another source of information for the complete requirements. Finding No. 2: The MiSiS Project Team did not Focus on Data Integrity MiSiS is a very large, complex, integrated and centralized system which pulls information from and pushes information to many other systems. In order to operate effectively, it was critical that MiSiS retrieve and effectively use legacy system data, It was discovered at “go live” that data My Integrated Student Information System Page 6 of 16, CA 15.996 (nisis) integrity issues were severe. Incomplete and erroneous data were in the system and bad data caused some MiSiS functions not to work effectively. Data integrity was the responsibility of the Project Director. Our review found that data integrity and compatibility was not a priority during the development phase of the MiSiS project. Although there was development time spent on interfaces between MiSiS and legacy systems, there was no focused effort to ensure that data integrity was adequate prior to the system going live. Interviews and document reviews indicated that there was inadequate data scrubbing, testing and coordination between MiSiS, the legacy system (LAUSDMax), ETL (the District's data warehouse) and other system interfaces, such as Welligent. Upon rollout, bad data caused data integrity and compatibility issues. This situation impeded proper operation of MiSiS and by extension, other systems that depended on MiSiS data, including Welligent. Finding No. 3: Inadequate Quality Assurance Testing During Project Lifecycle Quality assurance was the responsibility of the Project Director. During the project lifecycle, quality assurance testing was not adequately or thoroughly done for all parts of the system developed. The QA team did not have adequate resources and could not keep up with the pace of developmentichanges. Meeting minutes and analysis show that there was a need for additional QA resources as early as December 2013. The QA team consisted of only five testers until additional QA resources were brought in from Microsoft in June 2014. As a result, some system functionalities were not tested as thoroughly as they should have been, and some were not tested at all throughout the lifecycle of the project. Finding No. 4: Required MiSiS Hardware and Related IT Infrastructure was Poorly Coordinated Our review found that ITD did not have a well thought-out plan to ensure that the hardware and software requirements for access to MiSiS were met upon rollout. MiSiS is a web-based system which requires more modern hardware and operating system software configuration to work properly. Web-based systems are typically resource intensive and if the proper equipment and internet browser are not used, the process can become painfully slow and often times non- responsive. Our interviews with school personnel found that many schools were not adequately equipped with the necessary hardware and software, i.e. browser, to effectively access the MiSiS system. Some of the challenges experienced during the first week of school were attributed to improper hardware and software at some of the schools. In addition, although the server hardware installed met the requirements outlined by Microsoft, the servers were not adequately optimized to ensure that the system would work as intended. It is the responsibility of the CIO to develop a budget, which can be used to meet the infrastructure needs of the project. Further, it is the responsibility of the Chief Strategy Officer to review, approve and advocate for that budget to the Board. The Deputy CIO, the CIO and the Chief Strategy Officer failed to advocate for adequate funds to meet the hardware needs of the My Integrated Student Information System Page 7 of 16 15-996 (Misis) schools prior to the MiSiS launch. Funds were allocated in the MiSiS project budget for buying new computers for the schools but that budget was inadequate to meet the needs of the schools. In February 2014, the MiSiS project purchased and delivered one computer to each school and some larger schools received two computers. The number of computers purchased for the schools was based on available funding and not on the needs of the schools. Realizing that this effort would not be sufficient to ensure a successful experience at go-live, the project team, through the Organizational Change Management (OCM) department, sent out notices to schools in April 2014 informing them of the baseline requirements necessary to effectively use the new system. Schools were required to use their general fund monies to purchase additional computers. However, although the hardware and software requirements were posted on the MiSiS website, a complete list of the requirements was not distributed to every school with follow-up confirmation of receipt and understanding from the appropriate school official Therefore, many school officials may not have been aware of the hardware and software requirements to adequately prepare for the MiSiS launch, Another issue that caused a poor user experience was an inadequately configured server. Prior to go-live, Microsoft communicated to the District the configuration settings it believed would be best for optimal system configuration. However, after MiSiS launched, it became clear that the server configuration was not optimized enough to guarantee a smooth user experience. Subsequent to the launch, Microsoft provided the project with extra resources, in part, to analyze and offer updated configuration settings, which would improve system performance. One inherent risk of customized systems developed from scratch is that it is sometimes more challenging to identify and implement the appropriate system configuration to optimize performance. In this case, MiSiS was a system developed using Microsoft technologies and as such the District relied on Microsoft's expertise and guidance to adequately advise on the configuration necessary for optimal project performance. Poor coordination between the load testing resource and Microsoft, along with insufficient initial configuration settings and sporadic software bugs, contributed to poor system performance during the first week of school. Finding No. 5: Load Test was not Adequately Performed ‘The Project Director was responsible for the testing function. Our interview with the load testing engineer and document reviews indicated that load testing was inadequate prior to the MiSiS launch. Typically, load testing is performed in a test environment where hardware, software and configurations are identical to the production environment so that an accurate and realistic load test can be performed. However, the MiSiS load test environment was a scaled down version of the MiSiS production environment. Additionally, because the load testing engineer was not provided with information regarding the expected number of users during average and peak usage after go-live, he performed the load testing using an arbitrary number of 1,200 users as the maximum load, Part of the reason this number of users was chosen was because the system became sluggish if 1,200 users were exceeded. My Integrated Student Information System Page 8 of 16 CA 15-996 (isis) According to the load testing engineer, performance improvement rather than realistic load testing became the focus of the load testing. He indicated that the response of the servers was very slow at the beginning of the test and the testing was used to identify and isolate problems and helped with server configuration and code optimization, While the response times improved with these efforts, no realistic load testing, sufficient enough to mimic go-live scenarios, was performed. As a result, the system was painfully slow when it went live for school openings due to the number of users exceeding the 1,200 users used for load testing. With almost 1,000 schools within the District, it is apparent that just two users per school (or 2,000 users) during the first week of school would greatly exceed the load testing scenarios that were used. Finding No. 6: End-to-end Testing and User Acceptance Testing were not Performed Before Rollout A ccitical phase in the software development life cycle prior to rollout is the testing phase. End- to-end testing ensures the system is performing as designed from start to finish. User acceptance testing is the last testing carried out before the system “goes live” to ensure that it can handle required tasks in real-word scenarios, according to specifications. ‘The Project Director was responsible for the testing function, However, our interviews found that the system was rolled out before any end-to-end testing was performed and user acceptance testing was conducted for certain but not all funetional areas. The Project Director stated that in the original project plan, there was supposed to be a “code freeze” in April 2014 where all significant development efforts would cease and the team would spend extensive time performing robust end-to-end and user acceptance testing from May 2014 through July 2014 in order to have the system ready by August 2014 for school opening. Testing scenarios were fo be run that simulated using the system as a whole in an effort to identify any anomalies or bugs within the software. However, because software development efforts continued in order to deal with the multiple change and bug fix requests after the rollout of the scheduling module in April 2014 and later with the roll out of the other modules in the summer, ‘end-to-end testing of the system was never performed and only limited user acceptance testing was performed before go-live. Finding No. 7: MiSiS User Training was Ineffective and Grossly Inadequate End user training and user acceptance training are critical steps in any system development and implementation life cycle. With regard to the MiSiS project, the user community was not adequately trained on the MiSiS software prior to rollout. A series of three-hour trainings for the scheduling module were provided in early 2014 before the module’s rollout. However, interviews with the training participants found that those training sessions were considered not to be effective because the trainers did not sufficiently understand their job function, and some participants attended sessions where they were only shown video simulations of the system without any hands-on training. My Integrated Student Information System Page 9 of 16 CA 15-996 (isis) Beginning in February 2014, the three-hour training sessions began utilizing Sandboxes, which ‘were more helpful than the simulations. However, the Sandboxes did not have the full functionality of the module, In addition, there was no simulated live data within these Sandboxes to fully mimic the way the production system would operate. Furthermore, there was no training on the attendance and enrollment modules prior to rollout. Documents we reviewed showed that only around 3,300 users out of approximately 26,000 users were trained as of September 30, 2014. In addition, school personnel criticized ITD’s “train the trainer” approach (ITD expected each school to send one person to be trained and for that person to train the rest of the users at his or her school), indicating that it was not realistic for one person to be able to effectively train all the users on all the various functions after attending the training session. According to the Director of ITD’s Organizational Change Management (OCM), who had the responsibility to supervise the training team, she only had five trainers who were tasked with training the administrative staff, counselors and teachers in almost 1,000 schools. She said her team was scheduling with schools to offer face-to-face training for the gradebook module in 2013, but because principals were only allowing them to schedule the training on the schools’ limited number of professional development days, it was impossible to cover all the schools with only five trainers. With regard to the lack of training on the attendance and enrollment modules prior to rollout, she icated that it was decided that no training was necessary on those modules because those modules were similar enough to the legacy system making it easy for users to adapt. This was a miscalculation on the part of the MiSiS project team because attendance and enrollment problems with the new software were extensive. In addition, the Director of ITD’s OCM indicated that there was no training environment, ic. a suitable location to train the teachers, and that those modules were not ready until June or July 2014 and by then school was already out. She opined that the problems with those modules were largely due to the software not working properly and not because of a lack of training. Finding No. 8: No all-hands Meeting was Held Resulting in Poor Communication There were no periodic “all-hands” meetings with all the key stakeholders including, various project teams, users and interface owners. Several MiSiS project related meetings were held periodically including a weekly PMO meeting, a weekly MiSiS meeting, daily team lead meetings, a monthly executive meeting, and more recently a daily meeting with SMEs, quality assurance (QA) staff and developers. However, no periodic “all-hands” meetings with all the key stakeholders including the users and related project personnel were ever held. We believe the Project Director’s failure to hold periodic comprehensive “all-hands” meetings contributed to poor project performance. Finding No. 9: Project Resource Management was Inadequate My Integrated Student Information System Page 10 of 16 CA 15-996 (wisis) Throughout the project there were challenges related to the availability of resources to effectively manage the workload, particularly resources for software development and quality assurance, The process of on-boarding new project team members involves the Project Director ‘making requests to the Deputy CIO (direct report). Many of the resources that were brought onto the MiSi$ team were outside consultants, and the process of bringing on board consultants took time, Through interviews and documentation review, it became evident that resource management at certain points throughout the project was problematic. We obtained project status reports dating back to November 2013 where concems were expressed regarding insufficient resources, Requests were made for additional resources several times; however, it was the lack of QA resources that was a significant issue. The MiSiS QA team consisted of only five testers until additional QA resources were brought in from Microsoft in June 2014. On a complex project with a short implementation timeline, not having resourees on board timely had a detrimental effect on the quality of the deliverables and the ability to meet project timelines. The need for bug fixes and system changes after the rollout of the scheduling module in April 2014, and the other modules in the summer of 2014, exacerbated the resource issue and negatively impacted the quality of the software put into production. Finding No. 10: Program Oversight was Inadequate ITD’s Project Management Office (PMO) was charged with oversight of the MiSiS project. The Project Management Office (PMO) consisted of the Deputy CIO, the Deputy CIO's Administrative Assistant (both of whom are not full-time in the PMO), and the Director of Software Project Management, who is full-time in the PMO. According the Deputy CIO, the PMO had approximately 50 ongoing projects to oversee, and one full- time person and two part- time persons were not sufficient to effectively oversee the MiSiS project and the 49 other District projects. The Deputy CIO stated that he reviewed the weekly project status reports provided by the Project Director. However, he was not aware of the magnitude of the problems associated with the project because, according to the status reports, there did not seem to be any cause for concem, He stated that the status reports usually indicated “green,” and even if one week it indicated “red,” the next week it would show “green” again. He stated that since ITD did not contract for any independent verification and validation (IV&V) services, he could not have known how accurate the status reports were. Typically, large and complex undertakings like the MiSiS project would utilize an independent contractor to provide IV&V services. Verification and validation (V&V) processes are used to determine whether the development products of a given activity conform to the requirements of that activity and whether the product satisfies its intended use and user needs. V&V life cycle process requirements are specified for different integrity levels. The scope of V&V processes encompasses systems, software and hardware, and it includes their interfaces. This standard applies to systems, software and hardware being developed, maintained, or reused (legacy, ‘My Integrated Student Information System Page 11 of 16 CA 15-996 (Mi commercial off-the-shelf (COTS), non-development items). V&V processes include the analysis, evaluation, review, inspection, assessment, and testing of products. This is a best practice which ITD has previously adhered to when it implemented the District’s current SAP platform. An IV&V contractor would sit in on project meetings and review key project documents and offer an independent assessment of the progress and health of the project. However, for the MiSiS project, no discussions were held to utilize an independent third party to monitor project performance, and software development and implementation. The absence of this critical element of project management control, combined with the lack of resources for adequate program oversight by the PMO, created the “perfect storm” wherein the decision was made for MiSiS to go-live before it was ready Il, MiSiS PROBLEMS EXPERIENCED BY SCHOOLS We visited seven District schools and interviewed school personnel to obtain feedback about their MiSiS user experience. We found that school staff had experienced numerous problems with MiSiS since the launch. The feedbacks were unanimous: (i) the system was too slow, (ii) the programmed functionalities did not work, (iii) many needed functionalities/reports were not included, (iv) data in the system was incorrect, (v) the training/support offered was inadequate, and (vi) the schools’ exis ing equipment was inadequate for use with the new system. Subsequent to the interviews, we conducted a survey of 210 District schools (10 elementary schools, 10 middle schools and 10 high schools selected from each Board District) to determine the extent and impact of those problems. We received responses from 191 of the surveyed schools. All of the respondents experienced various problems with the MiSiS rollout (see Table 1D. Our survey found that on a scale of 1 (lowest) to 10 (highest), MiSiS scored an average of 2 upon school opening. The respondents gave an average score of 2 to the quality and quantity of training they received for the scheduling, enrollment and attendance modules, and an average score of 1 to the quality and quantity of training received for the transcripts module. Most respondents indicated that the training was not enough, the instructors were not effective because they did not know school operations, and the training was not hands-on, The MiSiS Help Desk received an average score of 3; the respondents indicated that the wait time was extremely long and the staff could not answer their questions. Most (69%) of the respondents reported having received some resources (new computers, compensation for overtime, and technical assistance from ITD) for implementing MiSiS, but additional resources are still needed to operate MiSiS. These include: New computers More hands-on trai g for all MiSiS users A more responsive Help Desk Additional clerical help for data input Compensation for overtime incurred by staff My Integrated Student Information System Page 12 of 16 CA 15-996 (Misis) ‘The survey also indicated that the continuing efforts from the MiSi$ team to address the various issues has made some improvements to the system, though many problems still remain to be resolved. The respondents gave an average overall score of 4 to the system as of the date of our survey, an increase of 2 on the rating scale since school opening. Table I Problems Reported by Schools Isthis a problem | The degree of the Impact | Has the problem been inyourschoo!? | ofthe problem solved? Description of Problems ‘Yes [No _| High | Moderate | Slight [ Yes [ No_| Not Sure Overall [Siam i eo slow wom ee | oe ae see | Jo interface wth Weligent and other | yy se bsou la ae ve som | am [osm | 20% | 4% | 9% | so%| 3% generated using MISS me os 7 ESfiisimemesintmaion | 9% | % [ore | ow | os | os [oa corec student data, eg inconest | yy ; noorrect student dal, ei 97% | 3% | 79%] 17% | 3% | 12% | srm%| 31% ony enteredsaveddsta not saved | gga, mm loim |) ae lanl sm) aes the system {Computers to oto ran MS om Tim [tom | ae ae | 9 | id not receive adequate waining | 89% Tie —1a% | 5%] 9% | H5%% | 8% ‘Scheduling fried Geri |, | aay, | eam | tse | am | 20 | ame | am jot working a intended tcl information Tepons neooSaEy or scheduling nat avaiable (es we | am | aay . checling not valible 8) 1 gays | io | sor | 12% | 2% | 12% | 53% | 35% se Conflicts) Manually input schedules not saved | 80% | 20% | 74% | 21% | 31% | 31% | 35% | 34% [Cannot assign counselor to students | 60% | 40% | 63% | 27% | 10% | 23% | 55% | 20% fitaleprs camara] (Cass Enrolment Repors, Student | 86% rom | 19% | 3% | 28% | 47% | 250% Schedules, te) | | Enrolment i t ent enrollment data doesnot save | yso, | 74 | aava | 15m | 2% | 20% | 43% | 29% roperly nconect student enrollment asian | 57x, | 394 | gam | tare | ar | 22% | 46% | 32% | system. | Attendance | {Cannot print 5 Colum Raster aa ome | ee |_| ame [a [9% Pubmitedatendance information Tsay [12m | sare | 15% | 1% | 25% | 55% | 20% 9 show reports comet [wee _| Tame [nine [15m | ae ae a | 56m absite teachers having problems gay, | 190 Tong | ag% | sre | anne | amr asve with accessing module Transcript Transript cannot be printed [36% [sare _| Tame | 1 ia | Sim | a% por card cannot be printed ore | 40% | 75% | 18% 399% | 39% | 20%6 [Transcripts incomect 70% [30% | 75% | 2096 996 | 61% | 28% port cardi incorrect Tie | 29% [68% | 20% 19% | se | —20%% My Integrated Student Information System Page 13 of 16 cA 15.996 (Misis) Ill. SINGLE SOURCE SELECTIO! The Project Director’s firm, Dr. Bria Enterprises, LLC, was awarded a single source contract by ITD in June 2013 to provide services as the MiSiS Project Director. The initial term of the contract was from July 1, 2013 through June 30, 2014 for $280,800. In April 8, 2014, ITD extended the contract through June 30, 2015 and increased the contract amount to $561,600. We reviewed documentation maintained by ITD and also interviewed ITD and procurement personnel and contractors to obtain an understanding of the circumstances under which the single source contract was awarded and determine whether there were any violations of District policies. Before July 1, 2013, the Project Director had been working in ITD in the same capacity but through another ITD contractor, P. Murphy & Associates (P. Murphy). According to the president of P. Murphy, her firm had recruited the Project Director from Arizona for the Project Director position when ITD had requested for resources to fill the position. Documentation provided by ITD indicated that ITD had made the request for resource from three firms and two hhad sent candidates for interviews. The two candidates were interviewed by ITD’s Deputy Director and the Project Director was selected based on her higher score. She came on board ITD on June 25, 2012. In May 2013, ITD management made the request to the Contract Administration Branch to issue a single source contract to her firm, rather than using P. Murphy's contract for her services. Under this single source contract, the Project Director’s firm was to be paid $135 per hour as opposed to the $116 hourly rate ITD was paying P. Murphy for the Project Director's services under P. Murphy's contract. ‘The District allows single source selection in cases where procurement and the sponsoring department agree via justification that only a single vendor will properly meet the needs of the District. Unlike sole source selection where there is only one source, single source selection is applicable when there are multiple sources available for the product or services, but the District determines that only one particular source is preferred due to specific reasons, such as quality, availability, or capacity. In making the single source contract request, ITD followed the District’s protocol by providing justifications for the single source contract, and the request was approved by the Contract Administration Branch and the Chief Operating Officer. According 10 ITD’s single source request, the Project Director’s project management and program oversight services were necessary to provide continuity to the MiSiS project and to ensure the completion of the MiSi$ milestones based on the timeline approved by the Office of the Independent Monitor (OM). Contract Administration Branch staff indicated to us that they had evaluated the request and had determined that a pre-award rate review was necessary to censure that the proposed hourly rate was supported and reasonable. ‘The OIG performed the pre- award rate review per Contract Administration Branch’s request and found that (i) Dr. Bria Enterprises, LLC did not have any significant overhead expenses to justify the proposed $135 hourly rate, and (ii) the proposed rate was higher than ITD’s average hourly rate of $120 which it ‘My Integrated Student Information System Page 14 of 16 CA 15-996 (Misis) paid other vendors for the same labor category. However, when we informed the Project Director that her proposed $135 hourly rate was not supported and not reasonable, she indicated to us that she needed a higher compensation than what she was receiving from P. Murphy in order to continue on with the MiSiS project. The former Chief Information Officer also stated that her services were critical to the MiSiS project. In the end, our rate review recommended an hourly rate not to exceed the average hourly rate of $120 that ITD had with its other contractors for the same labor category. Subsequent to our report, the former Chief Information Officer negotiated an hourly rate of $120 with Dr. Bria Enterprises, LLC. Our review found that ITD and the Contract Administration Branch followed the required procedures for a single source contract by documenting the justifications, obtaining the required approvals for the request, requesting for a pre-award rate review by the OIG, and implementing the O1G’s recommendation to negotiate a lower rate. IV. PROJECT EXPENDITURES A spreadsheet provided by ITD indicated that as of October 22, 2014, ITD’s cost for the MiSiS project totaled approximately $13.3 million. Table II shows a summary of the information shown on the spreadsheet. Table Project Costs FY IHS ca Bud; FY 13-14 Total ca Asoftonzia Budget Labor Cost Employee § Benefits Project Team $9,072,554 $4,097,996 $1,890,477 $5,988,473 83,084,081 Se 961,949 603,941 251.397 855.338. 106,611 Help Desk Overtime 153,77 mn : 1,777 152,000 School Sta 1,560,000 e 4147441474 1,518,526 Training Related Mileage & Other 189,046 _ 139,046 : 50,000 11,937,326 4,842,760 2,183,348 7,026,108 4 911,218 Consultants 10,900,828 5,088,656 1.237.637 __ 6.326.293 4,574,535 Total Labor Cost 22,838,154 9.931416 3,420,985 13,352,401 9,485,753 My Integrated Student Information System Page 15 of 16 cA 15.996 (Misis) Project Costs FY 1415 Unspent Budget FYIS14 asorio2a14 70a Budget Non-Labor Cost: Professional Development 120,000 S a - 120,000 Materials & Supplies : : : 36,000 Equipment 15,986 - 15,986 __ 3,674,014 Total Non-Labor Cost 3,846,000 15,986 - 15,986 3,830,014 Contingency 8,792 - - = 4,518,792 Total $31,202,946 $9,947,402 _ $3,420,985 _ $13,368,387 $17,834,559 ‘The Board of Education approved an additional $3.6 million in support of the MiSiS project to purchase up to 3,340 desktop computers for 784 school site offices that have 6,853 staff members who use MiSiS on a regular basis on October 14, 2014. REPORT DISTRIBUTION This report may not be released, in full or part, to any entity outside LAUSD or to any intemal department without prior approval of the Office of the Inspector General This special review was conducted by the Office of the Inspector General’s Contract Audit Unit and consultants from BCA Watson Rice LLP Team: Stella Lai, Audit Manager Shelly Guo, Principal Auditor Rachel Chow, Senior Auditor BCA Watson Rice, LLP My Integrated Student Information System (Misis) Page 16 of 16 CA 15-996 Know about fraud, waste or abuse? Tell us about it. Maybe you are a School District Employee, or maybe you are a private citizen, Either way, you area taxpayer. Maybe you know something about fraud, or waste, or some other type of abuse in the School District. The Office of the Inspector General has a hotline for you to call. You can also write to us. If you wish, we will keep your identity confidential. You can remain anonymous, if you prefer. And you are protected by law from reprisal by your employer. Call the hotline: (213) 241-7778 or (866) LAUSD-O1G Write to us: Fraud Hotline Center 333 S, Beaudry Ave., 12" Floor Los Angeles, CA 90017 Website: http://achieve.lausd.net/oig

Das könnte Ihnen auch gefallen