Wendy Greuel Controller: An Equal Employment Opportunity - Affirmative Action Employer

WENDY GREUEL CONTROLLER
July 18, 2012 Dr. Daniel LaVista, Chancellor Los Angeles Community College District 770 Wilshire Boulevard Los Angeles, CA 90017 Dear Dr. LaVista: The Los Angeles Community College District (LACCD) is an integral part of educating our City's future workforce and serves over 250,000 students annually at nine colleges. Today I am releasing an independent review of the Processes Used to Select an Inspector General for the Los Angeles Community College District. My review found that although the solicitation process generally reflected the intent of the Board of Trustees, the Request for Proposal (RFP) and subsequent evaluations were deficient in several areas. In 2010, LACCD's Board of Trustees established an Office of Inspector General to oversee the District's $5.75 billion bond-funded construction program, following a series of audits and investigations. More than 10 bids were submitted for the April 2010 RFP, and in October of 2010, Policy Masters, Inc. was selected for the contract, despite not having experience in directing audits and investigations or a history of successful work elsewhere as a firm since it was newly established. Additionally, a subcontractor to the firm had financial audit experience but did not have performance or operational audit experience, as required in the RFP. My review determined that the original RFP was vaguely worded, leading to significant confusion about the deliverables expected for the project. In one instance, the District issued an addendum, three business days before proposals were due, directing bidders that they would be expected to provide a full-time consultant to serve as Inspector General. This was insufficient time for a bidder to arrange for a full-time, senior level consultant with sufficient experience and qualifications. My review also found numerous other irregularities in the interview process including: The District failed to actively vet potential conflicts of interest.
200 N. MAIN STREET, SUITE 300, LOS ANGELES, CA 90012 (213) 978-7200 HTTP://CONTROLLER.LACITY.ORG
AN EQUAL EMPLOYMENT OPPORTUNITY - AFFIRMATIVE ACTION EMPLOYER
Dr. Daniel LaVista, Chancellor July 18, 2012 Page 2
The applications received were, not evaluated in a consistent or methodological manner. The Deputy Chancellor chose different panelists for the proposal review and the interview process and as a result, those conducting the interviews did not have full information about all bidders and how the finalists compared to the other bidders. The two finalists made significant changes to their proposals between the first and second round of interviews, without clear evidence of permission from LACCD management.
I urge the LACCD Board of Trustees to take a series of steps, including revising its policies and procedures, to help prevent the district from entering into future flawed RFP processes, as these deviations from the procedures did not allow for the full benefit of the competitive bidding process. Our community college system is a valuable asset to Los Angeles and an essential part of developing our workforce for the future. We must ensure that taxpayer funds are spent effectively and that proper procedures for contractor selection must not only be in place, but also adhered to in all transactions. Sincerely,
W~~
WENJ"GREUEL City Controller
Review of the Processes Used to Select an Inspector General for the Los Angeles Community College District
Submitted by
Harvey M. Rose Associates, LLC

http://www.harveyrose.com July 16, 2012
HARVEY M. ROSE
ASSOCIATES, LLC
1390 Market Street, Suite 11 50' Sa n Francisco,California 94102 (41 5) 5529292 . (4 15) 252-046 1 (FAX) info@harveyrose.com
public sector management consulting
L::::".
D oorn
DDODD
HMR
July 16,2012
Ms. Wendy Greuel City Controller City of Los Angeles 200 North Main Street, Room 300 Los Angeles, CA 90012 Dear Ms. Greuel: Harvey M. Rose Associates, LLC is pleased to present this Review of the Processes Used to Select an Inspector General for the Los Angeles Community College District. This report was prepared in response to a request from the Chancellor and President of the Board of Trustees of the Los Angeles Community College District to have your office conduct an independent review of the Inspector General contractor selection process. Specific project objectives included: determining how the selection process considered the experience and qualifications of proposing firms to ensure that the selected firm met the expectations of the District; determining whether the District's own procurement rules were followed; and, determining if there is evidence of inappropriate actions occurring in the District's selection and contract award for Inspector General services. Thank you for providing our firm with the opportunity to conduct this review for the Los Angeles Community College District. Upon your request, we are available to present this report to you and/or District representatives and to respond to any questions about our results from you and your staff.
Sincerely,
Fred Brousseau Project Manager
Table of Contents
Executive Summary ................................................................................................................... i Background, Objectives, Scope and Methodology ................................................................... 1 1. 2. Analysis of Process for Selection of Contract Inspector General ................................... 9 Assessment of Qualifications and Experience of Bidders ............................................. 24
Appendix: LACCD Professional Services Procurement Policies
Executive Summary
In 2010, the Los Angeles Community College District Board of Trustees authorized establishment of an Office of Inspector General to oversee the Districts $5.725 billion bondfunded construction program through a program of audits and investigations. The Office had not previously existed and the District decided to fill this function with a contractor rather than by inhouse staff. A Request for Proposals was issued in April 2010, eleven bids were received in May, 2010 and a contract was awarded to one of the bidders, Policy Masters, Inc., in October, 2010. In August 2011 the State Controller issued an audit of the Los Angeles Community College Districts (LACCD) bond construction program for the period of July 1, 2001 through December 31, 2010. The audit noted four primary findings, all of which were contested by LACCD, including the following, noted as Finding Number 3: LACCD ignored its internal procurement rules and guidelines in the selection of the Inspector General which, at least in appearance, compromised the integrity of the Inspector General to carry out the essential functions of the office. LACCD disagreed with this finding, stating that the State auditors based their conclusion on incorrect and incomplete information. The LACCD Board and Chancellor requested that the City Controller conduct an independent investigation into the process and circumstances that led to the selection of the Inspector General, as well as the qualifications of the selected firm. The objectives of this Review of the Processes Used to Select an Inspector General for the Los Angeles Community College District conducted for the Los Angeles City Controller at the request of the Chancellor of the Los Angeles Community College District were to: 1. Evaluate the process used by LACCD during 2010 for the solicitation, selection, and contract award for Inspector General services; 2. Determine how the selection process considered the experience and qualifications of proposing firms, to ensure that the selected firm met the expectations of the LACCD Board of Trustees for an Inspector General; 3. Determine whether LACCD followed its own procurement rules for the selection of Inspector General services, given the overarching objective of adhering to a fair and open process that would ensure that the most qualified proposer was selected; and, 4. Evaluate the selection process and recommended award to determine if there is evidence of any inappropriate actions that occurred with regards to the selection and award.
Summary of Review
This review found that LACCDs solicitation process for contract Inspector General services generally reflected the intent of the Board of Trustees. However, the process
Executive Summary
suffered from weaknesses in the clarity and specificity of contractor requirements in the Request for Proposal (RFP), weak methods and poor documentation of the proposal review and evaluation process and unclear evaluation criteria for, and documentation of, the finalist interview process. The requirements and expectations for the Inspector General contractor changed between what was included in the RFP and how finalists were evaluated by the finalist interview panel. The firm ultimately awarded the contract was not the most qualified based on the RFP requirements, but appears to have best met the expectations of the interview panel and the Chancellor. As a result, the District did not obtain the full benefits of competitive bidding since the actual criteria used for selection varied from that in the RFP upon which bidders based their proposals. LACCDs solicitation process for contract Inspector General services generally adhered to the District special and professional service procurement policies and procedures, though with some exceptions. A review of those policies and procedures found that they are generally adequate though weaknesses were identified in the areas of: documentation requirements; bidder evaluation and scoring protocols; and, conflict of interest vetting requirements. The Inspector General solicitation process was found deficient in these areas as well.
A summary of individual findings in this report is as follows.
Current District Policies and Procedures for Procurement of Professional Services Contain Weaknesses
District policies established for the selection of contractors for professional and specialized services is consistent with legal requirements and is generally adequate. However, our review of existing District policies identified gaps in (1) documentation requirements; (2) scoring methods and protocols; (3) bidder evaluation guidance; and, (4) conflicts of interest. Many of these weaknesses were observed in the Office of Inspector General solicitation process.
The RFP Lacked Specificity and Clarity

The process for soliciting and selecting a contractor for Inspector General services varied from common District practice for professional services in that it: (1) was a new function that the District had no prior experience with; (2) was requested by the Board of Trustees rather than a department familiar with the services to be provided that would have typically prepared the scope of work, contract duration, and provided a listing of firms for outreach, and; (3) occurred at a time when two key executive management positions were vacant: the LACCD Chancellor and the Director of Business Services. Development of the RFP and scope of work, as well as the process for review of proposals received, was delegated to the Deputy Chancellor. While the RFP provided a general overview of the work to be done by the new function of the Office of Inspector General, it lacked specificity and clarity in key areas pertaining to the functions. The RFP did not clearly state a
ii
Executive Summary
District expectation that the selected contractor should provide a full-time position to serve as Inspector General. Though this information was communicated to potential bidders in an addenda to the RFP, that addenda was issued May 7, 2010, giving bidders an unrealistically short three business days to secure a full-time senior level consultant with the appropriate skills and qualifications. Consequently, many of the bidders did not propose a team member to serve as full-time Inspector General. The RFP was vague on contractor deliverables. Specific deliverable requirements were not specified other than references to the Office being required to conduct audits and investigations, make regular reports to the Board of Trustees and operate a whistleblower hotline. The required number of audits or investigations to be conducted or reports to be made to the Board of Trustees was not provided, making it challenging for the bidders to determine the level of effort required and to prepare price estimates. The lack of specificity in this regard also limited the Districts ability to determine how the bidders price estimates compared to each other. The RFP was prepared without a full listing of standards, priorities and requirements of the role of the Inspector General. There was no mention in the RFP of required Inspector General professional standards in key areas such as independence and conflicts of interest. Given the nature of the work to be performed, such standards are critical to the successful functioning of the Office. A number of bidders that submitted questions prior to submitting their proposals demonstrated that they did not have a clear understanding of what the District currently had in place in the way of Inspector General services and what exact services the selected contractor would be required to perform. Such information should have been provided in the RFP.
Proposal Review Process Poorly Planned, Executed, and Documented

The eleven proposals received were reviewed and scored by a panel of District personnel and outside parties based on the following criteria: Qualifications, Experience, and References (25 points) Staffing, Project Organization (20 points) Work Plan Technical Approach (20 points) Fee (25 points) Miscellaneous, Exceptions, and Deviations (10 points)
The scoring sheets used for this purpose were vague and the review panelists did not receive detailed guidance about how they should be applied to the bids for the Inspector Generals Office contract. For example, the panel was not instructed on what appropriate experience and qualifications should include and awarded the highest score to a firm with no auditing experience at all, though auditing was one of the two key functions to be performed by the Office of Inspector General. That highest scoring firm was subsequently disqualified.
iii
Executive Summary
While each member of the four person proposal review panel was expected to assign scores to each of the eleven bidders, three bidders were not fully scored, resulting in lower overall scores for these firms. Specifically, one panelist did not provide a score for one firm due to a personal conflict of interest, and another panelist did not rate two other proposals because the firms had bid only on the whistleblower hotline service. Through these panelists own interpretations of how the scoring process should work, the panelists effectively rated the three firms as zero, and these firms thus received significantly lower scores through no fault of their own. In addition, the total scores were not used to provide overall rankings, which is an exception to District policy. Rather, bidders were advanced as finalists based on a group discussion- after individual scoring was complete- in which panelists expressed their opinions about the proposals. District policy calls for firms that receive the highest cumulative score to be recommended for contract award. But the firm with the highest score was disqualified by the panel because it was considered too small to do the needed work.
First Round of Interviews, Critical to Selection of an Inspector General, was Poorly Planned and Executed
A separate interview panel was assembled to interview the four finalist firms identified by the proposal review panel. The interview panel consisted of District staff, members of the Board of Trustees and two parties external to the District. Again, little information was provided to the panelists about District expectations for the Inspector Generals Office. Questions were prepared in advance, but they were not used consistently for all bidders. According to District policy, information obtained through the interviews may then be considered by the proposal evaluators in assigning final scores to the proposals. LACCD management departed from this policy by not assigning final scores. Though the interviews were key to final selection of a bidder, the criteria used to evaluate the bidders was not documented, and the interview results were not scored or formally documented. Rather, consistent with the proposal evaluation process, a group discussion took place after the interviews, and agreement was reached on bidder rankings. Reference checks were conducted by an outside contractor on all four finalist firms that were interviewed1. Though the reference interviews were transcribed in a written report, is not clear how these results were used by the interview panel or District staff. A second round of interviews was conducted with the same bidders in August, 2010 to ask more detailed questions in a less formal setting. Unlike common District practice, there was no set of formal questions to be asked of each bidder, as was the case for the first round of interviews. The second set of interviews was also not formally documented. Two of the firms made changes in their proposed staffing between the first and second round of interviews. In one case, the firm added investigator staff to their team. In the other case, the bidder clarified who would actually serve as the Inspector General. Both of these changes
We could not verify the validity of the reference checks as names of the individuals interviewed were redacted on the copies provided to our review team. Harvey M. Rose Associates, LLC
iv
Executive Summary
reportedly improved the firms qualifications but it is not clear if all bidders were given the opportunity to do this or if these two firms made these changes on their own. The two bidders that made the changes were recommended to the Chancellor as top finalists, one of which was ultimately selected and awarded the contract.
LACCD Did Not Actively Vet Conflicts of Interest

The bidder reference checks conducted after the first round of interviews did not include background checks on the firms or checks for potential conflicts of interest, which would have been useful given the sensitive nature of the work to be performed by the Office of Inspector General. For example, the fact that the president of Policy Masters, Inc. had previously worked for Gateway Science and Engineering, Inc., a company that had been contracting with the District for several years to oversee bond-funded construction work, was not disclosed or addressed in the process as a potential conflict of interest for an Office of Inspector General that would be auditing and investigating that firms work.
Bidders Experience and Qualifications were not Consistently or Methodically Assessed

The RFP emphasized that the Office of Inspector General would be responsible for audits and investigations, but the eleven proposals submitted were not systematically reviewed for their audit and investigation experience and qualifications. In fact, the top ranked bidder had no audit experience whatsoever. The RFP required performance, operations, financial and compliance audit experience, but only one of the four firms advanced as a finalist reported performance or operations audit experience in their proposal.
Inspector General Contract Award was Based on Qualifications other than those Emphasized in the RFP
The firm ultimately selected by the District to serve as the Inspector General was Policy Masters, Inc. Members of the interview panel reported that this firm made the best presentation at the two rounds of interviews and had a team with knowledge and experience in the construction industry and knowledge of the District and its needs. While these can all be considered valuable attributes for the selected contractor, the firm did not have experience in directing audits and investigations or a history of successful work elsewhere as a firm since it was newly established. A subcontractor to the firm had financial audit experience but did not have performance or operational audit experience, as required in the RFP. The firm did not have investigation experience initially though it added a team member who could perform that role between the first and second round of interviews. The originally proposed team members had related experience in school district construction projects, development of policies and procedures and financial audits. The interview panel appears to have redefined the role of the Inspector Generals Office through the solicitation process compared to how it was defined in the RFP. While it may have ended up
Executive Summary
with the contractor that best met its needs out of the bidder pool, it did not obtain the full benefits of competitive bidding since the actual criteria used for selection varied from that in the RFP.
Report Review by District

The District was provided a draft version of this report for review. Subsequently, an exit conference was conducted with District representatives to obtain any factual clarifications and comments on the report. The District also provided a written response with more detailed responses to the specific findings and details of the report. Key points in the Districts written responses were agreement with the Inspector General Office solicitation process not being well documented and the need for clarification of several points in the Districts Professional Servicers Procurement Policy (PP-04-08) and plans for improvement in both of these areas. While better documentation and a more clear professional services procurement policy would have improved the Inspector General solicitation process, a few areas warrant additional mention: 1. In response to the point in the report that the Inspector General interview panel did not score or methodically rank the bidders that were interviewed, the District states a number of times in their written response that scoring the interviews is not required by current policy. The more important deficiency represented by the absence of formal scoring is that there is no record of the criteria used to assess the interviewed bidders and no record showing that the interview evaluation process was systematic or methodical. A final scoring sheet, incorporating the results of the interview process, or some other documentation of a methodical assessment using standardized criteria, would provide important evidence of an objective and transparent process. 2. The District states in its written response that having two panels for the proposal review and interview processes was designed for greater objectivity. Whatever its intent, a negative impact of this approach was that the interview panel did not have information on how the bidders had been assessed relative to the criteria in the Request for Proposals, which would have provided areas to focus on in the interviews relative to standardized evaluation criteria. Combined with a lack of familiarity with inspector general functions, the interview panel was not well equipped to evaluate the strengths and weaknesses of the firms relative to the functions the contractor would be expected to perform.
vi
Background, Objectives, Scope and Methodology

Background
Harvey M. Rose Associates, LLC (HMR) was retained by the Los Angeles City Controller to conduct a Review of the Processes Used to Select an Inspector General for the Los Angeles Community College District. This independent review was conducted at the request of the Chancellor and President of the Board of Trustees of the Los Angeles Community College District based on concerns raised by the State Controllers Office1 and others regarding the selection process for Inspector General services and qualifications of the selected bidder for these services. The purpose of the review was to assess the processes used by the Los Angeles Community College District (LACCD) to select a contract Inspector General to oversee the Districts bond-funded construction program. LACCD, serving over 250,000 students annually at nine colleges, is the largest community college district in the United States. From 2001 to 2008 the voters in the LACCD approved three bond measures totaling $5.725 billion to expand and make improvements to LACCD campuses. Due to a number of deficiencies in the resulting capital facilities development program, as identified in a report by a District consultant, the LACCD Board of Trustees adopted a motion in March 2010 authorizing implementation of one of the consultants recommendations: creation of an Office of Inspector General to oversee the bond-funded capital program and to ensure that it performs with the utmost integrity and efficiency.2 District management decided to engage a contractor to fill this role rather than creating a new District position because bond proceeds cannot be used to pay for District administrative staff and District management was concerned about committing General Funds to cover the new expense. Between approximately March and September 2010, the District engaged in a professional services solicitation process in which a Request for Proposals for a contract Office of Inspector General was developed and issued, eleven proposals were received and evaluated by the District, four finalist bidders were interviewed, and one firm, Policy Masters, Inc. was selected and approved by the Board of Trustees to serve as the Districts Inspector General under contract. The agreement between the District and Policy Masters, Inc. calls for an annual payment of $701,680 to perform audits and investigations related to the Districts bond-funded construction program and to operate a whistleblower hotline. The contractor has been functioning in this role since October 2010. Timeline of Solicitation, Selection, and Contract Award Process Events Exhibit 1.1 shows a timeline of major events in the process of soliciting, selecting and awarding a contract for Inspector General services. The exhibit provides the date of the event as well as a brief description of the event and, when relevant, the individuals involved.
The State Controllers Office conducted an audit of the LACCD Proposition A/AA and Measure J bond expenditures. Their report, made public in August 2011, included a finding indicating irregularities in the procurement of Inspector General Services. 2 Board of Trustees Resolution BT8, adopted March 10, 2010. Harvey M. Rose Associates, LLC
Background, Objectives, Scope and Methodology As shown in Exhibit 1.1, the entire process took LACCD management a total of seven months beginning on March 10, 2010 when the Board of Trustees passed a resolution adopting a recommendation from a consultant report to create an Office of Inspector General. On April 26, 2010, approximately six weeks after the Board resolution calling for the creation of an Office of Inspector General, the District issued a Request for Proposals (RFP). Prospective bidders were provided nine days to submit written questions and 17 days to submit their proposals. With input from various District staff members, the LACCD Deputy Chancellor assembled a proposal review panel on May 14, 2010, the day after proposals were due. After reviewing the eleven proposals submitted to the District, this panel, comprised of District staff and the Districts Bond Counsel, identified four bidders as finalists to be interviewed. The following week, the Deputy Chancellor had staff conduct reference checks on two firms: (1) Thompson, Cobb, Bazilio & Associates, P.C. and (2) Policy Masters, Inc. Although staff may have conducted reference checks on the two other finalists (Ernst and Young; Deloitte), we have no evidence that occurred. That same week, on May 20, 2010 (six days after the initial proposal review) the Deputy Chancellor assembled a panel comprised of members of the Board of Trustees, the District Chief Financial Officer and an external attorney and forensic accountant to conduct interviews with the four finalist firms. On August 4, 2010, or about two and a half months after the District interviewed the four finalists, at the direction of the president of the District Board of Trustees and the incoming Chancellor, the Deputy Chancellor reassembled a slightly modified version of the interview panel to conduct a second round of interviews with the same four firms. Between the first and second round of interviews, LACCD management arranged for an outside firm, Community College Search Services, to conduct reference checks and prepare reports on the four firms invited to interview. On August 12, 2010, about one week after the second round of interviews, the Deputy Chancellor sent a memo to the incoming Chancellor stating that the selection committee had narrowed the finalists to two firms: (1) Ernst and Young and (2) Policy Masters, Inc. The Chancellor ultimately chose Policy Masters, Inc. after receiving the memo from the Deputy Chancellor and conducting his own interviews. On October 6, 2010 the Chancellor and Deputy Chancellor made a formal recommendation to the Board of Trustees to approve an agreement with Policy Masters, Inc. for Inspector General services. The Board adopted the recommendation and the contract with Policy Masters, Inc. was executed five days later on October 11, 2010.
Background, Objectives, Scope and Methodology
Exhibit 1.1 LACCD Inspector General Process for Procuring Inspector General Services Event
Capstone Report Recommendations to LACCD Board of Trustees & Interim Chancellor Board Resolution on Establishment of an Office of Inspector General RFP 10-12 Issued by LACCD Deadline for Written Questions
Date
3/10/2010
Comments
Capstone Advisory Group, LLC recommended that the District create an Office of Inspector General.
3/10/2010 4/26/2010 5/5/2010
Proposal Submission Deadline
5/13/2010
Initial Review (& Scoring) of Firms Memo from Deputy Chancellor to Members of the Interview Committee for OIG 1st Round of Interviews
5/14/2010 5/17/2010
5/20/2010
Reference Checks on Four Finalist Firms Memo from Deputy Chancellor to Members of the Interview Committee for OIG 2nd Round of Interviews
Late May Late July 2010 6/18/2010
LACCD Board of Trustees passes a resolution directing the Chancellor to establish an OIG. LACCD issues RFP 10-12 for the establishment of Inspector General services. The deadline established for proposers to submit any questions (required to be in writing) with a 2:00 PM deadline to Contracts Manager. Deadline for submission of proposals was 2:00PM and had to be a physical delivery. LACCD receives 11 proposals total. According to interviewees, only 9 were fully responsive (for OIG and whistleblower hotline services). Firms are reviewed by District staff: (1) James Watson, (2) Tom Hall, (3) Adriana Barrera; & (4) Lisalee Wells, Bond Counsel Memo from Deputy Chancellor states that another committee reviewed and rated the proposals received in response to the RFP and the proposals which are forwarded are the four most highly rated firms. Interviews conducted with (1) Thompson, Cobb, Bazilio & Associates; (2) Policy Masters, Inc.; (3) Deloitte Services, LP; (4) Ernst & Young LLP. The interview panel: (1) Jeanette Gordon, District CFO; (2) Stuart Rudnick, private attorney; (3) Mona Field; and (4) Sylvia Scott-Hayes, District trustees; and (5) Anna Lim, KPMG forensic accountant. Lisalee Wells and Lloyd Silberstein served as "resources" to the committee. LACCD utilizes Community College Search Services to conduct comprehensive reference reports on the four firms interviewed by the panel. Memo from Deputy Chancellor was for the purpose of providing Interview Committee members reports on the reference calls for OIG. Interviews conducted with same firms as first round of interviews. The committee was comprised of: (1) Stuart Rudnick, private attorney; (2) Mona Field and (3) Sylvia Scott-Hayes, District trustees, and (4) Doug Farrow, KPMG forensic accountant. Lisalee Wells and Lloyd Silberstein served as "resources" to the committee. Memo from Deputy Chancellor to incoming Chancellor re. results of 2nd Round Interviews. Chancellor was notified that the committee narrowed the finalists to (1) Ernst & Young and (2) Policy Masters, Inc. Chancellor chooses Policy Masters, Inc. as the finalist after interviews with Policy Masters, Inc. and Ernst & Young. Chancellor and Deputy Chancellor formally recommended establishing a contract with Policy Masters, Inc. for OIG services to the Board of Trustees. LACCD executes a contract with Policy Masters, Inc. with a start date of 10/7/2010 and an end date of 10/6/2015.
8/4/2010
Deputy Chancellor Memo to Incoming Chancellor on Finalists New Chancellor Chooses Awardee Board Item on OIG Contract
8/12/2010
8/2010 10/6/2010
Contract executed with Policy Masters, Inc.
10/11/2010
Source: LACCD RFP 10-12 Documentation File
Background, Objectives, Scope and Methodology Common District Practices for Solicitation of Professional Services We conducted a review of a sample of other professional services solicitations in order to obtain an understanding of common District practices for the solicitation of professional services. Specifically, we reviewed six other professional services solicitations that occurred between 2008 and 2011. Our findings from this review found the following: Consistent with the Inspector General solicitation, five of the six solicitations advertised the RFP for two weeks in the LA Daily Journal. Additionally, four of the six solicitations included documentation that the RFP was sent directly to several firms that could be considered prospective bidders. The average number of days between the posting of an RFP and the submission deadline was 22 days, which is five days longer than what was established for the Inspector General solicitation. The amount of time for other solicitations ranged from 11 days to 28 days. Four of the six other solicitation files included proposal scoring sheets of some kind. The detail of these scoring sheets varied from a basic form without instructions on how to score each category, to a detailed sheet that included a description of how each subcategory should be scored. Three of the six solicitations included a pre-proposal meeting for prospective bidders, while three did not. The District did not hold a pre-proposal meeting for the Inspector General solicitation. Two of the six other solicitations appeared to include one round of interviews. Unlike the Inspector General solicitation, none of the six other solicitations included a second round of interviews. Unlike the Inspector General solicitation, the results of the two solicitations that included interviews were consistent with District policy that indicates that: 1) scores will be assigned to proposals; and, 2) the results of interviews, if conducted, may be considered by proposal evaluators to determine each bidders final score. While interviews and interview scoring is optional, the policy does indicate that a final score will be assigned. Therefore, some process of scoring or documenting the Inspector General bidder interviews would have been beneficial in assigning final scores supporting the selection decision.
Other District Professional Services Solicitation Procedures
BuildLACCD, the District organization responsible for managing the bond-funded construction program, maintains its own set of professional service solicitation policies and procedures since they hire contractors for bond-funded activity separate from the District process. For professional services solicitations greater than $78,900 (threshold for approval by the Board of Trustees), BuildLACCDs guidelines call for formal establishment of an Evaluation Team, evaluation grading, interview and selection criteria and a process and schedule. The policies also call for documentation of all steps in the process. The solicitation of Inspector General services did not
Background, Objectives, Scope and Methodology fall under these procedures because of the independent nature of the Office of Inspector General. Specifically, the Office of Inspector General would be responsible for oversight of the BuildLACCD program. It therefore would be inappropriate for the District to have allowed BuildLACCD staff to oversee the solicitation and selection of Inspector General services.
Objectives and Scope

The scope of this independent review focused on an evaluation of LACCDs procurement processes for specialized professional services. The specific review objectives, as developed by the Office of the City Controller and agreed to by LACCD, were as follows: 1. Evaluate the process used by LACCD during 2010 for the solicitation, selection, and contract award for Inspector General services; 2. Determine how the selection process considered the experience and qualifications of proposing firms, to ensure that the selected firm met the expectations of the LACCD Board of Trustees for an Inspector General; 3. Determine whether LACCD followed its own procurement rules for the selection of Inspector General services, given the overarching objective of adhering to a fair and open process that would ensure that the most qualified proposer was selected; and, 4. Evaluate the selection process and recommended award to determine if there is evidence of any inappropriate actions that occurred with regards to the selection and award. Methods The audit team conducted this review in accordance with Government Auditing Standards, December 2011 Revision by the Comptroller General of the United States. To achieve the project objectives, the review team conducted the following tasks: 1. Conducted an entrance conference with the Districts Chancellor and General Counsel and Controllers staff to introduce ourselves, describe the review process and protocol, and to present a Request for Information. The District provided a number of key documents at this meeting. 2. The HMR team reviewed key documents pertaining to the Inspector General solicitation, including: a) the Districts written professional services procurement policies; b) Inspector General Request for Proposal (drafts and final document); c) 11 proposals received; d) completed proposal scoring sheets; e) interview questions for interviewed bidders; f) staff memorandum and file documents pertaining to the subject solicitation; g) email and other communications among and between District staff, Board members and bidders; h) bidder reference check documentation; i) the agreement with Policy Masters, Inc. for contract Inspector General services; j) invoices submitted to the District by Policy Masters, Inc. for its Inspector General services; k) bid protests filed by bidders who submitted proposals for the Inspector General contract; l) draft District conflict of interest policies; m) District and Office of Inspector General organization charts; n) Statement of Economic Interests (State of California Form 700) for all members of the District Board of Trustees and for District staff
Background, Objectives, Scope and Methodology who played a role in the Inspector General solicitation process; o) the State Controllers Office Audit of Proposition A/AA and Proposition J Bond Expenditures dated August 2011, which included an analysis of the Inspector General selection process; and, p) the District reporting responding to the State Controllers audit. 3. Interviews were conducted with members of the Districts Board of Trustees, managers and staff and external parties who either played an administrative role in the solicitation and selection process or were members of the proposal review and/or interview panels. All members of the two panels from external organizations were also interviewed, except for the representatives of one firm who did not agree to be interviewed. 4. A sample of other professional services contract solicitation documents was reviewed to assess common District practices in the preparation of Requests for Proposals, communications with vendors, review and scoring of proposals and contract awards. 5. A draft version of this report was submitted to the City Controllers Office and District management for review and comments prior to preparation and transmittal of the final report. Criteria Legal Requirements for the Procurement of Professional Services State law exempts community college districts from awarding special and professional service contracts to the lowest responsible bidder.3 This is in contrast to State requirements governing community college district contracts for commodities and repair and maintenance services greater than $50,000 and public project contracts greater than $15,000, all of which must be awarded to the lowest responsible bidder.4 Consistent with these State laws, the Districts policies and procedures do not require formal bidding or award of professional and special service contracts to the lowest responsible bidder, though competitive bidding is encouraged whenever possible. District Policies and Procedures for Solicitation of Professional Services District policies established for the selection of contractors for professional and specialized services is consistent with legal requirements and is generally adequate. Existing District polices lay out a multi-step process for the solicitation, evaluation, and selection of prospective professional services contractors. Specifically, key aspects of the policies and procedures stipulate that the process for procurement of special and professional services should include the following: 1. The RFP should include a scope of work that describes the work to be accomplished, the time frame for completion, any relevant professional or industry standards to be followed, and the deliverables- or work products- to be provided during and at the end of the engagement.
3 4
Government Code Section 53060. Public Contract Code Sections 20650-20660 Harvey M. Rose Associates, LLC
Background, Objectives, Scope and Methodology 2. The RFP should include evaluative criteria, which utilizes qualitatively objective standards for determining whether a vendor should be selected. The policies state that more subjective evaluation criteria also may be applied (such as previous clients satisfaction as reflected in reference checks). The policies also state that price (hourly rate or overall project pricing) should always be solicited in the RFP and evaluated as part of the overall proposal. The polices further state that points or weights are to be assigned to the various evaluation criteria and published in the RFP, so that both proposers and evaluators know in advance how proposals will be comparatively scored. 3. For professional services anticipated to be equal or exceed the statutory bid threshold, the District Contracts and Purchasing Section should legally advertise for proposals in the same manner as bids for Large Purchases are advertised. A notice requesting proposals will be published at least once a week for two weeks in a newspaper of general circulation published in the district. A copy of the RFP should also be posted on the LACCD website. 4. The RFP document should generally contain the following information: a. Background on the District, the requesting college (if applicable), and other information relevant to describing the District as a prospective customer; b. Explanation of why the specific services are needed; c. Instructions to proposers on how and where to submit their proposals; d. Instructions on what information to include in the proposal and how to organize the information; e. Evaluation criteria to be applied and their relative weights; f. Scope of work; g. Price proposal form, on which the proposer enters its identifying information, pricing, and signature; h. Terms and conditions of the contract or the proposed contract template to be offered to the successful proposer, if award is made; i. Miscellaneous forms and instructions. 5. Any material changes to the RFP are to be made only through written addendum distributed to all who received the original RFP. Copies of the addendum should also be posted on the LACCD website. 6. Whenever it is deemed necessary in order to facilitate the communication of information between the District and prospective proposers, a pre-proposal meeting will be convened by the District prior to the submittal deadline.
Background, Objectives, Scope and Methodology 7. Proposals received after the deadline will not be considered. Unlike sealed bids, proposals are typically not opened in a public setting. Instead, they are opened by District employees in private so as not to expose the contents of any proposal to competitors. 8. Proposals will be evaluated in accordance with the criteria and weights published in the RFP. To further the objectivity of the process, a panel of evaluators- including representatives from other District locations or from outside LACCD altogether- may be convened to score the offers independently. The proposal receiving the highest score will be deemed the offer most advantageous to the District and be recommended for contract award. 9. After the proposals have been initially evaluated, but before the final selection of a supplier, interviews may be conducted with the proposers judged to be in the competitive range based on the evaluation scores. Competitive range is defined as the field of proposers which, by virtue of the written proposal, could conceivably be awarded the contract resulting from the RFP. Information obtained through the interviews may then be considered by the proposal evaluators in assigning final scores to the proposals. a. In addition, negotiations may be conducted with proposers in the competitive range with the objective of eliciting a more advantageous offer for the District. When conducting negotiations, care must be taken to not (a) divulge the contents of one proposers offer to any other proposer, and (b) give one proposer a competitive advantage over another. 10. Any award of a contract to a successful proposer must be authorized or ratified by the Board of Trustees at a regular public meeting. 11. A copy of the original RFP, including all addenda issued thereto, and originals of all proposals, correspondence and other proposer-generated documents will be retained by the issuer of the RFP for the duration of the contract award or for a longer period if required by the Districts records retention policy. The policies also include a clause stating that, at its sole discretion and without cause, the District may, without awarding a contract, reject all proposals, cancel the RFP and either re-issue the RFP or cancel the entire solicitation. A copy of the Districts written policies and procedures for Procurement of Special and Professional Services is attached as an Appendix to this report.
1.
Process Used for Procuring Inspector General Services

District policies established for the selection of contractors for professional and specialized services are consistent with legal requirements and are generally adequate. However, gaps in District procedures in general and specifically for the solicitation of Inspector General services include: (1) documentation requirements; (2) scoring; (3) panelist guidance; and, (4) conflicts of interest that should be addressed to ensure District adherence to best practices for professional services solicitations. The process used by the Los Angeles Community College District (LACCD) for the solicitation, selection, and contract award for Inspector General services was poorly documented and did not consistently follow all District policies and common practices. The Inspector General Request for Proposals (RFP), which was prepared by the Contracts and Purchasing Manager and the Deputy Chancellor, adequately reflected the general intentions of the Board of Trustees. However, the RFP did not include a fully developed listing of professional standards, priorities and requirements for the Inspector General role. The RFP did not state that contractors should propose a full-time consultant to serve as Inspector General, a point that was not clarified until close to the proposal due date, making it unlikely for many bidders to meet this expectation. The proposal review and scoring process was poorly planned and executed and did not consistently follow District policy and common practices. The scoring sheets used for proposal evaluation did not adequately link back to the specifications in the RFP. Proposal review panelists were not provided with written or detailed verbal guidance prior to commencing evaluations. The proposal review process was poorly documented and individual scores of all raters were not aggregated to determine each firms score, as District policy suggests. Interviews with the four finalist bidders, critical to the selection of a firm, were poorly planned and executed. Interview panelists were not provided with written guidance about the role of, or District expectations for, the Inspector Generals Office. District policy states that interview results may be used to determine final proposal scores, but final proposal scores were never assigned to the bidders nor was any formal documentation of the interview panels assessment prepared. Rather, a consensus was reportedly reached after a group discussion took place with interview panelists expressing their opinions about the bidders. LACCD management did not vet potential conflicts of interest among the panelists or the bidders. Though the independence of the Office of Inspector General was critical to its success, District management took no steps to vet conflicts of interest. At the time, there was no District policy in place addressing potential bidder or bidder evaluation panelist conflicts of interest in the procurement of professional services.
Section 1: Process Used for Procurement of Inspector General Services
Finding #1: Current District Policies and Procedures for Procurement of Professional Services Contain Weaknesses
District policies established for the selection of contractors for professional and specialized services is consistent with legal requirements and is generally adequate. However, our review of existing District policies identified gaps in (1) documentation requirements; (2) scoring methods and protocols; (3) bidder evaluation guidance; and, (4) conflicts of interest. We found that existing policies do not specify the extent of documentation necessary for the proposal evaluation or interview process. Our review of this particular solicitation found that LACCDs selection hinged critically on the interview process, which was not well documented. Additionally, the Districts policy does not appear to provide sufficient guidance on the use of separate panelists for the proposal evaluation and the interview processes as was done for this particular solicitation. Finally, existing policies do not address vetting potential conflicts of interest of panelists or proposers. Our review found that LACCD management took no steps to ensure that the bidder evaluation panelists did not have conflicts of interest with bidders or that the bidders themselves didnt have any conflicts of interest that would prevent them from serving as an independent Inspector General for the District.
Finding #2: The RFP Lacked Specificity and Clarity

The process for soliciting and selecting a contractor for Inspector General services varied from common District practice for professional services in that it: (1) was a new function that the District had no prior experience with; (2) was requested by the Board of Trustees rather than a department familiar with the services to be provided that would typically prepare the scope of work statement, determine the contract duration, and provide a listing of firms for outreach; and, (3) occurred at a time when two key executive management positions were vacant: the LACCD Chancellor and the Director of Business Services. It was also unusual because, unlike most District professional services solicitations, the Board of Trustees was heavily involved in the process since they had authorized the creation of the Office of Inspector General, the new Office would be reporting directly to the Board, and there had been significant publicity pertaining to deficiencies in the Districts bond-funded capital expansion program. The RFP posted for the solicitation of Inspector General services, prepared by the Contracts and Purchasing Manager with oversight from the Deputy Chancellor, appears to have adequately reflected the general intentions of the Board of Trustees, but it did not include details on the level of staffing expected from the contractor, specific deliverables, or a listing of standards, priorities or requirements for the Inspector Generals Office. RFP not Clear on Expectation that Contractor Provide a Full-Time Designated Position to Serve as Inspector General Prior to the development of an RFP, there was some uncertainty amongst the LACCD Board of Trustees and management over whether the Office of Inspector General should be an in-house position or filled by a contractor, and whether it could be funded through general funds or bond
10
Section 1: Process Used for Procurement of Inspector General Services funds. LACCD management ultimately determined that the function should be filled by a contractor since the bond-funded program would not be permanent. District management also concluded that it would be inappropriate for BuildLACCD, the Districts bond-funded building program, to assume responsibility for the solicitation and selection of an Inspector General since it would be an oversight function of their operations. Under these circumstances, the Interim Chancellor put the Deputy Chancellor in charge of the solicitation and selection process for Inspector General services. However, a job description initially prepared for an in-house Inspector General position was used as the basis for the Scope of Work section of the RFP. Typical Duties are listed in the RFP for the Office of Inspector General (emphasis added), not for an individual. However, in interviews, members of the interview panel indicated that they were concerned with bidders that had not proposed providing a full-time individual to serve as the Inspector General for the District. The language of the RFP states that the prospective bidders could be an individual, similar to a civil service approach to hiring for a specific position. For instance, the first paragraph of the introduction of the RFP states in part that, the purpose of this RFP is to solicit offers from a qualified and interested individual or firm or joint venture (emphasis added). Further, the scope of work section lists the minimum requirements of individual or firm (emphasis added). Based on written questions submitted to the District, a number of bidders appeared to have been unclear about whether they were supposed to provide a mix of staff to support an Inspector General, fill the role with a number of individuals, or provide one individual to serve as Inspector General, and support staff. While LACCD management has asserted that by virtue of the background statement and description of the Districts requirements, the RFP communicated the Districts need for a full time operation, the need for a full time position is not explicitly stated in the RFP and, as stated under Finding #3 below, responses from the District to written questions submitted by potential bidders on this topic confirms the lack of clarity in the RFP on this matter. RFP Vague on Contractor Deliverables The critical Scope of Work section of the RFP was prepared by the Deputy Chancellor with some input from District stakeholders such as BuildLACCD representatives and the Districts general counsel, but it appears, with limited knowledge of the role and duties or standards and qualifications expected of an inspector general. Specific deliverable requirements were not specified for the Office of Inspector General in the RFP other than references to the Office being required to conduct audits and investigations and make regular reports to the Board of Trustees. The required number of audits or investigations to be conducted or reports to be made to the Board of Trustees was not provided, making it challenging for the bidders to prepare estimates of resources needed and related pricing. RFP Lacked a Full Listing of Standards, Priorities and Requirements of Inspector General Role The RFP for Inspector General services was prepared without a full listing of standards, priorities and requirements of the role of the Inspector General. While the RFP Scope of Work emphasized the conduct of investigations and audits, there was no mention of required Inspector General professional standards in areas such as independence and conflicts of interest. The
11
Section 1: Process Used for Procurement of Inspector General Services Association of Inspectors Generals Principles and Standards for Offices of Inspector General and the U.S. Government Accountability Offices Government Auditing Standards both contain specific independence and other standards applicable to Inspector General offices and public agency financial and performance audits, respectively, that would have been valuable to include in the RFP as standards to which the contractor would be expected to adhere. While the RFP listed current certification as a Certified Internal Auditor (CIA) or Certified Fraud Examiner (CFE) as a minimum requirement, there is no mention of whether it would also be required or desirable for the selected individual or firm to have a current certification as a Certified Inspector General (CIG). Further, there is no reference to the selected individual or firm adhering to the continuing professional education standard for CIGs and whether that would be funded by the District or by the selected bidder.
Finding #3: Pre-Deadline Outreach and Communication with Bidders Consistent with District Policy and Practices, but Bidders Questions Indicated RFP Not Clearly Written and District Responses Provided Inadequate Time for Bidders to Modify their Proposals
The Contracts and Purchasing Manager managed the outreach and solicitation process, consistent with District policy and practices. The Contracts and Purchasing Manager posted the RFP in the LA Daily Journal, on the LACCD website, and in journals relating to risk management and Inspector General activities. The Contracts and Purchasing Manager also directly solicited some firms, consistent with District policy and practices, but it is unclear which firms were chosen and why. While the Districts policies for procurement of special and professional services states that the requestor is to provide a list of suggested suppliers to which the RFP should be sent, interviewees have indicated that there was no designated requestor due to the unique nature of this solicitation. Written Questions from Prospective Bidders Indicate that Elements of the RFP were Not Clear LACCD management did not hold a pre-proposal meeting, nor were they required to do so under District policy. Rather, District management permitted bidders to submit written questions. While the Contracts and Purchasing Manager received written questions and posted answers, the Deputy Chancellor reviewed the questions and prepared formal responses for posting on the Districts website. An addenda to the RFP was issued May 7, 2010 with responses to the questions. While the Contracts and Purchasing Manager has indicated that the number of questions that were received were about average, the content of the written questions received by LACCD management indicate that fundamental aspects of the Inspector General role had not been fully articulated in the RFP. Some examples of these questions include: Does the LACCD currently have a whistleblower hotline or similar process in place? Is the RFP asking for services to establish the office (policies, procedures, governance) or for actually staffing?
12
Section 1: Process Used for Procurement of Inspector General Services Is the LACCD organization looking to retain an individual IG and one staff person and allow the IG to hire consultants to perform the duties of the office or is the LACCD organization looking to retain a firm to include an IG and whatever staff is required to perform the necessary duties of the office?
In its written responses to these and other questions in RFP Addenda 3, the District addressed the question of whether or not the bidders would be expected to provide a full-time consultant to serve as Inspector General. However, Addenda #3 was issued Friday, May 7, 2010, leaving only three full business days before proposals were due (on Thursday, May 13 at 2 p.m.) for bidders who had not understood the District staffing expectation to arrange for a full-time senior level consultant with sufficient experience and qualifications to serve as full-time Inspector General. Given the specialized nature of this position, it was not realistic to expect that firms would be able to secure such a person or would have one on-staff who could make that commitment with such short notice. In fact, a number of bidders submitted proposals that did not include a fulltime Inspector General. Bidders had a Short Time to Assemble Teams and Prepare their Proposals The RFP was issued April 26, 2010 and proposals were due May 13, 2010, slightly less than three weeks later. There does not appear to be any District policy or requirement about the amount of time to allow for bidders to prepare a proposal but, given the specialized and varied scope of services to be provided, this turnaround time seems short, since many bidders not only had to prepare the proposal document but had to assemble a team of subcontractors to be able to provide all required services. As mentioned in Section 1, the average amount of time from RFP posting to the submission of a proposal for a sample of six similar solicitations was 22 days, or five days longer than allotted for the Inspector General solicitation. Consistent with the Districts policies and procedures, three addenda were issued to the RFP and sent to all bidders (as acknowledged in their proposals). These addenda covered: (1) District procedures in the event of any material changes to the RFP; (2) an announcement of the date for the first round of interviews; and (3) a compilation of all questions and answers submitted by potential bidders. Additionally, consistent with the Districts policies and procedures, the RFP was advertised for a period of 10 business days.
Finding #4: Proposal Review Process Poorly Planned, Executed, and Documented
The proposal review and scoring process was poorly planned and executed and did not consistently follow District policy and common practices. Specifically, the scoring sheets used for the proposal evaluation were devised the morning of the evaluation and did not contain enough detail to adequately link back to the specifications in the RFP. Further, the proposal review panelists were not provided with written or detailed verbal guidance about how to apply the evaluation criteria prior to commencing the evaluations. Additionally, the proposal review process was poorly documented. The review and evaluation process of all eleven proposals received took place in its entirety on one day, May 14, 2010, the day after the proposals were received from the eleven bidders. The
13
Section 1: Process Used for Procurement of Inspector General Services Deputy Chancellor convened the proposal review process and chose herself and three other individuals: (1) the Executive Director of Facilities Planning and Development; (2) the Districts Bond Counsel; and, (3) the Contracts and Purchasing Manager to score the proposals and choose finalists to be evaluated by a separate panel through an interview process. Scoring Sheets were Vague and Not Accompanied by Written Guidance The scoring sheet used by each of the four panelists, which was devised by a District staff person at the request of the Deputy Chancellor two days before the review, did not contain enough detail to adequately link back to the requirements established in the RFP. Specifically, the scoring sheet was simply modeled from the broad set of evaluation criteria listed in the RFP as follows: Qualifications, experience, references (25 points); Staffing, Project Organization (20 points); Work Plan/Technical Approach (20 points); Fee (25 points); Miscellaneous, including exceptions/deviations (10 points)
Rather than expanding on these categories, the scoring sheet only included the broad categories as shown in Exhibit 1.1, which illustrates the layout of the sheet. Notably, the scoring sheet provided 10 points to bidders under the category of Miscellaneous, including exceptions/deviations. For this category it is not clear if panelists were to provide points for deviations from the RFP or take away points for deviations from the RFP. In fact, it is unclear why this is a category for scoring bidders. At least one panelist that we interviewed indicated that they were unclear how to score this section. In addition to the rows and columns shown in Exhibit 1.1, the actual score sheets had a place at the bottom for the evaluator to write in the date and their name. Although there was no designated area for reviewers to leave notes, some wrote comments in the margin or at the bottom of the page. The vague nature of the scoring sheet and evaluation criteria appears to have led to differing interpretations of the criteria. This is illustrated by wide discrepancies in scoring between the reviewers for the same firm under the same category between reviewers. For instance, one firm was scored a 13 for qualifications by one proposal reviewer while the other reviewers scored the firm between 20 and 23. In another instance, a different firm was awarded 25 points for fees by one reviewer while the others scored the firm between zero and eight points. The vague nature of the scoring sheets was compounded by a complete absence of written directions and little verbal guidance to reviewers prior to conducting the reviews. No instructions were given as to how to deal with reviewer conflicts of interest. While one reviewer recused himself from scoring one firm, the decision to do so was self-initiated and voluntary.
14
Exhibit 1.1 Proposal Evaluation Scoring Sheet Illustration

Bidder* [Pre-assigned Proposal #] 1 2 3 4 5 6 7 8 9 10 11 Source: LACCD RFP 10-12 File * Bidders names did not appear on the scoring sheet; instead, each bidder was assigned a number. Qualifications, Experience, and References (25) Staffing, Project Organization (20) Work Plan Technical Approach (20) Fee (25) Miscellaneous, Exceptions, and Deviations (10) TOTAL Points
Proposal Scores Not Aggregated The proposal scoring for Inspector General Services was conducted in an informal manner and was not fully consistent with LACCD policy. Specifically, District policy states that if a panel is convened to score proposals, the proposal receiving the highest score will be deemed the offer most advantageous to the District and be recommended for contract award. Rather, bidders were advanced as finalists based on a group discussion- after scoring was complete- in which panelists expressed their opinions about the proposals. Further, individual reviewer scores were not aggregated for the reviewers information prior to this discussion. Top Bidder Disqualified and Three Bidders Not Fully Scored Top Bidder Disqualified The firm that received the highest score on its proposal, as shown in Exhibit 1.2, was not awarded the contract nor chosen as one of the finalists to advance to the interview stage of the solicitation process. Rather, this bidder, Fu-Gen, Inc., was reportedly disqualified by the proposal review panel on the basis of being too thinly staffed. This decision was apparently made during the group discussion (after scoring had taken place), but was never documented. This decision is apparently at odds with District policy that the firm with the highest score should be deemed the
15
Section 1: Process Used for Procurement of Inspector General Services offer most advantageous to the District. Fu-Gen was not given an opportunity to address the reviewers concerns, though two other finalist firms were allowed to enhance or change their teams after their proposals were submitted. Three Bidders Not Fully Scored Three bidders were not fully scored, resulting in lower overall scores for these firms. The absence of full scoring for these firms was not based on District policy or guidance provided by the Deputy Chancellor. Rather, two reviewers determined on their own to not score a firm as shown in Exhibit 1.2. Specifically, one reviewer recused himself from scoring the firm Capstone Advisory Group, LLC because the reviewer had a professional relationship with the firm. A different reviewer did not provide scores at all for two firms that had only bid on the whistleblower hotline component of the contract. As a result of these individual decisions made by two of the reviewers, these three firms had lower overall scores, through no fault of their own since the reviewer(s) who chose not to score these proposals effectively awarded them a zero.
Exhibit 1.2 Proposal Review Scores for Eleven Bidders Sorted by Total Score
Reviewer1 Bidder 1 2 3 4 5 6 7 8 9 10 Fu-Gen Ernst & Young Deloitte TCBA2 Policy Masters Magallanes Business Controls Capstone Advisory RGL Forensics Network, Inc. #1 68 81 81 65 73 65 52 62 52 31 #2 68 79 77 75 71 72 58 50 49 -#3 83 55 51 61 55 52 52 -28 72 #4 75 70 70 75 70 65 60 65 40 35 Total 294 285 279 276 269 254 222 177 169 138 Whistleblower Hotline proposal only; not scored by one reviewer, resulting in a significant change to overall score Whistleblower Hotline proposal only; not scored by one reviewer, resulting in a significant change to overall score One reviewer recused self from scoring Disqualified Selected for interview Selected for interview Selected for interview Selected for interview Notes
11
Ethics Point
23
--
66
35
124
Source: LACCD RFP 10-12 File

1 2
Names of reviewers have been redacted. Thompson, Cobb, Bazilio, and Associates Harvey M. Rose Associates, LLC
16
Section 1: Process Used for Procurement of Inspector General Services Proposal Review Process Poorly Documented The proposal review process was poorly documented, which led to some confusion. Specifically, the Contracts and Purchasing Manager created a score sheet key with assigned numbers for each firm based on alphabetical order. This score sheet listing did not match up with the otherwise identical proposal log, which listed firms in the order that their proposal was received. This discrepancy is shown in Exhibit 1.3. The score sheet key and the proposal log looked identical, leading to confusion by the State Controllers Office in their audit pertaining to which Inspector General bidders received which scores. The State Controllers Office3 apparently was under the impression that the numbers assigned to firms on the proposal log put together by the Contracts and Purchasing Manager on May 13, 2010 (the deadline for proposal submission) were the same numbers assigned to the bidders on the scoring sheets.
Exhibit 1.3 Firm Listing (Chronological by Time Received) vs. Score Sheet Key (Alphabetical)
Assigned Number 1 2 3 4 5 6 7 8 9 10 11 Bids Received Proposal Log Created by Contracts & Purchasing Manager on 5/13/2010 Ernst & Young Deloitte Services LP Magallanes Associates International Fu-Gen Inc. Policy Masters, Inc. RGL Forensics Capstone Advisory Group, LLC The Network, Inc. Business Controls, Inc. Thompson, Cobb, Bazilio, & Associates, P.C. Ethics Point Business Controls Capstone Advisory Group, LLC Deloitte Services LP Ernst & Young Ethics Point Fu-Gen Magallanes Associates International Policy Masters, Inc. RGL Forensics The Network, Inc. Thompson, Cobb, Bazilio, & Associates, P.C.
Score Sheet
Source: LACCD RFP 10-12 File
Reference Checks Conducted on Two Firms, but Basis Not Documented Following the proposal review on May 14, 2010, the Deputy Chancellor requested staff to conduct reference checks on two of the bidders. Specifically, reference calls were made
The State Controllers Office conducted an audit of the LACCD Proposition A/AA and Measure J bond expenditures. Their report, made public in August 2011, included a finding on the procurement of Inspector General Services. Harvey M. Rose Associates, LLC
3
17
Section 1: Process Used for Procurement of Inspector General Services regarding QIU Accountancy (a partner of Policy Masters, Inc.) and Thompson, Cobb, Bazilio, & Associates, P.C. The Districts file for the Inspector General solicitation does not document why LACCD management chose to make reference checks only on these two firms and not the other two finalists (Deloitte and Ernst and Young).
Finding #5: First Round of Interviews, Critical to Selection of an Inspector General, was Poorly Planned and Executed
The first round of interviews with the four firms picked as finalists was critical to the selection of a firm, but was poorly planned and executed. Interview panelists were not provided written guidance about the role of Inspector General or District expectations for the selected contractor outside of the RFP. Further, in contrast to District policy and common practice, standardized scoring sheets were not provided or used to evaluate the interviewees/presentations. Rather, the panelists appear to have come to a consensus after a group discussion took place with panelists expressing their opinions about the bidders. The interview panel convened by the Deputy Chancellor was made up of a group of individuals different from the proposal review panel. The interview panel included two members of the Board of Trustees, the District Chief Financial Officer, and two outside representatives: (1) an attorney who had some experience representing LACCD and had previous experience as a prosecutor and, (2) a representative from KPMG, a firm which had previously been contracted by the District for audit and other services. Additionally, the Director of LACCDBuild and the Districts Bond Counsel sat in on the panel to serve as resources to the panelists, as did the Deputy Chancellor. Interview Process Poorly Planned Though interview logistics were adequately administered, the interview process was not well planned and the interview panel was not well prepared for the interviews. Specifically, the interview panelists were not provided with written guidance about the role of the Inspector General or District expectations for the new Office, outside of the original RFP. Further, the interview panelists were not provided with any information about the scores awarded by the proposal review panel. Rather, the Deputy Chancellor simply provided the interview panelists with a copy of the RFP, copies of the proposals submitted by the four finalists selected by the proposal review panel, and a very brief explanation that another committee reviewed and rated the proposals received in response to this RFP. The proposals which are being forwarded to you are of the four most highly rated firms. These materials were provided three days before the interviews took place. There was no discussion of the disqualification of top-ranked Fu-Gen. Interview Questions Not Strictly Followed The Deputy Chancellor, with some input from other District staff, prepared 15 questions, which were later narrowed to eight, to ask the bidders. However, the questions were not fully or formally used during the interviews. Rather, the format of the interviews was to allow the bidders to introduce their key members and make a presentation, followed by questions from the panel that were not restricted to those previously prepared. Interview panelists have indicated that all bidders were asked about deficiencies in their team or proposals.
18
Section 1: Process Used for Procurement of Inspector General Services While there is no District policy requiring that standardized questions be asked of all interviewed bidders, doing so is a common practice in public agency professional services solicitations. Giving every bidder an equal opportunity to demonstrate their skills and abilities and documenting this process are indicators of an objective, transparent process. Interview Process Did Not Follow District Policy or Common Practice LACCD management did not follow District policy or common practices in its conduct of the interview process. Though not explicitly required, District policy states that if interviews are conducted, the results may be used by the proposal evaluators in assigning final scores to the proposals,. The policy specifically states that,
after the proposals have been initially evaluated, but before selection of a supplier, interviews may be conducted with the proposers judged to be in the competitive rangeinformation obtained through the interviews may then be considered by the proposal evaluators in assigning final scores to the proposals.
LACCD management chose not to follow the policy option of scoring or otherwise methodically assessing each firms interview results for use in determining final bidder scores. In fact, final bidder scores, or any other formal final rankings of the bidders, were not assigned by the proposal evaluators. Rather, consistent with the proposal evaluation process used for this RFP, a group discussion took place after the interviews. The Deputy Chancellor then summarized in a written report to the incoming Chancellor what she felt was the consensus of this group discussion. Additionally, in contrast with District practice, standardized scoring sheets were not provided or used to evaluate the presentations made by the four finalist firms. Further, the process was not consistent with District practices and policy whereby the panel chosen to review proposals typically also determines final bidder scores. In contrast, the Deputy Chancellor chose different panelists for the proposal review process and for the interview process. The District states that this was done at the request of a member of the Board of Trustees and to ensure an objective process. While the composition of the interview panel was reasonable for this solicitation, a negative impact of using two panels for proposal review and interviews was that the interview panel did not have information on how the four finalists compared to all other bidders relative to the evaluation criteria in the Request for Proposals. Such information would have provided areas to focus on in the interviews relative to standardized evaluation criteria. Further, the interview results were not provided to the proposal evaluators to determine final scores, as District policy suggests. Interview Results Not Well Documented The results of the initial interview process were not well documented. As previously mentioned, there were no scoring sheets or other written summary, provided by or sent to panelists, of the interview process. Additionally, there is no direct record of bidder responses to the prepared questions or panelist assessments. The only documentation available of the initial interview process is an emailed written report sent to the incoming Chancellor by the Deputy Chancellor on June 28, 2010, about five weeks after the interviews were held. The email briefly summarized
19
Section 1: Process Used for Procurement of Inspector General Services the results of the interviews. Specifically, the email stated, in part, the selection committee interviewed the four firms and identified one firm as being far and away the best prepared for the interview and with the best working knowledge for the Office of the Inspector General. This written document was not provided to the interview panelists, though all panelists interviewed agreed with the primary conclusion in the report that Policy Masters, Inc. had performed best in the interviews.
Reference Checks Conducted on Four Finalist Firms by Outside Contractor

After the first round of interviews on May 20, 2010 the Deputy Chancellor was prepared to quickly wrap up the selection process and provide a recommendation to the Board of Trustees. Two of the four finalist firms, Policy Masters, Inc. and Thomspon, Cobb, Bazilio, and Associates had been contacted to schedule follow-up interviews and District staff conducted reference checks on some of the team members for these two bidders. However, after a discussion between one of the members of the Board of Trustees who sat on the interview panel and the incoming Chancellor about the process, the process was deliberately slowed down, to allow the incoming Chancellor to be involved in the selection. The member of the Board of Trustees directed the Deputy Chancellor to conduct background reference checks on the four finalist firms and their lead persons. Background Reference Checks Well Documented, but Had Little Effect on Selection of a Finalist In late May and June of 2010 the firm Community College Search Services, which was under contract with the District for executive search services typically used when hiring a new college president or manager, was requested by the District to conduct background reference checks on the four finalist firms and their lead persons. The background reference checks on the lead persons consisted primarily of 14 standard questions asked for each firms lead person, with a summary of responses received from the references listed by the bidders. The questions were devised internally by LACCD staff and provided to the search firm. The background reference checks also included bullet point lists of strengths and weaknesses. However, the reference checks did not include background checks on any of the firms or lead persons to identify any criminal records or potential conflicts of interest. The District paid $5,200 for this service. Although the background reference checks had little impact on the outcome of the selection process, as the references were all generally positive, they were probably the best documented step of the Inspector General procurement process. However, it should be noted that the names of the references were redacted in copies of these reports provided to our review team, so their authenticity cannot be verified. The background reference checks did not result in any bidders being disqualified or any significant change in the relative rankings from the initial interviews. Further, Community College Search Services was not provided specific direction by District management, beyond calling listed references, to conduct a meaningful reference or background check on the bidding firms as a whole. The Deputy Chancellor noted in a June 28, 2010 email to the incoming Chancellor that the information that Community College Search Services was receiving on the firms was very limited.
20
Section 1: Process Used for Procurement of Inspector General Services Reference Checks Did Not Include a Detailed Background Investigation There does not appear to be a District policy on conducting such background checks, but given the sensitive nature of the Office of Inspector General and independence standards generally applied on inspector generals, the need for the office to be independent of real and perceived impairments, checks for conflicts of interest would have been prudent.
Results of Second Round of Interviews Consistent with First Round

On August 4, 2010 the Deputy Chancellor convened a second round of interviews with the same four finalist firms. The planning, conduct, and results of the second round of interviews were similar to the first round. Several members of the panel indicated to us that the goal of the second round of interviews was to have a more informal discussion with the lead person and essential staff from each bidder. The interview panelists were similar to the first round including: the same two members of the Board of Trustees and the outside attorney; but two changes were made to the panel: (1) the Chief Financial Officer was unable to attend the second round of interviews and (2) KPMG sent a different representative because the original representative was unavailable. The prepared questions for the second round of interviews included four basic openended questions. As with the first round, there was no scoring or other direct documentation of the panels findings and, reportedly, not all questions were systematically asked of all bidders. Two Firms Made Changes to their Proposed Teams after the First Round of Interviews At least two firms came to the second round of interviews with significant changes to their proposals. Though the staff changes were accepted by the panel, it is unclear if the firms first asked for permission from LACCD officials if they could make the changes, were advised to do so by District officials, or if they took it upon themselves to make the changes without consulting LACCD. Specifically, Policy Masters, Inc. added an investigator to join their proposal and Ernst and Young changed their lead person who would serve as the Inspector General. Both firms were chosen as the two finalists to be recommended to the Chancellor. Neither Thompson, Cobb, Bazilio & Associates, PC nor Deloitte Services LP made changes to their proposals after the first round of interviews, nor were they recommended to the Chancellor as finalists.
Final Selection of an Inspector General Made by the Chancellor

On August 12, 2010, approximately one week after the second round of interviews, the Deputy Chancellor sent the Chancellor a memorandum summarizing the interviews and the panels recommendations. Specifically, the memo put forward Ernst & Young and Policy Masters, Inc. for further consideration by the Chancellor with a strong stated preference for Policy Masters. In the closing paragraph the Deputy Chancellor states:
Members [of the interview panel] expressed the view that their recommendation following the first round of interviews remained the same. If nothing else, committee members felt their previous recommendation had been affirmed and expressed the opinion that while they were forwarding the names of two firms, Policy Masters was far and ahead the best firm and that Ernst & Young was being forwarded as a distant second choice.
21
Section 1: Process Used for Procurement of Inspector General Services The Chancellor chose to recommend Policy Masters Inc. after interviewing both firms. On October 6, 2010 the Chancellor and Deputy Chancellor formally requested that the Board of Trustees approve the establishment of a contract with Policy Masters, Inc. for Inspector General Services.
Finding #6: LACCD Did Not Actively Vet Conflicts of Interest

Given the importance of inspector general offices being independent, LACCD did a poor job of vetting conflicts of interest from the bidder evaluation panelists and the bidders. District management took no steps to ensure that panelists had any compromising relationships with any of the bidders. Similarly, no action was taken to check if any of the bidders had compromising relationships with members of the Board of Trustees, District personnel or contractors who they would have to oversee if they were to be awarded the contract for Inspector General services. For example, the president of Policy Masters, Inc. had previously worked for Gateway Engineering, a company under contract with the District for bond-funded construction management services. This information was disclosed in her resume in the Policy Masters proposal, but the District did not require any statements from this or other bidders disclosing their current or past relationships with such firms. While there is currently no policy in place for vetting conflicts of interest in the procurement of specialized and professional services, District management should consider implementing such policies as a prudent control measure.
Conclusions
The process used by the Los Angeles Community College District (LACCD) for the solicitation, selection, and contract award for Inspector General services was poorly documented and did not consistently follow all District policies and common practices. The Request for Proposals (RFP) posted for the solicitation of Inspector General services, which was prepared by the Districts Contracts and Purchasing Manager and the Deputy Chancellor, appears to adequately reflect the general intentions of the Board of Trustees. However, the RFP did not contain a listing of priorities or requirements for the Inspector General role, specific deliverables to be provided by the contractor, the District expectation that the contractor should provide a full-time position to service as Inspector General, or required professional standards with which the contractor would be expected to comply. The proposal review and scoring process was poorly planned, executed, and documented and did not consistently follow District policy and common practices. The first round of interviews with the four firms picked as finalists was critical to the selection of a firm, but was poorly planned, executed, and documented. Interview panel members were not provided written guidance about the role of Inspector General or District expectations for the selected contractor outside of the RFP. Further, in contrast to District policy and common practice, standardized scoring sheets were not provided or used to evaluate the interviewees/presentations. LACCD management did a poor job of vetting potential conflicts of interest among the panelists as well as the bidders. District management took no steps to vet conflicts of interest and there is no District policy in place that addresses conflicts of interest in the procurement of professional services.
22
Recommendations
The LACCD Board of Trustees should: 1.1. Direct staff to revise the Districts Business Operations Policy and Procedures, PP-04-08, to include more specific guidance on: a. The evaluation of proposals, including specific guidance on how to develop a score sheet linked to standards and requirements specified in the RFP. The revised guidance should include a template to further assist District staff. b. The conduct of interviews, including specific guidance on how to develop an interview score sheet. The revised guidance should include a template to further assist District staff. c. How to properly document the solicitation and selection of special and professional services. d. What steps should be taken to ensure that no evaluators or bidders have any conflicts of interest that may impair their ability to serve the District.
Costs and Benefits

The costs associated with these recommendations would include minimal staff time devoted to drafting revisions to the Districts policy on the procurement of special and professional services and briefing the Board of Trustees on such changes. The benefits of these recommendations would include a more transparent and consistent method for the District to procure special and professional services. The benefits would also include a reduced likelihood of the District procuring services from individuals or firms with perceived or actual conflicts of interest.
23
2.
Assessment of Bidder Qualifications and Experience

The Request for Proposals (RFP) for Inspector General services stated that the primary services to be conducted would be audits and investigations of the Districts bond-funded capital program and operation of a whistleblower hotline. The RFP required that bidders present their experience and qualifications in these areas along with demonstrated knowledge of audit and investigation techniques, procedures, pertinent laws and regulations and principles of facility planning and development and construction management. None of the eleven bidders had experience and qualifications in all areas identified in the RFP. Though bidders with more relevant experience and qualifications were generally ranked higher by the proposal review and scoring panel, the Districts process for assessing qualifications and experience was not methodical or consistent. For example, the bidder awarded the highest score by the proposal review panel had no experience in auditing, one of the two key functions to be performed by the Office of Inspector General. One of the four bidders selected as a finalist to be interviewed, and ultimately awarded the contract, did not have any investigation experience listed in their initial proposal. Only one of the four finalist firms had performance audit experience and qualifications detailed in their proposals. The bidder awarded the contract did not meet all of the key criteria in the RFP. The firm did not have: Inspector General experience; investigation experience, (at least initially); and performance or operational auditing experience. The bidder did have public agency facility construction management and financial and compliance auditing experience, and added an investigator to their team before the second interviews. The Districts interview panel considered the winning bidders two presentations superior to the other bidders because the bidder was well prepared, had construction business experience, demonstrated knowledge about the District, and made a commitment to serving full-time as Inspector General. These characteristics ultimately outweighed the bidders deficiencies in the experience and qualifications required in the RFP. This reflects a change in the Districts definition of the role and related qualifications needed for an Inspector General that unfortunately occurred after eleven bids had been received based on the requirements in the original RFP. As a result, the District did not obtain the full benefits of competitive bidding in their solicitation for an Inspector General contractor.
This review was conducted to determine how the experience and qualifications of the eleven bidders for Inspector General services were considered to ensure that the expectations of the Board of Trustees for the services to be performed were met by the selected bidder. The Board of Trustees expectations were codified in a resolution adopted March 10, 20101 calling for the establishment of an Inspector General to be responsible for, objective ongoing review of
1
Resolution No. BT8 Harvey M. Rose Associates, LLC
24
Section 2: Assessment of Bidder Qualifications and Experience performance, financial integrity and legal compliance of the Districts bond program. The resolution also stated that the Board of Trustees recognized a need for ongoing monitoring of the management of bond-funded projects and appropriateness of related expenditures and that existing audits did not provide suggestions for proactive measures and course corrections. Based on this resolution and related Board of Trustees discussion, District staff prepared a Request for Proposals (RFP) for the Establishment of the Office of Inspector General for the Los Angeles Community College Districts Capital Funded Program and solicited proposals from qualified contractors, as discussed in Section 1 of this report. The RFP included language from the Board of Trustees resolution and stated that the purpose of the RFP was to solicit offers from a qualified and interested individual or firm or joint venture capable of performing the following Scope of Work: Plans, directs and conducts investigations and audits designed to promote accountability to the public and to insure economy, efficiency, effectiveness and integrity of the LACCDs bond funded capital program. (emphasis added) The description of typical duties and required knowledge and abilities noted in the RFP, investigations and audits, are clearly identified as the core duties of the Office of Inspector General and key areas in which the bidders should have experience and qualifications. Since the subject of the audits and investigations would be the Districts bond-funded capital program, knowledge and experience in public facilities planning and construction practices as well as fraud and waste investigation and detection techniques was also required. In conjunction with establishment of the Office of Inspector General, the RFP also required the establishment of a program to reduce the risk of fraud, waste and abuse at the District, including operation of an anonymous whistleblower hotline service. Though the term performance audit is not explicitly identified in the summary Scope of Work statement above, the importance of performance auditing experience and knowledge is embedded in the Scope of Work statement since the purpose of performance audits is to ensure economy, efficiency and effectiveness of an agencys operations. In fact, in the Typical Duties section of the RFP, performance audits are specifically identified, along with operational, financial, and compliance audits. The full list of Typical Duties for the Office of Inspector General is presented in Exhibit 2-1.
25
Section 2: Assessment of Bidder Qualifications and Experience
Exhibit 2-1 Typical Duties Identified in the RFP for the Office of Inspector General
Planning, directing and conducting complex operational, compliance, financial and performance audits of all aspects of the Districts bond program. Conduct investigations into whistleblower allegations of misconduct, conflicts of interest, improper financial and/or business practices, fraud, collusion, waste and/or abuse in the bond program. Develop and implement policies, procedures, objective and priorities for investigations and audits and ensures that investigations and audits are handled with tact, discretion and professionalism. Prepare corrective action plans based on results of audits and investigations. Make regular reports to the Board of Trustees on objectives, critical problems, audit schedule and related matters. Provide technical advice and recommend corrective internal controls, policies and procedures to bond program management, the Chancellor and Board of Trustees. Oversee the operation of a whistleblower hotline for reporting concerns related to the bond program. Evaluate computer software and systems applicable to the investigation unit and recommend system enhancements
Audits and investigations are also emphasized in the Minimum Bidder Knowledge and Ability Requirements identified in the RFP, as shown in Exhibit 2-2.
26
Exhibit 2-2 Minimum Bidder Knowledge and Ability Requirements Identified in the RFP for the Office of Inspector General
Proper techniques and procedures for conducting investigations and audits including advanced data collection and analysis techniques. Laws, rules, regulations, policies and legal precedents pertinent to investigation of allegations of misconduct, improper financial activity and fraud, waste or abuse. Business principles, methods and practices related to facilities planning and development (property acquisition, facilities planning, etc.). Principles and polices of financial management, accounting and auditing as they relate to construction of public building projects. Computer applications and hardware to support and enhance investigations. Current certification as a Certified Internal Auditor (CIA), Certified Fraud Examiner (CFE) or equivalent was listed as desirable. Minimum of five years of full-time paid professional-level audit and/or investigations experience with an investigations or auditing firm or public entity. Two years of the required experience must have been in a recent administrative or supervisory position. A work history which includes experience related to facilities planning, project development and construction management was listed as highly desirable.
To demonstrate their ability to perform the duties described above and their knowledge and abilities in the required areas, bidders were asked to provide information demonstrating their competence in the services to be provided (i.e., audits and investigations, public facility planning and construction, hotline services, etc.), the nature and relevance of similar work recently performed, references, a work plan and schedule detailing the bidders approach, and a budget and fee schedule for the first year of operation. Bidders were asked to provide indicators of their companys size and stability including date of companys founding, number of employees, any pending office closures or mergers, and any debarments, suspensions or other declarations of inability to contract by federal, state or local public agencies. Finally, bidders were also asked to provide any exceptions or deviations from the Districts standard Professional Services Agreement, which was attached to the RFP. Lacking from the Typical Duties, the Minimum Bidder Knowledge and Ability Requirements sections or any mention in the RFP was a specific description of required deliverables, staffing requirements for the new Office, or a clear distinction between staffing and deliverable
27
Section 2: Assessment of Bidder Qualifications and Experience requirements related to establishing the Office, as compared to what would be required over time once the Office was established and in full operation.
Finding #7: Bidders Experience and Qualifications were not Consistently or Methodically Assessed
Review of the eleven proposals shows that the bidders had varying levels of experience and qualifications in the core areas of audits, investigations, fraud detection, construction management and hotline operations. Only one of the bidders appeared to have experience and qualifications in all of the core required areas identified in the RFP. The proposal review and scoring panel appears to have considered bidder qualifications in their deliberations and, generally, awarded higher scores to the bidders with more experience and qualifications in the core areas of auditing and investigations and knowledge of the construction business. However, relevant experience and qualifications were not methodically assessed by the proposal review and scoring panel and were considered even less so, by the interview panels. In their scoring, the proposal review panel overlooked key areas needed to perform Inspector General services for some bidders. For example, the top ranked firm in the panels initial review of proposals only had experience in investigations, but none in audits, one of the two primary functions to be performed by the Office of Inspector General, according to the RFP. Though it received the top score, this bidder was precluded from advancing as a finalist to interviews because the panel decided the firm was too small to provide the services needed by the District. The bidder that was advanced for interviews and ultimately awarded the Inspector General contract did not have experience or qualifications in investigations, the other key function to be performed by the Office of Inspector General. Of the four bidders the proposal review panel selected as finalists to be interviewed, all had at least some audit experience, though only one reported experience in performance auditing, as required in the RFP. Only one of the four bidders advanced to interviews had experience operating a whistleblower hotline. The other three indicated they would work with the District to coordinate services with a separately selected third party whistleblower hotline firm. Only two bidders had actual Inspector General experience2 and most had not worked on audits or investigations pertaining to educational institution bond-funded construction programs. Exhibit 2-3 presents a summary of the experience and qualifications of all of the bidders in key areas required in the RFP. It should be noted that the two lowest scored bidders submitted bids on the whistleblower hotline component of the RFP only. The four bidders that were advanced by the proposal review panel as finalists to be interviewed are shown in the shaded rows of the following two tables.
The validity of one of the firms claims of being an inspector general for the Inglewood School District could not be verified. There is no reference to such an office on the Districts website or publically available documents and Board of Trustees minutes from March 25, 2009 explicitly state that such an office does not exist. Harvey M. Rose Associates, LLC
28
Exhibit23:ExperienceandQualificationsof11BiddersforInspectorGeneral
FuGen Ernst&Young* Deloitte* ThompsonCobbet.al* PolicyMasters* Magallanes BusinessControls Capstone RGL Network EthicsPt. Audits Investigations Construction Whistleblower Proposal &Facilities Hotline Score 294 285 279 276 269 254 222 177 169 138 124
Source: Proposals submitted to LACCD for Inspector General Services, May 13, 2010 *Firm selected by LACCD Proposal Review Panel to advance to interviews.
While most bidders had some audit experience, it was most commonly in the areas of financial audits. Most bidders did not have operational, performance or compliance audit experience, though all four types of audits were required by the RFP. Some bidders claimed to have experience in all four types of audits but provided no examples to back up such claims or indications that they knew the difference between audit types.
Finding #8: Inspector General Contract Awarded Based on Qualifications other than those Emphasized in the RFP
The proposal submitted by the bidder ultimately awarded the Inspector General contract, Policy Masters Inc., provided team experience in financial and compliance auditing but did not provide qualifications or experience in investigations or performance or operational auditing. According to members of the interview panel, these deficiencies were outweighed by the bidders presentation made at the first and second interviews. Positive attributes exhibited by this bidder, according to the interview panelists, included: being well prepared; familiarity with the District and its bond-funded program; the firm presidents experience and familiarity with construction programs from her years working at the Los Angeles Unified School District; and commitment to being full-time on the job.
29
Section 2: Assessment of Bidder Qualifications and Experience An assessment of the qualifications of the bidder awarded the contract relative to other key criteria in the RFP is as follows: The RFP called for descriptions of bidding firms stability and previous similar work and accomplishments: Policy Masters had not conducted Inspector General or other similar work previously because the firm only incorporated in 2010. Similarly, the firm could not report indicators of its stability such as number of years in business, any planned mergers or office closures or debarments, as requested in the RFP, since it was a new company and none of these factors applied. Policy Masters did indicate it had never been debarred or suspended, was not in bankruptcy and had no pending litigation. It indicated its team members, but not the firm, had performed related services for years and provided information about the related work and accomplishments of its team members from their previous positions or clients. The RFP called for client references for the firm: Policy Masters could not provide firm references because it had not served clients as a firm but it did provide team member references from previous positions and clients. The RFP defined the role of the Office of Inspector General as planning and directing investigations and audits and required a minimum of five years fulltime paid professional-level audit and/or investigations experience with an investigations or auditing firm or public entity. The Policy Masters team, and the proposed team lead, did not present direct experience planning and directing investigations and audits in its proposal. The prime bidders subcontracting accounting firm did report experience planning and directing financial and compliance audits, but not performance or operational audits. The team did not have experience planning and directing investigations in its original proposal, though an investigator was added to the team later, between the first and second round of interviews. The RFP called for knowledge and ability in business principles related to facilities planning and development and procurement and contracting policies: Policy Masters Incs president reported extensive experience in these areas. The RFP called for knowledge and ability in financial accounting and management as it relates to public building projects: the Policy Masters, Inc. team had financial accounting experience. The RFP called for current certification as a Certified Internal Auditor, Certified Fraud Examiner or equivalent as desirable: the Policy Masters, Inc. team had no such certifications.
30
Conclusions
In conclusion, the experience and qualifications specified in the RFP appear to have been considered, though not consistently or methodically, by the proposal review team and, to a lesser degree, by the interview panel. Review of the qualifications and experience presented in all the bidders proposals showed that those most closely matching the requirements of the RFP were generally scored better by the proposal review team. However, the panels top-ranked firm had no audit experience at all and one of the four bidders selected as a finalist, Policy Masters, Inc. did not report investigation or performance audit experience in their initial proposal and had no track record as a firm since the company had only recently incorporated (though this bidders team possessed experience in construction, public school facility planning and financial and compliance auditing). The deficiencies in qualifications and experience of Policy Masters, Inc., the bidder ultimately awarded the contract, were apparently of less concern to the interview panel, who was impressed by: (1) the firms presentation at the interview sessions; (2) its understanding of the Districts objectives for the Office of Inspector General; (3) its knowledge of public facility construction; and (4) its presidents commitment to serving full-time as Inspector General. Though the results of the interview process were not documented, as discussed in Section 2 of this report, these characteristics apparently outweighed any concerns about deficiencies in the qualifications and experience of the firm. The interview panel appears to have redefined the role of the Inspector Generals Office through the solicitation process compared to how it was defined in the RFP. While it may have ended up with the contractor that best met its needs out of the bidder pool, it did not obtain the full benefits of competitive bidding, since the actual criteria used for selection varied from that in the RFP.
31
LOS ANGELES COMMUNITY COLLEGE DISTRICT BUSINESS OPERA TIONS POLICY AND PROCEDURES
Date Updated: August 19, 2008 . Supersedes: Month Day, Year Origina//ssue Date: June 17, 2008
PP-04-08
PROCUREMENT OF SPECIAL AND PROFESSIONAL SERVICES
I.
OVERVIEW
"Special services and advice," also referred to as professional services, are described in Government Code section 53060. Special services include financial, auditing, legal, architectural/engineering, insurance brokerage, payroll issuance and preparation, and other administrative matters where the practitioner is specially trained and experienced to handle such matters. The key to determining a "special service" is whether the professional's independent knowledge, expert training, discretion and judgment is needed to render complete and adequate service under the contract. When procuring a professional service in combination with a personal service, it must be determined which service is the primary part of the overall services requested in order to direct whether an Invitation for Bid (IFB) or a Request For Proposals (RFP) is the legally appropriate means. of procurement. ,\
'xiJ
Special services/professional services procurements are deemed to be exempt from the legal requirement of formal bidding. Nonetheless, the District has adopted as its business practice to procure such services through a competitive process whenever possible. Specifically, the following situations may arise when it is advantageous for the District to competitively procure the services: (a) multiple suppliers of the service compete in the local market and may be interested in doing business with the District, and (b) the District could reasonably derive a material benefit, such as. significant cost savings or superior service, from inviting competitive offers, and (c) the service is expected to cost $5,000 or more. At the discretion of the procuring location, services costing l.ess than $5,000 may be obtained utilizing quotes. Professional services are to be procured through a RFP or Request For Qualifications (RFQ) process. Depending on the anticipated dollar amount for the service, the transaction is either handled by the college, College Procurement Specialist or District Purchasing and Contracts Section in accordance with the preceding section 04-01 and the division of authority set forth in Administrative Regulation B-19. Unlike the RFP, the RFQ omits the requirement for submittal of a price proposal and limits the District's evaluation of dffers to an assessment of the relative qualifications of the competing firms to perform the services. Generally, RFQs are used only for engineering and architectural services. Accordingly, this method of procurement is addressed in the Facilities Section.
Date Updated: August 19, 2008 Supersedes: Month Day, Year Original ISsue Date: June 17, 2008
As with the sealed bidding process used for Large Purchases, the purpose of the RFP is to create a fair, equitable and objective process for qualified suppliers of a particular service to compete for theDistrict's business on a "blind bid" basis (that is, each proposer not knowing what its competitors are offering), with the goal of selecting the most advantageous offer to the District, price and other relevant factors considered.
II. A.
PROCESS
Request for Proposals (RFP)
1. Specifications (Scope of Work) - The requestor will initiate the RFP process by preparing a set of service specifications and deliverables. Often referred to as the Scope or Statement of Work, the specifications describe the work to be accomplished, the time frame for completion, any relevant professional or industry standards to be followed, and the deliverables-or work products-to be provided during and at the end of the engagement. If possible, a requestor should also provide with the Scope of Work a list of suggested suppliers to which the RFP should be sent. 2. Evaluation Criteria - Accompanying the Scope of Work should be recommendations for evaluating the proposals submitted. Unlike invitations for bids, in which award is made to the lowest responsible and responsive bidder, the RFP evaluates qualitative factors in the proposal in addition to price. However, evaluation criteria should still utilize qualitatively objective standards for determining whether a vendor should be selected. Some examples of objective but qualitative criteria are the following: a) Number of years of experience in the service field being procured; b) Number of years of experience serving clientele similar to the District; c) Experience and education of the personnel proposed for the project; d) Possession of a specific license or certification in order to provide the service.
''wJ
Other more subjective evaluation criteria also may be applied, such as previous clients' satisfaction as reflected in reference checks, and the quality of the technical approach proposed for accomplishing the work objectives. Finally, price (hourly rate or overall project pricing) should always be solicited in the RFP and evaluated as part of the overall proposal.
Date Updated: August 19, 2008 Supersedes: Month Day, Year Origina//ssue Date: June 17, 2008
To enhance the objectivity ofthe evaluation process, points or weights are to be assigned to the various evaluation criteria and published in the RFP, so that both proposers and evaluators know in advance how proposals will be comparatively scored. .
3. Notice - For professional services anticipated to equal or exceed the statutory bid threshold, the District Contracts and Purchasing Section will legally advertise for proposals in the same manner as bids for Large Purchases are advertised. A notice requesting proposals will be published at least once a week for two weeks in a newspaper of . general circulation published in the district, if no such paper, a newspaper of general circulation in the county. The notice must include, at a minimum, the scope of work to be performed and when proposals are due. If a pre-proposal meeting is scheduled, the date, time and place of the meeting also must be included in the notice. If a pre-proposal meeting is mandatory to all proposers, this information should also be included in the notice. Notices also may be advertised in trade journals or other publications targeted at suppliers of the services being procured. For services expected to cost less than the statutory bid threshold, legal advertising of the notice is optional. In addition to advertising, copies of the notice or RFP may be sent directly to suppliers that have requested to receive such solicitations or which have been identified by the requestor or District as potential proposers. Finally, a copy of the RFP also will be posted on the LACCD Website. 4. RFP Document - The RFP document will generally contain the following information: Background on the District, the requesting college (if applicable), and other information relevant to describing the District as a prospective customer; Explanation of why the specified services are needed; Instructions to proposers on how and where to submit their proposals; Instructions on what information to include in the proposal and how to organize the information; Evaluation criteria to be applied and their relative weights; Scope of Work;
/\
t~J
Price Proposal Form, on which the proposer enters its identifying information, pricing and signature; Terms and conditions of the contract or the proposed contract template to be offered to the successful proposer, if award is made; Miscellaneous forms and instructions.
'\
5. Changes to the RFP - Any material changes to the RFP will be made only through written addenda distributed to all who received the original RFP. Distribution may be made by first-class US Mail, email, facsimile or personal delivery to addresses or fax numbers furnished by the RFP recipients. A copy of the addendum also will be posted on the LACCD website http://www.laccd.edu/business services/ along with the original RFP. Issuance of an addendum will be provided far enough in advance of the proposal-submittal deadline to provide a reasonable period of time for proposers to receive the addendum and modify their offers accordingly; otherwise, the submittal deadline must be extended. Any addendum should be sent to all interested proposers and posted on the LACCD website. 6. Pre-proposal Meeting - Whenever it is deemed necessary in order to facilitate the communication of information between the District and prospective proposers, a pre-proposal meeting will be convened by the District prior to the submittal deadline. The date, time and place of the meeting will be included in any RFP advertisement placed or notice issued. The pre-proposal meeting may be conducted at the site where the work is to be performed and conducted as a "job-walk" for prospective proposers. Any questions and answers of a material nature from the pre-proposal meeting, together with any material changes to the RFP announced at the meeting by District representatives, will be documented in an addendum sent to all recipients of the original RFP and posted on the LACCD website. 7. Proposal Submission - Proposals, contained in a sealed wrapper, will be accepted up to the noticed proposal submittal date and time. Time and date of receipt will be mechanically stamped or handwritten accompanied by the signature of the recipient, on the wrapper. Proposals received after the deadline will not be considered. If required in the RFP instructions, an electronic file of the proposal (e.g., in MS Word or PDF format) must be submitted on diskette or CD-ROM together with the paper proposal.
Proposals may be delivered in person by the proposer, through the US Mail or via a private third-party carrier (e.g., overnight delivery or courier service), but in no event will a delay caused by such delivery method be allowed by the District as a valid reason for acceptance of a late proposal. Only if the District is the direct cause of late receipt of a proposal will such offer be accepted. Faxed proposals will not be accepted. If permitted in the RFP instructions, as an alternative to the submittal of a paper proposal, proposals may be prepared in MS Word or PDF format and submitted electronically as an attachment to an email message sent to a secure mail box designated by the District. Unlike sealed bids, proposals are typically not opened in a public setting. Instead, they are opened by District employees in private so as not to expose the contents of any proposal to competitors. This serves to preserve the District's ability to negotiate changes favorable to LACCD with one or more proposers without their knowledge of what their competitors may be offering. 8. Proposal Evaluation - Proposals will be evaluated in accordance with the criteria and weights published in the RFP. To further the objectivity of the process, a panel of evaluators-including representatives from other District locations or from outside LACeD altogether-may be convened to score the offers independently. The proposal receiving the highest score will be deemed the offer most advantageous to the District and be recommended for contract award. 9. Interviews/Negotiation - After the proposals have been initially evaluated, but before the final selection of a supplier, interviews may . be conducted with the proposers judged to be in the competitive range based on the evaluation scores. "Competitive range" is defined as the field of proposers which, by virtue of their written proposal, could conceivably be awarded the contract resulting from the RFP. Information obtained through the interviews may then be considered by the proposal evaluators in assigning final scores to the proposals. In addition, negotiations may be conducted with proposers in the competitive range with the objective of eliciting a more advantageous offer for the District. When conducting negotiations, care must be taken to not (a) divulge the contents of one proposer's offer to any other proposer, and (b) give one proposer a competitive advantage over another. 10. Proposal Rejection - At its sole discretion and without cause, the District may, without awarding a contract, reject all proposals, cancel the RFP and either re-issue the RFP or cancel the entire solicitation.
!\
.'.-..:111 ,,;.
B.
Award - Any award of a contract to a successful proposer must be authorized or ratified by the Board of Trustees at a regular public meeting. All proposers will be notified in writing of the recommendation of award or the actual award. Any vendor who had submitted a proposal shall have five working days from the date of notice to protest an award made to another vendor. Recordkeeping - A copy of the original RFP, including all addenda issued thereto, and originals of all proposals, correspondence and other proposer-generated documents will be retained by the issuer of the RFP for the duration of the contract award or for a longer period if required by the District's records retention policy.
C.
'\
The District is subject to the California Public Records Act set forth in Government Code sections 6250-6270. As such, proposals, correspondence and other proposer-generated documents are subject to public inspection, unless otherwise excluded by the Public Records Act. For this reason, proposers are to ' be instructed to identify in their proposals information they deem to be proprietary in nature. Identification of information as "proprietary" by the proposer does not guarantee non-disclosure of the information if public disclosure is otherwise permitted or required by law.
III. LEGAL AUTHORITY AND CITATIONS

Public Contract Code sections 20650-20660 Education Code sections 81550 et seq, 81641, 81651 Government Code section 53060 Board Rules 7100 and 7102.16 04-02 - Types of Transactions G'overnment Code sections 6250-6270

Wendy Greuel Controller: An Equal Employment Opportunity - Affirmative Action Employer

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Wendy Greuel Controller: An Equal Employment Opportunity - Affirmative Action Employer

Hochgeladen von

Copyright:

Verfügbare Formate

WENDY GREUEL CONTROLLER

Dr. Daniel LaVista, Chancellor July 18, 2012 Page 2

Harvey M. Rose Associates, LLC

public sector management consulting

Fred Brousseau Project Manager

Appendix: LACCD Professional Services Procurement Policies

A summary of individual findings in this report is as follows.

The RFP Lacked Specificity and Clarity

Proposal Review Process Poorly Planned, Executed, and Documented

Harvey M. Rose Associates, LLC

LACCD Did Not Actively Vet Conflicts of Interest

Bidders Experience and Qualifications were not Consistently or Methodically Assessed

Harvey M. Rose Associates, LLC

Report Review by District

Harvey M. Rose Associates, LLC

Background, Objectives, Scope and Methodology

Harvey M. Rose Associates, LLC

Background, Objectives, Scope and Methodology

3/10/2010 4/26/2010 5/5/2010

Proposal Submission Deadline

Late May Late July 2010 6/18/2010

Contract executed with Policy Masters, Inc.

Source: LACCD RFP 10-12 Documentation File

Harvey M. Rose Associates, LLC

Other District Professional Services Solicitation Procedures

Objectives and Scope

Harvey M. Rose Associates, LLC

Harvey M. Rose Associates, LLC

Process Used for Procuring Inspector General Services

Section 1: Process Used for Procurement of Inspector General Services

Finding #2: The RFP Lacked Specificity and Clarity

Harvey M. Rose Associates, LLC

Section 1: Process Used for Procurement of Inspector General Services

Exhibit 1.1 Proposal Evaluation Scoring Sheet Illustration

Source: LACCD RFP 10-12 File

Source: LACCD RFP 10-12 File

Reference Checks Conducted on Four Finalist Firms by Outside Contractor

Results of Second Round of Interviews Consistent with First Round

Final Selection of an Inspector General Made by the Chancellor

Harvey M. Rose Associates, LLC

Finding #6: LACCD Did Not Actively Vet Conflicts of Interest

Section 1: Process Used for Procurement of Inspector General Services

Costs and Benefits

Harvey M. Rose Associates, LLC

Assessment of Bidder Qualifications and Experience

Resolution No. BT8 Harvey M. Rose Associates, LLC

Harvey M. Rose Associates, LLC

Section 2: Assessment of Bidder Qualifications and Experience

Harvey M. Rose Associates, LLC

Section 2: Assessment of Bidder Qualifications and Experience

Section 2: Assessment of Bidder Qualifications and Experience

Harvey M. Rose Associates, LLC

Harvey M. Rose Associates, LLC

Section 2: Assessment of Bidder Qualifications and Experience

Harvey M. Rose Associates, LLC

PROCUREMENT OF SPECIAL AND PROFESSIONAL SERVICES

III. LEGAL AUTHORITY AND CITATIONS

Das könnte Ihnen auch gefallen