In their latest report, Orange County grand jurors say the county government does a good job overall at overseeing its computer services, while certain areas need more focus, such as better tracking of project results and creating critical backup systems in the event of a disaster.
The county’s information technology staff “are delivering services that are highly rated by users, and IT costs, organization, and governance are very comparable to other California counties with similar population size,” the panel concludes.
“The Grand Jury is concerned, however, that the County has not demonstrated a current capability to recover critical computing resources in the event of a major disaster.”
In particular, they point to “the lack of a backup datacenter for Sheriff’s Department functions that are vital to the Department’s law enforcement and public protection responsibilities.”
(Click here to read the grand jury's report.)
The panel also criticized a lack of analysis by the central IT office of whether projects meet their business goals.
“Information Technology and project management best practices generally recommend that post implementation reviews or audits of projects include evaluation of the achievement of project goals,” their report states.
“Some agencies/departments do conduct their own limited post-implementation reviews. However, none of the agencies the Grand Jury studied have formal reviews or reports on achievement of project business case or business goal achievement,” they continued.
“The Grand Jury believes that this is an area where proper IT governance is lacking,” the report states, adding that the central IT office should be evaluating whether projects achieve their business goals.
The report is the most recent evaluation of the county’s information technology, or IT, services, which have long been the subject of controversy.
Among their findings, grand jurors disputed Voice of OC’s reporting on cost overruns, saying many of increases were instead extensions of maintenance and ongoing service contracts.
“IT project cost overruns do not plague the County. However, policies and procedures are not in place in the IT governance structure to adequately measure and evaluate achievement of benefits and goals of IT projects over their entire project life cycle,” the panel wrote.
Grand jurors did point to three contracts over a year-long period that experienced overruns ranging from 10 percent to 29 percent. In all, the overruns amounted to to $635,000, according to the panel.
Additionally, grand jurors said county officials need to require user satisfaction surveys across the board and adopt a more modern approach to developing software systems.
County officials weren’t able to show that departments conducted “regular, comprehensive survey programs,” according to the report.
The central IT office should require such surveys, set up a format for departments to follow and publish the results, grand jurors wrote.
As for creating software systems, grand jurors want the county to reduce its reliance on an older approach and instead adopt the more collaborative “agile” approach.
The county currently uses “waterfall”-style development, where each part of the project is finished before starting on the next phase.
It attempts to be predictive, rather than adaptive, and know all requirements before design is started, all design issues before programming is started,” the panel wrote.
“In reality, problems and issues encountered in subsequent phases often cause changes to previous phases, and that is one major source of cost overruns on projects.”
The “agile” approach, meanwhile, emphasizes regular collaboration between end-users and software developers.
“A collaborative and iterative approach is a creative process wherein each learns from the other incrementally as they build a solution,” the panel wrote, recommending that it be used where appropriate.
Information technology is crucial to the functioning of county government, whether it’s being used to make phone calls, send emails, track crime hotspots, ensure accurate property taxes, or a myriad of other uses.
While technology costs generally decline over time, the county has experienced steady increases to its IT spending.
They’ve risen more than 30 percent over the past three years, from $137 million to $150 million to $180 million this fiscal year.
Grand jurors also credited Orange County officials with creating annual reports on total IT costs, which other counties like San Diego and San Bernardino have yet to do.
Orange County’s IT spending per resident, and as a percentage of the overall budget, was also found to be in line with other counties of its size.
Troubles at the county IT department date back years. An earlier chief technology officer, Reza Khayyami, was charged in 2006 with taking bribes from a software contractor.
Many of the county’s issues were supposed to be fixed by a 2006 task force, the IT Working Group. But that panel waited years to begin reviewing the very problem that prompted its creation — the delivery of a $6-million mainframe system before county supervisors had a chance to approve it.
Then in 2009, a county audit found that $45 million in no-bid technology contracts were awarded over a four-year period.
And last year, after a software firm’s costs ballooned from $8 million to $16 million, the county filed a federal lawsuit claiming fraud.
Even after the county paid millions more than the original amount, the software had errors that “could have threatened the confidentiality and accuracy of the information for hundreds of thousands of taxpayers,” the county claims.
Controversy also arose over two large-scale IT contracts that were awarded last year.
In the first, the county awarded a $74 million contract to SAIC, which in the prior year had admitted to one of the largest municipal fraud schemes in U.S. history.
Supervisor Nguyen directly accused SAIC of underbidding its county contract.
“My concern is still that I truly believe there was a low-balling on the part of SAIC,” Nguyen said before her colleagues approved the deal. “I fear this contract is heading for a train wreck.”
Other supervisors disagreed, saying the fraud case in New York City was in no way emblematic of the company.
The second major IT contract controversy last year revolved around an $11-million price hike by Xerox after it had promised to stick to its final offer.
“I don’t like what I’m seeing, and I’m not going to get nickel-and-dimed,” Nelson said of the Xerox contract last summer. “We want honesty from the get-go, not hide the pea under the walnut.”
Under state law, county leaders must respond to the grand jury report within 90 days.