● Breaking

CIO Urges Human Oversight In AI Workflow At Summit

CIO Urges Human Oversight In AI Workflow At Summit - human oversight
"After about five minutes, they said, ‘Well, that’s Claude’s fault,’" Brady said, referring to the tool developed by Anthropic. Photo: Vizito Visitor Management System/Pexels

The need for human oversight in artificial intelligence initiatives dominated discussions at the recent Forbes CIO Summit. Ryan Brady, the chief information officer at Select Health, argued that as adoption accelerates, keeping people present in the workflow is essential.

Human Oversight in AI Development

Brady shared a personal story from a recent meeting. He challenged a presenter on specific elements of a slide deck, questioning their accuracy. His persistence revealed that the presentation had been generated by an AI assistant, and the presenter had missed several errors. When confronted, the presenter blamed the software.

“After about five minutes, they said, ‘Well, that’s Claude’s fault,'” Brady said, referring to the tool developed by Anthropic.

This incident highlights a risk in relying on autonomous tools. While AI can generate content quickly, it does not always catch mistakes or understand context. The assistant provided the output, but a human must review it before it is presented to an audience.

Brady also discussed the evolution of technology skills. He noted that people no longer calculate math on paper because smartphones have automated that task. Similarly, AI will likely replace manual steps in many jobs.

He suggested a shift in how people learn. Instead of focusing on rote memorization or basic problem-solving, education should prioritize critical thinking. Brady proposed that schools keep technology out of the classroom for younger students, introducing laptops and AI tools only in high school. This approach would allow students to build a strong foundation before relying on machines for answers.

Read Also: U.S. Cheapest for iPhone 18 Pro Series

Select Health has already begun implementing these principles through a partnership with Salesforce. The organization uses a new platform to handle provider credentialing, a process that verifies a healthcare professional’s licenses and education.

The old system took six months to complete. The new Salesforce-powered tool, which includes AI resources, reduced that timeframe to just six days. This reduction was not achieved by removing people from the process, but by streamlining the workflow to reduce manual effort.

Brady conducted an in-person visit to a contact call center to understand the work his team was doing. He observed staff members working with multiple screens and dozens of windows open. While they successfully communicated, the physical effort of working through the antiquated software was cumbersome. This observation drove the decision to upgrade the system.

Brady warned against “vibe coding” for sensitive enterprise applications. He described the practice of using AI to write code without deep technical knowledge as insufficient for regulated industries. Select Health ensured the new platform was HIPAA-compliant and met other regulatory needs, working with Salesforce to handle the technical details.

Mark Wanish, CIO and chief architect at Salesforce Regulated Industries, emphasized that AI requires context to function correctly. He noted that without this context, AI cannot accurately reason through business operations. The system must understand the relationship between data points, such as associating listings under the same doctor’s name across different locations.

“Now we’ve got a single pane of glass,” Brady said.

Read Also: Nvidia CEO resigns, China AI chip market rattled

Human in the Flow

“I hate the phrase ‘human in the loop.’ … I love the phrase, ‘human in the flow,'” he said, as he described using the technology in tandem with his team. For example, Brady said he used AI to extract action items from pictures taken of whiteboards where the team collaborated on ideas.

Wanish noted that deploying AI at Select Health required adding context to the organization’s data for the AI to interpret it effectively.

He said that at the moment, there is no single all-encompassing platform for how AI is used across business processes, with everything being multimodal, multi-agent and spanning the gamut of operations. That requires humans in the mix to ensure the AI orchestration runs with confidence.

“Agents don’t reason. AI doesn’t reason off of data. It does off of context,” Wanish said. That context helps the model better associate listings under the same doctor’s name at three different locations as actually belonging to one doctor. He said agentic operating systems need access to operational knowledge in order to align with business processes.

“That’s how you actually start getting the results and not just a lot of very fast intelligence,” Wanish said.