Background
I recently undertook a high-stakes project for a small Australian regional bank: managing the selection of a new core banking system. I've been around the banking and tech sectors for a long time—most recently serving as an Assistant Secretary in Federal Treasury responsible for the technical standards underpinning the Consumer Data Right—so I was fairly confident I had the background and now-how to tackle the challenge.
Why Speed Mattered
The bank had an extremely tight window to choose and implement a new core banking solution. Any delay in selection risked throwing the entire project off track. We had just six to seven weeks to go from a blank sheet of paper to a final recommendation to the Board—over the Christmas, New Year’s, and January holiday period, no less. Not exactly an ideal timetable.
Undercutting the Competition
A major consulting firm had already recommended replacing the bank’s core system and identified a shortlist of potential vendors. Naturally, they expected to manage the final selection. However, at Shepherd Thomas, we knew that one knowledgeable resource, backed by AI, could produce a sharper result faster than a large, traditional consulting team. This approach allowed us to beat their fee quote and still deliver effectively.
Crafting the RFP with AI
Ordinarily, gathering requirements would involve time-consuming workshops with subject matter experts. Given the constraints, we took a different approach. The previous consultant’s work did at least confirm the bank’s needs were mostly “vanilla,” which helped. By carefully prompting an AI model (in this case, Anthropic’s Claude Sonnet 3.5) and feeding it relevant reference documents, we generated a solid draft of the RFP within a single day. Another couple of days were spent cross-checking for “hallucinations” and confirming requirements with the client.
With the RFP in hand, we went straight to the vendors. Happily, all but one of the shortlisted vendors submitted proposals despite the tight holiday timeline.
Prepping the AI for Proposal Analysis
While the vendors were busy, we tested different Large Language Models to see which could best handle the soon-to-arrive flood of proposal documents. This was before the release of “reasoning” models like DeepSeek or ChatGPT-o1. Even so, the older generation of LLMs provided rapid analysis of the proposals—especially helpful given that some of them were awash in marketing speak and irrelevant detail.
The Proposal “Shred”
We had just a weekend plus two working days after the RFP closed to deliver our initial recommendation. Our LLM-based approach let us quickly “shred” each of the seven vendor proposals down to the essential facts, providing a clear side-by-side analysis and an initial recommendation for the Board.
Spotlighting Ambiguities
When the Board wanted clarifications, our AI tools helped again. One especially powerful tactic was asking the AI to highlight vague or ambiguous sections in each proposal. This acted like a metaphorical blacklight, revealing potential trouble spots hidden behind walls of text.
Hallucinations and Self-Checks
Of course, the AI wasn’t flawless. It sometimes confidently asserted complete untruths—the dreaded “hallucinations.” We tackled this by having the AI check its own work: after it generated a report, we fed that report back and asked it to validate every statement against the source documents. A round or two of self-checking could work wonders, although pushing it too far eventually caused the AI to go off the rails.
Final Observations
In the end, we demonstrated that AI, even in an early form, dramatically increases a consultant’s productivity. Boutique firms can compete with (or outperform) major consulting houses on speed and cost, and possibly even on quality. That said, a human in the loop remains indispensable. AI struggles to steer itself in complex, human-driven projects. It’s also prone to missing its own mistakes. But when paired with a genuine expert, the potential of these “centaur” consulting methods is huge.
AI’s trajectory is sure to evolve, and the major players in consulting will need to adapt. But in this particular project—under massive time pressure and with limited resources—the combination of human expertise and AI muscle got the job done faster, cheaper, and arguably better than the old-school approach.
留言