Tax is a highly complex area of knowledge work that is critical for business. Large enterprises, governmental bodies, entrepreneurs, and small businesses alike rely on tax professionals to navigate their tax compliance, strategy, and planning needs; each of which play a crucial role in shaping downstream business decisions. Tax professionals must have a detailed understanding of regulation and compliance requirements, and an ability to apply that knowledge to each unique scenario, whether it’s for their company or a diverse range of clients. They also need an understanding of how to interact with tax authorities to apply the legislation appropriately.

Large language models (LLMs), and AI-native solutions built on top of them, have the potential to radically transform traditional tax workflows. The challenge, however, is that general-purpose LLMs are not well-suited for complex tasks that require specialized tax expertise. Together with PwC, we aim to change that.

PwC’s tax professionals span 150+ regions internationally and advise businesses on how to navigate global tax systems across a wide range of practice areas including: corporate tax, indirect tax, international tax, M&A tax, transfer pricing, and many more. The largest projects require hundreds of tax experts to collaborate in understanding complex multinational tax issues, analyzing thousands of documents, and sharing detailed findings with stakeholders.

In alliance with PwC and OpenAI we have developed the Tax AI Assistant: a suite of custom models and LLM systems that assist tax professionals in solving a wide range of real-world tax problems. The Tax AI Assistant uniquely combines curated tax datasets with Harvey’s LLM expertise, and is fine-tuned with the feedback from PwC’s tax experts.

Data: Our research team worked with PwC’s tax experts across 10+ jurisdictions to collect, categorize, and analyze hundreds of thousands of authoritative tax documents. These include legislation, case law, revenue guidance, PwC tax materials, and more.

LLM System: We developed a novel language model program that answers complex tax queries and provides accurate citations to source material. This system is powered by in-house models and semantic search technology, and then additionally tuned on tax datasets and expert-annotated examples for further domain specialization.

Expert Feedback: A key aspect of Tax AI Assistant is its ability to incorporate human expertise and preferences to improve quality. First, expert responses and guidance were used to optimize the data set and model to solve problems in the way that a tax specialist would. Then, a feedback loop was applied, whereby tax experts worked with Harvey engineers to further improve the quality of outputs. This human-guided approach helps to maintain accuracy and output quality, so that the model meets the needs of the user base and learning from individual user prompts can be switched off, making the model safer.

As a result of these unique attributes, the Tax AI Assistant is able to rapidly produce high-quality research output with citations for tax professionals, ready for their experienced human review.

Tax AI Assistant is now used by many of PwC’s tax teams around the globe as a daily augmentation to their work. To evaluate its performance, PwC tax experts curated a set of realistic, simulated tax queries across a variety of tasks and practice areas. Then, they conducted a blind side-by-side human preference evaluation against other AI solutions and annotated their judgements for both for overall quality and more specific dimensions of answer correctness, citation accuracy, and organization.

In aggregate, Tax AI Assistant was preferred 91% of the time by experts. For a number of practice areas, including Indirect Tax, Innovation & Capital Incentives, International Tax & Treasury, Risk, and Stamp Duties, Tax AI Assistant attained 100% preference.

We are excited to continue working with PwC to develop custom models to solve the world’s most challenging tax work. To add to the success we have seen in numerous existing regions such as the UK, Japan, and Australia, we will continue our human-guided model training process to expand our specialized models to more than 30 tax jurisdictions around the world.

Credits: Calvin Qi [1], Aravind Srinivasan [1], Adham Elarabawy [1], Oliver Brady [1], Matt Zerweck [1], Tanvi Kapur [1], Emilie McConnachie [1], Michael Gauer [2], Julio Pereyra [1], Niko Grupen [1], Bivek Sharma [2], Gabriel Pereyra [1]

1 Harvey

2 PwC