Commit `51c534c9b9`

51c534c9b90c02cc71b0bbd52a53a96f6e5e170d

parent: e350c6b81d

Unsigned

cmc <hello@cleberg.net> · 2025-11-24 03:03 UTC

publish new post: it-audit-ai

Layout: unified · split

content/blog/2025-11-23-it-audit-ai.org added +300

		@@ -0,0 +1,300 @@
	1	#+date: <2025-11-23 17:43:57>
	2	#+title: Practical Uses for AI in IT Audit
	3	#+description: Learn how I'm currently using AI in my role in Technology Assurance (i.e., IT Audit) at KPMG.
	4	#+slug: it-audit-ai
	5
	6	* My AI Toolkit
	7
	8	As others in the Big 4 will probably tell you, AI is a strategic tool that
	9	allows for a far more efficient and impact audit. However, the key is
	10	understanding when to use it, how to use it, and how to translate the output
	11	from your AI into actions.
	12
	13	This post will walk you through how I personally leverage AI in my role.
	14
	15	Let's start by stepping through the tools I have at my disposal.
	16
	17	** Gemini Enterprise (formerly Agentspace)
	18
	19	First and foremost is [[https://cloud.google.com/gemini-enterprise][Gemini Enterprise]], which has a large context window, is
	20	connected to the Enterprise's internal documentation and websites, and allows
	21	for specialized use of custom agents, [[https://notebooklm.google/][NotebookLM]], and more.
	22
	23	I use Gemini for a handful of specific use-cases, notably:
	24
	25	1. NotebookLM notebooks for team-based usage. For example, you can build a
	26	notebook for a specific client (using public and private information about
	27	that client) and then chat with the data in the notebook, generate an audio
	28	overview, and save documents, such as mind maps or study guides, for future
	29	use.
	30	- I love this because it provides a "run once and save" functionality, where
	31	I can generate things like an audio overview or document, save it, and then
	32	my team can reference it without re-running the prompt or even needing to
	33	open the notebook, if they download it.
	34	2. Custom Agents for customized personas that benefit from the private
	35	information that Gemini has access to, such as internal SharePoint sites.
	36	- For example, I have a custom agent designed to operate as an IT Audit
	37	Assistant that can help you ask questions that can be answered by the
	38	company's news, documentation sites, and more (e.g., "I'm testing logical
	39	access for AWS and want to know if anyone has created a solution or
	40	guidance for XYZ.").
	41
	42	** Microsoft Copilot
	43
	44	Likely the most common AI in use at company's today, due to its integration with
	45	every Microsoft product, Copilot offers the best AI solution for productivity.
	46	However, it's usage is likely also the most volatile due to the large gap in
	47	skill sets between new-hires and senior employees.
	48
	49	For associates and other early-career employees, I suggest that they use Copilot
	50	to perform two primary tasks:
	51
	52	1. Check their tone, such as making an email or messages more professionalism,
	53	softer, or perhaps to analyze the conversation up to that point to understand
	54	how to respond better. This is something that /should/ come naturally to
	55	senior employees, but is often a learning curve for associates.
	56	2. For written documentation, such as guides or presentations, use Copilot and
	57	provide it with ample information on the goal of the documentation in order
	58	for Copilot to suggest an outline, gaps in the writer's thinking, or areas
	59	for further research before writing the document. The key here is to use
	60	Copilot to improve your thinking and not to replace it. It shouldn't generate
	61	the content for you, it should remove the tedium of creating boilerplate
	62	structures or to design the structure of a document.
	63
	64	** Clara AI
	65
	66	[[https://kpmg.com/us/en/capabilities-services/audit-services/kpmg-clara.html][Clara AI]] is KPMG's audit-based AI platform, known as the "KPMG Clara Smart Audit
	67	Platform". The major benefit to Clara AI is that it lives within a sandboxed
	68	environment, running on white-labelled models, fine-tuned on firm-specific
	69	methodologies, and built directly into KPMG's audit tools.
	70
	71	As noted in KPMG's Clara AI page linked above, Clara AI is built directly into
	72	the Clara audit platform in order to provide systematic risk assessments, drive
	73	consistent and efficient audits by providing all audit teams with relevant
	74	information for their audit, and connects to other tools (such as KPMG Client
	75	Collaboration, where evidence is requested and provided).
	76
	77	I absolutely love this tool, as it solves some of the bigger problems for
	78	auditors at large firms:
	79
	80	1. Connecting younger staff to the firm's guidance and regulatory material,
	81	lowering the barrier to auditors trying to learn which pieces of regulation
	82	apply and when they apply.
	83	2. As you can imagine, the audit process for a firm of KPMG's size can be
	84	complex at times and changes depending on the client, the client's industry,
	85	the type of engagement, and more. Clara AI helps break down each of these
	86	challenges and provide detailed, accurate guidance for auditors to learn
	87	about their specific use-case without getting lost in irrelevant data.
	88
	89	Clara AI does a lot more than I've written about here and I know teams are using
	90	these features beyond what my teams do in Tech Assurance. You can read about
	91	more features, such as automation, dynamic dashboards, and more on the product
	92	page: [[https://kpmg.com/us/en/capabilities-services/audit-services/kpmg-clara.html][Clara AI]].
	93
	94	* Use Cases for Each Tool
	95
	96	While this section won't be exhaustive, it provides a few basic examples of what
	97	my teams do at KPMG with AI, as well as some suggestions for how other auditors
	98	can utilize AI at their own companies.
	99
	100	** Gemini
	101	*** Client and Project-Based Agents or Notebooks
	102
	103	As I mentioned above, one great use-case is to build notebooks for your client
	104	team or even for internal teams, such as a team that's formed to increase
	105	automation within the audit practice.
	106
	107	I build notebooks for these teams, upload as many relevant documents and links
	108	as I can, invite my team members, and then save useful artifacts within the
	109	notebook so that my team members don't need to recreate the wheel in order to
	110	gain access to that same information.
	111
	112	*** Technical Research
	113
	114	Through purely anecdotal experiences, I have found Gemini to be superior to my
	115	other AI tools when it comes to technical topics. It provides more accurate
	116	information and better context for my specific role.
	117
	118	This has replaced the experience of walking team members through topics, such as
	119	Snowflake architecture, AWS logical security, or SQL permissions.
	120
	121	In addition, it helps me and others who are more experience dive deeper into
	122	subjects that we historically did not have the time or knowledge to research.
	123	For example, Gemini recently helped me use my knowledge of similar systems to
	124	understand Kafka's architecture and risks as they related to my client's
	125	specific implementation of Kafka.
	126
	127	*** Data Analysis
	128
	129	As I mentioned above, Gemini seems to excel with technical tasks. I've been
	130	using this to enhance the data analysis we perform.
	131
	132	For example, Gemini is able to help write queries (e.g., SQL or AWS commands) to
	133	extract the information we need to test a control. While we have lots of
	134	internal documentation for popular platforms, there are always situations where
	135	you need data from a system and don't have any guidance to extract it.
	136
	137	For users with data, Gemini is able to provide a more natural language querying
	138	process where the user can simply describe what analysis they need, or even
	139	generate a dashboard for the user. I find this most helpful in identifying
	140	relevant metrics or analytics and then have our teams use their skills to build
	141	out Alteryx workflows and Power BI dashboards to implement what Gemini has
	142	helped them investigate.
	143
	144	Lastly, data cleaning is wonderful in Gemini. Gemini is able to help structure,
	145	clean and deduplicate data much faster than humans can and seems to have a lower
	146	potential for errors than if someone manually combed through the data trying to
	147	correct it.
	148
	149	** Microsoft Copilot
	150	*** Meeting Transcripts
	151
	152	One of the built-in features of Copilot in Teams is [[https://support.microsoft.com/en-us/office/view-live-transcription-in-microsoft-teams-meetings-dc1a8f23-2e20-4684-885e-2152e06a4a8b][transcription]]. This is a
	153	killer feature for auditors, as one of the biggest pain points for an audit (on
	154	both sides) is having to meet multiple times to rehash the same topic or send
	155	constant follow-ups throughout the year to re-confirm facts.
	156
	157	With a transcript, we can spend one meeting going fully through all questions
	158	and simply refer back to the transcript whenever we have a question.
	159
	160	If you also save the video recording, you can also save the time of requesting
	161	evidence later, as you already have a recording where you walked through the
	162	evidence you needed.
	163
	164	This saves:
	165
	166	1. An absolute ton of time on both the audit and client sides, in terms of
	167	questions and follow-ups.
	168	2. Even more time by not needing the ask the client to take a million
	169	screenshots. Instead, the auditor can take screenshots from the recording
	170	after the fact. No more stopping every 5 seconds in a meeting waiting for
	171	someone to take a screenshot or sending a request for screenshots after the
	172	fact.
	173
	174	*** Meeting Facilitator
	175
	176	Another killer feature of Copilot is the [[https://learn.microsoft.com/en-us/microsoftteams/facilitator-teams][Facilitator]]. Facilitator takes the
	177	agenda in the meeting invite (or determines the agenda throughout the meeting),
	178	tracks progress of the discussion against the agenda, takes notes, and tracks
	179	to-do items with assignments for users to take action after the end of the
	180	meeting.
	181
	182	*** Microsoft Office Integrations
	183
	184	I talked about the Microsoft Office integrations above, so I won't dive deep in
	185	this section.
	186
	187	For auditors, some useful examples are:
	188
	189	- Generating boilerplate for an audience you're not familiar with writing for
	190	(e.g., engineers vs. CFOs). Note that you shouldn't generate the content,
	191	you should only let Copilot guide you to the proper format and then use your
	192	knowledge to write the content.
	193	- Checking your tone, especially if Copilot has access to historical chats, in
	194	order to help you respond professionally and appropriately.
	195	- In Excel or Word, helping you analyze your data.
	196
	197	I'm not a big fan of generating content with Copilot, so I don't have any
	198	examples for that. Perhaps I am naive, but I still think we should be honing our
	199	interpersonal skills and the content of our communications with others should
	200	still be written by us, not AI.
	201
	202	*** Managing Emails, Chats, and More
	203
	204	I almost titled this section "Manager Mode", since I'm feeling a different level
	205	of intensity from the sheer number of emails, Teams messages, tags in channels,
	206	and more that I experience now as a Manager.
	207
	208	One small but effective use for Copilot is to check at the end of the day if
	209	you've missed anything that day, such as responding to an email or message,
	210	actions items assigned to you in meetings, or important announcements that,
	211	while not directed at you individually, may require action from you.
	212
	213	I currently do not have access to Copilot Studio, but I'd love to build a custom
	214	agent around this idea if I could.
	215
	216	** Clara AI
	217	*** Firm & Methodology Guidance
	218
	219	For an auditor who has to deal with SOX, SOC 1-3, AUP, and more engagements, you
	220	can imagine the sheer number of potential pitfalls there are when working
	221	through an engagement.
	222
	223	Being able to have Clara AI break down the regulatory requirements for your
	224	specific situation is amazing.
	225
	226	For example, how do you risk assess Client ABC, who has unique IT systems and
	227	business processes, and operates in the oil & gas industry? What if they are
	228	also impacted by the US's tariffs? Even further, what if they experienced a data
	229	breach in the prior year and have experienced changes in numerous key roles?
	230
	231	Clara AI is able to handle of this and more, guiding team members to the
	232	relevant guidance and assisting them in the audit process.
	233
	234	*** Workflow Assistance
	235
	236	On top of the various regulations and internal guidance frameworks, auditors at
	237	the Big 4 also have to learn how to use their firm's specific audit platforms.
	238
	239	KPMG built the KPMG Clara Workflow (KCw) for audits and SOC reports, which
	240	requires teams to fill out the audit file for their engagement, answer all
	241	questions that appear, perform their testing, and abide by the required items.
	242
	243	Clara AI helps by connecting auditors to the internal guidance that supports
	244	each section of the audit platform so that they can complete their engagement in
	245	compliance with their requirements.
	246
	247	* A Day with AI
	248
	249	Before I finish, I wanted to provide an example of how an auditor might utilize
	250	AI throughout their day. You can probably summarize this already from the
	251	information provided above, but I wanted to lay out in plain terms here at the
	252	end:
	253
	254	- Strategic planning in the morning with any AI. Review regulatory news,
	255	summarize key decisions made by the government, industry leaders, and more.
	256	- If you have a lot of emails and messages, use Copilot to summarize actions
	257	items and make a prioritized plan of action.
	258	- As you test or review items in the audit file, use Clara AI to identify gaps
	259	in approach.
	260	- During meetings, enable transcriptions and Facilitator. Use these tools so
	261	that people attending the meeting don't need to take detailed notes. Instead,
	262	each person can be fully invested in the topics they're discussing.
	263	- Use Copilot to summarize and draft outlines for emails to the team and/or
	264	client.
	265
	266	** Conclusion
	267	*** What /Can/ AI Do?
	268
	269	AI /can/ increase efficiency, consistency, and audit quality, but there are some
	270	big assumptions when we state that. We first need to make sure we, as auditors,
	271	are using the tools themselves effectively. For an auditor, a general AI chat
	272	won't be very useful. Instead, we should be training and using focused models
	273	that solve specific purposes. The model will perform better if it's trained for
	274	specific purpose and isn't expected to know everything that's ever existed.
	275
	276	But what happens when we /do/ use AI effectively? Well, at that point, we are
	277	able to elevate ourselves and automate the manual and tedious tasks that have
	278	taken up time historically.
	279
	280	Staff who may have been asked to spends hours cleaning data from the client
	281	before can now spend a shorter length of time reviewing the data output instead
	282	and then using that additional time to enhance their critical thinking. If you
	283	are used to performing these manual tasks, think about what your seniors or
	284	managers do with your results. Can you take on those tasks instead and elevate
	285	yourself into those roles?
	286
	287	*** The Role of the Human in an AI World
	288
	289	Last, but not least, I want to highly stress the importance of human in the
	290	loop (HITL) concept. What we do as auditors is strategically important, whether
	291	you're an internal or external auditor, and cannot be replaced by the current
	292	landscape of AI tools. Therefore, when we're using AI, we need to make sure we
	293	validate everything coming out the tool. At no point can the AI take
	294	responsibility, so we need to take responsibility for our actions when we use
	295	AI.
	296
	297	However, when we do all of this, we are able to elevate the role of the auditor
	298	to one of judgment, critical thinking, validation, and client relationship
	299	management. This means faster, higher quality, and more impactful audits
	300	throughout the industry.

Commit 51c534c9b9

Commit `51c534c9b9`