Getting better results

In our last lesson, we dealt with data privacy and security—what you absolutely need to protect and how to do it. So now, let's talk about the question that's probably stopped you from using AI for data analysis in the first place: How can I trust the results?

Today's lesson is about the delegation diligence loop, specifically building confidence in AI's analytical capabilities for your specific work by systematically testing it against data you already understand. By doing this, you can better understand how AI will support your specific circumstances.

The process starts with delegation. Here's how this works. First, identify a specific analytical task you do regularly that you want to delegate to AI. Find past data where you already completed the analysis, and then work with AI to reproduce what you did, evaluating what works and what doesn't. Refine your approach and test again. If AI can match your known results, you know how to use it and trust it for similar future tasks. And if not, you've learned that this task is something you shouldn't delegate.

So let me show you what this looks like in practice, and then we'll talk through what to do if you're not that data savvy to begin with.

Meet Rio, the program director at Valley Veterans Services. Every quarter, he analyzes program attendance alongside employment outcomes, calculating participation rates, tracking monthly changes, and determining whether attendance correlates with job placement success. This analysis consistently takes him hours.

Considering delegation, Rio knows he wants to continue using the results of this analysis to improve his program. He wants to interpret the results himself, but he could do without the data cleaning and formula mayhem he usually finds himself in to do the actual analysis. So in order to test whether AI is appropriate in this scenario, he's going to evaluate it using last quarter's data. He knows exactly what this data showed after he analyzed it without AI, and he has the raw messy data from before he started. This is his test case.

Rio uploads the data and starts to work with AI, using description and discernment to perform his analysis. Only each time AI responds, Rio is going to check the results against what he knows to be true and jot down potential gaps in AI's reasoning. Sometimes additional description helps AI get the outcome he's looking for. In these cases, Rio knows he has to include that kind of information for future data analysis tasks. Other times, Rio might find legitimate capability gaps. This is the delegation diligence loop in action. Rio's diligence to evaluate the model's capabilities can change what he chooses to delegate to AI in the future.

His first attempt might look like: "I'm sharing attendance data and employment outcome data from our job training program last quarter. Please analyze the participation patterns across the three months and graph the correlations between attendance levels and employment success. I'm particularly interested in understanding whether consistent attendance predicts better job placement outcomes."

AI responds with a summary. But rather than assuming this is fact, Rio checks this against his records and notes what's good and what's not. AI correctly identified the correlation between program attendance and job placement, but it missed a critical insight around the combined housing assistance and job placement program. So Rio refines his description, asking AI to try again but pay special attention to the program type. This time AI catches its mistake. So Rio notes that for future quarters, he'll need to specifically request the AI to consider the program type when performing its analysis.

Then he does something harder: "Can you also look at this based on when participants enrolled?" AI responds as Rio observes that despite not knowing the enrollment data, AI could help extract it. He makes a note to cross-reference these results later on.

By going through this process, Rio has systematically validated what AI can and can't do for his quarterly reporting. He's learned that with the right description, AI can accurately reproduce the analysis he used to do manually. But he's also identified clear limitations and areas for follow-up. AI needs enrollment dates in the data to do cohort analysis; otherwise it'll try to infer them, which he doesn't want. And most importantly, Rio now has a tested approach that he can confidently use with this quarter's data and clear notes about what information he needs to include and what context he still needs to add himself.

When Rio uses this validated approach with new data, his diligence continues. He'll check whether numbers make sense based on what he knows about his programs. He'll take accountability for the final report, and he'll be transparent about AI's role if asked. But now he's working from validated confidence, not guesswork.

So here's the framework. Identify a specific analytical task that you want to delegate. Be precise about what you need. Then find past data where you already completed that analysis. You need the right answers to evaluate whether AI can arrive at them. Work with AI to reproduce your past analysis and systematically evaluate the results. What did AI produce? How did it approach the task? How did it communicate findings? Identify gaps, refine your delegation, and then test again. If you can validate that AI produces correct results, you've built an approach that you can confidently use on new data. But if you can't get there after several refinements, you've learned that this isn't a task you should delegate.

So this is all great, but what if you're not very comfortable with the data to begin with and wouldn't be able to spot those process gaps yourself? AI can also be a useful tool to brainstorm and implement solutions you might not have thought of on your own. Because AI models are uniquely good at coding, they can help with things like writing Excel formulas, reformatting messy data, and more. In these cases, you can simply bring your question or idea to AI and specifically ask for help understanding what a solution could look like, just like how you would work with a data analyst on your team. As you work with AI, just keep asking for clarifications and explanations so that you can follow the process and understand the final output.

Just remember, validation builds confidence, but it doesn't eliminate responsibility. You're still accountable for checking that these results make sense and being transparent about AI's role in your analysis process.

This testing works for any analytical task you're considering: donor analysis, budget forecasting, survey synthesis, outcome tracking. Test first, validate what works, then apply with more confidence, or learn what you shouldn't delegate at all.

In our next lesson, we'll look at workflow augmentation and how to apply these same principles when AI handles routine tasks on your behalf.