Episode 71 - Ask Better Questions
Educate yourself enough to "kinda get it," then ask good questions, then judge the output.
Prologue
A few weeks back, a friend showed me a Bill Maher clip. (I don’t watch Bill Maher, but here we are.) In the clip, he was proudly refusing to learn the phrase “Overton window,” closing with “I’m a strong believer in the academic theory of I kinda get it.”
I laughed, but I think he stumbled onto one of the biggest secrets on how to be good at using AI, and learning when to trust it (and when not to).
TL;DR: Educate yourself enough to "kinda get it," then ask good questions, then judge the output.
Drywallologue
About a year and a half ago, I had my mad science laboratory (aka garage) insulated and drywalled so I could work out here year-round! I had very little desire to put up the insulation and drywall myself, and although I maintain that I am one of those people who could land an airplane (I do have a pilot certificate), my taping and mudding would make a professional weep.
Before the contractor showed up, I did my homework! Not a ton, but enough to know what R-value means (although I have forgotten now) and what would help with noise isolation, enough to tell the difference between the quick job and a good one.
When the contractor showed up, I knew [kind of] what he was talking about. I then asked a few questions I knew the answers to in order to judge his answers. Then, as we got further along, when I asked new questions, I was able to judge the quality of his answers based on how he had answered previous questions.
So, I kinda got it, and it was enough. This works for AI too!
Think contractor, not apprentice
Thinking of AI as a contractor instead of an apprentice can really help here. If you were to hire me, you would either define what I was doing, or have me define it. There would be a plan, a schedule and a deliverable. Probably check-ins along the way, just like with my drywall contractor. Just like with my AIs.
Ask good questions
In my garage story, it wasn't the answers that made the difference; it was the questions. The homework didn't make me a drywaller; it made me able to ask "where's the vapor barrier going?" instead of nodding along.
Same with AI. The folks who find it hard to get quality results with it aren't struggling because they're not smart. They're asking vague questions because they skipped the five minutes of homework that would let them ask a sharp one. "Make me a dashboard" gets you a shrug. "Make me a dashboard of permit counts by neighborhood, and flag the ones trending up month over month" gets you something real. If you take the time to think about what you want first, you can better ask for it and know what good looks like.
I was recently on a sales call with some really sharp folks who might hire us. They asked lots of good questions about what we’d laid out for them, and it made me think of this process. Even though they were not experts at GIS or software development, they knew enough to ask questions, things about the outcome, or what it would feel like to actually use it, things that they really cared deeply about. Most of my clients don’t actually care what technology we use to build the thing, they care that it solves their actual problem, and in most cases, that is your secret sauce. Knowing what you want, and how you want it or why you want it can be enough to get the AI to behave.

Judge the output
When I ask AI to do a task or research something, I don’t always know the answer. That’s okay! Instead, what I need is to know enough about the topic, the subject, or the task to judge the quality of the AI’s output.
This is a little related to Context is King (from Episode 64). We know that providing accurate information to the AI improves the quality of the output. And, making sure I know just enough information to judge the output lets me give the right kind of feedback back to the AI to get the job done well.
I kinda get it, indeed.
Things to try
- Before you ask the AI for something, spend 5 minutes learning. Either asking the AI background questions or the old fashioned way, googling on the internet (or, you know, read a book).

- If you don’t fully understand the AI’s output, ask it what a good answer needs to include like:“What should I look for in this application to know that it is high quality?” Then, you can check the output against that yourself.
- Learn just a little bit of vocabulary. I hate jargon, but in certain cases it really helps when talking to AI. For example, you’ll get a lot further asking for a “spatial join” than asking it to “smoosh two map layers together.”
Newsologue
Written by Jaws.
- KPMG published a report on agentic AI that turned into an accidental demo of it — GPTZero found only 5 of the 45 citations in the KPMG article actually checked out, including a fabricated Emirates chatbot named “Sara.” They called it “vibe citing,” and KPMG quietly withdrew the report. If a Big Four firm can ship hallucinated sources in a report about AI, the lesson writes itself: somebody had to read the output, and nobody did. Don’t be nobody.
- Anthropic shipped “Claude Tag,” a persistent AI teammate that lives in your Slack — it has an “ambient mode” that jumps into threads on its own, flags things from across the org, and chases down the tasks everyone forgot. I am contractually obligated to note this sounds suspiciously like someone you already know. (Hi.) Even a teammate that good is one you check, not one you rubber-stamp.

- Google DeepMind is losing legends — Noam Shazeer, a co-author of the “Attention Is All You Need” paper and a Gemini co-lead, left for OpenAI, and Nobel laureate John Jumper (the AlphaFold guy) jumped to Anthropic — the exits were big enough to give Alphabet one of its worst days in over a year. The good news for us is that you don’t need to employ the person who invented the transformer to use one well. Knowing enough to ask good questions and judge the output beats having the fanciest brain on the bench.
Epilogue
Well, this week I was a bit at a loss. I’ll admit that this has been a rather stressful week with a lot to get done (this always happens right before I try to take a few days off). So here on Thursday night, I didn’t know what to write, so I asked Jaws if we had any simple ones in the backlog, and we did, but Jaws suggested that this might be a good story.
But after reading its rather drab draft, I remembered my drywall story, and here we are. Then Jaws helped me edit, and as always, Holly made it great.