Tool-watching isn’t keeping up

Not long ago I realized how much time I’d spent watching YouTube videos about AI tools I wasn’t using.

It was more hours than I’d expected. An hour here on a new model release. Forty minutes on a framework I’d heard about twice. A deep dive on something that turned out to be three months from being usable. Meanwhile, the actual pile — the work I needed AI to help me move — sat there.

I wasn’t falling behind. I was performing the act of keeping up. That’s a different thing, and it took me longer than I’d like to admit to name it.

The problem isn’t that there’s too much to track. That’s always been true and always will be. The problem is that tracking feels like progress. It has the texture of productivity — you learn things, you form opinions, you can hold a conversation about it at dinner. But none of it moves the pile.

This piece is about the two questions I now use to decide whether to engage with a new AI tool — and the gate those questions feed into. It’s not a framework I read somewhere. It’s the rule I built after getting burned by the alternative.

The leak

Tool-watching is a productivity leak with unusually good PR.

It disguises itself as due diligence. As professional development. As the reasonable thing a person in a responsible role should be doing when the landscape is moving this fast. And sometimes it is those things. But most of the time — if you’re honest about the ratio — it’s awareness masquerading as progress.

The tell is the pile. If the pile isn’t moving, and you’re still spending time on new tools, you’re not doing research. You’re deferring the harder thing with something that feels adjacent to it.

I’m not arguing against curiosity. I’m arguing against undisciplined curiosity in a context where you have actual work to do with these tools. The question isn’t whether to pay attention — it’s how to structure the attention so it serves the work instead of replacing it.

The first question: does this help the pile today?

The first question I ask about any new tool, framework, or model release is the simplest possible one: does this help with what’s actually in front of me right now?

Not in principle. Not eventually. Today.

Most of the time, the honest answer is no. And “no” is fine — it’s not a dismissal, it’s a parking decision. The tool goes on a list. I’ll look at it again when it’s had time to mature, when someone whose judgment I trust has used it in anger, and when I can see a concrete fit with work I’m actually doing.

What this question does is force the pile back to the center. The pile is the real constraint. Everything else is noise until it reduces that constraint.

The payoff isn’t that you stop learning — it’s that you concentrate. One tool you’ve actually mastered does more than ten you’re sampling. I run Claude in a few different configurations as my primary AI layer, and I’ve gotten enough depth with it that I can use it under real operational load. That took time and repetition. Time I wouldn’t have had if I was still watching videos about the next eight things.

The Claude Code story is useful here, because I got it wrong before I got it right.

When Claude Code launched, I parked it immediately. My read was: this is for developers. I write prompts, not code. Not today.

Then Cowork launched — a desktop tool that made the same underlying capability legible to non-coders like me. I adopted it day one. Went straight to Claude Code in the desktop app shortly after. The tool hadn’t changed much. My ability to see the fit had.

Looking back, deferring two or three months wasn’t a miss. It was the right call. The tool wasn’t legible to my situation yet. Waiting until it was meant I adopted it cleanly, without a tinkering phase, without wasted time trying to make something work that wasn’t ready for how I needed to use it.

The $100 a month I now pay is a straightforward trade. The alternative — buying a Mac Mini to run local models and tinker my way to something equivalent — would have cost days. Days I don’t have. When I frame it that way, the question answers itself.

The second question: does this fix something I know is broken?

“Not today” is only a safe answer if you have a second question running underneath it.

Otherwise you’re not filtering — you’re just ignoring. And ignoring is how you miss the thing that actually matters while you weren’t paying attention to everything else.

The second question is: does this new development address a known weakness in a stack I already run?

Not a weakness in general. Not a weakness someone else described. A weakness I have named, in a workflow I use, that I know is costing me something.

The difference between these two is the difference between following the hype feed and following your own gaps. The hype feed will tell you what’s loud. Your gaps will tell you what’s load-bearing.

Here’s what this looks like in practice. I run a knowledge pipeline — a four-step process for turning inputs (a course, a video, a book) into something I can actually use and retrieve later. The shape is: a source → Whisper transcribes it → Recall summarizes it → ChatGPT reshapes it for my context → Claude Code files it into an Obsidian vault. Simple enough. But I know exactly where it breaks.

First: it’s a relay of separate metered tools. Whisper, Recall, ChatGPT, Claude Code — each hand-off is a cost and a fragility point. The more steps, the more things that can go wrong, and the more the bill adds up.

Second: for video it reads only the audio track. Slides, diagrams, demo screens — anything visual — get discarded. For a leadership course that’s fine. For something with dense visual content, I’m losing real signal.

Third: the hand-offs are manual and context-finicky. Each input type needs different prompting. And I have to feed Claude Code deliberately to avoid context compacting, which degrades output quality when it hits.

Those three weaknesses define exactly two things I watch closely. Local models — because better synthesis plus the ability to process video frame-by-frame would fix weaknesses one and two simultaneously. And orchestration tools — n8n, Make, or something like Codex operating as an orchestrator — because automating the hand-offs would fix weakness three.

I don’t track every model release. I watch what people actually do with local models and orchestration tools. When someone I trust shows me they’ve used one of these to solve the kind of problem I have — not a demo, not a benchmark, but a working thing in a real workflow — I pay attention.

Neither of those watch-points has cleared the gate yet. Which brings me to the gate.

The gate

Before I move from watching to adopting, a tool has to clear three things.

Mature enough. Is this past the tinkering phase? Can I pick it up, apply it to a real workflow, and get a predictable result — or does it still require the kind of trial-and-error investment that costs more than it returns? The Claude Code story is the model here: I waited until it was past that phase. I got clean adoption with no wasted time.

Simple enough. Does using it require a side project to set it up? If the answer is yes, the real cost isn’t the tool — it’s the distraction. I’ve got one shot at attention per day. I’m not spending it on infrastructure unless the payoff is clearly worth it and clearly scoped.

Applicable enough. Is there a concrete fit with something I’m actually doing? Not “I could imagine a use case” — a specific workflow, a specific gap, a specific thing that would move differently if this tool were in the stack.

All three have to be true. One or two isn’t the gate.

Both of my current watch-points — local models and orchestration — are interesting. Neither is through yet. Local models are maturing fast but still require more setup than I’m willing to invest for the gain I’d currently get. Orchestration tools are capable, but the workflows I’d apply them to still need more manual oversight than automation would allow. I’ll keep watching. When one of them clears all three bars, I’ll move.

What this actually does

The two questions and the gate don’t reduce how much is worth knowing about. They reduce how much of your attention it can claim before it’s earned it.

Most new tools don’t need your time yet. A small number of things — the ones that map to your actual weaknesses — deserve real attention when they’re ready. The discipline is knowing which is which, and not letting the loudness of the first category crowd out the clarity you need about the second.

The leak is real. And it will fill as much space as you give it.

The pile doesn’t get smaller while you watch.