The most critical piece in the work of product and engineering?
Data.
Facts, metrics, and conclusions help us understand the world better.
Empirical evidence is what enables system thinking. Fortunately, in the age of AI, it’s easier than ever to get access to information, right?
Yes and no.
Even with all the tricks and solutions under the hood of AI chats — scripts, interpreters, MCP servers — LLMs are still non-deterministic language models.
They’re also trained to always provide an answer.
That means that even though information is always quickly available, validity isn’t guaranteed.
Data-Driven Engineering Leaders
Regular engineering leaders focus on execution: scheduling, task prioritization, resolving blockers.
High-performing engineering leaders navigate — they gather data, make decisions, pivot, and predict.
We calculate costs, forecast performance bottlenecks, aggregate incidents, count SLIs, or set SLOs.
We turn data into information, information into knowledge, and knowledge into wisdom (I covered that in How to Be a Data-Driven Engineering Leader).
So where does AI fit? Today, we upload CSVs, logs, even full system structures, and ask ChatGPT:
“What’s the slowest API call?”
“How many endpoints does this monolith really have?”
“What’s the average number of files uploaded by a user?”
How is this being calculated under the hood? We don’t really know.
Even if the results look legitimate, confirming them isn’t always straightforward.
Don’t Ask for Conclusions. Ask for the Formula.
We can do better.
Rather than asking AI for results, ask it for code that will interpret your data in a deterministic way.
Sure, it takes longer — maybe tens of minutes instead of seconds.
But in return, you get auditability, repeatability, and version control.
You can store the script, rerun it, tweak it, and know exactly how the data was processed.
“In the CSV file, you’ll find logs from the last seven days.
I want you to create a TypeScript script that parses this data and returns a CSV with endpoints, average, p90, p99 latencies, and number of calls.
When aggregating, use wildcards for IDs — e.g.,
/report/123/summaryand/report/abc/summaryshould count as one endpoint/report/*/summary.”
That’s how you move from answers to proof.
Open Source for You
If you fell into the trap of the “non-coding leader” and your programming skills got a bit rusty, I’ve got you covered.
Here’s an open-source project you can use 👉 github.com/frogermcs/leaders-scripts
Clone it, add your scripts, and use it as your personal automation toolkit.
The idea is simple:
One repository for all scripts.
Each script is a small, independent project.
Each script solves exactly one problem.
Many engineering leaders I know keep a similar repo.
Inside — dozens of small tools that save minutes, hours, sometimes days of manual work.
Don’t Write Code — Use AI
Here’s the best part: you don’t need to write these scripts yourself.
The repo includes an AGENTS.md file — a specification most AI coding agents (Cursor, Claude Code, Gemini, Copilot) can follow (check it here).
It tells the agent:
How to set up the project,
How to create a new script,
How to run it, and
How to prepare output.
Then you just request:
I have the file example_logs.csv.
I want you to create a new script named `logs-statistics` that processes this CSV and exports endpoints statistics based on average duration in milliseconds.
In a minute, you’ll get:
A working script,
Instructions on how to run it,
Hints where to find the output.
You don’t even need to know how to run it. Just ask your agent in VS Code:
“Can you run the logs-statistics script for me?”
My Real Use Cases
I also use my own version of this setup — and AI helps me automate parts of my daily work.
Here are a few examples from the last months:
The entire article is available only for paid subscribers.
You can use the training budget (here’s a slide for your HR).
Thanks for supporting Practical Engineering Management!