I Build Code Auditing App For Engineering Leaders
Engineering Leaders Write Code Too—Here’s My Story
There are two extremes that rule the world of engineering leadership. Leaders either write too much code or don't write any.
You Develop Too Much
At the beginning of a career as an engineering manager or team leader, we tend to do too much software development. It's hard to let it go—passing your beloved projects into the hands of less experienced devs. So we keep taking regular tasks as we used to do as software engineers.
It quickly becomes apparent that we neglect our duties as people leaders by doing so. Our job now is to give feedback, set expectations, grow our teammates, build relationships with the product organization, and zoom out to manage the "big picture" and strategy rather than focusing on implementation details. The more coding tasks we take on, the more our team depends on us.
Most of the content on Practical Engineering Management is exactly about that—how to move from an individual contributor role to one who manages a broader group.
In more than 100 articles, I covered:
And Suddenly, You Stop Coding
You master all of these to be an impactful and empowering leader, and suddenly, there comes a moment when you push the last commit. It happened to me as well, a few times already.
It's funny because, as a Team/Tech Leader, I had deep contempt for higher-level managers who didn't write at least a few lines of code per week—until I became one.
There were long periods where, as Head of Engineering or Engineering Director, I spent all my time on 1:1s, data analysis, brainstorming discussions, career leveling, hiring activities, and many other back-to-back meetings.
In rare moments of coding nostalgia, I desperately tried to use a 1-hour gap in my calendar, only to realize that those 60 minutes were just enough to update my IDE, dependencies, and code repository and maybe launch the project.
I firmly believe that this is a bad place to be.
In managing software engineers, it's critical to at least scratch the surface of their daily activities. Otherwise, you will never understand their points of view—why tech debt and poor dev experience are so painful, or why "this is just one line of code" takes an entire week to develop.
If you cannot at least run, test, deploy, or commit a simple change to the project you manage, you will never be able to make good strategic decisions in leading your team and the tech stack.
Fast forward to today: while managing the entire site of my employer, leading tens of engineers, and running my side business under Practical Engineering Management, I prioritize at least one or two deep-focus coding blocks per week. During these blocks, I build something meaningful, play with scripts, or run/contribute to some of my teammates' projects.
In one of my future posts, I will explain why there has never been a better time for engineering leaders to return to coding than today and explain how I do this while managing an engineering team or organization.
I've Been Building An App For Engineering Leaders
Today, I want to share a sneak peek of the project I've been working on for a few months. If you are a paid subscriber, you will also have access to the source code (see the bottom of this article).
In the spirit of "solving customers' problems instead of just building the code," here's the context of my work.
The Problem I Try To Solve
As a middle-level engineering leader, my job, among others, is to shape the long-term strategy for the codebase under my management. A few years ago, as a full-time mobile engineer, I knew exactly the direction my team and I should follow—architecture, quality, complexity management, etc.
Today, I work with technology-wide teams on Java, TypeScript/JavaScript, Kotlin, Swift, IaC code, Python, and several frameworks. Strategic planning for such a tech stack can be extremely challenging.
Fortunately, with the rise of Generative AI, working with such codebases is easier than ever. In my work, I use GitHub Copilot for autocompletion and inline code generation. I also use ChatGPT and Google Gemini to make more complex contributions. This toolset is enough for me in most coding tasks, but it doesn't provide everything I need for "big picture" analysis (architecture, overall quality, high-risk areas of complexity, etc.).
I'm aware that there are plenty of tools on the market that analyze and generate code more holistically and with a broader context. But before jumping blindly into the sea of AI-as-a-service and the promise of "robust code generation," "10x productivity boost," or "generate a working app in one weekend," I wanted to build something that would give me at least a basic understanding of what's happening under the hood.
Codebase as a Prompt
One day, I saw this LinkedIn post from a friend,
(if you understand Polish, I highly recommend his newsletter on Substack). Kamil built a simple Python script that parses the entire codebase into a single .txt or .md file, which can later be used as an LLM prompt.He also published tens of example prompts that analyze such a codebase and provide expected results (e.g., user stories based on the code, basic documentation, possible bugs, or code enhancements). Take the codebase prompt, give instructions on what you want, send it to LLM and voilà—simple as that!
I have to admit, at that moment, I had my "aha!" moment and started working on a tool that would solve my own problems as an engineering leader.
CodeAudits.ai
This is how CodeAudits.ai was born.
Disclaimer: The app is still in its early development phase (let's call it pre-alpha). There is a good chance that, at the moment of reading this, it could be down due to usage limits (I use the free version of Vercel infra and a free API key for Google Gemini).
Over time, the app will become more stable, ultimately with some SLIs and SLOs for its reliability. In the meantime, if you are interested in setting it up as on-prem (on your own Vercel account), contact me at mirek@practicalengineering.management.
How Does CodeAudits Work?
The app works in two steps:
First, it generates a single Markdown or text file containing an LLM prompt that describes the repository’s code.
Based on this file, the app generates audits using LLM services (currently Google Gemini, with plans to add ChatGPT, Claude, and others in the future).
This project was initially built for personal use. When managing multiple engineering teams with dozens of software engineers, I needed a simple way to review their projects—especially those written in technologies I have little or no experience with.
The audits provide a general overview of the feature set, architecture, potential bugs, and ways to improve each project in the future.
How to Use It
There are several ways to generate code audits. Each one starts with the codebase-dump app, a Python open-source project that parses the codebase and pushes it to CodeAudits.
Here's how you can use it:
Run it in Colab: Use an online Python notebook. Launch it here: Code Audits - Quickstart Guide. To test it out, run all the code via
Runtime -> Run All
.Run it locally on your machine: The app is available on PyPI. Read the Readme file for more details.
Run it as a GitHub Action: Check this example workflow. Here is the example output.
Run it from the source code: Instructions are also described in the project’s Readme file.
Once the code is submitted by codebase-dump, you can visit the generated link to access the Code Audits App.
If the current audit list is insufficient, you can explore it on your own.
In the Codebase Input section, click the Copy to clipboard button. Then, go to your preferred LLM (Gemini, Claude, ChatGPT, etc.) and paste it as the first prompt. From there, you can start asking questions about the repository.
What's The Promise
My central vision for this product is to provide critical input to your technical strategy. Here, AI is just a companion to your work as an engineering leader, but it should give you enough context to understand where your technical project(s) should be in 3–6–12 months.
You can use audits as input to your Engineering Strategy Framework.
Will I Commercialize It?
Before I consider monetization strategies, I need to ensure that the product has enough value for you. Hence, I’d be grateful for your feedback—either as a comment here, a direct message, or via the CodeAudits feedback form. I'm also open to discussing a dedicated instance of CodeAudits for your organization.
Paid plans of CodeAudits will also be presented once I hit the paid limits of the underlying infrastructure or the amount of work I put into this project goes beyond my regular engagement for Practical Engineering Management.
What Are The Future Plans?
In recent years, I have focused on sharing content with engineering leaders through this newsletter. By doing so, I aim to help engineering managers, directors, team leads, and others become more impactful and build high-performing engineering teams.
Working closely with source code is part of our leadership duties, so I see CodeAudits.ai as a complementary addition to my work here.
Moreover, I want to share the app not only as a working product but also to reveal the process behind the scenes:
How to manage such a project
How to build it using modern tools while being a "regular-level" software engineer
How to navigate phases like PoC, Prototyping, MVP, and future iterations
How to distinguish between what’s critical, important, and nice-to-have
How to embrace imperfection when time and other resources are extremely limited
…and much more on this topic.
Please let me know if this is the content you would like to see more of on Practical Engineering Management.
Technical Details of CodeAudits.ai
If you are a paid subscriber, here I’m sharing some of the technical insights behind CodeAudits.ai.