When The App You Built With AI Starts Handling Real Data

Our client operates several affiliated entities under a common leadership team. In early 2026, staff at one of them discovered what many small teams have: with a browser-based AI app-building platform, you can produce working software without hiring a developer.

So they did exactly that. Over several months, non-engineers shipped multiple internal applications using Replit: a case management platform used daily by front-line staff, a resource planning tool, and a third operational application. All of the applications worked, as they replaced spreadsheets and eliminated the need for duplicate data entry.

But that was exactly the problem: the software visibly worked. However, their team didn’t have the technical expertise to understand how it worked and whether it was built properly for security and long-term viability.

None of the people building these applications were software engineers. They were smart and highly motivated, but they built systems that a more experienced development team would have flagged for security issues and durability concerns during the build process.

One platform handles sensitive personal information subject to federal and state reporting requirements, which raises the stakes on every engineering decision for it. It also operated near a licensed healthcare entity elsewhere in the portfolio, which means HIPAA compliance must be considered. However, nobody on our client’s team was confident they had accounted for compliance and the associated risks and liabilities when building their platforms.

The client’s leadership team came to Clear Function with a direct question: what are we not seeing?

We spent the first weeks reviewing their code, mapping the data, and interviewing team members who use these systems daily. The gaps were ones we usually find in AI-built software, and none of them were the result of anyone being careless.

• Credentials stored in the wrong places: Sensitive digital information, like API keys, lived where the code lived, rather than in dedicated, secure storage. Any key that had been sitting in a repository had to be treated as exposed, so those were rotated.
• No separation between staging and production data: Real client data was sitting in the development environment where changes were tested.
• No clear path from change to production: Work happened directly on the platform. There was no second set of eyes on a change before end-customers encountered it.
• No encryption or access controls for sensitive information: Social Security numbers were accessible to anyone with database access, and there was no encryption or audit trail to record access or changes.
• Bug reports living in Slack or email: Requests and defects were submitted via Slack messages without a ticketing system to track status or confirm fixes were completed.

The most consequential finding wasn’t even technical; it was structural. Our client’s portfolio companies are separate legal entities, and the boundaries between them determine which regulations apply to which system. We worked with the client to map the structure and the data flowing across it. The finding: the licensed health provider carries full HIPAA obligations and needs its own compliant infrastructure. The other platforms under review do not, but they do need security upgrades to properly protect personally identifiable information (PII) to meet the requirements of government programs they work with.

Getting the boundaries right saved our client from extending HIPAA compliance across systems that didn’t need it at a sizable cost, and from under-protecting the data in the system that did.

We are not writing this client’s software. Our engagement consists of a weekly office hours call, plus asynchronous guidance during the week as needed. We review what they’ve built, explain what needs to change and why, and hand them the steps. Their team takes our instruction and does the work with Replit.

We deliberately chose this structure because our client’s team has real in-house capability to produce using AI coding applications. What the team was missing was the highly discerning judgment that comes from having shipped and maintained software before. Our role is more of a fractional engineering leader rather than a one-stop development shop (although we do that too).

The client is working through our list of recommendations in order of priority.

• Documented compliance boundaries: Each entity now has a defined regulatory scope and an infrastructure plan that matches it to prevent scope creep.
• Encryption and access controls: Sensitive fields are now encrypted, with role-based access and an audit trail.
• Breach detection: With audit controls in place, breach detection protocols were implemented not only to detect security incidents but also to document them for proper reporting and recordkeeping.
• Production data removed from staging: The team is working from scrubbed copies rather than live records.
• Repository pruning: Stale Git branches were consolidated, and the live version is unambiguous.
• Defining the path to production: Changes are now made on a branch, pushed for review, approved by a second person, and only then merged and deployed. It is still more manual than it will eventually be, but it has replaced a process with no checkpoints at all.
• Work is tracked via a ticketing system: Requests and bug reports now live there to improve visibility.

Once they have fully implemented these recommendations, we will move on to phase two, which focuses on future-proofing for long-term success.

You do not need a compliance problem to benefit from this kind of review. If your team has built something real with AI tools and any of the following is true, it is worth a conversation.

• You are storing sensitive data and cannot say how it is protected.
• No one reviews changes before they go live.
• One person understands how the whole thing works.
• You found out about the last outage from a user.
• You are not certain which regulations apply to which of your systems.

The work needed doesn’t have to be a full rebuild. The application logic is sound, as your team has already validated it through use. What is missing lies around the application rather than inside it, and it can be added to or improved without taking the software offline.

Ready to find out what you’re not seeing? See what a fixed-price assessment covers, or get in touch and we’ll tell you whether you need us.