Policy is Dead, Principles are King: Scaling Security at the Speed of AI

Introduction

When comparing the growth of a tech company to a traditional enterprise (e.g., an insurance company), tech companies consistently outpace the growth figures of the enterprise. In 2023, we stepped into another era called the AI generation. There is a clear differentiator between traditional SaaS companies and AI companies. Comparing traditional SaaS to an AI company, traditional enterprises are built with strict hierarchies, compliance, and governance. These act as guardrails to ensure the quality of work delivered to customers and internally, and to meet societal compliance expectations. On the other hand, AI companies (most often startups) typically feature a simple hierarchy, a flat culture, and swift turnarounds when encountering problems.

AI creates a lot of risk, which enterprises don’t like. Enterprises need control over the company to maintain customer trust, societal expectations, and compliance. In a startup, risk management is much swifter. When a problem is found, a small team can have a 30-minute discussion and make decisions, all within half a day.

AI isn’t just creating risk every 30 days, it changes every day, and quite possibly with every API call. 

“If we don’t micro-manage the risk of AI use within the company, we lose our control.” 

However, it seems this AI era shouldn’t rely solely on governance policy anymore. Instead, we need closer collaboration with the actual users regarding how they use AI with security awareness.

Hurdle in AI Product Development and Customer Satisfaction

AI products often encounter a specific problem: accuracy. Due to the probabilistic behavior of AI, customers can sometimes be dissatisfied with the product's output. This has led to fundamental changes in how products must be built in order to solve this problem.

Back when we built traditional software with predictable outputs, there wasn't nearly as much effort needed for testing compared to AI testing, as we simply followed the input-processing-output model with static results. Now, measuring unpredictable outputs requires extensive comparison against large datasets. 

“We don’t usually consider using customer information for testing or as part of measuring quality. But now, it seems inevitable to use it, as it provides a sufficient amount of data for measuring the quality of development.”

Because of this, we have to be very conscious about obtaining customer consent to use their information for development or testing. Furthermore, the intellectual property rights of the information input into your product usually remain owned by your customer.

Managing Information Security at an AI Company

Running an information security program in a startup versus an enterprise is extremely different, as they operate on two completely different scales. People in an enterprise often rely on strict policies to follow, they are treated as one-size-fits-all solutions, and there isn’t an easy way to change them. The person drafting the policy rarely gets a chance to look at what the hands-on staffs actually need.

People, especially top performers, are always curious about what AI can do. You shouldn’t take away their chance to use AI for work. Otherwise, they will find ways to bypass controls and use AI with no oversight from your team. 

"You know what? Most proprietary LLMs don’t train their models with your input information if you are on a paid tier."

Once you have given the tool to your team, the next problem is controlling how they use it. You should already have an information classification system in your company, but this alone is not enough for people using AI. The problem with traditional information classification is that the definitions are often too broad and compliance-oriented. You can't easily control how information flows while still reaping the benefits of AI, whose very nature is gathering and analyzing large amounts of data. The real priority is that people need to be aware of what information should be inputted into AI and how they should use it. 

"Of course, people's awareness always comes first, no matter if it's general security or AI security."

As a security practitioner, I am inspired by Indu Sajeev’s point that "practitioners take a principle-based, rather than policy-based, approach to governance". Enforcing security or governance should not slow down innovation, your job is to ensure people avoid doing what we don't want them to do. If policy-based governance is no longer able to control the growth and needs of AI, we should take a step back, identify the basic principles of security, and tailor our approach depending on the people’s actual needs.

Information Security Culture: Startup vs. Enterprise

In the startup world, people wear multiple hats. You will rarely do just one thing as your primary responsibility. Having multiple responsibilities means you need to be highly collaborative with different people and units. Making changes is extremely fast because of the shared knowledge everyone possesses and the small number of people involved.

In an enterprise, hierarchy, meetings, and governance are often blockers before a decision is made. A simple change might take a month or even a year to complete. This simply cannot align with the speed at which AI grows and the rapid countermeasures required from the security team.

At Tofu, the IT and security team functions essentially as a single unit - it is blended into various engineering processes and involved in management decisions. The security team standardizes the procedures and guidelines needed for engineering, and we also provide and manage the AI tools the engineering team needs while ensuring they align with our security principles. Because the engineering team is the end user of these tools, the security team actively collects feedback on their daily use cases to ensure the balance between business needs and security is properly maintained. The security team also reports directly to the CEO to remain independent in its decision-making. This structure allows our AI products to succeed without sacrificing quality, while also aligning with compliance requirements and acceptable risk appetites.

Based on my personal experience, this structure is hard to implement in a traditional enterprise because it requires massive cultural, hierarchical, or even political changes.

AI is flipping governance and security into completely different management concepts

AI is entirely flipping how security governance has been implemented so far. I think we need a security AI agent to manage the incremental risks caused by AI, but at the same time, an AI agent can’t be the one handling principle-based governance. What do you think about the role of security alongside the rapid growth of AI?

"I see the full potential of AI in boosting productivity and helping humans. Doing governance has never been this difficult when balancing the needs of AI. One thing I am sure of is that we shouldn't be the ones slowing down innovation, we should be the enablers helping AI grow, not banning it."

‍