A model that runs in a notebook serves one request at a time. A production endpoint receives many requests at once, and each one needs GPU memory for its key-value (KV) cache, a slot in the batch, and a response stream. Plain PyTorch or Hugging Face Transformers can serve the model, but you have to

Sanjay Beri, Netskope
Netskope is a cybersecurity company that helps organizations securely adopt cloud and AI technologies by protecting data, applications, users, and AI interactions through real-time security and zero-trust controls.
You founded Netskope in 2012, long before generative AI became the phenomenon it is today. What problem were you trying to solve at the time, and how has AI changed that mission?
We built the ability to protect data and transactions worldwide through our private cloud network. AI itself is not new for us; we have had AI labs for nine years, working on deep learning rather than LLMs, and we have built more than 190 deep learning models. When LLMs and smaller models started being adopted, our platform was positioned to do the same thing we had done for cloud and remote work: enable people to use them securely. Today, our mission is to enable this new era of cloud and AI.
Financial institutions are moving quickly to adopt AI while handling some of the world’s most sensitive data. In your mind, is the industry approaching AI with sufficient responsibility?
Financial services is our number one sector worldwide, and we work with some of the largest financial services companies globally. What I have seen is that they are leaning forward with AI because they know it is transforming every part of their industry. If they do not embrace it, they are going to fall behind. AI changes virtually every process and creates new opportunities to serve customers, so leveraging it for internal and external innovation is not really optional.
At the same time, because of who financial institutions serve and the regulatory environment they operate in, they are doing this in a very measured way. They are trying to balance moving quickly with putting the right constraints and controls in place. I cannot get 30 seconds into a conversation with a financial services company without them asking: How do you safely enable AI? How do you protect agents? How do you zero trust LLMs? They understand the challenge and are adopting security controls around it.
How can Netskope help financial institutions enable AI safely?
A financial services company’s brand takes years to build, but a couple of incidents can ruin it. I take our brand equally seriously, so I do not focus simply on selling what we do. I focus on the right architecture to enable and secure AI. It starts with a few core principles, the first being to zero trust every part of your AI ecosystem. Do not trust agents; every agent is a potential malicious insider. Do not inherently trust a model because you do not know what it is going to say. You need to monitor these entities in real time, understand and govern their intent, know where your sensitive data is, and ensure that data is used only in the right way.
The first step is visibility. The majority of organizations do not know exactly what AI is being used within their company because much of that usage is driven by end users or business units rather than IT. We give them visibility into their agents, MCP servers, models and generative AI applications, including the risks and data associated with them.
They can then apply real-time controls—for example, allowing only corporate versions of certain AI applications, preventing sensitive financial data from being entered, or stopping an agent from accessing data outside its intended purpose. At the same time, our worldwide network is optimized to connect customers to AI destinations quickly. The goal is to balance performance and security: companies need to move quickly, but they also need to be secure.
Couldn’t zero-trust controls create too much friction and ultimately slow companies’ ability to compete?
Every company—and, frankly, every nation—is balancing the fact that AI is an unstoppable force for innovation. Financial institutions cannot afford to be laggards because they will fall behind competitively. At the same time, they know that consumer trust in finance can take decades to build and one misstep to destroy. If they leverage AI without doing it securely, they could leak customer data, train models on data they should not be using or create serious problems for their customers. Ultimately, that would undermine adoption.
They therefore have to do both. Many companies have labs where they experiment, innovate and tightly control AI usage. They also want to democratize innovation rather than confining AI to one centralized group, so teams across finance, legal, HR and other functions can innovate. What they then put around that is a common set of corporate controls: shared visibility, security controls and governance around how AI should be used. That is largely how I see financial services approaching the challenge.
How do legacy institutions’ approach to cybersecurity and AI differ from that of challengers and fintechs?
Fintechs were created to disrupt. They tend to be smaller, more digital and more engineering-oriented, and they often started with a relatively greenfield security environment. That can be good and bad. The advantage is that they can adopt modern security quickly because they do not have legacy technology to replace or migrate. The disadvantage is that they do not necessarily have the decades of controls and maturity that established institutions have.
Many traditional banks are moving very quickly to modernize, but many are also still using legacy security equipment that was not designed for the AI world. They need to modernize because legacy security frameworks cannot handle AI at scale. AI is becoming a competitive advantage, so this is forcing financial institutions to modernize their security infrastructure more quickly. They know that without doing so, they cannot safely enable AI.
Netskope went public with the goal of increasing awareness. How has that played out?
For many of us, Netskope is our professional legacy. We are deeply motivated by our mission to stop criminals, hackers, nation-states and malicious insiders. I have sat in hospitals that could not perform critical surgeries because criminals had taken them over and held them for ransom. These are some of the most depraved people in the world that we are fighting. Going public was about furthering that mission by broadening awareness of Netskope. We win more than 80% of the time when we reach a proof of concept where a customer can try our product, so we felt that becoming a public company could bring another level of awareness and confidence.
It also means operating with a completely transparent financial model. We have traditionally invested heavily in R&D—close to half of the company is R&D—and focused on building the best platform while perhaps investing less in marketing. Being public has helped amplify our name. But you cannot operate a public company quarter by quarter. You need a mid- to long-term view. Whether you are a financial services company or a cybersecurity company like us, the mentality has to be: innovate or die. You have to continue investing in R&D.
Looking ahead, which initiatives at Netskope excite you most?
Netskope was founded on the idea of enabling companies and people to say yes to great technology while putting the proper security guardrails around it. In financial services, AI could democratize access to financial advice that was previously available only to a select group of customers. Retail banking customers could potentially receive the same kind of hyper-personalized, high-quality portfolio or tax advice that may previously have been reserved for elite wealth management customers. You can also think about fraud: rather than identifying a fraudulent credit card transaction after the fact, can you stop it at the point of sale? The same principle extends to other sectors such as healthcare, where securely leveraging AI could transform research and the development of treatments.
What excites me most is that one of the biggest barriers to realizing that potential is security and privacy—leveraging AI safely—and that is what we do. Our mission is to help good people use AI securely while protecting them from criminals who have been given one of the greatest weapons they have ever had. We live on both sides of that equation: enabling financial services companies to revolutionize areas such as financial advice and fraud detection, while trying to stop malicious actors from using AI in the wrong way.
