The recent rapid adoption of the AI application “DeepSeek” has gained significant global attention. Becoming the #1 app on both the Apple Store and Google Play Store within its first few days, seeing over 10 million downloads.  

While this explosive growth of DeepSeek R1 highlights the public’s fascination with AI-driven tools, the security community and policymakers have been less enthusiastic. Governments and cyber security professionals have raised alarms about its potential risks to privacy, security, and misinformation, sparking debates over whether the convenience of such technology outweigh its dangers. 

With AI technologies evolving at an unprecedented pace, are we truly prepared to handle the security challenges they pose to individuals and businesses alike? 

Let's consider DeepSeek R1 as a critical case study and review some of the issues that may arise with rapid adoption of new AI tools. 

Content Biases – Censorship and Misinformation

 clarify that the technology operates under the strict regulatory frameworks of the Chinese government. This may be a concern to many, as these regulations mandate that the AI must adhere to the country’s “

” This means that the platform must systematically censor responses on political topics while simultaneously promoting narratives supported by the Chinese state. Evidence of this has already been documented with some users being quick to discover that the AI

 regarding sensitive political topics such as Tiananmen Square. 

This becomes clear very quickly when using the AI and asking it to comment on sensitive political topics and can be replicated easily, as seen below:  

This behaviour could be bypassed by crafting less explicit prompts, resulting in a more useful - less censored - response:  

However, shortly after the response was displayed in the web interface, it was deleted by the application and replaced with the message “Sorry, that's beyond my current scope. Let’s talk about something else”:

The same behaviour was also observed within the mobile version of the application, where sensitive responses were retroactively deleted after being presented to the user. 

This selective censoring and dissemination of information raises ethical concerns regarding how AI models influence public opinion and the risks of AI-driven propaganda, in a world where it is already too easy for people to become misinformed or radicalised by existing media and information resources.

It also raises questions about the safeguards in place to prevent the unintended disclosure of sensitive information. If the tool is capable of revealing politically sensitive data – even if it’s self-aware enough to retroactively delete it - then it is reasonable to assume that an organisation’s proprietary or confidential data could also be exposed, especially if it has been unknowingly (or knowingly) included in its training data.

This presents a significant risk for organisations whose employees may unknowingly be inputting sensitive work-related information into their AI tools, potentially exposing proprietary data which could then be used for training.

Security researchers were also quick to assess the AI model's susceptibility to attacks and abuse. Research performed by 

, determined that the model was “highly vulnerable”, in no small part due to its susceptibility to Jailbreak attacks, which can be leveraged to bypass the built-in safety measures of an AI and cause it to generate harmful, biased, or inappropriate responses.

During our initial analysis, CultureAI found that it was trivial to bypass the platforms “security controls” using basic prompt injection techniques without the need to perform a jailbreak:

Interestingly, unlike responses to political subjects, forcing the model to divulge malicious code did not result in it being retroactively deleted by the application. This could allow an attacker to copy and expand upon the responses to: 

This conclusion is also supported by research performed by Donato Capitella using 

, which also concluded that the initial DeepSeek AI models ranked extremely low when it came to resisting common prompt injection attacks. All these factors considered indicate that “security” may not have been a priority throughout the model’s training.

The Ecosystem - A Goldmine for Attackers?

Focusing on the individual tool however does not capture the big picture! When it comes to the exploitability of AI, the rapid rise of tools like DeepSeek emphasises both innovation and risk. Attackers will be quick to exploit this trend and develop malicious applications, disguised as the latest helpful AI assistants, luring users in and harvesting their sensitive and valuable data. We already know this works the tactic has been leveraged many times before. In 2024 it was reported that there were hundreds of malicious apps on the Google Play Store which had been downloaded 

. Now we can compound this with the public and business appetite to rapidly adopt AI, and the threat becomes even more concerning. 

Without proper caution, the eagerness of users and organisations get their hands on the latest AI tools may unknowingly drive sensitive data such as login credentials, financial data, and intellectual property right into the hands of attackers.

Data Privacy – Who’s Really in Control?

One of the largest concerns highlighted by DeepSeek AI is the data collection and storage policies for AI tools. When reviewing DeepSeek’s 

Is this really something that should be a concern though? I hear some of you ask. Would employees really input business IP, confidential information, personal details, or other sensitive data into something that can help them automate their jobs? That answer is: Absolutely they would.

 mandates that all enterprises, organisations, and individuals must “support, assist, and cooperate” with the country’s intelligence agencies. This means that any data processed by DeepSeek could be accessed by the Chinese government, posing significant risks for businesses, government employees, and individuals using the platform.

However, DeepSeek's handling of user data also highlights the more often overlooked risk of third-party integrations. As we’ve seen with the likes of ChatGPT, AI technologies are now also integrated into other solutions to provide AI features and functionalities. This expands data privacy concerns beyond the AI tool itself to the plugins, browser extensions, and productivity tools that make use of it.

It is however not surprising that user data is leveraged by AI technology providers, and even when asked, DeepSeek was very helpful in confirming that our data may not always be handled with the highest level of confidentiality:

Interestingly, unlike responses to political subjects, forcing the model to divulge malicious code did not result in it being retroactively deleted by the application. This could allow an attacker to copy and expand upon the responses to:

. Now we can compound this with the public and business appetite to rapidly adopt AI, and the threat becomes even more concerning.

Whether data enters an AI model directly or through integrations, once processed, tracking its use, preventing misuse, and ensuring compliance with global regulations becomes significantly more challenging. And, while China-related cases dominate headlines, businesses worldwide may already have users unknowingly exposing proprietary or sensitive information through existing AI-powered services without realising it.

As AI becomes even more deeply integrated into daily workflows, effective AI security is more critical than ever. To defend against AI security and privacy risks, organisations and users must take proactive steps to evaluate and control AI adoption, including knowing:

In an era where data is one of the most valuable assets, blindly adopting AI tools like DeepSeek without understanding the risks it may pose could have severe consequences. Whether it’s data privacy violations, information leaks, cyber security threats, or widespread misinformation, DeepSeek may serve as a case study in the dangers of unchecked AI adoption.

Initial Responses – What are people doing?

Data privacy concerns have already prompted action from government agencies and regulators. The U.S. Navy 

, advising its personnel against the use of DeepSeek for any purpose, be it work-related or personal. In addition to this, Italian and Irish regulators 

within their respective countries among concerns about how personal data is collected, from which sources it’s collected, and for what purposes its used.

These decisions highlights how seriously governments are taking the potential security threats posed by foreign AI systems. and how companies and individuals a like should do the same.

Conclusion - A Cautionary Tale for AI Adoption

The true impact of DeepSeek's meteoric rise has yet to unfold, but its emergence - alongside other AI advancements - serves as an early warning about the risks at this intersection of AI, data security, and geopolitical influence. While AI holds the potential to revolutionise industries it must be implemented and adopted responsibly. The security, privacy, and ethical challenges posed by AI tools cannot be ignored. Governments, businesses, and individuals alike must remain cautious and establish proactive safeguards to mitigate the risks before they become emergencies.

The rapid (and wow is it rapid!) pace of AI innovation is exciting, but reckless adoption will come at a cost. The future of AI won’t just be defined by what it can do, but by how responsibly we use it. We must adopt AI wisely, and ensure that AI innovation enhances security, privacy, and ethical integrity, rather than undermine it.

With AI technologies evolving at an unprecedented pace, are we truly prepared to handle the security challenges they pose to individuals and businesses alike? Let's consider DeepSeek as a critical case study and review some issues that may arise with adoption of new AI tools. 

The Rapid Adoption of DeepSeek: A Security Nightmare

, Lead Cyber Security Researcher at CultureAI

The Rapid Adoption of DeepSeek

As GenAI adoption ramped up across the business, this fast-moving fintech hit a familiar wall: how to let teams explore tools like ChatGPT and Gemini without exposing sensitive data or breaching compliance rules.

Despite having modern DLP and CASB tools in place, they lacked the behavioural insights and real-time context needed to guide employee use of GenAI tools. Shadow AI use was growing, and SecOps lacked clear visibility into which incidents required intervention.

They turned to Culture AI, the Secure AI Usage Enablement Platform, to solve this.

The Challenge: Shadow AI, Manual Triage, and Cultural Friction

By mid-2025, the fintech’s teams were already using GenAI widely - but not always safely:

The Solution: Secure GenAI Usage Without Lockdown

The deployment was fast - live in under 6 hours - and began delivering impact almost immediately.

Beyond Detection: Shaping Secure AI Culture

One of the most valuable outcomes was the cultural shift:

With initial success validated, the fintech is:

This case study shows how a leading fintech organisation used our platform to gain control over generative AI usage without blocking innovation. By focusing on human behaviour, real-time intervention, and team-level trends—not just raw telemetry or just blocking access—the company reduced AI-related risk, saved time for security teams, and laid the groundwork for scalable, compliant AI adoption.

Want to learn more about how Culture AI can help your teams use AI safely, securely, and smartly? Book a Demo 

Learn how a global fintech company stopped over 20 sensitive data leaks to GenAI tools daily—without blocking innovation. This case study explores how Culture AI delivered real-time risk insights, behaviour-led coaching, and automated triage to enable secure, scalable AI adoption across 3,600 employees.

Empowering Safe GenAI Adoption at a 3,600-Employee Fintech — And Stopping 20+ Data Leaks a Day

Empowering Safe GenAI Adoption at a 3,600-Employee Fintech

As generative AI tools such as ChatGPT became widely available, employees at a prominent international law firm began using them informally across functions - drafting documents, updating CVs, and exploring ways to automate research. While the firm had implemented its own internal AI assistant based on OpenAI, leadership knew that unsanctioned usage of external tools - also known as shadow AI - was growing fast.

The firm’s CISO recognised that these tools were being used with good intent but posed significant risks. Confidential client data, PII, and even fragments of source code could potentially be exposed to public AI models, breaching privacy obligations and regulatory standards. At the same time, a blanket ban on GenAI tools was seen as overly restrictive and likely to drive such behaviour further underground.

“We were caught between risk and reality,” the CISO explained. “Our people wanted to experiment with AI to work more efficiently. We just needed to ensure it was being done safely.”

What the firm needed was a way to see exactly what tools were being used, identify when risk was introduced, and guide employees in real time, all without blocking innovation or adding friction to daily work.

The firm partnered with CultureAI to implement a secure AI usage enablement strategy that combined visibility, proactive protection, and behaviour change at scale.

CultureAI was deployed to track GenAI activity across the web, including both sanctioned and shadow tools. Usage was categorised and surfaced through a central dashboard, allowing the security team to identify which GenAI applications were being used, by whom, and for what types of tasks. Crucially, tools like ChatGPT, Claude, and Gemini were flagged automatically, and new ones could be detected as soon as they appeared.

To tackle high-risk behaviours, CultureAI introduced real-time controls that could detect when users were about to paste sensitive information - such as PII, source code, or confidential client data - into GenAI tools. These prompts were either blocked automatically or intercepted with a customised in-browser coaching message explaining the risk and suggesting safer alternatives, such as the firm’s internal AI assistant.

This in-the-moment intervention proved far more effective than after-the-fact alerts. Employees received contextual guidance while working, helping them understand the organisation’s expectations and reinforcing safe AI use.

CultureAI also helped the firm automate triage for critical risks. When source code or regulated data was detected, alerts were routed to the security team via lightweight playbooks directly into existing SIEM workflows. Meanwhile, the platform’s “audit mode” allowed the firm to see broader trends in AI usage, helping the CISO evaluate policies based on real data before enforcing changes.

Privacy concerns were addressed through CultureAI’s privacy-safe approach, which ensured that employee actions were monitored without invasive logging and that no sensitive documents were used to train any models.

Over the first three months, the firm saw measurable risk reduction and increased employee awareness:

Importantly, the firm did not need to default to a full ban on public GenAI use. Instead, the security team could justify its position with real data, and interventions were tailored to specific behaviours rather than applied indiscriminately.

“Our goal wasn’t to shut everything down,” said the CISO. “We wanted to enable AI safely, not stifle it. CultureAI gave us the confidence to do that.”

With the foundation in place, the firm is planning to expand its approach by:

This case demonstrates that even in highly regulated, risk-sensitive environments, organisations can embrace the future of work—so long as they pair visibility with real-time behavioural intervention. With CultureAI, the firm didn’t just reduce AI risk. It helped build a safer, smarter culture of innovation. 

Discover how a leading international law firm secured generative AI (GenAI) usage without sacrificing productivity. Facing growing concerns over shadow AI tools like ChatGPT, Bard, and Claude, the firm partnered with CultureAI to gain full visibility into GenAI activity, prevent data loss, and coach employees in real time. Within just three months, they blocked 98% of high-risk AI submissions—such as PII, source code, and confidential data—while avoiding a blanket ban.

Learn how CultureAI enabled proactive GenAI governance through browser-based controls, automated risk triage, privacy-first monitoring, and behaviour-based nudging. The firm identified 20+ shadow tools, delivered over 120 monthly in-the-moment interventions, and created a safer culture of AI innovation—proving that regulated industries can adopt GenAI responsibly without compromising compliance.

How an International Law Firm Prevented 98% of High-Risk GenAI Submissions Without Locking Down Innovation

Case Study: How an International Law Firm Prevented 98% of High-Risk GenAI Submissions Without Locking Down Innovation

When a fast-scaling digital bank began seeing widespread employee adoption of generative AI tools like ChatGPT and Gemini, their security team faced a growing dilemma: how do you protect sensitive data without shutting down innovation?

Despite having a mature security stack — including SIEM, DLP, CASB, and MDM - the team lacked clear visibility into how AI was being used, especially when accessed through personal accounts on corporate devices. Risky behaviours, like copying internal code into AI tools or reusing passwords across unmonitored SaaS apps, weren’t being detected in real time. And even when alerts were triggered, the signal-to-noise ratio was poor — overwhelming the SecOps team and eroding trust in the data.

The bank needed a new approach: not just another tool to block usage, but a smarter, more nuanced way to 

From Risk to Resilience: Enabling GenAI Without Fear

By deploying our secure AI usage enablement platform, the bank was able to quickly gain visibility and control over employee AI behaviour - without slowing productivity.

The first step was understanding who was using AI, how, and where. Our platform provided real-time telemetry on prompt-level activity, flagging use of generative AI tools via personal accounts on corporate machines - a major data loss vector that existing tools couldn’t see.

Next, the security team created role-based policies. Sets of employees from specific departments licensed to use Gemini were placed in an “approved” group, while unlicensed users remained under stricter monitoring. This conditional logic allowed the platform to suppress noise and 

, while still escalating the risks that truly mattered.

Rather than punishing users or forcing them to work around controls, the bank used real-time browser nudges to guide behaviour at the moment of risk. If someone tried to paste source code into an AI tool, for example, a discreet message explained the risk and offered a course-correct.

Over the first month, the platform delivered more than 

, helping employees learn what was safe and what wasn’t, without slowing them down.

In parallel, the platform’s SaaS visibility layer surfaced several AI-enabled tools that had been adopted organically across the business. In one instance, more than 100 employees had accessed a new AI-powered productivity app that wasn’t part of the approved stack. The security team received an alert before the tool gained deeper traction, allowing them to assess the risk and take appropriate action - all within a few days, rather than weeks.

In just a few weeks, the bank saw tangible results:

Buoyed by the early success, the bank is now planning to expand deployment across customer support and operations, and introduce deeper integrations into their security tech stack. 

Their journey reflects a broader shift happening across the industry: from blocking AI to 

enabling it — responsibly, safely, and at scale

Want to understand how Culture AI can help your business? Book a call with our team

See how a high-growth digital bank protected sensitive data while enabling safe GenAI adoption across the enterprise. Faced with growing employee use of tools like ChatGPT and Gemini, the bank’s security team needed more than traditional DLP or SIEM. They lacked visibility into shadow AI usage, especially via personal accounts, and were overwhelmed by noisy alerts.

By partnering with CultureAI, the bank deployed a secure AI usage enablement platform that delivered real-time visibility, contextual nudges, and role-based risk policies — without blocking tools or slowing teams down. Within weeks, they reduced shadow AI usage by 80%, eliminated false positives, improved SaaS discovery time by 70%, and delivered over 2,000 just-in-time behavioural interventions.

This case study highlights how modern security teams can manage AI risk without creating friction -  by shifting from control to enablement and building trust with employees.

How a Digital Bank Reduced Shadow AI Risk by 80% - Without Blocking Innovation

Case Study: How a Digital Bank Reduced Shadow AI Risk by 80% — Without Blocking Innovation

Discover our latest blogs, insights and cyber security tips from our team of experts. 

Blog & Case Studies

🚀Stay in the loop

The AI Hunger Games:

Empowering Safe GenAI Adoption at a 3,600-Employee Fintech — And Stopping 20+ Data Leaks a Day

How an International Law Firm Prevented 98% of High-Risk GenAI Submissions Without Locking Down Innovation

How a Digital Bank Reduced Shadow AI Risk by 80% - Without Blocking Innovation