8.5 AI, Human Rights, and National Security Issues

The Boundaries of Intelligence: AI’s Profound Impact on Human Rights and National Security

Artificial Intelligence (AI), as a General-Purpose Technology (GPT) with infinite potential and extremely broad impact, its rapid development and deepening societal penetration have effects extending far beyond mere economic efficiency gains, industrial upgrades, or technological innovation itself. It is beginning to touch upon, intertwine with, and profoundly affect the two most core, fundamental pillars upon which modern civilized society exists and sustains itself: the system for guaranteeing Fundamental Human Rights, centered on individual dignity, freedom, and equality, and the mechanism for maintaining National Security, centered on state sovereignty, territorial integrity, political stability, and social order.

Undoubtedly, AI technology demonstrates positive potential for promoting the realization of human rights. For instance, AI-assisted medical diagnosis can improve the accessibility and quality of healthcare services, thus better safeguarding the right to health. AI-driven assistive technologies (like smart prosthetics, speech recognition/synthesis) can help people with disabilities overcome barriers and participate more fully in social life, promoting their right to equality. AI applications in education might also provide higher quality, more personalized learning resources for remote areas or disadvantaged groups, contributing to educational equity.

However, on the other hand, the improper design, unregulated deployment, or malicious misuse of AI technology could also pose unprecedented, systemic, and sometimes imperceptible threats to individuals’ fundamental rights and freedoms. From the erosion of the right to privacy by ubiquitous intelligent surveillance, to the challenge posed by algorithmic bias to the rights to equality and non-discrimination, the potential impact of personalized recommendations and content moderation on freedom of expression and thought, and even the future impact of more advanced AI on fundamental rights like the right to life and human dignity, all compel us to deeply reflect on and effectively regulate AI’s human rights implications.

Simultaneously, at the national level, AI’s critical, strategic applications in areas such as defense and military intelligence, critical information infrastructure protection, national intelligence analysis and counter-terrorism capabilities, cybersecurity offense and defense, and enhancing the modernization of state governance systems and capabilities make it indisputably a key enabling technology for measuring and enhancing national comprehensive strength, safeguarding core national security interests, and influencing international strategic standing and discourse power in global governance. Mastering advanced AI and effectively applying it in national security has become a focal point of strategic competition among major powers.

Yet, this powerful enabling force is inevitably accompanied by novel, complex, potentially even disruptive national security risks and challenges. From the potential for AI technology gaps to exacerbate international strategic imbalances, to the emergence of AI-driven new forms of cyber attacks and hybrid warfare, the ethical catastrophe and conflict escalation risks potentially brought by lethal autonomous weapons systems, and the new vulnerabilities faced by critical infrastructure due to reliance on AI, all pose severe tests to traditional national security concepts and governance models.

Deeply understanding the extremely complex impacts and potential risks of AI on these two grand, intertwined, and sometimes potentially conflicting major issues (e.g., surveillance measures for national security might infringe on individual privacy rights) is crucial for every member of our society—be they legal professionals, policymakers, technology developers, business managers, educators, or ordinary citizens. Only with a full awareness of these impacts and risks can we collectively strive, through effective legal frameworks, responsible technology ethics, sound governance mechanisms, and open social dialogue, to guide the direction of AI development. The goal is to build a truly human-centric intelligent future that can fully enjoy the prosperity, security, and convenience brought by technological progress, while also steadfastly upholding fundamental human rights, ensuring social fairness, and defending the foundations of the rule of law.

I. Potential Impacts & Protection Challenges of AI on Fundamental Human Rights: A “New Exam” for Rights in the Intelligent Era

The widespread application and deep integration of AI technology present a novel, challenging “examination paper” to the system for guaranteeing fundamental human rights, which human society has long strived to build and maintain. We need to carefully examine in what ways and how AI technology might profoundly impact our cherished core human rights (both positively and negatively), and how we should update concepts, adjust institutions, and apply legal and ethical norms to address these challenges, ensuring these fundamental rights are not eroded but perhaps even better realized and protected in the intelligent age.

• Right to Privacy - Risking “Stranding” in the Ubiquitous Data Flood:

  • Core Risk: AI technology, especially its core driver—the ability to collect, store, process, analyze, correlate, and predict based on massive data—poses arguably the most severe and systemic threat to the right to privacy since modern society began. Privacy, as a foundational right for individuals to maintain tranquility, preserve personal dignity, and achieve self-development, faces immense risks of being comprehensively, deeply, and continuously eroded in the age of AI.
    • Ubiquitous Automated Surveillance: AI dramatically lowers the cost and technical barriers for implementing large-scale surveillance, while enhancing its efficiency and scope. E.g.:
      • Numerous cameras deployed in city streets, public spaces, transportation, even private spaces (smart home devices), combined with AI facial recognition, human pose estimation, gait recognition, etc., make continuous, automated tracking and analysis of individuals’ physical movements, social interactions, behavioral patterns technically feasible.
      • Developments in AI speech recognition and voice biometrics also increase the potential risks of mass eavesdropping and analysis of phone calls, online meetings, even public conversations.
      • In the digital space, AI-driven web crawlers, user tracking technologies (cookies, fingerprinting), social media content analysis (sentiment analysis, relationship mining), etc., enable systematic collection, integration, and deep profiling of individuals’ browsing history, search queries, shopping preferences, social connections, expressed opinions, even emotional states. This omnipresent, automated surveillance potential covering physical and digital spaces, online and offline activities, throughout all aspects of life, increasingly fosters a sense of anxiety of “nowhere to hide” and continuous compression of personal space.
    • Intrusive Profiling & Sensitive Inference based on AI: AI algorithms (especially complex ML models) excel at inferring extremely private, highly sensitive, sometimes even subconsciously held deep information about individuals by mining hidden statistical patterns and complex correlations from seemingly ordinary, fragmented, non-sensitive user data. E.g., based solely on web browsing history, online shopping carts, social media likes/follows, even typing rhythm and style, AI models might infer with considerable probability an individual’s health status (e.g., having a certain disease), political leanings, religious beliefs, sexual orientation, true financial situation (income, debt), family relationships, personality traits, potential psychological vulnerabilities, or even predict future behaviors (propensity to resign, purchase intent, crime risk). This “digital mind-reading” capability allows for extremely deep profiling, potentially biased or inaccurate, often without the user’s full knowledge or explicit consent, with results potentially used for various purposes (from targeted marketing to discriminatory decisions).
    • Persistent Threats of Data Breach & Misuse: (Detailed in Sections 6.1, 6.2) AI systems and their operating environments, storing and processing vast personal data, inherently face continuous risks of large-scale data breaches from external hacking, internal malicious theft or mishandling, system security vulnerabilities, or management negligence. Once leaked, sensitive personal data can lead to severe consequences for innumerable individuals, including identity theft, financial loss, reputational damage, even threats to physical safety. Additionally, even without breaches, data risks being improperly used or misused by collectors or their partners (e.g., using data for purposes not consented to, discriminatory pricing/servicing, sharing with third parties without legal basis).
  • Human Rights Protection Challenges & Responses: Protecting privacy in the AI age requires a multi-layered, multi-dimensional comprehensive defense system:
    • Robust, Evolving Data Protection Legal Frameworks: Enact and strictly enforce data protection laws (like EU GDPR’s strict principles, China PIPL’s clear requirements for consent, sensitive PI handling, automated decision-making, cross-border transfer) capable of effectively regulating AI-era data processing activities. Laws need to clearly define legal bases for processing, core principles (purpose limitation, minimization, transparency), data subject rights (informed consent, access, rectification, erasure, portability, objection to automated decisions), and data controllers’/processors’ security obligations and liabilities. The framework must also remain flexible and forward-looking to adapt to rapid technological change.
    • Strict Regulation of Intrusive Surveillance Technologies: Impose extremely strict legal regulations on all forms of highly intrusive AI surveillance technologies (e.g., large-scale facial recognition, social credit scoring, predictive policing), especially those potentially deployed by government authorities for public security, counter-terrorism, or crime investigation purposes. Ensure their deployment and use have full legal basis, pursue legitimate aims, strictly adhere to principles of necessity and proportionality (interference with privacy must be proportionate to the goal, no less intrusive alternatives available), are subject to independent, effective judicial or administrative oversight and remedy mechanisms, and operate with maximum possible transparency.
    • Embedding Privacy Protection into AI Design & Development (Privacy by Design & by Default): Proactively and systematically embed privacy protection principles and requirements into the entire lifecycle of AI system design, development, and deployment, rather than as an afterthought. This includes:
      • Data Minimization: Collect only data strictly necessary for achieving specific, legitimate purposes from the outset.
      • Promote Use of Privacy-Enhancing Technologies (PETs): Actively explore and apply technologies that enhance data privacy without (or acceptably reducing) compromising data utility. Examples include:
        • Differential Privacy: Adding precisely controlled random noise during statistical analysis or model training to prevent inference of individual sensitive info from results.
        • Homomorphic Encryption: Allowing computation directly on encrypted data without decryption, protecting data during processing.
        • Federated Learning: A distributed ML paradigm allowing multiple parties to collaboratively train a model without sharing their local raw data. Training happens locally; only model updates (not raw data) are shared and aggregated.
        • Secure Multi-Party Computation (MPC): Enabling multiple mutually distrusting parties to jointly compute a function without any party learning anything beyond their own input and the final output.
        • Zero-Knowledge Proofs: Allowing one party (prover) to prove to another (verifier) that they know a secret or a statement is true, without revealing any additional information about the secret/statement itself.
      • Highest Privacy Protection by Default: System default settings should be configured for maximum user privacy protection; users need to actively opt-in to lower protection levels or share more information.
      • Conduct Privacy Impact Assessments (PIAs): For AI applications potentially posing high risks to privacy, conduct systematic PIAs before deployment.
    • Effectively Guaranteeing Individuals’ Data Subject Rights: Establish convenient, effective channels and procedures enabling individuals to genuinely exercise their legal rights over their personal data, including rights to be informed, access, rectify, erase (“be forgotten”), restrict or object to processing, withdraw consent, data portability, and (for significant automated decisions) request human intervention, express views, and seek explanation.

• Right to Equality and Non-discrimination - Beware Algorithms Entrenching or Amplifying Social Injustice:

  • Core Risk: Algorithmic Bias poses one of the most significant, insidious, potentially far-reaching threats from AI technology to the fundamental human rights of equality and non-discrimination (sources and manifestations discussed in Sections 6.1, 6.4, 6.5).
    • Automated Reproduction & Systemic Entrenchment of Discrimination: If AI systems are trained on “biased” data reflecting historical or existing structural discrimination and inequality (e.g., based on race, gender, region, class), or if their algorithm design fails to adequately consider fairness goals, they are highly likely to unconsciously, systemically replicate, perpetuate, or even amplify these pre-existing discriminatory patterns in their automated decisions (e.g., in critical areas impacting life chances like hiring screening, credit approval, insurance pricing, housing eligibility, education resource allocation, judicial risk assessment). This risks turning technology, which should promote fairness, into an accomplice aggravating social injustice and hindering upward mobility for disadvantaged groups.
    • Digital Divide Exacerbating Opportunity Inequality: Beyond algorithmic bias itself, AI application can also worsen social inequality through the “Digital Divide.” Groups who, due to age, education, economic status, geographic location (lack of network coverage in remote areas), disability, or language barriers, cannot equally and effectively access or utilize advanced AI technologies and related services, might fall further behind those proficient with these tools in terms of accessing information, education, job opportunities, healthcare, participating in social life and public affairs. This could lead to new social stratification based on digital competence and further inequality of opportunity.
  • Human Rights Protection Challenges & Responses: Ensuring equality and non-discrimination in the AI era requires multi-pronged efforts:
    • Strengthen & Enforce Robust Anti-Discrimination Legal Frameworks: Ensure existing anti-discrimination laws (constitutional equality principles, specific laws in labor, education, credit, housing) can be clearly applied to decisions made or disparate impacts caused by AI systems. Pay special attention to addressing “Indirect Discrimination” or “Disparate Impact” (neutral practices having disproportionate negative effects on protected groups without sufficient justification), which is harder to prove intent for.
    • Develop & Promote Algorithmic Fairness Assurance Technologies & Governance Tools: Continuously invest in R&D and promote technical tools for effectively detecting, assessing, and mitigating algorithmic bias (fairness metrics libraries, bias audit tools, debiasing algorithms) and governance processes (integrating fairness assessment into model lifecycle, independent ethical review boards).
    • Proactively Promote Inclusivity & Accessibility in AI Design & Deployment: When developing/deploying AI (especially for public services or affecting basic rights), proactively consider and strive to promote its Inclusivity (e.g., ensuring diverse, representative training data; ensuring algorithms work equally well across groups) and Accessibility (e.g., multilingual interfaces, assistive features for disabilities, easy-to-use interaction designs). Pay special attention to the unique needs of vulnerable groups to avoid technology deepening social exclusion.
    • Invest Heavily in Bridging the Digital Divide: Requires joint efforts from governments, businesses, educational institutions, civil society through expanding digital infrastructure, providing accessible digital skills training, developing user-friendly AI tools, and offering necessary economic/technical support to actively bridge the digital divide in AI access and capability across different populations, ensuring benefits of AI development are shared more broadly.

• Freedoms of Expression, Thought & Access to Information - New Challenges Amidst Algorithmic Filtering, Recommendation & Generation:

  • Core Risk: AI technologies (especially LLMs and recommendation algorithms) play increasingly central roles in modern information production, dissemination, and consumption, posing novel, complex challenges to traditional freedoms of expression, thought, and access to information:
    • Risk of “Over-blocking” & “Selective Silence” from Automated Content Moderation: To manage massive user-generated content and legal risks, online platforms increasingly rely on AI algorithms for large-scale, automated Content Moderation to identify and remove (or limit spread of) illegal, harmful, or policy-violating content. However, due to AI’s significant limitations in understanding complex context, distinguishing satire from malice, grasping nuanced cultural boundaries, and recognizing novel/subtle harmful info, and the often opaque rules and processes of automated moderation, large amounts of legitimate, valuable, even merely controversial speech, opinion, or artistic expression can be mistakenly removed, blocked, down-ranked, or flagged (“false positives” or “over-blocking”). This algorithm-driven, large-scale, sometimes opaque censorship can severely suppress legitimate freedom of expression and diversity of views. In some cases, such systems could even be abused for targeted political censorship or suppression of dissent.
    • Potential for “Filter Bubbles” & Societal Polarization from Personalized Recommendation Algorithms: Algorithms personalizing content recommendations based on user history, preferences, social connections, while greatly improving information matching efficiency and user engagement, also risk trapping users persistently and unconsciously within “Filter Bubbles” or “Echo Chambers” aligned with their existing views, interests, and social circles. In such environments, information exposure becomes highly homogenous, significantly reducing opportunities to encounter diverse perspectives, counter-arguments, or varied information beyond their usual sphere. In the long run, this can exacerbate individual cognitive biases and rigidity, and worsen societal polarization, group antagonism, and the difficulty of rational public discourse.
    • Erosion of Right to Information & Rational Decision-Making by AIGC-driven Disinformation: Generative AI makes it possible to mass-produce, cheaply and efficiently, various forms of disinformation, deepfakes, online rumors, and political propaganda. This not only severely pollutes the authenticity of the entire information ecosystem, making it hard for the public to discern truth, but also greatly interferes with citizens’ ability to engage in rational thought, participate in public debate, and make informed decisions (e.g., voting, consumption, health) based on reliable facts, thus fundamentally undermining their effective right to information.
  • Human Rights Protection Challenges & Responses: Protecting freedoms of expression, thought, and information in the AI age requires efforts on multiple fronts:
    • Seeking a Delicate Balance in Content Governance: Need to find an extremely delicate, continuously adjusted balance, supported by clear legal basis and procedural safeguards, between effectively governing and restricting the spread of clearly illegal, severely harmful information (incitement to violence, hate speech, child exploitation, terrorism propaganda, proven major harmful disinformation) versus maximally protecting legitimate, diverse, even controversial or offensive freedom of expression and space for debate. Any content restriction measures must strictly adhere to principles of legality, necessity, and proportionality.
    • Enhancing Transparency, Accountability & Redress Mechanisms for Algorithmic Recommendation & Moderation: Platforms need to increase transparency of their recommendation algorithms and automated moderation rules/processes (e.g., explain to users why content recommended/removed). Establish clear, effective, user-friendly appeal mechanisms allowing users to challenge content decisions and receive timely, reasonable handling. Strengthen external oversight and accountability for platform content governance practices.
    • Developing & Responsibly Applying Technologies to Identify Disinformation: Invest heavily in R&D for more effective, accurate technical tools to identify and label (ideally label, not just delete, respecting user judgment) disinformation, especially deepfakes. Explore how to responsibly deploy these tools in the information chain without infringing privacy or free speech (e.g., displaying “potentially false info” warnings).
    • Strengthening Media Literacy & Critical Thinking Education Across Society: Improving citizens’ (from youth to adults) ability to discern information authenticity, assess source credibility, understand algorithmic mechanisms, and engage in independent, rational, critical thinking is the most fundamental and sustainable way to resist the flood of disinformation, escape filter bubbles, and maintain a healthy public sphere.

• Freedom of Thought, Conscience and Religion - Potential Intrusion into Integrity of the Inner Realm:

  • Core Risk: AI development, particularly cutting-edge applications attempting to directly or indirectly detect, interpret, or even influence human inner thoughts, emotions, beliefs, or decision-making processes, could pose potential, profound threats to this freedom often considered most fundamental, most private, the basis of individual personality integrity (“last fortress of the inner realm”).
    • Risks of Misusing AI Emotion Recognition & Intent Inference: Using AI Emotion Recognition (from facial expressions, voice tone, possibly physiological signals like heart rate variability, galvanic skin response) or Intent Inference techniques (crucially, the scientific validity, accuracy, cross-cultural universality of these techs are highly debated) if misused in certain contexts (e.g., to judge if suspects are “lying” in interrogations, monitor employee “emotional state” or “loyalty,” or for mass political/social surveillance to identify “dissidents” or “potential threats”), could severely infringe individuals’ freedom of thought, right to remain silent, and right to be free from undue mental interference.
    • Future Risks from Neurotechnology & “Mind Reading”: More advanced future Brain-Computer Interfaces (BCIs) combined with powerful AI algorithms could theoretically potentially (though reliable implementation is likely very far off) achieve deeper decoding of individual brain activity signals, allowing some degree of “reading” or inferring thoughts, intentions, even subconscious states. This possibility of “Mind Reading” undoubtedly poses the most fundamental, unprecedented challenge to freedom of thought and mental privacy.
    • Personalized Manipulation or Thought-based Discrimination via Deep Profiling: If AI deep profiling based on massive personal data can accurately infer individuals’ religious beliefs, political stances, core values, personality weaknesses, or psychological vulnerabilities, this highly sensitive personal information could be exploited by malicious actors (certain companies, political groups, foreign powers) for:
      • Highly customized, extremely subtle, hard-to-detect manipulative propaganda or psychological interventions aimed at changing their beliefs, thoughts, or voting behavior.
      • Implicit discrimination based on their thoughts, beliefs, or values in areas like employment, credit, insurance, education, even social interactions, which is hard to prove.
  • Human Rights Protection Challenges & Responses:
    • Strict Regulation of Intrusive Neuro- and Emotion-Technologies: Must impose extremely strict ethical review and legal regulation on all AI technologies potentially directly probing, interfering with, or manipulating individuals’ inner thoughts, emotional states, or mental activities (esp. invasive neurotech, emotion recognition, intent inference from biosignals). Set very high application thresholds, explicitly prohibit illicit or unethical uses, and establish strong oversight and accountability.
    • Exploring Emerging Rights like “Cognitive Liberty”: Faced with these frontier tech challenges, discussions are emerging in international human rights law and legal theory about the need to explicitly recognize and protect “Cognitive Liberty” or “Mental Privacy” as a new, distinct fundamental human right. This might include freedom of thought from external coercion/manipulation, right to privacy of mental state information from illicit access/use, etc. Future theoretical construction and rule development at international and constitutional levels might be needed.

• Right to Work, Decent Work & Social Security - Dual Pressures from Automation Impact & Algorithmic Management:

  • Core Risk: (Impact on jobs/skills discussed in Section 6.6)
    • Structural Unemployment & Transition Pressure from Automation: AI-driven automation and intelligence wave might, in coming decades, significantly displace large numbers of jobs currently performed by humans (esp. routine, repetitive tasks based on information processing or simple manual operation). This could lead to large-scale structural unemployment, posing immense difficulty and survival pressure for workers whose skills become obsolete, directly threatening their Right to Work and right to earn a decent, stable income for a basic living standard.
    • Erosion of Labor Rights by Algorithmic Management: AI is increasingly used by employers for workplace management and control, e.g., via algorithms for intelligent scheduling, automated task assignment, real-time productivity monitoring, employee behavior pattern analysis, automated performance reviews, even termination decisions. If the design or application of these “Algorithm Bosses” lacks transparency, fairness, human consideration, and effective grievance mechanisms, it could severely infringe workers’ privacy rights (excessive monitoring), right to rest (unreasonable schedules), right to fair remuneration (opaque performance reviews), right to non-discrimination (biased algorithms), and basic labor dignity and autonomy.
  • Human Rights Protection Challenges & Responses:
    • Implement Active Labor Market Transition Policies: Requires joint efforts from government, business, unions, education institutions to implement proactive, forward-looking labor market policies. Includes: massive investment in providing future-oriented, high-quality vocational skills training and lifelong learning opportunities to help workers acquire skills needed for new AI-era jobs; establishing better unemployment early warning and re-employment assistance systems; encouraging and supporting creation of new jobs and employment forms.
    • Strengthen and Reform Social Security Systems: Need to seriously review and reform existing social safety nets (unemployment insurance, pensions, health insurance, minimum living standards) to make them more effective in coping with potential future larger-scale, longer-duration structural unemployment or income instability, providing a more reliable, resilient basic living safety net for all members of society. More exploratory solutions like Universal Basic Income (UBI) also warrant deep research and discussion regarding feasibility and impact.
    • Update Labor Laws to Regulate Algorithmic Management: Need to amend existing labor laws or enact new specific regulations to effectively regulate and oversee the use of algorithms in the workplace. E.g., require greater transparency of algorithmic management (explain how algorithms work and impact employees), ensure fairness and non-discrimination in algorithmic decisions, guarantee employees’ data privacy rights and right to appeal algorithmic decisions, and potentially use collective bargaining to balance rights/obligations regarding algorithm use.

• Right to Life, Health & Physical Security - Direct Threats from High-Risk AI Applications:

  • Core Risk: In certain specific high-risk domains, the failure, error, or misuse of AI systems could directly pose lethal threats to individuals’ right to life, health, or physical security.
    • Risks in Medical AI: As discussed, medical AI for diagnosis or treatment assistance, if making serious errors or exhibiting bias (missing life-threatening disease, recommending wrong treatment, surgical robot malfunction), could directly cause delayed treatment, improper harm, or even death.
    • Safety Risks of Autonomous Vehicles: AV sensor failures, perception algorithm errors, decision logic flaws, or system compromise by hacking could all lead to extremely severe traffic accidents causing serious injury or death to occupants, other road users, or pedestrians.
    • Fundamental Threat from Lethal Autonomous Weapons Systems (LAWS): AI application in the military domain, especially the development and potential deployment of Lethal Autonomous Weapons Systems (LAWS, “killer robots”) designed to autonomously select and engage targets with lethal force without real-time human intervention or final approval, poses the most fundamental and alarming threat to the right to life, ethics of war, and global security stability. Risks include algorithmic errors causing civilian deaths, accidental conflict escalation beyond control, difficulty assigning responsibility for war crimes, and triggering devastating arms races.
  • Human Rights Protection Challenges & Responses:
    • Impose Strictest Safety Regulations on High-Risk AI Systems: Must establish and enforce the most stringent safety standards, most thorough testing & validation processes, and most independent regulatory approval & post-market surveillance mechanisms for all high-risk AI systems directly or indirectly involving human life safety and physical health (esp. in healthcare, transportation, critical infrastructure, military). Safety must be the absolute prerequisite and non-negotiable bottom line for AI application in these domains.
    • Seek International Legal Regulation of Lethal Autonomous Weapons Systems (LAWS): The international community needs, through multilateral diplomatic efforts (e.g., within the UN Convention on Certain Conventional Weapons (CCW) framework’s Group of Governmental Experts meetings), to reach broadest possible consensus on the ethical and legal issues of LAWS, and strive to develop legally binding international rules to effectively regulate, strictly limit, or even completely prohibit lethal weapon systems lacking “Meaningful Human Control” and capable of autonomously deciding life and death.

• Human Dignity, Autonomy & Freedom of Mind - Beware the Future “Ruled by Algorithms”:

  • Core Risk: Beyond threats to specific rights, pervasive AI might, in deeper, subtler ways, challenge the very existential status of humans as independent, autonomous beings with inherent dignity:
    • “Outsourcing” Decision-Making & Loss of Autonomy: If we increasingly rely on AI algorithms for personalized recommendations, suggestions, or “optimal” solutions in all aspects of life—from news/music/product choices, to decisions on education/career paths, even personal health plans or investment strategies—will we gradually lose the willingness and ability for independent thought, information discernment, critical judgment, weighing pros and cons, and making autonomous choices? Are we unconsciously outsourcing our life decisions to algorithms?
    • Virtual Immersion & Real-World Alienation: If individuals become chronically, excessively immersed in highly personalized, instantly gratifying virtual information environments (filter bubbles), social networks, or immersive entertainment experiences (future metaverses) carefully crafted by AI algorithms, potentially deviating from reality, could this weaken their connection to the real physical world, genuine interpersonal relationships, and diverse social realities? Could it reduce their willingness and capacity to engage in public affairs and social responsibilities? Could it affect their sense of self and perception of “reality”?
    • Suppression of Free Spirit by Ubiquitous Surveillance & Scoring: If intrusive AI surveillance and associated social credit scoring systems or other forms of quantitative evaluation become ubiquitous, permeating all aspects of life, could individuals’ every word, action, even expressed thought be constantly recorded, assessed, “scored,” directly impacting their access to opportunities and resources? Could this invisible, pervasive monitoring and pressure significantly curtail individual freedom of action and mental space, forcing people into self-censorship, hiding true thoughts, conforming to singular “standard” behaviors to get “good scores,” thereby damaging inherent human dignity, creativity, and the basic freedom to pursue individual development?
  • Human Rights Protection Challenges & Responses:
    • Steadfastly Uphold “Human-Centricity” as Core Principle in AI Design & Application: Ensure technology development serves to enhance, not diminish, human capabilities, autonomy, and dignity.
    • Emphasize and Legally Guarantee Ultimate Human Control & Choice in Key Decisions: Especially in areas involving significant personal rights or public interest, ensure humans retain the right and ability to review, intervene in, override, and take final responsibility for AI decisions.
    • Be Vigilant Against Technology’s Over-Intrusion & Potential “Dehumanizing” Risks: Requires continuous societal reflection and discussion (in culture, ethics education, public policy) on the deep potential impacts of AI on humanity, maintaining social oversight and value guidance on technology’s direction, preventing it from ultimately developing contrary to human well-being.
    • Promote Digital Well-being: Encourage development and use of tools and practices that help users engage with digital technology in healthy, balanced, autonomous ways, resisting AI applications potentially leading to addiction, manipulation, or harm to mental health.

Human Rights Impact Assessment (HRIA) as a Governance Tool: To more systematically and comprehensively identify and address potential human rights risks from AI, it is increasingly advocated internationally (by human rights bodies and some regulators) that for planned AI systems potentially having significant adverse impacts on one or more fundamental human rights (especially those of vulnerable groups)—particularly systems implemented by government or widely deployed by large tech platforms—a dedicated, proactive Human Rights Impact Assessment (HRIA) should be conducted before the project starts or at key decision points, with participation from multiple stakeholders including representatives of affected groups. HRIA aims, through a structured process, to systematically identify all potential human rights risks (direct/indirect, intended/unintended) posed by the AI application throughout its lifecycle, assess their likelihood and severity, and proactively seek and design specific mitigation measures (technical, procedural, policy, legal) to effectively prevent, reduce, or remedy these negative impacts. Integrating HRIA as a key component of the AI governance framework helps embed respect for and protection of human rights earlier and deeper into the entire lifecycle of AI technology.

II. AI and National Security: The Complex Game of Empowering Defense, Maintaining Stability & Addressing New Global Risks

AI, as a typical dual-use General-Purpose Technology (GPT) with strategic enabling power, its development level and application capability not only profoundly affect a nation’s economic competitiveness and technological innovation capacity, but also increasingly directly relate to its national security posture, defense strength, international strategic standing, and discourse power in the global governance system. AI provides unprecedented powerful tools for enhancing a nation’s ability to safeguard its own security, but simultaneously, its rapid development and proliferation also generate novel, complex, potentially even disruptive national security risks and global challenges. Understanding this complex situation of coexisting empowerment and risk is fundamental for formulating effective national AI strategies and participating in global AI governance.

• Significant Enhancement of National Security Capabilities by AI:

  • Revolution in Intelligence Analysis & Prediction: Using AI (advanced ML, NLP, CV, knowledge graphs) enables automated processing and deep analysis of massive, heterogeneous, multi-modal intelligence data from diverse sources (satellite/drone imagery, SIGINT, OSINT, diplomatic cables, HUMINT reports) at unprecedented speed and accuracy. AI can:
    • Automatically detect and track targets of interest (military facilities, weapon systems, specific individuals/organizations).
    • Discover hidden patterns, potential correlations, or behavioral regularities in vast data imperceptible to human analysts.
    • Fuse multi-source information to build more comprehensive, accurate situational awareness pictures or intelligence assessments on specific regions/topics.
    • Assist in predictive analysis, e.g., assessing risks of potential military conflicts, terrorist attacks, political instability, or supply chain disruptions for critical resources (energy, food). This allows intelligence agencies to grasp information faster, more accurately, and more proactively, providing stronger support for national security decision-making.
  • Intelligent Upgrade of Cybersecurity Offense and Defense Capabilities:
    • On the Defense Side: AI is increasingly used to enhance the intelligence level of cyber defenses. E.g.:
      • Using ML for anomalous traffic detection to identify intrusions or malware spread earlier (Intelligent NIDS).
      • Automated Threat Hunting: AI assists security analysts in proactively searching vast logs/network data for unknown, latent threats.
      • Intelligent Security Orchestration, Automation and Response (SOAR): AI automates parts of security incident analysis, triage, and response processes, improving speed and efficiency.
      • Advanced Malware Analysis & Identification: Using AI for static/dynamic analysis of malware to identify family, function, potential harm.
    • On the Offense Side - (Risk Aspect): Need to be aware that attackers (state-sponsored hackers, cybercriminals, terrorists) can also leverage AI to significantly enhance the efficiency, stealth, and destructive power of cyber attacks. E.g.:
      • Using AI to automatically scan for and discover unknown vulnerabilities (Zero-days) in target systems.
      • Using AIGC (esp. LLMs) to generate more deceptive, harder-to-detect personalized phishing emails, fake login pages, or social engineering baits.
      • Potentially developing novel AI-driven malware or cyber weapons capable of autonomously adapting to network environments, evading detection, and executing attack tasks.
  • Enhancement of Border Security Control & Counter-Terrorism Capabilities:
    • Using AI-driven Computer Vision (high-accuracy facial/gait/vehicle recognition, anomaly detection) and Biometrics (fingerprint, iris, voiceprint) combined with big data correlation analysis and risk profiling, can significantly strengthen monitoring, identity verification, and automated identification/alerting of suspicious individuals (terrorism suspects, fugitives) at border crossings, airports, train stations, critical event venues, etc.
    • (Crucial Human Rights & Compliance Warning): Such applications directly involve collection and processing of vast amounts of personal information (esp. highly sensitive biometric data) of numerous citizens (domestic and foreign), and assessment results could directly impact individuals’ freedom of movement, liberty, even life safety. Therefore, their deployment and use must be conducted under the strictest legal authorization and regulatory framework, fully safeguarding individuals’ rights to privacy, non-discrimination, and procedural remedy, ensuring technology accuracy and fairness, and conducting thorough assessment and effective control of potential human rights risks. Any approach sacrificing fundamental human rights for absolute security is unacceptable.
  • Intelligent Protection of Critical Information Infrastructure (CII): Power grids, water supply, transportation networks (rail, air, ports), communication backbones, core financial systems, critical industrial control systems, etc., are the “nervous system” of modern nations and society, and potential primary targets for hostile actors or terrorists. AI technology can:
    • Enable real-time, high-precision monitoring of the operational status of these complex systems.
    • Predict potential equipment failures or performance degradation in advance for predictive maintenance, ensuring stable operation.
    • Intelligently identify anomalous operational patterns or early warning signals indicative of physical sabotage, operational errors, or cyber attacks.
    • Assist in rapid fault localization, isolation, and emergency recovery during attacks or failures. Thereby enhancing the overall security resilience and risk resistance of these vital national infrastructures.
  • Military AI & Profound Transformation of Future Warfare: This is the area where AI’s impact on national security is most direct and sparks most ethical/legal controversy. AI is being explored for application across virtually all aspects of military affairs with unprecedented depth and breadth, potentially fundamentally changing the nature, speed, and rules of future warfare:
    • Intelligence, Surveillance, and Reconnaissance (ISR): Using AI for real-time, automated target detection, recognition, classification, tracking from massive, multi-modal (imagery, video, radar, signals, sonar) surveillance data collected from various platforms (drone swarms, LEO satellite constellations, ground/underwater sensor networks), as well as rapid battlefield environment assessment and situation generation, greatly enhancing information acquisition and processing capabilities.
    • Intelligent Logistics, Equipment Maintenance & Force Planning: Using AI for weapon system health monitoring & predictive maintenance, optimizing smart scheduling & replenishment planning for ammunition, fuel, spares, etc., assisting in more scientific, flexible force deployment, war gaming, and course of action analysis, improving overall combat system efficiency and resilience.
    • Command and Control (C2) Decision Support: AI can serve as “intelligent staff officers” or decision support systems for commanders at various levels. By rapidly integrating and analyzing complex information from across the battlefield, it provides more comprehensive, accurate situational pictures, assists in evaluating potential benefits, risks, resource costs of different courses of action, potentially even recommending optimal actions, thereby shortening decision cycles (OODA Loop), improving command efficiency and decision quality (though final decision authority must remain with human commanders).
    • Development & Immense Controversy of Lethal Autonomous Weapons Systems (LAWS): This is the most watched and worrisome direction in military AI. LAWS typically refer to weapon systems capable of autonomously searching for, detecting, identifying, tracking, selecting targets, and ultimately deciding to use Lethal Force against them without real-time human intervention or final approval (e.g., autonomous attack drones, unmanned ground combat vehicles, autonomous underwater vehicles, or defensive systems).
      • Arguments for Potential Military Advantage: Proponents suggest LAWS might offer significant military advantages, such as reaction speeds far exceeding humans (potentially crucial in high-speed engagements), ability to operate in environments inaccessible or extremely dangerous for humans, reducing casualties for one’s own soldiers, and potentially being more “precise” and “rational” in adhering to rules of engagement than emotional human soldiers (?? highly debatable point).
      • Overwhelming Ethical & Legal Risks: However, opponents and mainstream international opinion raise extremely profound ethical concerns and legal challenges regarding LAWS, including:
        • Risk of Algorithmic Errors Causing Innocent Deaths: AI systems (esp. based on current tech) have far-from-perfect perception and identification capabilities in complex, dynamic, deceptive real battlefield environments. They might mistakenly identify civilians as combatants, civilian objects as military targets, or err in distinguishing surrendering soldiers from active ones, leading to unlawful attacks against persons and objects protected under International Humanitarian Law (IHL / Law of Armed Conflict, LOAC).
        • Inability to Satisfy Core IHL Principles: IHL requires adherence to principles of Distinction (combatants vs. civilians, military vs. civilian objects), Proportionality (incidental harm not excessive compared to concrete military advantage), and Precautions in Attack (taking all feasible measures to avoid/minimize civilian harm). Critics argue that autonomous weapon systems, lacking human complex contextual understanding, empathy, and value judgment capability, simply cannot, in practice, truly understand and reliably comply with these nuanced principles requiring sophisticated judgment.
        • Huge Risk of Accidental Conflict Escalation & Loss of Control: If opposing nations deploy large numbers of high-speed autonomous weapon systems, rapid, cascading conflict escalation triggered by algorithmic misjudgment or unintended interactions could occur at speeds far exceeding human commanders’ ability to comprehend, intervene, and de-escalate, potentially leading to regional conflicts spiraling out of control into major wars, or even, in extreme cases, risking nuclear escalation.
        • Fundamental Ethical Dilemma of “Killer Robots”: Delegating the ultimate decision to take human life entirely to a machine lacking morality, emotion, and genuine moral responsibility itself challenges the fundamental dignity of humans as moral agents and sparks deep ethical debates about the necessity of “Meaningful Human Control” over the use of lethal force.
        • Extreme Difficulty Assigning Responsibility for War Crimes: If an autonomous weapon system commits acts violating IHL (i.e., war crimes), who bears legal responsibility? The engineers designing the algorithm? The commanders approving deployment? The manufacturing company? Or the AI system itself (if granted some subjectivity)? Existing frameworks of international criminal law and state responsibility struggle to effectively address this issue.
        • Triggering Destructive Global Arms Race: LAWS development and deployment could trigger a new round of extremely destructive, potentially uncontrollable global arms race, making future wars more brutal, less predictable, and posing greater threats to human civilization. Consequently, the international community (e.g., within the UN CCW GGE meetings) is engaged in extremely difficult but vital negotiations and debates on how to effectively regulate LAWS legally and ethically, set clear limitations, or potentially even ban systems capable of fully autonomous lethal decision-making.

• Novel National Security Risks & Global Challenges Posed by AI:

  • AI Technology Gap Exacerbating International Strategic Imbalance: Significant disparities between nations in AI R&D investment, top talent pool, mastery of core technologies (advanced chips, foundation models), and ability to effectively integrate AI into economy and defense could further widen gaps in comprehensive national power, economic competitiveness, and military strength in the future. This might alter existing international power balances, exacerbate strategic competition and mistrust among major powers, potentially even inducing new geopolitical tensions or regional conflicts.
  • AI-driven Cyber Attacks, Information Warfare & Hybrid Warfare Become New Normal:
    • Intelligent Cyber Attacks: AI is being used to significantly enhance the efficiency, stealth, adaptability, and destructiveness of cyber attacks. E.g., using AI for automated vulnerability discovery, generating highly deceptive personalized phishing/social engineering baits, developing self-learning/mutating malware, or coordinating large-scale DDoS attacks.
    • Intensified Information Warfare & Cognitive Domain Confrontation: AI (esp. AIGC) has become a core tool in modern Information Warfare and “Cognitive Warfare” aimed at influencing target populations’ psychology, cognition, and decision-making. Hostile states or non-state actors can use AI to mass-produce, automate, even personalize the dissemination of fake news, deepfakes, political propaganda, socially divisive narratives, or psychological manipulation messages targeting specific groups, attempting to disrupt social order, undermine national unity, interfere with democratic processes (elections), weaken morale and resistance, or internationally discredit rivals. This “warfare hitting minds without smoke” is an increasingly severe new threat to national security.
  • New Vulnerabilities for Critical Information Infrastructure (CII) from AI: As CII sectors (power grids, transport, finance, telecom, energy, water) increasingly rely on complex AI systems for automated control, optimized scheduling, and status monitoring, these AI systems themselves become new, highly attractive targets for attack. Potential software vulnerabilities, algorithmic flaws, or compromise/manipulation of these systems by malicious actors (via cyber intrusion, supply chain attacks, insider threats) could lead to catastrophic consequences (widespread blackouts, transport paralysis, financial market collapse, communication breakdown), directly threatening national operations and public safety. Thus, protecting the security and resilience of AI systems in these critical domains has become a core component of national cybersecurity strategies.
  • Risks of Loss of Control & Accidental Escalation from Lethal Autonomous Weapons Systems (LAWS): As mentioned, potential deployment of LAWS could dramatically shorten decision timeframes in conflict initiation and escalation. Their behavior in complex, chaotic, uncertain, deceptive real battlefield environments might be hard to fully predict. More worryingly, if opposing nations deploy large numbers of high-speed autonomous weapon systems, extremely rapid, cascading conflict escalation triggered by algorithmic misjudgment, unintended interactions, or positive feedback loops could occur at speeds far exceeding human commanders’ ability to comprehend, intervene, and control, potentially leading to regional conflicts uncontrollably expanding into major wars, or even risking nuclear use in extreme scenarios. Ensuring “Meaningful Human Control” over all lethal force systems is considered key to preventing such catastrophic scenarios.
  • Risk of Proliferation of Advanced AI Tech & Exacerbation of Asymmetric Threats: If powerful AI technologies (advanced facial recognition algorithms, easy-to-use deepfake tools, potentially weaponizable open-source AI models/code) proliferate uncontrollably, or fall into the hands of terrorist organizations, transnational criminal groups, or irresponsible rogue states, they could be used to conduct more efficient, stealthy, harder-to-defend terrorist attacks, organized crimes, or disruptive/subversive activities against other nations. This could grant asymmetric threat capabilities disproportionate to traditional power to weaker non-state actors or small states, bringing new uncertainties to the global security landscape.
  • Supply Chain Security & National Technological Sovereignty Risks for Core AI Tech/Resources: Development and operation of high-performance AI systems are highly dependent on a few key core technologies and resources, such as most advanced AI chips (high-end GPUs/TPUs), high-end semiconductor manufacturing equipment (advanced lithography machines), core Electronic Design Automation (EDA) software, and the ultra-large foundation models and massive high-quality training datasets controlled by a few tech giants. If a nation becomes overly reliant on importing these critical, strategic AI supply chain elements from a few foreign countries (especially potential strategic rivals), it faces huge risks of supply chain disruption (e.g., due to tech embargoes or export controls), concerns about embedded security backdoors, or being “choked” on critical technologies during international tensions, geopolitical conflicts, or trade disputes. This could jeopardize its own AI development autonomy and national security. Thus, securing the safety, diversity, and resilience of domestic AI core technology supply chains, striving for self-reliance in key segments, has become a major component of national AI strategies for leading countries.
  • Data Security & Protection of National Data Sovereignty: Data is the “new oil” of the AI era. For a nation, its core, sensitive, strategically valuable large-scale data (e.g., population & health data of all citizens, national geospatial mapping data, economic operational data of key industries, operational data of national energy grids & transport hubs, important scientific research data, core data related to national defense & government operations) if massively collected, processed, analyzed, or transferred cross-border by foreign governments, foreign companies, or AI systems under their control (whether deployed domestically or abroad) without adequate regulation, could directly threaten national data sovereignty, economic security, social stability, and even core national security interests (e.g., critical info used for intelligence analysis, strategic decision-making, or targeted attacks). Therefore, establishing and refining legal frameworks to strictly manage cross-border flows of important data and personal information (e.g., China’s DSL, CSL, PIPL and related systems for security assessment, standard contracts), clarifying the meaning and protection measures for national data sovereignty, has become a top priority in data governance and national security strategies for many countries (esp. China and EU).

• National AI Governance Strategies & Increasingly Complex International Strategic Dynamics:

  • Formulating & Implementing National AI Development Strategies: Major countries and regions worldwide (including China, US, EU, UK, Japan, South Korea, Singapore, etc.) have all issued or are actively developing comprehensive national AI strategic plans. These typically define mid-to-long term development goals, priority technology directions, key application sectors, talent cultivation/attraction plans, and corresponding policy support measures. They also usually include principled frameworks for AI ethics, risk governance, and legal regulation, aiming to secure advantageous positions in the global AI race, enhancing national economic competitiveness, technological innovation capacity, and national security strength.
  • Using “Strategic Tools” like Technology Export Controls & Investment Security Reviews: Driven by complex considerations of maintaining national security, preserving critical technological leadership, or containing strategic rivals, some countries (notably the US) have increasingly employed national security tools to intervene in the international flow of AI technology. E.g., imposing progressively stricter export controls on most advanced AI chips (high-end GPUs/NPUs), related semiconductor manufacturing equipment (advanced lithography), and core software technologies (EDA tools) to specific countries (like China) or entities deemed national security threats. Simultaneously, strengthening national security reviews for foreign investments in sensitive AI technology sectors (esp. from perceived rival nations), e.g., the expanding scope and power of CFIUS in the US. These measures aim to restrict critical technology flow to rivals, maintain own tech advantage and national security, but also risk disrupting global AI industry supply chain stability and exacerbating techno-nationalism and strategic confrontation.
  • Strengthening Data Localization Requirements & Cross-Border Data Flow Rules: (Discussed in Section 6.4) To safeguard national data sovereignty, protect citizens’ personal information security, and ensure regulatory control over critical data, more countries (esp. China and EU) are establishing legal frameworks to strictly manage cross-border flows of important data and personal information. These frameworks typically require cross-border transfer to meet specific legal conditions (e.g., passing data export security assessment by authorities, signing government-approved standard contractual clauses with foreign recipients, or relevant entities obtaining specific data protection certification), and might even impose mandatory data localization requirements for certain data types (e.g., important data from CII operators, personal information reaching certain volume thresholds). While aimed at protecting national interests and rights, these rules can increase compliance costs and complexity for multinational corporations and potentially pose barriers to global digital trade and free data flow.
  • Establishing National AI Safety Research, Testing & Assessment Institutions: Facing the complexity and potential risks of AI (esp. high-risk AI), some countries are establishing government-led or supported specialized AI safety research centers, technical standard-setting bodies, and independent testing & evaluation platforms. These aim to conduct cutting-edge research on AI safety issues (model robustness, explainability, alignment), develop national AI safety technical standards and best practice guidelines, and be responsible for independent, authoritative technical testing, safety assessment, and compliance certification for AI systems planned for deployment in critical sectors (public services, CII, defense) or assessed as high-risk.
  • Actively Participating In & Seeking to Lead Global AI Governance Rulemaking: Recognizing AI’s global impact and transnational challenges, all major nations and international organizations are actively participating in, and even attempting to lead, the ongoing global rulemaking and international coordination processes concerning key issues like AI ethical principles, safety standards, data governance rules, IP protection, military application limits, etc. This occurs within various multilateral international frameworks (UN & agencies like UNESCO, ITU; G7, G20; OECD; WTO) and bilateral dialogue mechanisms. Each nation strives to infuse its own values, governance philosophies, and national interest considerations into the future global AI governance system, seeking greater influence and leadership in shaping the rules. This itself constitutes a new, complex international strategic game centered around future technology rules and global order construction.

Conclusion: Seeking Prudent Balance Between Empowering Security & Inducing Risk, Exploring Future Paths Between Safeguarding Sovereignty & Engaging in Global Governance

The development of AI technology intertwines, in an unprecedented, profoundly complex, and irreversible manner, with the two grand narratives crucial for the well-being, stability, and future existence of human society: safeguarding fundamental human rights for every individual and maintaining overall national security. These two aims can be mutually reinforcing (e.g., enhanced social governance via AI might help ensure public safety, indirectly protecting citizens’ rights to life and property), but also harbor deep, inherent tensions requiring careful balancing (e.g., excessive national security surveillance might severely infringe citizens’ rights to privacy and freedom).

On one hand, we must, with highest vigilance, continuously scrutinize and proactively address the multifaceted, sometimes systemic potential erosion and challenges posed by widespread AI application to individuals’ fundamental human rights—privacy, equality, freedom of expression, autonomy of thought, right to work, right to life and health, human dignity, etc. This demands concerted efforts through sound, evolving legal frameworks, responsible technology design principles (embedding privacy/fairness by design), effective internal and external governance mechanisms (mandatory HRIA, independent audits), and continuous social dialogue, oversight, and public empowerment, to ensure AI development and application remain human-centric, premised on respecting and protecting fundamental rights, and constrained by core ethical values.

On the other hand, we must also objectively recognize and responsibly leverage AI’s immense, irreplaceable enabling potential in enhancing the modernization of state governance, strengthening national security capabilities (against traditional and new non-traditional threats), and tackling global challenges (climate change, pandemics, transnational crime). Simultaneously, we must soberly confront and actively mitigate the novel, complex, potentially disruptive national security risks it brings in areas like cyber warfare, military AI (esp. autonomous weapons), critical infrastructure protection, information warfare and cognitive confrontation, and international strategic stability. This requires nations to formulate forward-looking, adaptive national AI strategies and security governance frameworks that balance development and security, strive to enhance self-reliance in core technologies and supply chain resilience, and participate actively and constructively, with responsible leadership, in building an open, inclusive, fair, effective global AI governance system reflecting legitimate concerns of all parties.

For legal professionals, entrusted with the special mission of upholding the rule of law, protecting rights, resolving disputes, and promoting justice—whether you are a human rights lawyer, a legal advisor to national security agencies, or an expert in corporate compliance, data protection, IP, or international law—a deep, comprehensive, continuously updated understanding of the profound impacts, complex challenges, and frontier dynamics of AI across these two critically important, highly sensitive, interconnected, and sometimes tense grand dimensions is essential. Only then can we, in concrete legal practice and policy engagement, provide truly valuable, far-sighted, responsible legal analysis, compliance advice, risk management solutions, and institutional design wisdom on how to seek prudent balance between empowerment and risk, how to explore constructive paths between safeguarding sovereignty and participating in global governance, and how to ensure technological progress while steadfastly upholding core human values and fundamental principles of the rule of law. In this era of infinite possibilities and potential perils driven by intelligence, contributing the indispensable wisdom and responsibility of the legal profession to our shared, hopefully better future is our critical task.