An AI for National Security Curriculum

Members of the military unit at Armstrong Technical High School gathered around a table studying a map, photographed by Marjory Collins in 1942.

Members of the military unit at the Armstrong Technical High School studying a map, by Marjory Collins (1942). From the Library of Congress.

The White House released a recent National Security Presidential Memorandum (NSPM-11) that calls for a new AI for National Security Curriculum to be developed and implemented in 120 days.

The official text is this:

Within 120 days of the date of this memorandum, the DNI and the Secretary of War, in coordination with the OMB Director and IC elements, shall develop and implement an AI for National Security Curriculum, coordinated with existing Federal AI and cyber training programs. This initiative shall ensure that relevant personnel across the national security enterprise are trained to employ AI systems in accordance with applicable guidance and maintain literacy on the current AI frontier, including its capabilities, limitations, and implications for national security.

Building a curriculum from scratch in 120 days is a big ask. Fortunately, I have been thinking through what just such a curriculum might look like for Learning Journey AI, so here is my suggested framework. It covers AI's capabilities and limitations, safe and effective use, and the basics of building and evaluating AI systems.

The curriculum is divided into two tracks:

Core: for all relevant personnel across the national security enterprise — analysts, operators, leaders, acquisition officers, and support staff — anyone who will employ AI systems or whose decisions are shaped by AI outputs.
Advanced: for those who design, evaluate, or engineer AI systems: data scientists, AI engineers, T&E personnel, and system architects responsible for fielding AI capabilities.

Both tracks assume working operational context, but only the Advanced track assumes a computer-science background.

Core track

Applicable policy guidance

Objective: After this module, learners can identify the federal AI policies and DoD directives that govern their work, and locate current authoritative guidance.

DoD Directive 3000.09 on Autonomy in Weapon Systems
- This policy document describes what autonomous weapons systems are and how they should be governed.
- Note: NSPM-11 calls for this document to be updated within 90 days.
National Security Presidential Memorandum-11
OMB M-25-21 Accelerating Federal Use of AI through Innovation, Governance, and Public Trust
OMB M-25-22 Driving Efficient Acquisition of Artificial Intelligence in Government
NIST AI Risk Management Framework
ODNI/IC directives
Procurement guidance and approved systems
- Currently authorized enterprise systems by IL level
- Commercial systems preference
- Effective acquisition practices for AI systems
Foreign-developed model policy

Foundations & vocabulary

Objective: After this module, learners can correctly use the core technical vocabulary of modern AI when speaking with engineers, vendors, and policy makers.

Model
Token
Context window/Context limit
Harness/scaffolding
Effort level/test-time compute
Skill
Model Context Protocol (MCP)
Command Line Interface (CLI)
Agent

Non-generative AI capabilities and limitations

Objective: After this module, learners can recognize the classes of AI built before LLMs (computer vision, signal processing, classical ML, autonomy stacks) and explain where they remain the right tool over generative AI.

Capabilities
- Fast
- Low SWaP
- Can be more accurate than general purpose generative AI systems on specific tasks
- Comparatively simpler to evaluate and secure than generative AI systems
Limitations
- Require labeled data
- Poor performance outside training distribution
- Cannot be easily adapted to other tasks
Example use cases:
- Object detection in satellite imagery
- On-UAS target recognition
- Signal classification
- Predictive maintenance with tabular data

Foundation model capabilities and limitations

Objective: After this module, learners can articulate what current foundation models can and cannot do reliably, and identify the limits of frontier capabilities in a national-security context.

Capabilities
- Jagged frontier
  - E.g. Solving Erdos problem vs counting r's in strawberry
- Graphs of renown
  - METR 50% reliability software tasks graph
    - Task horizon length on agentic coding
    - Lack of reliability
  - Remote Labor Index
    - Performance on real remote work (Upwork gigs)
  - ARC-AGI 1, 2, 3
    - Reasoning advances via visual puzzles
  - Towards a Science of AI Agent Reliability
    - Slower progress in reliability than accuracy
  - UK AISI cyber evals
    - Cyber hacking progress
- Modalities
  - Text, code, imagery, speech, video
- Example use cases in national security context
  - Cyber offense and defense
  - Biological threat creation and defense
  - Drafting presentations and other back office communications
  - Data analysis
  - Labeling data for other AI models
  - Scientific research
  - Connecting bespoke IT systems
  - COA planning
  - Surveillance
  - Targeting
Limitations
- Limited contextual understanding
- Lack of continual learning
- Lack of reliability
- Lack of interpretability
- Lack of steerability
- Size, weight, and power tradeoffs for cloud vs local models
- Slow responsiveness
- Sycophancy
- Evaluation difficulties such as construct validity

Maintaining human judgment

Objective: After this module, learners can use AI as a thought partner without offloading professional judgment, and recognize when AI output should be challenged or discarded.

Literature review of cognitive effects
Avoiding loss of human autonomy
- Reviewing primary materials
- Never accepting AI output as final
- Ask for hints instead of answers when learning
- Recognizing AI-generated content patterns
  - Detector tools and limitations
Using AI as a thought partner
- Prompting for questions
- Using planning modes
- Persona-based critiques
- Distilling tacit criteria into skills
- Developing systems that surface relevant context alongside outputs

AI security — user-side

Objective: After this module, learners can choose appropriate AI environments for their data, recognize prompt-injection and data-leakage risks, and understand the role of guardrails in protecting them.

Recognizing and avoiding classification by compilation spills
Safeguarding data privacy
Prompt injection
Lethal Triad
Understanding what guardrails are
LLM backdoors/data poisoning
Incident reporting & escalation procedures

Using modern agents effectively

Objective: After this module, learners can prompt, scaffold, and direct modern AI agents to accomplish real work, and choose between skills, MCPs, and CLIs for a given task.

Prompting strategy
Extending toolsets with skills, MCPs, & CLIs and when to use each
Explore→Plan→Execute→Iterate workflow
Context engineering
Sharing context between teams

Staying up to date with AI advancements and best practices

Objective: After this module, learners can locate vetted sources, join a community of practice, and use AI itself to track frontier developments responsibly.

Official resources
- Approved AI systems
- Policies portal
Forming or locating and joining a community of practice
Online learning resources
- Online courses
- Substacks
- Military publications
- X.com lists
- Podcasts
Using AI to compile research reports

Advanced track

Common architectures with applications

Objective: After this module, learners can choose an architecture (supervised classifiers, RAG, coding agent, multi-agent) that fits a mission requirement and reason about its tradeoffs.

Supervised classifiers
RAG system
Coding/cowork agent
Multi-agent architectures

AI security — builder-side

Objective: After this module, learners can apply OWASP LLM Top 10 mitigations, perform red teaming, design security-specific harnesses, construct guardrails, and evaluate neurosymbolic approaches when building AI systems.

OWASP Top 10 for LLMs
MITRE ATLAS Framework
Manual and automated red teaming
Security-specific harnesses
Constructing guardrails
Neurosymbolic agent security approaches
- E.g. Sondera.ai

Evaluating frontier AI-based systems

Objective: After this module, learners can design and operate an evaluation pipeline — including dataset construction, LLM-judge calibration, and secondary-metric tracking.

Trace logging
Producing synthetic data for evaluations
Error analysis
Establishing actionable evaluation metrics
Building an evaluation dataset
Calibrating LLM judges
Automating and deploying evaluations
Measuring secondary task performance metrics
- Latency
- Token usage

AI engineering

Objective: After this module, learners can build with agent SDKs, curate fine-tuning datasets, and optimize models for deployment constraints (quantization, distillation, token economy).

Building with agent SDKs
The customization ladder of ascending difficulty
- Prompt engineering
- Skills
- Retrieval via MCPs/CLI
- Custom harnesses
- Supervised fine tuning
- Continued pretraining
Hosting and serving open weight models
Curating high quality fine tuning datasets
Resource optimization for edge deployment
- Quantization
- Knowledge Distillation/Model Distillation
- Reducing token usage
- Model routing

Thoughts on implementation

I think the government will be tempted to just buy a subscription to a large, online commercial training platform, but this is a mistake. For one, this curriculum does not exist on a single platform. Two, almost no online courses are tailored to a national security audience's data and workflows. And three, most commercial online courses fail to take advantage of modern coding agents, relying instead on boring videos and quizzes.

There are so many possibilities to produce hands-on trainings that better engage national security professionals. At Learning Journey AI, we specialize in designing and delivering these special learning experiences. If you are interested in adapting this curriculum or want to work with someone who understands the material deeply, please reach out.

This post was originally published on Learning Journey, our Substack. Subscribe there for new posts on AI and society.