Safe & Secure Agentic AI: Governance, Identity, and Red-Teaming Training Course

Foundations: Threat Modeling for Agentic AI

• Categories of agentic threats: misuse, privilege escalation, data leakage, and supply-chain vulnerabilities.
• Adversary profiles and attacker capabilities specifically relevant to autonomous agents.
• Mapping assets, trust boundaries, and key control points for agent environments.

Governance, Policy, and Risk Management

• Governance frameworks for agentic systems, including roles, responsibilities, and approval gates.
• Policy formulation covering acceptable use, escalation protocols, data handling, and auditability.
• Compliance requirements and evidence collection for audit purposes.

Non-Human Identity & Authentication for Agents

• Creating identities for agents using service accounts, JWTs, and short-lived credentials.
• Implementing least-privilege access models and just-in-time credential issuance.
• Strategies for identity lifecycle management, rotation, delegation, and revocation.

Access Controls, Secrets, and Data Protection

• Fine-grained access control models and capability-based patterns for agents.
• Secrets management, encryption in transit and at rest, and data minimization practices.
• Securing sensitive knowledge sources and PII from unauthorized agent access.

Observability, Auditing, and Incident Response

• Designing telemetry for agent behavior, including intent tracing, command logging, and provenance tracking.
• SIEM integration, defining alerting thresholds, and ensuring forensic readiness.
• Developing runbooks and playbooks for managing and containing agent-related incidents.

Red-Teaming Agentic Systems

• Planning red-team exercises, defining scope, rules of engagement, and safe failover mechanisms.
• Adversarial techniques such as prompt injection, tool misuse, chain-of-thought manipulation, and API abuse.
• Performing controlled attacks to assess exposure and impact.

Hardening and Mitigation Strategies

• Engineering controls including response throttling, capability gating, and sandboxing.
• Policy and orchestration controls involving approval flows, human-in-the-loop mechanisms, and governance hooks.
• Model and prompt-level defenses such as input validation, canonicalization, and output filtering.

Operationalizing Safe Agent Deployments

• Deployment strategies for agents, including staging, canary releases, and progressive rollouts.
• Change control, testing pipelines, and pre-deployment safety checks.
• Cross-functional governance involving security, legal, product, and operations playbooks.

Capstone: Red-Team / Blue-Team Exercise

• Executing a simulated red-team attack within a sandboxed agent environment.
• Acting as the blue team to defend, detect, and remediate using established controls and telemetry.
• Presenting findings, remediation plans, and proposed policy updates.

Summary and Future Directions