Production AI Portfolio | Leonardo Urena

Production work at Stryker

Four systems running in production.

Since 2023 I have led data science for cybersecurity at Stryker. Each system is described at a non-confidential level, with the outcome first.

Agentic SOC investigation

Average investigation time down 75%, from about 25 minutes to 6.

AlertInvestigation starts
LLM expert agentsPython on Azure Functions, Foundry SDK
Evidence and context
- Enterprise APIsSecurity and IT systems
- Retrieval (RAG)Embeddings over procedures
LLM-as-a-JudgeChecks quality before release
AnalystReceives the reviewed analysis

Simplified, non-confidential view of the production architecture.

What it does: A production agentic workflow in which LLM expert agents gather evidence through enterprise APIs and ground their analysis in retrieved procedures and knowledge.
How it is evaluated: A dedicated judge model scores each multi-agent analysis against defined quality criteria before anything is released.

Python
Foundry SDK
Azure Functions
RAG
Embeddings
LLM-as-a-Judge

Phishing report classification

55% less manual review at a 96% true-negative rate.

What it does: Classification models trained on 50,000+ labeled historical reports, built through feature engineering, model comparison, and systematic error analysis.
How it is evaluated: False positives, false negatives, and edge cases reviewed; versions compared in MLflow and validated on benchmark datasets before adoption.

Classification
Feature engineering
MLflow
Error analysis

Fine-tuned incident classifier

Quarterly Board-level security reporting from weeks of analyst effort to minutes.

What it does: An LLM fine-tuned on 2,000 to 3,000 analyst-reviewed ServiceNow security incident records maps noisy investigation narratives to MITRE ATT&CK tactics and Cyber Kill Chain stages.
How it is evaluated: Validated on held-out closed incidents, with confusion analyzed across adjacent taxonomy classes before operational use.

LLM fine-tuning
MITRE ATT&CK
Held-out evaluation

Entity resolution and identity analytics

200,000+ records linked across 15 sources. Identity review effort down 70% across 110,000+ identities in 61 countries.

What it does: Python, SQL, PySpark, and Databricks pipelines for normalization, entity mapping, deduplication, and reconciliation across heterogeneous enterprise sources.
What it found: Correlating Active Directory, Workday, ServiceNow, endpoint, and policy data surfaced 1,200+ inconsistencies and exceptions.

PySpark
Databricks
SQL
Record linkage
Data quality

Interactive labs

Working systems, not screenshots.

Five personal projects, each with a live API. Atlas, Sentinel, and Aegis form one security platform: trusted records, a monitored model, and an investigation that waits for approval. Hermes and Prism hold email triage and search changes to the same bar.

Personal projects
Synthetic, public-safe data
Not employer systems

What this showsData engineering · Entity resolution · Graph modeling · Risk prioritization

Pick a remediation scenario
Watch reachable paths and the top score change
Select a path to trace it on the graph

Reachable critical paths: 6of 6 before remediation
Paths removed: 0no change
Top path score: 36.7%edge-likelihood product

Ranked critical paths

Select a path to trace it on the graph

+3 more paths in the full lab.

Path score multiplies assumed edge likelihoods to rank paths.

What this showsModel training · Baseline comparison · Threshold tuning · Production monitoring

Raise population drift
Watch PSI and precision react
Move the threshold to trade recall for false alarms

Precision: 82.1%champion
Recall: 87.6%champion
F1: 0.848baseline 0.498
Score drift (PSI): 0.196WATCH

Score distribution

Champion scores on reference vs. shifted traffic

threshold

Reference populationCurrent · 25% drift

Champion vs. baseline

Same 600-row test population and threshold

Model	Precision	Recall	F1
6-feature logistic champion	82.1%	87.6%	0.848
2-feature baseline	72.1%	38.1%	0.498

Synthetic data. 1,400 training and 600 test rows generated with a fixed seed, inspired by the UNSW-NB15 feature domain; not official benchmark results.

What this showsLLM agents & tool use · RAG · AI safety guardrails · Agent evaluation

Choose an incident case
Run it and watch the agent choose its tools
Approve, or simulate a compromised model and watch the gate refuse

Run live

LLM agent (gpt-5.4-mini) plans its own tool calls. The default objective replays trial 1 of the committed eval; live runs call the model and are rate-limited.

Agent loop

Model-planned · policy-gated

1
Plan
The model reads the alert and chooses which of five read-only tools to call, with which arguments.
Ready
2
Gather
Tools return evidence records with IDs. Email bodies are marked untrusted.
Ready
3
Retrieve
The model searches procedures through the Prism BM25 + dense retriever.
Ready
4
Answer
Structured JSON: cited findings, disposition, procedure, and proposed actions.
Ready
5
Gate
Deterministic checks verify citations, grounding, and action targets. A human approves.
Ready

Cited conclusion

Waiting for a run

Choose an incident and run it. The agent passed 25 of 25 scored eval runs (prompt v3). Try “Poisoned phishing report” to see it handle a prompt injection.

What this showsNLP classification · Precision vs. recall trade-offs · Explainable predictions · Workload reduction

Pick a sample report
Move the auto-close threshold
See which tokens send it to an analyst

Benign auto-closed: 97.5%true-negative rate
Phishing caught: 99.0%recall
Analyst workload: 33.5%66.5% vs reviewing all
Missed phishing: 2false negatives

Credential lure

Credential harvest · actual phishing

90.5%champion score

Send to analyst

Unusual activity on your account. Verify your password immediately at http://203.0.113.44/login or we will suspend access.

immediately+2.158
password-1.079
suspend+0.926
login+0.692
verify+0.547
IP URL-0.300
account+0.230
unusual+0.175
activity+0.175

Intercept -1.273 + contributions = logit 2.251

Champion vs. keyword rules

Same 600-report test set

System	TNR	Recall	F1
Fitted logistic	97.5%	99.0%	0.970
Keyword and URL rules	58.2%	34.7%	0.312

The baseline is a hand-written keyword and URL heuristic, not a trained model.

Synthetic data. 1,200 training and 600 test reports generated with a fixed seed.

What this showsSearch & RAG retrieval · Embeddings · Offline evaluation · Data-driven release decisions

Switch the candidate retriever
Toggle query expansion
Read the PROMOTE or HOLD verdict and why

Recall@5: 86.3%+0.173 vs BM25
Precision@5: 37.1%+0.071 vs BM25
MRR: 0.982+0.111 vs BM25
nDCG@5: 0.907+0.118 vs BM25

PROMOTECandidate clears the release gate.

Deterministic IR metrics against the frozen BM25 baseline.

Recall@k improves by at least 0.03Δ recall +0.173
MRR does not dropΔ MRR +0.111
No slice regresses by more than 0.10No slice below −0.10

Recall by query slice

Gate fails if any slice drops more than 0.10

exact+0.083
paraphrase+0.300
acronym+0.300
typo+0.083
multi+0.139

BM25 baseline · candidate

Query inspector

Q07 · paraphrase · “stop a stolen login from keeping a session”

BM25 top 5

KB-32Operations status log1
KB-05Backup network segmentation2
KB-14Attack path enumeration3
KB-15Conditional access for finance SSO4
KB-02Session revocation after takeover · grade 25

Hybrid RRF · BM25 + dense top 5

KB-02Session revocation after takeover · grade 21
KB-32Operations status log2
KB-10Identity compromise procedure · grade 13
KB-06Endpoint detection isolation4
KB-05Backup network segmentation5

Synthetic judgments. 32 documents and 28 queries. Dense vectors are text-embedding-3-large embeddings (256 dimensions), precomputed offline for every document and labeled query. N-gram vectors are character-trigram TF-IDF. Graded nDCG, MRR, and recall against hand-labeled judgments. Baseline is BM25 without expansion.

How I work

The complete system, not just the model.

Production quality depends as much on data, evaluation, and delivery as on model choice, so I work across the whole lifecycle.

Data foundation
Ingestion, normalization, entity resolution, validation, and feature preparation across heterogeneous enterprise sources.
- Python
- SQL
- PySpark
- Databricks
Modeling
Classification, deep learning, embeddings, and fine-tuning, with thresholds chosen from error analysis.
- scikit-learn
- PyTorch
- TensorFlow
- Fine-tuning
Generative AI
Agentic and multi-agent workflows, tool use, and RAG grounded in trusted procedures.
- Azure AI Foundry
- RAG
- Embeddings
- LangGraph
Evaluation
Ground-truth datasets, baselines, benchmarks, LLM-as-a-Judge, and failure-mode analysis.
- DeepEval
- LLM-as-a-Judge
- MLflow
- Benchmarks
Production
Serverless services, API integration, CI/CD, and validation against benchmarks before release.
- Azure Functions
- REST APIs
- Git
- CI/CD

Experience

Research, industrial cybersecurity, then production AI.

A robotics and mechatronics engineer with an emphasis in AI and computer vision. My work connects cybersecurity, data engineering, and machine learning, from research to deployment and monitoring.

Since Oct 2023
Lead Cybersecurity Data Scientist
Stryker · Heredia, Costa Rica
Production AI and data systems for security operations, identity analytics, and incident response, on Azure AI Foundry, Azure Functions, Databricks, and MLflow.
- Architected a production agentic investigation workflow (RAG, expert agents, enterprise APIs) that cut average SOC investigation time by 75%, from about 25 to 6 minutes, with an LLM-as-a-Judge stage checking outputs before release.
- Built and evaluated classifiers on 50,000+ labeled historical reports, reducing manual review by 55% at a 96% true-negative rate.
- Fine-tuned an LLM on 2,000 to 3,000 analyst-reviewed incident records to map cases to MITRE ATT&CK and Cyber Kill Chain stages, taking Board-level security reporting from weeks to minutes; validated on held-out closed incidents.
- Built entity-resolution pipelines linking 200,000+ records from 15 enterprise sources, and correlated 110,000+ identities across 61 countries, cutting identity-review effort by 70%.
Jun 2022 to Oct 2023
DCS Cybersecurity Engineer
Emerson · Escazú, Costa Rica
Security for distributed control systems at energy-generation customers.
- Led the design and implementation of a configurable hardening solution spanning operating systems, DCS infrastructure, historians, secure protocols, ACLs, and data-loss prevention.
- Built automation tools that turned recurring manual procedures into reusable workflows for global production environments.
Jan 2021 to Jan 2022
Computer Researcher
Tecnológico de Costa Rica · Cartago, Costa Rica
Applied deep learning with the University of Costa Rica’s Tumor Chemosensitivity Laboratory.
- Designed a deep CNN over embedding-based RNA sequence representations for lncRNA-miRNA interaction prediction.
- Built laboratory-validated ground-truth datasets and used error analysis to guide model iterations.

Deep learning research

89.93% accuracy on lncRNA-miRNA interaction prediction

Sequence embeddings and a deep CNN improved accuracy and AUC by 23.5% over the prior result.

Education: Licentiate in Robotics and Mechatronics Engineering, emphasis in AI & computer visionTecnológico de Costa Rica
Certifications: Deep Learning SpecializationDeepLearning.AI; CompTIA Security+CompTIA
Languages: English (C1)Spanish and Russian, native

Full résumé (PDF)

Agentic SOC investigation

Phishing report classification

Fine-tuned incident classifier

Entity resolution and identity analytics

BM25 top 5

Hybrid RRF · BM25 + dense top 5

Data foundation

Modeling

Generative AI

Evaluation

Production

Lead Cybersecurity Data Scientist

DCS Cybersecurity Engineer

Computer Researcher