Hiring a role,
hiring a model.
A simple frame for thinking about AI projects in PPS: treat each one like a hire. Start from the business need, write the “talent portrait,” and only then go looking for the model that fits the job.
Mapping business to role.
When a team has an internal-efficiency problem, you don’t hire a generic person — you look for someone who has built workflows, optimized processes, driven automation. The mapping from need to profile is what makes the hire work. AI projects deserve the same discipline.
Need → talent portrait → hire
You define the business need first. Then sketch the kind of person who has solved that need before. Then go find them. You evaluate the hire against the original need, not against generic resume traits.
Need → capability portrait → project
Define the business need and the pain points. Map them to expectations of a model or workflow. That mapping is the project — and it’s also how you tell whether the project is running well.
Four areas where PPS needs AI.
From Eric’s original whiteboard. Each area breaks down into the scenarios where AI gets work, and each scenario into the abilities a model must demonstrate.
Efficiency Improvement
Training & Improvement
Moderation Output
RCA 归因
Six rules for setting up an AI project.
If the project doesn’t pass these, the “hire” isn’t real — it’s a model on a leaderboard, not a worker on the team.
Lead by business metric
AI success metrics must map to business metrics — accuracy, leakage, overkill, productivity. Pure model metrics alone are not a deliverable. Training set, test set, and success metric must all live in the same business scenario.
Blind-test first
The test set is invisible to the modeling team. It’s maintained and audited by an independent project POC. No mixing, no leakage, no “just one peek.”
Multi-locale by default
Cover at least the Top X market languages and cultural contexts. A model that works only in one locale isn’t shipping for TikTok — it’s a demo.
Freshness
Refresh at least Y% of the test set every X period. A frozen test set rewards memorization, not capability.
Reproducibility
Sampling SQL, annotation SOPs, and evaluation scripts must be archived and version-controlled. If you can’t rerun last quarter’s eval, you don’t actually know what changed.
Human-AI alignment
Test alignment against a Golden set — not just model vs. model, or model vs. generic moderation output. The bar is human ground truth, not the next-best machine.
“Find the exact business requirement. Write the talent portrait. Then build the project — and judge it by the same standard you’d judge the hire.”