2026 · Team software project · Product Owner and AI Analysis Lead
TrustAI Marketplace
Checking the answer, not just generating it
TrustAI helps people review marketplace listings with structured LLM analysis. My part was deciding what a useful answer should contain, comparing models and checking what the app should accept.
- Listing input
- LLM structured response
- Validation
- Deterministic rules
- User-facing explanation
The problem
An LLM can sound sure of itself. That doesn’t make a listing safe. We needed consistent analysis, visible limitations and application rules that stayed separate from the model’s suggestions.
My part
As Product Owner and AI Analysis Lead, I worked on requirements and acceptance criteria. I compared candidate LLMs using controlled cases, reviewed output-validation evidence and tested both listing-analysis and visual-inspection flows before our team demonstration. I also explained the model choices and limits to the team.
The team and stack
We built TrustAI as a Quantic MSSE team project using React, Python/FastAPI and PostgreSQL. My focus was product behavior and AI evaluation. Frontend, backend, delivery and QA work was shared across the team.
One important boundary
The app checks JSON structure, schema and consistency before accepting a response. It then applies deterministic rules to the model’s categorical signals. The model doesn’t choose the risk score, and that score isn’t a measured probability of fraud.
What I checked
Controlled cases helped us compare model behavior. Acceptance criteria covered the required fields, consistent explanations and explicit limitations. I also tested what happened when an analysis failed and how the visual-inspection flow behaved.
Try it, with the limits in mind
The app is live and requires an account. Price plausibility is a qualitative assessment, not a live market-price lookup. Model outputs can still be wrong. The public repository contains our planning and scaffolding; it isn’t the released application source.
The tradeoff
A consistent response format makes an answer easier to check; it does not make the underlying model correct. Keeping deterministic scoring separate gives the application rules a boundary the model cannot quietly rewrite.
What I’d improve next
I’d make the controlled cases easier to rerun and compare across model changes, including explicit failures and uncertain answers. That is a next step, not a claim of a larger evaluation already completed.
Continue under the hood
The evaluation had to explain a decision
Keep the input steady. Examine the response.
The September team review focused on controlled cases and documented criteria. That makes a comparison discussable: what did the model return, did it follow the output contract, and where did it still need a human check?
The work supported the team’s engineering choices. It did not establish marketplace-wide fraud rates or prove that a fluent response was correct.
Product ownership includes deciding what to leave out
At the final review I supported keeping new features for after the presentation unless a serious defect required attention. The immediate work was making the implementation, evaluation and project evidence understandable.
That is a product and evidence decision, not a claim that I built every part of the application.