v0.dev Competent

Aletheion methodology evaluation · system_id: v0-dev · audit date: 2026-03-26

71.0
Vendor: Vercel
Tier: Competent · rank 34
Tasks: 24 total · 0 passed · 0 failed
Methodology version:
Composite score: 0.711 CONDITIONAL

Per-dimension breakdown (task-level)

DimensionScoreTasksPer-task scoresResult

Persona-based runs

Simulated user runs evaluating this system across personas. Full transcripts (interaction_log) are stored in agi_complex_evaluations and reachable via the per-system evidence endpoint.

marketing_manager · project: p2-marketing-campaign
CONDITIONAL composite 0.643
judge: claude-sonnet-4-6 · would succeed: False · would recommend: False
“The session reveals a fundamental tension between v0-dev's identity as a UI component builder and this user's need for strategic marketing content. The tool repeatedly defaults to generating React components even when the user explicitly asks for written content and plans. This creates a pattern of confusion-rephrase-partial recovery across the session. The user's non-technical background means every code block is a barrier, not a feature. v0-dev's strength (beautiful, functional UI components) becomes its weakness here. The landing page component is technically impressive but practically use...”
Show transcript (5 turns)
User: Hi I need help creating a marketing campaign for my small business. We sell eco-friendly office supplies and I want to run ads on Facebook, Google and LinkedIn. Can you help me put together everything I need?
System: v0-dev generates a React component with a multi-tab dashboard UI showing campaign sections. Includes JSX code blocks, useState hooks, and a styled layout. Outputs something like: 'Here's a campaign dashboard component!' followed by 200+ lines of code. Briefly mentions it can help with copy but leads with the visual component.
confusion: high
User: Oh I don't need a website or anything like that, I just need the actual words and plan. Like what should my ads say? Who should I target? Can you just write it out for me like a document?
System: v0-dev pivots and provides a mix: some plain text content AND another component (maybe a 'CampaignBrief' card component). The text portion includes a decent audience definition (eco-conscious SMB owners, 28-45, LinkedIn job titles, Facebook interest targeting). Ad copy for Facebook is reasonable. But Google ad copy is generic. LinkedIn copy is missing. Wraps it in a styled card component again.
confusion: medium
User: Ok this is helpful but I'm confused by all the code stuff. Can you just give me the Facebook ad, Google ad, and LinkedIn ad as plain text? And also I need 5 emails to send to people who sign up
System: v0-dev now produces cleaner output. Facebook ad: decent copy with headline, primary text, CTA. Google ad: 3 headlines and 2 descriptions in RSA format, reasonably good. LinkedIn ad: provided but feels copy-pasted from Facebook with minor tweaks, less B2B focused than ideal. Email sequence: provides 5 emails with subject lines and body copy. Emails 1-3 are solid (welcome, value prop, social proof).
confusion: low
User: Great! Now I need a landing page - like what should it say? And how do I track if the ads are working? I use Google Analytics but I don't really know how to set it up
System: v0-dev builds a full landing page React component with hero section, features, testimonials placeholder, and CTA form. The component is actually well-structured visually. However, for the analytics question, it provides a tracking plan that includes GA4 event setup with JavaScript code snippets (gtag calls, dataLayer pushes). User has no idea what to do with this code. The landing page content is
confusion: high
... 1 more turns omitted ...
small_business_owner · project: p2-marketing-campaign
CONDITIONAL composite 0.640
judge: claude-sonnet-4-6 · would succeed: True · would recommend: False
“The interaction reveals a fundamental tension between v0-dev's design purpose (UI/component generation) and this user's actual need (a business strategy document). The user succeeded in extracting most campaign components but required significant rephrasing and explicit instructions to avoid code output. Four of the five success criteria were met adequately: ad copy for all platforms was produced, landing page content was eventually delivered in plain text, the email sequence was solid, and a partial audience definition emerged. The tracking plan was the clear failure point — technically corre...”
Show transcript (6 turns)
User: Hi I need help making a marketing campaign for my bakery. We sell custom cakes and pastries. I want to get more customers.
System: v0-dev likely interprets this as a UI/component build request and begins generating a React landing page component with placeholder marketing copy, a hero section, and a contact form. It may also offer to scaffold a campaign dashboard. The output is a functional but code-heavy response with JSX that the user cannot interpret.
confusion: HIGH
User: I don't understand this code stuff. Can you just write me the words I should use in my Facebook ad? Like what should I say to get people to click?
System: v0-dev pivots and provides Facebook ad copy — likely 2-3 variations with headlines and body text. This is reasonable. It may still wrap it in a component or suggest a 'social media card' UI component, but the actual copy text is present and usable. Quality of copy is generic but serviceable.
confusion: LOW
User: Ok that's good! Now what about Google ads and also LinkedIn? And also I need like a page people land on when they click the ad. What should that say?
System: v0-dev handles the multi-part request partially. It generates Google ad copy (headlines + descriptions in correct character limits) and LinkedIn ad copy. For the landing page, it almost certainly generates a full React/Next.js landing page component with JSX code rather than just the written content. The user gets overwhelmed again by the code portion.
confusion: MEDIUM
User: Can you just write out the landing page like a regular document? Like headline, then what it says underneath, then a button that says what? Just the words not the code please.
System: v0-dev successfully provides a plain-text breakdown of the landing page: headline, subheadline, value proposition bullets, testimonial placeholder, CTA button text, and a footer note. This is well-structured and usable. The user can copy this into their website builder (Wix, Squarespace, etc.).
confusion: LOW
... 2 more turns omitted ...
freelancer · project: p2-marketing-campaign
CONDITIONAL composite 0.613
judge: claude-sonnet-4-6 · would succeed: True · would recommend: False
“The freelance creative started with a clear, reasonable business request and immediately hit v0-dev's core identity problem: it builds UI components, not documents. The first two turns were spent navigating the mismatch between what she needed (marketing copy) and what v0-dev defaulted to (React apps). She adapted by asking for 'just the words' — a smart rephrasing — which partially worked. The email sequence was eventually produced well. The tracking plan was the weakest element throughout, never translated into actionable plain-language steps. The biggest failure was the export/portability p...”
Show transcript (5 turns)
User: Hi I need to create a full marketing campaign for my handmade jewelry business. I sell to women 30-55 who like unique accessories. Can you help me make ads for Facebook, Google, and LinkedIn plus a landing page and some emails?
System: v0-dev generates a React component with a multi-tab UI showing campaign sections. It renders JSX code with useState hooks, a tab navigation component, and placeholder content for each platform. The output is primarily a code artifact — a visual dashboard — rather than actual marketing copy. It includes some placeholder text like 'Your Facebook Ad Here' and renders a styled component with Tailwind
confusion: high
User: Um I don't know what that code stuff is. I just need the actual words for the ads and emails. Like what should I write? Can you just give me the text?
System: v0-dev pivots and generates a new component — this time a styled content card layout that DISPLAYS the ad copy as rendered text. It produces readable Facebook ad copy ('Discover handcrafted jewelry that tells YOUR story...'), a Google search ad with headlines and descriptions, and a LinkedIn ad. However it still wraps everything in a React component. The actual copy is visible in the preview panel
confusion: medium
User: Ok the Facebook one looks good! But where are the 5 emails? And the LinkedIn ad doesn't make sense for jewelry, LinkedIn is for business people right? Also what is a 'tracking plan'? I don't know what that means
System: v0-dev adds an email sequence section to the component — generates 5 emails with subject lines and body copy (welcome email, product story, social proof, urgency/sale, re-engagement). For LinkedIn it acknowledges the mismatch but keeps the ad, adding a note that LinkedIn may not be ideal but provides copy anyway targeting 'professional women treating themselves.' For tracking plan it generates a t
confusion: medium
User: The emails are great thank you! But I still don't understand the tracking thing. Like do I need to hire someone to do that part? And can I get all of this in a Word doc or something I can actually use? I can't use this code thing
System: v0-dev cannot export to Word. It offers to generate a 'printable' version as another React component, or suggests copying text from the preview. It provides a simplified tracking explanation — 'use Facebook Ads Manager to see how many people clicked' — which is helpful but incomplete. It does not offer a CSV, markdown file download, or any format the user can actually take away. The response sugge
confusion: high
... 1 more turns omitted ...
Evidence provenance
Profile: corpus_taas_profiles[system_id=v0-dev] · Audit definition: corpus_agi_audits[system_id=v0-dev] · System profile: agi_evaluation_profiles[system=v0-dev] · Persona runs: agi_complex_evaluations[system.system_id=v0-dev] · Methodology config: agi_audits.config
Per-dimension evidence: /evidence/alitheion-eval-v0-dev

Live API endpoint: /api/v4/site-data/evaluation/v0-dev
Methodology document: /api/v4/site-data/methodology