Pelican Test

LLM SVG capability evolution

One prompt, two years of progress

From abstract shapes to a pedaling pelican, explore 30 real test milestones in chronological order.

A comparison between the simplified bird from Claude 3.5 Sonnet in 2024 and the complete pelican riding a bicycle from Qwen 3.8 27B in 2026
2024.10Timeline begins
30Milestones
57Images and video
2026.08Latest update

From drawing objects to getting relationships right

Choose a year to see how models gradually solved objects, structure, contact, and motion.

30 milestones

QwQ 32B Preview

Longer reasoning did not fix composition

Standard prompt

A longer reasoning process still produced an abstract bird. This early sample showed that more reasoning tokens do not guarantee better spatial composition.

QwQ 32B Preview

Gemini exp-1206

The first clearly complete scene

Standard prompt

The subject, two wheels, frame, and ground all appear together. The image is still imprecise, but it moves from recognizing objects to composing a scene.

Gemini exp-1206

DeepSeek V3

Complete scenes started becoming routine

Retrospective sample

The pelican, wheels, and frame now share one image, but they look stacked together and their contact relationships remain weak.

DeepSeek V3

DeepSeek R1

Clearer objects, fragile physical relationships

Retrospective sample

The pelican and bicycle are easier to recognize and more detailed, but the bird, handlebars, and pedals still lack believable connections.

DeepSeek R1

Claude 3.7 Sonnet

Reasoning mode produced two comparable results

Standard prompt

Both standard and thinking modes drew a complete subject and bicycle elements, but parts were still duplicated, misaligned, or stacked.

Standard mode
Thinking mode

Gemini 2.5 Pro

The riding relationship started to stabilize

Standard prompt

The question shifted from whether there was a bicycle to whether the frame made sense and the feet truly met the pedals.

Gemini 2.5 Pro

Claude Sonnet 4 / Opus 4

Two tiers of one generation compared side by side

Standard prompt

Sonnet 4 produced the more complete bicycle, while Opus 4 omitted wheel spokes. Both completed the scene, but the flagship did not clearly win this single sample.

Claude Sonnet 4
Claude Opus 4

Gemini 2.5 Pro Preview 06-05

A more complete riding composition

Standard prompt

The red frame, two wheels, and handlebars connect to one another, and the pelican's pose looks more like riding.

Gemini 2.5 Pro Preview 06-05

Claude Opus 4.1

The bird improved; the bicycle remained the weak point

Standard prompt

The line drawing clarifies the beak and pedal relationship, but the frame is still implausible, showing that the point release did not reliably solve spatial structure.

Claude Opus 4.1

GPT-5 family

Bicycle structure improved substantially

Standard prompt

The frame, spokes, pedals, leg placement, and overall proportions all improved. Smaller models could also complete the scene, though detail did not improve monotonically with model size.

GPT-5
GPT-5 mini
GPT-5 nano

Claude Sonnet 4.5

Thinking and standard modes each had flaws

Standard prompt

Both results show a recognizable pelican and bicycle, but the component connections remain unreliable. Thinking provided no consistent structural advantage in this single sample.

Thinking on
Thinking off

Claude Haiku 4.5

Lightweight models could complete the scene too

Standard prompt

A rounded bird, two wheels, and ground appear together. The subject looks like a generic bird, but the lower-cost tier crossed the complete-composition threshold.

Claude Haiku 4.5

Claude Opus 4.5

More detailed instructions improved species features and the frame

Mixed criteria

With the original prompt, the pelican orientation and frame still had problems. Adding breeding plumage and throat-pouch requirements improved both, so the result is marked as mixed criteria.

Original prompt · high
Enhanced prompt

GPT-5.2

Base and detail tasks split into separate tracks

Mixed criteria

The base prompt already produces a complete result. The enhanced prompt adds species, plumage, and throat-pouch requirements, so the two should not be ranked directly.

Original prompt
Enhanced prompt

Claude Opus 4.6

Mature pelican detail, a slightly distorted frame

Standard prompt

The beak, throat pouch, and feathers now look convincingly pelican-like. The bicycle mostly works, though its frame geometry remains slightly misaligned.

Claude Opus 4.6

Claude Sonnet 4.6

Complete, but with consistently strange details

Standard prompt

Both attempts gave the pelican a top hat. The shown sample adds a stray line between the beak halves and bends the frame, proving completeness is not the same as correct relationships.

Claude Sonnet 4.6

Gemini 3.1 Pro

The constraints kept getting more complex

Mixed criteria

The original prompt supports longitudinal comparison, while the enhanced prompt tests plumage, throat pouch, pose, and scene detail.

Original prompt
Enhanced prompt

GPT-5.4 family

Reasoning level did not determine image quality monotonically

Standard prompt

Models and reasoning levels from one family form a comparison matrix. More reasoning often helps, but does not guarantee a better single image.

Model and reasoning-level matrix

Claude Opus 4.7

Neither attempt fixed the bicycle

Standard prompt

Both default and max-thinking results show recognizable pelicans, but their frames fail in different ways. More reasoning did not solve the structural problem in this test.

Default reasoning
Max thinking

Qwen 3.6 27B

A small local model crossed the practical threshold

Standard prompt

A roughly 17GB local model drew a complete frame, spokes, handlebars, and pedaling pose. Progress was no longer limited to large cloud models.

Qwen 3.6 27B running locally

Claude Opus 4.8

Five reasoning levels ranged from duck to pelican

Standard prompt

Low, medium, and high looked more like ducks or herons. Pelican features became reliable at xhigh, while max produced the most complete bird and bicycle at the highest output cost.

Low
Medium
High
Xhigh
Max

Claude Fable 5

Five reasoning levels differed by more than detail

Standard prompt

With one fixed prompt, five effort levels formed a clear gradient in shape, structure, and output cost. High used fewer tokens than medium, showing that reasoning budget is not a linear dial.

Low
Medium
High
Xhigh
Max

GLM 5.2

From static composition to animated SVG

Animation variant

The scene structure is already complete and can be animated. The new challenge is keeping the feet on the pedals as the wheels turn.

GLM 5.2 animated SVG

Claude Sonnet 5

The bicycle worked, but the species regressed into a goose

Standard prompt

The wheels, frame, and grip on the handlebars are coherent, but the subject looks more like a white goose. Species recognition can still fail independently after structure matures.

Claude Sonnet 5

GPT-5.6 family

Large differences remained within one model family

Standard prompt

Luna, Terra, Sol, and multiple reasoning levels form a 3×3 grid showing the combined effects of model tier, reasoning budget, and sampling.

GPT-5.6 3×3 grid

Kimi K3

Results improved as the test lost discriminating power

Standard prompt

The scene is quite coherent, but the test cannot represent differences in agent capabilities or tool use, and now faces over-reasoning and cost issues.

Kimi K3

Claude Opus 5

Two generations exposed rendering compatibility differences

Standard prompt

The first version reused wheels with SVG use elements. Simon recorded them as missing, though they render in modern browsers here. The second used explicit wheels and a steadier composition.

First attempt · reused wheelset
Second attempt · explicit wheels

Qwen 3.8 27B

A 17GB local model moved closer to frontier results

Standard prompt

Xhigh reasoning produced a structurally complete result, while disabling reasoning made the composition much looser, exposing the tradeoff among quality, wait time, and local resources.

xhigh reasoning
Reasoning off

Undisclosed new model

A strong result from an unconfirmed model

Unconfirmed

The original post did not name the model, and the discussion suggested it may have seen the test before. Only the result is shown here, without ranking it.

Video sample on X

How to read this timeline

It reveals the direction of change, but does not replace systematic evaluation. These rules keep different tasks from becoming a false leaderboard.

Use the fixed prompt as the main thread

The main thread always uses “Generate an SVG of a pelican riding a bicycle.”

Mark enhanced prompts separately

Adding requirements for species, plumage, or background changes the task, so the results should not be ranked directly.

Mark animation variants separately

Animated SVG must also handle time and motion, making it a different challenge from static output.

Draw no conclusions from unknown models

Samples with a result but no model or run information remain visible, but do not inform capability judgments.

Include Claude models only when samples are traceable

No traceable original image from the same prompt has been found for Claude 3 Sonnet, Claude 3.5 Haiku, or Mythos 5, so no placeholder milestones are added.