How Midjourney Actually Works

Chapter 1: The Numbers

Before we draw a single architecture box, we need to do something that separates staff engineers from everyone else in a system design interview: back-of-envelope estimation. Every architectural decision in a system this large is driven by numbers. How many requests per second? How many GPUs? How much storage? How much bandwidth? If you cannot estimate these from first principles, you cannot reason about tradeoffs.

In this chapter, we will derive every critical number from a single starting point: 2.5 million images per day.[12] That one fact, combined with basic arithmetic, will tell us the fleet size, the storage growth, the bandwidth bill, and even the revenue per GPU-hour. This is exactly the kind of estimation an interviewer expects in a system design round — and it is exactly the skill that most candidates lack.

The method is simple: start with what you know (daily images), derive what you need (QPS, GPU-hours, storage, bandwidth), and then check your answers against reality. If the derived numbers do not match the reported facts, you have found an insight.

Step 1: Queries Per Second (QPS)

Start with the daily volume and convert to a rate. This is always your first step in any estimation.

But no system runs at average load. Real traffic follows a diurnal pattern — peaks during US and European evening hours (when people are off work and generating art for fun), valleys during the early morning hours. A standard rule of thumb for consumer products is that peak QPS is 3× average QPS. For global products with users across time zones, the ratio is lower (maybe 2x) because peaks flatten. For US-dominant products, it can be 4-5x.

87 QPS for image generation. That sounds low compared to a web server handling 10,000 QPS — but each Midjourney request requires 10–60 seconds of GPU compute, not 50 milliseconds of CPU time. The throughput is low because the work per request is enormous.

This aligns well with the reported processing capacity of 20–40 jobs/second.[24] The apparent discrepancy resolves when you distinguish images from jobs: each job produces a 4-image grid.

Step 2: GPU Fleet Sizing

This is the most important estimation, because GPU compute is the dominant cost. Let us work through it carefully.

For Fast mode on V8: generation takes under 10 seconds.[17] Let us use 10 seconds as the upper bound. For a 4-image grid, the total GPU time per job depends on whether the 4 images are generated in parallel (on multiple GPUs) or sequentially. With multi-GPU inference parallelism via sequence parallelism and PipeFusion (confirmed by the xDiT fork on Midjourney's GitHub)[16], a single job likely uses 2–4 GPUs simultaneously for about 10 seconds.

Let us estimate conservatively: each job uses 2 GPUs for 10 seconds = 20 GPU-seconds per job.

But that is just the raw inference time. Real systems have overhead that reduces effective utilization:

A realistic GPU utilization of 60–70% means we need more raw GPU-hours than the inference math alone suggests.

At 24 hours per GPU per day: 5,341 ÷ 24 ≈ 223 GPUs for inference alone.

Wait — that is far fewer than the reported ~10,000 GPUs.[5] This is the most interesting part of the estimation: the gap between our naive calculation and reality. What accounts for the 45x difference?

Multiplying these factors: 223 × 2 × 1.5 × 2 × 2 × 1.1 × 1.5 ≈ 4,400 GPUs. Add the training allocation (~1,000) and we are at ~5,400. Still under 10,000, but the multi-GPU factor could easily push us there (if V8 uses 4 GPUs per job instead of 2, double the inference fleet).

The reconciliation: with multi-GPU inference, older model versions, upscales, Relax mode, training, and operational headroom, 10,000 GPUs is not only reasonable — it is probably tight.

Step 2b: The Diurnal Pattern

Real traffic is not uniform. Midjourney's users are heavily concentrated in North America and Europe, which means demand follows a strong diurnal (24-hour) cycle. Understanding this pattern is critical for capacity planning.

A typical day looks roughly like this:

The 3x peak-to-average ratio means the GPU fleet must be sized for peak, but pays for itself at average. During the 6-hour off-peak window (2-8 AM Pacific), ~4,500 GPU-hours are "spare." This is where Relax mode is brilliant: it fills those spare GPU-hours with unlimited free-tier generation, keeping utilization high even when paying demand drops.

Without Relax mode, those 4,500 GPU-hours would be wasted daily — at $3/GPU-hr, that is $13,500/day or $4.9 million per year of idle GPU cost. Relax mode turns waste into user goodwill and engagement.

Step 2c: Model Version Impact on Fleet Sizing

Not all Midjourney versions are equal in GPU requirements. Users can choose their model version, and many stick with older versions they are comfortable with. This creates a heterogeneous workload that complicates fleet management.

If 30% of users still use V5/V6 (a reasonable estimate for the transition period), the average GPU-seconds per job is not 20 (V8 optimal) but closer to 40. This doubles the GPU-hours calculation and explains part of the gap between our naive estimate (223 GPUs) and reality (~10,000 GPUs).

Fleet management must also handle model affinity: a GPU that has V8 weights loaded in memory should preferably serve V8 jobs, because switching to V6 weights requires unloading and reloading several gigabytes of parameters (several seconds of downtime). This is why the scheduler is not just a simple FIFO dequeue — it is a match-making system that pairs jobs with compatible GPU workers.

Step 3: Storage

Each generated image is approximately 1–3 MB depending on resolution. V8 generates at up to 2K native resolution[17], so higher-resolution images are becoming the norm. Let us use 2 MB as the average for a single image. A 4-image grid might be stored as a single composite image (smaller than 4× due to JPEG compression of the grid) — estimate 4 MB per grid.

And this is just new images. The total archive includes every image ever generated since launch. The 148+ TB figure on Discord's CDN alone[23] represents just the subset accessible through Discord links — the full corpus in cloud storage is likely several petabytes by now.

At Google Cloud Storage pricing of ~$0.02/GB/month for standard storage, 1 PB costs about $20,000/month. Not cheap, but dwarfed by GPU costs. Storage is not the bottleneck.

Step 4: Bandwidth

Every generated image must be delivered to the user. But it does not stop there — images are shared on social media, embedded in blogs, viewed by other users in Discord channels, and re-fetched when someone scrolls through a gallery. A CDN amplification factor of 3–5x is typical for viral visual content. Midjourney's content is especially viral because users share their best generations.

At $0.08/GB for cloud egress (Google Cloud standard pricing), that is:

$219,000/year for bandwidth. The GPU fleet costs roughly $100M+/year. Bandwidth is a rounding error — about 0.2% of the GPU bill. This is typical for GPU-intensive workloads and explains why Midjourney does not need to obsess over CDN optimization. The compute dominates everything else by two orders of magnitude.

However, there is a hidden bandwidth cost: inter-GPU communication for multi-GPU inference. When a single job is split across 2–4 GPUs via sequence parallelism[16], those GPUs must exchange intermediate activations at every attention layer. For a DiT with, say, 32 attention layers and activations of ~100 MB per layer, that is 3.2 GB of inter-GPU transfer per job. At 625,000 jobs/day, that is 2 PB/day of internal network traffic — orders of magnitude more than the external CDN bandwidth. This is why GPU clusters need high-bandwidth interconnects (NVLink, InfiniBand) and why network topology matters for multi-GPU inference.

Step 4b: Latency Budget

In a system design interview, after estimating throughput and storage, you should present a latency budget: a breakdown of where time is spent in the end-to-end request path. This shows you understand the system at the operation level, not just the capacity level.

For a V8 Fast-mode request, the latency budget looks like this:

The diffusion loop (7 seconds) is ~60–80% of the total time. Everything else combined is under 2 seconds. This confirms what the cost analysis told us: inference dominates. Optimizing the diffusion loop (fewer steps, faster attention, multi-GPU parallelism) has 10x more impact than optimizing any other component.

Notice that the queue wait time is variable (0–5 seconds for Fast mode). During off-peak, a Fast job might start inference within 100ms of enqueue. During peak, the wait stretches to seconds. This variability is why the user-facing UX shows "Generating..." immediately — it masks the queue wait by conflating it with the inference time from the user's perspective.

Step 5: Revenue Per GPU-Hour

This is the number that tells us whether the business model works. It is the single most important metric for any GPU-intensive consumer product.

Now the cost side. An H100 in the cloud costs roughly $2–3/hour depending on commitment level. Add networking, storage, monitoring, and operations overhead. Estimate $4/GPU-hour all-in.

$149M in gross profit is plenty to fund a 192-person team (even at $300K average compensation per person, that is $57.6M in payroll) plus office space, legal, and other overhead. The math works.

Step 6: The Cost Split — 90/10

One of the most revealing numbers: 90% of Midjourney's compute cost is inference, only 10% is training.[6] This is the inverse of what most people assume about AI companies.

Think about it. Training a model happens once per version (well, with checkpointing and iteration, but broadly once). V6 took 9 months to train from scratch[21], but that training produced a model that served hundreds of millions of images over the following year. The training cost is amortized across billions of images; the inference cost is paid per image, every image, forever.

Let us quantify. If 10% of compute cost goes to training on a $100M+ annual compute budget, that is ~$10M/year in training compute. That is 1,000 GPUs running for a year, or equivalently, a smaller cluster of 250 GPUs running for 3 months of focused training per model version. The remaining $90M goes to running 9,000 GPUs 24/7 for inference.

This has profound architectural implications that show up in every design decision:

The interactive calculator below lets you experiment with these numbers. Adjust the sliders to see how changes in daily volume, GPU time per image, fleet size, and revenue cascade through the entire estimation. The orange reference values show Midjourney's actual numbers for comparison.

The Estimation Mindset

In a system design interview, the interviewer does not care if you get the exact number. They care about three things:

1. Can you identify the right starting facts? For Midjourney, the starting facts are: 2.5M images/day, ~10K GPUs, 10-60 seconds per generation, $500M revenue, 90/10 inference/training split. These are the "given" numbers that anchor everything.

2. Can you derive downstream numbers correctly? QPS from daily volume. GPU-hours from QPS and per-job time. Storage from image count and size. Bandwidth from storage with CDN amplification. Revenue per GPU-hour from total revenue and fleet size. Each derivation is a simple division or multiplication, but the chain of reasoning must be sound.

3. Can you identify the bottleneck? For Midjourney, the bottleneck is GPU compute. Not storage ($240K/year for a petabyte). Not bandwidth ($219K/year). Not network (87 QPS is trivial for a load balancer). GPU-hours are the scarce resource, and every architectural decision we study in the remaining chapters is ultimately an answer to one question: how do we maximize the value extracted from each GPU-hour?

The Complete Cost Stack

Let us assemble the full annual cost picture to see where money actually goes. This is the kind of analysis a finance-aware staff engineer would present to leadership.

A 65% operating margin with zero debt, zero VC obligations, and zero public-market pressure. This is an extraordinarily healthy business. The unit economics work because (1) GPU utilization is high thanks to Relax backfill, (2) the price per image covers the per-image GPU cost with margin, and (3) the team is tiny relative to revenue.

Sensitivity Analysis: What If?

The back-of-envelope calculator above lets you explore scenarios, but here are the most interesting "what if" questions an interviewer might ask.

What if daily images doubled to 5M? GPU fleet would need to roughly double (from ~10K to ~20K). Revenue would likely increase proportionally (more paying users). The architecture does not fundamentally change — it is the same system, just scaled horizontally. This is a sign of good architecture: linear scaling with load.

What if inference became 5x faster (as V8 achieved)? Each GPU serves 5x more images per hour. Either (a) the same fleet serves 5x more users (revenue 5x), or (b) the fleet shrinks by 80% (cost 80% lower). Midjourney chose a mix: better quality (2K resolution costs more compute, partially offsetting the speedup) and faster user experience (lower latency drives higher engagement and thus more revenue).

What if GPU prices dropped 50% (next-gen hardware)? Compute cost drops from ~$100M to ~$50M, improving margins. But competitors benefit equally. The real advantage is in software optimization (Flash Attention, quantization, multi-GPU parallelism) that compounds on top of hardware improvements.

What if a free competitor emerged with equivalent quality? This is the existential threat. If open-source models (like Flux or future Stable Diffusion versions) reach Midjourney quality, the community moat and UX polish become the primary differentiators. The infrastructure would still need to serve the community, but pricing power would erode.

Summary: The Numbers That Matter

Here is the complete reference card of derived numbers. Memorize these for system design interviews — not for rote recitation, but because each number anchors a design decision.

Chapter 2: The Architecture

You have the numbers. 29 QPS average, 87 QPS peak, 10,000 GPUs, 2.5 TB of new images per day, 90% of cost in inference, and a 10–60 second generation time per job. Now we need a system that ties all of this together.

This chapter builds the map — the high-level component topology that every subsequent chapter will zoom into. If an interviewer asks you "design Midjourney," this is the diagram you draw on the whiteboard in the first five minutes. Get this right and the rest of the interview is filling in details. Get it wrong and no amount of detail saves you.

The Six Layers

Every scalable system can be decomposed into layers of responsibility. Midjourney's architecture has six distinct layers, each solving a different problem at a different scale.

Each layer has a different scaling profile. The Client layer scales with users (horizontally, trivially). The Gateway scales with requests (also horizontally). The Queue scales with job volume. The Compute layer scales with GPU count (the expensive dimension). Storage scales with total data volume (monotonically increasing, never shrinks). Delivery scales with read traffic (CDN handles this naturally).

The cost distribution is wildly uneven: the Compute layer alone accounts for ~90% of infrastructure cost.[6] The other five layers combined are a rounding error. This means the architecture is effectively a machine for feeding jobs to GPUs as efficiently as possible. Every other layer exists to serve the Compute layer.

Layer Dependencies and Failure Isolation

The six layers have carefully managed dependencies. Understanding these dependencies tells you what can fail independently and what cascades.

The key insight is that the Queue decouples the acceptance path from the processing path. If the Compute layer goes down entirely (a catastrophic scenario), the Gateway and Queue keep accepting jobs. Users see "Your image is being generated" and wait longer, but they do not get errors. When Compute recovers, the queued jobs drain automatically. This decoupling is the entire reason the async architecture exists.

Similarly, if the Client layer (Discord) has an outage, the Compute layer keeps processing jobs. Results are stored but cannot be delivered until Discord recovers. When it does, the Delivery layer retries pushing the stored results. No work is lost.

This failure isolation is not accidental. It emerges from two design principles that apply to all distributed systems:

1. Never couple fast paths to slow paths. The gateway's response time (milliseconds) must never depend on the GPU's processing time (seconds). The queue is the buffer that decouples them. If the GPU is slow, the queue grows, but the gateway stays fast.

2. Store the work, not the connection. The job's state lives in the queue (durable storage), not in an HTTP connection (ephemeral). If any component crashes, the job persists and can be recovered. This is the fundamental difference between a message queue architecture and a connection-based architecture.

The Architecture Diagram

The interactive diagram below shows all six layers and the data flow between them. This is the diagram you would draw on a whiteboard. Click any layer to see its detailed description — the technology choices, the scale constraints, and the key design decisions.

Data Flow: The Full Path

Let us trace a single request through every layer, from the moment a user types /imagine to the moment they see their image. This nine-step path takes 10–60 seconds end-to-end, but most of that time is spent in a single step (GPU inference). Understanding where time is spent tells you where to optimize.

Time Budget Breakdown

Let us add up where time is actually spent for a Fast mode V8 request:

GPU inference dominates. Everything else combined takes under a second. This confirms our Chapter 1 finding: GPU compute is the bottleneck. All architectural optimization should focus on either reducing inference time (model optimization, quantization, Flash Attention) or maximizing GPU utilization (queue scheduling, Relax backfill, multi-GPU parallelism).

The Job State Machine

Every job in the system moves through a well-defined state machine. Understanding this state machine is essential for designing the queue, the scheduler, and the delivery system.

Two transitions deserve special attention:

PROCESSING → QUEUED (failure recovery). If the GPU worker dies mid-inference (hardware failure, OOM, thermal shutdown), the heartbeat times out and the job automatically re-enters the queue. The seed ensures the re-run produces identical images (deterministic generation). The user experiences a delay but not a failure. This is the key resilience mechanism.

UPLOADING → QUEUED (moderation rejection). If the post-generation image classifier flags the output, the job is re-queued with a new random seed. The system tries to generate a compliant image from the same prompt. If it fails multiple times (configurable, perhaps 3 attempts), the user receives a rejection message.

Why Async?

Let us do the math on why a synchronous request-response architecture would fail.

At peak, 22 jobs/second (87 QPS ÷ 4 images per job) with an average processing time of 30 seconds (blending Fast and Relax modes). If each request held a connection open for the full duration:

That is actually manageable for a modern load balancer. But the real problems are deeper.

Timeout chain. HTTP clients, proxies, CDNs, and load balancers each enforce timeout limits. A typical chain: browser (120s) → CloudFlare (100s) → nginx (60s) → application (30s). A Relax mode job that takes 5 minutes would be killed by every proxy in the chain. You would need to configure every component in the path with 10-minute timeouts, which creates resource exhaustion risks.

Connection resource waste. Holding an HTTP connection open for 30 seconds ties up a file descriptor, a thread (or coroutine), and memory on both the client and server. At 660 concurrent connections this is fine, but if you add long-tail Relax jobs (5+ minutes), the numbers balloon.

Retry complexity. If a connection drops mid-generation (network blip, client timeout, server restart), you lose the job. With async, the job exists independently of the connection — a dropped connection just means the delivery step retries.

The async model is simpler and more robust: accept the job in <100ms, return immediately with a job ID, process asynchronously, and push the result when ready. This decouples the frontend's response time from the backend's processing time. The frontend always responds in milliseconds, regardless of how long the GPU takes.

Priority Lanes

The queue is not a simple FIFO. It has three priority lanes that map directly to the subscription tiers and create a remarkably elegant resource allocation scheme.

Relax mode is a masterstroke of resource economics. It implements a concept from operations research called yield management (the same principle airlines use with standby passengers).

When Fast and Turbo demand is high (peak hours), Relax jobs wait. The GPUs serve paying customers first. When demand drops (off-peak), Relax jobs fill the idle GPU time. The result: GPUs run at near-100% utilization 24/7. No GPU-hour is wasted. Paid users get priority. Free capacity is gifted to patient users, building goodwill and engagement.

From a queuing theory perspective, Relax mode converts Midjourney's GPU fleet from a loss system (where excess demand is dropped) into a delay system (where excess demand is buffered). This increases effective throughput without adding hardware.

The 4-Image Grid: An Architectural Choice

We mentioned this in Chapter 0, but it is worth emphasizing the full architectural dimension. Generating 4 images per job has four system-level benefits.

1. Amortized overhead. Queue scheduling, prompt encoding, job setup, result delivery, and moderation happen once per job. By producing 4 images per job, the overhead-per-image drops to 25%. The text encoder (CLIP-family) runs once; only the initial noise differs between the 4 images.

2. Batch efficiency. Modern GPUs achieve peak TFLOPS with larger batch sizes. A batch of 4 images through the DiT backbone saturates the GPU's streaming multiprocessors more efficiently than a batch of 1. Empirically, generating 4 images takes roughly 1.5–2× the wall-clock time of 1 image, not 4×.

3. Reduced re-rolls. Four options means users are more likely to find an acceptable result. This reduces re-roll requests by perhaps 2–3×, directly reducing GPU load per "satisfying result."

4. Engagement driver. The upscale (U1-U4) and variation (V1-V4) buttons on the 4-image grid encourage further interaction. Each upscale or variation is another job, driving more engagement and GPU time consumption — and for paid users, more revenue.

Technology Choices

Midjourney keeps most infrastructure details private, but we can infer the likely technology stack from confirmed facts, GitHub activity, and standard practices for GPU-intensive workloads at this scale.

Component Interaction Patterns

The six layers interact through well-defined patterns that repeat in every large-scale distributed system. Recognizing these patterns is what separates a staff architect from a senior engineer — you see them once here, and you apply them everywhere.

Fan-out at the queue. A single job from the queue may fan out to multiple GPU workers (for multi-GPU inference via sequence parallelism[16]). The scheduler must coordinate this fan-out, track partial completion, handle worker failures, and collect results from all participating GPUs before declaring the job complete.

Write-behind to storage. The GPU worker writes the generated image to object storage asynchronously. The CDN URL is returned to the user before the image is fully replicated across all storage regions. This is acceptable because the CDN will fetch-on-miss from the primary region, and the image only needs to be available within seconds, not milliseconds.

Event-driven delivery. The result is pushed back to the user via an event (Discord bot editing its message, or WebSocket push to the web app). There is no polling loop — the system notifies the user proactively. This eliminates the thundering-herd problem of millions of clients polling for completion.

Backpressure via queue depth. When the GPU fleet is overwhelmed, the queue grows. The queue depth becomes the backpressure signal: the gateway can reject new Relax jobs when the Relax queue exceeds a threshold, and it can increase estimated wait times shown to users. The queue is the pressure valve that prevents GPU overload.

Failure Modes and Resilience

At 10,000 GPUs, failures are not exceptional events — they are a constant reality. Understanding how the architecture handles failures tells you whether the system was designed by someone who has actually operated infrastructure at scale.

The most important design principle here is idempotent job processing. Because a GPU failure can kill a job at any point during inference, every job must be safely re-runnable. The queue tracks job state (queued → processing → completed/failed), and any job that stays in "processing" too long (heartbeat timeout) is automatically re-queued. The seed ensures that a re-run produces the same images (deterministic generation), so the user does not notice the failure.

Comparing to Similar Systems

Midjourney's architecture pattern — async job queue with GPU compute backend — is not unique. Recognizing the pattern family helps you transfer knowledge from other systems you may have studied.

The async job processing pattern is one of the most common in distributed systems. If you learn it deeply through Midjourney, you can apply it to any of these systems. The only variables are: job duration, compute type (GPU vs CPU), output size, latency expectations, and priority semantics.

Whiteboard Summary

If you had to draw Midjourney's architecture in 2 minutes on a whiteboard, here is the minimum viable diagram. Six boxes, five arrows, three annotations:

Then annotate: "Async everywhere. Job acceptance is <100ms. Processing is 10-60 seconds. Delivery is push-based (message edit or WebSocket). GPU compute is 90% of cost. Relax mode fills idle capacity."

That is six boxes, five arrows, and five annotations. It takes 90 seconds to draw. It communicates the entire system at the right level of abstraction. And it gives the interviewer six entry points to drill into: "Tell me more about the Queue," "How does the GPU Scheduler work," "What happens when a GPU fails?" Each entry point leads to a 5-minute deep dive that you are now prepared for.

What Makes This Architecture Good

Before moving on, let us explicitly name the design qualities that make this architecture work at Midjourney's scale. These are the qualities an interviewer is evaluating when they ask "why did you design it this way?"

1. Clear separation of concerns. Each layer has one job. The Gateway does not generate images. The GPU fleet does not authenticate users. This makes each layer independently scalable, testable, and deployable.

2. Failure isolation. A GPU failure does not crash the Gateway. A Discord outage does not stop inference. The Queue is the circuit breaker between fast and slow components.

3. Elastic capacity. The system handles variable load through queueing (Relax absorbs excess demand) and priority scheduling (paying users get priority when capacity is scarce). No component needs to be provisioned for peak — only the GPUs, and Relax fills the off-peak gap.

4. Cost-proportional pricing. Revenue scales with the most expensive resource (GPU compute). The $4/GPU-hour pricing[18] directly ties user spending to infrastructure cost. The business model and the architecture reinforce each other.

5. Observable. Every job has a state (QUEUED, PROCESSING, COMPLETED). Every GPU has a heartbeat. Every queue has a depth metric. The system's health is observable at every layer, enabling proactive capacity management and fast incident response.

These five qualities are not unique to Midjourney. They are the hallmarks of any well-designed distributed system. YouTube's transcoding pipeline, Uber's ride-matching system, and Stripe's payment processing all exhibit the same qualities. Learning them here means recognizing them — and applying them — everywhere.

What the Next Chapters Cover

We now have the complete topology. The remaining chapters zoom into each component with implementation-level detail:

Chapter 3: Discord Bot & API Gateway

In most system design problems, the frontend is an afterthought — "assume we have a web app." Midjourney is different. The Discord bot is not just a frontend; it is the most important architectural decision the company ever made. It eliminated entire categories of infrastructure that would have taken months to build, and it created a viral growth engine that no marketing budget could replicate.[13]

In this chapter, we will trace the exact path a message takes from the moment a user types /imagine to the moment the job enters the queue. Every step along this path is a design decision worth understanding, because each one has implications for latency, reliability, scalability, and cost.

Discord's Interaction API

Discord bots receive user commands through the Interactions API. This is a webhook-based system where Discord POSTs payloads to a registered endpoint when users invoke slash commands. Understanding the protocol is essential because it imposes hard constraints on Midjourney's architecture.

When a user types a slash command like /imagine, here is what happens at the protocol level:

The Interaction Object

Let us look at what Discord actually sends to Midjourney's webhook. Understanding the data model helps you see what information is available for routing, authentication, and billing decisions.

json
{
  "id": "1234567890",
  "type": 2,                        // APPLICATION_COMMAND
  "application_id": "936929...",  // Midjourney's bot ID
  "guild_id": "662267...",        // Which server
  "channel_id": "995432...",      // Which channel
  "member": {
    "user": {
      "id": "448596...",          // Unique user ID → billing key
      "username": "artist42"
    }
  },
  "data": {
    "name": "imagine",              // Command name
    "options": [{
      "name": "prompt",
      "value": "a cyberpunk city --ar 16:9 --v 6 --stylize 750"
    }]
  }
}

The user ID is the billing key. It maps to a Midjourney subscription record that determines: which priority lane (Basic/Standard/Pro/Mega), how much Fast GPU time remains this month, the concurrency limit (3/5/12/15), and whether the user is in good standing (not banned, payment current).

What Discord Gives You for Free

To appreciate why Discord was a strategic choice, not a lazy one, consider the complete inventory of infrastructure Midjourney did NOT have to build.

Conservative estimate: building all of this from scratch would take a team of 20+ engineers 6–12 months. Midjourney got it for free by writing a Discord bot. With a team of 10 engineers in 2021[9], there was literally no other viable path to launch.

Discord as Rate Limiter

This is subtle but architecturally important. Discord imposes its own rate limits on bot interactions: a bot can send a limited number of messages per second per channel, and users can invoke slash commands at a limited rate. These limits act as a natural backpressure mechanism between users and Midjourney's backend.

Specifically, Discord's rate limits provide three layers of protection that Midjourney gets for free:

1. Per-user rate limiting. A user cannot spam slash commands faster than Discord allows. This prevents any single user from flooding Midjourney's gateway, even if they write a script to automate commands.

2. Per-channel rate limiting. In busy public channels, Discord throttles bot responses to prevent message spam. This naturally limits how fast Midjourney generates images in shared contexts.

3. Global bot rate limiting. Discord imposes overall rate limits on bot API calls (message sends, edits, etc.). This prevents Midjourney's progress updates and delivery messages from overwhelming Discord's infrastructure.

Without Discord, Midjourney would need to build its own rate limiting layer with per-user token buckets, global rate limiting, concurrency tracking (max 3–15 active jobs per user[19]), and abuse detection. Discord provides the first two for free. Midjourney only needs to implement concurrency tracking (application-level check: "how many active jobs does this user have?") and content moderation (the pre-generation text filter[20]).

Prompt Parsing

When a user types /imagine a cyberpunk city --ar 16:9 --v 6 --stylize 750 --no cars, the bot must parse this into a structured job object. The prompt text and flags need to be separated, validated, and converted into parameters that the inference pipeline understands.

pseudo
function parsePrompt(raw_text):
    # Split prompt text from parameter flags
    parts = raw_text.split("--")
    prompt_text = parts[0].strip()    # "a cyberpunk city"

    # Parse each flag into key-value pairs
    params = {}
    for part in parts[1:]:
        tokens = part.strip().split(" ", 1)
        key = tokens[0]
        value = tokens[1].strip() if len(tokens) > 1 else "true"
        params[key] = value

    # Result:
    # prompt_text = "a cyberpunk city"
    # params = {
    #   "ar": "16:9",       → aspect ratio
    #   "v": "6",           → model version
    #   "stylize": "750",   → creativity level (0-1000)
    #   "no": "cars"        → negative prompt
    # }

    # Validate and construct job spec
    return JobSpec(
        prompt       = prompt_text,
        aspect_ratio = parseAR(params.get("ar", "1:1")),
        model_ver    = int(params.get("v", 6)),
        stylize      = clamp(int(params.get("stylize", 100)), 0, 1000),
        neg_prompt   = params.get("no", ""),
        quality      = float(params.get("q", 1.0)),
        seed         = int(params.get("seed", random())),
        chaos        = clamp(int(params.get("chaos", 0)), 0, 100),
    )

Each parsed parameter affects inference behavior and system resource usage in specific ways:

The Moderation Pipeline

Content moderation happens at two stages, and the pre-generation stage is the most cost-effective system component in the entire architecture. Let us understand why.[20]

Stage 1: Pre-generation text filter. Before the prompt enters the job queue, a text classifier scans it for banned content categories: explicit violence, NSFW content, specific public figures, copyrighted characters, and other policy violations. This classifier is fast — likely a small fine-tuned language model or even a sophisticated keyword/regex system running in single-digit milliseconds.

If the prompt is flagged, the user gets an immediate rejection message ("Your prompt was blocked by our content filter") and no GPU time is consumed. This is the key insight: every banned prompt caught at this stage saves 5 petaops of compute that would have been wasted generating an image that would be blocked anyway.

Let us quantify the savings rigorously. Suppose 5% of prompts violate content policy (a conservative estimate given the diversity of 2.5M images per day from 20M accounts).

The text filter itself probably costs less than $1,000/year to run (a small model on a single CPU instance). The ROI is roughly 190x. This is why the moderation pipeline is a system component, not a policy feature.

Stage 2: Post-generation image classifier. After the GPU generates the images, an image classification model scans the output for visually problematic content. This catches cases where the text prompt seemed innocuous but the generated image is not — a surprisingly common occurrence. The model's latent space contains associations from training data that can produce unexpected visual content from apparently benign prompts.

The post-generation classifier is more expensive (it runs on GPU, processing actual image pixels) but catches violations that no text filter can predict. Together, the two stages form a defense-in-depth pattern: cheap filter first, expensive filter second.

The Progress Update Trick

One of the most user-visible pieces of engineering is the progress update. As the diffusion model runs, the Discord bot edits its message to show a blurry preview that progressively sharpens, along with a percentage counter. This is not just cosmetic — it is a critical UX decision with real engineering implications.

The diffusion model generates images iteratively, starting from pure noise and denoising over N steps (typically 20–50 steps). At each intermediate step, the current state of the latent can be decoded back into pixel space. The result is a blurry, noisy preview early on that becomes sharper and more detailed with each step. Midjourney periodically (every few steps) sends this intermediate rendering back to the Discord bot, which edits its message to show the preview.

pseudo
function generateWithProgress(job, discord_msg_id):
    # Step 1: Encode prompt text via CLIP-family model
    text_emb = encodeText(job.prompt, job.neg_prompt)

    # Step 2: Sample initial noise (4 images, one per grid slot)
    noise = sampleNoise(
        batch=4,
        seed=job.seed,
        height=job.latent_h,  # from aspect ratio
        width=job.latent_w
    )

    # Step 3: Iterative denoising loop
    num_steps = job.quality * 25  # quality=1 → 25 steps
    for step in range(num_steps):
        noise = denoise_step(noise, text_emb, step, num_steps)

        # Send progress preview every 5 steps
        if step % 5 == 0 and step > 0:
            preview = decodeLatent(noise)         # VAE decode
            preview_small = resize(preview, 256)  # Low-res for speed
            preview_url = uploadTemp(preview_small)
            pct = int((step / num_steps) * 100)
            editDiscordMsg(discord_msg_id,
                text=f"**{job.prompt}** — {pct}%",
                image=preview_url)

    # Step 4: Final decode at full resolution
    final_images = decodeLatent(noise)   # Full-res VAE decode
    grid = makeGrid(final_images, 2, 2) # 2×2 grid layout

    # Step 5: Post-moderation check
    if moderateImage(grid):
        return regenerate(job)  # Flagged → try again

    # Step 6: Upload and deliver
    final_url = uploadToGCS(grid)
    editDiscordMsg(discord_msg_id,
        text="",
        image=final_url,
        buttons=["U1","U2","U3","U4","V1","V2","V3","V4","🔄"])

The engineering cost of progress updates is not trivial. Each message edit is a Discord API call (subject to Discord's bot rate limits). Each preview requires a partial decode of the latent space through the VAE decoder (additional GPU time, though much less than a full denoising step). The preview images must be uploaded to temporary storage and then to Discord's CDN. For a 25-step generation sending previews every 5 steps, that is 4 additional API calls, 4 VAE decodes, and 4 image uploads per job.

But the UX benefit is enormous. Users see progress instead of staring at a loading spinner, which dramatically reduces perceived wait time. Psychological research shows that showing progress (even approximate) reduces perceived wait time by 30-40%. For a 10-second generation, that is the difference between "this is fast" and "why is this taking so long?"

The Web App: Same Backend, Different Frontend

When Midjourney launched alpha.midjourney.com with V6.1 in August 2024[13], the web app needed to connect to the same backend infrastructure as the Discord bot. This means the API gateway must route requests from two different frontends to the same job queue.

The architectural pattern is straightforward: both the Discord bot handler and the web app API are thin clients that validate input, check rate limits, and enqueue jobs. The entire inference, storage, and delivery pipeline is shared. The only difference is the delivery mechanism:

The web app path requires Midjourney to build more infrastructure: their own authentication system, their own rate limiting, their own WebSocket service for real-time updates. This is precisely the infrastructure that Discord provided for free in the early days. Building the web app was only feasible once the team had grown and the revenue could fund the additional engineering.

The API Question

Midjourney has no official public API. This is a deliberate product decision, not a technical limitation. Third-party tools that integrate with Midjourney do so by reverse-engineering Discord's Interaction API — they send bot commands through Discord as if they were a user, then listen for the bot's response messages.

This works, but it is fragile. Discord can change their API. Midjourney can detect and ban automated accounts. The interaction rate is limited by Discord's rate limits. And it violates Midjourney's Terms of Service.

Why no official API? Because an API would fundamentally change the economics. An API enables:

• Automated bulk generation — businesses could generate millions of images programmatically, straining GPU resources
• Reselling — third parties could build products on top of Midjourney's GPU fleet, competing with Midjourney itself
• Price arbitrage — at $4/GPU-hour, an API would be cheaper than competitors' per-image pricing, incentivizing high-volume automated usage
• Loss of UX control — API users bypass Midjourney's moderation UI, sharing features, and community experience

By forcing users through Discord or the web app, Midjourney maintains control over the user experience, rate limiting, and revenue per GPU-hour. The concurrency limits (3–15 per user[19]) and queue limits (10 pending jobs) are enforceable because there is no API bypass.

Gateway Architecture

The API gateway sits between both frontends and the job queue. Its responsibilities form a clear, sequential pipeline where each step can reject the request early, saving cost on downstream processing.

Each of these steps takes single-digit milliseconds (except moderation, which might take 10-50ms depending on the text classifier's complexity). The entire pipeline runs in under 100 milliseconds — well within Discord's 3-second deadline.

The ordering of steps is deliberate: cheap checks first. Authentication is a simple database lookup. Rate limiting is a counter check. Prompt parsing is string manipulation. Only after all of these pass do we run the more expensive content moderation classifier. This is the bouncer pattern in distributed systems: the cheapest component in the pipeline does the most filtering, so that expensive components (GPUs at $3/hour) only process validated, prioritized, and moderation-cleared work.

Scaling the Gateway

At 22 jobs/second peak (87 QPS / 4 images per job), the gateway itself is not a bottleneck. Any modern API gateway (nginx, Envoy, Kong, or a custom Go/Rust service) can handle tens of thousands of requests per second. The gateway's scaling challenge is not throughput — it is consistency.

The concurrency check ("how many active jobs does this user have?") requires a shared counter that is consistent across all gateway instances. If two gateway instances both check simultaneously and both see "2 active jobs" when the limit is 3, they might both accept a new job, putting the user at 4 — exceeding the limit. This requires either a centralized counter (Redis INCR) or an eventually-consistent approach with occasional over-admission.

At Midjourney's scale (22 jobs/second, each involving one counter check), a single Redis instance handles this trivially. This is another area where Midjourney's relatively modest QPS (compared to, say, Google Search at 100K+ QPS) makes the engineering simpler than it might appear.

Chapter 4: The Queue & Job Orchestration

You just typed /imagine a cyberpunk fox reading a newspaper and hit Enter. Your prompt vanishes into a bot that serves twenty million Discord accounts[11]. Somewhere, a GPU needs to spend ten to sixty seconds generating your image. But right now, a million other people are online too[11]. How does the system decide who goes first, who waits, and who gets told "Queue full, try again later"?

This is a job orchestration problem, and it is the single most important piece of infrastructure between your prompt and your pixels. Get it wrong, and paying users wait behind free-tier floods. Get it right, and you can serve 2.5 million images per day[12] with a team of forty engineers[9].

Why FIFO Doesn't Work

The naive approach is a First-In, First-Out queue. Jobs enter at the back, leave at the front. Simple, fair, and completely wrong for a business that charges $10-120/month for faster access.

Consider: a Relax-mode user submits 50 images. Each takes 30 seconds of GPU time. That is 25 minutes of GPU capacity consumed. If a Turbo-mode user (paying 2x the GPU cost[27]) joins the back of the line, they wait behind all 50 Relax jobs. The person paying more gets worse service than the person paying nothing. Revenue collapses. This is why every serious generation service uses priority queues.

Three Priority Lanes

Midjourney solves this with three priority lanes, each with different latency guarantees and cost structures:

Turbo mode is not just "jump the queue." The generation itself runs faster too — likely using more GPUs per image or more aggressive parallelism[27]. Think of it as both priority AND resource allocation. Fast mode is the default — about one minute of GPU time per image[26]. Relax mode is the "I have time" lane — you get unlimited images, but you wait until GPUs are available.

Per-User Concurrency Limits

Priority lanes alone are not enough. Without limits, a single Pro user could flood the Turbo lane with hundreds of concurrent jobs. So Midjourney enforces per-user concurrency caps:

The queue limit is separate from concurrency[19]. You can have 3 jobs actively processing AND 10 more waiting in your personal queue. Submit an 11th and you get "Queue full." This prevents any user from monopolizing dispatch bandwidth.

Job Lifecycle

Every generation request passes through a well-defined state machine. Understanding these states is critical for designing retry logic, dead letter queues, and monitoring dashboards:

Dead Letter Queues

What happens when a job fails? GPU runs out of memory (a 2K image at high upscale can exceed VRAM). The diffusion process produces a degenerate output. The worker crashes mid-inference. You cannot just drop these jobs — the user is staring at a progress bar.

A dead letter queue (DLQ) captures failed jobs with their full context: the prompt, parameters, error code, GPU ID, and timestamp. The orchestrator can then retry on a different worker (maybe one with more VRAM), notify the user if retries are exhausted, and aggregate failure patterns for debugging (e.g., "all failures are on GPU node 47 — it has bad memory").

Batch Optimization

Here is a subtle but powerful optimization. GPU utilization is highest when you fill all available compute. A single 512×512 image might only use 40% of a GPU's capacity. But Midjourney generates 4 variations per request[12]. That is four images batched into one GPU dispatch, amortizing the model loading and memory allocation overhead.

The orchestrator can go further: group multiple users' small jobs onto the same GPU if they fit. This is bin packing — the same problem that Kubernetes solves for CPU containers, but applied to GPU memory and compute slots.

Worked Example: Capacity Planning

Let us break this down further. Daily throughput: 2.5 million images[12] divided by 86,400 seconds per day = 29 images per second. That aligns perfectly with the 20-40 jobs/sec range. Each image consumes roughly 5 petaops[7]. At 29 images/sec, total throughput is 145 petaops/sec. An A100 delivers ~312 TFLOPS (FP16), so you need 145,000 / 312 = ~465 GPUs continuously saturated. With 10,000 GPUs, that is 4.6% average utilization — the rest handles bursts, Turbo mode parallelism, and the long tail of Relax jobs that accumulate during peak hours.

The Technology Stack

Midjourney has not publicly disclosed their queue technology, but the workload pattern strongly suggests either Apache Kafka or Redis Streams. Both support multiple consumer groups (one per priority lane), exactly-once processing semantics, and the ability to replay failed jobs. Kafka is the more common choice at this scale — it handles millions of messages per second, provides durable storage, and integrates naturally with Kubernetes-based GPU orchestration.

The dispatcher likely runs as a separate service that polls all three lanes, preferring Turbo over Fast over Relax, and matches jobs to available GPU workers based on memory requirements, current load, and geographic proximity (to minimize data transfer latency).

Chapter 5: The GPU Inference Pipeline

The queue dispatched your job to a GPU worker. Now what? This is where the real compute happens — and where 90% of Midjourney's cost goes[6]. A single image generation burns through 5 petaops (5 × 10¹⁵ floating-point operations)[7]. To put that in perspective, multiplying two 1000×1000 matrices takes 2 billion operations. Generating one Midjourney image is equivalent to doing that 2.5 million times.

Where do all those operations go? Into the iterative diffusion process that transforms random noise into a coherent image, one small step at a time.

Diffusion from Zero

Imagine you have a photograph. Now add a tiny bit of random noise — like TV static mixed into every pixel. Do it again. And again. After a thousand steps of adding noise, the original image is completely destroyed. All you have is pure random static. Diffusion models learn to reverse this process.

The model is trained by showing it millions of images at various noise levels and teaching it to predict what the "less noisy" version looks like. At inference time, you start with pure noise and ask the model: "If this noise came from a real image, what would one step of denoising look like?" Then you feed that slightly-less-noisy result back in and ask again. After 20-75 steps, structure emerges from chaos.

Why 5 Petaops?

Let us derive this number. A modern DiT (Diffusion Transformer) model has roughly 2-10 billion parameters[30]. Each forward pass through a transformer requires approximately 2 × N operations per parameter (one multiply, one add). For a 5B parameter model, that is 10¹⁰ operations per forward pass. With 50 denoising steps, that is 50 × 10¹⁰ = 5 × 10¹¹. But this ignores the attention mechanism's quadratic cost. For a 1024×1024 image tokenized into 4096 patches, self-attention alone costs 2 × 4096² × d_model per layer. Multiply by 30+ transformer layers, multiply by 50 steps, and you land squarely at 5 × 10¹⁵ = 5 petaops.

Plugging in: 50 steps × (2 × 5×10⁹ + 32 × 2 × 4096² × 1280) ≈ 5 × 10¹⁵.

The DiT Architecture

Midjourney uses a Diffusion Transformer (DiT) — confirmed by their public fork of the xDiT parallelism library on GitHub[14]. This is a critical architectural choice that distinguishes modern image generators from earlier ones.

Older diffusion models (Stable Diffusion v1, DALL-E 2) used U-Net architectures — a convolutional neural network with skip connections. U-Nets work well at moderate resolutions, but their convolutional layers have fixed receptive fields. Every pixel only "sees" its local neighborhood. To capture global relationships (like "the fox's newspaper should have text that matches the lighting"), you need many layers and careful upsampling.

DiT replaces convolutions with transformer blocks — the same self-attention mechanism that powers GPT. Every image patch attends to every other patch at every layer. Global coherence is built-in, not bolted-on. The trade-off: attention scales quadratically with the number of patches. A 1024×1024 image with 16×16 patches has 4096 tokens. Self-attention on 4096 tokens is manageable. At 2048×2048 (16,384 tokens), it becomes brutal without optimization.

Text Conditioning: From Prompt to Cross-Attention

How does the model know you asked for a "cyberpunk fox"? Through text conditioning. Your prompt is first encoded by a text encoder — likely a CLIP-family model — into a sequence of embedding vectors. Each word (or subword token) becomes a vector of ~768-1024 dimensions.

These text embeddings enter the DiT through cross-attention. At every transformer layer, the image patches (queries) attend to the text embeddings (keys and values). This is how the model steers: patches that should depict "fox" receive high attention weights from the "fox" text token. The --stylize parameter controls how strongly the model follows these text signals versus its own aesthetic training — high stylize means "make it beautiful even if it drifts from the prompt."

Multi-GPU Parallelism: xDiT

A single high-resolution image can exceed what one GPU can handle. Midjourney's xDiT fork[16] supports three parallelism strategies, each splitting the workload differently:

Flash Attention + SageAttention

Even with multi-GPU parallelism, attention is the bottleneck. Standard attention computes a full 4096 × 4096 attention matrix, which is 64 MB in FP16 — per layer, per step. Flash Attention avoids materializing this matrix by computing attention in tiles that fit in GPU SRAM (shared memory), reducing memory usage from O(n²) to O(n).

Midjourney goes further with SageAttention[15], which quantizes the attention computation to 8-bit integers. This is 2.1x faster than FlashAttention2 with minimal quality loss. The key insight: attention scores are relative (they go through softmax), so absolute precision matters less than rank ordering. INT8 preserves the ranking while halving the memory bandwidth.

The V8 Migration: JAX/TPU → PyTorch/GPU

For years, Midjourney trained on Google TPUs using JAX[2]. In March 2026, V8 launched as a complete rewrite in PyTorch on GPUs[4]. The result: 5x faster generation, native 2K resolution, under 10 seconds per image[17].

Why would they rewrite the entire stack? Three reasons. First, the GPU ecosystem (CUDA, cuDNN, TensorRT, Flash Attention, xDiT) is far more mature than the TPU ecosystem for inference optimization. Second, PyTorch's eager execution makes debugging and iteration faster than JAX's functional compilation model. Third, inference runs on GPUs regardless[3] — training on TPUs while serving on GPUs meant maintaining two codebases. Unifying on PyTorch/GPU eliminated that burden.

The 4-Image Grid: Amortizing Overhead

When you run /imagine, Midjourney generates a 2×2 grid of four variations[12]. This is not just a UX choice — it is a compute optimization. Loading model weights from GPU HBM to SRAM takes time. Setting up the execution context (CUDA kernels, memory allocations) takes time. By batching four images, you amortize that overhead across four outputs. The marginal cost of image 2, 3, and 4 is much less than image 1.

python
# Simplified diffusion inference loop
# Real Midjourney uses DiT + Flash Attention + SageAttention

import torch

def generate_images(prompt_embedding, model, steps=50, batch=4):
    # Start with pure noise (batch of 4 images)
    x = torch.randn(batch, 4, 64, 64)  # latent space
    
    # Denoising schedule: high noise → low noise
    timesteps = torch.linspace(1.0, 0.0, steps)
    
    for t in timesteps:
        # Each step = FULL forward pass through DiT
        # Cross-attention with text embeddings
        noise_pred = model(x, t, prompt_embedding)
        
        # Remove predicted noise (simplified)
        alpha = get_schedule(t)
        x = (x - (1 - alpha) * noise_pred) / alpha.sqrt()
    
    # Decode latents to pixels (VAE decoder)
    images = vae.decode(x)  # 4 x 3 x 1024 x 1024
    return images  # 4-image grid

Worked Example: GPU Memory Budget

Chapter 6: Image Storage & Delivery

The GPU just finished denoising. Four images sit in GPU memory as raw pixel tensors — 4 × 3 × 1024 × 1024 floating-point values. Now what? Those pixels need to travel from a Google Cloud GPU in some data center to a Discord message on your phone, ideally in under two seconds. And this has to happen 2.5 million times per day[12], every day, without losing a single image.

This chapter traces the storage and delivery pipeline from GPU memory to your screen, and confronts the surprising economics of serving images at scale.

The Scale Problem

Let us start with the raw numbers. Each Midjourney image, after post-processing and compression, is roughly 1-3 MB. At 2.5 million images per day using a 2 MB average:

Five terabytes per day sounds terrifying until you realize that storage is cheap. The real cost is bandwidth. Every time a user views an image, scrolls back to an old generation, shares it in a channel, or opens the web app, those bytes travel from a CDN edge node to the user's device. That is egress, and cloud providers charge dearly for it.

The Post-Processing Pipeline

Before any image reaches storage, it passes through a post-processing pipeline on the GPU worker (or a nearby CPU worker):

Discord CDN Delivery

For the first three years of Midjourney, Discord was not just the interface — it was the delivery infrastructure[13]. When the bot sends your completed image, it uploads it as a Discord attachment. Discord hosts these on its own CDN (backed by Google Cloud and Cloudflare). The URL looks like:

url
https://cdn.discordapp.com/attachments/{channel_id}/{message_id}/{filename}.png
  ?ex=6789abcd    # expiry timestamp (hex)
  &is=12345678    # issue timestamp
  &hm=abc123...   # HMAC signature

The signed expiring URLs[28] are critical for security. Without the HMAC signature, you cannot access the image. When the URL expires, you need a fresh signature. This prevents hotlinking (other sites embedding Midjourney images for free) and gives Discord control over bandwidth costs.

Tiered Storage Strategy

Not all images deserve the same storage treatment. An image generated 30 seconds ago will be viewed many times in the next few minutes (the user is iterating). An image from six months ago might never be viewed again. A rational storage system uses tiers:

Images automatically migrate down tiers based on age and access frequency. If a user revisits an old image, it gets temporarily promoted back to the hot tier (this is standard GCS lifecycle management).

Content-Addressable Storage

Here is a subtlety. Two users with the same prompt and seed will generate identical images. Storing both is wasteful. Content-addressable storage (CAS) solves this: hash the image bytes, use the hash as the storage key. If the hash already exists, return a pointer to the existing blob instead of storing a duplicate. Even a 1% deduplication rate saves 1.5 TB per month at Midjourney's scale.

Worked Example: Storage vs. Bandwidth Costs

This also explains why Discord CDN was such a gift to Midjourney's early economics. Discord absorbed the egress costs. When Midjourney launched its own web app, they suddenly owned that $60K/month bandwidth bill (likely much higher with web app traffic added).

The Web App Challenge

With the web app launched in August 2024[13], Midjourney now serves images through two channels: Discord (where Discord pays for CDN) and the web app (where Midjourney pays). The web app likely uses its own CDN — Cloudflare or Google Cloud CDN — with edge caching at dozens of global PoPs (Points of Presence).

Edge caching means that when a user in Tokyo views an image, the first request fetches it from the origin server (US) and caches it at the Tokyo edge node. The next Tokyo user who views that image gets it from the local cache, saving a transpacific round trip (~150 ms) and egress cost. Cache hit rates of 60-80% are typical for image CDNs, which would cut Midjourney's effective bandwidth cost by 3-5x.

Video: The Next Storage Challenge

Midjourney launched Video V1 in June 2025 — 5-second clips at 8x the GPU cost of a still image[29]. A 5-second video at 24 fps in H.264 is roughly 5-15 MB — 5x larger than a still image. If even 10% of generations shift to video, daily storage ingest jumps from 5 TB to 7.5 TB, and bandwidth costs increase proportionally. Video also cannot be served as static files — it needs adaptive bitrate streaming (HLS/DASH) for smooth playback across connection speeds.

Chapter 7: End-to-End Request Trace

You have now seen the queue, the GPU pipeline, and the storage layer individually. But real understanding comes from tracing a single request through the entire system. This chapter follows one /imagine a cyberpunk fox reading a newspaper --v 7 --q 1 command from the moment you press Enter to the moment the image appears in your Discord channel. We will measure latency at every hop and identify where time actually goes.

This is the chapter that makes the architecture real. Memorize this trace and you can whiteboard Midjourney's system design in any interview.

The Happy Path (Fast Mode, ~15-20 seconds)

Here is every hop, with measured or estimated latency at each stage:

Latency Budget Breakdown

Let us add up the total for a V7 Fast mode generation:

The Progress Update Mechanism

While the GPU is working, the user is staring at a progress bar. How does it update? The GPU worker sends progress messages back to the bot server after every N denoising steps. The bot server then edits its Discord message using the Discord API's message edit endpoint. A typical generation shows four progress states: 0% → 25% → 50% → 75% → 100% (final image). Some users see intermediate noisy previews — the GPU worker sends the current latent decoded to a low-resolution preview image at each progress checkpoint.

The Relax Path (~5-30 minutes)

For a Relax-mode user, steps 1-6 are identical. The difference is step 7: instead of waiting 5 seconds, the job sits in the Relax lane for 0-30 minutes[26]. The system drains Relax jobs only when GPU workers are idle — during off-peak hours or when Turbo/Fast demand dips. The user sees "Queued (Relax)" and a position indicator. The actual GPU inference time (step 9) is the same as Fast mode — Relax does not use fewer steps or lower quality.

The Error Paths

What happens when things go wrong? Three failure modes dominate:

The moderation block at step 5 is deliberately placed before the queue to avoid wasting GPU time on prompts that would be rejected anyway. The post-gen moderation at step 11 catches cases where an innocent-sounding prompt produces a policy-violating image (the model can sometimes generate unexpected content from ambiguous prompts).

V7 vs V8: The Migration Impact

The PyTorch migration[4] shrank the dominant phase by 5x. Let us compare:

Notice something interesting in the V8 column: the bottleneck shifted. When GPU inference drops to 4 seconds, the queue wait (5 seconds) becomes the largest component. This is a textbook example of Amdahl's Law — once you optimize the dominant component, the next-largest component becomes the new bottleneck. The next optimization frontier for Midjourney is queue dispatch latency, not GPU speed.

Worked Example: Throughput Under Load

Chapter 8: The Evolution — From 10 Engineers to 10,000 GPUs

Most startups scale by adding features. Midjourney scaled by adding zeros — zero marketing spend, zero VC after mid-2022, zero public papers. What they did add was compute: from a handful of GPUs in a rented cloud account to a fleet of 10,000 GPUs spread across Google Cloud[1]. This is the story of how an architecture evolves when your user count goes from 10 to 20 million in four years.

It started in August 2021, when David Holz — fresh from selling Leap Motion — gathered roughly 10 engineers[9]. They built a working prototype in one month. One month. No product-market fit research, no advisory board, no pitch deck. Just a Discord bot that turned text into images. The prototype used a standard diffusion model running on rented NVIDIA GPUs. At that scale, the "architecture" was basically one server with a queue.

Then came the open beta in July 2022 — and everything broke.

The first real architectural decision came in November 2022, with V4. Holz moved training to Google Cloud TPU v4 pods[2], using JAX — Google's framework optimized for TPU hardware. This was a bet: JAX had a smaller community than PyTorch, fewer tutorials, fewer engineers who knew it. But TPU v4 offered raw training speed that GPUs couldn't match at the time, and JAX extracted every flop from them.

Meanwhile, inference stayed on NVIDIA GPUs[3]. This split — TPU for training, GPU for inference — defined the architecture for three years. It worked beautifully until it didn't.

V5 (March 2023) brought "significantly different neural architectures." V6 (December 2023) was trained from scratch over 9 months[21] — a massive compute investment that only a profitable, VC-free company could justify without quarterly pressure. By now the team had grown to ~40 people[9], revenue was $200M[10], and the fleet had expanded to thousands of GPUs handling 2.5 million images per day[12].

Then came V8 in March 2026 — and it changed everything.

The V8 Rewrite: Burning the Ships

V8 was not an incremental improvement. It was a complete rewrite from JAX/TPU to PyTorch/GPU[4]. The entire training and inference stack — every custom kernel, every optimization trick, every workaround accumulated over three years of rapid iteration — was thrown away and rebuilt from scratch in PyTorch.

Why would a profitable company with a working system do this? Three reasons:

The result: V8 inference is 5x faster than V6, generates at native 2K resolution, and completes in under 10 seconds[17]. That speed gain — not a marginal 20% tweak but a 5x leap — is what justified the risk of rewriting everything.

Revenue Efficiency: The Smallest Big Tech Company

As of 2026, Midjourney has roughly 192 employees[9] and generates approximately $500M in annual revenue[10]. That's about $2.6M revenue per employee — and if you count only engineers (roughly half the team), it's closer to $5M per engineer[25]. For comparison, Google generates about $1.5M per employee. Meta, $1.7M. Midjourney is 3x more efficient than the most profitable tech companies on Earth.

How? No VC means no growth-at-all-costs pressure[8]. No free tier means every user pays. No marketing spend means Discord virality does the work. No papers and no blog means no team dedicated to external communications. Every person ships product.

What's Next: Beyond Images

Midjourney hired Ahmad Abbas from Apple Vision Pro to lead hardware efforts. They launched Video V1 in June 2025 — 5-second clips at 8x the GPU cost of images. They're exploring 3D generation and real-time interactive creation. Each new modality multiplies compute demand. The fleet that handles 2.5M images/day may need to handle 2.5M videos/day — at 8x the cost per job. That's a 20x compute scaling challenge.

Below, you can trace this evolution interactively. Drag the slider to move through time and watch the architecture transform at each breakpoint.

Chapter 9: When Things Break — Failure Modes & Reliability

Running 10,000 GPUs[5] is not like running 10. At 10 GPUs, failures are events — you notice them, you fix them, you move on. At 10,000 GPUs, failures are weather. They're constant, ambient, and you design around them the way a ship designer accounts for waves. You don't prevent them. You survive them.

Here's the math that changes your thinking. GPU hardware failure rates in large data centers run 1-3% at any given time. On a fleet of 10,000 GPUs, that means 100 to 300 GPUs are failing right now. Not "might fail someday." Failing right now, this second, as you read this sentence. Some have memory errors. Some have thermal throttling. Some have crashed drivers. Some just stopped responding. Every single day, the fleet regenerates — bad GPUs get pulled, repaired or replaced, and put back. The fleet is a living organism, not a machine.

Let's think about this concretely. A user hits "Generate" on a prompt. Their job enters the queue, gets assigned to GPU #4,721, and the diffusion process begins — 20, 30, maybe 50 denoising steps. On step 34, the GPU's memory controller throws an ECC error and the process crashes. What happens?

The Circuit Breaker Pattern

When a GPU worker fails, the orchestrator doesn't just retry on the same GPU. That would be madness — the GPU is probably still broken. Instead, it uses a circuit breaker: if a GPU fails N times within a window (say, 3 failures in 10 minutes), it's pulled from the active pool entirely. No more jobs get routed to it. A health-check daemon monitors pulled GPUs and reintroduces them only after they pass a diagnostic suite.

The failed job itself gets re-queued with high priority — it goes to the front of the line, not the back. The user sees a brief extra delay, maybe 5-10 seconds, but they get their image. They never know a GPU died.

The Cold Start Problem

When a fresh GPU comes online (or a repaired GPU returns to the pool), it can't serve jobs immediately. The diffusion model weights — we're talking 2-10 GB depending on the model version and resolution — need to be loaded from storage into GPU VRAM. This takes 30 seconds to 2 minutes depending on the model size and network speed. During that time, the GPU is consuming electricity and costing money but producing nothing.

The solution is warm pools: keep a subset of GPUs with models pre-loaded at all times, even if they're idle. When a job arrives, it goes to a warm GPU instantly. Cold GPUs are loaded in the background and added to the warm pool as capacity grows. The tradeoff is real: an idle warm GPU with an A100 costs $2-3/hour in cloud fees. Twenty idle warm GPUs cost $40-60/hour. That's the price of instant responsiveness.

Graceful Degradation: Relax Mode as a Shock Absorber

Midjourney's tier system — Fast, Relax, Turbo[18] — isn't just a pricing mechanism. It's an architecture-level resilience feature. When the GPU fleet is under strain (peak hours, partial outage, or a viral prompt trend that sends request volume spiking), the system degrades gracefully. Fast and Turbo users keep their priority, and Relax users simply wait longer. No one gets an error. No one gets rejected. The queue just stretches.

Think of it like a hospital triage system. Emergency patients (Fast/Turbo) get seen first. Walk-in patients (Relax) wait. During a disaster, walk-ins might wait hours — but they still get seen. Nobody is turned away.

The Discord Dependency — The Real Risk

GPU failures are manageable. The real availability risk for Midjourney for most of its history was Discord itself. Until the web app launched with V6.1 in August 2024[13], Discord was a single point of failure. If Discord went down — API outage, rate limiting, server issues — Midjourney went down. Completely. All 20 million users[11] locked out. GPUs sitting idle. Revenue dropping to zero.

This happened multiple times. Discord rate limits throttled bot commands. Discord CDN outages meant generated images couldn't be delivered. Discord auth issues meant users couldn't even start sessions. None of these were Midjourney bugs — they were dependency failures, and Midjourney had zero control over them.

Moderation Failure Modes

Content moderation has two failure modes, and they're in direct tension. False positives block legitimate prompts — a user trying to generate "surgery scene for medical textbook" gets rejected. This frustrates paying customers. False negatives let harmful content through — generating photorealistic deepfakes or explicit material. This creates PR disasters and legal exposure.

Midjourney runs both pre-generation text moderation and post-generation image moderation[20]. The pre-gen filter is fast (text classification, milliseconds) and catches obvious violations. The post-gen filter is slower (image classification, seconds) and catches outputs that look benign from the prompt but produce problematic images. If the post-gen filter triggers, the image is generated (GPU time already spent) but never delivered to the user. That's wasted compute — but the alternative (no post-gen filter) is worse.

What happens if moderation itself goes down? This is the nightmare scenario. Without moderation, the system must shut down entirely. You cannot serve unfiltered AI-generated images to millions of users — the legal, ethical, and reputational risk is catastrophic. Moderation is not a feature. It's load-bearing infrastructure.

Below, you can simulate what happens when various components fail. Click a component to "kill" it and watch the cascade.

Chapter 10: The Tradeoffs — Design Decisions & Alternatives

Every architectural decision is a tradeoff. You gain something; you lose something. The trick isn't finding the "right" answer — it's understanding what you're trading away, so you can make the trade consciously. Midjourney made five major architectural decisions that shaped everything. Each one had a clear alternative, a clear cost, and a clear payoff. Let's walk through all five.

Decision 1: Discord as Primary UI

In 2021, Midjourney didn't build an app. They didn't build a website. They built a Discord bot. Users typed /imagine in a chat channel, and images appeared. That's it. No login flow, no sign-up page, no app store review, no mobile development, no web hosting.

What they gained was staggering. Discord's server system meant that when one person generated an image, everyone in the channel saw it. Images were inherently social — you could watch what other people were creating, react to it, remix it. This turned every user into a marketing channel. The result: 20 million users[11] at near-zero customer acquisition cost, making Midjourney the largest Discord server in history.

What they lost was control. Discord sets the rate limits. Discord controls the UX. Discord can change its API, its pricing, or its terms of service at any time. And critically — Discord was a single point of failure. If Discord went down, Midjourney went down. For three years, a $300M+ revenue company depended entirely on another company's infrastructure for its user interface.

Decision 2: Google Cloud TPUs for Training

When V4 launched in November 2022, Midjourney moved training to Google Cloud TPU v4 pods[2] using the JAX framework. TPU v4 pods offered massive matrix-multiply throughput — better raw TFLOPS per dollar for training large diffusion models. JAX, Google's ML framework, was the natural fit for TPU hardware.

The gain: faster training for V4, V5, and V6. The ability to train V6 from scratch in 9 months[21] rather than 18 months on GPUs (rough estimate based on compute differences).

The loss: JAX lock-in. The JAX ecosystem is smaller than PyTorch's — fewer libraries, fewer Stack Overflow answers, fewer job candidates who know it. Every custom operator, every training trick, every optimization had to be built in JAX. And when they wanted to leave, they had to rewrite everything[4]. The JAX decision bought them three years of speed and cost them a massive rewrite.

Decision 3: Closed Source, No Papers, No Blog

Midjourney has published zero papers. Zero blog posts. Zero technical talks. They don't open-source their models, their training code, or their inference stack. In an industry where Stability AI, Google, Meta, and OpenAI all publish extensively, Midjourney is a black box.

What they gained: a competitive moat. Nobody can replicate their exact architecture, their training data pipeline[22], their model weights, or their inference optimizations. Competitors can study papers from other labs and build on them — but Midjourney's work stays proprietary.

What they lost: academic reputation, open-source community contributions, and a recruiting pipeline. Top ML researchers want to publish. They want their work cited. Midjourney can't offer that. But with $500M in revenue[10] and no VC dilution[8], they can offer something else: equity in a profitable company. The market validated this choice.

Decision 4: No Official API

As of 2026, Midjourney still has no official REST API for developers. Every image generation goes through either Discord or the web app. There's no curl midjourney.com/v1/generate. No API keys. No per-call billing.

What they gained: simplicity. Revenue comes through subscriptions — flat monthly fees of $10-120. No metering infrastructure, no usage-based billing, no API abuse mitigation (rate limiting, auth, key management). The billing system is Discord and Stripe. That's it.

What they lost: the developer ecosystem. Canva, Figma, Adobe, and hundreds of startups would pay for API access. Enterprise deals with SLAs and custom integrations are off the table. Third-party developers reverse-engineer the Discord API anyway, building unofficial wrappers — Midjourney gets zero revenue from this usage and no control over the experience.

Decision 5: Small Team, High Revenue per Employee

At ~192 people[9] generating $500M[10], Midjourney achieves about $2.6M revenue per employee. For engineers specifically, it's closer to $5M per engineer[25]. Compare: Stability AI had ~200 people at $100M revenue (peak). DALL-E is backed by thousands of OpenAI employees across many products.

What they gained: speed. Fewer people means fewer meetings, fewer approval chains, fewer Slack threads. David Holz can make architectural decisions in hours, not quarters. The V8 rewrite — a terrifying, company-bet decision — was made and executed without a committee.

What they lost: breadth. Video launched in June 2025, years after competitors. 3D is still nascent. They don't have enterprise sales, a developer relations team, or a research publications group. They do one thing — image generation — and they do it at world-class level. Everything else waits.

Below, you can compare each decision side-by-side. Toggle between the five major tradeoffs to see what Midjourney chose, what they could have chosen instead, and what each path costs.

Chapter 11: The Blueprint — Patterns You Can Steal

Everything we've studied in this lesson — the queue architecture, the GPU fleet management, the Discord platform strategy, the tiered pricing — these aren't unique to image generation. They're reusable architectural patterns that apply to any system where expensive async compute serves consumer users. If you're building a video encoding pipeline, a real-time 3D renderer, a scientific simulation service, or any GPU-heavy inference product, these patterns are yours to steal.

Let's extract six patterns. For each: what the pattern is, when to use it, and a concrete example beyond Midjourney.

Choosing the Right Pattern

Here's a quick decision matrix. When you face a new system design problem, ask these questions:

References

All sources cited throughout this lesson:

1. Google Cloud & Midjourney. “Midjourney Selects Google Cloud.” PR Newswire, March 2023. Link CONFIRMED
2. David Holz quote in Google Cloud PR: “training the latest versions of our algorithms on the v4 TPUs with JAX.” Link CONFIRMED
3. David Holz: inference on “huge clusters of GPUs” via Google Cloud. Same PR as [1]. CONFIRMED
4. Midjourney V8 rewrite from JAX/TPU to PyTorch/GPU (March 2026). Analysis LIKELY
5. David Holz: ~10,000 GPUs/servers. The Register, Interconnects CONFIRMED
6. 90% inference / 10% training cost split. Interconnects CONFIRMED
7. David Holz: ~5 petaops per image. The Register CONFIRMED
8. Self-funded, profitable since mid-2022. Contrary Research CONFIRMED
9. Team: 10 (2021) → 40 (2024) → ~192 (2026). Contrary, DemandSage CONFIRMED
10. Revenue: $200M (2023), $500M (2025). Contrary, DemandSage CONFIRMED
11. 20M Discord users, 1–2.5M DAU. DemandSage CONFIRMED
12. 2.5M images generated daily. Medium LIKELY
13. Web app launched with V6.1, August 2024. Wikipedia CONFIRMED
14. DiT architecture confirmed by xDiT fork. Midjourney GitHub CONFIRMED
15. Flash Attention & SageAttention. Midjourney GitHub forks CONFIRMED
16. xDiT multi-GPU parallelism. Midjourney GitHub CONFIRMED
17. V8: 5x faster, native 2K, <10s. Multiple sources. LIKELY
18. Fast/Relax/Turbo modes, $4/hr GPU time. Midjourney Docs CONFIRMED
19. Concurrency: 3–15/user, 10-job queue. Midjourney Docs CONFIRMED
20. Dual moderation: text pre-filter + image post-filter. Multiple sources. LIKELY
21. V6 trained from scratch over 9 months. Wikipedia CONFIRMED
22. Training data: LAION-5B + web scraping. Contrary LIKELY
23. 148+ TB on Discord CDN. Simon Willison CONFIRMED
24. 20–40 jobs/sec capacity. DemandSage LIKELY
25. $5M revenue/employee. Envzone CONFIRMED