MENTAL MODEL · M1009

Latency vs. Throughput

TechnicalHigh supportSoftware Engineering

Included

Version 1.0.0 · Updated 2026-07-30

CORE DEFINITION

Latency is the response time of a single request (how fast it is); Throughput is the total amount processed per unit time (how much). Low latency does not equal high throughput (e.g., sports car transporting goods vs. train transporting goods).

SCAFFOLDING EFFECT

psychology

Reduce cognitive load

Clarify optimization goals. Is your goal 'instant message replies' (low latency) or 'complete 10 proposals in a day' (high throughput)? These two are often contradictory. For high throughput (deep work), you must sacrifice latency (not replying to WeChat); for low latency (immediate response), you inevitably sacrifice throughput.

anchor

Anchor fast decisions

Latency measures how quickly a single request returns, throughput measures the amount processed per unit time; they often constrain each other—batching to increase throughput often raises latency. Understanding the trade-off helps set reasonable goals.

MINIMUM ACTION

In progress 0/4

Practice this model in one real situation:

Clarify the priority of latency vs. throughput for the business
use Little's law to relate the two (in-flight = throughput × latency)
stress test to observe inflection points
choose batch or streaming processing based on the scenario.

Check to track your progress (stored locally)

Learning progress0%

account_treeGenealogyexpand_more

menu_bookReferencesexpand_more

Source support: Explicit

link
baike.baidu.comhttps://baike.baidu.com/item/%E7%BD%91%E7%BB%9C%E5%90%9E%E5%90%90%E9%87%8F/646450ZH · Explicit
verified

RELATED MODELS