The mistakes are not caused by carelessness. They are caused by the way JMeter is designed and the way people first meet it.

JMeter Does Not Argue

Many tools refuse to run when something is clearly wrong. JMeter does not. It will run a plan with no pauses, no checks, and a single fixed user name, and it will report numbers at the end. It leaves the judgement to you. That freedom is powerful for experts and dangerous for beginners.

The Tool Looks Friendly

JMeter has a graphical window with menus and a tree on the left. It feels like any other desktop program, so people treat it like one: they leave every window open, click start, and wait. But a load testing tool is not a spreadsheet. It creates heavy work on the machine that runs it, and the way you run it changes your results.

Success Is Hard to See

In ordinary programming, a mistake usually stops the program. In load testing, a mistake usually produces a perfectly normal-looking result that happens to be false. The report shows green numbers, and you believe them. The failure is invisible until someone compares the test with real life.

The Five Kinds of Mistakes

Eight small ideas appear again and again in the mistakes that follow. Learn them once and the rest becomes easier.

Why These Eight Matter

Almost every beginner mistake touches one of them. Wrong numbers of virtual users, missing timers, missing assertions, heavy listeners, and weak load generators account for most of the false results in the world of beginner tests.

A trustworthy test passes through five stages. Beginners tend to rush through the first and last, which is exactly where many mistakes live.

flowchart TD
    P["1. Plan: goal, scenario, numbers, environment"] --> B["2. Build: script, data, checks, timers"]
    B --> D["3. Debug: one user, read real responses"]
    D --> R["4. Run: command line mode, light listeners, healthy generator"]
    R --> A["5. Analyse: errors, percentiles, server metrics, repeat runs"]
    A --> C["6. Report and record: versions, settings, raw files"]
    C --> P
    P -.-> M1["Mistakes: no goal, unrealistic users"]
    B -.-> M2["Mistakes: fixed data, no assertions, no correlation"]
    D -.-> M3["Mistakes: skipping this stage entirely"]
    R -.-> M4["Mistakes: graphical mode, weak machine, heavy listeners"]
    A -.-> M5["Mistakes: only averages, ignoring errors, one run"]
    

Reading the Diagram

The solid arrows show the healthy flow: plan, build, debug, run, analyse, and record, then back to planning for the next round. The dotted lines point to the typical mistakes at each stage. Notice that debugging with one user is a stage of its own. Beginners often skip it, because it feels slow, and then waste hours finding problems in a large test that one user would have revealed in a minute.

Data Flow in One Sentence

Your data and settings flow into virtual users, the virtual users send requests, responses come back, assertions judge them, listeners record them, and you read the results. A mistake at any point in this chain damages everything after it.

The Mistakes in This Guide Follow the Same Order

• Planning mistakes come first, because they spoil everything that follows.
• Building mistakes come next, covering data, correlation, checks, and element placement.
• Running mistakes follow, covering modes, listeners, and load machines.
• Reading mistakes come last, covering how to interpret what you measured.

The most common mistake of all happens before JMeter is even opened.

What It Looks Like

A beginner is told, “Load test the website.” They record a few clicks, set a hundred users, press start, and produce a report. When someone asks, “So, is the system good enough?”, nobody knows how to answer, because nobody ever said what “good enough” meant.

Why It Is a Problem

Without a goal, there is no way to decide whether the result is a success or a failure. Numbers float in the air with no ruler to measure them against. A test that cannot fail also cannot prove anything.

The Fix: Write Down the Questions and the Limits

Agree on these numbers with the business owners before you start. Then every result has a clear verdict: it met the limit, or it did not.

A test is only as meaningful as the behaviour it imitates. Imitating the wrong behaviour gives the right answer to the wrong question.

What It Looks Like

Every virtual user logs in, searches for the same product, and logs out, over and over, as fast as possible. Real visitors browse in many ways: some only read, some search, some buy, some abandon baskets. They also arrive in waves, not in one flat line.

Typical Unrealistic Choices

• One journey for everyone. Real traffic is a mix of journeys, and the mix matters. Ninety percent browsing and ten percent buying stresses a system very differently from the reverse.
• No variety in data. Everyone uses the same account and the same search word.
• Too many users, too fast. A hundred users arriving in one second is a sudden burst that real life rarely produces.
• Only the happy path. Real users make mistakes, mistype searches, and click back.
• Ignoring what is cached on the visitor’s side. Real browsers keep copies of images and scripts, so returning visitors send fewer requests.

The Fix: Learn the Real Traffic

Look at real usage numbers if they exist: web analytics, server logs, past incidents, and business forecasts. Build a small table of journeys with the share of visitors who follow each one, and let your plan reflect that mix. If no data exists, ask the business for their best estimates and write them down as assumptions, so they can be corrected later.

These shares are invented for illustration. Your own system will have its own mix, and finding it is part of the work.

The graphical window is for building and debugging. Using it for the real run is one of the most expensive beginner habits.

What It Looks Like

The beginner builds the plan in the window, adds several listeners to see results, sets a few hundred users, and presses the green start button. The window updates constantly as results arrive. After a while, the window slows down, the numbers look odd, and sometimes JMeter freezes or runs out of memory.

Why It Is a Problem

• The window uses the machine’s power. Drawing tables and graphs for every result takes processor time and memory that should go to generating load.
• Listeners keep results in memory. Some store every sample, and memory fills up in long tests.
• Measurements become distorted. When the machine is busy with its own display, requests leave late, and the response times include that delay.

The Fix: Two Modes, Two Jobs

In non-graphical mode, JMeter runs without windows. You tell it which plan to run and which file to write results to, and afterwards you open the results file, or generate the built-in HTML report, to see what happened. The official guidance of the JMeter project is the same: build in the window, run from the command line.

Listeners are windows into the results. Some of them are very heavy, and beginners add them to everything.

What It Looks Like

The plan has a View Results Tree, a graph results listener, a table listener, and a few more, all attached at the top of the plan. They show wonderful details during small tests, and nobody remembers to remove them before the big one.

Why It Is a Problem

The View Results Tree stores the full request and response of every sample. For a thousand samples it is fine. For a million, it can exhaust memory and freeze the program. Graph listeners redraw constantly. Together, they can take more processor time than the load itself.

Cost of Common Listeners

The Fix

• Use the View Results Tree only while debugging, and disable or delete it afterwards.
• For real runs, write results to a file and analyse them afterwards.
• Generate the built-in HTML dashboard report from that file for graphs and percentiles.
• Keep at most one light listener in the plan for a quick live view.

Recording a browser session and playing it back is a good way to start. Playing it back unchanged is a good way to fail.

What It Looks Like

The beginner records a login and a purchase, then replays the script with many users. The first request works, but everything after the login fails with redirects to the login page, error messages, or forms that are rejected. The report shows a flood of errors, or worse, shows success codes on error pages.

Why It Happens

Websites hand out values that are valid for one visit only: session ids, security tokens, order numbers. The recording contains the values from the recorded visit. On replay, the server issues new values, but the script still sends the old ones, and the server refuses them.

The Fix: Correlation

Correlation means catching a changing value from a response and using it in the next request. In JMeter, small elements called extractors find the value in the response and store it in a variable. The later request uses the variable instead of the recorded value.

1. Record the same journey twice, and compare the two recordings. Values that differ are dynamic.
2. Find the earlier response in which each dynamic value first appeared.
3. Add an extractor to that request, to catch the value.
4. Replace the fixed value in later requests with the variable.
5. Run one user and confirm that every step really succeeds.

If every virtual user sends the same name, password, and search word, the server soon notices.

What It Looks Like

A single test account is typed straight into the login request. All two hundred virtual users log in with it at the same time. The search always uses the same word, and the order always uses the same product.

Why It Is a Problem

• Caching makes the system look too fast. Repeating the same search lets the server answer from memory.
• Sessions clash. Many applications limit or interfere with simultaneous logins on one account.
• Locks and queues form. Many users updating one record wait for each other, which is not what real traffic does.
• Duplicate data is rejected. A second registration with the same email address fails.

The Fix: Parameterize

Parameterization means replacing fixed values with placeholders that are filled in differently for each user. The most common tool is the CSV Data Set Config, which reads rows from a simple text file and hands one to each user. Prepare enough rows: for single-use data, the number of users multiplied by the number of iterations each will run, plus a safety margin.

By default, JMeter judges a request by its status code alone. That is a much weaker test than beginners imagine.

What It Looks Like

The report shows zero percent errors. Everyone celebrates. Later, someone opens the application and finds that search has been showing “Sorry, something went wrong” for the whole test. The server answered every request with a normal success code, and the friendly error page was delivered perfectly.

Why It Happens

JMeter marks a sample as failed when the status code signals a problem, for example a server error or a missing page. Many applications, however, return a success code together with an error message in the body. JMeter sees a success code and moves on. It has no idea that the words in the page mean “this failed”, unless you tell it what to look for.

The Fix: Add Meaningful Checks

An assertion is a check attached to a request. If the check fails, the sample is marked as failed and counted in the error percentage.

The Thread Group decides how many virtual users appear and how fast. Beginners often pick the numbers by guessing.

Quick Definitions

• Threads (users): the number of virtual users that run in parallel.
• Ramp-up period: the time over which all those users are started, one after another at an even pace.
• Loop count: how many times each user repeats the journey.

Typical Mistakes

• A ramp-up of one second for hundreds of users. All the users start almost together, which produces a sudden spike unlike real arrivals. The server may fail for reasons that have nothing to do with steady load.
• Counting users as if they were requests. A hundred users who pause and read send far fewer requests per second than a hundred users who never pause.
• Picking a round number for no reason. “A thousand sounds good” is a guess, not a requirement.
• Starting at the maximum. You never learn where the system begins to struggle.

The Fix: Work From Numbers You Can Defend

A useful rule of thumb comes from queueing theory, known as Little’s Law: the number of users inside the system equals the arrival rate multiplied by the time each stays. If you expect six new visitors per second and each stays about ten minutes, there are about thirty-six hundred users active at once. Plain arithmetic like this gives you a defensible thread count.

A sensible ramp-up is long enough that load grows smoothly. Many testers use several minutes for hundreds of users, then hold the load steady for the measuring period.

Real people pause. They read, scroll, and decide. A script that never pauses is a machine, not a crowd.

What It Looks Like

A virtual user finishes one request and sends the next in the same instant, thousands of times a minute. Fifty such users can produce the traffic of several thousand human visitors. The system seems to collapse at a tiny user count, and the beginner concludes that the application is terribly slow.

Why It Is a Problem

Think time is the pause between actions. Without it, each virtual user is far more aggressive than any human. The test measures something real life never produces, so the verdict is unfair in one direction or the other: either the system fails unrealistically, or it passes because the test is too gentle in some other way.

The Fix: Use Timers

A timer adds a delay before a request. A constant timer pauses for a fixed time; a random timer varies the pause around an average, which looks more human and avoids all users moving in step.

Two settings control how long a test lasts, and they interact in a way that catches nearly everyone once.

What It Looks Like

The tester wants a thirty-minute test. They tick the scheduler option, enter a duration of thirty minutes, and leave the loop count at one. After two minutes the test ends. Each user finished a single pass through the journey, and with the loop count at one, there was nothing more for them to do. The duration never had a chance to apply.

How the Two Settings Work Together

The Fix

If you want the test to run for a fixed time, set the loop count to infinite and set the duration. If you want a fixed number of journeys, set the loop count and leave the duration alone. Mixing them is allowed, but make sure you know which one will end the test first.

The first minutes of a test are not typical. Mixing them into your final numbers blurs the picture.

Why the Start Is Different

When a system has just started, or just received its first traffic, it is cold. Caches are empty, database connections are not yet open, and compiled code has not yet been optimised by the runtime. Early requests are slower. During the ramp-up, the load is also changing, so numbers from that period describe a moving target rather than a steady state.

The Mistake

The tester takes the average of the whole test, including the cold start and the ramp. The average is dragged by early slowness, and the result does not describe how the system behaves at the planned steady load.

The Fix: Define Phases

When you read the results, select the steady period only. Many reporting tools, including the HTML dashboard, let you filter by time range, or you can trim the results file before analysis.

An average can hide the very people who are having the worst time.

A Small Example

Suppose ten customers wait for a response. Nine of them wait one second. One waits eleven seconds. The average is two seconds, which sounds fine. But one customer in ten had a terrible experience, and the average never told you.

What to Read Instead

The Fix

State your targets in percentiles, such as “ninety-five percent of searches within two seconds,” and report against them. Keep the average as a secondary figure. If the average and the median are far apart, a slow tail is pulling the average upward, and that is where to look.

The machine running JMeter can itself become the slowest part of the test, and then you are measuring your machine, not the application.

What It Looks Like

Response times climb steadily as users are added, yet the application server shows low processor use and plenty of spare capacity. The slowness is real in the report but not real in the application. The load generator is out of processor power, memory, or network capacity, so requests leave late and responses are processed late.

sequenceDiagram
    participant U as Virtual User
    participant G as Load Generator (overloaded)
    participant S as Application Server
    U->>G: Prepare request
    Note over G: Busy with too many threads and a heavy window
    G->>S: Send request (late)
    S-->>G: Reply quickly
    Note over G: Busy again, reads reply late
    G-->>U: Record time (inflated)
    Note over U,S: Report shows slow application, but the delay was in the generator
    

Reading the Diagram

The server replies quickly, but the generator is too busy to send the request on time and too busy to read the reply on time. The recorded time includes both delays, so the report blames the server for a problem that belongs to the testing machine.

Signs and Fixes

Watch the generator with the same care as the server. If its processor stays above about eighty percent, treat the measurements with suspicion.

Real browsers remember things. A virtual user that remembers nothing behaves in ways no browser would.

Cookies

A cookie is a small note a website asks the browser to keep, often holding a session identity. Without an HTTP Cookie Manager, JMeter does not keep these notes, so every request looks like a brand-new visitor. Logins appear to fail, baskets appear empty, and the beginner blames the application.

Caching

Real browsers keep copies of images, styles, and scripts and do not download them again on every page. Without an HTTP Cache Manager, every virtual user downloads everything every time, which can be much heavier than real traffic. The opposite mistake also exists: enabling caching but forgetting to clear it each iteration, so repeat visitors seem to be everyone.

Headers and Defaults

Browsers send headers describing what they accept and who they are. Interfaces often need specific headers, such as a content type for JSON. An HTTP Header Manager sets them once for many requests. HTTP Request Defaults holds the server name, port, and protocol once, so you can change environments in one place.

A raw recording contains everything the browser touched, including things you should never load test.

What a Raw Recording Contains

• Static files: hundreds of images, styles, and scripts, usually served by a cache or delivery network.
• Third-party calls: analytics, advertising, fonts, and social widgets that belong to other companies.
• Browser noise: background update checks and extension traffic.
• Hundreds of unnamed samples: impossible to read in a report.

Why It Is a Problem

Hitting someone else’s analytics service with thousands of requests is rude and may breach their terms. Loading every image from your own server may be unrealistic if real users get them from a delivery network. A script of three hundred requests is also hard to maintain.

The Fix: Clean and Filter

1. Use include and exclude patterns while recording, to skip static files and outside hosts.
2. Delete anything that does not belong to the system under test.
3. Group related requests into logical steps, such as “Login” or “Search”.
4. Decide deliberately whether to fetch embedded resources, and if so, restrict them to your own host.

A load test looks very much like an attack. Pointing it at the wrong target can cause real harm and real legal trouble.

What It Looks Like

A beginner practises on a public website they do not own, or runs a heavy test against a shared production system during business hours. In the first case the traffic may break the law or the website’s terms. In the second it may slow down real customers, or take them offline.

The Rules to Follow

• Only test systems you own or have written permission to test. Practising on public sites is not acceptable.
• Warn the people who run the system. Operations, security, and hosting teams should know the dates and sources of test traffic.
• Prefer a dedicated test environment. If you must test production, agree on limits, times, and a way to stop quickly.
• Keep third parties out. Payment gateways, email services, and partner interfaces should be replaced with stand-ins or sandboxes.
• Practise on safe targets. Use a small application of your own, or a demo site that explicitly welcomes testing.

A report full of rows called “HTTP Request” is a puzzle that only the person who built it can solve, and sometimes not even them.

What It Looks Like

Every sampler keeps its default name. The summary table shows twelve rows all labelled “HTTP Request”. Worse, JMeter groups samples by label, so different requests with the same name are merged into one row. The slow login and the fast home page average together, and the slow one disappears from view.

Why It Is a Problem

• Merged rows hide problems. A slow step blends with fast ones of the same name.
• Reports are unreadable. Nobody else can tell which row matters.
• Failures are hard to trace. An error in “HTTP Request 7” gives no clue where to look.

The Fix: A Naming Convention

Use a number prefix so rows sort in the order of the journey, use words that a business person would understand, and keep the same names across test runs so results can be compared month after month.

In JMeter, where you put an element decides what it affects. The same element in two places can behave very differently.

The Scope Rule

Most helper elements, such as timers, assertions, extractors, and managers, apply to everything at their own level and below. Place a timer directly under the Thread Group and it affects every sampler in the group. Place it under one sampler, and it affects only that sampler.

flowchart TD
    TP["Test Plan"] --> TG["Thread Group"]
    TG --> CM["Cookie Manager: applies to all below"]
    TG --> T1["Timer here: delays EVERY sampler"]
    TG --> S1["Sampler: Home Page"]
    TG --> S2["Sampler: Login"]
    S2 --> A2["Assertion here: checks ONLY Login"]
    S2 --> X2["Extractor here: reads ONLY Login reply"]
    TG --> S3["Sampler: Search"]
    TG --> L["Listener here: records ALL samples"]
    

Reading the Diagram

The cookie manager, timer, and listener sit directly under the Thread Group, so they reach every sampler. The assertion and extractor sit under the Login sampler, so they touch only that request. The same building blocks produce very different behaviour depending on placement.

Common Slips

• A strict assertion placed at group level, failing every step that does not contain the same phrase.
• An extractor at group level that tries to read every response, wasting time and capturing the wrong value.
• A timer meant for one slow step applied to all steps, making the whole test slower than planned.
• Elements placed in a different Thread Group from the requests they were meant to serve.

When behaviour surprises you, check the tree: what sits above the request, and what sits beside it?

JMeter has two kinds of named values, and they live in different places.

The Difference

What Goes Wrong

• The beginner expects a command-line setting to appear as a variable, but it arrives as a property, and the placeholder stays empty or shows the default.
• A value that should belong to one user is written to a property, so all users see each other’s data.
• A property is changed halfway through and every user is suddenly affected.

The Fix

Remember a simple rule: values that belong to one user are variables, and settings for the whole run are properties. To read a property inside the plan, use the property-reading function, ideally with a default, so the plan still works in the window when no setting is given. Naming properties clearly, such as a prefix for your project, avoids clashes with JMeter’s own settings.

Thread Groups are a tool for modelling different kinds of users. Beginners often use them as folders, or forget they all start together.

What It Looks Like

The plan has ten Thread Groups, one per page, each with its own user count. When the test runs, all ten groups start at the same time, so the load is a strange mixture that no real user would produce. Or the opposite: one huge Thread Group holds three unrelated journeys in a row, so every user does everything.

Better Organisation

• One Thread Group per type of user. For example, browsers, searchers, and buyers, each with its own user count and pace.
• Use controllers for steps. Folders inside a group, such as Simple Controllers and Transaction Controllers, organise steps without changing how users behave.
• Use a throughput or random controller for mixed journeys. These let one user pick a journey by percentage.
• Decide start order deliberately. If groups must run one after another, say so in the plan settings; otherwise they begin together.

JMeter runs inside a Java virtual machine, and that machine has a memory limit set at start-up. The default is modest.

What It Looks Like

A long or large test slows down, pauses for several seconds at a time, and finally stops with an out-of-memory error. The beginner assumes the application caused it, when in fact JMeter ran out of room to hold its own results.

Why It Happens

The JVM reserves a fixed amount of memory, called the heap. When it fills, a clean-up process called garbage collection runs. Frequent, long clean-ups freeze the generator for moments, and these freezes show up as long response times. In the worst case there is no room left at all.

The Fix

• Raise the heap size in a sensible, measured way. A larger heap lets JMeter hold more, but a very large heap can cause long clean-up pauses of its own.
• Do not set the heap larger than the physical memory available, or the machine begins swapping to disk, which is slower still.
• Reduce what JMeter has to remember: remove heavy listeners, avoid storing response bodies, and save only the fields you need.
• Use several smaller generators instead of one enormous one when load is very large.

The exact setting names and recommended sizes depend on the JMeter version and Java version, so check the documentation that matches your installation before changing them.

A test is a statement about a particular system. If the test system is very different from the real one, the statement does not carry over.

Typical Differences

The Fix

• Make the test environment as similar to production as you can afford, and write down every known difference.
• Load the database with production-like volumes of invented data.
• Reserve the environment, so nobody else uses it during the run.
• Be careful about scaling results. Doubling a small environment does not always double its capacity.

The numbers on the screen disappear when the program closes. The raw results file is the real evidence.

What It Looks Like

A test finishes, the tester takes a screenshot of a summary, closes JMeter, and writes a short email. A week later someone asks about the ninety-ninth percentile for the checkout step. The data is gone, and the test would need to be run again.

The Fix: Always Write a Results File

In command-line mode, give JMeter a results file to write. This file, often called a JTL file, holds one line for every sample, with the time, label, response time, status, and more. From it you can build the HTML report, draw your own graphs, and answer questions you did not think of at the time.

Choose the fields to save thoughtfully. Storing full response bodies is useful for debugging but costly for large tests, so save them only when needed, or only for failed samples.

A test plan is code. Treating it like a casual document leads to confusion about which version produced which result.

What It Looks Like

The folder holds files named “plan_final”, “plan_final2”, and “plan_FINAL_really”. Nobody knows which one was run last Thursday, or why the results changed after that run.

Why It Matters

A load test only means something if you can say exactly what was tested and how. If a result gets better or worse, you need to know whether the application changed, or the test did.

The Fix: Use Version Control

• Store plans, data files, and scripts in a version control system such as Git.
• Write a short note with each change explaining what and why.
• Tag the exact version used for each important run and record the tag in the report.
• Keep the plan in a readable form, and avoid mixing unrelated changes in one save.
• Keep sensitive values out of the repository, using properties or secret stores instead.

Plan files are saved in a format that is text-based, so differences between versions can be reviewed. Still, large edits made in the graphical window can reorder many lines at once, so keep commits small and review them before saving.

When one machine cannot create enough load, JMeter can use several. This adds power and new ways to go wrong.

How It Works in Brief

One machine, the controller, tells several worker machines to run the same plan. Each worker creates its own share of the load. The controller collects the results.

Common Mistakes

The Fix

• Divide the user count by the number of workers, and check the arithmetic.
• Split data files so that each worker gets its own part.
• Keep every machine on the same versions.
• Monitor each worker’s processor and memory as you would a single generator.
• Make sure the controller is not drawing heavy listeners.

Distributed testing also needs security care: the communication between machines should happen on a private network, and the ports should not be open to the whole internet.

Timers and loops each work well alone. Together, with careless placement, they produce surprising results.

Surprises Worth Knowing

• A timer inside a loop runs for every pass of the loop, not once. A ten-second timer in a loop of twenty adds more than three minutes.
• A timer with no sampler beside it does nothing. Timers act on the samplers in their scope, so an isolated timer is easy to forget and has no effect.
• A timer attached to a controller affects all samplers inside that controller. A pause meant for one step spreads to many.
• Thread Group loops plus a controller loop multiply. Ten loops of the group and five loops of a controller inside means fifty passes through that part.
• Loops without pauses are tight and aggressive. A while or loop controller that retries instantly can hammer the server.

The Fix

Check your arithmetic by running a short test with one user and counting the requests in the results file. The count should match what you calculated.

The error column is one of the most watched numbers in a report, and one of the most misread.

Three Ways to Misread It

• Zero means perfect. As seen earlier, without assertions an error page counts as a success. Zero errors can mean zero checking.
• A small number means harmless. One percent sounds tiny, but at ten thousand requests a minute it is a hundred failures every minute, each one a real person having a bad time. Errors may also cluster on a single important step, such as payment, while the overall figure looks small.
• A big number means the application is broken. Often the cause is the test: expired data, a wrong environment, a missing correlation, or an overloaded generator.

Reading Errors Properly

JMeter tells you what the visitor experienced. It cannot tell you why. For the why, you need to look inside the system.

What It Looks Like

The report says search is slow at five hundred users. The tester writes “search is slow” and hands the report over. The developers ask, “Why?” and nobody can answer. Was the processor busy? Was the database waiting on a lock? Did memory run out? The test alone has no way to say.

What to Watch During a Test

The Fix

Arrange monitoring before the test starts, on every important machine, and keep the clocks of all machines in step so graphs can be laid side by side. Mark the start and end times of each phase. After the test, line up the response-time graph with the server graphs. When response times rise exactly when the database processor reaches its limit, you have a lead worth chasing.

One run is one sample. Systems vary from run to run, and a single result may be unusually good or unusually bad.

Why Runs Differ

• Background jobs, such as backups or scheduled tasks, may run during one test and not another.
• Other users or systems share the network or hosts.
• Caches are warm in one run and cold in another.
• Cloud machines share hardware with neighbours whose load changes.

The Fix: Repeat and Compare

1. Run the same test at least three times under the same conditions.
2. Compare the key figures: percentiles, throughput, and errors.
3. If they agree closely, you can trust the result. If they vary a lot, find out why before reporting.
4. Change only one thing between runs when comparing versions of the application.

If you had run only the first test, you would have called the system healthy. The third run shows that something sometimes goes wrong, and that is exactly the kind of thing a test should find.

When a script misbehaves, many beginners add more users, hoping a bigger test will explain itself. It will not.

The Right Way to Debug

1. Run one user, one loop. Everything becomes easy to follow.
2. Turn on the View Results Tree. Read the request, the response, and the headers of each step. Compare them with what a browser sends.
3. Check variables. A Debug Sampler can show every variable and its value at that moment, which reveals extractors that found nothing.
4. Read the JMeter log. Warnings about missing files, bad variables, or script errors appear there.
5. Fix one problem at a time. Run again after each change.
6. Only then raise the load, step by step: five users, then fifty, and so on.

Common Debugging Slips

• Looking only at the status column and never opening the response.
• Changing several things at once, so nobody knows which change fixed the problem.
• Leaving debugging elements switched on during the real run.
• Ignoring the log file entirely.

JMeter lets you write small pieces of code inside the plan. This is powerful, and easy to overuse.

The Slips

• Using the old BeanShell elements. They are slower and are generally discouraged. The JSR223 elements with a compiled-friendly language such as Groovy are the usual recommendation.
• Not enabling script caching. Without it, the script may be recompiled for every execution, which wastes time and memory in a large test.
• Heavy work in scripts. Reading big files, or slow outside calls, inside a script holds the thread and distorts timing.
• Reaching for code when a built-in element exists. A built-in extractor or function is clearer, safer, and easier for the next person.
• Unreadable, uncommented code. No one else can maintain it.
• Sharing state carelessly. Writing to a shared object from many threads without protection creates unpredictable results.

A Sensible Rule

Exact option names and recommendations vary slightly between versions, so check the documentation for yours.

Test plans and result files often contain secrets, and they get copied around freely.

Where Secrets Leak

The Fix

• Use dedicated test accounts with the least permission needed, never personal or administrator accounts.
• Pass secrets in at run time through properties or a secret store, not inside the plan.
• Use invented data. Where production data must be used, mask it first.
• Do not save sensitive headers or bodies in results unless needed, and protect result files like any other confidential file.
• Clean up test accounts and test data after the run.
• Follow the rules of your organisation about where test traffic may come from.

These short stories are invented for illustration. They show how the mistakes appear in practice, and how they were repaired.

Avoiding these mistakes takes effort. It is fair to look at what that effort costs and what it buys.

Where to Spend Effort First

If time is short, protect the basics: a clear goal, assertions, realistic data, command-line runs, a healthy generator, and saved raw results. These few habits prevent most false conclusions. Polish, such as elegant naming and automation, can follow.

A pattern is a habit that works. An anti-pattern is a habit that looks sensible and quietly causes harm. Here are four anti-patterns, followed by the patterns that replace them.

Patterns to Adopt

If you remember only one chapter, make it this one. It condenses everything so far into habits.

When a result looks wrong, do not guess. Follow a fixed order of questions, from the cheapest to the most expensive to check.

flowchart TD
    Q["Result looks wrong"] --> A{"Do saved responses show real success?"}
    A -- No --> A1["Fix script: correlation, data, headers, assertions"]
    A -- Yes --> B{"Was the generator healthy during the run?"}
    B -- No --> B1["Use command line mode, lighter listeners, more memory or more machines"]
    B -- Yes --> C{"Was the load realistic: users, think time, mix?"}
    C -- No --> C1["Rework scenario and numbers with real traffic data"]
    C -- Yes --> D{"Was the environment comparable and quiet?"}
    D -- No --> D1["Fix environment or record the known differences"]
    D -- Yes --> E{"Do repeat runs agree?"}
    E -- No --> E1["Look for background jobs, shared hosts, cold caches"]
    E -- Yes --> F["Look at server metrics: likely a real bottleneck"]
    

Reading the Diagram

Each diamond is a question, and each “No” answer sends you to a repair. Only when every earlier question has a “Yes” do you conclude that the application itself has a real bottleneck. This order matters because mistakes in the test are far more common than faults in the application during a beginner’s first runs, and they are cheaper to rule out.

Why This Order

• Responses first. If the replies are errors, nothing else matters.
• Generator second. A weak generator contaminates every number.
• Load realism third. Wrong load gives a right answer to the wrong question.
• Environment fourth. A mismatched system limits what conclusions you can draw.
• Repeatability fifth. Unstable results cannot support a verdict.
• Server metrics last. Now the evidence points at the application, and server graphs tell you where.

Use this table as a first guess. It lists what you see, what often causes it, and where to check first.

These are starting hypotheses, not verdicts. Always confirm by reading the actual responses and logs before changing the plan.

Several words in performance testing sound alike and mean different things. Mixing them up leads to confused reports.

This invented case follows one test from its first, flawed version to a trustworthy one.

The First Version

A tester was asked to check a ticket-booking site before a concert sale. Their first plan recorded a login, a search, and a booking. It had five hundred users, a one-second ramp-up, no timers, no assertions, one test account, a View Results Tree and two graph listeners, and it ran in the graphical window on a laptop. The report showed average times of eight seconds and thirty percent errors. The conclusion, “the site cannot handle the sale,” caused a panic.

The Repairs, One at a Time

The Lesson

The original result blamed the site for the mistakes of the test. The final result found one real, fixable limit. Same site, very different story.

You do not have to learn everything at once. This order builds skill steadily and avoids early frustration.

One page that lists each mistake with its repair, for quick checking before a run.

Plain meanings for the words used in this guide.

Print it, copy it, or keep it beside your screen. Each line is a mistake you can now avoid.

Before You Build

• The goal, load, and pass limits are written down and agreed.
• You have permission, a test window, and a way to stop quickly.
• The environment is reserved and its differences from production are noted.

Before You Run

• One-user debug run succeeded and responses were read.
• Dynamic values are correlated and extractors have visible defaults.
• Data files contain enough unique rows and no real personal data.
• Assertions exist on every important step.
• Timers, cookie, cache, and header managers are in the right places.
• Names are meaningful and elements are placed deliberately.
• Heavy listeners are disabled, and results go to a file.
• Loop count and duration agree with the plan.
• Memory setting and generator health are checked.
• Monitoring is running on the system and the generators.

After the Run

• Only the steady period is analysed.
• Percentiles and per-step errors are reported.
• Runs are repeated and compared.
• Raw results, plan version, and settings are archived together.
• Test data and accounts are cleaned up.

Requests travel through cables, routers, and name lookups. A test that ignores this path can look either far too fast or far too slow.

Distance Changes the Numbers

If the load generator sits in the same room as the server, the network delay is almost nothing. Real visitors are spread across cities and countries, and each trip takes longer. A test run from a neighbouring machine therefore produces times that no real visitor will ever see. Run generators from places that resemble where your visitors are, or at least record the difference and keep it in mind when reading results.

Connections Are Reused, or Not

Opening a secure connection costs time, because the two sides must agree on how to talk. Browsers reuse connections for many requests. If a test opens a new connection for every request, it spends much of its effort on handshakes, which is heavier than real traffic. If it reuses connections too aggressively, it may hide a weakness in how the server handles many new visitors. Know which behaviour your test has, and choose it on purpose.

Name Lookups and Limits on the Machine

• Name lookups. Turning a website name into an address takes a moment. A slow or overloaded lookup service can add delay that has nothing to do with the application.
• Port limits. Each open connection uses a numbered door on the generator. A machine has a limited number of doors, and a heavy test can run out, causing connection errors.
• Open file limits. Operating systems restrict how many connections one program may hold. Beginners meet this limit suddenly at high user counts.
• Firewalls and balancers. Security devices may slow or block traffic that looks like an attack, and may need to be told that the test is expected.

A single result floats in the air. A result placed beside earlier results tells a story.

What It Looks Like

Each test is treated as a fresh event. Someone reports “ninety-fifth percentile of two seconds,” and nobody knows whether that is better or worse than last month. When a new version of the application ships, there is nothing to compare against, so a slow regression slips through unnoticed.

Why It Is a Problem

• Gradual slowdowns, a little each release, are invisible one run at a time.
• Without a reference, nobody can say whether a change helped.
• Arguments about whether the system “feels slower” cannot be settled with facts.

The Fix: Keep a Baseline and a History

1. Pick one trusted run, under well-known conditions, as the baseline.
2. Run the same plan, with the same data and settings, for every new version.
3. Record key figures in a simple table: percentiles per step, throughput, error rate, and server load.
4. Flag any step that moves by more than an agreed margin, such as ten percent.
5. Refresh the baseline only on purpose, and write down why.

No single release looked alarming until the table was drawn. Trends are visible only to those who keep a history, and many teams automate this by running the same plan on every build and storing the results in one place.

Beginner mistakes in JMeter are predictable, and so are their cures.

The tool will run whatever you give it, so the quality of your result rests on the quality of your plan, script, run, and reading. Mistakes cluster in four places: unclear goals and unrealistic scenarios, scripts without correlation, data, or checks, runs that distort their own measurements, and readings that trust a single comfortable number. Each can be prevented with a small, repeatable habit.