A local large language model (LLM) needs two things from a server. It needs enough RAM to hold the model, and enough memory channels to read it fast. Many guides stop at the first.

That gap is costly. In Lenovo's tests on EPYC, two memory modules reached just 28% of the bandwidth that eight modules did. So a server with plenty of RAM can still run a model at a crawl.

This guide covers server RAM for local LLM use. It shows how to size RAM for a model and how memory channels set LLM inference speed. It also compares DDR4 ECC RDIMM builds on AMD EPYC and Xeon servers. The numbers come from vendor data sheets, model cards, and owner test reports, all checked in October 2026. The speed ceilings are our own math.

Plan on about 0.6 GB of RAM per billion parameters at 4-bit. Then add room for context and the system. A 70B model needs 64 GB at the least and 128 GB to run comfortably. After that, put one module in every memory channel. The channels set your speed.

How much RAM to run an LLM locally

A model is a big set of numbers called weights. Its file size is the weight count times the bytes each weight takes. Quantization shrinks each weight to fewer bits. A 16-bit weight takes 2 bytes. An 8-bit weight takes 1. A 4-bit weight takes about half a byte. With the extras added, that is roughly 0.6 GB per billion weights.

Take Llama 3.1 70B. Its GGUF files run 74.98 GB at 8-bit and 42.52 GB at 4-bit. The 4-bit file is the usual pick. It trims the size by about 70% against 16-bit. Here is the same math for other sizes.

Dense models use all their weights for every token. MoE means mixture of experts, and we cover it below.

Context takes memory too. The model keeps a running record of your chat, called the KV cache. It grows with each token. Llama 3.1 70B has 80 layers and 8 key-value heads of 128 values each. That works out to about 0.33 MB per token at 16-bit. A chat of 32,768 tokens adds about 10.7 GB. Leave a few more GB for the system and your software. That is why the last column sits well above the file sizes.

Memory channels and LLM inference speed

Size decides whether a model runs. Speed is a separate question. A model writes its reply one token at a time. For each token, the CPU reads the model's active weights from RAM. Databricks explains that the first token is typically compute-bound, while the rest of the reply is memory-bound. AMD also calls prompt reading compute-bound.

So the top speed is easy to estimate. Divide memory bandwidth by the data read per token. Bandwidth is channels times transfer rate times 8 bytes. Eight channels of DDR4-3200 give 204.8 GB/s.

Now run the 70B model at 4-bit, which is 42.5 GB. A desktop caps out near 1.2 tokens per second (51.2 divided by 42.5). A 6-channel Xeon Gold 6230 caps at 3.3. An 8-channel EPYC caps at 4.8. Real speed lands below these ceilings, so treat them as the best case.

Owner tests show the same pattern. One owner ran a 296 GB DeepSeek model on three DDR4 machines. The results are in a llama.cpp thread. Four channels of DDR4-2133 gave 2.1 tokens per second. Six channels of DDR4-2666 gave 3.22. Eight channels of DDR4-2666 on an EPYC 7402P gave about 4.2. Speed rose with the channel count. These are one person's results, not lab data.

Fill every channel. In Lenovo's balanced-memory tests, eight modules, one per channel, hit 100% of the top bandwidth. Four modules hit 54%. Two hit 28%. So eight 64GB modules beat two 256GB modules for speed. Both reach 512 GB. If your board has empty channels, fill those before you add size anywhere else.

A second module in a channel adds room, not speed. On EPYC 7002 and 7003, two modules per channel also cut the speed from 3200 to 2933 MT/s. That costs about 8% of the bandwidth. Sixteen modules give you more room. Eight give you more speed.

To see what your server holds now, run sudo dmidecode -t memory on Linux. It lists each slot and the module in it. Your board manual shows which slots share a channel. Fill the first slot in each channel before you use any second slot.

Mixture-of-experts models change the math

Some models read only part of their weights for each token. A mixture-of-experts (MoE) model splits its weights into groups called experts. It uses a few experts per token. OpenAI's gpt-oss-120b holds 117B parameters and uses 5.1B for each token. DeepSeek-R1 holds 671B and uses 37B. Qwen3-235B-A22B uses 22B of its 235B.

The whole model still has to fit in RAM. But speed follows the active part. At 4-bit, gpt-oss-120b reads a few GB per token, while the dense 70B reads 42.5 GB. So big MoE models are a good match for a large DDR4 server. That also explains how the 296 GB DeepSeek model reached 4 tokens per second on DDR4.

A GPU can help too. You can keep the always-used layers on the card and leave the experts in RAM, as this llama.cpp guide shows.

AMD EPYC processors or Xeon for a DDR4 build

For a DDR4 build, AMD EPYC processors in the 7002 and 7003 families are a common pick. AMD's data sheet lists up to 8 DDR4 channels and up to 128 PCIe 4.0 lanes. The lanes leave room for GPUs later.

Check the chiplet count on low-core models. ServeTheHome measured about half the memory bandwidth on the EPYC 7232P, 7252, 7272, and 7282. Those chips use only two chiplets. Models built for all eight channels, like the 7262, avoid the problem.

More cores won't fix a memory limit. One owner in that llama.cpp thread found about three cores per channel worked best. Cores do speed up prompt reading, since that step is compute-bound.

A second CPU doubles the channels on paper, but gains vary. In an early-2025 test, one EPYC 7K62 ran a 462 GB DeepSeek model at 4.2 tokens per second. Two CPUs managed 2.9. Start with one socket.

Already own an Intel server? Dell's R640 and R740 have six channels per CPU. With 2nd Gen chips, each CPU tops out near 140.8 GB/s. The R750 has eight channels. Our Dell PowerEdge memory guide shows how to add modules to each.

64GB DDR4 server memory: build sizes that work

Pick the module size from the channel count. These builds put one module in each of eight channels.

A six-channel Xeon takes six modules, so six 64GB modules give 384 GB.

Eight 64GB modules reach 512 GB without a second module in any channel. On EPYC, that keeps the full 3200 MT/s. A dual-rank RDIMM rated 3200 MT/s is a common fit for these boards. CPU Republic lists one as 64GB DDR4 server memory. For other sizes and speeds, see our DDR4 ECC RDIMM range.

Match your modules. Lenovo's rules call for the same size and rank in each channel pair. Don't mix RDIMM with LRDIMM, since Dell's guides call that unsupported. Speed has a cap too. A 1st Gen Xeon runs DDR4-2666 at most, so faster modules gain nothing there.

Servers want registered ECC memory anyway. Dell's R740 spec sheet lists "registered ECC DDR4 DIMMs only." ECC fixes single-bit errors, which helps on jobs that run for days. Our ECC guide has more.

Plan the whole build before you start. TrendForce expects server DRAM to stay undersupplied in the fourth quarter of 2026.

Server RAM for local LLM FAQ

How much RAM do I need to run an LLM locally?

Plan on about 0.6 GB per billion parameters at 4-bit, plus room for context and the system. That means 16 GB for an 8B model, 64 GB for a 70B model, and 128 GB for gpt-oss-120b.

Is 64GB of RAM enough for an LLM?

For models up to about 70B at 4-bit, yes, if your chats stay short. The 70B file is 42.5 GB, and a long chat adds 10 GB or more. For 120B models and up, plan on 128 GB or more.

Does RAM size or RAM speed matter more?

Size decides whether the model runs. Channels and speed decide how fast. Channels count most. Eight channels of DDR4-2666 (170.6 GB/s) beat four channels of DDR4-3200 (102.4 GB/s).

How many memory channels do I need for LLM inference?

Use every channel the platform has, and fill them all. That is eight on EPYC 7002 and 7003, and on 3rd Gen Xeon, and six on 1st and 2nd Gen Xeon. A desktop has two.

Can I run an LLM on server RAM without a GPU?

Yes. Speed depends on memory bandwidth, and reading a long prompt is slow because that step is compute-bound. MoE models suit CPU-only servers best. A GPU helps most with prompts.

Can I use desktop DDR4 in a server?

Usually not. Many servers take registered ECC memory only. Check your board's memory list before you pick a module.

Size the RAM first, then fill every channel

Pick the model, work out the RAM with the 0.6 rule, then match the platform. Fill all eight channels with identical modules for the best speed. CPU Republic carries AMD EPYC processors and ECC registered DDR4 server memory. If you're unsure which size fits your board, contact CPU Republic with your CPU and board model. Weighing DDR5 too? Read our DDR4 vs DDR5 server memory guide.