Hardware

How PCIe Lanes Affect LLM Performance? Depending on the task. For training and multi-gpu inferrence - perdormance drop is significant.

I was hit hard by this issue. But you should investigate and test your PC if your BSOD is like mine. The cause was Intel’s CPU Degradation Issue in 13th and 14th generation.

I’ve got a theory to test - if utilising ALL cores on Intel CPU would raise the speed of LLMs? This is bugging me that new gemma3 27 bit model (gemma3:27b, 17GB on ollama) is not fitting into 16GB VRAM of my GPU, and partially running on CPU.

In the midst of the modern world’s turmoil here I’m comparing tech specs of different cards suitable for AI tasks (Deep Learning, Object Detection and LLMs). They are all incredibly expensive though.

When the Ollama server receives two requests at the same time, its behavior depends on its configuration and available system resources.

Installing ET-8500 on Windows is well documented in instruction. The ET-8500 Linux Driver installation is simple but not trivial.

Comparing prediction speed of several versions of LLMs: llama3 (Meta/Facebook), phi3 (Microsoft), gemma (Google), mistral(open source) on CPU and GPU.