Topic Hub
AI & ML on Cloud Native
Running AI/ML workloads on Kubernetes and modern infrastructure.
GPU scheduling, model serving, Kubeflow, LLMs on Kubernetes, NVIDIA NVCF, and the rapidly evolving AI infrastructure landscape.
Start here
NVCF Is Now Open Source: Inside NVIDIA's GPU Function Platform
NVIDIA just open-sourced the full NVCF platform under Apache 2.0. Not a thin SDK, not a client library. The actual control plane, invocation plane,…
K8sGPT Tutorial - When Kubernetes Meets AI
In this blog we’ll explore k8sGPT, a powerful tool that brings the capabilities of AI to change the way you manage Kubernetes.
Kubeflow: Machine Learning on Kubernetes - Part 1
Developing and deploying machine learning systems could be a pain with multiple things you need to manage. In this article, I introduce you and help you…
"SSH Into Your DGX Spark From Anywhere in the World Using Tailscale
Learn how to set up Tailscale on your NVIDIA DGX Spark for secure SSH access from anywhere in the world.
More on AI & ML on Cloud Native
Running a big LLM across multiple GPUs with vLLM
A plain-English guide to serving a model too big for one GPU, in two tracks: a runbook from download to serving with every flag and error explained, and a deep dive into how tensor, pipeline, and expert parallelism split the model, with measured numbers from a 235B model on four RTX PRO 6000 cards.
The Local LLM Glossary: Every Term, Flag, and Number in Plain English
Plain-English definitions for every term you hit in local LLM posts: prefill and decode, tokens per second, FP8 and NVFP4, Q4_K_M, KV cache, YaRN, Gated DeltaNet, speculative decoding, and every vLLM, llama.cpp, and Ollama flag worth knowing.
Running Qwen3.8-27B on DGX Spark
Qwen3.8-27B on DGX Spark with llama.cpp, Ollama, vLLM, and SGLang: the recipes, the tokens per second I measured, MTP speculative decoding, and the sharp edges I hit along the way.
I Ran an AI SRE Copilot on My Own Hardware. Here Is What It Actually Does.
Running NudgeBee v1.4.0 end to end - a self-hosted AIOps platform behind AI-SRE, AI-FinOps, AI-K8sOps, and agentic automation - on a Mac, a kiac cluster, and a DGX Spark.
Running Nemotron 3.5 Lightning on DGX Spark
NVIDIA's new Nemotron 3.5 Lightning on DGX Spark: how to run it with Ollama and vLLM, the tokens per second I measured, and how the two paths compare.
HAMi Dynamic MIG on RTX PRO 6000: A Live Kubernetes Test
Hands-on HAMi Dynamic MIG test on Kubernetes and RTX PRO 6000 Blackwell: setup commands, real allocations, mixed profiles, reclamation, and recovery.
How to Share GPUs in Kubernetes at Scale with HAMi (Software vGPU Slicing)
Share NVIDIA GPUs in Kubernetes with HAMi software vGPU slicing: memory and compute limits, Helm configuration, a verified PyTorch manifest, a real RTX PRO 6000 OOM test, and Prometheus monitoring.
Slicing GPUs in Kubernetes with NVIDIA Multi-Instance GPU (MIG)
GPU sharing in Kubernetes explained: time-slicing vs MPS vs MIG, every nvidia-smi command to enable and disable MIG on one GPU or eight, GPU Operator automation, pitfalls, and DCGM monitoring.
Day 5: Local LLM Inference Engines, Wrappers, and What to Pick
A beginner-friendly guide to local LLM inference, with the same Qwen model tested through Ollama, llama.cpp, Docker Model Runner, vLLM, SGLang, and TensorRT-LLM on NVIDIA DGX Spark.
Bonsai 27B on RTX PRO 6000 vs DGX Spark: what actually works
Real Bonsai 27B benchmarks on an RTX PRO 6000 and a DGX Spark, including the supported llama.cpp setup, ternary vs 1-bit results, and speculative decoding.
Day 4: Quantization Demystified. BF16, FP8, NVFP4, MXFP4, INT4, GGUF, and Why It All Matters
A practical, beginner-friendly guide to BF16, FP8, NVFP4, MXFP4, INT4, and GGUF Q4_K_M on NVIDIA DGX Spark. Bytes per parameter, quality vs size, and which format to pick when.
Day 3: The DGX Spark Unpacked. GB10, Unified Memory, sm_121, and the One Reason This Hardware Exists
A practical teardown of NVIDIA DGX Spark's GB10 Grace Blackwell Superchip, unified memory, sm_121, NVFP4 tensor cores, memory reporting, and decode limits.
Show 11 more AI & ML on Cloud Native articles
- mlxcel: A Rust-Native Inference Engine for Apple Silicon, Tested on My M1 Max28 min
- Day 2: Anatomy of an LLM Inference Request. From Prompt to Answer, Step by Step26 min
- Day 1: The Local LLM Revolution. Why Your Desk Just Became the New Datacenter13 min
- Day 6: Run an LLM on Your Laptop - With Docker10 min
- What Claude Code's Leaked Source Actually Teaches Us About Building AI Agents8 min
- Here's What I Learned About Nemotron 3 Super -I Ran a 120B Parameter Model on Nvidia DGX Spark14 min
- Ditch the Overheating Laptop: Supercharge Your Docker Workflow with Docker Offload5 min
- Docker MCP Catalog: Finding the Right AI Tools for Your Project6 min
- Testing Docker AI's "Gordon" – How Smart Is It?9 min
- Kubeflow Pipelines: Orchestrating Machine Learning Workflows - Part 317 min
- Kubeflow Notebooks: ML Experimentation Made Easier - Part 214 min