LLM-Manager is a lightweight, modular Bash suite with a dual JSON/Interactive interface designed to manage local and remote inference engines across Linux and WSL2.

When I started experimenting with Large Language Models (LLMs) to build an On-Premise RAG (Retrieval-Augmented Generation) application, I hit a massive roadblock: environment fragmentation.

Managing multiple inference engines like Ollama and Llama.cpp meant memorizing different command-line flags, environment variables, and configurations. Once my frontend and backend prototypes were ready for testing, I realized I was spending too much time manually starting, stopping, loading, and unloading models.

I looked online for solutions. Most people suggested complex Python scripts, heavy Docker setups, n8n workflows, or complicated web dashboards.

I didn't want the bloat. I wanted something lightweight that executed commands as if I were doing them manually, but with zero cognitive load.

That is why I built LLM-Manager: a modular orchestration suite written entirely in pure Bash.

Why Bash?

Choosing Bash wasn't about being old-school; it was a pragmatic engineering decision:

1. Zero Overhead: No python virtual environments, no npm install, no runtime dependencies. It’s native and lightning-fast.
2. OS-Level Access: It can natively probe hardware metrics (CPU load, RAM, Disk, GPU VRAM) and manage OS processes.
3. Cross-Platform via WSL2: By utilizing minor PowerShell bridges only when necessary (like starting a server on the Windows host side), the exact same Bash scripts run flawlessly across both native Linux and Windows/WSL2 environments.

The Architecture

The system is designed with a strict plug-and-play modular layout. At the center sits a single entry-point orchestrator (engine-run.sh) that validates arguments against whitelists and routes actions to engine-specific scripts.

.
├── engine.conf               # Global configuration constants
├── engine-models.json        # Model registry with per-engine metadata
├── engine-templates.json     # Prompt/Model templates by family
├── engine-run.sh             # Main orchestrator & entry-point
├── engine-common.sh          # Shared utilities (OS detection, JSON formatting)
├── engine-status.sh          # Cross-engine status aggregation
├── engine-system.sh          # Hardware metric probing
├── logs/                     # Centralized logs
├── llama/                    # Llama.cpp backend scripts
└── ollama/                   # Ollama backend scripts

Every engine directory implements a consistent interface (start.sh, stop.sh, status.sh, load.sh, unload.sh, show.sh, remove.sh). If an engine doesn't support a specific action, a simple stub script that exits with 0 keeps the pipeline happy.

The Dual-Output Contract (Human + Machine)

One of the core features of LLM-Manager is how it handles output.

• Interactive text (Logs, Help, Errors) is routed to stderr.
• Structured JSON data is routed to stdout.

This dual nature makes it perfect for local interactive use, but also means it acts as a local proxy. You can run it over Remote SSH and pipe the clean JSON straight into another monitoring script, custom Web UI, or automation tool.

Example 1: Global Status Check

Running ./engine-run.sh status probes the system metrics and queries active network ports, spitting out a comprehensive payload:

{
  "timestamp": "2026-05-29T06:38:30Z",
  "status": "success",
  "action": "status",
  "engine": "all",
  "data": {
    "system": {
      "os_type": "wsl",
      "memory": { "total_mb": 5927, "available_mb": 4555 },
      "gpu": { "detected": true, "name": "AMD Radeon(TM) Graphics", "vram_total_mb": 512 },
      "cpu": { "cores": 4, "load_1m": 1.11 }
    },
    "engines": {
      "ollama": { "state": "stopped", "port": 1234 },
      "llama": { "state": "stopped", "port": 12345 }
    }
  }
}

Example 2: Interactive Help Interface

If a command fails or is called without parameters, the machine gets the JSON error contract, and the human operator gets a clean, human-readable usage menu:

Error: LLM Manager
Usage: engine-run.sh <action> [engine] [args...]
actions:
    config                           Global config
    models [-h]                      List available models (-h human readable)
    status <engine>                  Show global or engine status
    start <engine> [model] [users]   Start an engine
    stop <engine>                    Stop an engine
...

Dynamic Modelfile Generation

Managing raw .gguf files on Ollama can be a chore since it requires a Modelfile. LLM-Manager abstracts this entirely in the backend via model loading strategies in engine-models.json.

If a model is configured with a gguf loader, the load.sh script dynamically generates the required Modelfile on the fly, injecting correct prompt templates based on the model family, and loading it into Ollama seamlessly. It also supports native strategies to pull directly from the official Ollama registry, or auto to fallback if the local file is missing.

Check out the Code

The project is fully open-source. If you want to see how the WSL2/PowerShell bridges are handled, how the dynamic Modelfiles are generated, or if you want to use it to clean up your own local LLM testing environment, check out the repository:

I am currently working on completing the vLLM engine integration and refining the startup health-checks into proactive retry loops.

Let me know what you think or if you've built similar lightweight alternatives for your AI workflows!