LLM Model Switcher

A simple web-based tool to switch between different llama.cpp model configurations on a headless AI server, featuring a retro-futuristic terminal interface.

How it works

The tool manages llama.cpp configurations by manipulating symbolic links and interacting with systemd:

1. Configurations: It scans /etc/llama.cpp.d for .conf files. Each file should contain the environment variables for a specific model.
2. Active Link: The systemd service for llama.cpp is expected to read its parameters from /etc/llama.cpp.conf.
3. Switching: When you select a model in the web UI:
   • The tool updates the symlink at /etc/llama.cpp.conf to point to the selected file in /etc/llama.cpp.d.
   • It restarts the llama.cpp systemd service to apply the changes.
4. Monitoring: The tool provides a real-time view of the last 100 lines of the llama.cpp service logs using journalctl.

Features

• Model Management: Easily swap between different model configurations.
• Service Logs: Integrated log viewer showing the last 100 lines of the systemd service (newest first).
• Retro UI: A "Cyberpunk CRT" aesthetic with scanlines, phosphor glow, and screen flicker effects.
• Zero Dependencies: The backend uses standard Python libraries, and the frontend uses Preact via ESM—no build step or npm install required.

Requirements

• Python 3.7+ (uses subprocess.run with capture_output).
• sudo privileges (to modify /etc, restart system services, and read journalctl logs).
• A systemd service named llama.cpp.

Usage

Run the script with sudo:

sudo python3 main.py

By default, the server will be available at http://127.0.0.1:7330.

Command Line Options

• --host: Set the listening address (e.g., 0.0.0.0 for all interfaces). If a path is provided (e.g., /tmp/model.sock), it will listen on a Unix socket. (Default: 127.0.0.1)
• --port: Set the TCP port. (Default: 7330)

Project Structure

• main.py: The Python backend using http.server and subprocess.
• static/: Contains the frontend assets (HTML/CSS/JS).
  • index.html: Preact-based application.
  • style.css: Custom "CRT Terminal" styling and animations.