# ruvllm

> Local LLM inference engine supporting GGUF models with hardware acceleration on Metal, CUDA, ANE, WebGPU. Features Flash Attention, MicroLoRA, RoPE, quantization (Q4-Q8, π-Quantization), MoE routing, and streaming tokens for browser and edge deployment.

- **URL:** https://crates.io/crates/ruvllm
- **Category:** LLM Tools
- **Tags:** llm-inference, Wasm, Quantization, Open Source
- **Updated:** 2026-04-07 10:19
- **Canonical page:** https://vectordb.ever.works/items/ruvllm

## Details

## Overview

ruvllm enables local AI without cloud APIs.

## Features

- ruvllm-wasm: Browser WASM runtime (435 KB)
- Hardware: Metal/CUDA/ANE/WebGPU
- Quantization: Q4, Q5, Q8, π-Quantization (2-bit weights)
- MoE Memory-Aware Routing
- Streaming tokens, KV cache

## Installation

```sh
npm install @ruvector/ruvllm
npm install @ruvector/ruvllm-wasm
cargo add ruvllm
```

## Pricing

Free and open-source.

---

_This Markdown mirror is generated by Ever Works for AI agents. The canonical HTML page is at https://vectordb.ever.works/items/ruvllm._