JARVIS

Overview

JARVIS is my local-first personal AI assistant. It connects a locally hosted language model to tools and persistent memory, with a React interface for voice or typed requests. AI runs inside the application: the model selects tools and uses their results to work through a task.

Memory and execution

The assistant keeps its long-term memory in a plain-text Markdown vault. Notes remain the source of truth; SQLite catalogs, links between notes, and optional vector indexes support retrieval without becoming the only copy of that memory.

A ReAct agent loop connects the model to a typed tool registry. Execution budgets, no-progress checks, and cancellation bound a run. Filesystem operations support snapshots, trash, and rollback records, while policy checks constrain tool execution in code.

FastAPI exposes the backend, and server-sent events carry run activity to the interface. LM Studio provides the local model endpoint.

Voice and text interaction

The frontend uses React, TypeScript, and Vite. Its voice flow records a request, sends it for local transcription with faster-whisper, and plays a synthesized response through a configured local speech engine. The interface reflects listening, thinking, and speaking states while a timeline displays tool activity and results.

Text mode provides another way to issue requests. Voice dependencies and model files are separate setup requirements: without a configured speech voice, responses can still appear as text. The interface also reports backend model availability so the user can distinguish a disconnected runtime from a pending response.

Current state

The public repository contains the backend, frontend, and automated tests. JARVIS remains in progress and requires a local model runtime and configuration to use. The source link includes setup instructions; there is no hosted demo linked here.