miniGPT

A local LLaMA3 chatbot that runs entirely offline through Ollama, with an automated setup script and in-session conversation memory.

Python
Ollama
LLaMA3
REST APIs

View GitHub All projects

~/miniGPT

$ python3 miniGPT.py
Simple Local AI Chat
------------------------------
Type 'exit' to quit
You > In one sentence: what does a Unix shell do?
AI >
A Unix shell is a command-line interface that allows users to interact with the
operating system by typing commands and receiving output, providing a way to
manage files, directories, processes, and system resources.
------------------------------
You > What did I just ask you? Answer in one sentence.
AI >
You asked me what a Unix shell does in one sentence.
------------------------------
You > exit
Bye.

Real run of the public repo against a local Ollama llama3 model. Blank lines are left out, and the model's replies differ between runs.

Overview

A lightweight command-line chatbot that runs a local LLaMA3 model through Ollama, with no cloud API or external service involved. A companion setup script automates the local environment end to end, installing missing Python dependencies, installing and starting Ollama if it isn't already running, pulling the LLaMA3 model, and launching the chat loop, so the whole thing works from a single command.

Architecture

The project is split into two scripts with a clear separation of concerns: one script handles environment setup, checking for and installing Python dependencies and Ollama, starting the Ollama server, and pulling the llama3 model, before launching the chatbot; the other owns the conversation loop itself, sending each user message plus the running message history to Ollama's local /api/chat endpoint and appending both the user and assistant turns to an in-memory list so the model has conversation context on each request.

Key Features

Fully local inference through Ollama's LLaMA3 model, no API keys, no cloud calls
One-command automated setup that installs Python dependencies and Ollama, pulls the model, and starts the chat loop
Conversation memory maintained for the duration of the session so follow-up questions have context

Technical Challenges

Automating a working local LLM environment from zero, detecting whether Ollama and the model are already installed, and installing them without user intervention when they aren't

Engineering Decisions

Kept conversation history in a simple in-memory list scoped to the session rather than persisting it to disk, since the project's goal was a minimal, fully local chat loop rather than a chat-history product
Called Ollama's REST API directly with the requests library instead of adding an SDK dependency, keeping the script's only external dependency small

Technologies Used

Python for the chatbot and setup automation, calling Ollama's local REST API to run the LLaMA3 model entirely on-device.

Lessons Learned

Practiced integrating with a local LLM runtime over HTTP instead of a hosted API
Learned to script cross-environment setup automation, dependency checks and conditional installs, so a project works out of the box on a clean machine