Ollama (Orchestrator / Inference Engine)
Ollama is a model runner (also called an inference engine). Ollama runs the LLM Models like: Llama, Gemma, Phi, qwen etc
[ Go Client ]
│ (HTTP / JSON REST API)
▼
[ Ollama Binary (Executable) ] ───(Loads into memory)───► [ Model Weights (Static Data File) ]
│
▼ (Performs Matrix Multiplication Math)
[ CPU / GPU Hardware Compute ]
Can Ollama run any LLM model?
Ollama understands GGUF format. Any model which can be converted to this
can run on Ollama
1. Direct Support: Ollama has an official library supporting
dozens of mainstream open models (Llama 3, Gemma 2, Mistral, Qwen,
DeepSeek). You can download these instantly with ollama run
2. Custom Models: If you find a niche or custom-tuned open-source
model on Hugging Face (the GitHub of AI), you can download its .gguf
file and use a simple file called a Modelfile to load it into Ollama
manually.
Installing & Running Ollama
$ curl -fsSL https://ollama.com/install.sh | sh
>>> Cleaning up old version at /usr/local/lib/ollama
>>> Installing ollama to /usr/local
>>> Downloading ollama-linux-amd64.tar.zst
Above script will create /etc/systemd/system/ollama.service
and place binary inside /usr/bin/ and libs inside /usr/libs
$ sudo systemctl restart ollama
$ sudo systemctl status ollama
● ollama.service - Ollama Service
Loaded: loaded (/etc/systemd/system/ollama.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2026-06-15 22:39:53 IST; 6s ago
Main PID: 134899 (ollama)
Tasks: 13 (limit: 9229)
Memory: 19.8M
CPU: 663ms
CGroup: /system.slice/ollama.service
└─134899 /usr/local/bin/ollama serve
Download qwen(Model) and run in Ollama
|
Check Model is running? |
Hit AI Model (qwen's) REST APIs
Input: "indarpur unfollowed for 2 weaks" (with a typo on 'weeks').
Output: The model interpreted the command ("Return JSON only."),
extracted the semantic meaning of misspelled input phrase, corrected the
spelling error, mapped it to key-value variables, and wrapped it cleanly
in a markdown code block.
{
"status": "Unfollowed",
"duration": "2 weeks"
}
$ cat test.json
{
"model":"qwen2.5:1.5b",
"messages":[
{
"role":"system",
"content":"Return JSON only."
},
{
"role":"user",
"content":"indarpur unfollowed for 2 weaks"
}
],
"temperature":0,
"stream":false
}
$ curl http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d @test.json
{"id":"chatcmpl-421","object":"chat.completion","created":1781543972,"model":"qwen2.5:1.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"message":{"role":"assistant","content":"```json\n{\n \"status\": \"Unfollowed\",\n \"duration\": \"2 weeks\"\n}\n```"},"finish_reason":"stop"}],"usage":{"prompt_tokens":28,"completion_tokens":24,"total_tokens":52}}
Qwen Logs
//Since Ollama is running inside a systemd service unit, logs are managed natively by journald.
$ journalctl -u ollama.service -f
Jun 15 22:49:31 Amit ollama[134899]: slot update_slots: id 0 | task 22 | cached n_tokens = 3, memory_seq_rm [3, end)
Jun 15 22:49:31 Amit ollama[134899]: slot init_sampler: id 0 | task 22 | init sampler, took 0.01 ms, tokens: text = 28, total = 28
Jun 15 22:49:31 Amit ollama[134899]: srv log_server_r: done request: POST /completion 127.0.0.1 200
Jun 15 22:49:32 Amit ollama[134899]: slot print_timing: id 0 | task 22 | prompt eval time = 405.33 ms / 25 tokens ( 16.21 ms per token, 61.68 tokens per second)
Jun 15 22:49:32 Amit ollama[134899]: slot print_timing: id 0 | task 22 | eval time = 786.05 ms / 24 tokens ( 32.75 ms per token, 30.53 tokens per second)
Jun 15 22:49:32 Amit ollama[134899]: slot print_timing: id 0 | task 22 | total time = 1191.39 ms / 49 tokens
Jun 15 22:49:32 Amit ollama[134899]: slot print_timing: id 0 | task 22 | graphs reused = 41
Jun 15 22:49:32 Amit ollama[134899]: slot release: id 0 | task 22 | stop processing: n_tokens = 51, truncated = 0
Jun 15 22:49:32 Amit ollama[134899]: srv update_slots: all slots are idle
Jun 15 22:49:32 Amit ollama[134899]: [GIN] 2026/06/15 - 22:49:32 | 200 | 1.409126234s | 127.0.0.1 | POST "/v1/chat/completions"
// Increase logs level
$ sudo vim /etc/systemd/system/ollama.service
Environment="OLLAMA_HOST=0.0.0.0"
Environment="OLLAMA_DEBUG=1"
$ sudo systemctl daemon-reload
$ sudo systemctl restart ollama
Use Cases
CRM Voice Search feature (Speak donot Write)
- Business Problem?
You have a frontend CRM app written in react native, which hits REST endpoints on golang backend and pushes and gets data. App has ViewAllRequirements page which shows all requirements in row format. When user clicks on 1 requirement it shows complete information about the requirement. Now we want a feature on frontend app(smart search), that employee or owner SPEAK on Frontend app example:
All requirements which are not followed up since 2 weeks OR
All requirements from Ajnara grace OR
(in Hindi) Mujhe piche 5 dino k untouched requirement do OR
indarpur unfollowed for 2 weaks //Spelling mistake. Area name mistake
Frontend should show the authorized data. What changes are required on frontend and what are required on backend? We want 0 cost solution, backend would hosted on AWS.
- Solution
Uninstall Ollama and Qwen
$ sudo systemctl status ollama
[sudo] password for amit:
● ollama.service - Ollama Service
Loaded: loaded (/etc/systemd/system/ollama.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2026-06-21 10:41:11 IST; 5min ago
$ ollama list
NAME ID SIZE MODIFIED
qwen2.5:1.5b 65ec06548149 986 MB 5 days ago
$ ollama rm qwen2.5:1.5b
ollama list
NAME ID SIZE MODIFIED
sudo rm -rf /usr/share/ollama/.ollama/models
sudo systemctl stop ollama
sudo systemctl disable ollama
sudo rm /etc/systemd/system/ollama.service
sudo systemctl daemon-reload
sudo rm -f /usr/local/bin/ollama
sudo rm -f /usr/bin/ollama
sudo rm -rf /usr/share/ollama
sudo userdel ollama
sudo groupdel ollama