Situatie
Ollama is an open-source tool that lets you run Large Language Models (LLMs) locally on your computer. It provides an easy way to download, run, and interact with AI models without needing cloud services.
A Large Language Model (LLM) is a type of AI model trained on massive text datasets to understand and generate human-like text.
Examples of LLMs:
Llama 3 (Meta)
Mistral 7B (Mistral AI)
Gemma 2B (Google DeepMind)
GPT-4 (OpenAI, not open-source)
Solutie
1)Install Ollama on Linux:
curl -fsSL https://ollama.com/install.sh | sh
2)Quickstart
To run and chat with Llama 3.2:
ollama run llama3.2 -This will run the Llama 3.2 with 3B parameters using 2.0GB of VRAM.
You should have at least 8 GB of VRAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.You should run “watch -n 1 nvidia-smi” to check and monitor your GPU usage (nvidia GPU).
3. Download More Models
Here are some example models that we can use with OLLAMA:
Model Parameters Size Download
DeepSeek-R1 7B 4.7GB ollama run deepseek-r1
DeepSeek-R1 671B 404GB ollama run deepseek-r1:671b
Llama 3.3 70B 43GB ollama run llama3.3
Llama 3.2 3B 2.0GB ollama run llama3.2
Llama 3.2 1B 1.3GB ollama run llama3.2:1b
Llama 3.1 8B 4.7GB ollama run llama3.1
Llama 3.1 405B 231GB ollama run llama3.1:405b
Phi 4 14B 9.1GB ollama run phi4
Phi 3 Mini 3.8B 2.3GB ollama run phi3
Gemma 2 2B 1.6GB ollama run gemma2:2b
Gemma 2 9B 5.5GB ollama run gemma2
Gemma 2 27B 16GB ollama run gemma2:27b
Mistral 7B 4.1GB ollama run mistral
Moondream 2 1.4B 829MB ollama run moondream
Neural Chat 7B 4.1GB ollama run neural-chat
Starling 7B 4.1GB ollama run starling-lm
Code Llama 7B 3.8GB ollama run codellama
Llama 2 Unc. 7B 3.8GB ollama run llama2-uncensored
LLaVA 7B 4.5GB ollama run llava
Solar 10.7B 6.1GB ollama run solar
To download a specific model: ollama run gemma2:2b. For full list visit https://ollama.com/search.
Leave A Comment?