Instructions to use mradermacher/DeepCoder-1.5B-Preview-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mradermacher/DeepCoder-1.5B-Preview-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("mradermacher/DeepCoder-1.5B-Preview-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use mradermacher/DeepCoder-1.5B-Preview-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/DeepCoder-1.5B-Preview-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/DeepCoder-1.5B-Preview-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/DeepCoder-1.5B-Preview-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/DeepCoder-1.5B-Preview-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mradermacher/DeepCoder-1.5B-Preview-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf mradermacher/DeepCoder-1.5B-Preview-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mradermacher/DeepCoder-1.5B-Preview-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mradermacher/DeepCoder-1.5B-Preview-GGUF:Q4_K_M

Use Docker

docker model run hf.co/mradermacher/DeepCoder-1.5B-Preview-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use mradermacher/DeepCoder-1.5B-Preview-GGUF with Ollama:
```
ollama run hf.co/mradermacher/DeepCoder-1.5B-Preview-GGUF:Q4_K_M
```

Unsloth Studio

How to use mradermacher/DeepCoder-1.5B-Preview-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for mradermacher/DeepCoder-1.5B-Preview-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for mradermacher/DeepCoder-1.5B-Preview-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for mradermacher/DeepCoder-1.5B-Preview-GGUF to start chatting

Docker Model Runner
How to use mradermacher/DeepCoder-1.5B-Preview-GGUF with Docker Model Runner:
```
docker model run hf.co/mradermacher/DeepCoder-1.5B-Preview-GGUF:Q4_K_M
```

Lemonade

How to use mradermacher/DeepCoder-1.5B-Preview-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mradermacher/DeepCoder-1.5B-Preview-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.DeepCoder-1.5B-Preview-GGUF-Q4_K_M

List all available models

lemonade list

Atomic Chat

mradermacher commited on Apr 8, 2025

Commit

d4d1901

verified ·

1 Parent(s): 1297417

uploaded from leia

Browse files

Files changed (13) hide show

.gitattributes +12 -0
DeepCoder-1.5B-Preview.IQ4_XS.gguf +3 -0
DeepCoder-1.5B-Preview.Q2_K.gguf +3 -0
DeepCoder-1.5B-Preview.Q3_K_L.gguf +3 -0
DeepCoder-1.5B-Preview.Q3_K_M.gguf +3 -0
DeepCoder-1.5B-Preview.Q3_K_S.gguf +3 -0
DeepCoder-1.5B-Preview.Q4_K_M.gguf +3 -0
DeepCoder-1.5B-Preview.Q4_K_S.gguf +3 -0
DeepCoder-1.5B-Preview.Q5_K_M.gguf +3 -0
DeepCoder-1.5B-Preview.Q5_K_S.gguf +3 -0
DeepCoder-1.5B-Preview.Q6_K.gguf +3 -0
DeepCoder-1.5B-Preview.Q8_0.gguf +3 -0
DeepCoder-1.5B-Preview.f16.gguf +3 -0

@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+DeepCoder-1.5B-Preview.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+DeepCoder-1.5B-Preview.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+DeepCoder-1.5B-Preview.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+DeepCoder-1.5B-Preview.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+DeepCoder-1.5B-Preview.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+DeepCoder-1.5B-Preview.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+DeepCoder-1.5B-Preview.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+DeepCoder-1.5B-Preview.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+DeepCoder-1.5B-Preview.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+DeepCoder-1.5B-Preview.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+DeepCoder-1.5B-Preview.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+DeepCoder-1.5B-Preview.f16.gguf filter=lfs diff=lfs merge=lfs -text

+version https://git-lfs.github.com/spec/v1
+oid sha256:1788e6bc54fcb02a0bf1024e9b98c9ace965eedc4eed2f139c1eeacd16920852
+size 1026163680

+version https://git-lfs.github.com/spec/v1
+oid sha256:91b89fcb113492ab92a9018c3195d285ce8cc85c31b048779d74d1ed3404d123
+size 752881632

+version https://git-lfs.github.com/spec/v1
+oid sha256:12978341b70fac965982f30778ecd207eaef9f0fb10e2abb05a7c593177bccd7
+size 980441568

+version https://git-lfs.github.com/spec/v1
+oid sha256:febff47eb6459767049974b114c01064971482cba9ed38c43666ba5323c164c6
+size 924457440

+version https://git-lfs.github.com/spec/v1
+oid sha256:2445ffb361c2e23c44f97576fa4ea11a801cd87905df0c30f19b9cd5919a5cb0
+size 861223392

+version https://git-lfs.github.com/spec/v1
+oid sha256:7d9be29579e424e57ff5221934c0fb6c601a1946364e3adbc744376891bdf4ec
+size 1117322208

+version https://git-lfs.github.com/spec/v1
+oid sha256:4bd4d898bad6d3bfc6f57be2e70407330677ec7a56e07612ba67885054a3fa2d
+size 1071586272

+version https://git-lfs.github.com/spec/v1
+oid sha256:c4261b99a56711ef88706efbd2e3c0e0283bd716714d242464516995f9aced63
+size 1285495776

+version https://git-lfs.github.com/spec/v1
+oid sha256:d9854f004340c6d6c2e309a6fc23a200052c61db5ec808013a111415941d3124
+size 1259174880

+version https://git-lfs.github.com/spec/v1
+oid sha256:d50c8d55e79c32589fdf5a20f737a66c720ac39d618de11c639ccd56f6417d07
+size 1464180192

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:eafbc67515e605f4fafdb360329a67b85ada951c9d0ec779005cf1ee8ad1ce56
+size 1894533600

+version https://git-lfs.github.com/spec/v1
+oid sha256:d629df337d620567378364958c794a3e713715de826dc497b5612b667195e77f
+size 3560417760