Instructions to use WokeAI/Tankie-DPE-12b-SFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use WokeAI/Tankie-DPE-12b-SFT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="WokeAI/Tankie-DPE-12b-SFT")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("WokeAI/Tankie-DPE-12b-SFT")
model = AutoModelForCausalLM.from_pretrained("WokeAI/Tankie-DPE-12b-SFT")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use WokeAI/Tankie-DPE-12b-SFT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "WokeAI/Tankie-DPE-12b-SFT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "WokeAI/Tankie-DPE-12b-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/WokeAI/Tankie-DPE-12b-SFT

SGLang

How to use WokeAI/Tankie-DPE-12b-SFT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "WokeAI/Tankie-DPE-12b-SFT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "WokeAI/Tankie-DPE-12b-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "WokeAI/Tankie-DPE-12b-SFT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "WokeAI/Tankie-DPE-12b-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use WokeAI/Tankie-DPE-12b-SFT with Docker Model Runner:
```
docker model run hf.co/WokeAI/Tankie-DPE-12b-SFT
```

Very interesting and unique model

by bobp - opened Feb 5

Discussion

bobp

Feb 5

I know it's supposed to basically be a model for discussing specific political ideas but just using it in any roleplay scenario I find that it has its own style of writing that is different from any RP finetune I've tried (most of them output the same sloppa with mildly different wordings) or base Nemo itself. Refreshingly unique while still maintaining a very usable level of intelligence so long as you are not overly liberal with token sampling. I hope there will be more experiments like this.

Fizzarolli

WokeAI org Feb 5

We actually found this too (we're mostly a roleplay org we just do actual science sometimes)! A lot of it, I think, is from training on Kimi synth data that was specifically not in the RP realm, so it didn't overfit to the specific Kimi RP phrases while still having that sort of """Kimi style""" to it

Fizzarolli

WokeAI org Feb 5

•

edited Feb 5

also you best not be overly liberal with sampling this is a TANKIE model. we are strictly anti-revisionist in this house!!! /j

bobp

Feb 5

Hah, also I forgot to mention it but I found the model because I was looking at the UGI leaderboard and this was the model with the lowest "semantic redundancy" score in the 12B size (and very low compared to any size really), suggesting it doesn't really repeat itself much using different phrases to say the same thing again which is a pretty big Nemo problem.

Fizzarolli

WokeAI org Feb 5

annoyingly this model seems to have done good on the ugi leaderboard at everything but a non-liberal political tilt, with the og kimi still beating it out

inflatebot

WokeAI org Feb 7

I continue to be irritatingly wrong about character training /silly

Fizzarolli

WokeAI org Feb 9

there's new ones now btw https://huggingface.co/collections/WokeAI/t-34

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment