Instructions to use MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python")
model = AutoModelForCausalLM.from_pretrained("MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python

SGLang

How to use MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python with Docker Model Runner:
```
docker model run hf.co/MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python
```

Llama-3.2-1B-Instruct-FFT-coder-python / README.md

MDDDDR

Update README.md

115cba5 verified over 1 year ago

preview code

raw

history blame contribute delete

3.34 kB

metadata

license: apache-2.0
language:
  - ko
base_model:
  - meta-llama/Llama-3.2-1B-Instruct
pipeline_tag: text-generation
library_name: transformers
datasets:
  - nayohan/CodeFeedback-Filtered-Instruction-ko

Model Card for Model ID

base_model : meta-llama/Llama-3.2-1B-Instruct

Training dataset

data_set : nayohan/CodeFeedback-Filtered-Instruction-ko
- 해당 데이터셋을 전부 사용한건 아니며 Python언어를 우선 추출한다음 데이터셋들의 생김새를 파악, 그 다음 전처리가 공통적으로 들어갈만한 데이터를 다시 추출하여 학습에 사용했습니다.
  - 총 학습 데이터 건 : 49,859 건

Basic usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = 'MDDDDR/Llama-3.2-1B-Instruct-FFT-coder-python'
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id,
                                             device_map="cuda:0",
                                             torch_dtype=torch.bfloat16)


instruction = '''LCS(Longest Common Subsequence, 최장 공통 부분 수열)문제는 두 수열이 주어졌을 때, 모두의 부분 수열이 되는 수열 중 가장 긴 것을 찾는 문제이다.

예를 들어, ACAYKP와 CAPCAK의 LCS는 ACAK가 된다.

###입력 : 첫째 줄과 둘째 줄에 두 문자열이 주어진다. 문자열은 알파벳 대문자로만 이루어져 있으며, 최대 1000글자로 이루어져 있다.
###출력 : 첫째 줄에 입력으로 주어진 두 문자열의 LCS의 길이를 출력한다.

###입력 예제 : 
ACAYKP
CAPCAK
###출력 예제 : 4
'''

messages = [
    {
        "role":"user",
        "content":"아래는 문제를 설명하는 지시사항입니다. 이 요청에 대해 적절하게 답변해주세요.\n###지시사항:{instruction}\n###답변:".format(instruction=instruction)
    }
]

with torch.no_grad():
  prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=False)
  inputs = tokenizer(prompt, return_tensors="pt", padding=False).to('cuda')
  outputs = model.generate(**inputs, 
                           use_cache=False, 
                           max_length=256, 
                           top_p=0.9,
                           temperature=0.7, 
                           repetition_penalty=1.0,
                           pad_token_id=tokenizer.pad_token_id)

output_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
final_output = output_text.split('답변:')[-1].strip()
print(final_output)
# ```python
# def longest_common_subsequence(str1, str2):
#     m = len(str1)
#     n = len(str2)
#     dp = [[0] * (n+1) for _ in range(m+1)]
#     
#     for i in range(m+1):
#         for j in range(n+1):
#             if i == 0 or j == 0:
#                 dp[i][j] = 0
#             elif str1[i-1] == str2[j-1]:
#                 dp[i][j] = dp[i-1][j-1] + 1
#             else:
#                 dp[i][j] = max(dp[i-1][j], dp[i][j-1])
#     
#     return dp[m][n]
# 
# print(longest_common_subsequence("ACAYKP", "CAPCAK"))  # Output: 4
# ```

Hardware

A100 40GB x 1
Training Time : 1 hour 45 minutes