README.md

---
base_model: unsloth/llama-3.2-3b-instruct-bnb-4bit
language:
- en
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- Llama-3.2-3B
- Medical-Coding
- Healthcare
- CMS
- OASIS
---

# OASISCoder-Llama-3.2-3B

### Model Description
**OASISCoder-Llama-3.2-3B** is a fine-tuned version of the **LLaMA 3.2 3B** model, trained specifically on a **medical coding dataset** focusing on **CMS (Centers for Medicare & Medicaid Services)** and **OASIS (Outcome and Assessment Information Set)** standards. The model is designed to assist healthcare professionals and organizations in generating accurate medical codes and streamlining documentation tasks. It supports **medical question-answering**, **medical coding**, and **clinical decision support**, with an emphasis on regulatory compliance and documentation quality in the US healthcare system.

### Intended Use Cases
- **Medical Coding (CMS, OASIS)**: Supports automated or semi-automated coding tasks in clinical documentation, reducing administrative burden for healthcare providers.
- **Clinical Decision Support**: Provides relevant, context-aware answers based on healthcare standards and medical queries.
- **Healthcare QA Systems**: Useful for building medical chatbots and virtual assistants that handle queries related to CMS regulations, OASIS standards, and healthcare procedures.
- **Medical Compliance**: Ensures accurate documentation for home healthcare assessments and improves regulatory compliance in clinical settings.

### Training Data
The model was fine-tuned on a **comprehensive medical coding dataset** integrating **CMS** and **OASIS** data, including real-world clinical documentation and coding tasks. The dataset includes examples of medical diagnoses, procedures, patient assessments, and coding annotations following CMS and OASIS regulations.

### Architecture
The model is based on **LLaMA 3.2 3B**, a powerful large language model architecture optimized for language understanding and generation tasks. Fine-tuning on the medical domain allows it to provide highly specialized and accurate outputs for healthcare tasks.

### Performance
- **Accuracy**: The model demonstrates a high accuracy rate in generating CMS and OASIS codes from clinical text and answering medical queries.
- **Efficiency**: Fine-tuning on specific healthcare tasks has reduced the model's processing time for medical coding and decision support tasks.

### Limitations
- **Not a Diagnostic Tool**: This model is not intended for making medical diagnoses and should not be used as a replacement for professional medical judgment.
- **Bias and Data Coverage**: The model's performance is best on US healthcare data (CMS, OASIS), and may not generalize well to other healthcare systems or international coding standards.

### License
The model is released under the **Apache License 2.0**, making it available for non-commercial research and development purposes.

### How to Use

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("exafluence/OASISCoder-Llama-3.2-3B")
model = AutoModelForCausalLM.from_pretrained("exafluence/OASISCoder-Llama-3.2-3B")

input_text = "What is the CMS code for a patient with diabetes?"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)
```

If you use this model, please cite:
```bibtex
@inproceedings{exafluence2024OASISCoder,
  title={OASISCoder-Llama-3.2-3B: A Medical Coding Language Model for CMS and OASIS},
  author={Exafluence Inc.},
  year={2024},
  doi={10.57967/hf/3260},
  url={https://huggingface.co/exafluence/OASISCoder-Llama-3.2-3B}
}
```

# Uploaded  model

- **Developed by:** exafluence
- **License:** apache-2.0
- **Finetuned from model :** unsloth/llama-3.2-3b-instruct-bnb-4bit

This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)