allenai/olmOCR-2-7B-1025-FP8
Image-Text-to-Text ā¢ 8B ā¢ Updated
ā¢ 265k ā¢ 210
Artem Darius Weber
milkomeda22
AI & ML interests
None yet
Recent Activity
liked
a model 12 days ago
allenai/olmOCR-2-7B-1025-FP8 liked
a model 25 days ago
OpenGVLab/InternVL3_5-241B-A28B upvoted a collection 25 days ago
InternVL3.5 Organizations
liked a model 12 days ago
liked a model 25 days ago
upvoted a collection 25 days ago
liked a model about 1 month ago
liked a Space 3 months ago
liked a model 3 months ago
upvoted a paper 4 months ago
upvoted an article 4 months ago
Article
Design Patterns for Building Agentic Workflows
ā¢
9
liked 3 models 5 months ago
reacted to codelion's post with š„ 6 months ago
Post
6199
I recently worked on a LoRA that improves tool use in LLM. Thought the approach might interest folks here.
The issue I have had when trying to use some of the local LLMs with coding agents is this:
Me: "Find all API endpoints with authentication in this codebase"
LLM: "You should look for @app .route decorators and check if they have auth middleware..."
But I often want it to search the files and show me but the LLM doesn't trigger a tool use call.
To fine-tune it for tool use I combined two data sources:
1. Magpie scenarios - 5000+ diverse tasks (bug hunting, refactoring, security audits)
2. Real execution - Ran these on actual repos (FastAPI, Django, React) to get authentic tool responses
This ensures the model learns both breadth (many scenarios) and depth (real tool behavior).
Tools We Taught:
-
-
-
-
-
-
Improvements:
- Tool calling accuracy: 12% ā 80%
- Correct parameters: 8% ā 87%
- Multi-step tasks: 3% ā 78%
- End-to-end completion: 5% ā 80%
- Tools per task: 0.2 ā 3.8
The LoRA really improves on intential tool call as an example consider the query: "Find ValueError in payment module"
The response proceeds as follows:
1. Calls
2. Gets 4 matches across 3 files
3. Calls
4. Analyzes context
5. Reports: "Found 3 ValueError instances: payment/processor.py:47 for invalid amount, payment/validator.py:23 for unsupported currency..."
Resources:
- Colab notebook https://colab.research.google.com/github/codelion/ellora/blob/main/Ellora_Recipe_3_Enhanced_Tool_Calling_and_Code_Understanding.ipynb
- Model - codelion/Llama-3.2-1B-Instruct-tool-calling-lora
- GitHub - https://github.com/codelion/ellora
The issue I have had when trying to use some of the local LLMs with coding agents is this:
Me: "Find all API endpoints with authentication in this codebase"
LLM: "You should look for @app .route decorators and check if they have auth middleware..."
But I often want it to search the files and show me but the LLM doesn't trigger a tool use call.
To fine-tune it for tool use I combined two data sources:
1. Magpie scenarios - 5000+ diverse tasks (bug hunting, refactoring, security audits)
2. Real execution - Ran these on actual repos (FastAPI, Django, React) to get authentic tool responses
This ensures the model learns both breadth (many scenarios) and depth (real tool behavior).
Tools We Taught:
-
read_file - Actually read file contents-
search_files - Regex/pattern search across codebases-
find_definition - Locate classes/functions-
analyze_imports - Dependency tracking-
list_directory - Explore structure-
run_tests - Execute test suitesImprovements:
- Tool calling accuracy: 12% ā 80%
- Correct parameters: 8% ā 87%
- Multi-step tasks: 3% ā 78%
- End-to-end completion: 5% ā 80%
- Tools per task: 0.2 ā 3.8
The LoRA really improves on intential tool call as an example consider the query: "Find ValueError in payment module"
The response proceeds as follows:
1. Calls
search_files with pattern "ValueError"2. Gets 4 matches across 3 files
3. Calls
read_file on each match4. Analyzes context
5. Reports: "Found 3 ValueError instances: payment/processor.py:47 for invalid amount, payment/validator.py:23 for unsupported currency..."
Resources:
- Colab notebook https://colab.research.google.com/github/codelion/ellora/blob/main/Ellora_Recipe_3_Enhanced_Tool_Calling_and_Code_Understanding.ipynb
- Model - codelion/Llama-3.2-1B-Instruct-tool-calling-lora
- GitHub - https://github.com/codelion/ellora
liked 5 models 6 months ago
litagin/anime-whisper
Automatic Speech Recognition ā¢ 0.8B ā¢ Updated
ā¢ 5.38k ā¢ 122
litagin/anime_speaker_embedding_ecapa_tdnn_groupnorm
Audio Classification ā¢ Updated
ā¢ 8
litagin/anime_speaker_embedding_by_va_ecapa_tdnn_groupnorm
Audio Classification ā¢ Updated
ā¢ 3
litagin/anime_speech_emotion_classification
Audio Classification ā¢ 0.2B ā¢ Updated
ā¢ 540 ā¢ 6
openbmb/MiniCPM-V-4_5
Image-Text-to-Text ā¢ Updated
ā¢ 69.6k ā¢ 1.07k
reacted to AdinaY's post with ā¤ļø 6 months ago
Post
5089
MiniCPM-V 4.5 š New MLLM for image, multi-image & video understanding, running even on your phone, released by OpenBMB
openbmb/MiniCPM-V-4_5
āØ SOTA vision language capability
āØ 96Ć video token compression > high-FPS & long video reasoning
āØ Switchable fast vs deep thinking modes
āØ Strong OCR, document parsing, supports 30+ languages
openbmb/MiniCPM-V-4_5
āØ SOTA vision language capability
āØ 96Ć video token compression > high-FPS & long video reasoning
āØ Switchable fast vs deep thinking modes
āØ Strong OCR, document parsing, supports 30+ languages
liked a dataset 6 months ago
liked a dataset 7 months ago