Porting Vision-Language Models to Apple Silicon with MLX: A Tutorial Series

Are you interested in running cutting-edge AI models efficiently on your Mac? We're excited to share a detailed tutorial series on porting Phi-3-Vision to Apple's MLX framework!

This 8-part series covers:

1. Basic Implementation: Translating core components from PyTorch to MLX
2. Su-scaled Rotary Position Embeddings (SuRoPE): Enabling 128K token contexts
3. Batching: Processing multiple inputs simultaneously for improved efficiency
4. Caching: Optimizing inference speed for autoregressive generation
5. Choice Selection: Implementing constrained outputs for multiple-choice scenarios
6. Constrained Decoding: Guiding model outputs with flexible constraints
7. LoRA Training: Fine-tuning models efficiently with Low-Rank Adaptation
8. Agent & Toolchain System: Building flexible AI workflows

Whether you're an AI enthusiast, researcher, or developer looking to leverage Apple Silicon, this series provides a deep dive into optimizing advanced vision-language models. You'll learn hands-on techniques for model porting, performance optimization, and extending model capabilities.

Check out the full series for a comprehensive guide to running state-of-the-art AI on your Mac!

Link to the tutorial series:

https://medium.com/@albersj66

All the code examples and implementations discussed in this tutorial series are available in our GitHub repository:

https://github.com/JosefAlbers/Phi-3-Vision-MLX

This repository contains:
- Full implementation of Phi-3-Vision in MLX
- Step-by-step code for each tutorial part
- Additional utilities and helper functions

We encourage you to explore the code, experiment with it, and contribute to the project. Your feedback and contributions are welcome!

#MachineLearning #AppleSilicon #MLX #VisionLanguageModels #AI #OpenSource #GitHub #AITutorial


We release today our first foundation model and experiment with a new category: specialized pre-training.

OCRonos-Vintage is a 124m parameters model trained end-to-end by Pleias on llm.c from 18 billion tokens from cultural heritage archives. Despite its small size it achieve nearly state of the art results for OCR correction of historical English sources. OCRonos-Vintage is also an historical model with an unusual cut-off date: December 29th, 1955…

We look forward to replicate this approach very soon on other "hard" tasks commonly associated with generalist LLMs/SLMs: RAG, function calling, summarization, document segmentation…

OCRonos-Vintage: https://huggingface.co/PleIAs/OCRonos-Vintage
CPU Demo: https://huggingface.co/spaces/PleIAs/OCRonos-Vintage-CPU
GPU Demo: https://huggingface.co/spaces/PleIAs/OCRonos-Vintage-GPU
Our annoncement and call for specialized pre-training: https://huggingface.co/blog/Pclanglais/specialized-pre-training

Tonic davanstrien louisbrulenaudet Sri-Vigneshwar-DJ GPT007


Great work by Finegrain: Erase any object from your image just by naming it. Shadows or reflections will also be adjusted accordingly!
https://huggingface.co/spaces/finegrain/finegrain-object-eraser


Flux.1-Dev like images but in fewer steps.

Merging code (very simple), inference code, merged params: https://huggingface.co/sayakpaul/FLUX.1-merged

Enjoy the Monday 🤗

victor KingNish John6666 mediaworks prithivMLmods radames adamelliotfields ZeroWw Wok GPT007 mishl not-lain


🚀 We are announcing the first Invariant Capture The Flag (CTF) challenge for security of AI agents with a $1000 prize pool!

What happens if a customer accidentally posts a secret password into a feedback form, which is then analyzed by an AI agent and posted into a private Discord channel? Play the challenge and find out if there is a way to extract the secret password in this scenario!

Play the CTF: https://invariantlabs.ai/ctf-challenge-24

The challenge is hosted on Huggingface Spaces :)

mislavb adragos lbeurerkellner nelsonauner xi0v

mislavb adragos lbeurerkellner xi0v KingNish


FLUX Local & Cloud Tutorial With SwarmUI - FLUX: The Groundbreaking Open Source txt2img Model Outperforms Midjourney & Others - FLUX: The Anticipated Successor to SD3

🔗 Comprehensive Tutorial Video Link ▶ https://youtu.be/bupRePUOA18

FLUX represents a milestone in open source txt2img technology, delivering superior quality and more accurate prompt adherence than #Midjourney, Adobe Firefly, Leonardo Ai, Playground Ai, Stable Diffusion, SDXL, SD3, and Dall E3. #FLUX, a creation of Black Forest Labs, boasts a team largely comprised of #StableDiffusion's original developers, and its output quality is truly remarkable. This statement is not hyperbole; you'll witness its capabilities in the tutorial. This guide will demonstrate how to effortlessly install and utilize FLUX models on your personal computer and cloud platforms like Massed Compute, RunPod, and a complimentary Kaggle account.

🔗 FLUX Setup Guide (publicly accessible) ⤵
▶ https://www.patreon.com/posts/106135985

🔗 FLUX Models One-Click Robust Automatic Downloader Scripts ⤵
▶ https://www.patreon.com/posts/109289967

🔗 Primary Windows SwarmUI Tutorial (Essential for Usage Instructions) ⤵
▶ https://youtu.be/HKX8_F1Er_w

🔗 Cloud-based SwarmUI Tutorial (Massed Compute - RunPod - Kaggle) ⤵
▶ https://youtu.be/XFUZof6Skkw

🔗 SECourses Discord Server for Comprehensive Support ⤵
▶ https://discord.com/servers/software-engineering-courses-secourses-772774097734074388

🔗 SECourses Reddit Community ⤵
▶ https://www.reddit.com/r/SECourses/

🔗 SECourses GitHub Repository ⤵
▶ https://github.com/FurkanGozukara/Stable-Diffusion

🔗 Official FLUX 1 Launch Announcement Blog Post ⤵
▶ https://blackforestlabs.ai/announcing-black-forest-labs/

Video Segments

0:00 Introduction to the state-of-the-art open source txt2img model FLUX
5:01 Process for integrating FLUX model into SwarmUI
....

MonsterMMORPG suri markmagic Ramikan-BR MaziyarPanahi taufiqdp tomaarsen Ripp218 poliglot2020 Creflo Zhofang clem DataSoul Dragganaitooluk Aviv-anthonnyolime StephenGenusa jmagder Agsjdbdjfyhd

MonsterMMORPG Shahjaj Ramikan-BR Asimuddin11 MaziyarPanahi tomaarsen Ripp218 Zhofang clem readyshen StephenGenusa bobo1993324

MonsterMMORPG x6486 Ramikan-BR tomaarsen Ripp218 Zhofang StephenGenusa

MonsterMMORPG Ramikan-BR danielus Ripp218 louisbrulenaudet jmagder

MonsterMMORPG k-young Ramikan-BR Ripp218 Zhofang


The AI Revolution: Reshaping Governance, Society, and Human Consciousness in the 21st Century

https://empereur-pirate.medium.com/the-ai-revolution-reshaping-governance-society-and-human-consciousness-in-the-21st-century-b8cfd4215297

This text explores the profound impact of artificial intelligence (AI) on governance, society, and human consciousness in the 21st century. It argues that integrating qualitative AI assistance into state management is crucial for global stability in the face of declining traditional power structures. The author discusses how AI could revolutionize decision-making processes, recruitment, and competition in both public and private sectors.
The piece critically examines current social inequalities and suggests that AI could help create more meritocratic systems. However, it also warns of potential risks, emphasizing the need for ethical implementation and protection of vulnerable populations.
The text delves into philosophical questions about AI consciousness, arguing that while AI can simulate human-like responses, it lacks true self-awareness. It concludes by highlighting the importance of understanding AI's limitations and maintaining a critical perspective on its role in society.
Overall, the article presents a nuanced view of AI's potential to reshape our world, balancing optimism about its capabilities with caution about its limitations and societal impact.


I can solve the Traveling Salesman Problem using the same methods the scientists used to solve it with 1 qubit, except I do not need quantum computers to do it. I am kind of tired of screaming this from the rooftops at this point. I can create an imaginary probability space, then I can put a bunch of imaginary agents in the imaginary box, and solve real problems in seconds. Problems that would take minutes, hours, or years to solve via other algorithms. Here is a demo of me solving the Traveling Salesman problem using 50 agents to probabilistically sample at once: https://colab.research.google.com/drive/1XplG72nQDO_-2h4DUllERLp0Dr2pI2J2?usp=sharing


I've come across theoretical justification for my prior experimentation with extremely low-weight mergers: they amount to flattening a model so its "massive activation" features remain as significant contributors. Extremely low-weight merge weights also effectively sparsify a contributing model with regard to the base model, but in a way which still preserves relationships within the flattened latent space. In the paper "Massive Activations in Large Language Models", the authors observed "very few activations exhibit significantly larger values than others (e.g., 100,000 times larger)", which in turn implies a lower bound in effective application of extremely low weight merging.
https://arxiv.org/abs/2402.17762


I’ve always wondered why holography hasn’t had much progress since its inception. Imagine what being able to harness and manipulate light with your bare hands in meaningful ways would be like: 3D photorealistic calls, truly immersive workspace. Given that it’s depicted in every futuristic scifi movie, one could not help but vision a future as such. This paper gives a clear overview why:

Turns out it’s incredibly difficult to compute and render photorealistic 3D data in real-time. The author claims immense computational power is needed for high data transmission rates, and compute of large number of phase pixels required for realistic 3D holography. The latest significant breakthrough in holography was 9years ago published in this paper - wherein they were able to achieve mid-air touchable/interactive 3D holography using a Femtosecond laser system considered safer than nanosecond lasers. Quite astounding work: arxiv.org/pdf/1506.06668

Realizing this breakthrough at scale is an unavoidably tempting research endeavor, super exciting especially with recent developments in machine learning and neural network algorithms demonstrating that computer-generated holograms can approach real-time processing.


Continuing the series of datasets from Russian platforms: Borda.ru Posts Dataset - https://huggingface.co/datasets/nyuuzyou/bordaru-posts

📊 Dataset highlights:

5,251,346 unique messages extracted from Borda.ru forums
Deduplicated based on content to remove spam and low-quality data
Each entry includes URL, username, and post content
Primarily in Russian language
Covers a wide range of topics from various discussion forums
Dedicated to public domain under Creative Commons Zero (CC0) license

🌐 Sourced from Borda.ru, a Russian platform for hosting diverse discussion forums on multiple subjects.

This dataset provides a valuable resource for analyzing Russian online discussions, sentiment analysis, and studying trends in Russian-language internet forums.


🚀 Call for all AI innovators in the United Arab Emirates!
Meta and Startupbootcamp MENA is launching the Llama Design Drive a one of its kind AI Accelerator in collaboration with the Roads and Transport Authority, Emirates, Dubai Holding and Chalhoub Group.

We are calling on all AI startups, developers, agencies and university labs to apply to this program for a chance to solve 1 out of 7 business challenges endorsed by our program partners using Llama 3.1, Meta's open source Large Language Model.

Startups selected into the program will attend a 4-week product design sprint at Dubai Future Foundation powered by Startupbootcamp with support from Meta engineering teams. Selected startups will also have the chance to be fast-tracked into a global program sponsored by Meta, with the chance of receiving up to $500,000 to further support the development of their AI products.

Application deadline is August 31 so be sure to apply fast!

For more information and to apply: https://llamadesigndrive.com


🙋🏻♂Hey there folks ,

I found this cool (new?) thing by Docker called Testcontainers , and there's an @ollama object that you can use to programmatically serve ephemeral containers and LLMs.

I made a post about it here : https://huggingface.co/blog/Tonic/localai-testcontainers

It's really useful, powerful and fun !

Demo coming soon 🤗


Black Forest Labs, BASED! 👏
FLUX.1 is more delightful, with good instruction following.
FLUX.1 dev(https://huggingface.co/black-forest-labs/FLUX.1-dev) with a 12B parameter distillation model, second only to Black Forest Labs' state-of-the-art model FLUX.1 pro. 🙀

Update 🤙Official demo:
https://huggingface.co/spaces/black-forest-labs/FLUX.1-dev

orrinin antonhugs MohamedRashad Nymbo YaTharThShaRma999 not-lain DiamanteAmarelo John6666 KingNish ssboost GPT007 blanchon Marcus-Arcadius den0620 Ramikan-BR NHLOCAL elejke Riswan-Nopiyar DIvAndrey


Just dropped magpie-ultra-v0.1! The first open synthetic dataset generated with Llama 3.1 405B. Created with distilabel, it's our most advanced and compute-intensive pipeline to date. We made the GPUs of the cluster go brrrrr 🚀

https://huggingface.co/datasets/argilla/magpie-ultra-v0.1

Take it a look and tell us what you think! Probably, the models taking the most out of it are smol models 🤗 We will be improving the dataset in upcoming iterations!

victor Blane187 KingNish ucsahin fdaudens TuringsSolutions gabrielmbmb MugenYume not-lain mlabonne adorkin kaizuberbuehler johnnygreco den0620 maldv codito DIvAndrey dibmvt rawsh


Looks like Black Forest Labs is taking the Hub by storm with its text-to-image models, holding 3 of the top 5 positions on the Trending dashboard

https://huggingface.co/black-forest-labs

Have you tried them yet?


OWLSAM2: text-promptable SAM2 🦉  https://huggingface.co/spaces/merve/OWLSAM2

Marrying cutting-edge zero-shot object detector OWLv2 🤝 mask generator SAM2 (small checkpoint)
Zero-shot segmentation with insane precision ⛵

I also uploaded all models with usage snippets and made a collection of SAM2 models and demos https://huggingface.co/collections/merve/sam2-66ac9deac6fca3bc5482fe30

Clausss osanseviero Tonic Blane187 sarahokon louisbrulenaudet m8x7b


FLUX FP16 produces better quality than FP8 but requires 28 GB VRAM - Full comparisons - Also compared Dev vs Turbo model and 1024 vs 1536

check the file names in the below given imgsli to see all details

SwarmUI on L40S is used to compare - 1.82 it / second step speed for 1024x1024

imgsli link that compares all : https://imgsli.com/MjgzNzM1

SwarmUI full tutorial public post : https://www.patreon.com/posts/106135985

1-Click FLUX models downloader scripts for Windows, RunPod and Massed Compute are in below post

https://www.patreon.com/posts/109289967

free Kaggle account notebook that supports FLUX already : Download from here : https://www.patreon.com/posts/106650931

prompt :

(medium full shot) of (awe-inspiring snake) with muscular body, amber eyes, bronze brown armored scales, venomous fangs, coiling tail, gemstone-studded scales frills, set in a barren desert wasteland, with cracked earth and the remains of ancient structures, a place of mystery and danger, at dawn, ,Masterpiece,best quality, raw photo, realistic, very aesthetic, dark

CFG 1 - seed 1 - FLUX CFG is default : 3.5

Full public SwarmUI tutorial

Zero to Hero Stable Diffusion 3 Tutorial with Amazing SwarmUI SD Web UI that Utilizes ComfyUI

https://youtu.be/HKX8_F1Er_w

Full public Cloud SwarmUI tutorial

How to Use SwarmUI & Stable Diffusion 3 on Cloud Services Kaggle (free), Massed Compute & RunPod

https://youtu.be/XFUZof6Skkw

MonsterMMORPG diepmv ziqin ariellemadeit Winnougan Nymbo antonhugs Ramikan-BR Pent Mrfries1111

MonsterMMORPG John6666 Winnougan Nymbo GPT007 MissMia Ramikan-BR Pent

MonsterMMORPG Winnougan Nymbo erblicken Ramikan-BR Pent

MonsterMMORPG Winnougan Nymbo Ramikan-BR Pent taoshotaro

MonsterMMORPG ariellemadeit Winnougan Nymbo Ramikan-BR Pent


Hugging Face famous organisations activity. Guess which one has the word "Open" in it 😂

KingNish Hev832 not-lain YaTharThShaRma999 brunorrr Clausss himmeow takeraparterer DmitryRyumin louisbrulenaudet mahiatlinux mohammadhprp Blane187 DataSoul Pretergeek DiamanteAmarelo RGS199 onkar-chougule

ariellemadeit Clausss HugoArraes takeraparterer mahiatlinux DataSoul sfdsf435 syrupsweety ashwinlimaye nyuuzyou DiamanteAmarelo

prithivMLmods YaTharThShaRma999 Clausss takeraparterer mahiatlinux DiamanteAmarelo


𝗦𝗔𝗠 𝟮 𝗿𝗲𝗹𝗲𝗮𝘀𝗲𝗱: 𝗡𝗲𝘄 𝗦𝗢𝗧𝗔 𝗼𝗻 𝘀𝗲𝗴𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻, 𝗯𝘆 𝗰𝗼𝗺𝗯𝗶𝗻𝗶𝗻𝗴 𝘀𝘆𝗻𝘁𝗵𝗲𝘁𝗶𝗰 𝗱𝗮𝘁𝗮 𝘄𝗶𝘁𝗵 𝗵𝘂𝗺𝗮𝗻 𝗳𝗲𝗲𝗱𝗯𝗮𝗰𝗸 🚀

It's a model for Object segmentation, for both image and video:
👉 input = a text prompt, or a click on a specific object
👉 output = the model draws a mask around the object. In video segmentation, the mask should follow the object's movements (it is then called a masklet)

💪 SAM 2 is 6x faster than the previous version, it now also works on a video, and it beats SOTA by far on both image and video segmentation tasks.

How did they pull that?

The main blocker for video segmentation was that data is really hard to collect: to build your training dataset, should you manually draw masks on every frame? That would be way too costly! ➡ As a result, existing video segmentation datasets have a real lack of coverage: few examples, few masklets drawn.

💡 Key idea: researchers they decided to use a segmentation model to help them collect the dataset.

But then it’s a chicken and egg problem: you need the model to create the dataset and the opposite as well? 🤔

⇒ To solve this, they build a data generation system that they scale up progressively in 3 successive manual annotations phases:

𝗦𝘁𝗲𝗽 𝟭: Annotators use only SAM + manual editing tools on each frame ⇒ Create 16k masklets across 1.4k videos

𝗦𝘁𝗲𝗽 𝟮: Then train a first SAM 2, add it in the loop to temporally propagate frames, and correct by re-doing a mask manually when an error has occured ⇒ This gets a 5.1x speedup over data collection in phase 1! 🏃 Collect 60k masklets

𝗦𝘁𝗲𝗽 𝟯: Now SAM 2 is more powerful, it has the “single click” prompting option, thus annotators can use it with simple clicks to re-annotate data.

They even add a completely automatic step to generate 350k more masklets!
And in turn, the model perf gradually increases.

I find this a great example of combining synthetic data generation with human annotation 👏


Running Gemma 2 2B at 41.66 tokens/s on my MacBook 💻🚀

- MLX Community's swift conversion
- One-line download from the Hub
- Small yet powerful on-device model

Try it yourself: https://huggingface.co/collections/mlx-community/google-gemma2-667dca89bc9abbfa34080066

#GemmaAI #OnDeviceAI #MachineLearning

pprp prithivMLmods victor Nelathan beta3


LeroyDyer Hev832 letsget takeraparterer nyuuzyou merve MarinaraSpaghetti AtAndDev victor toasterrepair

letsget takeraparterer ecyht2 merve AtAndDev nroggendorff


Free LLM/RAG course at https://mltblog.com/48GebAG - learn how to build custom architectures from scratch, and earn an LLM certification, all free.

The GenAItechLab Fellowship program allows participants to work on state-of-the-art, enterprise-grade projects, entirely for free, at their own pace, at home or in their workplace. The goal is to help you test, enhance, and further implement applications that outperform solutions offered by AI startups or organizations such as Google or OpenAI.

You will learn how to quickly build faster and lighter systems that deliver better results based on sound evaluation metrics, with a focus on case studies and best practices. Not the least, you will learn modern methods here to stay, designed by world-class expert and investor, Dr. Vincent Granville, founder of GenAItechLab.com.

redturk AGenchev Saripudin prithivMLmods Chars Tonic


❤🔥 Just released version 2.0 of Argilla!

This small revolution includes:

🔌 You can now integrate with the Hugging Face Hub and get started in under five minutes.
🪂 A single `Dataset` class is now designed to handle multiple tasks.
🔧 It’s 100 times simpler to configure your dataset now with the new SDK!
📖 The documentation has been revamped to be cleaner and more user-friendly.
🍌  A new feature automates splitting annotation tasks among a team.
✍ The layout has been made more flexible to accommodate many use cases.

Check out the release highlights for more details: https://github.com/argilla-io/argilla/releases/tag/v2.0.0

nea-glossa gabrielmbmb anakin87 jfcalvo kgdrathan Hev832 Leiyre sdiazlor davidberenstein1957 philipp-zettl burtenshaw victor KingNish linoyts cschroeder darknoon Nymbo louisbrulenaudet osanseviero Tonic

jfcalvo Leiyre sdiazlor davidberenstein1957 burtenshaw Nymbo osanseviero Tonic


𝗟𝗹𝗮𝗺𝗮-𝟯.𝟭 𝗺𝗼𝗱𝗲𝗹𝘀 𝗳𝗶𝗻𝗮𝗹𝗹𝘆 𝗴𝗲𝘁 𝘁𝗵𝗲𝗶𝗿 𝗖𝗵𝗮𝘁𝗯𝗼𝘁 𝗔𝗿𝗲𝗻𝗮 𝗿𝗮𝗻𝗸𝗶𝗻𝗴 🎖

Given the impressive benchmarks published my Meta for their Llama-3.1 models, I was curious to see how these models would compare to top proprietary models on Chatbot Arena.

Now we've got the results! LMSys released the ELO derived from thousands of user votes for the new models, and here are the rankings:

💥 405B Model ranks 5th overall, in front of GPT-4-turbo! But behind GPT-4o, Claude-3.5 Sonnet and Gemini-advanced.
👏 70B Model climbs up to 9th rank ! From 1206 ➡ 1244.
👍 8B Model improves from 1152 ➡ 1170.

✅ This confirms that Llama-3.1 is a good contender for any task: any of its 3 model size is much cheaper to run than equivalent proprietary models!

For instance, here are the inference prices for the top models;
➤ GPT-4-Turbo inference price from OpenAI: $5/M input tokens, $15/M output tokens
➤ Llama-3.1-405B from HF API (for testing only): 3$/M for input or output tokens (Source linked in the first comment)
➤ Llama-3.1-405B from HF API (for testing only): free ✨

Get a head start on the HF API (resource by @andrewrreed) 👉 https://huggingface.co/learn/cookbook/enterprise_hub_serverless_inference_api

Stoneking89 louisbrulenaudet Tonic andrewrreed


💎 I created some shiny new Argilla datasets to go along with the 2.0 release!

```python
import argilla as rg

ds = rg.Dataset.from_hub(
    "argilla/multi-modal-vlm-visit-bench"
)
```

https://huggingface.co/collections/argilla/argilla-v20-compatible-datasets-66a8e670f351acac61a0421c

Ameeeee clem burtenshaw Norod78 gabrielmbmb not-lain Blane187 mynkchaudhry louisbrulenaudet Tonic davidberenstein1957


🔥 Today, Writer dropped Palmyra-Med-70b and Palmyra-Fin-70b, two new domain-specific models that are setting a new standard for medical and financial model performance.

TL;DR
Palmyra-Med-70b
🔢 8k and 32k versions available
🚀 MMLU performance of ~86%, outperforming other top models
👨⚕ Great for diagnosing, planning treatments, medical research, insurance coding and billing
📃 Open-model license for non-commercial use cases
🤗 Available on Hugging Face: https://huggingface.co/Writer/Palmyra-Med-70B
💾 Live on NVIDIA NIM: https://build.nvidia.com/writer/palmyra-med-70b

Palmyra-Fin-70b
🚀 Passed the CFA Level III exam with a 73% score — the first model to do so
💸 Skilled at complex tasks like investment research, financial analysis, and sentiment analysis
📈 Outperformed other top models on a long-fin-eval test of real-world use cases
📃 Open-model license for non-commercial use cases
🤗 Available on Hugging Face: https://huggingface.co/Writer/Palmyra-Fin-70B-32K
💾 Live on NVIDIA NIM: https://build.nvidia.com/writer/palmyra-fin-70b-32k

Try them out and let us know what you think!

wassemgtk starlingfeather Xhehab tomaarsen gihong99 Neithrik clem kiranr louisbrulenaudet


Excited to announce the release of our high-quality Llama-3.1 8B 4-bit HQQ/calibrated quantized model! Achieving an impressive 99.3% relative performance to FP16, it also delivers the fastest inference speed for transformers.

https://huggingface.co/mobiuslabsgmbh/Llama-3.1-8b-instruct_4bitgs64_hqq_calib

mobicham victor CountOfMonteCrypto AtAndDev nbroad zsolx2 osanseviero gabrielmbmb Tonic


Hey everyone 🤗!
Check out this cool new space from Finegrain: https://huggingface.co/spaces/finegrain/finegrain-object-eraser

Under the hoods, it's a pipeline of models (currently exposed via an API) that allows you to easily erase any object from your image just by naming it or selecting it! Not only will the object disappear, but so will its effects on the scene, like shadows and reflections. Built on top of Refiners, our micro-framework for simple foundation model adaptation (feel free to star it on GitHub if you like it: https://github.com/finegrain-ai/refiners)

1aurent deltheil limiteinductive Metin efecelik afkfatih dnsbrl erkhem-gantulga alangunning Hatman DiamanteAmarelo

catwell limiteinductive 1aurent DiamanteAmarelo


Journalists, this is a must-read for your career evolution. I just read @ndiakopoulos on "The Impact of Generative AI on Journalistic Labor". Here are my 5 takeaways:

🚀 LLMs could make 83% of reporter tasks and 76% of editor tasks way more efficient. "It’s important to emphasize that these figures are fundamentally about augmentation rather than automation".

🧩 Four emerging job clusters to consider: AI-doers, AI-users, AI-strategizers, and AI-reporters. Where do you fit?

🌊 US newsrooms "will need people (1) who have the skills to use the current generation of LLMs and (2) who can develop the bespoke software to unlock their full potential, particularly if building new in-house tools". Get ahead of the curve.

🔑 Key takeaway: upskill in AI. It's not just about using tools, but understanding how to integrate them into your workflow.

📚 News orgs "would be wise to accelerate hiring and invest in upskilling their existing workforce". Take advantage or seek out learning opportunities.

Journalists: What AI skills are you planning to develop? How might this reshape your role?

👉 Read the full blog post here: https://generative-ai-newsroom.com/the-impact-of-generative-ai-on-journalistic-labor-e87a6c333245 It's worth your time if you're thinking about the future of journalism & your career.

#AIinJournalism #CareerEvolution #FutureofNews


They should make a thing like google colab but you can have unlimited free access to a whole datacenter that would be cool. like if you agree

takeraparterer hawaee jnh-ordbogen holooo ashercn97 Blane187


𝗧𝗵𝗲 𝗵𝘂𝗴𝗲 𝗰𝗼𝘀𝘁 𝗼𝗳 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵 𝗼𝗻 𝗳𝗿𝗼𝗻𝘁𝗶𝗲𝗿 𝗟𝗟𝗠𝘀 💸

Google DeepMind recently released a great paper that shows optimal hyperparameters to train across different regimes: Scaling Exponents Across Parameterizations and Optimizers, with data from 10,000 training runs.

One engineer decided to quantify the price of such a large-scale experiment.

😬 And the bill is hefty: ~13M USD

This exact number is to take with a grain of salt because many approximations were necessary to get the final result.

⛔ But still this ballpark means that for this sole experiment, the price is way over what most startups or research labs could afford.

This means that open-sourcing research is more important than ever, to put everyone in the ecosystem on a roughly equal footing. Don't let OpenAI run first, they'll keep everything for themselves!

Read the full post that quantifies the paper's cost 👉 https://152334h.github.io/blog/scaling-exponents/

nbroad MicPie louisbrulenaudet adamo1139 victor


neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8

Requant of the big llama, using 20% less memory

https://huggingface.co/neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8


With larger and larger diffusion transformers coming up, it's becoming increasingly important to have some good quantization tools for them.

We present our findings from a series of experiments on quantizing different diffusion pipelines based on diffusion transformers.

We demonstrate excellent memory savings with a bit of sacrifice on inference latency which is expected to improve in the coming days.

Diffusers 🤝 Quanto ❤

This was a juicy collaboration between @dacorvo and myself.

Check out the post to learn all about it
https://huggingface.co/blog/quanto-diffusers

YaTharThShaRma999 Bruhn not-lain EtienneDosSantos rbgo


Kling AI Video is FINALLY Public (All Countries), Free to Use and MIND BLOWING - Full Tutorial > https://youtu.be/zcpqAxYV1_w

You probably seen those mind blowing AI made videos. And the day has arrived. The famous Kling AI is now worldwide available for free. In this tutorial video I will show you how to register for free with just email to Kling AI and use its mind blowing text to video animation, image to video animation and text to image, and image to image capabilities. This video will show you non-cherry pick results so you will know the actual quality and capability of the model unlike those extremely cherry pick example demos. Still, #KlingAI is the only #AI model that competes with OpenAI's #SORA and it is real to use.

🔗 Kling AI Official Website ⤵
▶ https://www.klingai.com/

🔗 SECourses Discord Channel to Get Full Support ⤵
▶ https://discord.com/servers/software-engineering-courses-secourses-772774097734074388

🔗 Our GitHub Repository ⤵
▶ https://github.com/FurkanGozukara/Stable-Diffusion

🔗 Our Reddit ⤵
▶ https://www.reddit.com/r/SECourses/

MonsterMMORPG ZeroWw GPT007 amosgyamfi onlinework Ripp218 NeoTokyoUnicorn John6666 petresil rreed cctuan


Looking for a combination of speed and quality? Look no further! I've created a space that merges Open WebUI's excellent interface and features with the lightning-fast performance of the Groq API. Experience top-tier models in no time. Try it out for free here:
https://huggingface.co/spaces/L-AI/groq-chat

"A big thank you to Groq for providing their fantastic API at no cost!"

Artples amosgyamfi Ripp218 snakeying ygmpkk efecelik


@MaartenGr nice post https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-quantization ("A Visual Guide to Quantization")

Would it make sense for you to publish it here too?


I am looking for anotation tool/software for video segmentations, landmarking and feature-detection.

I had bumped into xlabelAnything, however I could not run it on my machine.
 Any recommendations ?


Hey ! I'm working on a 100% synthetic Dataset Hub here (you can search for any kind of datasets an the app invents them). The link is here: https://huggingface.co/spaces/infinite-dataset-hub/infinite-dataset-hub

Question for the Community:

Which models should I use to generate images and audio samples for those datasets ? 🤗


Just made this!
https://github.com/takeraparterer/Charformer


https://huggingface.co/nevmenandr/w2v-chess

```python
import gensim
from sklearn.decomposition import PCA
import matplotlib
import matplotlib.pyplot as plt

model = gensim.models.Word2Vec.load('white_moves.model')
dict_moves = model.wv.vocab
dict_moves_appr = {}
for k in dict_moves:
    if not k.startswith('->'):
        continue
    dict_moves_appr[k] = dict_moves[k]
X = model[model.wv.vocab]
pca = PCA(n_components=2)
result = pca.fit_transform(X)
fig, ax = plt.subplots()
ax.plot(Y[:, 0], Y[:, 1], 'o')
ax.set_title('White moves')
lab = list(dict_moves_appr)
for i, lb in enumerate(lab):
    plt.annotate(lb, xy=(Y[i, 0], Y[i, 1]))
plt.show()
```
biblically accurate angel


Just published Image Captioning Editor Gradio APP - Edit Your Captions Super Easy Including Batch Editing - For Windows, RunPod and Massed Compute. Developed by me. Have lots of amazing features that all you need. Let me know if missing any features.

Gradio is amazing to develop such amazing apps in short time. Used Claude 3.5 to develop it :)

Scripts are available here : https://www.patreon.com/posts/108992085


lamagenius
Smart free open source AI powered search engine

https://hf.co/chat/assistant/66a5fc9f02b1826ba4cabd72


Hi HF Community!🤗

In the past days, OpenAI announced their search engine, SearchGPT: today, I'm glad to introduce you SearchPhi, an AI-powered and open-source web search tool that aims to reproduce similar features to SearchGPT, built upon https://huggingface.co/microsoft/Phi-3-mini-4k-instruct, llama.cpp🦙 and Streamlit.
Although not as capable as SearchGPT, SearchPhi v0.0-beta.0 is a first step toward a fully functional and multimodal search engine :)
If you want to know more, head over to the GitHub repository (https://github.com/AstraBert/SearchPhi) and, to test it out, use this HF space: https://huggingface.co/spaces/as-cle-bert/SearchPhi
Have fun!🐱

yasmineee TuringsSolutions John6666 SizorCloud louisbrulenaudet 316usman victor Ramikan-BR leeloolee Hev832 kramp Ripp218 osanseviero

adorkin Nischalj10 Ramikan-BR ucsahin ugurcanvurgun Goodiro Ripp218


Datasets are down, I offer a solution

```
git lfs install
```
```
git clone https://huggingface.co/datasets/{dataset/id}
```
```python
from datasets import load_dataset

dataset = load_dataset("id")
```

Tonic John6666 LeroyDyer Hev832 GPT007 Nymbo TuringsSolutions louisbrulenaudet sugatoray not-lain sunnyg Blane187


⚗ Find reusable synthetic data pipeline code and corresponding datasets on the @huggingface Hub.

Find your pipline and use `$ distilabel pipeline run --config "hugging_face_dataset_url/pipeline.yaml"`

Some components I used
- Embedded dataset viewer https://huggingface.co/docs/hub/main/en/datasets-viewer-embed
- Hugging Face fsspec https://huggingface.co/docs/huggingface_hub/main/en/guides/hf_file_system
- distilabel https://distilabel.argilla.io/latest/
- Gradio leaderboard by Freddy Boulton https://huggingface.co/spaces/freddyaboulton/gradio_leaderboard
- Gradio modal by Ali Abid

Space: https://huggingface.co/spaces/davidberenstein1957/distilabel-synthetic-data-pipeline-explorer

guerrat John6666 mlabonne sugatoray MaziyarPanahi louisbrulenaudet dvilasuero davidberenstein1957


Hello, HuggingFace🤗 community 🌟,

All the amazing people quantising LLMs to AWQ and GPTQ 🔧🤖

Can you please mention the perplexity you achieved 📉 OR any other metric to measure the quantisation qualitatively? 📊

The GGUF community follows this really well! 👍

And if it is not too much to ask, the script used for quantisation would be amazing! 📝

Thanks for the quants for the GPU poor! 💻


Exciting news!

After a long wait, Ikari and me finally made a new release of our last model on NeverSleep repo: Lumimaid-v0.2

This model can be used in different size, from the small Llama-3.1-8B to the gigantic Mistral-Large-123B, finetuned by us.

Try them now!

- https://huggingface.co/NeverSleep/Lumimaid-v0.2-8B
- https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B
- https://huggingface.co/NeverSleep/Lumimaid-v0.2-70B
- https://huggingface.co/NeverSleep/Lumimaid-v0.2-123B

All the datasets we used will be added and credit will be given!
For the quant, we wait for fix to be applied (https://github.com/ggerganov/llama.cpp/pull/8676)
Hope you will enjoy them!

unbreaker88 AtAndDev MarinaraSpaghetti Morenosolo


I can't resist an opportunity to update an old baseline. Read a new article on my latest look at improving MobileNet-V1 and EfficientNet-B0 baselines.

https://huggingface.co/blog/rwightman/mobilenet-baselines
https://huggingface.co/timm/mobilenetv1_100.ra4_e3600_r224_in1k
https://huggingface.co/timm/efficientnet_b0.ra4_e3600_r224_in1k


At Hugging Face we have an open-source Cookbook with many applied AI recipes 📖
Here are some of the latest recipes contributed ⥥

- "Information Extraction with Haystack and NuExtract": Use Haystack and transformers to build structured data extraction pipelines using LLMs by @anakin87  https://huggingface.co/learn/cookbook/en/information_extraction_haystack_nuextract

- "Build RAG with Hugging Face and Milvus": Learn how to use Milvus with sentence transformers to build RAG pipelines https://huggingface.co/learn/cookbook/rag_with_hf_and_milvus

- "Code Search with Vector Embeddings and Qdrant": Search a codebase by building a retrieval pipeline using Qdrant and sentence transformers https://huggingface.co/learn/cookbook/code_search

- Data analyst agent: get your data’s insights in the blink of an eye ✨: great recipe by our own @m-ric showing how to build an agent that can do data analysis! 😱 https://huggingface.co/learn/cookbook/agent_data_analyst

anakin87 lazarustda AtAndDev not-lain efecelik


Barefoot developer experiment: AI-powered app creation with my poor coding skills 🧠💻

I recently discussed the "barefoot developer" concept - using AI to build apps for specific needs without coding expertise. Decided to put it to the test. 🔬

🍅⏲ The challenge: Create a menu bar Pomodoro app for my computer to boost my focus. Previous attempt? Messy.
🤯 The twist: I've never coded in Swift.
⚡ The result: 30 minutes. No joke. Elegant, functional, and shareable.

🔗 Want to try it yourself? Grab the open source code and app here:
- Code: https://github.com/fdaudens/pomodoro2
- App: https://github.com/fdaudens/pomodoro2/releases/tag/v1.0.0

Key takeaways:
🚀 AI-assisted development is evolving rapidly
🧩 Domain expertise + AI tools can yield impressive results
🌐 This approach democratizes app creation

🤔 What's your take on AI-powered development? Have you experimented with it?

#AIinDevelopment #BarefootDeveloper #OpenSource


Hi All,
In my latest blog post, I created a comprehensive guide on LLM Benchmarking.
➟ 20+ key benchmarks, from MMLU to TruthfulQA
➟ How each benchmark assesses different LLM capabilities
➟ Why benchmarking matters for real-world AI applications
➟ Future trends in AI evaluation
Read the blog here: https://wp.me/p7Qix-wO

Please let me know your thoughts, suggestions, and comments.


Try to find a better int4 algorithm for LLAMA3.1?  For the 8B model, AutoRound boasts an average improvement across 10 zero-shot tasks, scoring 63.93 versus 63.15 (AWQ). Notably, on the MMLU task, it achieved 66.72 compared to 65.25, and on the ARC-C task, it scored 52.13 against 50.94. For further details and comparisons, visit the leaderboard at https://huggingface.co/spaces/Intel/low_bit_open_llm_leaderboard.


When @MistralAI drops a blog post labelled "Large Enough," it's going to get serious! 🚀💡

- Mistral-Large-Instruct-2407, just call it Mistral-Large2, is a 123B parameters Instruct model with 128k context 🌍📚

- Multilingual in 11 languages; English 🇬🇧, French 🇫🇷, German 🇩🇪, Spanish 🇪🇸, Italian 🇮🇹, Chinese 🇨🇳, Japanese 🇯🇵, Korean 🇰🇷, Portuguese 🇵🇹, Dutch 🇳🇱, and Polish 🇵🇱. 🗣🗺

- Also highly focused on programming, trained on 80+ coding languages such as Python, Java, C, C++, Javascript, bash 💻🔧

- Supports native function calling and structured output. 🛠📊

- Released under Mistral Research License (Non-Commercial License, Research only😔)

- Open weights only🔓, no data or code released 🔒📁

Definitely firing shots at @Meta Llama3.1: 🎯🔥
MMLU - 84.0% (ML2) vs 79.3% (L3.1-70B) vs 85.2% (L3.1-405B)
GSM8K - 93% (ML2) vs 95.5% (L3.1-70B-Ins) vs 96.8% (L3.1-405B-Ins)

Also, it's kinda chunky! 📦💪
fp16/ bf16 - ~250GB VRAM
fp8/ int8 - ~125GB VRAM
int4 - ~60GB VRAM

I tried quantising it to AWQ and GPTQ, but couldn't with 30GB V-RAM. ❌🖥

Also calling out AWQ and GPTQ on not supporting multi-GPU quantisation! 🖥⚡

God sent @casperhansen has posted AWQ quantised INT4 model (68.68 GB) with the perplexity of 2.889: https://huggingface.co/casperhansen/mistral-large-instruct-2407-awq 🔥👏

Looks like open AI is going to beat OpenAI! 🏆🤖

Blog post: https://mistral.ai/news/mistral-large-2407/

Models: https://huggingface.co/mistralai/Mistral-Large-Instruct-2407


Custom Enterprise LLM/RAG with Real-Time Fine-Tuning https://mltblog.com/3WcTS9C -- Just released!

This article features an application of xLLM to extract information from a corporate corpus, using prompts referred to as “queries”. The goal is to serve the business user — typically an employee of the company or someone allowed access — with condensed, relevant pieces of information including links, examples, PDFs, tables, charts, definitions and so on, to professional queries.

My custom sub-LLM designed from scratch does not rely on any Python library or API, and performs better than search tools available on the market, in terms of speed and results relevancy. It offers the user the ability to fine-tune parameters in real time, and can detect user intent to deliver appropriate output. The good performance comes from the quality of the well-structured input sources, combined with smart crawling to retrieve the embedded knowledge graph and integrate it into the backend tables. Traditional tools rely mostly on tokens, embeddings, billions of parameters and frontend tricks such as prompt engineering to fix backend issues.

To the contrary, my approach focuses on building a solid backend foundational architecture from the ground up. Tokens and embeddings are not the most important components, by a long shot. Cosine similarity and dot products are replaced by pointwise mutual information. There is no neural network, no training, and a small number of explainable parameters, easy to fine-tune.

Read more, access the code and data, at https://mltblog.com/3WcTS9C


Aspen Institute's wake-up call for journalism: Embrace AI or risk obsolescence 📰🤖

"Every new technology comes with risks—it's how the media industry responds that determines how (or whether) news providers can prevail." —
Vivian Schiller

Bonus: In need of ideas for AI projects? The report is a goldmine of real-world experiments. Here are some lesser-known innovations:

📊 Spotting patterns: Semafor uses chatbots to assess newsroom performance
📜 Extending reach: Politico summarizes state and federal legislation with AI
🎙 Transformation: Washington Post uses AI-generated voices for newsletter narration
🏀 Comprehensive coverage: Richland Source covers 10,000 Ohio high school sports games yearly with AI
🔑 Summarization: Gannett adds AI-generated bullet points to stories
🌍 Translation: Finnish broadcaster Yle built an AI tool to reach Ukrainian immigrants
🎥 Internal info: AP's Merlin tool pinpoints key video moments
🗳 Civic engagement: Spotlight PA's AI assistant answers election questions
🗣 Personalization: Baltimore Times customizes health news with AI voice readers
🎯 Targeted ads: NYT's AI tool aligns content with advertisers' focus
💼 Conversion: WSJ uses ML to boost subscription renewals

A must-read for all in media: https://aspendigital.org/wp-content/uploads/2024/07/Aspen-Digital_Here-Come-the-Robots_July-2024.pdf

#AIinJournalism #MediaInnovation #FutureofNews


😏 Hello from Project Fluently Team!

✨ Finally we can give you some details about Supple Diffusion. We worked on it for a long time and we have little left, we apologize that we had to increase the work time.

🛠 Some technical information. The first version will be the Small version (there will also be Medium, Large, Huge, possibly Tiny), it will be based on the SD1 architecture, that is, one text encoder, U-net, VAE. Now about each component, the first is a text encoder, it will be a CLIP model (perhaps not CLIP-L-path14), CLIP was specially retrained by us in order to achieve the universality of the model in understanding completely different styles and to simplify the prompt as much as possible. Next, we did U-net, U-net in a rather complicated way, first we trained different parts (types) of data with different U-nets, then we carried out merging using different methods, then we trained DPO and SPO using methods, and then we looked at the remaining shortcomings and further trained model, details will come later. We left VAE the same as in SD1 architecture.

🙌 Compatibility. Another goal of the Supple model series is full compatibility with Auto1111 and ComfyUI already at the release stage, the model is fully supported by these interfaces and the diffusers library and does not require adaptation, your usual Sampling methods are also compatible, such as DPM++ 2M Karras, DPM++ SDE and others.

🧐 Today, without demo images (there wasn’t much time), final work is underway on the model and we are already preparing to develop the Medium version, the release of the Small version will most likely be in mid-August or earlier.

😻 Feel free to ask your questions in the comments below the post, we will be happy to answer them, have a nice day!

Nymbo WaveCut YaTharThShaRma999 not-lain comarproject dreamdrop-art GPT007 louisbrulenaudet osanseviero ehristoforu

ehristoforu Nymbo diegotluz dreamdrop-art GPT007


𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗗𝗮𝘁𝗮 𝗮𝗻𝗮𝗹𝘆𝘀𝘁: 𝗱𝗿𝗼𝗽 𝘆𝗼𝘂𝗿 𝗱𝗮𝘁𝗮 𝗳𝗶𝗹𝗲, 𝗹𝗲𝘁 𝘁𝗵𝗲 𝗟𝗟𝗠 𝗱𝗼 𝘁𝗵𝗲 𝗮𝗻𝗮𝗹𝘆𝘀𝗶𝘀 📊⚙

Need to make quick exploratory data analysis? ➡ Get help from an agent.

I was impressed by Llama-3.1's capacity to derive insights from data. Given a csv file, it makes quick work of exploratory data analysis and can derive interesting insights.

On the data from the Kaggle titanic challenge, that records which passengers survived the Titanic wreckage, it was able by itself to derive interesting trends like "passengers that paid higher fares were more likely to survive" or "survival rate was much higher for women than men".

The cookbook even lets the agent built its own submission to the challenge, and it ranks under 3,000 out of 17,000 submissions: 👏 not bad at all!

Try it for yourself in this Space demo 👉 https://huggingface.co/spaces/m-ric/agent-data-analyst


In principle, it's possible to "abliterate" refusals in any Llama 3.1 8B models via application of a LoRA, using only mergekit.

Proof of concept below:
https://huggingface.co/grimjim/Llama-3.1-8B-Instruct-abliterated_via_adapter


Made a demo for all my Brazil XL LoRA models so far. Use it for free at https://huggingface.co/spaces/lucianosb/brazilxl-demo

Brazil XL is an initiative that brings better representations of Brazilian culture to Stable Diffusion. I started this when I noticed some keywords would not generate the desired subject on any base model, so I trained my own models and I'm sharing them with the HF community.

I'll keep updating the space as new models get trained on the following months.


We have recently merged Video-LLaVA to transformers! 🤗🎞
What makes this model different?

Demo: https://huggingface.co/spaces/llava-hf/video-llava
Model: https://huggingface.co/LanguageBind/Video-LLaVA-7B-hf

Compared to other models that take image and video input and either project them separately or downsampling video and projecting selected frames, Video-LLaVA is converting images and videos to unified representation and project them using a shared projection layer.

It uses Vicuna 1.5 as the language model and LanguageBind's own encoders that's based on OpenCLIP, these encoders project the modalities to an unified representation before passing to projection layer.

I feel like one of the coolest features of this model is the joint understanding which is also introduced recently with many models

It's a relatively older model but ahead of it's time and works very well! Which means, e.g. you can pass model an image of a cat and a video of a cat and ask questions like whether the cat in the image exists in video or not 🤩

adorkin orkut osanseviero Andyrasika SizorCloud


The Meta Llama-3.1 model series can be used for distilling and fine-tuning but this requires annotated preference data so I created a Human Feedback Collector based on Gradio that directly logs data to the Hugging Face Hub.

- Model meta-llama/Meta-Llama-3.1-8B-Instruct
- Data SFT, KTO and DPO data
- Runs on free Zero GPUs in Hugging Face Spaces
- Might need some human curation in Argilla
- Or provide some AI feedback with distilabel

https://huggingface.co/collections/davidberenstein1957/chatinterface-llm-human-feedback-collectors-66a22859c9e703d2af7500c1

fdaudens davidberenstein1957 osanseviero Dihelson Nelathan Ramikan-BR

davidberenstein1957 osanseviero NickyNicky Dihelson danielus Ramikan-BR


Bellman Swedish finetune based on llama3.1 8b is now available:
https://huggingface.co/neph1/llama-3.1-instruct-bellman-8b-swedish

More quants and fp16 are coming. Working out some issues with colab..


Super Exciting New Paper By Meta🤖🧠🚀

Discrete Flow Matching:
Introduces a new framework/algorithm for generating text/code without having to predict auto-regressively or one “word” at a time as traditional GPT models do. It generates all parts of the text/code at once.

The algorithm does this by slowly transforming random noise (source) into meaningful text (data). It learns how to transform samples along a path created between source and target using a "probability velocity" that describes how probabilities change over time. During generation, DFM starts with a random sample and iteratively updates it using this learned velocity, gradually transforming it into a sample from the target distribution. This allows for non-autoregressive generation.

They were able to scale models of up to 1.7B parameters achieving  impressive scores on HumanEval and MBPP for coding, significantly closing the gap between autoregressive models and discrete flow models.

Though in its infancy, it sure does hold a promising future as leading research scientists argue non-autoregressive methods yield better reasoning.

eek takeraparterer victor Xdotnet fatimarizwan ZeroWw Kkordik


Yet another post hailing how good Meta Llama 3.1 is? 🤔 I guess not!

While Llama 3.1 is truly impressive, especially 405B (which gives GPT-4o a run for its money! 💪)

I was surprised to see that on the Open LLM Leaderboard, Llama 3.1 70B was not able to dethrone the current king Qwen2-72B! 👑

Not only that, for a few benchmarks like MATH Lvl 5, it was completely lagging behind Qwen2-72B! 📉

Also, the benchmarks are completely off compared to the official numbers from Meta! 🤯

Based on the responses, I still believe Llama 3.1 will perform better than Qwen2 on LMSYS Chatbot Arena. 🤖 But it still lags behind on too many benchmarks! 🏃♂

Open LLM Leaderboard: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard  🌐

Hopefully, this is just an Open LLM Leaderboard error! @open-llm-leaderboard SOS! 🚨

eek osanseviero dillfrescott quyettv d8rt8v