anychat

supports chatgpt, gemini, perplexity, claude, meta llama, grok all in one app

try it out there: https://huggingface.co/spaces/akhaliq/anychat


Interesting long read from @evanmiller-anthropic on having a better founded statistical approach to Language Model Evaluations:
https://www.anthropic.com/research/statistical-approach-to-model-evals

Worth a read if you're into LLM evaluations!

Cc @clefourrier


For those who want to try out the new https://huggingface.co/black-forest-labs/FLUX.1-Redux-dev
You can do this from my latest spaces https://huggingface.co/spaces/MohamedRashad/Flux-Redux


Estamos tratando de unir, aunar fuerzas y cooperar en experimentos de IA en América Latina. Te invito a unirte a nosotros en «LatinAI». La idea es compartir y organizar espacios, modelos y conjuntos de datos en español/portugués/guaraní/mapuche o ingles para el desarrollo en América Latina.
Siéntete libre de unirte a la organización : https://huggingface.co/LatinAI
---
We are trying to unite, join forces and cooperate in AI experiments in Latin America. We invite you to join us in “LatinAI”. The idea is to share and organize spaces, models and datasets in Spanish/Portuguese/Guarani/Mapuche or English for development in Latin America.
Feel free to join the organization : https://huggingface.co/LatinAI


Sorry, I just cannot get the hype behind F5 TTS. It has now gathered a thousand votes in the TTS Arena fork and **has remained in #8 spot** in the _mostly_ Open TTS adversaries.

The voice sample used is the same as XTTS. F5 has so far been unstable, being unemotional/monotone/depressed and mispronouncing words (_awestruck_).

If you have suggestions please give feedback in the following thread:
https://huggingface.co/spaces/mrfakename/E2-F5-TTS/discussions/32


Maybe that post I showed the other day with my Hyperbolic Embeddings getting to perfect loss with RAdam was a one-time fluke, bad test dataset, etc.? Anotha' one! I gave it a test set a PhD student would struggle with. This model is a bit more souped up. Major callouts of the model: High Dimensional Encoding (HDC), Hyperbolic Embeddings, Entropix. Link to the Colab Notebook: https://colab.research.google.com/drive/1mS-uxhufx-h7eZXL0ZwPMAAXHqSeGZxX?usp=sharing


I created bodybuilder and playful AI this week. Try them!

https://huggingface.co/migueldeguzmandev/bodybuilder_ai

https://huggingface.co/migueldeguzmandev/playful_ai


Using AI to teach English as a Foreign Language? EFL teachers often have busy schedules, variable class sizes, and unexpected cancellations. Introducting VocabSova: https://huggingface.co/spaces/ZennyKenny/VocabSova

VocabSova is a simple chatbot interface that helps teachers create topical vocabulary lists, custom worksheets using that vocabulary, and group activities on a defined theme for a specific English-speaking level (according to CEFR international standards).

There is a great use case for AI in nearly every field, and language learning is a particularly apt domain in my opinion. VocabSova is in active development during its Alpha release, all feedback welcome.


Good folks from @amazon, @Stanford, and other great institutions have released “A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models!”

This comprehensive survey examines over 32 cutting-edge techniques to combat hallucination in Large Language Models (LLMs). As LLMs become increasingly integral to our daily operations, addressing their tendency to generate ungrounded content is crucial.

Retrieval-Augmented Generation (RAG) Innovations:
- Pre-generation retrieval using LLM-Augmenter with Plug-and-Play modules
- Real-time verification through the EVER framework implementing three-stage validation
- Post-generation refinement via the RARR system for automated attribution

Advanced Decoding Strategies:
- Context-Aware Decoding (CAD) utilizing contrastive output distribution
- DoLa's innovative approach of contrasting logit differences between transformer layers

Knowledge Integration Methods:
- The RHO framework leveraging entity representations and relation predicates
- FLEEK's intelligent fact verification system using curated knowledge graphs

Novel Loss Functions:
- Text Hallucination Regularization (THR) derived from mutual information
- The mFACT metric for evaluating faithfulness in multilingual contexts

This research provides a structured taxonomy for categorizing these mitigation techniques, offering valuable insights for practitioners and researchers working with LLMs.

What are your thoughts on hallucination mitigation in LLMs?


Release a new virtual tryon flux fill finetuning model. Try it here.

https://huggingface.co/xiaozaa/catvton-flux-alpha


Repurposed my older AI workstation to a homelab server, it has received 2xV100 + 1xP40
I can reach huge 210k token context size with MegaBeam-Mistral-7B-512k-GGUF ~70+tok/s, or run Llama-3.1-Nemotron-70B-Instruct-HF-GGUF with 50k Context ~10tok/s (V100 only 40k ctx and 15tok/s).
Also able to Lora finetune with similar performace as an RTX3090.
It moved to the garage to no complaints for the noise from the family. Will  move to a Rack soon :D


p104-100s are beasts. 8 gigs of VRAM, 12 tok/s on qwen 14b at q4, and 18 tok/s on 7b at q6. best thing - 20 euros each.

https://furry.engineer/@cappuch/113500349547803802


SAM: I can segment anything!
SAMURAI: Hold my sake while I track it through time and space without even training 🎯

Researchers really said what if we gave SAM object permanence and it worked 🤯

https://www.aimodels.fyi/papers/arxiv/samurai-adapting-segment-anything-model-zero-shot


📢 If you were earlier interested in quick translator application for bunch of texts with spans of fixed parts that tolerant for translation, then this post might be relevant! Delighted to share a bulk_translate -- a framework for automatic texts translation with the pre-anotated fixed spans.

📦 https://pypi.org/project/bulk-translate/
🌟 https://github.com/nicolay-r/bulk-translate

🔑 Spans allows you to control your objects in texts, so that objects would be tollerant to translator.  By default it provides implementation for GoogleTranslate.

bulk_translate features:
✅ Native Implementation of two translation modes:
    - fast-mode: exploits extra chars for grouping text parts into single batch
    - accurate: pefroms individual translation of each text part.
✅ No strings: you're free to adopt any LM / LLM backend.
Support googletrans by default.

The initial release of the project supports fixed spans as text parts wrapped in square brackets [] with non inner space characters.

You can play with your data in CSV here on GoogleColab:
📒 https://colab.research.google.com/github/nicolay-r/bulk-translate/blob/master/bulk_translate_demo.ipynb

👏  This project is based on AREkit 0.25.1 pipelines for deployment lm-based workflows:
https://github.com/nicolay-r/AREkit


https://hf.co/spaces/hexgrad/Kokoro-TTS just got an upgrade that substantially improves TTS naturalness for short bursts while maintaining parity for longer utterances! 🔥

Read more and listen to before/after audio samples at https://hf.co/blog/hexgrad/kokoro-short-burst-upgrade

(Probably would have made that Article a Post instead, if audio could be embedded into Posts.)


🦋 Hug the butterfly! You can now add your Bluesky handle to your Hugging Face profile! ✨


NVIDIA Labs developed SANA model weights and Gradio demo app published —Check out this amazing new Text to Image model by NVIDIA

Official repo : https://github.com/NVlabs/Sana

1-Click Windows, RunPod, Massed Compute installers and free Kaggle notebook : https://www.patreon.com/posts/116474081

You can follow instructions on the repository to install and use locally. I tested on my Windows RTX 3060 and 3090 GPUs.

I have tested some speeds and VRAM usage too

Uses 9.5 GB VRAM but someone reported works good on 8 GB GPUs too

Default settings per image speeds as below

Free Kaggle Account Notebook on T4 GPU : 15 second
RTX 3060 (12 GB) : 9.5 second
RTX 3090 : 4 second
RTX 4090 : 2 second
More info : https://nvlabs.github.io/Sana/

Works great on RunPod and Massed Compute as well (cloud)

Sana : Efficient High-Resolution Image Synthesis
with Linear Diffusion Transformer

About Sana — Taken from official repo

We introduce Sana, a text-to-image framework that can efficiently generate images up to 4096 × 4096 resolution. Sana can synthesize high-resolution, high-quality images with strong text-image alignment at a remarkably fast speed, deployable on laptop GPU. Core designs include: Deep compression autoencoder: unlike traditional AEs, which compress images only 8×, we trained an AE that can compress images 32×, effectively reducing the number of latent tokens. Linear DiT: we replace all vanilla attention in DiT with linear attention, which is more efficient at high resolutions without sacrificing quality. Decoder-only text encoder: we replaced T5 with modern decoder-only small LLM as the text encoder and designed complex human instruction with in-context learning to enhance the image-text alignment. Efficient training and sampling: we propose Flow-DPM-Solver to reduce sampling steps, with efficient caption labeling and selection to accelerate convergence.


All wisper or transcriptions projects spaces (including Zero GPUs plans) are very slow or have many quotas bugs or processing errors. I tested all... anything can transcribe a single 3 min short audio file!!! How it´s possible???


I'm currently on a push to expand the scope of image based datasets on the Hub. There's certainly a lot already, but for anyone who's looked closely, there's not a whole lot of standardization. I am to fix that, datasets under the https://huggingface.co/timm and https://huggingface.co/pixparse orgs will serve as canonical examples for various task / modality combinations and be useable without fuss in libraries like `timm`, `OpenCLIP`, and hopefully more.

I just uploaded the first multi-label dataset that I'll support with `timm` scripts soon: https://huggingface.co/datasets/timm/plant-pathology-2021

Next up object detection & segmentation! I've got an annotation spec sorted out, a lot of datasets ready to rip, and yeah that means `timm` support for object detection, eventually segmentation, is finally under development :O

John6666 davanstrien lhoestq fgdrfgrgrdgdr


Trace LLM calls with Arize AI's Phoenix observability dashboards on Hugging Face Spaces! 🚀

✨ I just added a new recipe to the Open-Source AI Cookbook that shows you how to:
1⃣ Deploy Phoenix on HF Spaces with persistent storage in a few clicks
2⃣ Configure LLM tracing with the 𝗦𝗲𝗿𝘃𝗲𝗿𝗹𝗲𝘀𝘀 𝗜𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗔𝗣𝗜
3⃣ Observe multi-agent application runs with the CrewAI integration

𝗢𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗶𝘀 𝗰𝗿𝘂𝗰𝗶𝗮𝗹 for building robust LLM apps.

Phoenix makes it easy to visualize trace data, evaluate performance, and track down issues. Give it a try!

🔗 Cookbook recipe: https://huggingface.co/learn/cookbook/en/phoenix_observability_on_hf_spaces
🔗 Phoenix docs: https://docs.arize.com/phoenix


If I am correct and the LLM model changes the 'shape' of the data as it learns, then I should be able to track and utilize those shape changes as a backpropagation training mechanism, right? Well guess what, I can do that! Entropy, Sparsity, and Density, this is how I can measure the shape of the data the LLM model is creating. Nodes, Clusters, and Edges, these are the mechanisms within the neural network the LLM model updates as it learns these concepts. I measure the effects of these updates, via Entropy, Sparsity, and Density. Check out more in this video: https://youtu.be/jADTt5HHtiw


What a brilliant week for Open Source AI!

Qwen 2.5 Coder by Alibaba - 0.5B / 1.5B / 3B / 7B / 14B/ 32B (Base + Instruct) Code generation LLMs, with 32B tackling giants like Gemnini 1.5 Pro, Claude Sonnet
https://huggingface.co/collections/Qwen/qwen25-coder-66eaa22e6f99801bf65b0c2f

LLM2CLIP from Microsoft - Leverage LLMs to train ultra-powerful CLIP models! Boosts performance over the previous SOTA by ~17%
https://huggingface.co/collections/microsoft/llm2clip-672323a266173cfa40b32d4c

Athene v2 Chat & Agent by NexusFlow - SoTA general LLM fine-tuned from Qwen 2.5 72B excels at Chat + Function Calling/ JSON/ Agents
https://huggingface.co/collections/Nexusflow/athene-v2-6735b85e505981a794fb02cc

Orca Agent Instruct by Microsoft - 1 million instruct pairs covering text editing, creative writing, coding, reading comprehension, etc - permissively licensed
https://huggingface.co/datasets/microsoft/orca-agentinstruct-1M-v1

Ultravox by FixieAI - 70B/ 8B model approaching GPT4o level, pick any LLM, train an adapter with Whisper as Audio Encoder
https://huggingface.co/collections/reach-vb/ultravox-audio-language-model-release-67373b602af0a52b2a88ae71

JanusFlow 1.3 by DeepSeek - Next iteration of their Unified MultiModal LLM Janus with RectifiedFlow
https://huggingface.co/deepseek-ai/JanusFlow-1.3B

Common Corpus by Pleais  - 2,003,039,184,047 multilingual, commercially permissive and high quality tokens!
https://huggingface.co/datasets/PleIAs/common_corpus

I'm sure I missed a lot, can't wait for the next week!

Put down in comments what I missed! 🤗

YaTharThShaRma999 AdinaY cosmosgenius s3nh davidberenstein1957 John6666 Ameeeee lunarflu jsulz MingxingLi m-ric fullstuckdev mdpi-ai erinys IBOYAI

victor Vazdru davidberenstein1957 nicolollo lunarflu fullstuckdev Maitt

prithivMLmods lunarflu jsulz fullstuckdev


next version of https://huggingface.co/datasets/sequelbox/Celestia will be https://huggingface.co/datasets/microsoft/orca-agentinstruct-1M-v1 style. coming soon


Minimalistic Adapters 🎃

🚀Demo Here:
 https://huggingface.co/spaces/prithivMLmods/FLUX-LoRA-DLC

🚀Model:
{ Quote Tuner } : https://huggingface.co/prithivMLmods/Flux.1-Dev-Quote-LoRA
{ Stamp Art } : https://huggingface.co/prithivMLmods/Flux.1-Dev-Stamp-Art-LoRA
{ Hand Sticky } : https://huggingface.co/prithivMLmods/Flux.1-Dev-Hand-Sticky-LoRA
{ Poster HQ } : https://huggingface.co/prithivMLmods/Flux.1-Dev-Poster-HQ-LoRA
{ Ctoon Min } : https://huggingface.co/prithivMLmods/Flux.1-Dev-Ctoon-LoRA

🚀Collection:
{ Flux LoRA Collection} : https://huggingface.co/collections/prithivMLmods/flux-lora-collections-66dd5908be2206cfaa8519be
{ LoRA Space Collection } : https://huggingface.co/collections/prithivMLmods/lora-space-collections-6714b72e0d49e1c97fbd6a32

🚀For More Visit
https://huggingface.co/strangerzonehf
.
.
.
🤗@prithivMLmods

Ngrthm RenderIo darksfx ai4life44 hypergod rdrede victor John6666 merve clem OmbelineM


Kohya brought massive improvements to FLUX LoRA (as low as 4 GB GPUs) and DreamBooth / Fine-Tuning (as low as 6 GB GPUs) training - check attached images in full size to see full details

You can download all configs and full instructions

> https://www.patreon.com/posts/112099700 - Fine Tuning post

> https://www.patreon.com/posts/110879657 - LoRA post

Kohya brought massive improvements to FLUX LoRA and DreamBooth / Fine-Tuning (min 6GB GPU) training.

Now as low as 4GB GPUs can train FLUX LoRA with decent quality and 24GB and below GPUs got a huge speed boost when doing Full DreamBooth / Fine-Tuning training

You need minimum 4GB GPU to do a FLUX LoRA training and minimum 6 GB GPU to do FLUX DreamBooth / Full Fine-Tuning training. It is just mind blowing.

You can download all configs and full instructions > https://www.patreon.com/posts/112099700

The above post also has 1-click installers and downloaders for Windows, RunPod and Massed Compute

The model downloader scripts also updated and downloading 30+GB models takes total 1 minute on Massed Compute

You can read the recent updates here : https://github.com/kohya-ss/sd-scripts/tree/sd3?tab=readme-ov-file#recent-updates

This is the Kohya GUI branch : https://github.com/bmaltais/kohya_ss/tree/sd3-flux.1

Key thing to reduce VRAM usage is using block swap

Kohya implemented the logic of OneTrainer to improve block swapping speed significantly and now it is supported for LoRAs as well

Now you can do FP16 training with LoRAs on 24 GB and below GPUs

Now you can train a FLUX LoRA on a 4 GB GPU - key is FP8, block swap and using certain layers training (remember single layer LoRA training)

It took me more than 1 day to test all newer configs, their VRAM demands, their relative step speeds and prepare the configs :)

MonsterMMORPG remjie jayavibhav DennyDenn


Ok RNNs can rap too:)

Here we implement the seminal RNN paper “Generating Text with Recurrent Neural Networks"- we train a character-level multiplicative recurrent neural network model (~250k params) for 1000 epochs with Adam opt on 2pac's "Hit 'em Up", sample was fun lol.

Code: https://github.com/Jaykef/ai-algorithms/blob/main/generating_texts_with_rnns.ipynb


Finaly I realesed mediapipe-face animation space.

Mediapipe 68-points Eyes-Closed and Mouth-Opened
https://huggingface.co/spaces/Akjava/mediapipe-68-facial-guide-eyes-closed-mouth-opened

[Article]Results: Converted Guide Images(eyes-closed and mouth-opened) with Flux.1 schenll img2img/inpaint
https://huggingface.co/blog/Akjava/result-guide-image-eyes-mouth

All the other tools listed are designed to support Mediapipe Face Animation

https://huggingface.co/collections/Akjava/mediapipe-tools-672ffe8ee7b62763c31b70c7

https://huggingface.co/collections/Akjava/webp-3-frame-talking-animation-tools-672819ce4989f354cdbcc739


Good folks at @nvidia  and @Tsinghua_Uni have released LLAMA-MESH - A Revolutionary Approach to 3D Content Generation!

This innovative framework enables the direct generation of 3D meshes from natural language prompts while maintaining strong language capabilities.

Here is the Architecture & Implementation!

>> Core Components

Model Foundation
- If you haven't guessed it yet, it's built on the LLaMA-3.1-8B-Instruct base model
- Maintains original language capabilities while adding 3D generation
- Context length is set to 8,000 tokens

3D Representation Strategy
- Uses the OBJ file format for mesh representation
- Quantizes vertex coordinates into 64 discrete bins per axis
- Sorts vertices by z-y-x coordinates, from lowest to highest
- Sorts faces by the lowest vertex indices for consistency

Data Processing Pipeline
- Filters meshes to a maximum of 500 faces for computational efficiency
- Applies random rotations (0°, 90°, 180°, 270°) for data augmentation
- Generates ~125k mesh variations from 31k base meshes
- Uses Cap3D-generated captions for text descriptions

>> Training Framework

Dataset Composition
- 40% Mesh Generation tasks
- 20% Mesh Understanding tasks
- 40% General Conversation (UltraChat dataset)
- 8x training turns for generation, 4x for understanding

Training Configuration
- Deployed on 32 A100 GPUs (for Nvidia, this is literally in-house)
- 21,000 training iterations
- Global batch size: 128
- AdamW optimizer with a 1e-5 learning rate
- 30-step warmup with cosine scheduling
- Total training time: approximately 3 days (based on the paper)

This research opens exciting possibilities for intuitive 3D content creation through natural language interaction. The future of digital design is conversational!


OmniVision-968M: a new local VLM for edge devices, fast & small but performant
💨 a new vision language model with 9x less image tokens, super efficient
📖 aligned with DPO for reducing hallucinations
⚡ Apache 2.0 license 🔥

Demo hf.co/spaces/NexaAIDev/omnivlm-dpo-demo
Model https://huggingface.co/NexaAIDev/omnivision-968M

YaTharThShaRma999 John6666 quyet7779 typesdigital Csplk Norod78 not-lain Sri-Vigneshwar-DJ ai-everyday victor lhoestq Nydaym Catering3733 ogozcelik ucsahin appvoid FGOTYT OmbelineM


Kokoro: a small, fast 80M param TTS model hosted on ZeroGPU at https://hf.co/spaces/hexgrad/Kokoro-TTS


🚀 Introducing the Model Drops Tracker! 🕵♂

Feeling overwhelmed by the AI model release frenzy? 🤯 You're not alone!

I built this simple tool to help us all keep up:
- Filter recent models from the 🤗 Hub
- Set minimum likes threshold
- Choose how recent you want to go

Try it out and let me know what you think: https://huggingface.co/spaces/fdaudens/Model-Drops-Tracker

Any features you'd like to see added?
#AIModels


🤖💡Just tried out @m-ric 's new LLaMA-3.1 70B agent for data analysis. Impressive stuff.

🚢📊 Fed it the Titanic passenger dataset with minimal instructions. The agent autonomously dug in, tested hypotheses, and reached some intriguing conclusions:

"Lower class passengers less likely to survive, slight negative correlation with age, and positive correlation between fare price and survival."

📈It even generated charts to visualize the findings!

🧠💼 Great potential for business intelligence, research, and decision-making when we can just upload datasets and let AI agents loose on them.

👉 Check it out: https://huggingface.co/spaces/m-ric/agent-data-analyst

🤔 Any particular use cases you're excited about?

#AIinDataAnalysis #MachineLearning #DataScience


Hello there,

New model released, my goal was to try finetune on the last Llama-3.1-8B-Instruct but not a small train, I wanted to do something useful.
One of the rare model that I didn't made for RP, or in the goal to uncensor it (but I did anyway kek).

The model was trained on 9M Claude conversations ONLY, giving him another writting style.

https://huggingface.co/Undi95/Meta-Llama-3.1-8B-Claude > OG release fp32, it's the epoch 2
https://huggingface.co/Undi95/Meta-Llama-3.1-8B-Claude-bf16 > Base model resharded in bf16 waiting for available quant without issues

Since it's frustrating to be censored using a local model, orthogonal activation steering was used, trying to force the model to never refuse a prompt.

https://huggingface.co/Undi95/Meta-Llama-3.1-8B-Claude-68fail-3000total > Uncensored model, refuse 68 times on 3000 toxic prompt
https://huggingface.co/Undi95/Meta-Llama-3.1-8B-Claude-39fail-3000total > Uncensored model, refuse 39 times on 3000 toxic prompt

It still refuse some prompt but the majority of them is uncensored. OAS can make a model more dumb or make the base perplexity go higher, so I didn't snipe for 0 refusal.

I don't do non-RP model a lot so any feedback is welcome, I would like to re-use this base for some others future project if needed.


Professional Threads Post Writer
https://hf.co/chat/assistant/66a0ee6fc397ecb70cee100d


Llama 3.1 405B Instruct beats GPT-4o on MixEval-Hard

Just ran MixEval for 405B, Sonnet-3.5 and 4o, with 405B landing right between the other two at 66.19

The GPT-4o result of 64.7 replicated locally but Sonnet-3.5 actually scored 70.25/69.45 in my replications 🤔 Still well ahead of the other 2 though.

Sammple of 1 of the eval calls here: https://wandb.ai/morgan/MixEval/weave/calls/07b05ae2-2ef5-4525-98a6-c59963b76fe1

Quick auto-logging tracing for openai-compatible clients and many more here: https://wandb.github.io/weave/quickstart/


Hi HF community!🤗
Hope y'all are as excited as me for the release of Llama 3.1! 🦙
Following the release, I built a space exploiting  HF Inference API, thanks to a recipe you can find in this awesome GitHub repo (https://github.com/huggingface/huggingface-llama-recipes/): you can now run Llama-3.1-405B customizing its system instructions and other parameters, for free! 😇
Follow this link: https://huggingface.co/spaces/as-cle-bert/Llama-3.1-405B-FP8 and let the fun begin!🍕

Nymbo hg-crown victor Taylor658 osanseviero louisbrulenaudet


JUST RELEASED: Fireplace 2 for Llama 3.1 8b Instruct!

Fireplace 2 is an 'expansion pack' of structured outputs you can request during your chat, using special request tokens to let Llama know you're looking for specific types of responses:
Inline function calls
SQL queries
JSON objects
Data visualization with matplotlib

https://huggingface.co/ValiantLabs/Llama3.1-8B-Fireplace2


Meta Researchers: How many compute hours should we use to train Llama 3.1?
Mr. Zuck: Yes! 🤖💪

Good folks at @AIatMeta did not just release the models but also published a 92-page detailed paper 📄 on their findings and technical aspects of the models and their training process!

Generally, we just gobble up these weights and forget the compute infrastructure used to train these models. 🖥🚀

Here are some interesting findings about the computing infrastructure of Llamas:

- Llama 1 and 2 models were trained on @Meta 's AI Research SuperCluster. Llama 3 was migrated to Meta’s production clusters! 📊

- That's 16,000 H100 GPUs, with each GPU featuring 700W TDP and 80GB HBM3, arranged in Meta’s Grand Teton AI server platform. 🖥🔋

- What about storing checkpoints? Used Tectonic, a distributed file system, for storage, with capacities reaching 240 PB and peak throughput of 7 TB/s. 💾📈

- Meta's mad lads saved each GPU’s model state, ranging from 1 MB to 4 GB per GPU, for recovery and debugging. 🛠🔍

If this sounds big, well, they document the humungous challenges that come with it:

- In the 54-day training period, there were 466 job interruptions. 🕒🔄

- About 78% of unexpected interruptions were attributed to confirmed or suspected hardware issues. Mostly GPUs! 💥🖥

- Saving all checkpoints is cool until you do it for the 300B+ parameters model. The bursty nature of checkpoint writes, essential for state-saving during training, periodically saturated the storage fabric, impacting performance. 📉💾

- With all this, effective training time—measured as the time spent on useful training over the elapsed time—was higher than 90%. ⏱📊

I think this is the stuff that movies can be made on! 🎬🌟

Paper: https://ai.meta.com/research/publications/the-llama-3-herd-of-models/


🚀 Exciting news! We've just launched "Thundermoon" - the latest version of Moondream, our open-source vision language model! 🌙

Key improvements in this release:
1. Massive leap in OCR capabilities
2. Enhanced document understanding
3. Significant boosts across key metrics:
 * DocVQA: 61.9 (↑103%)
 * TextVQA: 60.2 (↑5.2%)
 * GQA: 64.9 (↑2.9%)

What does this mean? Moondream can now tackle complex document analysis tasks with unprecedented accuracy for a model of its size. From deciphering handwritten notes to interpreting data tables, the applications are vast.

Check out the image for a glimpse of Moondream in action, effortlessly extracting insights from a 1944 sugar industry document!

Why it matters:
* Democratizing AI: As an open-source project, we're making advanced vision AI accessible to all developers.
* Efficiency: Proving that smaller models can deliver big results.
* Real-world impact: From historical document analysis to modern business intelligence, the potential use cases are exciting.

Curious to try it out? Try out the live demo here! https://moondream.ai/playground

YaTharThShaRma999 Tom-Neverwinter catastropiyush rAIfle rbrisita louisbrulenaudet Csplk iky1e


I just had a masterclass in open-source collaboration with the release of Llama 3.1 🦙🤗

Meta dropped Llama 3.1, and seeing firsthand the Hugging Face team working to integrate it is nothing short of impressive. Their swift integration, comprehensive documentation, and innovative tools showcase the power of open-source teamwork.

For the curious minds:

📊 Check out independent evaluations: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard

🧠 Deep dive into the tech: https://huggingface.co/blog/llama31

👨🍳 Try different recipes (including running 8B on free Colab!): https://github.com/huggingface/huggingface-llama-recipes

📈 Visualize open vs. closed LLM progress: https://huggingface.co/spaces/andrewrreed/closed-vs-open-arena-elo

🤖 Generate synthetic data with distilabel, thanks to the new license allowing the use of outputs to train other LLMs https://huggingface.co/blog/llama31#synthetic-data-generation-with-distilabel

💡 Pro tip: Experience the 405B version for free on HuggingChat, now with tool-calling capabilities! https://huggingface.co/chat/

#OpenSourceAI #AIInnovation

clem gastonmorixe victor chuangxinlezhi Utha R-I-0816 fluxthedev osanseviero

clboetticher chuangxinlezhi Utha R-I-0816 osanseviero


Any idea if this "scheduled"/"dynamic" batch size is available in HF Trainers ? I've never seen it personally


🚨 Launching The Visual Haystacks (VHs) Benchmark: the first "visual-centric" Needle-In-A-Haystack (NIAH) benchmark to assess LMMs' capability in long-context visual retrieval and reasoning.

Check it out!
https://huggingface.co/datasets/tsunghanwu/visual_haystacks
https://visual-haystacks.github.io/
https://arxiv.org/abs/2407.13766
https://github.com/visual-haystacks/vhs_benchmark


I just launched an exciting new multiplayer app powered by GPT-4o, enabling collaborative AI-driven queries in a single shared session!

### 🔗 Try It Out!  👉 Check out the GPT-4o Multiplayer App
Experience the future of collaborative AI by visiting our space on Hugging Face: https://huggingface.co/spaces/awacke1/ChatStreamlitMultiplayer

🎉 This innovative tool lets you and your team reason over:

###📝 Text
###🖼 Image
###🎵 Audio
###🎥 Video

## 🔍 Key Features

### Shared Contributions
Collaborate in real-time, seeing each other's inputs and contributions.
Enhances teamwork and fosters a collective approach to problem-solving.

### Diverse Media Integration
Seamlessly analyze and reason with text, images, audio, and video.
Breakthrough capabilities in handling complex media types, including air traffic control images and audio.

## 🛠 Real-World Testing
This morning, we tested the app using images and audio from air traffic control—a challenge that was nearly impossible to handle with ease just a few years ago. 🚁💬

🌱 The Future of AI Collaboration
We believe AI Pair Programming is evolving into a new era of intelligence through shared contributions and teamwork. As we continue to develop, this app will enable groups to:

Generate detailed text responses 📝
Collaborate on code responses 💻
Develop new AI programs together 🤖


We https://mii-llm.ai just released a new LLM Italian benchmark and a set of evaluation: MMLU-PRO-ITA

Thanks to @efederici who released https://huggingface.co/datasets/efederici/MMLU-Pro-ita a machine translated version of MMLU-PRO and thanks to a community shared computational effort we published  in the "Eval Aggiuntive" tab of https://huggingface.co/spaces/FinancialSupport/open_ita_llm_leaderboard the results on Italian open source LLMs.

If you want to deepen read the blog article on hf https://huggingface.co/blog/giux78/mmlu-pro-ita


[New tool] Follow interesting ML persons 👩🎨 👨🎤 👩🏫 with Followgraph

https://huggingface.co/spaces/severo/followgraph

Please try it and tell me if it helped you discover high-quality content 👍 👎

I repurposed "Followgraph for Mastodon" (https://followgraph.vercel.app/).

My new follows: @TheBloke @mlabonne @teknium @KnutJaegersberg  @SkalskiP @AmelieSchreiber @lbourdois @ceyda @andrewyng @Pclanglais @karpathy

And you?

gokaygokay Ramikan-BR mmhamdy lhoestq appvoid kramp merve mikestaub osanseviero lbourdois louisbrulenaudet ZennyKenny

mlabonne Ramikan-BR mmhamdy lhoestq merve mikestaub clem osanseviero lbourdois efecelik


🙋🏻♂ Hey there folks

made a demo for Nvidia Minitron on an A100.

Minitron is a family of small language models (SLMs) obtained by pruning NVIDIA's Nemotron-4 15B model. We prune model embedding size, attention heads, and MLP intermediate dimension, following which, we perform continued training with distillation to arrive at the final models.

Deriving the Minitron 8B and 4B models from the base 15B model using our approach requires up to 40x fewer training tokens per model compared to training from scratch; this results in compute cost savings of 1.8x for training the full model family (15B, 8B, and 4B). Minitron models exhibit up to a 16% improvement in MMLU scores compared to training from scratch, perform comparably to other community models such as Mistral 7B, Gemma 7B and Llama-3 8B, and outperform state-of-the-art compression techniques from the literature. Please refer to our arXiv paper for more details.

Minitron models are for research and development only.

source : https://huggingface.co/nvidia/Minitron-8B-Base
demo : https://huggingface.co/spaces/Tonic/Minitron

nicolay-r TuringsSolutions ZeroWw clem Nymbo osanseviero louisbrulenaudet Tonic


🔥Thrilled to release our 8B version of Symbol-LLM-Instruct !

It follows the two-stage training strategy proposed in the original paper and is continually optimized on LLaMA3-Chat-8B model.

Symbol-LLM was accepted by ACL'24 main conference ! See you in Thailand !

Paper link: https://arxiv.org/abs/2311.09278
Paper Title: Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Models

Symbol-LLM Xdotnet Ramdevkijai louisbrulenaudet osanseviero


How to create custom LLMs from scratch

See my new podcast on this topic, at https://mltblog.com/3xS1bf5

Despite GPT, Claude, Gemini, LLama and the other host of LLMs that we have access to, a variety of organizations are still exploring their options when it comes to custom LLMs. Logging in to ChatGPT is easy enough, and so is creating a ‘custom’ openAI GPT, but what does it take to create a truly custom LLM? When and why might this be useful, and will it be worth the effort?


LazyLLM - Unusual Colab (Apple & Meta) Yields Impactful Work

LLM inference typically consists of two stages: prefilling/tokenizing and decoding. In the prefilling stage, the model processes the entire input prompt, computing and caching key-value (KV) pairs for each token, which can be time-consuming for long prompts. This is followed by the decoding stage, where the model generates tokens sequentially, reusing the cached KVs.

LazyLLM introduces a dynamic token pruning technique. Instead of computing KVs for all tokens during prefilling, LazyLLM selectively processes only the most important tokens based on attention scores, deferring less important ones to later steps if needed. It uses progressive token pruning across transformer layers and introduces an Aux Cache to store hidden states of pruned tokens.

This approach significantly reduces the time-to-first-token (TTFT) and overall generation time while maintaining accuracy across various tasks. LazyLLM outperforms baseline techniques like random token dropping and static pruning, and can be easily integrated into existing LLMs without fine-tuning, offering a practical solution for accelerating LLM inference, especially in long context scenarios.

IN SIMPLE TERMS
When you prompt a large language model (LLM), it usually looks at every single word/subword (or tokens) in your prompt before generating a response. This can be time consuming, especially for prompts with very long texts. This paper introduces a new technique that solves this problem by being more selective. Instead of looking at every word right away, it only focuses on the most important words first. It decides which words are important based on how much attention the model gives them. If it needs other words later, it can go back and look at them then. This approach is like skimming a text for key information before reading it in detail.

Read More: https://arxiv.org/pdf/2407.14057


SNN Image Diffusion V2

Billionaires have been made for less than this. This is only one of the things it can it do. It can do API calls, function calls, optimize poker and blackjack odds, anything that is an optimization problem. It costs fractions of a penny and requires fractions of the compute of an LLM model. It can even communicate two ways with an LLM model.


Hi HuggingFacers!🤗

Good news concerning https://huggingface.co/spaces/as-cle-bert/smolLM-arena, the chat arena where you can compare some of the Small Language Models (<1.7B) on the Hub and cast your vote to choose the best!📱
The space now has a new interface with chatbots instead of textboxs, it runs faster and it also comes with usage instructions :)
Have fun!🍕

prithivMLmods nicolay-r Ramikan-BR cahlen osanseviero quyettv louisbrulenaudet


Introducing Whisper Diarization: Multilingual speech recognition with word-level timestamps and speaker segmentation, running 100% locally in your browser thanks to 🤗 Transformers.js!

Tested on this iconic Letterman interview w/ Grace Hopper from 1983!
- Demo: https://huggingface.co/spaces/Xenova/whisper-speaker-diarization
- Source code: https://huggingface.co/spaces/Xenova/whisper-speaker-diarization/tree/main/whisper-speaker-diarization

freinold ZeroWw John6666 DmitryRyumin okamirvs Nymbo ssboost maruyasa Deddy GPT007 sudanenator Rsln dave3991 bmorphism toshvelaga osanseviero d8rt8v devstockgirl

ssboost prithivMLmods Sylvestre Deddy nicolay-r Gatozu35 AARon99 toshvelaga osanseviero

julien-rodriguez DataSoul BoscoTheDog clem toshvelaga osanseviero


"By the end of this blog post, you will have
- learnt all the new goodies accompanying the latest macOS release
- AND successfully run a 7B parameter model using less than 4GB of memory on your Mac."

Game-changer for local AI? Can't wait to try this!

Brillant work by @pcuenq  @osanseviero  @reach-vb  @FL33TW00D-HF

Check it out: https://huggingface.co/blog/mistral-coreml #apple


🎉 Ghost 8B Beta Released: Game-Changing Language Model
--
Ghost 8B Beta is a groundbreaking language model developed with a clear vision: to deliver exceptional multilingual support, superior knowledge capabilities, and all while remaining cost-effective. This model comes in two context length variations, 8k and 128k, ensuring flexibility for various tasks. Moreover, it boasts built-in multilingual functionality, making it a powerful tool for global communication and understanding.
--
* See detailed article: https://huggingface.co/blog/lamhieu/ghost-8b-beta-released-game-changing-language-mode
* Model card: https://huggingface.co/ghost-x/ghost-8b-beta
* Official website: https://ghost-x.org/docs/models/ghost-8b-beta

ZeroWw danielus nicolay-r ecyht2 John6666 MDalprato louisbrulenaudet


Hey everyone 🤗!
Check out this cool little reproduction of the Clarity Upscaler (https://github.com/philz1337x/clarity-upscaler) using refiners (https://github.com/finegrain-ai/refiners): https://huggingface.co/spaces/finegrain/enhancer

limiteinductive deltheil jaebumskiyomi piercus 1aurent clem John6666 Blane187 osanseviero


Questions about data, synthetic data, human feedback and data quality?

Argilla has moved its community from Slack to the Hugging Face Discord server!

When part of the Hugging Face Discord, you can select “Channels & roles” and select “Argilla” along with any of the other groups that are interesting to you. “Argilla” will cover anything about argilla and distilabel, and it will give you access to 1) #argilla-distilabel-general, for all general discussions and news, and 2) #argilla-distilabel-help, for any usage-focused questions.


ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG
  Capabilities (2407.14482)

🚀 Good folks at @nvidia just dropped: "ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilities" 🧠💡

In the past few months, the open LLM community has made significant progress in releasing open models (Llama-3-70B-Instruct (@Meta -AI) 🦙, QWen2-72BInstruct (@AlibabaGroup ) 🌐, Nemotron-4-340B-Instruct (@nvidia ) ⚙, and Mixtral-8x22BInstruct-v0.1 (@MistralAI ) 🌪) that are at par with proprietary models! 📈

But top models like GPT-4 are still outperforming them in certain domains! 🔝💪

This led us to having domain-focused open-LLMs (DeepSeek-Coder-V2 for coding and math 👨💻➕, ChatQA 1.5 for conversational QA and retrieval-augmented generation (RAG) 💬🔍, and InternVL 1.5 for vision-language tasks 🖼🗣)

The challenge that ChatQA 2 focuses on is of context length and RAG! 📏🔗

These are the two capabilities essential for LLMs to process large volumes of information that cannot fit into a single prompt and are complementary to each other, depending on the downstream tasks and computational budgets. 🧩📊

The solution is a detailed continued training recipe to extend the context window of Llama3-70B-base from 8K to 128K tokens, along with a three-stage instruction tuning process to enhance the model's instruction-following, RAG performance, and long-context understanding capabilities. 🔄🔧

📄 Paper: https://huggingface.co/papers/2407.14482

The interesting thing to notice from benchmarks was, how good QWen 2 is out of the box! 👏✨


Intelligence is all you need for roleplay.
Roleplay is overlooked as a special case of chain-of-thought, where context must be attended to and inferred state of the world and embodied minds must be persisted and evolved along credible narrative lines. LLMs are also being tasked to function as gamemasters. It's a challenging task which points to potential future benchmarks. The fact that the largest commercial LLMs are adept in generating text for roleplay intuitively implies that model intelligence is sufficient so long as it can generalize properly and pay attention to context without becoming confused.
This recent merge of mine composed using 3 academic fine-tunes, none of which were intended for roleplay, has survived the gauntlet of a Reddit post and appears to be a particularly strong 8B model when it comes to roleplay coherence.
https://huggingface.co/grimjim/llama-3-Nephilim-v3-8B (bf16 weights)
https://huggingface.co/grimjim/llama-3-Nephilim-v3-8B-GGUF (select quants)

John6666 ZeroWw SerialKicked Duttones takeraparterer Cohee jed-tiotuico


LivePortrait AI: Transform Static Photos into Talking Videos. Now supporting Video-to-Video conversion and Superior Expression Transfer at Remarkable Speed

A new tutorial is anticipated to showcase the latest changes and features in V3, including Video-to-Video capabilities and additional enhancements.

This post provides information for both Windows (local) and Cloud installations (Massed Compute, RunPod, and free Kaggle Account).

🔗 Windows Local Installation Tutorial ⤵
▶ https://youtu.be/FPtpNrmuwXk

🔗 Cloud (no-GPU) Installations Tutorial for Massed Compute, RunPod and free Kaggle Account ⤵
▶ https://youtu.be/wG7oPp01COg

The V3 update introduces video-to-video functionality. If you're seeking a one-click installation method for LivePortrait, an open-source zero-shot image-to-animation application on Windows, for local use, this tutorial is essential. It introduces the cutting-edge image-to-animation open-source generator Live Portrait. Simply provide a static image and a driving video to create an impressive animation in seconds. LivePortrait is incredibly fast and adept at preserving facial expressions from the input video. The results are truly astonishing.

With the V3 update adding video-to-video functionality, those interested in using LivePortrait but lacking a powerful GPU, using a Mac, or preferring cloud-based solutions will find this tutorial invaluable. It guides you through the one-click installation and usage of LivePortrait on #MassedCompute, #RunPod, and even a free #Kaggle account. After following this tutorial, you'll find running LivePortrait on cloud services as straightforward as running it locally. LivePortrait is the latest state-of-the-art static image to talking animation generator, surpassing even paid services in both speed and quality.

MonsterMMORPG Duartebrizza John6666 Ramikan-BR attyru Dihelson Blane187 Rsln osanseviero

MonsterMMORPG Ramikan-BR bakudas Dihelson abdeljalilELmajjodi Rsln 0xjorgev

MonsterMMORPG Ramikan-BR yutianCCNU bakudas NHLOCAL


Hi HF Community!🤗

As you may know, small language models like the SmolLM series have been on the rise recently: although it may not be completely fair to compare them with larger models, my thought  was that we could build a space where these SLMs could compete against each other in a chat arena, and here is what came out: https://huggingface.co/spaces/as-cle-bert/smolLM-arena 🚀
Even though there might be some little issues and hiccups due to GPU resources allocation, this space offers you the possibility to compare and play around with several Small Language Models, coming also with a leaderboard page (make sure to refresh it for the latest updates!)👑
Have fun!🍕


Introducing OpenCHAT mini: a lightweight, fast, and unlimited version of OpenGPT 4o.

https://huggingface.co/spaces/KingNish/OpenCHAT-mini2

It has unlimited web search, vision and image generation.

Please take a look and share your review. Thank you! 🤗

John6666 JackLiuAngel abbotk Blane187 Trillionare SvCy louisbrulenaudet AdamyaG noelpil peterciank


Annotated History of Modern AI and Deep Learning (2212.11279)

Reading https://huggingface.co/papers/2212.11279 by Jürgen Schmidhuber.

From the abstract: "A modern history of AI will emphasize breakthroughs outside of the focus of traditional AI text books, in particular, mathematical foundations of today's NNs such as the chain rule (1676), the first NNs (linear regression, circa 1800), and the first working deep learners (1965-)."


You can now find the OBIS - Ocean Biodiversity Information System, on Hugging Face with 128M rows, via the Datasets package stream 🤗

The datasets are integrated, allowing seamless search and mapping by species name, higher taxonomic level, geographic area, depth, time, and environmental parameters. OBIS originates from the Census of Marine Life (2000-2010) and was adopted as a project under IOC-UNESCO’s International Oceanographic Data and Information (IODE) programme in 2009.

Collectively, they have provided over 45 million observations of nearly 120,000 marine species, ranging from bacteria to whales, from the surface to 10,900 meters depth, and from the tropics to the poles.

Link to the dataset: https://huggingface.co/datasets/louisbrulenaudet/obis


New Trends in LLM: Overview with Focus on xLLM

Read full article and download PowerPoint presentation at https://mltblog.com/3KqlNO7

If you ever wondered how xLLM is different from other LLM and RAG architectures, what are the foundational changes that make it appealing to fortune 100 companies, and what are the innovations being copied by competitors, read on. In this article, I share the latest trends and provide a high-level summary of xLLM, describing the ground-breaking technologies that make it unique, faster, and better for professional users and experts. In particular, I share my PowerPoint presentation on the topic.

Search is becoming hot again, this time powered by RAG and LLMs rather than PageRank.  New LLMs may not use transformers, and energy-efficient implementations are gaining popularity, with an attempt to lower GPU usage, and thus costs. Yet all but xLLM still rely on Blackbox neural networks.

Great evaluation metrics remain elusive and will remain so probably forever: in the end, LLMs, just like clustering, are part of unsupervised learning. Two users looking at a non-trivial dataset will never agree on what the “true” underlying cluster structure is. Because “true” is meaningless in this context. The same applies to LLMs. With some exceptions: when used for predictive analytics, that is, supervised learning, it is possible to tell which LLM is best in absolute terms (to some extent; it also depends on the dataset).


I can do Time Series Predictions with Swarm Algorithms! When all you know how to use is a hammer, everything looks like a nail. An LLM model is a hammer. It is not a deity. It has computational and mathematical limitations. Very big ones. Swarm Algorithms do not have this same problem. They are like a screwdriver. The screwdriver is not better than the hammer, both are useful. Why are LLM models bad at things like Time Series Predictions and Function Calls? Because those are jobs better fit for a screwdriver as opposed to a hammer.


SpreadsheetLLM: Encoding Spreadsheets for Large Language Models (2407.09025)

Remember when you had a few hundred rows of data that could easily be opened in Excel. 📊

Well, we are far from that with billion-parameter LLMs trained on trillions of tokens. 🌐

@Microsoft  wants to bridge that using "SpreadsheetLLM": Encoding Spreadsheets for Large Language Models. 🤖📈

While it sounds simple, Spreadsheets, with their extensive two-dimensional grids, various layouts, and diverse formatting options, present notable challenges for large language models (LLMs). 🚧

They initially propose a vanilla serialization approach that incorporates cell addresses, values, and formats. However, this approach is limited by LLMs' token constraints, making it impractical for most applications. ⛔

Solution... A SheetCompressor, an innovative encoding framework that compresses spreadsheets effectively for LLMs. 🔧

It comprises three modules:
1⃣ Structural-anchor-based compression
2⃣ Inverse index translation
3⃣ Data-format-aware aggregation

It significantly improves performance in spreadsheet table detection task, outperforming the vanilla approach by 25.6% in GPT4's in-context learning setting. 🏆

Sounds exciting, sadly no code, models OR datasets are released. 🙁

Moreover, there is a lot of research in encoding 2D position embeddings and this work has not been benchmarked against that! 🧐

Paper: https://huggingface.co/papers/2407.09025


Wanted to share some brief comparison of early training of the two-stage PixArt e-diffi pipeline.

On the left, we have the full stage 1 model generating all 50 steps on its own. This model is not trained at all on the final 400 timesteps of the schedule. On the right, we have the combined pipeline where stage 1 output is fed into stage 2.

Currently, the difference is rather minimal - but the small details are reliably improved.

In the watercolour example, the full generation (right side) has the texture of the watercolour paper, and the partial generation (left side) has a more flat digital art look to it.

For the blacksmith robot, the sparks emitted from the operation have a more natural blend to it. The robot's clothing appears to be undergoing some interesting transformation due to the undertrained state of the weights.

The medieval  battle image has improved blades of grass, settling dust particles, and fabric in the flag.

The stage 2 model being trained does not seem to resolve any global coherence issues despite having 400 steps in its schedule, but it still noticeably changes the local coherence, eg. the consistency of fabrics and metals can be improved through stage 2 fine-tuning.

The stage 1 model is the workhorse of the output, as expected with the 600 timesteps in its schedule. Additional fine-tuning of this model will improve the overall global coherence of the outputs. I wish I could say it will not impact fine details, but a lot of that does seem to be carried forward.

As noted, these models are undertrained due to a lack of compute. But they are a promising look toward what an e-diffi PixArt might be capable of.

Does anyone want to build this out fully with me?


Check this out on Poe
StableDiffusionXL Prompt Generator
: https://poe.com/sdpai


We Got a Job Offer in SECourses Discord Channel Related to AI (Stable Diffusion)

This is kind of announcement sharing. I think the offer looks decent.

For who doesn’t know our channel here : https://discord.com/servers/software-engineering-courses-secourses-772774097734074388

The job offer is in ai-related-job-offers sub-channel


I like training LoRAs

https://huggingface.co/blog/nroggendorff/create-diffusers-dataset

Malumatra Blane187 merve AtAndDev Nymbo prithivMLmods


Websites slam doors on AI data harvesting 🚪🔒

New study "Consent in Crisis: The Rapid Decline of the AI Data Commons" reveals a rapid decline in open web access.

Key findings from 14,000 web domains audit:
- +5% of three common data sets (C4, RefinedWeb and Dolma) now fully restricted, +25% of the highest-quality sources now fully restricted
- 45% of C4 restricted by Terms of Service

Noteworthy trends:
🚫🔄 OpenAI banned 2x more than any other company
📰🔐 News sites leading restrictions: 45% of tokens off-limits

Two quotes in the NYT piece to ponder:

“Unsurprisingly, we’re seeing blowback from data creators after the text, images and videos they’ve shared online are used to develop commercial systems that sometimes directly threaten their livelihoods.” — @yjernite

“Major tech companies already have all of the data. Changing the license on the data doesn’t retroactively revoke that permission, and the primary impact is on later-arriving actors, who are typically either smaller start-ups or researchers.” — @stellaathena

👉 Dive into the research: https://www.dataprovenance.org/consent-in-crisis-paper
👉 Read the NYT story: https://www.nytimes.com/2024/07/19/technology/ai-data-restrictions.html

#AIEthics #DataPrivacy


InSPyReNet Background Removal

I've built a space for fast background removal.

- https://huggingface.co/spaces/gokaygokay/Inspyrenet-Rembg

- https://github.com/plemeri/InSPyReNet

John6666 DonkeySmall den0620 Ramikan-BR sxnr1 Rsln DIvAndrey Wok jeffcookio MoAusaf


New feature 🔥
Image models and LoRAs now have little previews 🤏

If you don't know where to start to find them, I invite you to browse cool LoRAs in the profile of some amazing fine-tuners: @artificialguybr, @alvdansen, @DoctorDiffusion, @e-n-v-y, @KappaNeuro @ostris

GPT007 osanseviero ayush7 John6666 Blane187 bghira artificialguybr Rsln edpresque DoctorDiffusion cbensimon OmbelineM


Chameleon: Mixed-Modal Early-Fusion Foundation Models (2405.09818)

Chameleon 🦎 by Meta is now available in Hugging Face transformers 😍
A vision language model that comes in 7B and 34B sizes 🤩
But what makes this model so special?

Demo: https://huggingface.co/spaces/merve/chameleon-7b
Models: https://huggingface.co/collections/facebook/chameleon-668da9663f80d483b4c61f58

keep reading ⥥

Chameleon is a unique model: it attempts to scale early fusion 🤨
But what is early fusion?
Modern vision language models use a vision encoder with a projection layer to project image embeddings so it can be promptable to text decoder (LLM)

Early fusion on the other hand attempts to fuse all features together (image patches and text) by using an image tokenizer and all tokens are projected into a shared space, which enables seamless generation 😏

Authors have also introduced different architectural improvements (QK norm and revise placement of layer norms) for scalable and stable training and they were able to increase the token count (5x tokens compared to Llama 3 which is a must with early-fusion IMO)

This model is an any-to-any model thanks to early fusion: it can take image and text input and output image and text, but image generation are disabled to prevent malicious use.

One can also do text-only prompting, authors noted the model catches up with larger LLMs (like Mixtral 8x7B or larger Llama-2 70B) and also image-pair prompting with larger VLMs like IDEFICS2-80B (see paper for the benchmarks https://huggingface.co/papers/2405.09818)
Thanks for reading!

sumandas atahanuz Nymbo ucsahin GPT007 fdaudens osanseviero alkzar90 dblasko Svngoku gokaygokay mikestaub abdulbasit-nubytes


Sparse MoE (SMoE) has an unavoidable drawback: the performance of SMoE heavily relies on the choice of hyper-parameters, such as the number of activated experts per token (top-k) and the number of experts.

Also, identifying the optimal hyper-parameter without a sufficient number of ablation studies is challenging. As the size of the models continues to grow, this limitation could result in a significant waste of computational resources, and in turn, could hinder the efficiency of training MoE-based models in practice.

(READ MORE ↓↓↓) Now, our DynMoE addresses these challenges! 🙌 DynMoE incorporates:
(1) a novel gating method that enables each token to automatically determine the number of experts to activate.

(2) An adaptive process automatically adjusts the number of experts during training. Extensive numerical results across Vision, Language, and Vision-Language tasks demonstrate the effectiveness of our approach to achieve competitive performance compared to GMoE for vision and language tasks, and MoE-LLaVA for vision-language tasks, while maintaining efficiency by activating fewer parameters.

Our code is available at https://github.com/LINs-lab/DynMoE, also see the checkpoints at https://huggingface.co/collections/LINs-lab/dynmoe-family-665ed5a331a7e84463cab01a

surajkachate123 GPT007 osanseviero Ramikan-BR hllj danielus flflow kenshinn Yongxin-Guo


Since it is release season, at PleIAs we announce our first suite of specialized language models for document processing tasks (OCR correction, text segmentation, bibliographic extraction) and the release of the largest multimodal dataset of financial document Finance Commons: https://huggingface.co/blog/Pclanglais/finance-commons-bad-data-toolbox

LLM research is currently focused on quality data. We went on the opposite direction and voluntarily trained models on bad data. Far from degrading models, it made them more resilient to text sources commonly used in production.

Having a wider range of real life data proved critical for this project. A few months after the release of Common Corpus, we expanded our pool of "training data commons" with a major multimodal ressource: document released as open financial data.  Finance commons comprises 17 billion tokens and 1.25 PDF corporate documents released by the SEC, WTO, AMF, EU Tenders In a multiple languages with a large variety of document layouts and challenging sources to train more robust models.

With HuggingFace compute support, we release an entire pipeline to process bad data sources and make them usable in production for LLMOps or simply retrieval: https://huggingface.co/spaces/PleIAs/PleIAs-Editor

This approach is based on our new series of specialized models for document processing, the "bad data toolbox" comprising:
*OCRonos, the best available model to date for OCR correction. https://huggingface.co/PleIAs/OCRonos
*Segmentext, a pure semantic small model for text segmentation, working without any visual reference. https://huggingface.co/PleIAs/Segmentext
*Bibtexer, a small model for bibliographic data extraction acting as a "reversed-Zotero." https://huggingface.co/PleIAs/BibTexer


Since new TTS (Text-to-Speech) systems are coming out what feels like every day, and it's currently hard to compare them, my latest project has focused on doing just that.

I was inspired by the https://huggingface.co/spaces/TTS-AGI/TTS-Arena (definitely check it out if you haven't), which compares recent TTS system using crowdsourced A/B testing.

I wanted to see if we can also do a similar evaluation with objective metrics and it's now available here:
https://huggingface.co/spaces/ttsds/benchmark
Anyone can submit a new TTS model, and I hope this can provide a way to get some information on which areas models perform well or poorly in.

The paper with all the details is available here: https://arxiv.org/abs/2407.12707

ZeroWw osanseviero John6666 Gatozu35 ayymen Ramikan-BR Dihelson MarinaraSpaghetti mrfakename Korakoe kramp

krushnabhosle Gatozu35 Ramikan-BR Dihelson Korakoe


Thrilled to share some AI insights for journalism! 📊🤖

Just wrote a guest post on @ndiakopoulos's blog about Hugging Face on Sheets tool.

Why it matters:
🔌 Brings AI power directly to spreadsheets
📈 Huge potential for data journalism
🚫💻 No coding required!

If you've read Nicholas' "Automating the News" (a must-read!), you'll appreciate how this tool fits into the evolving landscape of AI in journalism.

Read here: https://generative-ai-newsroom.com/bringing-open-source-models-to-spreadsheets-c440fc4818b4

#AIJournalism #DataJournalism


🇷🇸 New Benchmark for Serbian Language 🇷🇸

@DjMel and I recently released a new benchmark for Serbian language that measures General Knowledge of LLMs. We had to parse over 20 years of university entrance exams for University of Belgrade, so the dataset is of high quality.

🥇 OAI models still hold the podium places with a significant gap compared to open-source models
🤔 https://huggingface.co/Qwen/Qwen2-7B-Instruct and https://huggingface.co/VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct models show promising results considering they weren't trained on Serbian language
📈 Best open-source model seems to be https://huggingface.co/Stopwolf/Mustra-7B-Instruct-v0.2, a merge between https://huggingface.co/gordicaleksa/YugoGPT and https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2
📉 Some models like https://huggingface.co/google/gemma-2-9b-it turned out to be a disappointment with random guessing-like accuracy

Take a look at the whole results at the dataset page:
https://huggingface.co/datasets/DjMel/oz-eval

P.S. If you have any constructive criticism or ideas for improvement, feel free to use dataset's Discussions page!


This is the week of small AI language models!

ZeroWw rumbleFTW ZachZimm osanseviero GPT007 chenduo jpacifico thesven AtAndDev Fredithefish

Bharath182004 YaTharThShaRma999 rumbleFTW osanseviero GPT007 jessicagab AtAndDev louisbrulenaudet Fredithefish


🚀 Just released version 0.24.0 of the 𝚑𝚞𝚐𝚐𝚒𝚗𝚐𝚏𝚊𝚌𝚎_𝚑𝚞𝚋 Python library!

Exciting updates include:
⚡ InferenceClient is now a drop-in replacement for OpenAI's chat completion!

✨ Support for response_format, adapter_id , truncate, and more in InferenceClient

💾 Serialization module with a save_torch_model helper that handles shared layers, sharding, naming convention, and safe serialization. Basically a condensed version of logic scattered across safetensors, transformers , accelerate

📁 Optimized HfFileSystem to avoid getting rate limited when browsing https://huggingface.co/datasets/HuggingFaceFW/fineweb

🔨 HfApi & CLI improvements: prevent empty commits, create repo inside resource group, webhooks API, more options in the Search API, etc.

Check out the full release notes for more details:
https://huggingface.co/spaces/Wauplin/huggingface_hub/discussions/7
 👀

lhoestq plaguss clem Nymbo not-lain Goekdeniz-Guelmez


I have invented a method that is better than Diffusion. A company got a billion dollar valuation yesterday for less than what I am currently giving the world for free. I am starting to suspect that is the issue, I am giving it away for free. I meant it to be a gift to the world, but no one will even look at it. I am changing the licensing soon. It will no longer be free. At the moment, you can view exactly how Swarm Neural Networks can do everything Reverse Diffusion can do, for far less money. It can even make API calls.

SNN Image Generator: https://huggingface.co/spaces/TuringsSolutions/SNN-Image-Generator

SNN Function Caller (Controlled By TinyLlama): https://huggingface.co/spaces/TuringsSolutions/Qwen-2.0.5B-Swarm-Function-Caller

victor Augustindroid LeroyDyer Kkordik den0620 Joseph717171


#  Offensive Physical  Security Reconnaissance Planning Automation with public facing RTSP streams and Moondream

After some late night casual hacking about on VLMs for criminal attack vector reconnaissance automaton  experiments using Moondream (as usual)  based image-text-text with pre defined text prompts that are tuned for extracting weakness or customer identity and monitory  based   theft physical red team engagement reconnaissance and vector of malicious or criminal activity Working on a space. Thanks again for such a wonderful blessing of super power  image-text-to-text  model with minimal computational power needed @vikhyatk

I have started actually implementing a custom little tool with both static html space sand python gradio spaces  on the go which I shall share as  hf spaces when done  them.

---

https://huggingface.co/spaces/vikhyatk/moondream2

https://huggingface.co/vikhyatk/moondream2


Excited to share my "Focus Mode" Playlist, code name "The AI/ML Researcher's Playlist" :)

No lyrics, no beat, just a harmonious sequence of piano melody that will take you places beyond your reasoning/thinking prowess, trust me I’ve been there lol 🎹🎶

Thanks to an amazing pianist and composer on instagram @andreavanzo_composer who played all the songs in this playlist.

Currently have a total of 16 songs, I will keep adding more when I find them.

Full playlist: https://youtu.be/2ccxanKmzZY?si=x6weX2AgY5Zpadfw


GraphRAG-Ollama-UI

I've been working on a local version of Microsoft's GraphRAG that uses Ollama for everything. It's got a new interactive UI built with Gradio that makes it easier to manage data, run queries, and visualize results. It's not fully featured or set up to harness the entire GraphRAG library yet but it allows you to run all the standard commands for Indexing/Processing and chatting with your graph. Some key features:

Uses local models via Ollama for LLM and embeddings

3D graph visualization of the knowledge graph using Plotly

File management through the UI (upload, view, edit, delete)

Settings management in the interface

Real-time logging for debugging

https://github.com/severian42/GraphRAG-Ollama-UI

lucianosb Saugatkafley 200-cooller hubhamgupta AlbertSadday2891 Ruby-NewLeaf taufiqdp victor den0620


#ICLM 2024 is almost there 🔥🔥🔥 PM if you will be in Vienna next week, Glad to catchup with the  Hugging Face community there!

I would like to contribute 🎁 by releasing the sixth Knowledge Vault, with 100 lectures visualized from the last 10 years of ICML from 2014 to 2024, (10 from 2024 will be included after the conference) including knowledge graphs for all the Invited Lectures and some extras, with almost 3000 topics represented using AI.

You can explore it here:
🌏 https://theendofknowledge.com/Vaults/6/ICML-2015-2024.html

And you can learn more about the Vaults here:
📝https://www.linkedin.com/pulse/knowledge-vaults-david-vivancos-lbjef/

And previous Vaults relevant to the #huggingface community are:

🌏 [ @lexfridman 2018-2024 Interviews] https://theendofknowledge.com/Vaults/1/Lex100-2024.html

🌏 [ICLR 2014-2023] https://theendofknowledge.com/Vaults/2/ICLR2014-2023.html

🌏 [AIForGood 2017-2024] https://theendofknowledge.com/Vaults/4/AIForGood2017-2024.html

🌏 [CVPR 2015-2024] https://theendofknowledge.com/Vaults/5/CVPR-2015-2024.html

Hope you like them!

And great to see you all at #icml2024 @clem @thomwolf @julien-c and team

clem victor Rusvasul osanseviero louisbrulenaudet julien-c


Using the new viewer iframe support for the datasets viewer, I built a simple Space https://huggingface.co/spaces/davanstrien/collection_dataset_viewer to quickly explore all the datasets inside a collection.

The collection is loaded from an environment variable, so you can duplicate this Space to create a Space for exploring datasets in another collection!

Alexandro14 yjernite John6666 lucianosb Ramikan-BR Nymbo osanseviero


palmer-004 becomes 🔥turbo🔥  now is half the size, twice the speed and the best overall 0.5b language model in huggingface.

https://huggingface.co/appvoid/palmer-004-turbo


🤯 Ghost 8B Beta emerges as a clear leader, surpassing even proprietary models like xAI Grok 1, OpenAI GPT 3.5, and Mistral Mixtral 8x7B. This dominance extends to its parity with Mistral Medium, further solidifying its position as a top-tier language model. Furthermore, Ghost 8B Beta stands out as one of only three models employing the zero-shot method for evaluation, alongside Claude 2 and Claude 3, showcasing its unique capabilities and potential for groundbreaking applications.
---
💬 Chat with the model here:
- Playground with Ghost 8B Beta (β, 8k): https://huggingface.co/spaces/lamhieu/ghost-8b-beta-8k
- Playground with Ghost 8B Beta (β, 128k): https://huggingface.co/spaces/lamhieu/ghost-8b-beta-128k
- Official website: https://ghost-x.org/docs/models/ghost-8b-beta/

ZeroWw abidlabs tuanio reach-vb Dihelson Alexandro14 amosgyamfi cnmoro nicolollo GPT007 louisbrulenaudet


No on really pays attention to voyage ai , but probably you should 😉
https://huggingface.co/voyageai


Decensored Gemma2-27b

https://huggingface.co/TheDrummer/Big-Tiger-Gemma-27B-v1


Micrograd in pure C🤕
Port of Karpathy's micrograd in pure C.
Yo C does not negotiate with memory 😂
Code: https://github.com/Jaykef/micrograd.c

reach-vb jeffboudier abidlabs nbroad not-lain den0620 DIvAndrey


I've made a creative version of Tile Upscaler

- https://huggingface.co/spaces/gokaygokay/TileUpscalerV2

- https://github.com/gokayfem/Tile-Upscaler

- New tiling strategy
- Now it's closer to Clarity Upscaler
- It has more parameters to play and it has more room to fail because of that
- You should try different resolutions, strength and controlnet strength

Original Tile Upscaler
- https://huggingface.co/spaces/gokaygokay/Tile-Upscaler

reach-vb osanseviero jeffboudier abidlabs Blane187 louisbrulenaudet Alexandro14 John6666 HaiQinQin Wok Itsjustkev saxash jgitsolutions

John6666 reach-vb osanseviero abidlabs HaiQinQin kramp mah002 Wok


What an eventful day in Open Source LLMs today:

Mistral released Codestral Mamba 🐍
> Beats DeepSeek QwenCode, best model < 10B, competitive with Codestral 22B
> Mamba 2 architecture - supports up to 256K context
> Apache 2.0 licensed, perfect for local code assistant
> Transformers & llama.cpp integration upcoming!

Model checkpoint: https://huggingface.co/mistralai/mamba-codestral-7B-v0.1

Hugging Face dropped SmolLM 🤏
> Beats MobileLLM, Qwen 0.5B, Phi 1.5B and more!
> 135M, 360M, and 1.7B param model checkpoints
> Trained on 600B high-quality synthetic + FineWeb Edu tokens
> Architecture: Llama + GQA + 2048 ctx length
> Ripe for fine-tuning and on-device deployments.
> Works out of the box with Transformers!

Model checkpoints: https://huggingface.co/collections/HuggingFaceTB/smollm-6695016cad7167254ce15966

Mistral released Mathstral 7B ∑
> 56.6% on MATH and 63.47% on MMLU
> Same architecture as Mistral 7B
> Works out of the box with Transformers & llama.cpp
> Released under Apache 2.0 license

Model checkpoint: https://huggingface.co/mistralai/mathstral-7B-v0.1

Pretty dope day for open source ML. Can't wait to see what the community builds with it and to support them further! 🤗

What's your favourite from the release today?

reach-vb Presidentlin osanseviero dvilasuero louisbrulenaudet nawed JackCloudman ajibawa-2023 Alexandro14 tharrmeehan Mdubbya lucianosb WOOSAH nbroad


What could have been

"{{bos_token}}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '<|user|>\n' + message['content'] + '<|end|>\n' }}{% elif message['role'] == 'assistant' %}{{ '<|bot|>\n' + message['content'] }}{% if loop.last %}{{ '<|end|>' + eos_token }}{% else %}{{ '<|end|>\n' }}{% endif %}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}"


𝐊𝐞𝐲 𝐟𝐞𝐚𝐭𝐮𝐫𝐞𝐬

𝐊𝐞𝐲 𝐟𝐞𝐚𝐭𝐮𝐫𝐞𝐬

𝐏𝐨𝐭𝐞𝐧𝐭𝐢𝐚𝐥 𝐮𝐬𝐞 𝐜𝐚𝐬𝐞𝐬

𝐏𝐨𝐭𝐞𝐧𝐭𝐢𝐚𝐥 𝐮𝐬𝐞 𝐜𝐚𝐬𝐞𝐬

Small models, BIG impact: SmolLM is here! 🚀🔬

We're launching a series of small but mighty language models:
🏎 Super fast - runs on laptops, phones, you name it!
📏 3 sizes: 130M, 350M, and 1.5B parameters
🥇 Outperforms same size models from Meta, Microsoft, and Qwen
🔓 Fully open-source: datasets, training code, models

𝐊𝐞𝐲 𝐟𝐞𝐚𝐭𝐮𝐫𝐞𝐬
- Trained on FineWeb-Edu and Cosmopedia v2 (largest synthetic pre-training dataset)
- No cloud needed - run locally for privacy and energy efficiency
- Everything is public, from data curation to training steps

𝐏𝐨𝐭𝐞𝐧𝐭𝐢𝐚𝐥 𝐮𝐬𝐞 𝐜𝐚𝐬𝐞𝐬
- On-device autocomplete
- Local request parsing
- Custom fine-tuning for specific needs without the need for expensive GPUs

𝐆𝐨 𝐝𝐞𝐞𝐩𝐞𝐫
👉 Check it out: https://huggingface.co/collections/HuggingFaceTB/smollm-models-6695016cad7167254ce15966
👉 Run the 360M model in your browser, 100 % private: https://huggingface.co/spaces/HuggingFaceTB/SmolLM-360M-Instruct-WebGPU
👉 Read the blog explaining everything in detail: huggingface.co/blog/smollm

Kudos to the stellar team who worked on this project: @loubnabnl @anton-l @eliebak @lvwerra

YaTharThShaRma999 sgarbi sivan22 mdouglas nroggendorff John6666 reach-vb plaguss gabrielmbmb osanseviero jeffboudier AyoubChLin holooo raveninrhythm Ramikan-BR clem yjernite louisbrulenaudet

surfhb reach-vb osanseviero FaultyEntry ucsahin raveninrhythm Ramikan-BR clem yjernite


Cool things this week from @huggingface!

🌎AI math olympiad winner NuminaMath is here!
🤗Announcing New Hugging Face and Keras NLP integration
✨UI overhaul to HF tokens!
🧊 Embed our dataset viewer on any webpage!

https://huggingface.co/blog/winning-aimo-progress-prize
https://huggingface.co/blog/keras-nlp-integration
https://huggingface.co/settings/tokens
https://x.com/julien_c/status/1812099420726456457

Check out the full list on our discord! 👇
https://discord.com/invite/JfAtkvEtRb

reach-vb plaguss osanseviero jeffboudier not-lain yjernite louisbrulenaudet Blane187 apehex


The community Journalists on HuggingFace recently launched a tool (https://huggingface.co/spaces/JournalistsonHF/text-to-image-bias) to compare biases across several text-to-image models. I forked my own to evaluate the SDXL models I made.

## SinteticoXL Bias:

https://huggingface.co/spaces/lucianosb/sinteticoXL-bias


🔴⭐ New addition to the existing concept space! 🔴⭐

🏞 Space: https://huggingface.co/spaces/prithivMLmods/IMAGINEO-4K

🚀 Tried the Duotone Canvas with the image generator. Unlike the duotone filter in the Canva app, which applies hue and tints in RGBA, this feature applies duotones based purely on the provided prompt to personalize the generated image.

🚀 These tones also work with the gridding option, which already exists in the space.

🚀 The application of tones depends on the quality and detail of the prompt given. The palette may be distorted in some cases.

🚀It doesn't apply like a hue or tint in RGBA (as shown in canva app below); it is purely based on the prompts passed.

🏞 Check out the space: https://huggingface.co/spaces/prithivMLmods/IMAGINEO-4K
🏜Collection: https://huggingface.co/collections/prithivMLmods/collection-zero-65e48a7dd8212873836ceca2

```
huggingface.co/spaces/prithivMLmods/IMAGINEO-4K
```
🏞What you can do with this space:
✅ Compose Image Grid
👉🏻 "2x1", "1x2", "2x2", "2x3", "3x2", "1x1"
✅ Apply styles
✅ Set up Image tones
✅ Apply filters & adjust quality

.
.
.
Thanks for reading!
- @prithivMLmods

reach-vb KingNish GPT007 prithivMLmods NanyTVZ19 Ngrthm


Introducing Plugins in NiansuhAI (on July 20, 2024)

Plugin Names:
1. WebSearch: Tool for searching the web using search engines.
2. Calculator: Helps evaluate mathematical expressions; extends the base Tool class.
3. WebBrowser: Interacts with web pages to extract information or summarize content.
4. Wikipedia: Retrieves data from Wikipedia using its API.
5. Arxiv: Searches and fetches article information from Arxiv.
6. WolframAlphaTool: Answers questions on Math, Science, Technology, Culture, Society, and Everyday Life.

Similar to https://hf.co/chat

Niansuh renjithkathirolil lunarflu ucsahin reach-vb


IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans
  from Monocular Videos via Explicit Ray Tracing (2312.05210)

🚀🕺🌟 New Research Alert - CVPR 2024 (Avatars Collection)! 🌟💃🚀
📄 Title: IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans from Monocular Videos via Explicit Ray Tracing 🔝

📝 Description: IntrinsicAvatar is a method for extracting high-quality geometry, albedo, material, and lighting properties of clothed human avatars from monocular videos using explicit ray tracing and volumetric scattering, enabling realistic animations under varying lighting conditions.

👥 Authors: Shaofei Wang, Božidar Antić, Andreas Geiger, and Siyu Tang

📅 Conference: CVPR, Jun 17-21, 2024 | Seattle WA, USA 🇺🇸

🔗 Paper: https://huggingface.co/papers/2312.05210

🌐 Github Page: https://neuralbodies.github.io/IntrinsicAvatar/
📁 Repository: https://github.com/taconite/IntrinsicAvatar

📺 Video: https://www.youtube.com/watch?v=aS8AIxgVXzI

🚀 CVPR-2023-24-Papers: https://github.com/DmitryRyumin/CVPR-2023-24-Papers

🚀 WACV-2024-Papers: https://github.com/DmitryRyumin/WACV-2024-Papers

🚀 ICCV-2023-Papers: https://github.com/DmitryRyumin/ICCV-2023-Papers

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #IntrinsicAvatar #InverseRendering #MonocularVideos #RayTracing #VolumetricScattering #3DReconstruction #MachineLearning  #ComputerVision #DeepLearning  #AI #CVPR2024

DmitryRyumin prithivMLmods ClayFace Ramikan-BR reach-vb Alexandro14


mrfakename MandyGreatness louisbrulenaudet Tonic kramp osanseviero s3nh GPT007 enzostvs SeanCLucas d0rj hayden-donnelly adamelliotfields Taylor658 gabrielmbmb RoboApocalypse reach-vb jeffboudier Blane187 Ramikan-BR HDiffusion zhlh ThatOneCoder Hev832 Haleshot mmhamdy Fredithefish nroggendorff rmuthiah13 mvandermeulen createtheimaginable

prithivMLmods Tonic BiniyamAjaw osanseviero s3nh reach-vb jeffboudier Ramikan-BR Fredithefish

ijohn07 den0620 reach-vb dashfunnydashdash Ramikan-BR ThatOneCoder Tonic


- 6 languages: 🇨🇳 🇺🇸 🇯🇵 🇰🇷 🇩🇪 🇫🇷

- 6 languages: 🇨🇳 🇺🇸 🇯🇵 🇰🇷 🇩🇪 🇫🇷

Exciting news for audio AI enthusiasts! 🎙🌍

The Emilia dataset dropped last week, and it's a cool one:
- 101k+ hours of high-quality audio
- 6 languages: 🇨🇳 🇺🇸 🇯🇵 🇰🇷 🇩🇪 🇫🇷
- Diverse content: talk shows, interviews, debates, sports commentary, audiobooks

This dataset could improve multilingual speech generation and recognition. Opens up many possibilities for global media, language learning, and accessibility!

Explore it: https://huggingface.co/datasets/amphion/Emilia

#AIAudio

YaTharThShaRma999 ajibawa-2023 maywell osanseviero Dihelson John6666 reach-vb louisbrulenaudet ngphuchoangnam

Dihelson reach-vb HarryHe ngphuchoangnam

Dihelson HyperBlaze reach-vb ngphuchoangnam


🔥 🔥🔥
Excited to announce WizardLM new Paper: Auto Evol-Instruct!

🐦 Twitter: https://x.com/WizardLM_AI/status/1812857977122202087

📃 Paper: https://arxiv.org/pdf/2406.00770

🤖 1. Fully AI-Powered Pipeline

Auto Evol-Instruct automatically involves an iterative process of optimizing an Evol-Instruct V1 into an optimal one. The pipeline consists of two critical stages: Evol Trajectory Analysis, where the optimizer LLM analyzes the issues and failures exposed in instruction evolution performed by the evol LLM, and Evolving Method Optimization, where the optimizer LLM addresses these issues to progressively develop an effective evolving method. The optimal evolving method is then used to convert the entire instruction dataset into more diverse and complex forms, facilitating improved instruction tuning.

📈2. Scaling Evol-Instruct with Arena Learning

With Auto Evol-Instruct, the evolutionary synthesis data of WizardLM-2 has scaled up from WizardLM-1 to dozens of domains, covering tasks in all aspects of large language models. This allows Arena Learning to train and learn from an almost infinite pool of high-difficulty instruction data, fully unlocking all the potential of Arena Learning.

WizardLM YaTharThShaRma999 John6666 AndrewZeng ajibawa-2023 osanseviero mmhamdy Best-codes KingNish AbheekG sohampnow reach-vb Alexandro14 KakaRotting

AndrewZeng mmhamdy AbheekG TitleOS reach-vb davickyz Josiah654

AbheekG reach-vb liyongsea 1ucky1uke davickyz Josiah654


🚀 Meet Our New Line of Efficient and Accurate Zero-Shot Classifiers! 🚀

The new architecture brings better inter-label understanding and can solve complex classification tasks at a single forward pass.

Key Applications:
✅ Multi-class classification (up to 100 classes in a single run)
✅ Topic classification
✅ Sentiment analysis
✅ Event classification
✅ Prompt-based constrained classification
✅ Natural Language Inference
✅ Multi- and single-label classification

https://huggingface.co/collections/knowledgator/gliclass-6661838823756265f2ac3848
https://huggingface.co/spaces/knowledgator/GLiClass_SandBox
https://huggingface.co/knowledgator/gliclass-base-v1.0-lw


@seyonec It's a great help to experiment with ChemBERTa.

BTW, There are several models that handle SMILES in the model repository. Can you kindly recommend the one with the best performance in handling hERG dataset?
https://paperswithcode.com/dataset/herg

Best,
Joo-Haeng Lee, Pebblous Inc.
http://pebblous.ai


@theroot srry i accidentally tried to join your group-of-one, pls ignore :) but ur work seems super interesting!


Updated https://huggingface.co/blog/nroggendorff/train-with-llama-architecture so you can "train" your own tokenizer from your dataset.


New model drop...🥁

FROSTING LANE REDUX

The v1 of this model was released during a big model push, so I think it got lost in the shuffle. I revisited it for a project and realized it wasn't inventive enough around certain concepts, so I decided to retrain.

https://huggingface.co/alvdansen/frosting_lane_redux

I think the original model was really strong on it's own, but because it was trained on fewer images I found that it was producing a very lackluster range of facial expressions, so I wanted to improve that.

The hardest part of creating models like this, I find, is maintaining the detailed linework without without overfitting. It takes a really balanced dataset and I repeat the data 12 times during the process, stopping at the last 10-20 epochs.

It is very difficult to predict the exact amount of time needed, so for me it is crucial to do epoch stops. Every model has a different threshold for ideal success.

YaTharThShaRma999 AtAndDev NHLOCAL plrthink Alexandro14 enzostvs

prithivMLmods AtAndDev louisbrulenaudet plrthink


RodinHD: High-Fidelity 3D Avatar Generation with Diffusion Models (2407.06938)

🔥🎭🌟 New Research Alert - ECCV 2024 (Avatars Collection)! 🌟🎭🔥
📄 Title: RodinHD: High-Fidelity 3D Avatar Generation with Diffusion Models 🔝

📝 Description: RodinHD generates high-fidelity 3D avatars from portrait images using a novel data scheduling strategy and weight consolidation regularization to capture intricate details such as hairstyles.

👥 Authors: Bowen Zhang, @yiji, @chunyuwang, Ting Zhang, @jiaolong, Yansong Tang, Feng Zhao, Dong Chen, and Baining Guo

📅 Conference: ECCV, 29 Sep – 4 Oct, 2024 | Milano, Italy 🇮🇹

📄 Paper: https://huggingface.co/papers/2407.06938

🌐 Github Page: https://rodinhd.github.io/
📁 Repository: https://github.com/RodinHD/RodinHD

📺 Video: https://www.youtube.com/watch?v=ULvHt7dZx-Q

🚀 CVPR-2023-24-Papers: https://github.com/DmitryRyumin/CVPR-2023-24-Papers

🚀 WACV-2024-Papers: https://github.com/DmitryRyumin/WACV-2024-Papers

🚀 ICCV-2023-Papers: https://github.com/DmitryRyumin/ICCV-2023-Papers

📚 More Papers: more cutting-edge research presented at other conferences in the https://huggingface.co/spaces/DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin

🚀 Added to the Avatars Collection: https://huggingface.co/collections/DmitryRyumin/avatars-65df37cdf81fec13d4dbac36

🔍 Keywords: #RodinHD #3DAvatars #DiffusionModels #HighFidelity #PortraitTo3D #MachineLearning  #ComputerVision #DeepLearning  #AI #ECCV2024


Responsibly building AI also means knowing its impact on the environment and the hidden carbon costs associated with it🌱
If you're interested in the subject, you can check out my latest community article: https://huggingface.co/blog/as-cle-bert/is-ai-carbon-footprint-worrisome
Where I try to unravel AI's carbon footprint and potential solutions to reduce it🌻
Enjoy!🤗


Introducing: 'Synthetic Math Phi'! Literally just press a button, and receive 10 random math problems and 10 random answers, outputted in JSON format and ready to be fed back to an LLM model.

This app is inspired by the research in this research paper. Their findings show that in order to increase mathematical reasoning of models, we need to give them more math. Quality is kind of not the important factor (for math only). Not enough math exists currently to train the models, that is the bottleneck. So, press a button and generate some math for free!

MIT licensed, here is the Github: https://github.com/RichardAragon/Synthetic-Math-Phi


Check out this SD3 Inpainting space I've created:
https://huggingface.co/spaces/Kkordik/inpainting-sd3-sketch-pad


Large models are surprisingly bad storytellers.

I asked 8 LLMs to "Tell me a bedtime story about bears and waffles."

Claude 3.5 Sonnet and GPT-4o gave me the worst stories: no conflict, no moral, zero creativity.

In contrast, smaller models were quite creative and wrote stories involving talking waffle trees and bears ostracized for their love of waffles.

Here you can see a comparison between Claude 3.5 Sonnet and NeuralDaredevil-8B-abliterated. They both start with a family of bears but quickly diverge in terms of personality, conflict, etc.

I mapped it to the hero's journey to have some kind of framework. Prompt engineering can definitely help here, but it's still disappointing that the larger models don't create better stories right off the bat.

Do you know why smaller models outperform the frontier models here?

ijohn07 pabloce AtAndDev GPT007 KingNish ThatOneCoder takeraparterer alexb02 kirada Dihelson sosoai lordvader31 notimetosleep SicariusSicariiStuff VictorNanka PrimeD omarei ruben-wleon AIismine LyAA Wauplin Tonic DIvAndrey roccotool tgup mufeed eldogbbhed giovanibarcelos thanhkt

FlareRebellion AtAndDev prithivMLmods GPT007 ThatOneCoder takeraparterer alexb02 adamelliotfields den0620 Dihelson danielus Tonic

AtAndDev GPT007 ThatOneCoder takeraparterer alexb02 Tonic giovanibarcelos


🎉 The Ghost 8B Beta model outperforms prominent models such as Llama 3 8B Instruct, GPT 3.5 Turbo in the lc_winrate score. In addition, it also outperforms Claude 3 Opus, Claude 3 Sonnet, GPT-4, and Mistral Large when comparing the winrate score of AlpacaEval 2.0.

Ghost 8B Beta is a large language model developed with goals that include excellent multilingual support, superior knowledge capabilities, and cost-effectiveness. The model comes in two context length versions, 8k and 128k, along with multilingual function tools support by default.
The languages supported are 🇺🇸 English, 🇫🇷 French, 🇮🇹 Italian, 🇪🇸 Spanish, 🇵🇹 Portuguese, 🇩🇪 German, 🇻🇳 Vietnamese, 🇰🇷 Korean and 🇨🇳 Chinese.

Explore the Potential:
To learn more about this groundbreaking language model, visit the official website or explore the online demo platforms:
- Ghost 8B Beta (β, 8k) on Spaces: https://huggingface.co/spaces/lamhieu/ghost-8b-beta-8k.
- Ghost 8B Beta (β, 128k) on Spaces: https://huggingface.co/spaces/lamhieu/ghost-8b-beta-128k
- Official website: https://ghost-x.org/docs/models/ghost-8b-beta

prithivMLmods PatzEdi John6666 GPT007 darxkies AtAndDev Sergidev ngxson Ramikan-BR Vibeon letheviet ajibawa-2023

truc-nmt rreed-pha GPT007 darxkies AtAndDev Ramikan-BR Vibeon

deus-ex-machina Tom-Neverwinter Ramikan-BR maywell Vibeon


Use GPT-4, GPT-4 Turbo Preview, GPT-3.5 Turbo, BingAI, and other models. The interface is similar to ChatGPT, with a speedy API endpoint.

https://huggingface.co/spaces/NiansuhAI/Copilot


Announcing the second open model in our Aura series of media models at @fal: https://hf.co/fal/AuraFlow

Try it using diffusers or ComfyUI from publicly available weights, and read more about it in our blog https://blog.fal.ai/auraflow.

YaTharThShaRma999 KingNish dblasko impactframes DiamanteAmarelo gokaygokay

louisbrulenaudet YaTharThShaRma999 DiamanteAmarelo gokaygokay


Hello!

I've been in the lab synthesizing captions, with my trusty sidekick Blip, and along the way I had an interesting idea. I thought of designing an incredibly simple model that accepts simple instruction pairs, adjective noun pairs specifically, and outputs 2d vertices.

The current implementation has been implemented by myself then ran over with Claude, not because I am incompetent, but because I recognize tools written by experts may have more technique than my newbie self.

As with all projects, this will be updated with proportion to the feedback received, if someone's using it and wants to keep using it, i'm happy to keep working on anything. Thanks, all! 🤗

`-<3`

https://colab.research.google.com/gist/SMeyersMrOvkill/8d4686db803f6c5f43fafc1c94b1c8c6/polypathdelement.ipynb


Started a list of LLMs used to make HackerNoon happen:
https://huggingface.co/collections/HackerNoon/models-used-in-hackernoon-publishing-system-668c56a0d10c3be5d338b805


🚀 I'm excited to announce that huggingface_hub's InferenceClient now supports OpenAI's Python client syntax! For developers integrating AI into their codebases, this means you can switch to open-source models with just three lines of code. Here's a quick example of how easy it is.

Why use the InferenceClient?
🔄 Seamless transition: keep your existing code structure while leveraging LLMs hosted on the Hugging Face Hub.
🤗 Direct integration: easily launch a model to run inference using our Inference Endpoint service.
🚀 Stay Updated: always be in sync with the latest Text-Generation-Inference (TGI) updates.

More details in https://huggingface.co/docs/huggingface_hub/main/en/guides/inference#openai-compatibility

not-lain Crudepioneer kevindragan osanseviero seyf1elislam Svngoku Daemontatox ayush-thakur02 juhoinkinen


Have you looked at the Gemini Nano local LLM?
Gradio-Lite, the in-browser ver. of Gradio, gives it a rich interface using only Python code, even for such an in-browser AI app!

Try out a chat app that runs completely inside your browser 👇
https://www.gradio.app/playground?demo=Hello_World&code=IyBOT1RFOiBHZW1pbmkgTmFubyBtdXN0IGJlIGVuYWJsZWQgaW4geW91ciBicm93c2VyLiBTZWUgYXJ0aWNsZXMgbGlrZSBodHRwczovL3dyaXRpbmdtYXRlLmFpL2Jsb2cvYWNjZXNzLXRvLWdlbWluaS1uYW5vLWxvY2FsbHkKaW1wb3J0IGdyYWRpbyBhcyBncgpmcm9tIGpzIGltcG9ydCBzZWxmICAjIFB5b2RpZGUgcHJvdmlkZXMgYWNjZXNzIHRvIHRoZSBKUyBzY29wZSB2aWEgYGpzYCBtb2R1bGUuIFNlZSBodHRwczovL3B5b2RpZGUub3JnL2VuL3N0YWJsZS91c2FnZS9hcGkvcHl0aG9uLWFwaS5odG1sI3B5dGhvbi1hcGkKCiMgSW5pdGlhbGl6ZSBQcm9tcHQgQVBJCnNlc3Npb24gPSBOb25lCnRyeToKICAgIGNhbl9haV9jcmVhdGUgPSBhd2FpdCBzZWxmLmFpLmNhbkNyZWF0ZVRleHRTZXNzaW9uKCkKICAgIGlmIGNhbl9haV9jcmVhdGUgIT0gIm5vIjoKICAgICAgICBzZXNzaW9uID0gYXdhaXQgc2VsZi5haS5jcmVhdGVUZXh0U2Vzc2lvbigpCmV4Y2VwdDoKICAgIHBhc3MKCgpzZWxmLmFpX3RleHRfc2Vzc2lvbiA9IHNlc3Npb24KCgphc3luYyBkZWYgcHJvbXB0KG1lc3NhZ2UsIGhpc3RvcnkpOgogICAgc2Vzc2lvbiA9IHNlbGYuYWlfdGV4dF9zZXNzaW9uCiAgICBpZiBub3Qgc2Vzc2lvbjoKICAgICAgICByYWlzZSBFeGNlcHRpb24oIkdlbWluaSBOYW5vIGlzIG5vdCBhdmFpbGFibGUgaW4geW91ciBicm93c2VyLiIpCgogICAgc3RyZWFtID0gc2Vzc2lvbi5wcm9tcHRTdHJlYW1pbmcobWVzc2FnZSkKICAgIGFzeW5jIGZvciBjaHVuayBpbiBzdHJlYW06CiAgICAgICAgeWllbGQgY2h1bmsKCgpkZW1vID0gZ3IuQ2hhdEludGVyZmFjZShmbj1wcm9tcHQpCgpkZW1vLmxhdW5jaCgp

Note: Gemini Nano is currently only available on Chrome Canary, and you need to opt-in.
Follow the "Installation" section in https://huggingface.co/blog/Xenova/run-gemini-nano-in-your-browser .


RegMix: Data Mixture as Regression for Language Model Pre-training (2407.01492)

Still following your human intuition to mix corpora from different sources for pre-training 🧠? Everyone says that data mixture has a big impact on model performance, but how - and why🕵? Did you know that web corpora are actually highly impactful for downstream tasks 🏆?

Check out our preprint "RegMix: Data Mixture as Regression for Language Model Pre-training" 📄

🔬 In this paper, we've proposed an automatic data mixture method RegMix that achieves a 6.3% improvement over human selection on the widely used HellaSwag benchmark - and it only needs a 2% extra training FLOPs! 📈

📄 Paper: https://huggingface.co/papers/2407.01492
💻 Code: https://github.com/sail-sg/regmix
📊 Collection: https://huggingface.co/collections/sail/regmix-data-mixture-as-regression-6682b6caab37b9442877f0ce
 🎮 Demo: https://huggingface.co/spaces/sail/RegMix


Hi everyone 🤗

You can easily use the new Model Explorer from Google (https://github.com/google-ai-edge/model-explorer) here: https://huggingface.co/spaces/1aurent/model-explorer

Unfortunately, it doesn't look like it can easily be preloaded with some models, so you'll have to bring your own. Here is a quick selection of models you can use:
- https://huggingface.co/openai-community/gpt2/blob/main/64-8bits.tflite
- https://huggingface.co/qualcomm/ResNet50/blob/main/ResNet50.tflite
- https://huggingface.co/qualcomm/VIT/blob/main/VIT.tflite


Korvus: Single-query RAG with Postgres

Hey folks!

We built Korvus, an open-source RAG (Retrieval-Augmented Generation) pipeline that consolidates the entire RAG workflow - from embedding generation to text generation - into a single SQL query, significantly reducing architectural complexity and latency.

https://github.com/postgresml/korvus

Here's some of the highlights:

- Full RAG pipeline (embedding generation, vector search, reranking, and text generation) in one SQL query
- SDKs for Python, JavaScript, and Rust (more languages planned)
- Built on PostgreSQL, leveraging pgvector and pgml
- Open-source, with support for open models
- Designed for high performance and scalability

We're eager to get feedback from the community and welcome contributions. Check out our GitHub repo for more details:

https://github.com/postgresml/korvus


I am working to add automatic speech bubbles to the AI Comic Factory!

This is not finished yet - there are a few things to tweak - but so far results are pretty promising!

(it's available through a "beta" toggle)

prithivMLmods MattBee Norod78 fdaudens sirjamesgray o0Rodney0o Tkap0176 victor KingNish Tusharkn12 osanseviero John6666 nmdeal louisbrulenaudet Ajayk Ramikan-BR yazdaabdul Sohaibsoussi imangshu MaziyarPanahi SouthbayJay fifteen42 Pent

TNVMike Tkap0176 tusharsoni00786 Ramikan-BR Cassoni alandarkx UsmanComics HemiHDR korujaa

digiplay Ramikan-BR MaziyarPanahi MAGMA91 dongnt


Continuing the series of datasets with code from Russian platforms: GitFlic Code Dataset - https://huggingface.co/datasets/nyuuzyou/gitflic-code.
📊 Dataset highlights:

60 GB of unique code extracted from over 967 GB of analyzed data
12,527 repositories
5,983,358 unique code files
692 different file types
Multilingual: various programming languages

 🌐 Sourced from GitFlic, the first Russian service for storing and working with source code, based on the Git version control system.


Vision language models are blind (2407.06581)

Researchers from Auburn University and the University of Alberta have explored the limitations of Vision Language Models (VLMs) in their recently published paper titled "Vision language models are blind." (https://huggingface.co/papers/2407.06581)

Key Findings:🔍
VLMs, including GPT-4o, Gemini-1.5 Pro, Claude-3 Sonnet, and Claude-3.5 Sonnet, struggle with basic visual tasks.
Tasks such as identifying where lines intersect or counting basic shapes are challenging for these models.
The authors noted, "The shockingly poor performance of four state-of-the-art VLMs suggests their vision is, at best, like of a person with myopia seeing fine details as blurry, and at worst, like an intelligent person that is blind making educated guesses"(Vision Language Models Are Blind; 2024).

Human-like Myopia? 👓
VLMs may have a blind spot similar to human myopia.
This limitation makes it difficult for VLMs to perceive details.
Suggests a potential parallel between human and machine vision limitations.

Technical Details: 🔧
The researchers created a new benchmark called BlindTest.
BlindTest consists of simple visual tasks to evaluate VLMs low-level vision capabilities.
Four VLMs were assessed using BlindTest.
Many shortcomings were revealed in the models ability to process basic visual information.

Learn More: 🖼
For a deeper dive into this research, check out the project page: https://vlmsareblind.github.io/


Introducing Whisper Timestamped: Multilingual speech recognition with word-level timestamps, running 100% locally in your browser thanks to 🤗 Transformers.js! Check it out!
👉 https://huggingface.co/spaces/Xenova/whisper-word-level-timestamps 👈

This unlocks a world of possibilities for in-browser video editing! 🤯 What will you build? 😍

Source code: https://github.com/xenova/transformers.js/tree/v3/examples/whisper-word-timestamps

BoscoTheDog fdaudens boapps Ramikan-BR Noahka bwrasa GPT007 osanseviero John6666 ralphilius louisbrulenaudet GordonM fedyano


Ghost 8B Beta is a large language model developed with goals that include excellent multilingual support, superior knowledge capabilities, and cost-effectiveness. The model comes in two context length versions, 8k and 128k, along with multilingual function tools support by default.
* The languages supported are 🇺🇸 English, 🇫🇷 French, 🇮🇹 Italian, 🇪🇸 Spanish, 🇵🇹 Portuguese, 🇩🇪 German, 🇻🇳 Vietnamese, 🇰🇷 Korean and 🇨🇳 Chinese.
* 👨💻 Try on Spaces: https://huggingface.co/spaces/lamhieu/ghost-8b-beta-8k
* 📋 Official website: https://ghost-x.org/docs/models/ghost-8b-beta


I've built a small open utility pip package called LLM-Forwarder that allows you to inject context, such as adding a private RAG, into existing chat applications by forwarding the app through the LLM-Forwarder. In the forwarder server, you can configure custom code to re-process chat messages and alter the user prompt, for example, by adding extra context.

https://pypi.org/project/llm-forwarder/
More details
https://devquasar.com/llmforwarder/


Mann-E's new platform is up and running.

You can access our platform here at https://mann-e.com. We're still working on it and reducing the bugs and we also are trying to add a guest session which lets you make images as guests.

What do you think?


It's not every day you see the No. 1 ranked paper of the day open-sourcing a very powerful image editing app!

Fascinating to see MagicQuill - a groundbreaking interactive image editing system that makes precise photo editing effortless through advanced AI!

The system's architecture features three sophisticated components:

1. Editing Processor:
- Implements a dual-branch architecture integrated into a latent diffusion framework
- Utilizes PiDiNet for edge map extraction and content-aware per-pixel inpainting
- Features a specialized UNet architecture with zero-convolution layers for feature insertion
- Employs denoising score matching for training the control branch
- Processes both structural modifications via scribble guidance and color manipulation through downsampled color blocks
- Maintains pixel-level control through VAE-based latent space operations

2. Painting Assistor:
- Powered by a fine-tuned LLaVA multimodal LLM using Low-Rank Adaptation (LoRA)
- Trained on a custom dataset derived from Densely Captioned Images (DCI)
- Processes user brushstrokes through specialized Q&A tasks for add/subtract/color operations
- Features bounding box coordinate normalization for precise stroke localization
- Implements streamlined single-word/phrase outputs for real-time performance

3. Idea Collector:
- Built as a modular ReactJS component library
- Supports cross-platform deployment via HTTP protocols
- Compatible with Gradio and ComfyUI frameworks
- Features comprehensive layer management and parameter adjustment capabilities
- Implements real-time canvas updates and preview generation

The system outperforms existing solutions like SmartEdit and BrushNet in edge alignment and color fidelity while maintaining seamless integration with popular AI frameworks.

What are your thoughts on AI-powered creative tools?