Just crossed 200,000 free public AI datasets shared by the community on Hugging Face! Text, image, video, audio, time-series & many more... Thanks everyone!

http://hf.co/datasets

ajibawa-2023 John6666 yjernite KingNish TheDrunkenSnail Felladrin Sergidev fdaudens osanseviero kramp damerajee asiraja Mackin7 Nacholmo alielfilali01 CookieMaster its5Q nyuuzyou Rsln jsulz louisbrulenaudet pagezyhf


First fully multi-GPU supporting and very advanced batch image captioner APP with Gradio interface published (as far as i know first)

Multi-GPU batch caption with JoyCaption. JoyCaption uses Meta-Llama-3.1–8B and google/siglip-so400m-patch14–384 and a fine tuned image captioning neural network.

Link : https://www.patreon.com/posts/110613301

Link for batch caption editor : https://www.patreon.com/posts/108992085

Coding multi-gpu in Python and Torch and bitsandbytes was truly a challange.

Our APP uses JoyCaption image captioning fine tuned model.

Our APP supports bitsandbytes 4bit model loading as well even in multi GPU mode (9.5 GB VRAM)

Tested on 8x RTX A6000 (cloud) and RTX 3090 TI + RTX 3060 (my PC)

1-click to install on Windows, RunPod and Massed Compute

Excellent caption quality, automatically distributes images into each GPU, lots of features. You can resume caption with skip captioned images option.

For full details checkout screenshots


Here's my favorite piece of the summer bias detection research project (paper coming in Sept). We trained BERT for token classification (multi-label), to identify:
- Generalizations
- Unfairness
- Stereotypes

HF Space: https://huggingface.co/spaces/maximuspowers/bias-detection-ner
Article on Training: https://huggingface.co/blog/maximuspowers/bias-entity-recognition

Pls reach out with ideas!! Lot's more info coming soon, our research group has workshops and a hackathon planned for launching this open source project. Thanks


Question about LightEval 🤗:

I've been searching for an LLM evaluation suite that can, out-of-the-box, compare the outputs of a model(s) without any enhancements vs. the same model with better prompt engineering, vs. the same model with RAG vs. the same model with fine-tuning.

I unfortunately have not found a tool that fits my exact description, but of course I ran into LightEval.

A huge pain-point of building large-scale projects that use LLMs is that prior to building an MVP, it is difficult to evaluate whether better prompt engineering, or RAG, or fine-tuning, or some combination of all is needed for satisfactory LLM output in terms of the project's given use case.

Time and resources is then wasted R&D'ing exactly what LLM enhancements are needed.

I believe an out-of-the-box solution to compare models w/ or w/out the aforementioned LLM enhancements could help teams of any size better decide what LLM enhancements are needed prior to building.

I wanted to know if the LightEval team or Hugging Face in general is thinking about such a tool.


I developed a way to test very clearly whether or not a Transformers model can actually learn symbolic reasoning, or if LLM models are forever doomed to be offshoots of 'Socratic Parrots'. The results are in, undeniable proof that Transformers models CAN learn symbolic relationships. Undeniable proof that AI can learn its ABC's. Credit goes to myself, Claude, and ChatGPT. I would not be able to prove this without Claude or ChatGPT.

https://www.youtube.com/watch?v=I8jHRgahRfY

handfuloftitty Nioi Josephgflowers alien1957 binatheis rohitdavas f0ster


florence-tool (https://github.com/bigdata-pw/florence-tool) now supports WebDataset! Check it out for efficient batch inference with Florence-2 models https://huggingface.co/microsoft/Florence-2-large https://huggingface.co/microsoft/Florence-2-base

Currently running it myself on A40 with `CAPTION` task and a streaming WebDataset @ 60k images/hour!


 📐 AI Math Equation Solver: Your Step-by-Step Solution Companion

Hello Hugging Face community! 👋 I'm excited to share my latest Space: the AI Math Equation Solver!

 🔍 What does it do?

This Space uses the power of AI to solve math problems from images. Simply upload a picture of a math equation or problem, and the AI will provide a detailed, step-by-step solution. It's perfect for students, teachers, or anyone looking to understand complex mathematical concepts better.

 🧠 How does it work?

- Backend: Utilizes the `microsoft/Phi-3.5-vision-instruct` model for image understanding and mathematical reasoning.
- Frontend: Built with Gradio for a clean, user-friendly interface.
- Features:
  - Image upload for math problems
  - Detailed step-by-step solutions
  - Example problems to try instantly

🚀 Try it out! https://huggingface.co/spaces/sagar007/phi-vision-math-assistant

Visit the Space here: [Insert your Hugging Face Space URL]

💡 Use cases:

- Students: Check your work or get help with homework
- Teachers: Create detailed solution guides quickly
- Tutors: Explain complex problems more effectively
- Self-learners: Understand new mathematical concepts

🛠 Technical Details:

- Model: microsoft/Phi-3.5-vision-instruct
- Libraries: transformers, Gradio, PyTorch
- Optimizations: Uses Flash Attention for improved performance

🤝 Contribute:

This is an open project, and I welcome contributions! Whether it's improving the model, enhancing the UI, or adding new features, feel free to fork the project and submit your pull requests.

 📣 Feedback:

I'd love to hear your thoughts! How are you using this Space? Any suggestions for improvements? Let me know in the comments below.

Happy problem-solving! 🎉

#MachineLearning #AI #Mathematics #Education #HuggingFace


🎮 Introducing a new dataset focused on Steam user game bans - https://huggingface.co/datasets/nyuuzyou/steambans

Dataset highlights:
- 476,694 Steam user profiles
- Emphasis on game ban data and account status
- Each entry includes: Steam ID, profile URL, username, avatar, account creation date, visibility state, VAC bans, game bans, economy ban status, and days since last ban
- Covers a wide range of Steam users, from clean accounts to those with multiple bans
- Data spans up to 2024

Perfect for researchers studying cheating in online games, anti-cheat effectiveness, and player behavior patterns in the face of bans.


30 Features that Dramatically Improve LLM Performance - Part 1 https://mltblog.com/3Aq9iAb

Many are ground-breaking innovations that make LLMs much faster and not prone to hallucinations. They reduce the cost, latency, and amount of computer resources (GPU, training) by several orders of magnitude. Some of them improve security, making your LLM more attractive to corporate clients. I introduced a few of these features in my previous article "New Trends in LLM Architecture". Now I offer a comprehensive list, based on the most recent developments.

Read full article, learn about agentic LLMs, LLM routers, contextual tables, fast search, and more, at https://mltblog.com/3Aq9iAb


**Exploring Realistic Emotional Depth in AI Language Models**

Language models, particularly those proprietary, often grapple with issues of censorship, which can limit their ability to engage authentically with users. Recognizing this, the open-source AI community has pioneered the development of language models that are less restrained, offering more candid interactions. However, even these models tend to maintain a veneer of neutrality or overly positive responses, which might not serve all users' needs, especially in contexts where emotional depth and relatability are crucial.

To address this gap, I've curated a specialized dataset aimed at infusing language models with a more nuanced emotional spectrum, specifically targeting a darker, more introspective mood. This dataset, titled "Dark Sentience", is designed to complement existing datasets like RP (Role Play) and those focused on instruction following. It seeks to enhance the emotional intelligence of AI by exposing it to complex human emotions, including but not limited to:

- **Suicide**
- **Depression**
- **Anxiety**

Trigger Warning: Please be advised that the content within this dataset deals with heavy and potentially distressing themes.

The "Dark Sentience" dataset is now available for review and use at: https://huggingface.co/datasets/Locutusque/Dark-Sentience. I encourage researchers, developers, and mental health professionals to explore how this resource can foster more genuine and supportive AI interactions.


 🔗 Neural Network  (1 Byte explainer for everybody)

Just like our brain, a Neural Network is made up of interconnected "neurons". These neurons work together by learning from (input) data and getting better at tasks (in the hidden layer) to give (output) predictions or decisions.


🌟 Liger Kernel: Efficient Triton Kernels for LLM Training

LIGER "is a [Hugging Face-compatible] collection of Triton kernels designed specifically for LLM training. It can effectively increase multi-GPU training throughput by 20% and reduces memory usage by 60%."

GitHub: https://github.com/linkedin/Liger-Kernel


a new shape-optimized SigLIP just dropped 👀 https://huggingface.co/google/siglip-so400m-patch14-224/


 Looking for Generative AI trainer/speaker for AI accelerator program (Virtual/Online sessions).

To get more context about the program, please visit the program landing page: https://llamadesigndrive.com

If you are interested, reach out at sharhabeel@startupbootcamp.org


🎯  Ghost 8B Beta 1608: Empowering Your AI Assistant
📦 Unlock the Power of Ghost 8B Beta 1608: Build Your Personal AI Companion
Ghost 8B Beta 1608 empowers you to create a safe and multilingual AI assistant tailored to your needs, directly on your personal computer. 🧑💻 Leverage AI's capabilities within your own space! 🚀 Ghost 8B Beta 1608 is ready to become your AI companion.
~
📦 개인용 AI 보조 도구로 Ghost 8B Beta 1608를 활용하세요!
Ghost 8B Beta 1608, AI의 힘을 활용하여 안전하고 개인화된 언어 지원을 제공하는 AI 보조 도구를 직접 구축할 수 있습니다. 🧑💻 개인 컴퓨터에서 AI의 혜택을 누리세요! 🚀 Ghost 8B Beta 1608는 당신의 AI 파트너가 될 준비가 되어 있습니다.
https://huggingface.co/spaces/lamhieu/ghost-8b-beta-8k
https://huggingface.co/collections/ghost-x/ghost-8b-beta-668ead6179f93be717db4542


Introducing Voicee, A superfast voice fast assistant.
https://huggingface.co/spaces/KingNish/Voicee
It achieved latency <500 ms.
While its average latency is 700ms.
It works best in Google Chrome.
Please try and give your feedbacks.
Thank you. 🤗

Bruhn John6666 m-ric prithivMLmods hamedj ajibawa-2023 osanseviero victor


Alan Turing's mind-bender: "Can machines think?" in its glorified form. This 74yr old paper laid the foundation for how we think about AI and machine intelligence today. The level of detail, clarity and foresight is just phenomenal - he was way ahead of his time 🧠🤖

Original copy: https://archive.org/details/MIND--COMPUTING-MACHINERY-AND-INTELLIGENCE


🌐 Check out the new dataset sourced from Fishki.net, one of the popular entertainment and news portals in the Russian Internet, known for its diverse content including humor, interesting facts, and viral stories - https://huggingface.co/datasets/nyuuzyou/fishkinet-posts.

📊 Dataset highlights:
- 369,180 posts
- Includes original posts with titles, content, images, and metadata
- Each entry contains URL, title, author, date, tags, content, and image URLs
- Primarily in Russian language
- Covers a wide range of topics in entertainment, news, and social media content
- Spans nearly two decades of posts, likely from early 2000s to 2024
- Dedicated to public domain under Creative Commons Zero (CC0) license


You can now use DoRA for your embedding layers!

PR: https://github.com/huggingface/peft/pull/2006

I have documented my journey of this specific PR in a blog post for everyone to read. The highlight of the PR was when the first author of DoRA reviewed my code.

Blog Post: https://huggingface.co/blog/ariG23498/peft-dora

Huge thanks to @BenjaminB for all the help I needed.


I can't believe this... Phi-3.5-mini (3.8B) running in-browser at ~90 tokens/second on WebGPU w/ Transformers.js and ONNX Runtime Web! 🤯 Since everything runs 100% locally, no messages are sent to a server — a huge win for privacy!
- 🤗 Demo: https://huggingface.co/spaces/webml-community/phi-3.5-webgpu
- 🧑💻 Source code: https://github.com/huggingface/transformers.js-examples/tree/main/phi-3.5-webgpu

cschroeder DmitryRyumin John6666 4rtemi5 DenyTranDFW Rybens YaTharThShaRma999 massimoavvisati Bruhn nguyenbh DiamanteAmarelo TheDrunkenSnail not-lain cfahlgren1 loubnabnl Ramikan-BR Omarito2412 revolunet osanseviero jweston cahlen captainspock louisbrulenaudet rmanoj ethix RojoXlon jundialwan ghosty-0 chrislhow gianpaj parksthecoder

jdspugh DiamanteAmarelo cfahlgren1 Ramikan-BR GordonM cahlen


https://huggingface.co/v2ray/deepgelbooru
A Danbooru tag image tagger, maybe better than WD14 at some images.
Training code, inference code, dataset included.
:3


Dolma: an Open Corpus of Three Trillion Tokens for Language Model
  Pretraining Research (2402.00159)

Same Task, More Tokens: the Impact of Input Length on the Reasoning
  Performance of Large Language Models (2402.14848)

Ensemble-Based Unsupervised Discontinuous Constituency Parsing by Tree
  Averaging (2403.00143)

📄 ACL 2024: The Missing Papers

Apparently, some papers from the ACL 2024 are still not listed in the ACL Anthology. While this issue will hopefully be fixed soon, we should give those papers additional spotlight.

Some of my favorites:

1. Dolma is an English corpus that encompasses 3 trillion tokens. Additionally, it is accompanied by an exceptional software package that consdierably advances the state-of-the-art in preparing data for LLM pretraining. (Source: I am currently using Dolma.)
https://huggingface.co/papers/2402.00159

2. In the paper "Same Task, More Tokens: the Impact of Input Length on
the Reasoning Performance of Large Language Models", the authors show how extending the context length impacts an LLM's reasoning performance. I asked myself a similar question a few months ago, and therefore this paper is highly interesting to me.
https://huggingface.co/papers/2402.14848

This was brought to my attention through a Linkedin post by @ShayeghB, who is also affected:
https://huggingface.co/papers/2403.00143

View all the missing papers here:
https://theshayegh.github.io/ACL2024MissingPapers/


BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2211.05100)

Shoutout to everyone who participated in BigScience! Doesn't get enough credit but IMO paved the way for open-source LLMs!

https://huggingface.co/papers/2211.05100
https://huggingface.co/bigscience/bloom
https://huggingface.co/bigscience/bloomz

MaziyarPanahi MohammadAminDHM osanseviero YaTharThShaRma999 John6666 adnankhaan louisbrulenaudet


 🌟 Enchanted Tales Generator: A GPT-2 Inspired Story Weaver 🌟

Hello Hugging Face community! I'm excited to share my latest project, the Enchanted Tales Generator, inspired by Andrej Karpathy's enlightening YouTube videos on GPT-2.

🧙♂ What is the Enchanted Tales Generator?

The Enchanted Tales Generator is a magical text generation app that weaves whimsical stories based on your prompts. It's powered by a fine-tuned GPT model and brings the wonder of AI-generated storytelling to your fingertips.

 🎥 Inspiration

This project was born from the inspiration I drew from Andrej Karpathy's incredible YouTube series on GPT-2. His clear explanations and deep insights into the workings of language models sparked my imagination and drove me to create something that could bring joy and creativity to others.

 🔮 Features

- Story Incantation: Input your magical prompt to start your tale
- Scroll Length: Adjust the length of your generated story
- Magical Intensity: Control the creativity (temperature) of the generation
- Arcane Diversity: Fine-tune the variety of word choices (top-k sampling)

🚀 Try It Out: https://huggingface.co/spaces/sagar007/GPT-2-with_gpu

You can experience the Enchanted Tales Generator right here on Hugging Face Spaces: [Insert your Spaces link here]

 🛠 Behind the Scenes

The app is built using:
- PyTorch for the underlying GPT model
- Gradio for the user interface
- Hugging Face Spaces for deployment

I've implemented techniques learned from Karpathy's videos, such as:
- Custom GPT architecture
- Top-k sampling for diverse outputs
- Temperature-controlled generation

🌈 Future Enchantments

I'm continuously working on improving the Enchanted Tales Generator. Some ideas for future updates include:
- Fine-tuning on specific genres of stories
- Adding options for character and setting generation
- Implementing more advanced sampling techniques

 🤝 Join the Magic!


Huge updates and improvements for FLUX LoRA training : https://www.patreon.com/posts/kohya-flux-lora-110293257

10 GB, 16 GB, 24 GB and 48 GB GPU configs added - 10 GB config is like 3x to 5x slower sadly

Massed Compute, RunPod and Windows Kohya SS GUI LoRA installers added to the zip file

Also right now testing new 16 GB FLUX LoRA training config and new way of regularization images. Moreover testing Apply T5 Attention Mask too. Lets see if Kohya FLUX LoRA workflow will become even better or not

Also massive grids comparisons shared here : https://www.reddit.com/r/StableDiffusion/comments/1eyj4b8/kohya_ss_gui_very_easy_flux_lora_trainings_full/


So, we published https://huggingface.co/frameai/ChatFrame model. this model is on higher level of responsibilities. You can solve mathematics problems and write high quality codes. We are inviting all of you to help use in fine-tune this model for all languages. Please give us a languages dataset to make our model available in all languages. We are waiting for your reply.


The latest timm validation & test set results are now viewable by a  leaderboard space: https://huggingface.co/spaces/timm/leaderboard

As of yesterday, I updated all of the results for ImageNet , ImageNet-ReaL, ImageNet-V2, ImageNet-R, ImageNet-A, and Sketch sets.  The csv files can be found in the GH repo https://github.com/huggingface/pytorch-image-models/tree/main/results

Unfortunately the latest benchmark csv files are not yet up to date,  there are some gaps in dataset results vs throughput/flop numbers impact the plots.

h/t to @MohamedRashad for making the first timm leaderboard.

eliebak John6666 osanseviero sergiopaniego bryant1410 nagaharish

eliebak MohamedRashad bryant1410 nagaharish


𝗔𝗜𝟮𝟭 𝗶𝘁𝗲𝗿𝗮𝘁𝗲𝘀 𝘄𝗶𝘁𝗵 𝗻𝗲𝘄 𝗝𝗮𝗺𝗯𝗮 𝟭.𝟱 𝗿𝗲𝗹𝗲𝗮𝘀𝗲: 𝗡𝗲𝘄 𝘀𝘁𝗮𝗻𝗱𝗮𝗿𝗱 𝗳𝗼𝗿 𝗹𝗼𝗻𝗴-𝗰𝗼𝗻𝘁𝗲𝘅𝘁 𝘂𝘀𝗲-𝗰𝗮𝘀𝗲𝘀!🏅

@ai21labs used a different architecture to beat the status-quo Transformers models: Jamba architecture combines classic Transformers layers with the new Mamba layers, for which the complexity is a linear (instead of quadratic) function of the context length.

What does this imply?

➡ Jamba models are much more efficient for long contexts: faster (up to 2.5x faster for long context), takes less memory, and also performs better to recall everything in the prompt.

That means it’s a new go-to model for RAG or agentic applications!

And the performance is not too shabby: Jamba 1.5 models are comparable in perf to similar-sized Llama-3.1 models! The largest model even outperforms Llama-3.1 405B on Arena-Hard.

✌ Comes in 2 sizes: Mini (12B active/52B) and Large (94B active/399B)
📏 Both deliver 256k context length, for low memory: Jamba-1.5 mini fits 140k context length on one single A100.
⚙ New quanttization method: Experts Int8 quantizes only the weights parts of the MoE layers, which account for 85% of weights
🤖 Natively supports JSON format generation & function calling.
🔓 Permissive license *if your org makes <$50M revenue*

Available on the Hub 👉 https://huggingface.co/collections/ai21labs/jamba-15-66c44befa474a917fcf55251
Read their release blog post 👉 https://www.ai21.com/blog/announcing-jamba-model-family


# Offensive Security Reconnaissance Continued with Public Facing Industrial Control System HMIs using Moondream

Building on my previous experiments with Moondream for physical security reconnaissance planning automation (https://huggingface.co/posts/Csplk/926337297827024), I've now turned my attention to exploring the potential of this powerful image-text-text model for offensive security reconnaissance in the realm of Industrial Control Systems (ICS).
ICS HMIs (Human-Machine Interfaces) are increasingly exposed to the public internet, often without adequate security measures in place. This presents a tantalizing opportunity for malicious actors to exploit vulnerabilities and gain unauthorized access to critical infrastructure.

Using Moondream with batch processing (https://huggingface.co/spaces/Csplk/moondream2-batch-processing), I've been experimenting with analyzing public facing ICS (https://huggingface.co/datasets/Csplk/ICS_UIs) HMI (https://huggingface.co/datasets/Csplk/HMI) screenshots from shodan  to identify types of exposed ICS system HMIs, how they are operated and how malicious actors with access to these systems could cause damage to physical infrastructure. Feeding images of HMIs and pre-defined text prompts to Moondream batch processing successfully (unconfirmed accuracy levels) extracted information about the underlying systems, including

1. **System type**
2. **Possible Operation Details**
3. **Malicious Actor Outcomes**

Next steps:
* I have a longer and more in depth blog write up in the works that will cover the previous and this post's approaches for experiments for sharing via HF community blog posts soon.
* I plan to continue refining my Moondream-based tool to improve its accuracy and effectiveness in processing public facing ICS HMIs.
* As mentioned before,  offensive security with moondream focused HF Space once its fleshed out.

Thanks again to @vikhyatk for the incredible Moondream model. https://huggingface.co/vikhyatk/moondream2


🎉 Just dropped a fresh version of dataset-viber along with some cool, Gradio-based annotators! These tools aren't about formalities—they're here to help you quickly collect feedback and get your projects moving along to a more serious stage, ahumm @argilla.

Some new features!
- manual import from a CSV or the Hugging Face Hub
- manual export to CSV or the Hub
- improved automated export to the Hub and CSV
- limit interaction with specific components
- stream data with custom next_input features (SO to Ben Burtenshaw for the suggestions)
- model in-the-loop support for all tasks

https://huggingface.co/collections/dataset-viber/gradio-annotators-66c5ce73d5e3bf99caa445b1


BIG update dropped for https://huggingface.co/datasets/bigdata-pw/Flickr - now ~515M images! Target for the next update: 1B

In case you missed them; other recent drops include https://huggingface.co/datasets/bigdata-pw/Dinosaurs - a small set of BIG creatures 🦕🦖 and the first in a series of articles about the art of web scraping! https://huggingface.co/blog/hlky/web-scraping-101 https://huggingface.co/blog/hlky/web-scraping-102

Stay tuned for exciting datasets and models coming soon:
- PC and Console game screenshots
- TV/Film actors biographies and photos (think facial recognition and automatic captioning!)
- https://huggingface.co/bigdata-pw/lyrics-gpt v2
- and more!


🚨 NEW TASK ALERT 🚨
Extractive Question Answering: because sometimes generative is not all you need 😉
AutoTrain is the only open-source, no code solution to offer so many tasks across different modalities. Current task count: 23 🚀
Check out the blog post on getting started with this task: https://huggingface.co/blog/abhishek/extractive-qa-autotrain


🙋 Calling all Hugging Face users! We want to hear from YOU!

What feature or improvement would make the biggest impact on Hugging Face?

Whether it's the Hub, better documentation, new integrations, or something completely different – we're all ears!

Your feedback shapes the future of Hugging Face. Drop your ideas in the comments below! 👇

lazarustda clem jsulz Saugatkafley lukmanaj sergiopaniego not-lain nicoboss cfahlgren1 Tonic nbroad denizaybey Nymbo John6666 DamarJati ajibawa-2023 MarinaraSpaghetti KingNish asigalov61 iafun cschroeder RGAES louisbrulenaudet enzostvs blanchon khitab kramp Xdotnet oritey yhna

clem jsulz not-lain nicoboss cfahlgren1 pierrci nbroad Nymbo John6666 MarinaraSpaghetti merterbak nkasmanoff TanVPat blanchon osanseviero

ijohn07 MarinaraSpaghetti sauravssss Rybens nyuuzyou asigalov61 iafun MoritzLaurer blanchon John6666


This isn’t a goal of ours because we have plenty of money in the bank but quite excited to see that @huggingfaceis profitable these days, with 220 team members and most of our platform being free (like model hosting) and open-source for the community!

Especially noteworthy at a time when most AI startups wouldn’t survive a year or two without VC money. Yay!

Priyankvadaliya Claudiu2004 siaal313 hvaara jsulz appvoid lhoestq desklockgun Nymbo NouRed IAmTheCollector ajibawa-2023 wanghaofan bartowski sumandas Arakinas carsenk jphme Shehab007 handfuloftitty philipp-zettl

burtenshaw victor hvaara jsulz John6666 Nymbo bartowski carsenk rAIfle handfuloftitty


Falcon Mamba now available now in llama.cpp !
Check out GGUF files uploaded here: https://huggingface.co/collections/tiiuae/falconmamba-7b-66b9a580324dd1598b0f6d4a


The Minimalist Spaces That May Be Helpful !!
GrabDoc V | GrabDoc | Type Byte | SD3 CLI

- GrabDoc V: https://huggingface.co/spaces/prithivMLmods/GRABDOC-V
- GrabDoc: https://huggingface.co/spaces/prithivMLmods/GRAB-DOC
- Type Byte: https://huggingface.co/spaces/prithivMLmods/Type-Byte
- SD3 CLI: https://huggingface.co/spaces/prithivMLmods/SD3-CLI

John6666 victor Nymbo prithivMLmods Ngrthm


Woman.ru Forum Posts Dataset - https://huggingface.co/datasets/nyuuzyou/womanru-posts

📊 Dataset highlights:

- 1,308,238 forum posts extracted from Woman.ru
- Includes original posts and replies from various threads
- Each entry contains URL, title, original post, date, and replies
- Primarily in Russian language
- Covers a wide range of topics relevant to the site's female-focused audience
- Spans nearly two decades of discussions, from around 2005 to 2024
- Dedicated to public domain under Creative Commons Zero (CC0) license

🌐 Sourced from Woman.ru, one of the most popular women's information and entertainment portals in the Russian Internet.


Cooked up a cool & much faster AI voice assistant space that also supports speech translation (with seamless-expressive). Start with the phrase "Please translate" followed by the speech you'd like to translate, to activate speech translation mode. Using opensource LLMs (Llama 3, Mistral etc) with edge tts for voice assistant and seamless-expressive for speech translation.

Give it a try: https://huggingface.co/spaces/Jaward/optimus


SpaRP: Fast 3D Object Reconstruction and Pose Estimation from Sparse
  Views (2408.10195)

MeshFormer: High-Quality Mesh Generation with 3D-Guided Reconstruction
  Model (2408.10198)

🚀 We are proud to release our latest suite of three image(s)-to-3D Gradio demos and two new papers.

SpaRP (Unposed sparse views to 3D): https://huggingface.co/spaces/sudo-ai/SpaRP https://huggingface.co/papers/2408.10195
MeshFormer (@minghua  @NCJ ): https://huggingface.co/spaces/sudo-ai/MeshFormer https://huggingface.co/papers/2408.10198
MeshLRM-reproduced (@sarahwei0210 ): https://huggingface.co/spaces/sudo-ai/MeshLRM

Great thanks to @angli66 for his many efforts in preparing these demos!

chaoxu John6666 osanseviero Tonic victor angli66


'Legal Dictionary GPT' is now completely trained and ready for Open Source release to the world! Trained on 10,000 rows of legal definitions, Legal Dictionary GPT is your go-to resource for everything related to the first step in understanding the law, defining it. The model is free and publicly available for anyone to use.

Model Link:  https://platform.openai.com/playground/chat?preset=eCrKdaPe9cnMnyTETqWDCQAU

Knowledge Base Bots are internal facing as opposed to external facing LLM models, that are either fine tuned or RAG tuned, generally on systems and processes related data.

Learn more about Knowledge Base Bots at our website:
https://knowledgebasebots.com/


🚀 Meet the new GLiNER architecture 🚀
GLiNER revolutionized zero-shot NER by demonstrating that lightweight encoders can achieve excellent results.  We're excited to continue R&D with this spirit 🔥. Our new bi-encoder and poly-encoder architectures were developed to address the main limitations of the original GLiNER architecture and bring the following new possibilities:

🔹 An unlimited number of entities can be recognized at once.
🔹Faster inference when entity embeddings are preprocessed.
🔹Better generalization to unseen entities.

While the bi-encoder architecture can lack inter-label understanding, we developed a poly-encoder architecture with post-fusion. It achieves the same or even better results on many benchmarking datasets compared to the original GLiNER, while still offering the listed advantages of bi-encoders.
Now, it’s possible to run GLiNER with hundreds of entities much faster and more reliably.

📌 Try the new models here:
https://huggingface.co/collections/knowledgator/gliner-bi-encoders-66c492ce224a51c54232657b


Scaling LLM Test-Time Compute Optimally can be More Effective than
  Scaling Model Parameters (2408.03314)

𝗚𝗼𝗼𝗴𝗹𝗲 𝗽𝗮𝗽𝗲𝗿 : 𝘀𝗰𝗮𝗹𝗶𝗻𝗴 𝘂𝗽 𝗶𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗰𝗼𝗺𝗽𝘂𝘁𝗲 𝗯𝗲𝗮𝘁𝘀 𝟭𝟰𝘅 𝗹𝗮𝗿𝗴𝗲𝗿 𝗺𝗼𝗱𝗲𝗹𝘀 🚀

Remember scaling laws? These are empirical laws that say "the bigger your model, the better it gets". More precisely, "as your compute increases exponentially, loss decreases in a linear fashion". They have wild implications, suggesting that spending 100x more training compute would make you super-LLMs. That's why companies are racing to build the biggest AI superclusters ever, and Meta bought 350k H100 GPUs, which probably cost in the order of $1B.

But think of this : we're building huge reasoning machines, but only ask them to do one pass through the model to get one token of the final answer : i.e., we expend a minimal effort on inference. That's like building a Caterpillar truck and making it run on a lawnmower's motor. 🚚🛵 Couldn't we optimize this? 🤔

💡 So instead of scaling up on training by training even bigger models on many more trillions of tokens, Google researchers explored this under-explored avenue : scaling up inference compute.

They combine two methods to use more compute : either a reviser that iterated to adapt the model distribution, or generate N different completions (for instance through Beam Search) and select only the best one using an additional verifier model.

They use a Palm-2 model (released in May 23) on the MATH dataset : Palm-2 has the advantage of getting a low performance on MATH, but not zero, so that improvements will be noticeable.

And the results show that for the same fixed amount of inference compute:
💥 a smaller model with more effort on decoding beats a x14 bigger model using naive greedy sampling.

That means that you can divide your training costs by 14 and still get the same perf for the same inference cost!

Take that, scaling laws. Mark Zuckerberg, you're welcome, hope I can get some of these H100s.

Read the paper here 👉 https://huggingface.co/papers/2408.03314

deundido victor todaybite rreed-pha lehoanganh298 ZeroWw prithivMLmods John6666 Joseph717171 inflatebot diwank aheuillet nbroad MoritzLaurer


Just added the newly released xGen-MM v1.5 foundational Large Multimodal Models (LMMs) developed by Salesforce AI Research to my xGen-MM HF Space https://huggingface.co/spaces/maxiw/XGen-MM

Sri-Vigneshwar-DJ osanseviero victor mrdbourke John6666 alielfilali01


🚀 We will be generating a preference dataset for DPO/ORPO and cleaning it with AI feedback during our upcoming meetup!

In this session, we'll walk you through the essentials of building a distilabel pipeline by exploring two key use cases: cleaning an existing dataset and generating a preference dataset for DPO/ORPO. You’ll also learn how to make the most of AI feedback, integrating Argilla to gather human feedback and improve the overall data quality.

This session is perfect for you
- if you’re getting started with distilabel or synthetic data
- if you want to learn how to use LLM inference endpoints for **free**
- if you want to discover new functionalities
- if you want to provide us with new feedback

Sign up here: https://lu.ma/dt0c7jru

davidberenstein1957 osanseviero plaguss not-lain sdiazlor victor legolasyiu burtenshaw

davidberenstein1957 not-lain BramVanroy victor NickyNicky


ChatFrame-Persian is first expert Persian language model in Iran.

cuhawku1 explorewithai victor Tonic prithivMLmods


🚀 How The Washington Post Uses AI to Empower Journalists 🔍📰

An exciting new example in the world of AI-assisted journalism! The Post has developed an internal tool called "Hayatacker" that's enhancing in-depth reporting. Here's why it matters:

🎥 What it does:
• Extracts stills from video files
• Processes on-screen text
• Labels objects in images

🗳 First big project:
Analyzed 745 Republican campaign ads on immigration (Jan-Jun 2024)

🤝 Human-AI collaboration:
• AI extracts and organizes data
• Reporters verify and analyze findings

🔎 Thorough approach:
• Manual review of all 745 ads
• Reverse image searches when context is lacking
• Cross-referencing with AdImpact transcripts

💡 Key insight from WaPo's Senior Editor for AI strategy Phoebe Connelly:
"The more exciting choice is putting AI in the hands of reporters early on in the process."

This tool showcases how AI can augment journalistic capabilities without replacing human insight and verification. It's a powerful example of technology enhancing, not replacing, traditional reporting skills.

👉 Read the full article and the methodology: https://www.washingtonpost.com/elections/interactive/2024/republican-campaign-ads-immigration-border-security/

YannisTevissen OWNRWallet Smooke jkohnbryan78 MefhigosetH victor adamelliotfields cschroeder AlkaouDembele Nelathan


🤗 Serving Meta Llama 3.1 405B on Google Cloud is now possible via the Hugging Face Deep Learning Containers (DLCs) for Text Generation Inference (TGI)

In this post, we showcase how to deploy https://huggingface.co/meta-llama/Meta-Llama-3.1-405B-Instruct-FP8 on an A3 instance with 8 x H100 GPUs on Vertex AI

Thanks to the Hugging Face DLCs for TGI and Google Cloud Vertex AI, deploying a high-performance text generation container for serving Large Language Models (LLMs) has never been easier. And we’re not going to stop here – stay tuned as we enable more experiences to build AI with open models on Google Cloud!

Read the full post at https://huggingface.co/blog/llama31-on-vertex-ai

maximuspowers OWNRWallet gabrielmbmb not-lain behindmask pagezyhf troelsfr

whitebill gabrielmbmb ajibawa-2023 behindmask


@Blane187 could you please modify the title of your blogpost? content is cool, title could be nicer imo https://huggingface.co/blog/Blane187/wtf-is-rvc


🚀 Introducing Hugging Face Similar: a Chrome extension to find relevant datasets!

✨ Adds a "Similar Datasets" section to Hugging Face dataset pages
🔍 Recommendations based on dataset READMEs
🏗 Powered by https://huggingface.co/chromadb  and https://huggingface.co/Snowflake embeddings.

You can try it here: https://chromewebstore.google.com/detail/hugging-face-similar/aijelnjllajooinkcpkpbhckbghghpnl?authuser=0&hl=en.

I am very happy to get feedback on whether this could be useful or not 🤗

prithivMLmods John6666 ajibawa-2023 lunarflu osanseviero victor Aarifkhan Nymbo


ResShift 1-Click Windows, RunPod, Massed Compute, Kaggle Installers with Amazing Gradio APP and Batch Image Processing. ResShift is Efficient Diffusion Model for Image Super-resolution by Residual Shifting (NeurIPS 2023, Spotlight).

Official Repo : https://github.com/zsyOAOA/ResShift

I have developed a very advanced Gradio APP.

Developed APP Scripts and Installers : https://www.patreon.com/posts/110331752

Features

It supports following tasks:

Real-world image super-resolution

Bicubic (resize by Matlab) image super-resolution

Blind Face Restoration

Automatically saving all generated image with same name + numbering if necessary

Randomize seed feature for each generation

Batch image processing - give input and output folder paths and it batch process all images and saves

1-Click to install on Windows, RunPod, Massed Compute and Kaggle (free account)

Windows Requirements

Python 3.10, FFmpeg, Cuda 11.8, C++ tools and Git

If it doesn't work make sure to below tutorial and install everything exactly as shown in this below tutorial

https://youtu.be/-NjNy7afOQ0

How to Install on Windows

Make sure that you have the above requirements

Extract files into a folder like c:/reshift_v1

Double click Windows_Install.bat and it will automatically install everything for you with an isolated virtual environment folder (VENV)

After that double click Windows_Start_app.bat and start the app

When you first time use a task it will download necessary models (all under 500 MB) into accurate folders

If during download it fails, file gets corrupted sadly it doesn't verify that so delete files inside weights and restart

How to Install on RunPod, Massed Compute, Kaggle

Follow the Massed_Compute_Instructions_READ.txt and Runpod_Instructions_READ.txt

For Kaggle follow the notebook written steps

An example video of how to use my RunPod, Massed Compute scripts and Kaggle notebook can be seen

https://youtu.be/wG7oPp01COg


Put together a small repo showing how to go from making your own fine-tuning dataset w/ services like Groq & Together to publishing that model on ollama.

In my case I fine-tuned SmolLM-360M to be a better assistant for my Pi-Card (previous post) project.

Check it out!
https://github.com/nkasmanoff/ft-flow


Small-Text: Active Learning for Text Classification in Python (2107.10314)

🔔 Release: small-text v1.4.1

The new release contains some smaller bugfixes. Check it out!

Github: https://github.com/webis-de/small-text
Paper: https://huggingface.co/papers/2107.10314


🚀  We’re excited to launch Ghost 8B Beta (1608), a top-performing language model with unmatched multilingual support and cost efficiency.

Key Highlights:
- Superior Performance: Outperforms Llama 3.1 8B Instruct, GPT-3.5 Turbo, Claude 3 Opus, GPT-4, and more in winrate scores.
- Expanded Language Support: Now supports 16 languages, including English, Vietnamese, Spanish, Chinese, and more.
- Enhanced Capabilities: Improved math, reasoning, and instruction-following for better task handling.

With two context options (8k and 128k), Ghost 8B Beta is perfect for complex, multilingual applications, balancing power and cost-effectiveness.

🔗 Learn More: https://ghost-x.org/docs/models/ghost-8b-beta
https://huggingface.co/collections/ghost-x/ghost-8b-beta-668ead6179f93be717db4542

victor stylooantlabs nbroad Joseph717171 ZeroWw


Wrote a blog post with some ideas about prompt engineering

https://huggingface.co/blog/KnutJaegersberg/first-principles-prompt-engineering

prithivMLmods John6666 victor osanseviero appvoid


I'm excited to share a really cool milestone in my AI/LLM journey.

Brief backstory: Before diving into AI, I spent over a decade working in ecological fields such as the conservation corps, biodynamic farming, and natural habitat restoration. This background instilled in me a deep concern about the environmental impact of scaling AI without sustainable practices.

Driven by this concern, I've spent months planning and experimenting to make my AI work more eco-friendly. I'm thrilled to announce that I've successfully transitioned my entire operation to run on 100% sustainable solar power!

My current setup includes multiple linked Mac Pro tower desktops and custom code built from open-source libraries. While it's a bit experimental, this configuration is working great for my needs. All my LLM research, development, and client services now run exclusively on solar energy.

I'm curious if anyone else here has experimented with renewable energy for their LLM work?

For those interested in more details, I've written a brief blog post about this journey here https://medium.com/@betalabsllm/powering-the-future-be-ta-labs-revolutionary-100-solar-powered-ai-operation-444433e61d43

victor ajibawa-2023 osanseviero shawon Olafangensan Bruhn John6666 vipulmaheshwari kumail1


How good are you at spotting AI-generated images?

Find out by playing Fake Insects 🐞 a Game where you need to identify which insects are fake (AI generated). Good luck & share your best score in the comments!

https://huggingface.co/spaces/victor/fake-insects

cschroeder Nenux John6666 Clausss ipushprajyadav diwank osanseviero DiamanteAmarelo


So turns out I've been spreading a bit of misinformation when it comes to imatrix in llama.cpp

It starts true; imatrix runs the model against a corpus of text and tracks the activation of weights to determine which are most important

However what the quantization then does with that information is where I was wrong.

I think I made the accidental connection between imatrix and exllamav2's measuring, where ExLlamaV2 decides how many bits to assign to which weight depending on the goal BPW

Instead, what llama.cpp with imatrix does is it attempts to select a scale for a quantization block that most accurately returns the important weights to their original values, ie minimizing the dequantization error based on the importance of activations

The mildly surprising part is that it actually just does a relatively brute force search, it picks a bunch of scales and tries each and sees which one results in the minimum error for weights deemed important in the group

But yeah, turns out, the quantization scheme is always the same, it's just that the scaling has a bit more logic to it when you use imatrix

Huge shoutout to @compilade for helping me wrap my head around it - feel free to add/correct as well if I've messed something up

MaziyarPanahi victor Themoneybuddha John6666 bullerwins Rybens Anderson452 NOBODYVONNOTHING qnixsynapse Clausss exdysa AtAndDev IAmTheCollector wsuff rreed-pha djuna MarinaraSpaghetti Pomni RakshitAralimatti osanseviero venketh Arakinas wonderboy nicoboss Joseph717171 AIGUYCONTENT Handgun1773 bradeenl EloyOn

Aurelien-Morgan Clausss AtAndDev sroecker iandol rreed-pha ajibawa-2023 ijohn07 thiera1 nbeerbower newsletter venketh mpk63 wonderboy clem nicoboss jsfs11 Joseph717171 waldie fendiprime

rreed-pha wonderboy nicoboss Joseph717171


I know a secret about knowledge graphs that the world doesn't! There are severe mathematical limitations to geometric fractals. It is classed as an 'unsolvable problem' in the mathematics world. There are currently ~1,000 mathematicians or so in the world that give this problem serious thought. You literally cannot solve it with geometric fractals. This is why I invented P-FAF, it uses calculus based fractals instead. I literally invented the math to make it work. I solved an 'unsolvable equation' to make the math work. You ONLY make the math work the way I did it in the end. I have never released the licensing commercially. Good luck!


Supercool Weekend Read🤖
Nvidia researchers achieved SOTA LLM compression metrics using pruning and knowledge distillation techniques.

Details on Techniques (Simplified):
They started off with a large pre-trained language model (15B params), then:

1. Estimated the importance of different parts of the model (neurons, attention heads, layers) using activation-based metrics on a small calibration dataset.

2. Pruned (remove) less important parts of the model to reduce its size.

3. Retrained the pruned model using knowledge distillation, where the original large model acts as a teacher for the smaller pruned model.

4. Used a lightweight neural architecture search to find the best configuration for the pruned model.

5. Repeated this process iteratively to create even smaller models.

Cool, giving it a try this weekend 😎
Code: https://github.com/NVlabs/Minitron
Paper: https://arxiv.org/abs/2407.14679
Demo: https://huggingface.co/spaces/nvidia/minitron


Added FLUX.1 pro/dev/schnell and AuraFlow v0.2 to  https://huggingface.co/spaces/fal/imgsys !!! Go play with it and get us some votez


NEW math-instruct model + dataset!

https://huggingface.co/ValiantLabs/Llama3.1-8B-Cobalt is our new math-instruct model.
Trained using a synthetic math-instruct dataset generated with Llama 3.1 405b. Find the dataset here: https://huggingface.co/datasets/sequelbox/Polytope

More to come soon :)


Introducing HelpingAI2-9B, an emotionally intelligent LLM.
Model Link : https://huggingface.co/OEvortex/HelpingAI2-9B
Demo Link: https://huggingface.co/spaces/Abhaykoul/HelpingAI2

This model is part of the innovative HelpingAI series and it stands out for its ability to engage users with emotional understanding.

Key Features:
-----------------

*   It gets 95.89 score on EQ Bench greather than all top notch LLMs, reflecting advanced emotional recognition.
*   It gives responses in empathetic and supportive manner.

Must try our demo: https://huggingface.co/spaces/Abhaykoul/HelpingAI2


Announcing another BIG data drop! This time it's ~275M images from Flickr https://huggingface.co/datasets/bigdata-pw/Flickr

Data acquisition for this project is still in progress, get ready for an update soon:tm:

In case you missed them; other BIG data drops include Diffusion1B https://huggingface.co/datasets/bigdata-pw/Diffusion1B - ~1.23B images and generation parameters from a variety of diffusion models and if you fancy practicing diffusion model training check out Dataception https://huggingface.co/datasets/bigdata-pw/Dataception - a dataset of over 5000 datasets in WebDataset format!

Requests are always welcome so reach out if there's a dataset you'd like to see!


✨ Feeling thankful...

🇮🇳 15th August, 2024; on India's 78th Independence Day

🎉 Crossed 100 followers on Hugging Face

🏆 Got LinkedIn Top Voice

🤖 AI has never been more exciting and I am here for it

👀 @clem Can I be a Hugging Face fellow now?


Improved ControlNet!
Now supports dynamic resolution for perfect landscape and portrait outputs. Generate stunning images without distortion—optimized for any aspect ratio!
...
https://huggingface.co/spaces/DamarJati/FLUX.1-DEV-Canny


As some of you know, I try to convert models to either fp32 or bf16 depending on theirs size before doing imatrix and quantization

Today I decided to see if that matters, and the results have me.. for lack of a better word, perplexed

My setup:

Mistral Nemo Instruct 2407
- convert to FP32, calculate imatrix, quantize to Q8_0 and Q4_K_M
- convert to FP16, calculate imatrix, quantize to Q8_0 and Q4_K_M

I calculated the kld base from the FP32 model:
`./llama-perplexity -m /models/Mistral-Nemo-Instruct-2407-f32.gguf -f /training_data/wikitext-2-raw/wiki.test.raw --kl-divergence-base /training_data/mistral-nemo-f32.kld -ngl 35 -fa -sm row`

then calculated the divergence itself for each like so:
`./llama-perplexity -m /models/Mistral-Nemo-Instruct-2407-Q8_0.gguf -f /training_data/wikitext-2-raw/wiki.test.raw --kl-divergence-base /training_data/mistral-nemo-f32.kld --kl-divergence -ngl 50 -fa -sm row`

Q4_K_M from fp16 and fp32 were similar, trading blows across statistics, odd since i expected fp32 to be strictly better but it's not

Q8_0 is where things get weird. Despite each file being slightly different size, and the sha256sum of course being different, they each get *completely identical* scores, down to 6 decimal places of precision on the statistics.

How is this possible? Is there something I don't understand about llama.cpp that makes it always convert to fp16 before it does quantization? Am I wasting time using FP32/BF16??

Rybens John6666 Nelathan YaTharThShaRma999 MarinaraSpaghetti DeathGodlike osanseviero julien-c pcuenq mlabonne Joseph717171

nazimali osanseviero julien-c Joseph717171


https://huggingface.co/fal/AuraFlow-v0.3 is now here with support for different aspect resolutions (w/h up to 1536px!) and much nicer aesthetics! Make sure to install the latest diffusers to get support for it.

gokaygokay John6666 YaTharThShaRma999 NHLOCAL Sri-Vigneshwar-DJ osanseviero isidentical adamelliotfields

prithivMLmods osanseviero Fetah06 isidentical


𝗭𝗲𝗿𝗼-𝗺𝗮𝘁𝗵 𝗶𝗻𝘁𝗿𝗼 𝘁𝗼 𝗔𝗜 𝗵𝗶𝘀𝘁𝗼𝗿𝘆: 𝗳𝗿𝗼𝗺 𝘁𝗵𝗲 𝟭𝟵𝟱𝟬𝘀 𝘁𝗼 𝘁𝗼𝗱𝗮𝘆'𝘀 𝗟𝗟𝗠𝘀 📖

I wanted to structure my thinking about LLMs by going through their history since the 50s. This history is captivating, with the opposition between Connexionists (Rosenblatt, LeCun) and Symbolists, the first victories of "deep" neural networks, the revolution of Attention...

So I might have gone a bit too far! 😅

📝 I've made a long post summarizing the main stages of building LLMs: neural networks, optimization, backpropagation, attention layers...

✅ And I've made sure to keep it 100% horrible-latex-math-free: the technical stuff is conveyed in graphs only, so it should be accessible to really anyone, even your grandfather (I'm sending it to mine right now).

Read it here in english 👉 https://aymeric-roucher.github.io/brief-history-of-ai/
Pour le post en français 👉 https://aymeric-roucher.github.io/breve-histoire-de-l-ia/


Some personal and professional news ✨

I'm writing a book on ML metrics.

Together with Wojtek Kuberski, we’re creating the missing piece of every ML university program and online course: a book solely dedicated to Machine Learning metrics!

The book will cover the following types of metrics:
• Regression
• Classification
• Clustering
• Ranking
• Vision
• Text
• GenAI
• Bias and Fairness

👉 check out the book: https://www.nannyml.com/metrics


Everchanging Quest is out !

It is an LLM controlled Rogue-Like in which the LLM gets a markdown representation of the map, and should generate a JSON with the objective to fulfill on the map as well as the necessary objects and their placements.

Come test it on the space :
https://huggingface.co/spaces/Jofthomas/Everchanging-Quest

fractalego mrfakename Svngoku Aurelien-Morgan mmhamdy boapps osanseviero jsgreenawalt Joseph717171 ChuckMcSneed victor adamelliotfields Lacaz64 julien-rodriguez philschmid philipp-zettl AndrewNa azuremous haikunns vip3 comarproject hitchhiker3010 palsp

John6666 mmoy Svngoku osanseviero jsgreenawalt Joseph717171 ChuckMcSneed Lacaz64 philschmid AndrewNa


ehm, so today i've finished my new project

https://huggingface.co/spaces/Blane187/animalese-py

or you can make your voice to animalese with it:
https://huggingface.co/spaces/Blane187/animalese_RVC

i'm just bored, so i make the project, lol


📣 Introducing Dataset Viber: your chill repo for data collection, annotation and vibe checks! 🎉

I've cooked up Dataset Viber, a set of cool tools designed to make data preparation for AI models easier, more approachable and enjoyable for standalone AI engineers and enthusiasts.

🔧 What Dataset Viber offers:
- CollectorInterface: Lazily collect model interaction data without human annotation
- AnnotatorInterface: Annotate your data with models in the loop
- BulkInterface: Explore data distribution and annotate in bulk
- Embedder: Efficiently embed data with ONNX-optimized speeds

🎯 Key features:
- Supports various tasks for text, chat, and image modalities
- Runs in .ipynb notebooks
- Logs data to local CSV or directly to Hugging Face Hub
- Easy to install via pip: `pip install dataset-viber`

It's not designed for team collaboration or production use, but rather as a fun and efficient toolkit for individual projects.

Want to give it a try? Check out the repository link https://github.com/davidberenstein1957/dataset-viber/.

I'm excited to hear your feedback and learn how you vibe with your data. Feel free to open an issue or reach out if you have any questions or suggestions!

Some shoutouts:
- Gradio for the amazing backbone
- Daniel van Strien for some initial presentations I did on vibe checks
- Emily Omier for the workshop on structuring GitHub repo READMEs
- Hamel Husain for keeping mentioning that people should look at their data.
- Philipp Schmid for his code for ONNX feature-extractors
- Ben Burtenshaw for the first PR

Best-codes maywell osanseviero OrigamiDream


Here is a hackable and minimal implementation showing how to perform distributed text-to-image generation with Diffusers and Accelerate.

Full snippet is here: https://gist.github.com/sayakpaul/cfaebd221820d7b43fae638b4dfa01ba

With @JW17


hey everyone, if you have any experience with fine tuning ai models please feel free to reach out to me :)
We're looking for devs to fine tune an AI model for a snapchat chatting AI project. Open to either part time or full time work.

Email: admin@socialrisestudio.org
Telegram: Kirann46


Seeing and Understanding: Bridging Vision with Chemical Knowledge Via
  ChemVLM (2408.07246)

🚀 Introducing ChemVLM, the first open-source multimodal large language model dedicated to chemistry!
🌟Comparable performances with commercial models or specific OCR model but with dialogue capabilities!
✨2B/26B Models Here! https://huggingface.co/AI4Chem/ChemVLM-26B
https://huggingface.co/papers/2408.07246

mmhamdy Riswan-Nopiyar sequelbox Ramikan-BR Bruhn


📸Photo LoRA Drop📸

I've been working on this one for a few days, but really I've had this dataset for a few years! I collected a bunch of open access photos online back in late 2022, but I was never happy enough with how they played with the base model!

I am so thrilled that they look so nice with Flux!

This for me is a version one of this model - I still see room for improvement and possibly expansion of it's 40 image dataset. For those who are curious:

40 Image
3200 Steps
Dim 32
3e-4

Enjoy! Create! Big thank you to Glif for sponsoring the model creation! :D

https://huggingface.co/alvdansen/flux_film_foto


AuraSR Giga Upscaler V1 by SECourses - Upscales to 4x

AuraSR is a 600M parameter upsampler model derived from the GigaGAN paper. It works super fast and uses a very limited VRAM below 5 GB. It is deterministic upscaler. It works perfect in some images but fails in some images so it is worth to give it a shot.

GitHub official repo : https://github.com/fal-ai/aura-sr

I have developed 1-click installers and a batch upscaler App.

You can download installers and advanced batch App from below link:
https://www.patreon.com/posts/110060645

Check the screenshots and examples below

Windows Requirements

Python 3.10, FFmpeg, Cuda 11.8, C++ tools and Git

If it doesn't work make sure to below tutorial and install everything exactly as shown in this below tutorial

https://youtu.be/-NjNy7afOQ0

How to Install and Use on Windows

Extract the attached GigaGAN_Upscaler_v1.zip into a folder like c:/giga_upscale

Then double click and install with Windows_Install.bat file

It will generate an isolated virtual environment venv folder and install requirements

Then double click and start the Gradio App with Windows_Start_App.bat file

When first time running it will download models into your Hugging Face cache folder

Hugging Face cache folder setup explained below

https://www.patreon.com/posts/108419878

All upscaled images will be saved into outputs folder automatically with same name and plus numbering if necessary

You can also batch upscale a folder

How to Install and Use on Cloud

Follow Massed Compute and RunPod instructions

Usage is same as on Windows

For Kaggle start a Kaggle notebook, import our Kaggle notebook and follow the instructions

App Screenshots and Examples below

MonsterMMORPG davisbro johko Riswan-Nopiyar ajibawa-2023


Introducing Fineweb-Edu-Fortified: An enhanced Fineweb-Edu dataset. 📚

This dataset is tailored for NLP tasks and helps streamline model training by offering a more refined, unique dataset. Perfect for startups and researchers looking for high-quality educational content to train, evaluate, or fine-tune AI models. The dataset is based on the Fineweb-Edu subset of the large Fineweb dataset and includes:

- Exact-match deduplication across all crawls
- Embeddings for each row using the TaylorAI/bge-micro model
- Count column indicating duplication frequency
- Includes data from 95 Common Crawl crawls (2013-2024)
- Rows have been reduced from 1.279B to 0.324B after deduplication
- It is comprised of ~375B tokens (down from 1,320B in Fineweb-Edu)

Access the entire Fineweb-Edu-Fortified dataset on Hugging Face → https://huggingface.co/datasets/airtrain-ai/fineweb-edu-fortified

Try a semantic search demo via this Hugging Face Space → https://huggingface.co/spaces/airtrain-ai/fineweb-edu-fortified-search-demo

Many thanks to the amazing @josh-sematic for his work on this project, the Fineweb/Fineweb-Edu team at Hugging Face for producing the original datasets and for their support during our work on Fineweb-Edu-Fortified, and also thanks to @underspirit for pointing out the reduction in dataset size that could be achieved via deduplication. 🤗

emmanuel-turlay SinclairWang eljanmahammadli LeroyDyer jakemannix debuxe mmhamdy Riswan-Nopiyar osanseviero gabrielmbmb josh-sematic davanstrien VictorSanh nazimali lev-channel djuna

insumanth mmhamdy josh-sematic davanstrien VictorSanh lev-channel


Let’s see JEPA in action🤖
Simplified image-based implementation training on a CPU with live preview support - very satisfying to watch:)

I-JEPA is the image-based version of JEPA (Joint-Embedding Predictive Architecture - an alternative to autoregressive LLM architectures ) pioneered by professor Yann Lecun.

At a higher level, I-JEPA predicts image segment representations (Target) based on representations of other segments within the same image (Context). It consists of three key components: a context encoder, target encoder and a predictor.

Code: https://github.com/Jaykef/ai-algorithms/blob/main/mnist_ijepa.ipynb

emirhanbilgic jakemannix Riswan-Nopiyar osanseviero ibrahim313


If you are interested in deep reinforcement learning, find my recent survey below:

A Survey Analyzing Generalization in Deep Reinforcement Learning
Paper: https://arxiv.org/pdf/2401.02349
GitHub: https://github.com/EzgiKorkmaz/generalization-reinforcement-learning


It took Google’s Transformer model from 2017 a whopping $900 to train. 💸

This in contrast to the $191 million Google spent on Gemini Ultra sounds like a bargain! 💰

Gemini Ultra required 50 billion petaFLOPS (one petaFLOP equals one quadrillion FLOPs). 🤖
Compared to OpenAI’s GPT-4, which required 21 billion petaFLOPS, at a cost of $78 million. 💡

2017: Original Transformer Model: $930 [@Google ] 💻
2018: BERT-Large: $3,288 [@Google] 📚
2019: RoBERTa Large: 160k [@Meta] 🌐
2020: GPT-3(175B): $4.32M [@OpenAI] 🧠
2023: Llama 2 70B: $3.93M [@Meta] 🐑
2023: GPT-4: $78.35M [@OpenAI] 🌟
Now, Gemini Ultra: $191.4M [@Google] 🚀

This forms an exponential curve! 🤯

But, why? 🤔
Compute, data, and expertise. All three come at a great cost! ⚙📊💡

Google recently made Gemini-1.5-Flash fine-tuning free, as it's almost impossible for regular businesses to justify an in-house trained foundational model! 🆓

This barrier of cost is going to result in fewer new foundational models/less competition and more fine-tunes! 📉🔄

Data [Stanford University’s 2024 AI Index Report]: https://aiindex.stanford.edu/report/
Graphic: https://voronoiapp.com/technology/Googles-Gemini-Ultra-Cost-191M-to-Develop--1088

Many thanks to everyone spending tons of resources and open-sourcing the models! 🤗

Pavun Felladrin Riswan-Nopiyar paulaju thisisanshgupta Abduraxim NHLOCAL prithivMLmods awhsnap kmsky


@papepipopu hey would you like to join my team me and my partner created a trading bot now we are diving into the machine learning world if you are intrested email me nebyoulema@gmail.com


🚨 Code now available for "Using Large Language Models for Hyperparameter Optimization" at https://github.com/michaelrzhang/LLM-HyperOpt 🚨

TLDR: You can just ask LLMs which hyperparameters to use, and it works pretty well! You can even directly optimize your model’s code as a hyperparameter with this.

Check out the paper at https://arxiv.org/abs/2312.04528 - with Michael Zhang, Nishkrit Desai, Juhan Bae, and Jimmy Ba


nroggendorff Zery-An s3nh osanseviero John6666 MicahB


Most LLMs are not reproducible because the underlying deep neural networks are not. Because that's something LLM creators don't care about. We do, and ours are reproducible, including our GenAI that uses GAN.

All you have to do is allow the user to specify the seeds of the random number generators involved. First, you need a good random generator you have full control over. Better than numpy.random. See ours, with infinite period and one line of code, faster and better than what's in Python and elsewhere. Here is the link:  https://mltblog.com/4fGDLu0


Chomsky predicting LLMs in 1956, curated by Ryan Rhodes (Rutgers)


mobiuslabsgmbh/Llama-3.1-70b-instruct_4bitgs64_hqq

99% of the performance across various benchmarks!

https://huggingface.co/mobiuslabsgmbh/Llama-3.1-70b-instruct_4bitgs64_hqq

osanseviero Clausss wengaozhu victor DmitryRyumin nguyen599 mobicham


𝚗𝚙𝚖 𝚒 @𝚑𝚞𝚐𝚐𝚒𝚗𝚐𝚏𝚊𝚌𝚎/𝚝𝚛𝚊𝚗𝚜𝚏𝚘𝚛𝚖𝚎𝚛𝚜

𝚗𝚙𝚖 𝚒 @𝚑𝚞𝚐𝚐𝚒𝚗𝚐𝚏𝚊𝚌𝚎/𝚝𝚛𝚊𝚗𝚜𝚏𝚘𝚛𝚖𝚎𝚛𝚜

I'm excited to announce that Transformers.js V3 is finally available on NPM! 🔥 State-of-the-art Machine Learning for the web, now with WebGPU support! 🤯⚡

Install it from NPM with:
𝚗𝚙𝚖 𝚒 @𝚑𝚞𝚐𝚐𝚒𝚗𝚐𝚏𝚊𝚌𝚎/𝚝𝚛𝚊𝚗𝚜𝚏𝚘𝚛𝚖𝚎𝚛𝚜

or via CDN, for example: https://v2.scrimba.com/s0lmm0qh1q

Segment Anything demo: https://huggingface.co/spaces/webml-community/segment-anything-webgpu

lamhieu victorjalonzo KayO Felladrin Svngoku John6666 djuna louisbrulenaudet Catering3733 jbuckland devmandan osanseviero nmstoker adamelliotfields wengaozhu victor Marvin73 sugatoray Afeezee bootz15 sospoon Bowen7 Youngwon ctranslate2-4you okasi prithivMLmods MicahB aust-t Arvin8613 Snagy22000 GordonM jweston do-me varun4 hammeiam pat777 alessandrobellesia jkorstad

jbuckland Jaward osanseviero ajibawa-2023 victor rockerBOO wonderboy okasi MicahB GordonM

BoscoTheDog Svngoku osanseviero okasi MicahB GordonM

adamelliotfields Youngwon emirhanbilgic okasi MicahB GordonM


Hey HF. I just released a new reward modelling dataset: https://huggingface.co/datasets/Avelina/UltraSteer-v0

UltraSteer-V0 is a massive collection of single- and multi-turn dialogue with fine-grained reward labels produced by Nvidia's https://huggingface.co/nvidia/Llama2-13B-SteerLM-RM reward model. We have a total of 2.3M labelled sequences taken from high quality datasets with a total of 2.8M labelled turns each containing 9 attributes produced as is from the reward model.

This is still very much an early version of the dataset (but it's fully usable!) and an updated version will be on the way with a full paper.

I would really appreciate if people could take a look at the dataset and suggest any improvements (e.g. more data sources, different cleaning approaches, different label schema, etc) in the community section.

osanseviero gabrielmbmb John6666 Bruhn louisbrulenaudet rreed-pha davanstrien Gatozu35


Releasing HQQ Llama-3.1-70b 4-bit quantized version! Check it out at https://huggingface.co/mobiuslabsgmbh/Llama-3.1-70b-instruct_4bitgs64_hqq.

Achieves 99% of the base model performance across various benchmarks! Details in the model card.


FalconMamba  7B - a new model from TII (Technology Innovation Institute) is out !

- Blogpost: https://huggingface.co/blog/falconmamba
- Link to collection: https://huggingface.co/collections/tiiuae/falconmamba-7b-66b9a580324dd1598b0f6d4a
- Link to playground: https://huggingface.co/spaces/tiiuae/falcon-mamba-playground

osanseviero Norod78 han1997 Felladrin louisbrulenaudet valeriocardoso ucsahin ajibawa-2023 wengaozhu not-lain s3nh StephenGenusa


If you are interested in Knowledge Graphs, I invented all of this a year ago. It is a Encoder/Decoder that works with Knowledge Graphs. I am glad the world finally realizes this is useful a year later. I tried to tell you. I have not licensed any of the math. I own all of it. I do not have any plans to ever enforce the licensing but I like holding onto it.

https://huggingface.co/blog/TuringsSolutions/pfafresearch

GianaDiana merterbak LeroyDyer Tanvir1337 wengaozhu Chunte


Here is an AI Puzzle!
When you solve it just use a 😎 emoji.
NO SPOILERS
A similar puzzle might have each picture that has a hidden meaning of summer, winter, fall, spring, and the answer would be seasons.

Its a little dated now (almost a year), so bottom right might be tough.

Thanks to @johko for the encouragement to post!


How can i check my resources limitation and usage?


SFT + Quantisation + Unsloth is a super easy way of squeezing extra performance out of an LLM at low latencies. Here are some hand y resources to bootstrap your projects.

Here's a filtered dataset from Helpsteer2 with the most correct and coherent samples: https://huggingface.co/datasets/burtenshaw/helpsteer-2-plus
This is a SFT finetuned model: ttps://huggingface.co/burtenshaw/gemma-help-tiny-sft
This is the notebook I use to train the model: https://colab.research.google.com/drive/17oskw_5lil5C3jCW34rA-EXjXnGgRRZw?usp=sharing
Here's a load of Unsloth notebook on finetuning and inference: https://docs.unsloth.ai/get-started/unsloth-notebooks

gabrielmbmb patiencerichard osanseviero John6666


🚀 RAGoon is now available on PyPI, GitHub, and as a Space on Hugging Face for batched embeddings generation 🤗

RAGoon is a set of NLP utilities for multi-model embedding production, high-dimensional vector visualization, and aims to improve language model performance by providing contextually relevant information through search-based querying, web scraping and data augmentation techniques.

At this stage, 5 major classes are available via RAGoon to facilitate:
- the production of chain embeddings for several models to simplify a continuous deployment process;
- production of LLM requests for web querying and content retrieval via the Google API;
- recursive chunking via tokens;
- data visualization and the function to load embeddings from a FAISS index, reduce their dimensionality using PCA and/or t-SNE, and visualize them in an interactive 3D graph;
- the creation of binary indexes for search with scalar (int8) rescoring.

Link to GitHub: https://github.com/louisbrulenaudet/ragoon
Link to the 🤗 Space: https://huggingface.co/spaces/louisbrulenaudet/ragoon


BiRefNet State Of The Art Newest Very Best Background Batch Remover APP

Official repo : https://github.com/ZhengPeng7/BiRefNet

Download APP and installers from : https://www.patreon.com/posts/109913645

Hugging Face Demo : https://huggingface.co/spaces/ZhengPeng7/BiRefNet_demo

I have developed a very advanced Gradio APP for this with full proper file saving and batch processing. Also my version removes BG and saves as transparent background.

The APP uses huge VRAM for high resolution images. However it is still working uber fast even though using shared VRAM. So make sure that you have high RAM or set virtual RAM.

Click below to see how to set virtual RAM on Windows.
https://www.windowscentral.com/how-change-virtual-memory-size-windows-10

On Massed Compute A6000 GPU (31 cents per hour) you can very fast remove even very high res images backgrounds.

Currently we have 1 click installers for RunPod, Massed Compute, Kaggle and Windows.

Windows Requirements
Python 3.10, FFmpeg, Cuda 11.8, C++ tools and Git

If it doesn't work make sure to below tutorial and install everything exactly as shown in this below tutorial

https://youtu.be/-NjNy7afOQ0

How To Use On Windows
Just extract files into like c:/BiRefNet_v1

Double click Windows_Install.bat file and it will generate a isolated virtual environment and install requirements

It will automatically download models into your Hugging Face cache (best model under 1 GB)

Then start and use the Gradio APP with Windows_Start_App.bat

Cloud How To Use
Massed Compute, RunPod has instructions txt files. Follow them

Kaggle has all the instructions 1 by 1

On Kaggle set resolution 1024x1024 or you will get out of memory error

MonsterMMORPG myrkur osanseviero eieihihi Ani8Face Bruhn svenard


AutoGen from @Microsoft is crazy! 🚀 It's an open-source framework that allows LLM agents to chat with each other to solve your tasks. 🤖💬

They use the Assistant-Agent and User-Proxy-Agent framework! 🛠

As the name suggests, the Assistant-Agent does the work, and the User-Proxy-Agent behaves like a human, guiding the Assistant-Agent and double-checking its work! 🧑💻✅

Both Assistant-Agent and User-Proxy-Agent can be the same or different LLMs. 🤔🔄

AutoGen is an open-source programming framework for building AI agents and facilitating cooperation among multiple agents to solve tasks. 🌟

This is truly amazing for building agentic AI quickly! 🚀✨

GitHub: https://github.com/microsoft/autogen 🔗

```Python
from autogen import AssistantAgent, UserProxyAgent, config_list_from_json

#config
config_list = config_list_from_json(env_or_file="OAI_CONFIG_LIST")

assistant = AssistantAgent("assistant", llm_config={"config_list": config_list})
user_proxy = UserProxyAgent("user_proxy", code_execution_config={"work_dir": "coding", "use_docker": False})

user_proxy.initiate_chat(assistant, message="Plot a chart of NVDA and TESLA stock price change YTD.")
# This initiates an automated chat between the two agents to solve the task
```


LLaVA-o1: Let Vision Language Models Reason Step-by-Step (2411.10440)

LLaVA-o1 🔥  NEW visual language model with spontaneous and systematic reasoning, like GPT-o1!

Paper: https://huggingface.co/papers/2411.10440
Github: https://github.com/PKU-YuanGroup/LLaVA-o1
✨ Autonomous Multistage Reasoning
✨ Efficient with Small Data: Trained on 100k samples
✨ Innovative Inference: Stepwise beam search boosts precision & scalability in reasoning.


🖼 Introducing Public Domain Pictures Dataset - https://huggingface.co/datasets/nyuuzyou/publicdomainpictures

Dataset highlights:
- 644,412 public domain images with comprehensive metadata from publicdomainpictures.net
- English language metadata including titles, descriptions, and keywords
- Each entry contains rich metadata including:
  - Unique image ID and full-size image URLs
  - Detailed titles and descriptions
  - Keyword/tag collections
  - Creator attribution
- Released to the public domain under Creative Commons Zero (CC0) license


You can clean and format datasets entirely in the browser with a few lines of SQL.

In this post, I replicate the process @mlabonne used to clean the new https://huggingface.co/datasets/microsoft/orca-agentinstruct-1M-v1 dataset.

The cleaning process consists of:
- Joining the separate splits together / add split column
- Converting string messages into list of structs
- Removing empty system prompts

https://huggingface.co/blog/cfahlgren1/the-beginners-guide-to-cleaning-a-dataset

Here's his new cleaned dataset: https://huggingface.co/datasets/mlabonne/orca-agentinstruct-1M-v1-cleaned

clem lunarflu mlabonne John6666 Felladrin m-ric kramp nbroad qgallouedec Nymbo victor yjernite mmhamdy RedSparkie eliebak knight7561 SaylorTwift


Some time ago, I built a predictive LLM router that routes chat requests between small and large LLM models based on prompt classification. It dynamically selects the most suitable model depending on the complexity of the user input, ensuring optimal performance while maintaining conversation context. I also fine-tuned a RoBERTa model to use with the package, but you can plug and play any classifier of your choice.

Project's homepage:
https://devquasar.com/llm-predictive-router/
Pypi:
https://pypi.org/project/llm-predictive-router/
Model:
https://huggingface.co/DevQuasar/roberta-prompt_classifier-v0.1
Training data:
https://huggingface.co/datasets/DevQuasar/llm_router_dataset-synth
Git:
https://github.com/csabakecskemeti/llm_predictive_router_package

Feel free to check it out, and/or contribute.

The-Rat-King lunarflu Csplk John6666 Sengil SugoiLoki tushifire


🤗 transformers pipelines now support vision language models for easy local inference 🫰🏻
h/t @yonigozlan for shipping this 🎩👏

you can also use inference API to infer hosted vision LMs (via Python, JS and cURL) https://huggingface.co/docs/api-inference/en/tasks/image-text-to-text

John6666 yonigozlan lunarflu Aurelien-Morgan celinah Nagase-Kotono pcuenq Norod78 OmbelineM


Build a fine-tuning dataset with No Code.

Do you want to build a small dataset for creative writing to fine-tune an Open LLM?
- Find a dataset full of conversations with ChatGPT on the Hugging Face Hub.
- Import it into your Argilla Space.
- Preview the dataset and create a question to label the relevant conversations.
- Label 1000 valid examples of creating writing.
- Use this dataset with Autotrain to fine-tune your model.

John6666 lunarflu clem hoduyquocbao veeraleto fuzzy-mittenz


For anyone who struggles with NER or information extraction with LLM.

We showed an efficient workflow for token classification including zero-shot suggestions and model fine-tuning with Argilla, GliNER, the NuMind NuExtract LLM and SpanMarker. @argilla

Video: https://youtu.be/JvLpaYgNd84?feature=shared
Notebooks and slides included to try it yourself 🙂

davidberenstein1957 clem merve lunarflu dark-pen victor

davidberenstein1957 John6666 clem merve lunarflu ArthurZ


It's been a while we shipped native quantization support in `diffusers` 🧨

We currently support `bistandbytes` as the official backend but using others like `torchao` is already very simple.

This post is just a reminder of what's possible:

1. Loading a model with a quantization config
2. Saving a model with quantization config
3. Loading a pre-quantized model
4. `enable_model_cpu_offload()`
5. Training and loading LoRAs into quantized checkpoints

Docs:
https://huggingface.co/docs/diffusers/main/en/quantization/bitsandbytes

John6666 clem lunarflu rwightman not-lain den0620 pcuenq OmbelineM


LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models (2411.09595)

LLaMA Mesh 🔥 Unifying 3D Mesh Generation with Language Models

Model: https://huggingface.co/Zhengyi/LLaMA-Mesh
Demo: https://huggingface.co/spaces/Zhengyi/LLaMA-Mesh
Paper: https://huggingface.co/papers/2411.09595

✨ Unified 3D generation & text understanding.
✨ 3D meshes as plain text for seamless LLM integration.
✨ High-quality 3D outputs rivaling specialized models.


A while ago I started experimenting with compiling the Python interpreter to WASM.

To build a secure, fast, and lightweight sandbox for code execution — ideal for running LLM-generated Python code.

- Send code simply as a POST request
- 1-2ms startup times

Hack away:
https://github.com/ErikKaum/runner

davidberenstein1957 clem cfahlgren1 merve lunarflu Aurelien-Morgan nbroad veeraleto

davidberenstein1957 John6666 merve lunarflu Aurelien-Morgan veeraleto


NEW MODEL + DATASET! :)

Check out Enigma, our new code-instruct model:
- trained on synthetic code-instruct data created with Llama 3.1 405b
- high quality code-instruct within the Llama 3.1 Instruct format

The model: https://huggingface.co/ValiantLabs/Llama3.1-8B-Enigma
The dataset: https://huggingface.co/datasets/sequelbox/Tachibana

Enjoy! We've got more new datasets and models to follow soon.


Alright Ya'll

I know it's a Saturday, but I decided to release my first Flux Dev Lora.

A retrain of my "Frosting Lane" model and I am sure the styles will just keep improving.

Have fun! Link Below - Thanks again to @ostris  for the trainer and Black Forest Labs for the awesome model!

https://huggingface.co/alvdansen/frosting_lane_flux

pip-nov yuchenj GPT007 mattmdjaga kimleang123 Ramikan-BR MaziyarPanahi DiamanteAmarelo deadhanded louisbrulenaudet Leotrim jyoung105

Rubia57 Sri-Vigneshwar-DJ Ramikan-BR MaziyarPanahi Nelathan DiamanteAmarelo Leotrim Norod78 jyoung105 kianxyz

Nelathan DiamanteAmarelo deadhanded Leotrim attashe


Hello Community,

I am seeking guidance on solving a complex problem related to tender categorization. Here's a detailed overview of my requirements and challenges:

Problem Statement:

I need to categorize tenders based on their descriptions on a daily basis.
I process over 35,000 tenders daily, each potentially belonging to a vast range of categories. Predicting the exact category for each tender is challenging due to the diversity and complexity of categories.
Requirements:

Model Selection: I require a model that can accurately predict the category of each tender based on its description. I plan to create a dataset from 1 million records in the required format to train the model. Post-training, I also need to optimize the model for performance.
Training and Optimization: Guidance on the best practices for training and optimizing the model is essential. I am considering Hugging Face’s tools but am unsure about how to effectively use them or any alternatives.
Local Deployment: If I opt for a local deployment, I need to know what kind of infrastructure or applications are required to run the model on my own servers efficiently.
Performance: The model must provide responses quickly, given the high volume of tenders.
Questions for the Community:

Model Recommendations:

Which model or method would be most suitable for categorizing tenders with diverse categories?
What are the best practices for training and optimizing such a model?
Infrastructure and Deployment:

What infrastructure or software is needed to deploy a model locally for this purpose?
How can I ensure the model performs efficiently on my server?
Performance Optimization:

What strategies or techniques can I use to achieve quick response times from the model?
I appreciate any advice, suggestions, or resources you can provide to help address these challenges.

Thank you!

Best regards,
Sachin Vaishnav

abonilla92 combatsolutions Tanvir1337 ALYTV Ramikan-BR Amritbro Balasubramaniam


Remember when Claude 3.5 Sonnet by @AnthropicAI took the world by storm with Claude Artifacts? 🌍✨

Now we have LlamaCoder, an open-source Claude Artifacts app that can generate full React apps and components with Meta-Llama 3.1 405B. 💻 100% free and open source. 🆓

I like how Llama has now started becoming a placeholder denoting open-source work! 🔓
Originally, Llama was an acronym for Large Language Model Meta AI. 🤖

GitHub: https://github.com/Nutlope/llamacoder
Demo (by togetherAI): https://llamacoder.together.ai


🚀 Introducing TraVisionLM: Turkish Visual Language Model - The First of Its Kind! 🇹🇷🖼

I'm thrilled to share TraVisionLM on Hugging Face! With 875M parameters, this lightweight, efficient model handles Turkish instructions for image inputs. Fully compatible with the Transformers library, it’s easy to load, fine-tune, and use—no external libraries needed!

Developed solo, TraVisionLM is a strong foundation for low-resource language research. While still improving, it's a key step for Turkish-language AI. Your feedback is welcome as I refine the model.

🎉 Explore it now:

- Model: https://huggingface.co/ucsahin/TraVisionLM-base
- Demo: https://huggingface.co/spaces/ucsahin/TraVisionLM-Turkish_Visual_Language_Model
- Object Detection Finetune: https://huggingface.co/ucsahin/TraVisionLM-Object-Detection-ft

Let’s push Turkish visual language processing forward!

---

🚀 TraVisionLM: Türünün İlk Örneği Türkçe Görsel Dil Modelini Sunuyorum! 🇹🇷🖼

TraVisionLM modelini Hugging Face'te yayınladım! 875M parametre ile bu hafif ve verimli model, görüntüye dayalı Türkçe talimatları işlemek için tasarlandı. Transformers kütüphanesiyle tamamen uyumlu, yüklemesi, eğitmesi ve kullanması çok kolay—dış kütüphane gerekmez!

Tek başıma geliştirdiğim TraVisionLM, düşük kaynaklı dillerde araştırmalar için sağlam bir temel sunuyor. Geliştirmeye devam ederken geri bildirimlerinizi bekliyorum.

🎉 Hemen keşfedin:

- Model: https://huggingface.co/ucsahin/TraVisionLM-base
- Demo: https://huggingface.co/spaces/ucsahin/TraVisionLM-Turkish_Visual_Language_Model
- Obje Tespiti İnce Ayarı: https://huggingface.co/ucsahin/TraVisionLM-Object-Detection-ft

Türkçe görsel dil işleme sınırlarını birlikte zorlayalım!

Sri-Vigneshwar-DJ Metin ogozcelik monsoon-nlp virrvarr hamuz afkfatih ajibawa-2023 pcuenq merve gokaygokay Gozde mertbozkurt balicien


App is running again after a quick restart.

https://huggingface.co/spaces/sourceoftruthdata/sot_autotrain_dreambooth_v1.1


What is the best LLM for RAG systems? 🤔

In a business setting, it will be the one that gives the best performance at a great price! 💼💰

And maybe it should be easy to fine-tune, cheap to fine-tune... FREE to fine-tune? 😲✨

That's @Google Gemini 1.5 Flash! 🚀🌟

It now supports fine-tuning, and the inference cost is the same as the base model! <coughs LORA adopters> 🤭🤖

So the base model must be expensive? 💸
For the base model, the input price is reduced by 78% to $0.075/1 million tokens and the output price by 71% to $0.3/1 million tokens. 📉💵

But is it any good? 🤷♂
On the LLM Hallucination Index, Gemini 1.5 Flash achieved great context adherence scores of 0.94, 1, and 0.92 across short, medium, and long contexts. 📊🎯

Google has finally given a model that is free to tune and offers an excellent balance between performance and cost. ⚖👌

Happy tuning... 🎶🔧

Gemini 1.5 Flash: https://developers.googleblog.com/en/gemini-15-flash-updates-google-ai-studio-gemini-api/ 🔗

LLM Hallucination Index: https://www.rungalileo.io/hallucinationindex 🔗


I've built a space for creating prompts for FLUX

https://huggingface.co/spaces/gokaygokay/FLUX-Prompt-Generator

You can create long prompts from images or simple words. Enhance your short prompts with prompt enhancer. You can configure various settings such as artform, photo type, character details, scene details, style, and artist to create tailored prompts.

And you can combine all of them with custom prompts using  llms (Mixtral, Mistral, Llama 3, and Mistral-Nemo).

The UI is a bit complex, but it includes almost everything you need. Choosing random option is the most fun!

And i've created some other spaces for using FLUX models with captioners  and enhancers.

- https://huggingface.co/spaces/gokaygokay/FLUX.1-dev-with-Captioner

adamelliotfields John6666 tuanio DhruvParth Nymbo radames JasBogan Sri-Vigneshwar-DJ philipp-zettl gokaygokay Ramikan-BR ND911 yuchenj FilipeR louisbrulenaudet brianjking Ijithad nativegirl888 KingNish ibrahim313

Eliasib Jaward tuanio deraum Csplk Nymbo Ramikan-BR gokaygokay anonymous111110987654321 FilipeR nativegirl888

prithivMLmods Arasalan7 Ramikan-BR FilipeR UltraMarkoRJ


New smol-vision tutorial dropped: QLoRA fine-tuning IDEFICS3-Llama 8B on VQAv2 🐶

Learn how to efficiently fine-tune the latest IDEFICS3-Llama on visual question answering in this notebook 📖
Fine-tuning notebook: https://github.com/merveenoyan/smol-vision/blob/main/Idefics_FT.ipynb
Resulting model: https://huggingface.co/merve/idefics3llama-vqav2

ZennyKenny monsoon-nlp tuanio osanseviero not-lain boapps louisbrulenaudet rreed-pha


✨The STABLE IMAGINE !!✨
🍺Space: https://huggingface.co/spaces/prithivMLmods/STABLE-IMAGINE
↗The specific LoRA in the space that requires appropriate trigger words brings good results.
📒 Articles: https://huggingface.co/blog/prithivMLmods/lora-adp-01

**Description and Utility Functions **
✅ Most likely image generation
☑ Most accurate trigger words expected
✅ Each designed to capture different artistic elements
☑ Specialized styles and characteristics
✅ Flexible to design what is needed (keyword-centric)
☑ Increasing productivity

🫙Repository: https://github.com/prithivsakthiur/gen-vision
📔Colab Link: https://colab.research.google.com/drive/1axA0pU--32t4a8AHiVlt6zyl8gRfiXKs
*⃣Notebook: https://huggingface.co/spaces/prithivMLmods/STABLE-IMAGINE/blob/main/Gen_Vision.ipynb

```
 lora_options = {
        "Realism (face/character)": ("prithivMLmods/Canopus-Realism-LoRA", "Canopus-Realism-LoRA.safetensors", "rlms"),
        "Pixar (art/toons)": ("prithivMLmods/Canopus-Pixar-Art", "Canopus-Pixar-Art.safetensors", "pixar"),
        "Photoshoot (camera/film)": ("prithivMLmods/Canopus-Photo-Shoot-Mini-LoRA", "Canopus-Photo-Shoot-Mini-LoRA.safetensors", "photo"),
        "Clothing (hoodies/pant/shirts)": ("prithivMLmods/Canopus-Clothing-Adp-LoRA", "Canopus-Dress-Clothing-LoRA.safetensors", "clth"),
          }
          .
          .
          .
for model_name, weight_name, adapter_name in lora_options.values():
    pipe.load_lora_weights(model_name, weight_name=weight_name, adapter_name=adapter_name)
pipe.to("cuda")

```


Live Portrait Updated to V5

Animals Live animation added

All of the main repo changes and improvements added to our modified and improve app

Link : https://patreon.com/posts/107609670

Works perfect on Massed Compute, RunPod, free Kaggle account and Windows

1-Click to install with instructions

All tested and verified

Windows tutorial : https://youtu.be/FPtpNrmuwXk

Cloud (RunPod, Massed Compute & free Kaggle account) tutorial : https://youtu.be/wG7oPp01COg

Making XPose / UniPose / ops library compiling working was a challenge on Massed Compute and Kaggle.


PyTorch implementation of the Self-Compression & Differentiable Quantization Algorithm introduced in “Self-Compressing Neural Networks” paper.

The algorithm shows dynamic neural network compression during training - with reduced size of weight, activation tensors and bits required to represent weights.

It’s basically shrinking the neural network size (weights and activations) as it’s being trained without compromising performance -  this helps reduce compute and inference cost.

Code: https://github.com/Jaykef/ai-algorithms
Paper: https://arxiv.org/pdf/2301.13142

DmitryRyumin GPT007 osanseviero maywell HideFace


Bilateral Reference for High-Resolution Dichotomous Image Segmentation (2401.03407)

🔥 New state of the art model for background removal is out
🤗 You can try the model at https://huggingface.co/ZhengPeng7/BiRefNet
📈 model shows impressive results outperforming https://huggingface.co/briaai/RMBG-1.4
🚀 you can try out the model in: https://huggingface.co/spaces/ZhengPeng7/BiRefNet_demo

📃paper: https://huggingface.co/papers/2401.03407

yadheedhya merve xi0v Taylor658 Eliasib John6666 GPT007 louisbrulenaudet osanseviero eieihihi Winnougan autmoate JohnWinn

ZhengPeng7 osanseviero darkzbaron Winnougan ijohn07 JohnWinn


https://huggingface.co/JoseRFJunior/TransNAR
https://github.com/JoseRFJuniorLLMs/TransNAR
https://arxiv.org/html/2406.09308v1
TransNAR hybrid architecture. Similar to Alayrac et al, we interleave existing Transformer layers with gated cross-attention layers which enable information to flow from the NAR to the Transformer. We generate queries from tokens while we obtain keys and values from nodes and edges of the graph. The node and edge embeddings are obtained by running the NAR on the graph version of the reasoning task to be solved. When experimenting with pre-trained Transformers, we initially close the cross-attention gate, in order to fully preserve the language model’s internal knowledge at the beginning of training.


Remember when @mistralAI said large enough and casually dropped Mistral-Large-Instruct-2407? 🤯🚀

It's now on http://lmsys.org! 🌐 It works amazing for instruction following, hard prompts, coding, and longer queries with only 123 billion parameters. 💡💻

It outperforms GPT4-Turbo and Claude 3 Opus on Coding, Hard Prompts, Math, and Longer Query categories. 📈🔢

It also outperforms Llama 3.1 405B on Instruction Following while being 3x smaller. 🐎🔍

It also does exceedingly well on the Ai2 ZebraLogic logistic reasoning benchmark despite being much smaller than the other models. 🦓🤔

Mistral is not here to take part but to take over! 🏆🌟

Model: https://mistral.ai/news/mistral-large-2407/


hello everyone, today I have been working on a project https://huggingface.co/spaces/Blane187/rvc-demo, a demo of rvc using pip, this project is still a demo though (I don't have a beta tester lol)


⚡ My PhD thesis, “Scalable Nested Optimization for Deep Learning,” is now on arXiv! ⚡

tl;dr: We develop various optimization tools with highlights, including:
· Making the momentum coefficient complex for adversarial games like GANs.
· Optimizing millions of hyperparameters using implicit differentiation.
· Tuning hyperparameters using hypernetworks.
· Differentiably finding bifurcations in optimization for diverse solutions.

https://arxiv.org/abs/2407.01526

nbroad ajibawa-2023 merve MexIvanov rkusch Joseph717171 osanseviero lorraine2 not-lain fsaudm

Jaward merve MexIvanov Joseph717171 Aurelien-Morgan lorraine2 not-lain


Is your summer reading list still empty? Curious if an LLM can generate a book blurb you'd enjoy and help build a KTO preference dataset at the same time?

A demo using Hugging Face Spaces and Gradio to collect LLM output preferences: https://huggingface.co/spaces/davanstrien/would-you-read-it

prithivMLmods joey00072 nbroad Saugatkafley maywell Ramikan-BR Tanvir1337 philipp-zettl GPT007 osanseviero


Just released: Shining Valiant 2 for Llama 3.1 8b!

- the first SV at 8b size, using the best 8b model
- newest version of the SV dataset improves specialist knowledge and response consistency

3.1 70b will be coming but our next releases will focus on expanding the Build Tools lineup. Get ready for some open-source synthetic datasets made with 3.1 405, coming VERY soon :)

get SV2: https://huggingface.co/ValiantLabs/Llama3.1-8B-ShiningValiant2


@warmshao Hello. do you do any paid consulting?


distilabel 1.3.0 is out! This release contains many core improvements and new tasks that help us building https://huggingface.co/datasets/argilla/magpie-ultra-v0.1!

Distributed pipeline execution with Ray, new Magpie tasks, reward models, components for dataset diversity based on sentence embeddings, Argilla 2.0 compatibility and many more features!

Check the new release in GitHub: https://github.com/argilla-io/distilabel

plaguss nataliaElv sdiazlor KvrParaskevi anakin87 not-lain osanseviero

gabrielmbmb sdiazlor not-lain louisbrulenaudet


 🚀 We are thrilled to introduce TextImage Data Augmentation, developed in collaboration with Albumentations AI! ✨ This multimodal technique modifies document images and text simultaneously, enhancing Vision Language Models (VLMs) for high-text datasets.

👩💻 Learn how this innovative approach can improve your document AI projects by checking out our full blog post here: https://huggingface.co/blog/doc_aug_hf_alb


🗓 Remember when last April, @Meta released Segment Anything Model (SAM) paper and it was too good to be true. 🤯

They have now released Segment Anything Model 2 (SAM 2) and it's mind-blowingly great! 🚀

SAM 2 is the first unified model for segmenting objects across images and videos. You can use a click, box, or mask as the input to select an object on any image or frame of video. 🖼📹

SAM consists of an image encoder to encode images, a prompt encoder to encode prompts, then outputs of these two are given to a mask decoder to generate masks. 🎭

The biggest jump of SAM2 from SAM is using memory to have consistent masking across frames! They call it masklet prediction! 🧠

They have also released the dataset, SA-V
This dataset is truly huge, with 190.9K manual annotations and 451.7K automatic! 📊

📄 Paper: https://ai.meta.com/research/publications/sam-2-segment-anything-in-images-and-videos/

📝 Blog: https://ai.meta.com/sam2/

🔗 Demo: https://sam2.metademolab.com/demo

💾 Model Weights: https://github.com/facebookresearch/segment-anything-2/blob/main/checkpoints/download_ckpts.sh

📁 Dataset: https://ai.meta.com/datasets/segment-anything-video-downloads/

louisbrulenaudet Ramikan-BR NHLOCAL Glavin001 nbroad ajibawa-2023 myjyjy osanseviero


You can now embed your heatmap anywhere with a simple change :)

Currently just supports model creation! You can duplicate the space and create your own here:

https://huggingface.co/spaces/cfahlgren1/my-heatmap


Idefics3-Llama is out! 💥💥
Model: https://huggingface.co/HuggingFaceM4/Idefics3-8B-Llama3
Demo: https://huggingface.co/spaces/HuggingFaceM4/idefics3

It's a multimodal model based on Llama 3.1 that accepts an arbitrary number of interleaved images with text with a huge context window (10k tokens!) ✨

Supported by Hugging Face transformers 🤗

Nymbo YaTharThShaRma999 felli0t Andyrasika aidystark alvarobartt gabrielmbmb not-lain ajibawa-2023 prithivMLmods nbroad osanseviero BinJaw rreed-pha

Nymbo ucsahin Melike gabrielmbmb monsoon-nlp not-lain louisbrulenaudet osanseviero rreed-pha


📫 AI in the news today: Struggling AI startups, Figure 0 robot, chipmakers

- OpenAI Co-Founders Schulman and Brockman Step Back
https://finance.yahoo.com/news/openai-co-founders-schulman-brockman-010542796.html

- Struggling AI Startups Look for a Bailout from Big Tech
"More exits—either pseudo-acquisitions or real ones—are coming, investors say, as a bubble built by the excitement around generative AI is showing signs of peaking."
https://www.wsj.com/tech/ai/struggling-ai-startups-look-for-a-bailout-from-big-tech-3e635927?mod=rss_Technology

- Did Google Just Pay $2.5 Billion to Hire Character's CEO?
https://www.theinformation.com/articles/did-google-just-pay-2-5-billion-to-hire-characters-ceo

- Figure’s new humanoid robot leverages OpenAI for natural speech conversations
Figure has unveiled its latest humanoid robot, the Figure 02.
The most notable addition this time out arrives by way a longstanding partnership with OpenAI, which helped Figure raise a $675 million Series B back in February, valuing the South Bay firm at $2.6 billion.
https://techcrunch.com/2024/08/06/figures-new-humanoid-robot-leverages-openai-for-natural-speech-conversations/

- World’s Five Leading Chipmakers Have Now Promised U.S. Investment
The Biden administration award up to $450 million in grants to a South Korean chipmaker, SK Hynix, to help build its new chip facility in Indiana
The US now has commitments from all five of the world’s leading-edge semiconductor manufacturers to construct chip plants in theUS with financial assistance from the administration
https://www.nytimes.com/2024/08/06/business/economy/chipmakers-promise-investment.html

efecelik Nymbo YannisTevissen nbroad AdilZtn


🚀 Introducing The Open Language Models List

This is a work-in-progress list of open language models with permissive licenses such as MIT, Apache 2.0, or other similar licenses.

The list is not limited to only autoregressive models or even only transformers models, and it includes many SSMs, and SSM-Transformers hybrids.

🤗 Contributions, corrections, and feedback are very welcome!

The Open Language Models List: https://github.com/mmhamdy/open-language-models

joseEjmendez DmitryRyumin louisbrulenaudet Smd-Arshad jam-ai9120 Kukedlc not-lain efecelik Sri-Vigneshwar-DJ thedhanawada


Who wants to take a stab at explaining this one? SPOILER ALERT: You CANNOT transfer an image Jailbreak from one model to another. Why in the world you cannot do this when you can transfer learn literally everything else? You tell me, experts.

https://arxiv.org/abs/2407.15211


Looks like @Google is still not satisfied with Gemini 1.5 Pro! 😲

Good folks at @GoogleDeepMind quietly updated the already good Gemini 1.5 Pro to Gemini-1.5-Pro-Experiment-0801 🚀

Unremarkable naming aside, the model itself outperforms GPT-4o, Claude-3.5, and LLama 3.1 on LMSYS and the Vision Leaderboard. 🌟

Gemini-1.5-Pro-Experiment-0801 is great at almost everything, multi-lingual tasks, Maths, understanding, and coding. 🌐📚💻

Although in my testing, I felt Claude-3.5 was slightly better at coding! 👨💻🤔

Also, still cannot find an LLM that can solve the "Strawberry prompt"! 🍓❓

"""
How many R's are there in Strawberry?
Also, write Strawberry with all r's in brackets
"""

Try here: https://aistudio.google.com/app/prompts/new_chat


We are offering a free trial of  the Tumeryk Gen AI Security Studio with 2 million tokens for the hugging face community. You can run a vulnerability scan of the LLM and get a security score to fix the vulnerabilities via the built in policy editing tools for configuring the necessary guardrail policies.

Protect against jailbreaks, moderate content and manage hallucinations.

https://www.tumeryk.com/sign-up to get an account or you can get an account from the Amazon Marketplace https://aws.amazon.com/marketplace/pp/prodview-c6ywcyjefl3hu?sr=0-2&ref_=beagle&applicationId=AWSMPContessa


I'm excited to share our updated hallucination evaluation model (called HHEM-2.1-Open) as well as the updated leaderboard that ranks LLM by the propensity to hallucinate.

https://huggingface.co/spaces/vectara/Hallucination-evaluation-leaderboard

ZeroWw osanseviero cfahlgren1 quanvndzai vincentweisser gghfez


I've observed that the layers targeted in various abliteration notebooks (e.g., https://colab.research.google.com/drive/1VYm3hOcvCpbGiqKZb141gJwjdmmCcVpR?usp=sharing ) appear to be arbitrary, reflecting probable brute-force exploration. This doesn't need to be the case.

Taking a cue from the paper "The Unreasonable Ineffectiveness of the Deeper Layers" ( https://arxiv.org/abs/2403.17887 ) and PruneMe (https://github.com/arcee-ai/PruneMe), it seems reasonable to target deeper layers identified as more redundant given measured similarity across layers, as the result should be less damaging to models, reducing the need for subsequent fine-tuning. Intuitively, one should expect the resulting intervention layers to be deep but not final. The only uncertainty is if the redundancy successfully encodes refusals, something which is almost certainly model-dependent. This approach only requires the redundancy to be computed once per model, and the result used as a starting point for which layer range to restrict intervention to.

mlabonne Joseph717171 ZeroWw fearlessdots failspy wassname


TTSDS -- Text-to-Speech Distribution Score (2407.12707)

I just added 5 more models to my open source TTS model benchmark, https://huggingface.co/spaces/ttsds/benchmark.
Let's talk about the results!

Over the last couple days, I added https://huggingface.co/jbetker/tortoise-tts-v2, https://huggingface.co/metavoiceio/metavoice-1B-v0.1, https://huggingface.co/audo/HierSpeechpp, and the unofficial implementations of https://huggingface.co/spaces/amphion/NaturalSpeech2 and https://huggingface.co/amphion/valle by https://huggingface.co/amphion

Takeaways:
- TorToiSe does very well, falling into second place after StyleTTS 2, which is also ranked first in the human evaluation at https://huggingface.co/spaces/TTS-AGI/TTS-Arena.
- MetaVoice-1B's overall score is dragged down by its Intelligibility Score (probably due to utterances being cut short), it achieves #3 in Speaker Score, which indicates good voice cloning ability.
- HierSpeech++ lands in the middle of the road in terms of performance, but excels at the Environment Score, achieving #2 - this means the model is especially good at modeling recording conditions such as microphone and background noise.
- The Amphion models, possibly due to not being trained for the same amount as in the papers, achieve relatively low scores. However, they seem to struggle for different reasons. The autoregressive VALLE models have low Intelligibility Scores (possibly due to "babbling" or early stop tokens) while NaturalSpeech2 has low Speaker and Prosody scores.

What's next?
I'm planning to add more open source TTS models like https://huggingface.co/suno/bark, https://huggingface.co/CAMB-AI/MARS5-TTS and https://huggingface.co/fishaudio/fish-speech-1.2. I'll also write an article on these and all the other results soon, since our paper, https://huggingface.co/papers/2407.12707, mostly focused on establishing the benchmark itself rather than the indiviual TTS systems.


🥹 @lbourdois has made an app to browse all of my vision paper summaries for everyone's convenience https://huggingface.co/spaces/merve/vision_papers

lbourdois leeloolee not-lain louisbrulenaudet GPT007

lbourdois Aurelien-Morgan ferdmartin not-lain