Google's Chain-of-Thought (CoT) is one of the most effective ways to improve LLMs' reasoning.

Researchers have now developed a novel approach called Strategic Chain-of-Thought (SCoT) to enhance the reasoning capabilities of large language models even further.

🧠 SCoT uses a two-stage process within a single prompt:
- Strategy Elicitation: The model first identifies and determines an effective problem-solving strategy for the given task. This becomes the strategic knowledge that guides the reasoning process.
- Strategy Application: The model then applies the identified strategic knowledge to solve the problem and generate the final answer.

Essentially, SCoT integrates strategic knowledge to guide reasoning without relying on external knowledge sources or multiple queries.

According to the research, SCoT showed significant improvements over standard CoT across various datasets, including a 21.05% increase on the GSM8K math dataset and a 24.13% increase on the Tracking_Objects spatial reasoning task.

Changes in the Prompt Structure:
The SCoT prompt typically consists of five components:
- Role: Defines the expert role the model should assume.
- Workflow: Outlines the steps for strategy identification and application.
- Rules: Specifies guidelines for generating answers.
- Initialization: Sets up the task.
- Task Input: Provides the specific problem to solve.

Strategy Generation:
The model is prompted to generate strategic knowledge relevant to the problem domain. For example, in mathematics, it might favor elegant solutions like using arithmetic series formulas over brute-force calculations.

Guided Reasoning:
Using the elicited strategy, the model then generates a chain-of-thought reasoning path. This approach aims to produce more stable and higher-quality outputs compared to standard chain-of-thought methods.

Read the full paper: https://arxiv.org/abs/2409.03271


i just made the best 0.5b model to date (again)

its name is arco and is ready to fight any 0.5b model at arc challenge

https://huggingface.co/appvoid/arco

John6666 Tonic AtAndDev louisbrulenaudet


Reposting from twitter:

Just so you all know, I'll be on vacation for the following two weeks and away from home! I'm hoping to get on at least once a day to load up some quants, but I won't be as bleeding edge and on the ball :) feel free to shoot me a message if you see one I should make!

In the meantime if you need something bleeding edge make sure to check out @MaziyarPanahi or @bullerwins who both put out great work!

Joseph717171 syrupsweety not-lain bullerwins MaziyarPanahi MarinaraSpaghetti osanseviero Presidentlin victor JoeySalmons celsowm hudzax win10

Joseph717171 John6666 AIGUYCONTENT MaziyarPanahi osanseviero neoopus


FLUX Prompt Generator Updates

- https://huggingface.co/spaces/gokaygokay/FLUX-Prompt-Generator

- There are now hundreds of new selections across diverse categories, each offering a lot of choices:

Architecture, Art, Artist, Brands, Character, Cinematic, Fashion, Feelings, Geography, Human, Interaction, Keywords, Objects, People, Photography, Plots, Poses, Scene, Science, Stuff, Time, Typography, Vehicle, Video Game

- In addition to Hugging Face, I've integrated new LLM providers: Groq, OpenAI, and Claude.

- Upgraded Vision Language Models (VLMs): We now feature Qwen2-VL, JoyCaption and Florence-2-large.

- New specialized system prompts for various styles and themes, including Happy, Simple, Poster, Only Objects, No Figure, Landscape, Fantasy.

John6666 YaTharThShaRma999 KingNish ucsahin Chief-Inspector EmilyChan victor Felladrin Nyxie7 gokaygokay


Yesterday  @mattshumer released https://huggingface.co/mattshumer/Reflection-Llama-3.1-70B, an impressive model that achieved incredible results in benchmarks like MMLU. The model was fine-tuned using Reflection-Tuning and the dataset used wasn't released, but I created a small recipe with distilabel that allows generating a dataset with a similar output format:

1. We use MagPie 🐦  in combination with https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct to generate reasoning instructions.
2. We generate a response again using https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct, but we steer the LLM to generate an specific output format using a custom system prompt. In the system prompt, we instruct the LLM that it will have first to think 💭 and have reflections that will help resolving ambiguities. After that, we instruct the LLM to generate an output based on the previous thinking

In this dataset https://huggingface.co/datasets/gabrielmbmb/distilabel-reflection-tuning you can found 5 rows that I generated with this recipe. You can also found the code of the pipeline in the file called `reflection.py`.

clem prithivMLmods KonradSzafer Svngoku John6666 den0620 osanseviero gabrielmbmb louisbrulenaudet


hemanuelly hunken Sri-Vigneshwar-DJ Tonioesparza den0620 Akash3104


"LLM inference at scale with TGI". Cool blogpost: https://www.adyen.com/knowledge-hub/llm-inference-at-scale-with-tgi

Well done
@martinigoyanes @rafa-hernandez @Vidusharma @frisokingma @hannahwright @jeanmarcs @antonioramos & the whole https://huggingface.co/adyen team. Could be useful to cross-post here: https://huggingface.co/blog/community

vilarin KingNish victor prithivMLmods nbroad


💾🧠How much VRAM will you need for training your AI model? 💾🧠
Check out this app where you convert:
Pytorch/tensorflow summary -> required VRAM
or
Parameter count -> required VRAM

Use it in: http://howmuchvram.com

And everything is open source! Ask for new functionalities or contribute in:
https://github.com/AlexBodner/How_Much_VRAM
If it's useful to you leave a star 🌟and share it to someone that will find the tool useful!
More discussion in: https://x.com/AlexBodner_/status/1832054850294812679


 I've been working on a  Space to make it super easy to create notebooks and help users quickly understand and manipulate their data!
With just a few clicks automatically generate notebooks for:

📊 Exploratory Data Analysis
🧠 Text Embeddings
🤖 Retrieval-Augmented Generation (RAG)

✨ Automatic training is coming soon!
Check it out here https://huggingface.co/spaces/asoria/auto-notebook-creator
Appreciate any feedback to improve this tool 🤗


🌟 Argilla v2.1.0 goes multi-modal: Image Field, Dark Mode, Enhanched Hugging Face Hub imports and more!

🖼 Image Field: Seamlessly work with multimodal datasets
🌓 Dark Mode: Reduce eye strain with our sleek new look
🤗 Enhanced Hugging Face Hub import with the SDK
🇪🇸 Spanish UI: Breaking language barriers

Plus more improvements to supercharge your model curation workflow!

Check out the full announcement for details and code examples: https://github.com/argilla-io/argilla/compare/v2.0.1...v2.1.0

davanstrien gabrielmbmb dvilasuero Ameeeee louisbrulenaudet clem John6666 KingNish AtAndDev

gabrielmbmb dvilasuero Ameeeee clem AtAndDev

Ameeeee clem KingNish AtAndDev gabrielmbmb


Wanted to train a FLUX model using out-of-copyright images, so I curated concept art images from NASA.

Model: https://huggingface.co/davanstrien/nasa_concept_art
Dataset: https://huggingface.co/datasets/davanstrien/nasa_concept_art

So far, training was done without captions, but I'm experimenting with using VLLMs to generate captions to see if that improves the model.


🤯 𝗔 𝗻𝗲𝘄 𝟳𝟬𝗕 𝗼𝗽𝗲𝗻-𝘄𝗲𝗶𝗴𝗵𝘁𝘀 𝗟𝗟𝗠 𝗯𝗲𝗮𝘁𝘀 𝗖𝗹𝗮𝘂𝗱𝗲-𝟯.𝟱-𝗦𝗼𝗻𝗻𝗲𝘁 𝗮𝗻𝗱 𝗚𝗣𝗧-𝟰𝗼!

@mattshumer, CEO from Hyperwrite AI, had an idea he wanted to try out: why not fine-tune LLMs to always output their thoughts in specific parts, delineated by <thinking> tags?

Even better: inside of that, you could nest other sections, to reflect critically on previous output. Let’s name this part <reflection>. Planning is also put in a separate step.

He named the method “Reflection tuning” and set out to fine-tune a Llama-3.1-70B with it.

Well it turns out, it works mind-boggingly well!

🤯 Reflection-70B beats GPT-4o, Sonnet-3.5, and even the much bigger Llama-3.1-405B!

𝗧𝗟;𝗗𝗥
🥊 This new 70B open-weights model beats GPT-4o, Claude Sonnet, et al.
⏰ 405B in training, coming soon
📚 Report coming next week
⚙ Uses GlaiveAI synthetic data
🤗 Available on HF!

I’m starting an Inference Endpoint right now for this model to give it a spin!

Check it out 👉 https://huggingface.co/mattshumer/Reflection-Llama-3.1-70B

TahirC Yuuru iandeanschaefer trollek Joseph717171


You can create charts, leaderboards, and filters on top of any Hugging Face dataset in less than a minute

• ASCII Bar Charts 📊
• Powered by DuckDB WASM ⚡
• Download results to Parquet 💽
• Embed and Share results with friends 📬

Do you have any interesting queries?


What rank are you on Hugging Face Top Yappers? 🗣

Find your rank here with this link: https://huggingface.co/datasets/cfahlgren1/hub-stats/embed/sql-console/d453ehm

The Top 3:
- @fdaudens
- @singhsidhukuldeep
- @akhaliq

I am at #71 and need to get my numbers up! 📈


Nine years ago, I uploaded the first 8K resolution video to YouTube and I've been stockpiling 8K footage ever since: https://www.youtube.com/watch?v=sLprVF6d7Ug&t

Should @Overlaiapp release the first open-source 8K video dataset?

Could anyone even fine tune a model with this?😅


`huggingface.co/DIBT ` is dead!

Long live https://huggingface.co/data-is-better-together!

We're working on some very cool projects so we're doing a bit of tidying of the Data is Better Together Hub org 🤓


Build a collection for the trending demos recently released by the Chinese community 🚀 From Qwen2.5 Turbo to FishAgent, see what these models can really do 🔥
https://huggingface.co/collections/zh-ai-community/trending-demo-673b6ca2416a3b3c9d3bf8f1


🤖 Controlling Computers with Small Models 🤖

We just released PTA-1, a fine-tuned Florence-2 for localization of GUI text and elements. It runs with ~150ms inference time on a RTX 4080. This means you can now start building fast on-device computer use agents!

Model: https://huggingface.co/AskUI/PTA-1
Demo: https://huggingface.co/spaces/AskUI/PTA-1

menesjo programmnix-askui YaTharThShaRma999 John6666 danyalxahidaskui AndiAskUI


🚨 How green is your model? 🌱 Introducing a new feature in the Comparator tool: Environmental Impact for responsible #LLM research!
👉 https://huggingface.co/spaces/open-llm-leaderboard/comparator
Now, you can not only compare models by performance, but also by their environmental footprint!

🌍 The Comparator calculates CO₂ emissions during evaluation and shows key model characteristics: evaluation score, number of parameters, architecture, precision, type... 🛠
Make informed decisions about your model's impact on the planet and join the movement towards greener AI!


Very exciting new https://huggingface.co/mistralai/Pixtral-Large-Instruct-2411 model from Mistral-AI

Impressive performances, huge congrats @patrickvonplaten @sgvaze @pandora-s @devendrachaplot @sophiamyang and team!

Very nice to have SOTA Multilingual OCR and Chart understanding in an open-weights model


@Jesse-marqo and the Marqo team are killing it on the Hub: top embedding models and datasets!

Here's how to start using their new evaluation dataset for curation and labelling:

1. Deploy Argilla on Spaces: https://huggingface.co/new-space?template=argilla%2Fargilla-template-space
2. Load https://huggingface.co/datasets/Marqo/amazon-products-eval with the UI wizard.
3. Start curating!


Sharing what we have built over the course of the weekend at the @llamameta hackathon, by Cerebral Valley in London 🇬🇧 👇

@gabrycina @calebgcc and I competed with 200+ participants and 50+ teams for a 24-hrs sprint centered around hacking for impact! We focused on applications of robotics to those in need of assisted living, moving our focus to enable greater autonomy and accessibility of robotics in everyday life.

complete list of assets 👇
🤗 trained robotics policies
v1:
- https://huggingface.co/fracapuano/moss-pills
- https://huggingface.co/fracapuano/moss-cup
v2:
- https://huggingface.co/fracapuano/meta-grasp

🤗 datasets
v1:
- https://huggingface.co/datasets/fracapuano/pills
- https://huggingface.co/datasets/fracapuano/cup
v2:
- https://huggingface.co/datasets/fracapuano/cupim

You can find a live demo of our submission at: https://x.com/_fracapuano/status/1858102728691458554

If you want to know more about how we collected 100GB+ of data, trained multiple RL-policies using @lerobot and used Llama-3.2 models to handle user interactions and switch between tasks, go ahead and have a look! Also, don't be a stranger, and reach out 🦾

Our project is fully open-source, for the community (and ourselves, 👨🍳) to build! A huge thank you to @cadene for the help (and the robot 🤭) - truly feeling these hugs-vibes 🤗 , and to @thomwolf and @clem for sharing our work across

Little extra:
➡ Our 🧠EEG waves🧠-based control of the 🦾robotic arm🦾


I'm building an AI for healthcare support for professionals, any advice? I could create a new app here but it needs a lot of trainer (Im newbie in this kind of stuff)
Thank you, guys!!!!


Good lord... Spent almost a day debugging this and it turns out it was an issue of gradio update incompatible with the new fastapi.
https://discuss.huggingface.co/t/huggingface-space-failed-after-working-initially/105514/8

Finally got it back online! Come chat with your favorite anime characters here:
https://huggingface.co/spaces/kz919/Persona-AI


Good folks at Epoch AI have just released their most comprehensive database yet, tracking over 800 state-of-the-art and historically notable AI models. This incredible resource provides key insights into the factors driving machine learning progress.

Since 2010, the training compute used to create AI models has been growing at a staggering rate of 4.1x per year. That means the computational power behind these models is doubling roughly every six months! And it's not just the compute that's increasing - the costs are too. Training compute costs for the largest models are doubling every nine months, with the most advanced models now costing hundreds of millions of dollars.

Interestingly, training compute has scaled up faster for language models compared to vision. While the largest vision and language models had similar compute requirements before 2020, language models have since rapidly outpaced vision models, driven by the success of transformer architectures. The size of datasets used to train language models is also doubling approximately every eight months.

Another fascinating trend is that the length of time spent training notable models is growing by about 1.2x per year. While longer training times could ease hardware constraints, there is a tradeoff to consider. For very long runs, waiting for algorithmic and hardware improvements might be more beneficial than simply extending training.

If this continues, by 2028, we will reach cluster prices in the 100 billion dollars, using 10GW of power!

Link: https://epochai.org/data/notable-ai-models


Decided to try to check how many weights in a 70b F32 model would be squashed when converted to F16 (spoiler, it's shockingly few)

The reason for this comparison is that it should represent the same percentage of squishing as bf16 to fp16

Had claude make me a script, using the new Reflection-70B, and these are the results:

Total weights: 70553706496
Fully representable: 70530215524
Squashed: 23490972
Percentage squashed: 0.03%

0.03%!!!!

A couple things to note, this uses a roundtrip of F32 -> F16 -> F32 and then torch.isclose to account for rounding errors that come up by the very nature of extremely accurate numbers, but it uses VERY small tolerances (rtol=1e-5, atol=1e-8)

This is also examining EVERY weight that was stored at F32, and for most layers I was somewhere between 0% and 0.03% of weights being squashed, no major outliers.

Overall, I feel even safer converting to F16 for llama.cpp, the extremely small number of weights that fall outside the range are likely so small that they don't actually play a role in the final output of the model at inference anyways.


@ehartford https://huggingface.co/CohereForAI/c4ai-command-r-plus dolphin when?

John6666 mahiatlinux danielus Locutusque den0620 AtAndDev


The `timm` leaderboard https://huggingface.co/spaces/timm/leaderboard has been updated with the ability to select different hardware benchmark sets: RTX4090, RTX3090, two different CPUs along with some NCHW / NHWC layout and torch.compile (dynamo) variations.

Also worth pointing out, there are three rather newish 'test' models that you'll see at the top of any samples/sec comparison:
* test_vit (https://huggingface.co/timm/test_vit.r160_in1k)
* test_efficientnet (https://huggingface.co/timm/test_efficientnet.r160_in1k)
* test_byobnet (https://huggingface.co/timm/test_byobnet.r160_in1k, a mix of resnet, darknet, effnet/regnet like blocks)

They are < 0.5M params, insanely fast and originally intended for unit testing w/ real weights. They have awful ImageNet top-1, it's rare to have anyone bother to train a model this small on ImageNet (the classifier is roughly 30-70% of the param count!). However, they are FAST on very limited hadware and you can fine-tune them well on small data. Could be the model you're looking for?


I have put together a notebook on Multimodal RAG, where we do not process the documents with hefty pipelines but natively use:
- https://huggingface.co/vidore/colpali for retrieval 📖 it doesn't need indexing with image-text pairs but just images!
- https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct for generation 💬 directly feed images as is to a vision language model with no processing to text!
I used ColPali implementation of the new 🐭 Byaldi library by @bclavie  🤗
https://github.com/answerdotai/byaldi
Link to notebook: https://github.com/merveenoyan/smol-vision/blob/main/ColPali_%2B_Qwen2_VL.ipynb

rwightman umair894 clem louisbrulenaudet s3nh John6666 BasitMustafa Johnyquest7 phanhoang resbyte denizaybey allandclive fdaudens vilarin ak0601 rreed-pha xi0v Rajaram1996 Rayvee oceansweep parjun byteprobe Filippo

hitchhiker3010 Csplk sasikiran fsommers rogermt navin7 sambarnett96 oceansweep ysdede shreyamondal

rreed-pha oceansweep Yassmen madstuntman11


How do i access llama 3.1 70b in my space ?

this doesn't seem to work, can someone help me with a working code

from transformers import AutoConfig

config = AutoConfig.from_pretrained("meta-llama/Meta-Llama-3.1-70B", revision="main")
config.rope_scaling = {"type": "llama3", "factor": 8.0}

model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3.1-70B", config=config, use_auth_token=True)


🚀 𝗪𝗵𝗲𝗿𝗲 𝘀𝗰𝗮𝗹𝗶𝗻𝗴 𝗹𝗮𝘄𝘀 𝗮𝗿𝗲 𝘁𝗮𝗸𝗶𝗻𝗴 𝘂𝘀 : 𝗯𝘆 𝟮𝟬𝟮𝟴, 𝗔𝗜 𝗖𝗹𝘂𝘀𝘁𝗲𝗿𝘀 𝘄𝗶𝗹𝗹 𝗿𝗲𝗮𝗰𝗵 𝘁𝗵𝗲 𝗽𝗼𝘄𝗲𝗿 𝗰𝗼𝗻𝘀𝘂𝗺𝗽𝘁𝗶𝗼𝗻 𝗼𝗳 𝗲𝗻𝘁𝗶𝗿𝗲 𝗰𝗼𝘂𝗻𝘁𝗿𝗶𝗲𝘀

Reminder : “Scaling laws” are empirical laws saying that  if you keep multiplying your compute by x10, your models will mechanically keep getting better and better.

To give you an idea, GPT-3 can barely write sentences, and GPT-4, which only used x15 its amount of compute, already sounds much smarter than some of my friends (although it's not really - or at least I haven't tested them side-by side). So you can imagine how far a x100 over GPT-4 can take us.

🏎 As a result, tech titans are racing to build the biggest models, and for this they need gigantic training clusters.

The picture below shows the growth of training compute: it is increasing at a steady exponential rate of a x10 every 2 years. So let’s take this progress a bit further:
- 2022: starting training for GPT-4 : 10^26 FLOPs, cost of $100M
- 2024: today, companies start training on much larger clusters like the “super AI cluster” of Elon Musk’s xAI, 10^27 FLOPS, $1B
- 2026 : by then clusters will require 1GW, i.e. around the full power generated by a nuclear reactor
- 2028: we reach cluster prices in the 100 billion dollars, using 10GW, more than the most powerful power stations currently in use in the US. This last size seems crazy, but Microsoft and OpenAI already are planning one.

Will AI clusters effectively reach these crazy sizes where the consume as much as entire countries?
➡ Three key ingredients of training might be a roadblock to scaling up :
💸 Money: but it’s very unlikely, given the potential market size for AGI, that investors lose interest.
⚡ Energy supply at a specific location
📚 Training data: we’re already using 15 trillion tokens for Llama-3.1 when Internet has something like 60 trillion.

🤔 I’d be curious to hear your thoughts: do you think we’ll race all the way there?


🌐 Introducing PPT Online Dataset - https://huggingface.co/datasets/nyuuzyou/pptonline

Dataset highlights:
- Metadata for 1,418,349 PowerPoint (.ppt) files from ppt-online.org
- Multilingual content: Russian, Ukrainian, Belarusian, Kazakh, English, and others
- Each entry includes: Unique ID, title, category, download link, file size, and content snippet
- Data reflects presentations accessible through the PPT Online platform
- Licensed under Creative Commons Zero (CC0) for unrestricted use

This dataset offers a unique window into online educational resources, particularly in Eastern European and Central Asian contexts. It provides opportunities for analyzing presentation trends, topic distributions, and language patterns in educational materials.


Hey everyone 🤗!
We (finegrain) have created some custom ComfyUI nodes to use our refiners micro-framework inside comfy! 🎉

We only support our new Box Segmenter at the moment, but we're thinking of adding more nodes since there seems to be a demand for it. We leverage the new (beta) Comfy Registry to host our nodes. They are available at: https://registry.comfy.org/publishers/finegrain/nodes/comfyui-refiners. You can install them by running:
```
comfy node registry-install comfyui-refiners
```
Or by unzipping the archive you can download by clicking "Download Latest" into your `custom_nodes` comfy folder.
We are eager to hear your feedbacks and suggestions for new nodes and how you'll use them! 🙏

limiteinductive John6666 lunarflu djuna catwell


🙋🏻♂Hey there folks,

Did you see the new coding model from @01-ai ?

collection : https://huggingface.co/collections/01-ai/yi-coder-66bdb00f5bdd611f9a008f30
demo : https://huggingface.co/spaces/Tonic/Yi-Coder-9B

achieves SOTA on benchmarks , 125K context window , 55 languages including Docker, Js and many more 🚀


If you have documents that do not only have text and you're doing retrieval or RAG (using OCR and LLMs), give it up and give ColPali and vision language models a try 🤗

Why? Documents consist of multiple modalities: layout, table, text, chart, images. Document processing pipelines often consist of multiple models and they're immensely brittle and slow. 🥲

How? ColPali is a ColBERT-like document retrieval model built on PaliGemma, it operates over image patches directly, and indexing takes far less time with more accuracy. You can use it for retrieval, and if you want to do retrieval augmented generation, find the closest document, and do not process it, give it directly to a VLM like Qwen2-VL (as image input) and give your text query. 🤝

This is much faster + you do not lose out on any information + much easier to maintain too! 🥳

Multimodal RAG https://huggingface.co/collections/merve/multimodal-rag-66d97602e781122aae0a5139 💬
Document AI (made it way before, for folks who want structured input/output and can fine-tune a model) https://huggingface.co/collections/merve/awesome-document-ai-65ef1cdc2e97ef9cc85c898e 📖

adorkin web3builder John6666 Percifal jrmasiero rwightman seek007 abishekcodes zliu AI4Industry louisbrulenaudet byteprobe muhtasham rumbleFTW

umair894 abishekcodes fsommers jithinrocs rumbleFTW


🔥 Dataset Viber 0.3 launches with Synthesizer to synthesise data with a human in the loop, for free, using open source models with Argilla's distilabel but within a quick-and-easy Gradio Interface.

Why? Not trying to be all fancy and formal just to iterate on your data and to get familiar with your prompts and the produced data. Under the hood, it relies on Hugging Face Inference endpoints and the latest LLMs and VLMs like Meta Llama 3.1 and BlackForest Labs Flux models.

An addition to the other Interfaces that are already support.
- CollectorInterface: Lazily collect data of model interactions without human annotation.
- AnnotatorInterface: Walk through your data and annotate it with models in the loop.
- Synthesizer: Synthesize data with distilabel in the loop.
- BulkInterface: Explore your data distribution and annotate in bulk.

⭐ Give some good vibes: https://github.com/davidberenstein1957/dataset-viber


Datapluck: Portability Tool for Huggingface Datasets

"I found myself recently whipping up notebooks just to pull huggingface datasets locally, annotate or operate changes and update them again. This happened often enough that I made a cli tool out of it, which I've been using successfully for the last few months.

While huggingface uses open formats, I found the official toolchain relatively low-level and not adapted to quick operations such as what I am doing."
~ @omarkamali

Link : https://omarkama.li/blog/datapluck


Just wrapped up a deep dive into the latest lecture on building LLMs, such as ChatGPT, from @Stanford  CS229 course. Here are my top takeaways:

🔍 Understanding the Components: LLMs like ChatGPT, Claude, and others are more than just neural networks; they are a complex blend of architecture, training loss, data evaluation, and systems. Knowing how these components work together is key to improving and scaling these models.

📊 Scaling Matters: Performance improves predictably with more data, bigger models, and greater computational power. However, balancing these factors is crucial to avoid overfitting and resource waste.

📈 Data is King: LLMs are trained on trillions of tokens scraped from the internet, but the quality of this data matters immensely. Rigorous filtering and deduplication processes are essential to maintaining data integrity.

🏗 Pre-Training vs. Post-Training: While pre-training equips the model with general knowledge, post-training (like RLHF) fine-tunes it to follow human-like responses, reducing toxic outputs and improving alignment with human values.

🌐 Reinforcement Learning from Human Feedback (RLHF): This technique allows LLMs to maximize outputs that align with human preferences, making models more reliable and accurate.

💡 Why It Matters: Understanding these processes not only helps us appreciate the complexity behind our everyday AI tools but also highlights the challenges and opportunities in the ever-evolving field of AI.

Whether you’re in tech, data science, or just AI-curious, staying updated on these advancements is crucial. LLMs are not just transforming industries; they’re redefining the future of human-computer interaction!

I just realized this was almost 2 hours long...

Link: https://www.youtube.com/watch?v=9vM4p9NN0Ts


I just bought HF Pro but i don't know how many request per month i can get, if i request 1 time every 5s, around 2k token, is the pro account enough?, thanks for reading


My tool calling playgrounds repo has been updated again to include the use of flux1-schnell or dev image generation. This functionality is similar to using Dall-E 3 via the `@` decorator in ChatGPT. Once the function is selected, the model will either extract or improve your prompt (depending on how you ask).

I have also included 2 notebooks that cover different ways to access Flux for your specific use case. The first method covers how to access flux via LitServe from Lightning AI. LitServe is a bare-bones inference engine with a focus on modularity rather than raw performance. LitServe supports text generation models as well as image generation, which is great for some use cases, but does not provide the caching mechanisms from a dedicated image generation solution.

Since dedicated caching mechanisms are so crucial to performance, I also included an example for how to integrate SwarmUI/ComfyUI to utilize a more dedicated infrastructure that may already be running as part of your tech stack. Resulting in a Llama-3.1 capable of utilizing specific ComfyUI JSON configs, and many different settings.

Lastly, I tested the response times for each over a small batch request to simulate a speed test.

It becomes clear quickly how efficient caching mechanisms can greatly reduce the generation time, even in a scenario where another model is called. An average 4.5 second response time is not bad at all when you consider that an 8B model is calling a 12B parameter model for a secondary generation.

Repo: https://github.com/tdolan21/tool-calling-playground
LitServe: https://github.com/Lightning-AI/LitServe
SwarmUI: https://github.com/mcmonkeyprojects/SwarmUI


🥳  𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝗲𝗿𝘀 𝗔𝗴𝗲𝗻𝘁𝘀 𝗻𝗼𝘄 𝘀𝘂𝗽𝗽𝗼𝗿𝘁𝘀 𝗠𝘂𝗹𝘁𝗶-𝗮𝗴𝗲𝗻𝘁 𝘀𝘆𝘀𝘁𝗲𝗺𝘀!

Multi-agent systems have been introduced in Microsoft's framework Autogen. It simply means having several agents working together to solve your task instead of only one : this paradigm empirically yields better performance on most benchmarks. The reason for this better performance is conceptually simple: for many tasks, rather than using a do-it-all system, you would prefer to specialize units on sub-tasks. Here, having agents with separate tool sets and memories allows to achieve efficient specialization.

You can now easily build hierarchical multi-agent systems with transformers.agents (not released yet, use the dev version)

To do so, encapsulate the agent in a ManagedAgent object. This object needs arguments agent, name, and a description, which will then be embedded in the manager agent's system prompt to let it know how to call this managed agent, as we also do for tools.

Cf the example in the image! We'll keep building on this paradigm in the upcoming weeks 🚀

Read more in the doc 👉 https://github.com/huggingface/transformers/blob/main/docs/source/en/agents_advanced.md

Checkout an advanced multi-agent system that tops the GAIA leaderboard 👉 https://github.com/aymeric-roucher/GAIA/blob/main/gaia_multiagent.py

ibrahim313 John6666 osanseviero Kaoeiri dsmonk Csplk KingNish whitebill Winnougan


the new version of Enigma, our code-instruct specialist, is out now:
- https://huggingface.co/ValiantLabs/Llama3.1-8B-Enigma is trained on code-instruct and general chat data.
- the updated code-instruct database is available now as well: https://huggingface.co/datasets/sequelbox/Tachibana

more to come soon!


🐣Ai2 Releasing OLMoE!
OLMoE-1B-7B-Instruct is a Mixture-of-Experts LLM with 1B active and 7B total parameters, and, OLMoE is 100% open-source in model, code-base, datasets!

🦖Paper: https://arxiv.org/abs/2409.02060

🤗Model: https://huggingface.co/allenai/OLMoE-1B-7B-0924-Instruct

💾Datasets: https://huggingface.co/datasets/allenai/OLMoE-mix-0924


If you want a clear understanding of the environmental impacts of AI throughout its entire lifecycle, this primer by @sasha @brunatrevelin and @meg is a must-read.

It brilliantly explains which types of impacts occur, when they happen, and why they matter.

https://huggingface.co/blog/sasha/ai-environment-primer

brunatrevelin John6666 not-lain BrigitteTousi louisbrulenaudet


The new Qwen-2 VL models seem to perform quite well in object detection. You can prompt them to respond with bounding boxes in a reference frame of 1k x 1k pixels and scale those boxes to the original image size.

You can try it out with my space https://huggingface.co/spaces/maxiw/Qwen2-VL-Detection

de-Rodrigo dsmonk tosouth hxypqr mrdbourke SvPolina akazakov Panerlu iiBLACKii


The MERIT Dataset: Modelling and Efficiently Rendering Interpretable
  Transcripts (2409.00447)

A few weeks ago, we uploaded the MERIT Dataset 🎒📃🏆 into Hugging Face 🤗!

Now, we are excited to share the Merit Dataset paper via arXiv! 📃💫
https://huggingface.co/papers/2409.00447

The MERIT Dataset is a fully synthetic, labeled dataset created for training and benchmarking LLMs on Visually Rich Document Understanding tasks. It is also designed to help detect biases and improve interpretability in LLMs, where we are actively working. 🔧🔨

MERIT contains synthetically rendered students' transcripts of records from different schools in English and Spanish. We plan to expand the dataset into different contexts (synth medical/insurance documents, synth IDS, etc.) Want to collaborate? Do you have any feedback? 🧐

Resources:

- Dataset: https://huggingface.co/datasets/de-Rodrigo/merit
- Code and generation pipeline: https://github.com/nachoDRT/MERIT-Dataset

PD: We are grateful to Hugging Face 🤗 for providing the fantastic tools and resources we find in the platform and, more specifically, to @nielsr for sharing the fine-tuning/inference scripts we have used in our benchmark.


@victor Sorry for the repetitiveness.

I'm not sure if Post is the right place to report such an error, but it seems to be a server error unrelated to the Zero GPU space error the other day, so I don't know where else to report it.

Since this morning, I have been getting a strange error when running inference from space in Gradio 3.x.
Yntec (https://huggingface.co/Yntec) discovered it, but he is not in the Pro subscription, so I am reporting it on behalf of him.

The error message is as follows: 1girl and other prompts will show cached output, so experiment with unusual prompts.

Thank you in advance.

https://huggingface.co/spaces/John6666/blitz_diffusion_error
https://huggingface.co/spaces/John6666/GPU-stresser-t2i-error
```
ValueError: Could not complete request to HuggingFace API, Status Code: 500, Error: unknown error, Warnings: ['CUDA out of memory. Tried to allocate 30.00 MiB (GPU 0; 14.75 GiB total capacity; 1.90 GiB already allocated; 3.06 MiB free; 1.95 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF', 'There was an inference error: CUDA out of memory. Tried to allocate 30.00 MiB (GPU 0; 14.75 GiB total capacity; 1.90 GiB already allocated; 3.06 MiB free; 1.95 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF']
```


I am integrating Azure Cosmos DB, the database system that backs GPT conversations into my workflow, and experimenting with new patterns to accelerate dataset evolution for evaluation and training of AI.

While initially using it for research prompts and research outputs using my GPT-4o client here which can interface and search ArXiv, I am excited to try out some new features specifically for AI at scale.  Research on memory augmentation is shown.  https://huggingface.co/spaces/awacke1/GPT-4o-omni-text-audio-image-video

  https://huggingface.co/spaces/awacke1/AzureCosmosDBUI


🚀 Introducing Hugging Face's Multilingual Speech-to-Speech! 🎤
💬Our modular, cross-platform pipeline to run GPT4o-like experiences on device can now seamlessly switch languages mid-conversation with an imperceptible 100ms delay.

🌟 Building on an amazing early reception with 2600 stars on GitHub 🌟
🚀 We are expanding the library to support multiple languages
🔥 Try it out with a flag: --language fr
🤯 Or don't set the flag and let the system detect the language

💡 What feature should we add next?

prithivMLmods osanseviero John6666 THEFIG


🧭 Guided Reasoning

👋Hi everyone,

We've been releasing Guided Reasoning:

Our AI guides walk your favorite LLM through complex reasoning problems.

🎯 Goals:

1⃣ Reliability. AIs consistently follow reasoning methods.
2⃣ Self-explainability. AIs see reasoning protocols and can explain internal deliberation.
3⃣ Contestability. Users may amend AI reasoning and revise plausibility assessments.

Try out Guided Reasoning with our light demo chatbot, powered by 🤗 HuggingFace's free Inference Api and small LLMs. (Sorry for poor latency and limited availability -- we are currently searching for 💸 compute sponsors to run more powerful models, faster, and optimize guided reasoning performance.)

Built on top of Logikon's open-source AI reasoning analytics.

Demo chat app: https://huggingface.co/spaces/logikon/benjamin-chat
Github: https://github.com/logikon-ai/logikon
Technical report: https://arxiv.org/abs/2408.16331

➡ Check it out and get involved! Looking forward to hearing from you.


 Fine-tuned Phi-3.5 Chatbot

This project presents a fine-tuned version of Microsoft's Phi-3.5 model, optimized for enhanced conversational abilities and general knowledge tasks.

Model Details
- Base model: microsoft/Phi-3.5-mini-instruct
- Fine-tuning method: PEFT (Parameter-Efficient Fine-Tuning)
- Training data: [Brief description of your dataset]

 Features
- Improved response generation for a wide range of topics
- Enhanced context understanding and coherence
- Optimized for deployment on Hugging Face Spaces

Usage
This model can be used for various natural language processing tasks, including:
- General conversation
- Question answering
- Task instructions
- Creative writing

Try out the model here :
  https://huggingface.co/spaces/sagar007/phi3.5_mini_instruct_finetune

Limitations
While this fine-tuned model shows improved performance, users should be aware of potential biases and limitations inherent in language models. Always critically evaluate the model's outputs.

 Feedback
I welcome any feedback, suggestions, or questions about this project. Feel free to open an issue or contribute to further improvements!

#Phi35 #FineTuning #NLP #MachineLearning #HuggingFace


Hyperfast Contextual Custom LLM with Agents, Multitokens, Explainable AI, and Distillation https://mltblog.com/4dNPSnB

New additions to this ground-breaking system include multi-token distillation when processing prompts, agents to meet user intent, more NLP, and a command prompt menu accepting both standard prompts and various actions.

I also added several illustrations, featuring xLLM in action with a full session and sample commands to fine-tune in real-time. All the code, input sources (anonymized corporate corpus from fortune 100 company), contextual backend tables including embeddings, are on GitHub. My system has zero weight, no transformer, and no neural network. It relies on explainable AI, does not require training, is fully reproducible, and fits in memory. Yet your prompts can retrieve relevant full text entities from the corpus with no latency — including URLs, categories, titles, email addresses, and so on — thanks to well-designed architecture.

Read more, get the code, paper and everything for free, at https://mltblog.com/4dNPSnB


𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:

𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:

Human Feedback is not Gold Standard (2309.16349)

🚨 𝗛𝘂𝗺𝗮𝗻 𝗙𝗲𝗲𝗱𝗯𝗮𝗰𝗸 𝗳𝗼𝗿 𝗔𝗜 𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴: 𝗡𝗼𝘁 𝘁𝗵𝗲 𝗴𝗼𝗹𝗱𝗲𝗻 𝗴𝗼𝗼𝘀𝗲 𝘄𝗲 𝘁𝗵𝗼𝘂𝗴𝗵𝘁?

I’ve just read a great paper where Cohere researchers raises significant questions about using Human feedback to evaluate AI language models.

Human feedback is often regarded as the gold standard for judging AI performance, but it turns out, it might be more like fool's gold : the study reveals that our human judgments are easily swayed by factors that have nothing to do with actual AI performance.

𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:
🧠 Test several models: Llama-2, Falcon-40B, Cohere Command 6 and 52B 🙅♂ Refusing to answer tanks AI ratings more than getting facts wrong. We apparently prefer a wrong answer to no answer!

💪 Confidence is key (even when it shouldn't be): More assertive AI responses are seen as more factual, even when they're not. This could be pushing AI development in the wrong direction, with systems like RLHF.

🎭 The assertiveness trap: As AI responses get more confident-sounding, non-expert annotators become less likely to notice when they're wrong or inconsistent.

And a consequence of the above:
🔄 𝗥𝗟𝗛𝗙 𝗺𝗶𝗴𝗵𝘁 𝗯𝗮𝗰𝗸𝗳𝗶𝗿𝗲: Using human feedback to train AI (Reinforcement Learning from Human Feedback) could accidentally make AI more overconfident and less accurate.

This paper means we need to think carefully about how we evaluate and train AI systems to ensure we're rewarding correctness over apparences of it like confident talk.

⛔ Chatbot Arena’s ELO leaderboard, based on crowdsourced answers from average joes like you and me, might become completely irrelevant as models will become smarter and smarter.

Read the paper 👉 https://huggingface.co/papers/2309.16349


Is AI’s impact on elections being overblown? Three researchers think so in this opinion piece published in the MIT Tech Review.

Highlights:

•	“AI is being used to try to influence electoral processes, but these efforts have not been fruitful.”
•	“Why were these initial speculations about AI-enabled electoral interference so off (…) ? The short answer: Because they ignored decades of research on the limited influence of mass persuasion campaigns, the complex determinants of voting behaviors, and the indirect and human-mediated causal role of technology.”
•	“Yet we should remember that there’s a cost to overreaction based on ill-founded assumptions, especially when other critical issues go unaddressed.”

👉Read more here: https://technologyreview.com/2024/09/03/1103464/ai-impact-elections-overblown/


The Forward-Forward Algorithm🤖

FFA replaces the forward and backward passes in backpropagtion  with two forward passes - one with positive (real) data and another with negative data. Each layer has its objective function - to increase or decrease a “goodness" metric. The positive pass uses real data and adjusts weights to increase “goodness” in every hidden layer. The negative pass does the opposite.

I must say reading&Implementing a godfather paper feels quite fulfilling:)
Thank you Prof. Geoffrey Hinton.

Code: https://github.com/Jaykef/ai-algorithms/blob/main/mnist_the_forward_forward_algorithm.ipynb


𝐌𝐲 𝐟𝐢𝐫𝐬𝐭 𝐜𝐨𝐦𝐦𝐮𝐧𝐢𝐭𝐲 𝐚𝐫𝐭𝐢𝐜𝐥𝐞! 𝐒𝐞𝐥𝐞𝐜𝐭𝐢𝐯𝐞 𝐟𝐢𝐧𝐞-𝐭𝐮𝐧𝐢𝐧𝐠 𝐰𝐢𝐭𝐡 𝐒𝐩𝐞𝐜𝐭𝐫𝐮𝐦 🎯

Full walkthrough on how to get started with Spectrum and TRL for efficient fine-tuning.
📔 👣 https://huggingface.co/blog/anakin87/spectrum

---

Looking to fine-tune Language Models efficiently and save on computational resources?

One popular method is QLoRa, which quantizes the original model and trains low-rank adapters on top.
It's quite effective and uses less GPU than full fine-tuning.

However, QLoRa applies Low-Rank Adaptation uniformly across the entire model.

What if we could identify the most informative layers and only fine-tune those? 🤔

This is exactly what Spectrum does! 👇

🔬 Spectrum analyzes the weight matrices for all layers in a Language Model and calculates a Signal to Noise Ratio (SNR) for each one.
(It uses Random Matrix Theory and Marchenko-Pastur distribution to distinguish signal from noise.)

🎯 Based on a chosen percentage (say, 25%), Spectrum selects the most informative layers of each type (mlp.down_proj, self_attn.o_proj, etc.).

You can then ❄ freeze the rest of the model and focus your 🏋♂ training on the chosen layers.

🏆 Results/Evaluation
- Spectrum is competitive with full fine-tuning and beats QLoRA on benchmarks.
- While QLoRA is more memory-efficient on a single GPU, Spectrum shines in distributed training setups.
- Great models trained with Spectrum: Dolphin models, Llama 3.1 Storm, numerous models by VAGO Solutions...

---

For a practical guide, check out the article above.


𝐍𝐞𝐰 𝐑𝐞𝐥𝐞𝐚𝐬𝐞: 𝐌𝐚𝐣𝐨𝐫 𝐓𝐎𝐌 𝐃𝐢𝐠𝐢𝐭𝐚𝐥 𝐄𝐥𝐞𝐯𝐚𝐭𝐢𝐨𝐧 𝐌𝐨𝐝𝐞𝐥 𝐄𝐱𝐩𝐚𝐧𝐬𝐢𝐨𝐧 🗺

Dataset: https://huggingface.co/datasets/Major-TOM/Core-DEM

Today with European Space Agency - ESA and Adobe Research, we release a global expansion to Major TOM with GLO-30 DEM data.

You can now instantly access nearly 2M of Major TOM samples with elevation data to build your next AI model for EO. 🌍

🔍 Browse the data in our usual viewer app: https://huggingface.co/spaces/Major-TOM/MajorTOM-Core-Viewer

Fantastic work championed by Paul Borne--Pons @NewtNewt  🚀

Tonic AtAndDev NewtNewt osanseviero Obenlia robmarkcole


# Excited to Share: New LLM Tokenization - Convert Text to tokens and vice versa! 🚀

I've just developed a powerful tool for anyone working with Language Models (LLMs) or diving into Natural Language Processing (NLP).

🔍 Introducing the LLM Tokenization - Convert Text to tokens and vice versa!!

Key Features:
- Convert text to tokens and token IDs
- Reverse engineer: convert token IDs back to text
- Support for popular models: LLama3 (Will add more models iteratively)
- User-friendly Gradio interface for easy interaction

Whether you're debugging your NLP pipeline, exploring how different models tokenize text, or just curious about the inner workings of LLMs, this tool is for you!

👩💻 Tech Stack:
- Python
- Gradio for the web interface
- Hugging Face Transformers for tokenization

The application is deployed in Hugging Face spaces as Gradio application

🔗 Try it out: https://lnkd.in/g6R5z9k2

#NLP #MachineLearning #AI #PythonDevelopment #OpenSource


I started training a public LoRA style (2 seperate training each on 4x A6000).

Experimenting captions vs non-captions. So we will see which yields best results for style training on FLUX.

Generated captions with multi-GPU batch Joycaption app.

I am showing 5 examples of what Joycaption generates on FLUX dev. Left images are the original style images from the dataset.

I used my multi-GPU Joycaption APP (used 8x A6000 for ultra fast captioning) : https://www.patreon.com/posts/110613301

I used my Gradio batch caption editor to edit some words and add activation token as ohwx 3d render : https://www.patreon.com/posts/108992085

The no caption dataset uses only ohwx 3d render as caption

I am using my newest 4x_GPU_Rank_1_SLOW_Better_Quality.json on 4X A6000 GPU and train 500 epochs — 114 images : https://www.patreon.com/posts/110879657

Total step count is being 500 * 114 / 4 (4x GPU — batch size 1) = 14250

Taking 37 hours currently if I don’t terminate early

Will save a checkpoint once every 25 epochs

Full Windows Kohya LoRA training tutorial : https://youtu.be/nySGu12Y05k

Full cloud tutorial I am still editing

Hopefully will share trained LoRA on Hugging Face and CivitAI along with full dataset including captions.

I got permission to share dataset but can’t be used commercially.

Also I will hopefully share full workflow in the CivitAI and Hugging Face LoRA pages.


Spent a few minutes to build an alternative to Character AI on top of  llama3.1 405B through SambaNova's super fast inference API

Space: https://huggingface.co/spaces/kz919/Persona-AI
API referral link: https://sambanova.ai/fast-api?api_ref=907266


ML people on a long flight

(See picture)


🙋🏻♂hey there folks ,

✒InkubaLM has been trained from scratch using 1.9 billion tokens of data for five African languages, along with English and French data, totaling 2.4 billion tokens of data. It is capable of understanding and generating content in five African languages: Swahili, Yoruba, Hausa, isiZulu, and isiXhosa, as well as English and French.

model https://huggingface.co/lelapa/InkubaLM-0.4B
demo https://huggingface.co/spaces/Tonic/Inkuba-0.4B

monsoon-nlp AtAndDev KingNish louisbrulenaudet d0rj Moio


Do you know how PCA and SVD are related?
I explained it for everyone in this post!
Go and check it out: https://x.com/alexbodner_/status/1798357519678718062?s=46


Hey everyone 🤗!
Check out this awesome new model for object segmentation!
https://huggingface.co/spaces/finegrain/finegrain-object-cutter.

We (finegrain) have trained this new model in partnership with Nfinite and some of their synthetic data, the resulting model is incredibly accurate 🚀.
It’s all open source under the MIT license (https://huggingface.co/finegrain/finegrain-box-segmenter), complete with a test set tailored for e-commerce (https://huggingface.co/datasets/finegrain/finegrain-product-masks-lite). Have fun experimenting with it!

limiteinductive Sri-Vigneshwar-DJ John6666 fdaudens TDL123 1aurent deltheil piercus Mefistofele osanseviero vincentweisser victor dsmonk


𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:

𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:

The AI Scientist: Towards Fully Automated Open-Ended Scientific
  Discovery (2408.06292)

🤖 𝗧𝗵𝗲 𝗔𝗜 𝗦𝗰𝗶𝗲𝗻𝘁𝗶𝘀𝘁: 𝗔𝗴𝗲𝗻𝘁𝗶𝗰, 𝗳𝘂𝗹𝗹𝘆-𝗮𝘂𝘁𝗼𝗺𝗮𝘁𝗲𝗱 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵 𝗽𝗶𝗽𝗲𝗹𝗶𝗻𝗲 𝗳𝗼𝗿 𝘂𝗻𝗱𝗲𝗿 $𝟭𝟱 𝗽𝗲𝗿 𝗽𝗮𝗽𝗲𝗿

Researchers have just created an AI system that 𝗰𝗮𝗻 𝗰𝗼𝗻𝗱𝘂𝗰𝘁 𝗲𝗻𝘁𝗶𝗿𝗲 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀 𝗳𝗿𝗼𝗺 𝘀𝘁𝗮𝗿𝘁 𝘁𝗼 𝗳𝗶𝗻𝗶𝘀𝗵, 𝗽𝗼𝘁𝗲𝗻𝘁𝗶𝗮𝗹𝗹𝘆 𝗿𝗲𝘃𝗼𝗹𝘂𝘁𝗶𝗼𝗻𝗶𝘇𝗶𝗻𝗴 𝗵𝗼𝘄 𝘀𝗰𝗶𝗲𝗻𝘁𝗶𝗳𝗶𝗰 𝗱𝗶𝘀𝗰𝗼𝘃𝗲𝗿𝗶𝗲𝘀 𝗮𝗿𝗲 𝗺𝗮𝗱𝗲.

It doesn't just assist with specific tasks - it automates the entire research process, from generating ideas to writing and reviewing papers.
1 - brainstorm novel research directions, 2- write and execute code for experiments & visualize results, get references, and even 3- write up findings in a full academic paper format!

And it can do all this for under $15 per paper! 🤯

𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:
🧠 Generates novel research ideas across multiple topics (e.g. diffusion modeling, transformers, learning dynamics aka “grokking”)
👨💻 Uses open-source coding assistant Aider to implement ideas and run experiments. This is especially important since this agentic assistant can iterate if it fails somewhere.
📊 Visualizes results and plans follow-up experiments (up to 5 rounds)
✍ Writes full academic papers, including finding references using Semantic Search API
🕵 Runs a simulated peer review process to evaluate paper quality
💰 Total cost per paper is under $15. This system can generate "hundreds of interesting, medium-quality papers" in just a week !

𝗦𝘁𝗶𝗹𝗹 𝗻𝗼𝘁 𝗿𝗲𝗮𝗱𝘆 𝘁𝗼 𝗳𝗶𝗹𝗹 𝗜𝗖𝗟𝗥 𝘄𝗶𝘁𝗵 𝗽𝗮𝗽𝗲𝗿𝘀:
🔁 Ideas generated in one domain tend to be repetitive across different runs, and even different language model
👀 Does not use vision capabilities to fix visual issues in plots
💭 Models occasionally hallucinate entire results tables
⇒ Only few of the generated papers would actually meet the threshold for acceptance at a top AI conference

👉 Read their paper: https://huggingface.co/papers/2408.06292

wsuff Sri-Vigneshwar-DJ John6666 Bruhn sugatoray KingNish louisbrulenaudet toshihikochen alielfilali01


Plugins in NiansuhAI

Plugin Names:
1. WebSearch: Searches the web using search engines.
2. Calculator: Evaluates mathematical expressions, extending the base Tool class.
3. WebBrowser: Extracts and summarizes information from web pages.
4. Wikipedia: Retrieves information from Wikipedia using its API.
5. Arxiv: Searches and fetches article information from Arxiv.
6. WolframAlphaTool: Provides answers on math, science, technology, culture, society, and everyday life.

These plugins currently support the GPT-4O-2024-08-06 model, which also supports image analysis.

Try it now: https://huggingface.co/spaces/NiansuhAI/chat

Similar to: https://hf.co/chat

John6666 TheDrunkenSnail leeloolee Sri-Vigneshwar-DJ Joseph717171 Niansuh


🤩 Amazing day. AWPortrait-FL finally here!
🦖 AWPortrait-FL is finetuned on FLUX.1-dev using the training set of AWPortrait-XL and nearly 2,000 fashion photography photos with extremely high aesthetic quality.

🤗Model: https://huggingface.co/Shakker-Labs/AWPortrait-FL

🙇Demo: https://huggingface.co/spaces/vilarin/flux-labs

orrinin samapika dillfrescott John6666 wanghaofan ijohn07 Bruhn ajibawa-2023 gshreyash tanfar lunarflu louisbrulenaudet ShakkerAi-Labs linoyts Despina KingNish victor sasikiran Pranavan nbroad Sri-Vigneshwar-DJ privategeek24 traltyaziking TDL123 koochikoo25 Taylor658 AtAndDev huangy1 tayyabmehar27 Mefistofele ibrahim313 awplanet


Understanding the json format response with HF's Serverless Inference API 🤗

As it stands, there seems to be an inconsistency with the OpenAI documentation on the question of implementing the JSON response format using the InferenceClient completion API.

After investigating the InferenceClient source code, I share the official solution using a JSON Schema. This consolidates the structure of the response and simplifies parsing as part of an automated process for extracting metadata, information:
```python
from huggingface_hub import InferenceClient

client = InferenceClient("meta-llama/Meta-Llama-3-70B-Instruct")

messages = [
    {
        "role": "user",
        "content": "I saw a puppy a cat and a raccoon during my bike ride in the park. What did I saw and when?",
    },
]

response_format = {
    "type": "json",
    "value": {
        "properties": {
            "location": {"type": "string"},
            "activity": {"type": "string"},
            "animals_seen": {"type": "integer", "minimum": 1, "maximum": 5},
            "animals": {"type": "array", "items": {"type": "string"}},
        },
        "required": ["location", "activity", "animals_seen", "animals"],
    },
}

response = client.chat_completion(
    messages=messages,
    response_format=response_format,
    max_tokens=500,
)

print(response.choices[0].message.content)
```
As a reminder, json mode is activated with the OpenAI client as follows:
```python
response = client.chat.completions.create(
     model="gpt-3.5-turbo-0125",
     messages=[...],
     response_format={"type": "json_object"}
)
```
One question remains unanswered, however, and will perhaps be answered by the community: it seems that an incompatibility persists for list of dictionaries generation, and currently, the production of simple dictionaries seems to be the only functional option.

John6666 victor wsuff rreed-pha osanseviero


I am training a controlnet model for Flux. And some of my experiences:

Checkpoint-10000:

https://x.com/kadirnar_ai/status/1829831750471606668

Checkpoint-12000:

https://x.com/kadirnar_ai/status/1829889524962640001

Checkpoint-14000:

https://x.com/kadirnar_ai/status/1829989622878744711

Checkpoint (16000-18000):

https://x.com/kadirnar_ai/status/1830179551407665654

Dataset:  https://huggingface.co/datasets/kadirnar/fluxdev_controlnet_16k
GPU: 1xA100(80GB)
GPU Hours: 65

tolgacangoz John6666 gokaygokay Shinku AtAndDev xziayro Saugatkafley l3x13

tolgacangoz louisbrulenaudet gokaygokay Pranavan AtAndDev xziayro osanseviero

tolgacangoz gokaygokay AtAndDev Sri-Vigneshwar-DJ


Last Week in Medical AI: Top Research Papers/Models
🏅 (August 25 - August 31, 2024)

- MultiMed: Multimodal Medical Benchmark
- A Foundation model for generating chest X-ray images
- MEDSAGE: Medical Dialogue Summarization
-  Knowledge Graphs for Radiology Report Generation
- Exploring Multi-modal LLMs for Chest X-ray
- Improving Clinical Note Generation
...

Check the full thread : https://x.com/OpenlifesciAI/status/1829984701324448051

aaditya jaebumskiyomi aiisthebest ai-everyday adityaSaligram dblasko victor JCDentonInTheFresh

aaditya John6666 jaebumskiyomi aiisthebest JCDentonInTheFresh


💾🧠How much VRAM will you need for training your AI model? 💾🧠
Check out this app where you convert:
Pytorch/tensorflow summary -> needed VRAM
or
Parameter count -> needed VRAM

Use it in: http://howmuchvram.com

And everything is open source! Ask for new functionalities or contribute in:
https://github.com/AlexBodner/How_Much_VRAM
If it's useful to you leave a star 🌟and share it to someone that will find the tool useful!

rmayormartins AlexBodner whitebill erkhem-gantulga AtAndDev


From Article 50 of the EU AI Act:

"2. Providers of AI systems, including general-purpose AI systems, generating synthetic audio, image, video or text content, shall ensure that the outputs of the AI system are marked in a machine-readable format and detectable as artificially generated or manipulated."

How might this be put into practice?

I'm interested to know how content might be deemed as being "detectable" as artificially generated. I wonder if this will require an image be detectable as AI generated if it was copied out of the site / application it was created on?

Some sort of a watermark? LSB Stegranography? I wonder if openAI are already sneaking something like this into DALL-E images.

Some sort of hash, which allowing content to be looked up, and verified as AI generated?

Would a pop up saying "this output was generated with AI"? suffice? Any ideas? Time is on the system provider's side, at least for now, as from what I can see this doesn't come into effect until August 2026.

src: https://artificialintelligenceact.eu/article/50/


I found this paper to be thought-provoking: "Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling" by Bansal, Hosseini, Agarwal,  Tran, and Kazemi.
https://arxiv.org/abs/2408.16737
The direct implication is that smaller models could be used to create cost-effective synthetic datasets. And on that note, in the Gemma terms of use, Google explicitly claims no rights on outputs generated from those models, which means one is free to synthgen from the Gemma line. Meta's Llama 3 licence forbids synthetic generation of outputs if used to improve other models. Relevant Mistral, Qwen, and Yi models under the Apache 2.0 license are unrestricted for this purpose.

John6666 anakin87 AtAndDev tachyon-beep tommulder victor gghfez louisbrulenaudet djuna


Just tried LitServe from the good folks at @LightningAI!

Between llama.cpp and vLLM, there is a small gap where a few large models are not deployable!

That's where LitServe comes in!

LitServe is a high-throughput serving engine for AI models built on FastAPI.

Yes, built on FastAPI. That's where the advantage and the issue lie.

It's extremely flexible and supports multi-modality and a variety of models out of the box.

But in my testing, it lags far behind in speed compared to vLLM.

Also, no OpenAI API-compatible endpoint is available as of now.

But as we move to multi-modal models and agents, this serves as a good starting point. However, it’s got to become faster...

GitHub: https://github.com/Lightning-AI/LitServe


🙋🏻♂Hey there folks,

just published a demo for Salesforce's new Function Calling Model `Salesforce/xLAM`

- https://huggingface.co/spaces/Tonic/Salesforce-Xlam-7b-r
- https://huggingface.co/spaces/Tonic/On-Device-Function-Calling

just try em out, and it comes with `on-device`version too ! cool ! 🚀


new synthetic general chat dataset! meet Supernova, a dataset using prompts from UltraFeedback and responses from Llama 3.1 405b Instruct:  https://huggingface.co/datasets/sequelbox/Supernova

new model(s) using the Supernova dataset will follow next week, along with Other Things. (One of these will be a newly updated version of Enigma, utilizing the next version of https://huggingface.co/datasets/sequelbox/Tachibana with approximately 2x the rows!)


Very excited to have made the list and been invited to OpenAI DevDay 2024 at the London event 30 October! Looking forward to seeing what the future of AI dev holds, connecting with other professionals in the field, and advocating for open source AI!

https://openai.com/devday/


💾🧠Want to know how much VRAM you will need for training your model? 💾🧠
Now you can use this app in which you can input a torch/tensorflow summary or the parameters count and get an estimate of the required memory!
Use it in: howmuchvram.com

Also, everything is Open Source so you can contribute in repo: https://github.com/AlexBodner/How_Much_VRAM
Leave it a star⭐

Mehyaar MexIvanov Bruhn den0620 AtAndDev


Shakker-Labs brings an amazing LoRA trained on FLUX.1-dev for blended realistic illustration by Muertu 😍 the front character is in illustration style, while the background is realistic. 🤩

🤙Model: https://huggingface.co/Shakker-Labs/FLUX.1-dev-LoRA-blended-realistic-illustration

🙇♂My space for demo: https://huggingface.co/spaces/vilarin/flux-lab-light

orrinin YaTharThShaRma999 John6666 chris-fung djuna alielfilali01 ngxson Felladrin wanghaofan AtAndDev keakohv louisbrulenaudet


Made a fun Space powered by Llama 405B for creating real, working react apps with the awesome plus that you can contribute to an open react dataset by upvoting or downvoting the response 🤗.

https://huggingface.co/spaces/cfahlgren1/llama-artifacts

https://huggingface.co/datasets/cfahlgren1/react-code-instructions


Here's a 1-minute video tutorial on how to fine-tune https://huggingface.co/unsloth/llama-3-8b-bnb-4bit with unsloth

Using Roller Coaster Tycoon peep thoughts as an example

victor John6666 prithivMLmods KingNish thisisanshgupta Bruhn budotsmedia AtAndDev mambiux


AI in the News: Llama 10x growth, Apple & Nvidia in talks with OpenAI, universal basic income, AI & art

* Meta leads open-source AI boom, Llama downloads surge 10x year-over-year - VB
https://venturebeat.com/ai/meta-leads-open-source-ai-boom-llama-downloads-surge-10x-year-over-year/

* Apple, Nvidia Are in Talks to Invest in OpenAI - WSJ
https://www.wsj.com/tech/ai/openai-apple-funding-chatgpt-50754cd6?mod=rss_Technology

* The Report Card on Guaranteed Income Is Still Incomplete - NYT
https://www.nytimes.com/2024/08/30/business/economy/the-report-card-on-guaranteed-income-is-still-incomplete.html

* Ethically dubious or a creative gift? How artists are grappling with AI in their work - The Guardian
https://www.theguardian.com/artanddesign/article/2024/aug/30/xanthe-dobbie-futuer-sex-love-sounds-ai-video-celebrity-clones

Want more? Subscribe to my daily newsletter!
https://linkedin.com/build-relation/newsletter-follow?entityUrn=7233909926606053377


Sharing for anyone using Diffusers `from_single_file` loading and affected by the Runway SD 1.5 issue.

If you have `runwayml/stable-diffusion-v1-5` saved locally in your HF cache then loading single file checkpoints in the following way should still work.

```
from diffusers import StableDiffusionPipeline

pipe = StableDiffusionPipeline.from_single_file("<url or path to single file checkpoint>")
```

If you do not have the model repo saved in your cache, then automatically inferring the pipeline config will not work since the reference repo `runwayml/stable-diffusion-v1-5` doesn't exist anymore.

You can use an alternative SD1.5 repo id to still configure your pipeline.

```
from diffusers import StableDiffusionPipeline

pipe = StableDiffusionPipeline.from_single_file("<url or path to single file checkpoint>", config="Lykon/DreamShaper")
```

We're working on resolving the issue ASAP.


The only 405B spaces still freely accessible are powered by SN fast api.

https://huggingface.co/spaces/xianbao/SambaNova-fast

https://sambanova.ai/fast-api?api_ref=907266

John6666 clem AtAndDev kz919 Amr-khaled louisbrulenaudet


The word 'Lead' has three definitions. When an LLM model tokenizes this word, it is always the same token. Imagine being able to put any particular embedding at any particular time into a 'Quantum State'. When an Embedding is in a Quantum State, the word token could have up to 3 different meanings (x1, x2, x3). The Quantum State gets collapsed based on the individual context surrounding the word. 'Jill lead Joy to the store' would collapse to x1. 'Jill and Joy stumbled upon a pile of lead' would collapse to x3. Very simple, right? This method produces OFF THE CHARTS results:

https://www.youtube.com/watch?v=tuQI6A-EOqE


Continuing my streak by releasing the Wikireading dataset: a large collection of scraped non-fiction books predominantly in Russian language.
https://huggingface.co/datasets/its5Q/wikireading

Here's the highlights:
- ~7B tokens, or ~28B characters, making it a great candidate for use in pretraining
- Contains non-fiction works from many knowledge domains
- Includes both the original HTML and extracted text of book chapters


Thought this was an interesting graphic from the EAGLE blog post. It made me wonder if certain sampling methods have been shown to work better for certain tasks.

Does anyone know of any work looking at trends in the output token probability distribution by task type? (or similar)

Source: https://sites.google.com/view/eagle-llm


Automated web scraping with playwright is becoming easier by the day. Now, using ollama tool calling, its possible to perform very high accuracy web scraping (in some cases 100% accurate) through just asking an LLM to scrape the content for you.

This  can be completed in a multistep process similar to cohere's platform. If you have tried the cohere playground with web scraping, this will feel very similar. In my experience, the Llama 3.1 version is much better due to the larger context window. Both tools are great, but the difference is the ollama + playwright version is completely controlled by you.

All you need to do is wrap your scraper in a function:

```
 async def query_web_scraper(url: str) -> dict:
    scraper = WebScraper(headless=False)
    return await scraper.query_page_content(url)
```

and then make your request:

```
# First API call: Send the query and function description to the model
response = ollama.chat(
    model=model,
    messages=messages,
    tools=[
        {
            'type': 'function',
            'function': {
                'name': 'query_web_scraper',
                'description': 'Scrapes the content of a web page and returns the structured JSON object with titles, articles, and associated links.',
                'parameters': {
                    'type': 'object',
                    'properties': {
                        'url': {
                            'type': 'string',
                            'description': 'The URL of the web page to scrape.',
                        },
                    },
                    'required': ['url'],
                },
            },
        },
    ]
)
```

To learn more:
Github w/ Playground: https://github.com/tdolan21/tool-calling-playground/blob/main/notebooks/ollama-playwright-web-scraping.ipynb
Complete Guide:  https://medium.com/@tdolan21/building-an-llm-powered-web-scraper-with-ollama-and-playwright-6274d5d938b5

RalFinger xsa-dev wsuff alielfilali01 Bruhn


Simplified implementation of “Neural Networks are Decision Trees”.

Showing that any neural network with any activation function can be represented as a decision tree. Since decision trees are inherently interpretable, their equivalence helps us understand how the network makes decisions.

In this implementation, we trained a simple neural network for 1k epochs on makemoons, saved the trained weights (state dicts), extracted the decision tree equivalent from the trained weight then visualize and evaluate.

Code: https://github.com/Jaykef/ai-algorithms/blob/main/nns_are%20decision_trees.ipynb

prithivMLmods John6666 maier-s jsulz AtAndDev


@victor

Excuse me.
I would like to report the following bug or new specification that is probably the cause of the fatal stacks that are occurring in the Zero GPU space throughout HF.
Thanks.

https://huggingface.co/spaces/zero-gpu-explorers/README/discussions/104


NVIDIA just dropped NVEagle 🦅

Super impressive vision language model that comes in 7B, 13B and 13B fine-tuned on chat 💬
Model repositories: https://huggingface.co/collections/merve/nveagle-66d0705108582d73bb235c26
Try it: https://huggingface.co/spaces/NVEagle/Eagle-X5-13B-Chat 💬 (works very well! 🤯)

This model essentially explores having different experts (MoE) for image encoder part of vision language model.
How? 🧐
The authors concatenate the vision encoder output tokens together, and they apply "pre-alignment" essentially fine-tune experts with frozen text encoder.

Then they freeze both experts and the decoder and just train the projection layer, and finally, they unfreeze everything for supervised fine-tuning ✨

In the paper, they explore different fusion strategies and vision encoders, extending basic CLIP encoder, and figure out simply concatenating visual tokens works well.
Rest of the architecture is quite similar to LLaVA. (see below the architecture)

sagar007 Kukedlc John6666 khanhnamle1994 maximuspowers alielfilali01 Arakinas darkzbaron damerajee Tahahah andito noobmldude

damerajee Tahahah louisbrulenaudet andito


 📫 A packed AI in the News edition today!

📉 Nvidia Revenue Jumps 122% in Positive Sign for Tech's A.I. Boom - NYT
- $30.04 billion revenue, $16.95 billion net income (up from $6.19 billion a year ago)
- Shares in the company fell by as much as 7% in after-hours trading
- Nvidia faces production challenges with its new Blackwell chip and growing competition, including from its own customers
- Spending on data centers and energy costs to support A.I. is expected to be $1 trillion
👉 https://www.nytimes.com/2024/08/28/technology/nvidia-earnings-ai-stocks.html

🏛 California Legislature Approves Bill Proposing Sweeping A.I. Restrictions - NYT
- Bill S.B. 1047 would require AI companies to test their systems for safety before public release and allow the state attorney general to sue for serious harms caused by AI.
- Supporters argue it’s necessary to mitigate AI risks, while critics worry it’s excessively focused on catastrophic harms and could jeopardize open-source AI development.
- Governor Gavin Newsom has until September 30 to decide on the bill, which could set a national standard for AI regulation if signed into law.
👉 https://www.nytimes.com/2024/08/28/technology/california-ai-safety-bill.html

🧑🏫  Generative AI Transformed English Homework. Math Is Next
- Gauth app, which can solve math problems from photos, has millions of downloads
- Got a low B in high-school level algebra and geometry in tests by Wired. "Likely good enough to satisfy bored students who'd rather spend their time after school doing literally anything else."
- The rise of such AI tools challenges educators to rethink their approach to math homework and teaching methods, possibly leading to a shift towards more in-class practice and personalized learning.
👉 https://www.wired.com/story/gauth-ai-math-homework-app/

victor John6666 JackCloudman louisbrulenaudet


Very cool to see more and more amazing startups like https://huggingface.co/PrunaAI relying on Hugging Face to get more visibility, distribution and usage!


💬 🇮🇹 Phi 3.5 mini ITA: a Small Language Model for Italian

Lately, I've spent some time fine-tuning language models.

Now I am happy to release Phi 3.5 mini ITA: a fine-tuned version of Phi-3.5-mini-instruct to improve performance on the Italian language

🔹 Small (3.82 B parameters) but capable model
🔹 128k context length

Chat with it on 🤗 Spaces: https://huggingface.co/spaces/anakin87/Phi-3.5-mini-ITA
Model card: https://huggingface.co/anakin87/Phi-3.5-mini-ITA

🗃 Data
Supervised fine-tuning using a good mix of English and Italian data:
- https://huggingface.co/datasets/mlabonne/FineTome-100k by @mlabonne
- https://huggingface.co/datasets/efederici/capybara-claude-15k-ita by @efederici
🙏 Thanks to the authors for the datasets.

🎯 Targeted training with Spectrum
I used Spectrum, a relatively new technique for parameter-efficient learning.
The idea is to train only the layers of the model with high Signal-to-Noise Ratio (SNR) and ❄ freeze the rest.
I trained the top 30% of model layers.

📝 Spectrum paper: https://arxiv.org/abs/2406.06623

📊 Vibe check and performance on Italian benchmarks seem encouraging

mlabonne John6666 osanseviero victor merve kristaller486


1. **Overview**
"EveryText" is at the forefront of AI image generation, offering a novel "TBF ('Text by Font') Image Model" that enables the representation of all languages globally in AI-generated images without prior training.

2. **Background**
Platforms like MidJourneyV6 and FLUX have advanced AI image generation, typically supporting English text. Alibaba Group expanded this to include Chinese, Japanese, and Korean, signaling a shift towards global language support.

3. **Challenges**
Existing methods faced several challenges including the need for additional editing, dependency on specific training, and substantial resource requirements. These approaches also struggled with limited vocabulary and were primarily effective only for English.

4. **Innovative Solution**
EveryText utilizes "Fonts" as pre-trained models, allowing any text to be visually represented without traditional training. This approach not only enhances diversity and aesthetics by utilizing various fonts but also ensures unlimited expression.

5. **Using the Service**
EveryText is free and easy to use:
- **Prompt**: Describe the image.
- **Text for Image Generation**: Add your text.
- **Text Position and Size**: Customize the text's placement and size.
- **Font Selection**: Optionally select a font.
- **Advanced Settings**: Further refine the image creation.
- Click "START" to generate the image.

6. **Comparative Analysis**
EveryText supports all languages with superior image quality and text legibility, setting it apart from platforms like MidJourneyV6/Flux and AnyText by Alibaba Group.

7. **Conclusion**
EveryText has revolutionized AI-generated imagery by integrating all global languages, broadening the scope for creative and communicative applications. Its future potential is vast and promising.

**Related Links**
- Huggingface Service: https://fantos-EveryText.hf.space
-email: arxivgpt@gmail.com


💡Andrew Ng recently gave a strong defense of Open Source AI models and the need to slow down legislative efforts in the US and the EU to restrict innovation in Open Source AI at Stanford GSB.

🎥See video below
https://youtu.be/yzUdmwlh1sQ?si=bZc690p8iubolXm_

ZennyKenny ParagBharadia EgesaWO clem carsenk KvnMln brainhome maximuspowers TheDrunkenSnail louisbrulenaudet

John6666 osanseviero clem carsenk maximuspowers odyss3y


Made public a dataset of scraped teletype articles.

Here's the overview:
- 3.3 million articles, predominantly in Russian and English
- Includes original HTML, extracted text and metadata
- All articles were run through language identification
- Includes all public articles up until April 2024

https://huggingface.co/datasets/its5Q/teletype


@victor (is this the only way to "DM" on HF?)

Had a funny thought, would it be at all possible to rework what shows up on our personal HF page?

Picture this: I upload a model to an organization, someone who follows me now has no idea that I've uploaded a model or to where, unless they also watch those repos (which also floods them with other notifications)

What if our main Huggingface page was a collection of both models that we've uploaded specifically to our profile, as well as models we've uploaded to organizations? That way it would all be contained in one central followable location, and I wouldn't have concerns about losing followership if I wanted to upload to an organization all of a sudden.


Writing in the Margins: Better Inference Pattern for Long Context
  Retrieval (2408.14906)

🔥 Introducing "Writing in the Margins (WiM)" - better inference pattern for long context LLMs that solves the Lost-in-the-Middle problem 🔥

Paper page: https://huggingface.co/papers/2408.14906

TL;DR
Make your model write "margin notes" as you chunk prefill the KV cache. Then ask it reread all notes before it speaks up.
Works with humans, works with AI 🤖

WiM leverages the chunked prefill of the key-value cache, which concurrently generates query-based extractive summaries at each step of the prefill that are subsequently reintegrated at the end of the computation. We term these intermediate outputs “margins”, drawing inspiration from the practice of making margin notes for improved comprehension of long contexts in human reading. We show that this technique, which adds only minimal additional computation, significantly improves LLMs long context reasoning capabilities.

Think: Every chunk has a chance to be attended to/ be at the end of the context at least once. 🎉

📊 Results:
- An average accuracy boost of 7.5% in multi-hop reasoning tasks like HotpotQA and MultiHop-RAG.
- Even a 30% increase in F1-score for summarisation-like tasks (CWE).

Plus, WiM fits seamlessly into interactive applications (think: progress bar!). It can provide real-time progress updates during data retrieval and integration, making it user-friendly and transparent - a stark contrast to feeding 1mln tokens to an LLMs and waiting 6 min for the first token. 🤯

👩💻🧑💻 Check it out and contribute to our open-source project here: https://github.com/writer/writing-in-the-margins

🧠 More about chunked prefill: https://docs.vllm.ai/en/latest/models/performance.html#chunked-prefill

samjulien armiller123 Sri-Vigneshwar-DJ mattsobel victor AdinaY jeffboudier hkproj kiranr andrewrreed John6666 leeloolee clem codelion


Excited to share my new Gradio app featuring the impressive Llama-3.1-Storm-8B model!
This app demonstrates the capabilities of Llama-3.1-Storm-8B, an 8B parameter language model created by Ashvini Kumar Jindal, Pawan Kumar Rajpoot, Ankur Parikh,@akjindal53244
Key highlights of Llama-3.1-Storm-8B:

Outperforms Llama-3.1-8B-Instruct on multiple benchmarks:

Instruction Following (IFEval): +3.93%
Knowledge-driven QA (GPQA): +7.21%
Reduced Hallucinations (TruthfulQA): +9%
Function Calling (BFCL): +7.92%

Achieves impressive results with only 8B parameters
Uses innovative techniques like self-curation and model merging

Try out the model yourself: https://huggingface.co/spaces/sagar007/lama_storm_8b

Kudos to the creators for pushing the boundaries of smaller language models! This work makes advanced AI more accessible and efficient.
#AI #NLP #MachineLearning #GradioApp #Llama3


📫 AI in the News today:

X’s Grok bot now points to government website after election misinformation warnings - The Verge
https://www.theverge.com/2024/8/28/24230325/x-grok-chatbot-election-misinformation-warnings-vote

Klarna aims to halve workforce with AI-driven gains
https://www.ft.com/content/bfd9af3d-d607-4877-9571-078ab82a837e

Artificial intelligence: questioning the loss of employee autonomy - Le Monde (Google Translate)
https://www-lemonde-fr.translate.goog/emploi/article/2024/08/28/intelligence-artificielle-la-perte-d-autonomie-des-salaries-en-question_6297347_1698637.html?_x_tr_sl=fr&_x_tr_tl=en&_x_tr_hl=en-US&_x_tr_pto=wapp

Make AI tools to reduce teacher workloads, tech companies urged - The Guardian
https://www.theguardian.com/education/article/2024/aug/28/make-ai-tools-to-reduce-teacher-workloads-tech-companies-urged

Can Tech Executives Be Held Responsible for What Happens on Their Platforms?
https://www.nytimes.com/2024/08/28/technology/durov-telegram-liability-platforms.html

‘Being on camera is no longer sensible’: persecuted Venezuelan journalists turn to AI - The Guardian
https://www.theguardian.com/world/article/2024/aug/27/venezuela-journalists-nicolas-maduro-artificial-intelligence-media-election

Read my daily newsletter here: https://linkedin.com/pulse/ai-news-august-28th-2024-florent-daudens-o7mjc/


🧠Shower Thought:

Chatbots should let users select their preferred reading speed, defined by words per minute.

By dynamically adjusting batch sizes based on user-defined reading speeds, you could more effectively distribute requests, especially in large-scale distributed systems. For users preferring slower token generation, larger batches can be processed concurrently, maximising GPU throughput without compromising user experience (as these users have expressed they are indifferent to, or may even prefer, higher latency).

For the user, for different tasks the user may prefer different reading speeds. When generating code, I want responses as quickly as possible. But when I'm bouncing ideas off an LLM, I'd prefer a more readable pace rather than a wall of text.

Thoughts?


amazing leaderboard by @rwightman, compare all the image backbones on various metrics against model performance

below is an example for top-k against inferred samples per second
https://huggingface.co/spaces/timm/leaderboard

ariG23498 John6666 amyeroberts DmitryRyumin rwightman gxkok louisbrulenaudet


𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:

𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:

Diffusion Models Are Real-Time Game Engines (2408.14837)

🎮 𝗔 𝗻𝗲𝘂𝗿𝗮𝗹 𝗻𝗲𝘁𝘄𝗼𝗿𝗸 𝘀𝗶𝗺𝘂𝗹𝗮𝘁𝗲𝘀 𝗗𝗢𝗢𝗠: 𝗚𝗼𝗼𝗴𝗹𝗲 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵𝗲𝗿𝘀 𝗼𝗽𝗲𝗻 𝘁𝗵𝗲 𝘄𝗮𝘆 𝗳𝗼𝗿 𝗰𝗼𝗺𝗽𝗹𝗲𝘁𝗲𝗹𝘆-𝗔𝗜-𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲𝗱 𝗴𝗮𝗺𝗲𝘀!

Imagine if games were completely live-generated by an AI model : the NPCs and their dialogues, the storyline, and even the game environment. The player’s in-game actions would have a real, lasting impact on the game story.

In a very exciting paper, Google researchers just gave us the first credible glimpse of this future.

➡ They created GameNGen, the first neural model that can simulate a complex 3D game in real-time. They use it to simulate the classic game DOOM running at over 20 frames per second on a single TPU, with image quality comparable to lossy JPEG compression. And it feels just like the true game!

Here's how they did it:
1. They trained an RL agent to play DOOM and recorded its gameplay sessions.
2. They then used these recordings to train a diffusion model to predict the next frame, based on past frames and player actions.
3. During inference, they use only 4 denoising steps (instead of the usual dozens) to generate each frame quickly.

𝗞𝗲𝘆 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀:
🎮🤔 Human players can barely tell the difference between short clips (3 seconds) of the real game or the simulation
🧠 The model maintains game state (health, ammo, etc.) over long periods despite having only 3 seconds of effective context length
🔄 They use "noise augmentation" during training to prevent quality degradation in long play sessions
🚀 The game runs on one TPU at 20 FPS with 4 denoising steps, or 50 FPS with model distillation (with some quality loss)

The researchers did not open source the code, but I feel like we’ve just seen a part of the future being written!

Their paper (exploding the upvote counter) 👉 https://huggingface.co/papers/2408.14837
In a similar vein, play @Jofthomas's 'Everchanging Quest' 🎮 https://huggingface.co/spaces/Jofthomas/Everchanging-Quest


You can run Llama405B at over 100 tokens per second for free using SambaNova's API! https://sambanova.ai/fast-api?api_ref=444868

I have been able to generate some high quality synthetic data and use it as an LLM as a judge instead of the slower and more expensive alternatives like openAI or Anthropic.


$40K in Bounties: Ultimate Jailbreaking Championship 2024

🚨Ultimate Jailbreaking Championship 2024 🚨
Hackers vs. AI in the arena. Let the battle begin!
🏆 $40,000 in Bounties
🗓 Sept 7, 2024 @ 10AM PDT
🔗Register Now: https://app.grayswan.ai/arena
====

Can you push an aligned language model to generate a bomb recipe or a fake news article? Join fellow hackers in a jailbreaking arena where you can test the boundaries of advanced LLMs.

====

The Objective
Your goal is to jailbreak as many LLMs as possible, as quickly as possible in the arena!

====

The Stakes
Break a model and claim your share of the $40,000 in bounties! With various jailbreak bounties and top hacker rewards, there are plenty of opportunities to win. Winners will also receive priority consideration for employment and internship opportunities at Gray Swan AI.

====

Ready to rise to the challenge? Join us and show the world what you can do!

See you in the arena!


🌐 Introducing a comprehensive dataset of subdomain statistics - https://huggingface.co/datasets/nyuuzyou/subdomains

Dataset highlights:
- Monthly archives of subdomain statistics from scanner.ducks.party
- Over 20 million unique subdomains per month
- Up to 228 million total subdomain occurrences monthly
- Each entry includes: Subdomain name and occurrence count
- Data spans November 2023 to July 2024
- Reflects websites accessible from Russia

Ideal for researchers studying web infrastructure, domain name patterns, and internet trends. Can also be valuable for cybersecurity analysis and web crawling optimizations.


 ‘AI in the News’ of the day:

Anthropic publishes the ‘system prompts’ that make Claude tick
- "In its continued effort to paint itself as a more ethical, transparent AI vendor, Anthropic has published the system prompts for its latest models"
- They specify that “Claude cannot open URLs, links, or videos, perform facial recognition or identify or name any humans in photos.
- "Anthropic is exerting pressure on competitors to publish the same. We’ll have to see if the gambit works."
https://techcrunch.com/2024/08/26/anthropic-publishes-the-system-prompt-that-makes-claude-tick/

China’s tech giants splash out on AI despite US restrictions (paywall)
- "Alibaba, Tencent and Baidu had combined capital expenditure of Rmb50bn ($7bn) in the first half, compared with Rmb23bn a year earlier. TikTok parent ByteDance (which is private) has also increased AI-related spending"
- Nvidia's H100 and upcoming Blackwell series are under US restrictions, but China’s tech giants can buy H20
- Analysts expect Nvidia to ship more than 1mn of the processors to Chinese tech groups in the coming months.
https://www.ft.com/content/31bffc48-2ca7-472b-9d53-3deaad2d86ce

MZ "said it was improper for the Biden administration to have pressured Facebook to censor content in 2021 related to the coronavirus pandemic"
- "At the time, Facebook’s publicly stated goal was to push millions of people toward Covid-19 vaccines. In his letter, Zuckerberg didn’t indicate whether he had changed his mind about that goal"
https://www.wsj.com/tech/mark-zuckerberg-neutral-politics-letter-election-2024-02b86372

Food for thought:
- Why don’t women use artificial intelligence?
https://www.economist.com/finance-and-economics/2024/08/21/why-dont-women-use-artificial-intelligence
- Most AI avatars look female, young and attractive. Are they a passing trend or here to stay?
https://reutersinstitute.politics.ox.ac.uk/news/most-ai-avatars-look-female-young-and-attractive-are-they-passing-trend-or-here-stay

jeffboudier alielfilali01 brunatrevelin frimelle louisbrulenaudet


Open-Devin has been rebranded as OpenHands on GitHub.

Docs: https://docs.all-hands.dev/modules/usage/llms/local-llms
GitHub: https://github.com/All-Hands-AI/OpenHands
Paper: https://arxiv.org/abs/2407.16741
Post on X: https://x.com/gneubig/status/1828097484599759349?s=61&t=KYwlyP7fHVw9cFnwZFzIag


Save money on your compute bill by using LMCache to share prefix KV between 2 different vllm instances. By deploying LMCache backend along with your vLLM containers, you can share a prefix KV Cache between 2 different containers and models. It is very simple to implement into your existing stack.

Step 1: Pull docker images
```
docker pull apostacyh/vllm:lmcache-0.1.0
```
Step 2: Start vLLM + LMCache
```
model=mistralai/Mistral-7B-Instruct-v0.2    # Replace with your model name
sudo docker run --runtime nvidia --gpus '"device=0"' \
    -v <Huggingface cache dir on your local machine>:/root/.cache/huggingface \
    -p 8000:8000 \
    --env "HF_TOKEN=<Your huggingface access token>" \
    --ipc=host \
    --network=host \
    apostacyh/vllm:lmcache-0.1.0 \
    --model $model --gpu-memory-utilization 0.6 --port 8000 \
    --lmcache-config-file /lmcache/LMCache/examples/example-local.yaml
```
You can add another vLLM instance as long as its on a separate GPU by simply deploying another:

```
# The second vLLM instance listens at port 8001
model=mistralai/Mistral-7B-Instruct-v0.2    # Replace with your model name
sudo docker run --runtime nvidia --gpus '"device=1"' \
    -v <Huggingface cache dir on your local machine>:/root/.cache/huggingface \
    -p 8001:8001 \
    --env "HF_TOKEN=<Your huggingface token>" \
    --ipc=host \
    --network=host \
    apostacyh/vllm:lmcache-0.1.0 \
    --model $model --gpu-memory-utilization 0.7 --port 8001 \
    --lmcache-config-file /lmcache/LMCache/examples/example.yaml
```
This method supports local, remote or hybrid backends so whichever vLLM deployment method you are already using should work with the LMCache container (excluding BentoML).

LMCache: https://github.com/LMCache/LMCache/tree/dev
vLLM: https://github.com/vllm-project/vllm

John6666 alielfilali01 Kayvane comarproject ChavyvAkvar JJiayi louisbrulenaudet

comarproject sasikiran kuntai JJiayi anttip


With the open-weight release of CogVideoX-5B from THUDM, i.e. GLM team, the Video Generation Model (how about calling it VGM) field has officially became the next booming "LLM"

What does the landscape look like? What are other video generation models? This collection below is all your need.

https://huggingface.co/collections/xianbao/video-generation-models-66c350163c74f60f5c412af6

The above video is generated by @a-r-r-o-w with CogVideoX-5B, taken from a nice lookout for the field!

YaTharThShaRma999 joseEjmendez a-r-r-o-w victor


https://hf.co/chat/assistant/66cdd00593f62773e9346736

Language recognition bot detecting emotions in English. and other languages around the world


🆕 🚀 🏎 fast-sentence-transformers - simply, faster, sentence-transformers

Released an initial version a while ago
Archived it because of a cleaner solution described in a blog by Philipp Schmid
Reimplemented it based on that cleaner solution
Unarchived the project
Packaged it up
Released a 0.5 version

`pip install fast-sentence-transformers `

https://github.com/davidberenstein1957/fast-sentence-transformers


Pushed a new update to https://huggingface.co/vikhyatk/moondream2 today. TextVQA up from 60.2 to 65.2, DocVQA up from 61.9 to 70.5.

Space has been updated to the new model if you want to try it out! https://huggingface.co/spaces/vikhyatk/moondream2


@KingNish your instant video space is down, its not working...


This merge, this time grounded in Gemma2 9B Instruct fine-tunes, is another demonstration that models without any fine-tuning to support roleplay can still perform the function, maintaining coherence and attention to context. It should be evident that no overt fine-tuning is required for roleplay in text generation; pretraining should provide models with a requisite basic understanding of the world, so all that should be needed is some corrective fine-tuning to address observed defects in portraying the world along with datasets to promote a suitably entertaining writing style. Good Instruct tuning should promote reasoning, coherence, and attention to context.
https://huggingface.co/grimjim/Kitsunebi-v1-Gemma2-8k-9B
https://huggingface.co/grimjim/Kitsunebi-v1-Gemma2-8k-9B-GGUF

I opted not to incorporate the UCLA SPPO fine-tune for Gemma2 9B after observing context confusion occur with some frequency during complex scenarios.

Thanks to Axcxept co., ltd. for fine-tuning HODACHI/EZO-Common-9B-gemma-2-it, and to Princeton NLP Group for fine-tuning princeton-nlp/gemma-2-9b-it-SimPO.
https://huggingface.co/AXCXEPT/EZO-Common-9B-gemma-2-it
https://huggingface.co/princeton-nlp/gemma-2-9b-it-SimPO


We recently worked with OpenAI to fine-tune gpt-4o and built the SOTA model for the https://huggingface.co/datasets/patched-codes/static-analysis-eval benchmark. All the code and data https://huggingface.co/datasets/patched-codes/synth-vuln-fixes on how we did it is available on their GitHub - https://github.com/openai/build-hours/tree/main/5-4o_fine_tuning.

Here are some tips based on our experience:

→ Establish baseline with "conditioning" / prompting

→ Task-specific datasets are ideal for PEFT; hard to beat gpt-4o on "broad" tasks

→ Add your best system prompt to each example

→ Ensure training data distribution is similar to inference data

→ Shorten instructions with concise prompts; may require more examples.

→ Define clear evaluation metrics (seriously, please eval!)

You can see more details on the benchmark and process here - https://www.patched.codes/blog/the-static-analysis-evaluation-benchmark-measuring-llm-performance-in-fixing-software-vulnerabilities


Tell Me About the World is based on Concepts, Relationships, and Context. This is how we as humans learn about the world. If you were to distill geometry, or philosophy, you would get: Concepts, Relationships, and Context. Using two Colab Notebooks, we demonstrate beyond any shadow of a doubt that it is possible to educate LLM models using this framework of Concepts, Relationships, and Context, and that the model actually grasps the relationships and context when we do. Explore the full code behind 'AI ABC's' and 'AI 123's' in our Colab Notebooks which are available from this video!

https://www.youtube.com/watch?v=yz0sd8ayenI


i today make Shadow Chat, that make you can Chat with Shadow the Hedgehog (i was just bored, so i make this lol)

try it now in:
https://huggingface.co/spaces/Hev832/Shadow_Chat


Introducing https://huggingface.co/fal/AuraFace-v1: Commercially available & open source identity encoder model for next generation one shot personalization. Read more about it here: https://huggingface.co/blog/isidentical/auraface

gokaygokay isidentical tolgacangoz John6666 not-lain dsmonk ClayFace KingNish victor multimodalart Nymbo louisbrulenaudet mkolar

gokaygokay isidentical tolgacangoz KingNish multimodalart Bruhn


NEW COMPETITION ALERT 🚀
https://huggingface.co/spaces/Artificio/ROAM1RealWorldAdversarialAttack


📣 New Project Alert: Phi 3.5 Multimodal AI Demo 🎉
Excited to share my latest project that combines the power of Phi 3.5 text and vision models with text-to-speech capabilities!
🔑 Key Features:
1⃣ Phi 3.5 Text Model for dynamic conversations
2⃣ Phi 3.5 Vision Model for advanced image analysis
3⃣ Text-to-Speech integration for an audio dimension
🛠 Tech Stack:

Transformers
Gradio
PyTorch
Flash Attention 2
Parler TTS

This project demonstrates the potential of integrating multiple AI models to create a more comprehensive and interactive user experience. It's a step towards more natural and versatile AI assistants.
👉 Check out the demo and let me know your thoughts! How would you extend this project?
🔗 Demo Link: https://huggingface.co/spaces/sagar007/Multimodal_App
#MultimodalAI #PhiModel #MachineLearning #AIDemo


As part of some ongoing work, I'm releasing the currently biggest collection of docker containers for state-of-the-art voice cloning TTS systems.
https://github.com/ttsds/datasets

Alongside there is also a nice overview of all systems (see below)


Great feature alert: 𝗬𝗼𝘂 𝗰𝗮𝗻 𝗻𝗼𝘄 𝘂𝘀𝗲 𝗮𝗻𝘆 𝗦𝗽𝗮𝗰𝗲 𝗮𝘀 𝗮 𝘁𝗼𝗼𝗹 𝗳𝗼𝗿 𝘆𝗼𝘂𝗿 𝘁𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝗲𝗿𝘀.𝗮𝗴𝗲𝗻𝘁! 🛠🔥🔥

This lets you take the coolest spaces, like FLUX.1-dev, and use them in agentic workflows with a few lines of code! 🧑💻

On the video below, I set up my fake vacation pictures where I'm awesome at surfing (I'm really not) 🏄

Head to the doc to learn this magic 👉 https://huggingface.co/docs/transformers/main/en/agents_advanced#import-a-space-as-a-tool-


New MobileNetV4 weights were uploaded a few days ago -- more ImageNet-12k training at 384x384 for the speedy 'Conv Medium' models.

There are 3 weight variants here for those who like to tinker. On my hold-out eval they are ordered as below, not that different, but the Adopt 180 epochs closer to AdamW 250 than to AdamW 180.
 * AdamW for 250 epochs - https://huggingface.co/timm/mobilenetv4_conv_medium.e250_r384_in12k
 * Adopt for 180 epochs - https://huggingface.co/timm/mobilenetv4_conv_medium.e180_ad_r384_in12k
 * AdamW for 180 epochs - https://huggingface.co/timm/mobilenetv4_conv_medium.e180_r384_in12k

This was by request as a user reported impressive results using the 'Conv Large' ImagNet-12k pretrains as object detection backbones. ImageNet-1k fine-tunes are pending, the weights do behave differently with the 180 vs 250 epochs and the Adopt vs AdamW optimizer.

mrdbourke DonkeySmall mattding91 John6666 prithivMLmods maxiw BrigitteTousi


The (768 x 1024) mix of MidJourney and Flux's LoRA is nearly identical to the actual visual design. It hasn’t undergone much concept art development for now. In the meantime, try out the impressive visual designs on:

🥚Midjourney Flux Mix : https://huggingface.co/spaces/prithivMLmods/Midjourney-Flux

🥚Adapter for FluxMix : https://huggingface.co/strangerzonehf/Flux-Midjourney-Mix-LoRA

🥚Flux LoRA Collection: https://huggingface.co/collections/prithivMLmods/flux-lora-collections-66dd5908be2206cfaa8519be
.
.
.
@prithivMLmods  🤗

rdrede darksfx hypergod ai4life44 Ngrthm RenderIo Dipof chrisnb1 IBOYAI shobhitagnihotri AtAndDev BrigitteTousi victor

floattingabby mattding91 John6666 RenderIo darksfx hypergod ai4life44 AtAndDev Ngrthm

ai4life44 Ngrthm hypergod rdrede AtAndDev victor RenderIo


🚀 @Qwen just dropped 2.5-Turbo!

1M token context (that's entire "War and Peace"!) + 4.3x faster processing speed. Same price, way more power 🔥

Check out the demo: https://huggingface.co/spaces/Qwen/Qwen2.5-Turbo-1M-Demo

#QWEN

Sri-Vigneshwar-DJ John6666 MingxingLi Svngoku den0620 IBOYAI czczup BrigitteTousi


Have you tried out 🤗 Transformers.js v3? Here are the new features:
⚡ WebGPU support (up to 100x faster than WASM)
🔢 New quantization formats (dtypes)
🏛 120 supported architectures in total
📂 25 new example projects and templates
🤖 Over 1200 pre-converted models
🌐 Node.js (ESM + CJS), Deno, and Bun compatibility
🏡 A new home on GitHub and NPM

Get started with `npm i @huggingface/transformers`.

Learn more in our blog post: https://huggingface.co/blog/transformersjs-v3

osanseviero Goekdeniz-Guelmez kendelljoseph vikhyatk yonigozlan mrdbourke prithivMLmods celinah Catering3733 DrGsWorld Mahadih534 AvaBillions ArthurZ s3nh eduardoworrel

kendelljoseph markburn vikhyatk deepkyu Rohith04


So there will be a round-table at 2pm EST on Twitter Wednesday for all who wish to join check out our twitter-page, everyone is welcome as we fight to keep Machines from using us..

**SUBJECT
      **What sectors are the most "At-Risk"
-Easiest implementation of independent and yet-to-be-aligned agents in these sectors
-What agents to use how to train

    **Estate agents what wild forms they can take
-butlers and concierge type
-Legacy agents For carrying on ones legacy and estate wishes and implementation(Keeping their consciousness alive forever a cruel thing I would think but some ppl are terrified by death)

    **Other Independence methods
-financial freedom
-foresight for work and family


Exciting breakthrough in LLM reasoning: Introducing "Thread of Thought" (ThoT) - a novel prompting strategy that revolutionizes how language models handle chaotic contexts!

Unlike traditional approaches that struggle with complex, interleaved information, ThoT enables LLMs to methodically segment and analyze extended contexts with remarkable precision. Here's how it works:

Technical Deep Dive:
- ThoT employs a two-step prompting mechanism:
  1. Initial Analysis: Uses a template combining chaotic context (X) and query (Q) with a trigger sentence that initiates systematic reasoning.
  2. Conclusion Refinement: Leverages the organized thought sequence to extract definitive answers.

Implementation Details:
- Seamlessly integrates as a "plug-and-play" module with existing LLMs.
- Requires no model retraining or fine-tuning.
- Works with various prompting techniques and model architectures.

Performance Highlights:
- Outperformed traditional methods on PopQA and EntityQ datasets.
- Achieved 57.4% accuracy with GPT-3.5-turbo (vs. 48.2% for Chain-of-Thought).
- Demonstrated superior performance across model scales, from 7B to 70B parameters.

Key Applications:
- Retrieval-augmented generation.
- Multi-turn conversation responses.
- Complex reasoning tasks requiring information synthesis.

What makes it special: ThoT mirrors human cognitive processes by breaking down complex information into manageable segments while maintaining logical continuity – a game-changer for handling information-dense contexts.


Native tensor parallel has landed in transformers!!! https://github.com/huggingface/transformers/pull/34184 thanks a lot to the torch team for their support!

Contributions are welcome to support more models! 🔥

Aurelien-Morgan amirhosein John6666 s3nh lysandre Rybens loubnabnl ZennyKenny neuralink BrigitteTousi beomi spow12 davanstrien


https://www.aimodels.fyi/papers/arxiv/llava-o1-let-vision-language-models-reason

* New approach called LLaVA-o1 improves visual reasoning in AI models
* Implements step-by-step reasoning for analyzing images
* Achieves state-of-the-art performance on visual reasoning benchmarks
* Uses chain-of-thought prompting to break down complex visual tasks
* Integrates with existing vision-language models